+如果您熟悉中文，可以[阅读中文版本的README](./README_CN.md)
## 📢 Latest Updates
+**[March 25, 2025]** The online demo of ModelScope Space has been released: 500M model Only.
+**[March 24, 2025]** Is SAM2 difficult to install😭😭😭? 👉 It is compatible with rembg!
+**[March 20, 2025]** Release video motion processing pipeline
+**[March 19, 2025]** Local Gradio App.py optimization: Faster and More Stable 🔥🔥🔥
+**[March 15, 2025]** Inference Time Optimization: 30% Faster
**[March 13, 2025]** Initial release with:
✅ Inference codebase
✅ Pretrained LHM-0.5B model
✅ Pretrained LHM-1B model
-✅ Real-time rendering pipeline
+✅ Real-time rendering pipeline
+✅ Huggingface Online Demo
### TODO List
- [x] Core Inference Pipeline (v0.1) 🔥🔥🔥
-- [ ] HuggingFace Demo Integration
-- [ ] ModelScope Deployment
-- [ ] Motion Processing Scripts
+- [x] HuggingFace Demo Integration 🤗🤗🤗
+- [x] ModelScope Deployment
+- [x] Motion Processing Scripts
- [ ] Training Codes Release
## 🚀 Getting Started
+We provide a [video](https://www.bilibili.com/video/BV18So4YCESk/) that teaches us how to install LHM step by step on bilibili, submitted by 站长推荐推荐.
+
### Environment Setup
Clone the repository.
```bash
git clone git@github.com:aigc3d/LHM.git
cd LHM
```
+### Windows Installation
+Set Up a Virtual Environment
+Open **Command Prompt (CMD)**, navigate to the project folder, and run:
+```bash
+python -m venv lhm_env
+lhm_env\Scripts\activate
+install_cu121.bat
-Install dependencies by script.
+python ./app.py
```
# cuda 11.8
+
+```bash
+pip install rembg
sh ./install_cu118.sh
# cuda 12.1
@@ -47,7 +144,7 @@ Or you can install dependencies step by step, following [INSTALL.md](INSTALL.md)
### Model Weights
-Download pretrained models from our OSS:
+Please note that the model will be downloaded automatically if you do not download it yourself.
| Model | Training Data | BH-T Layers | Link | Inference Time|
| :--- | :--- | :--- | :--- | :--- |
@@ -67,16 +164,16 @@ tar -xvf LHM-1B.tar
### Download Prior Model Weights
```bash
# Download prior model weights
-wget https://virutalbuy-public.oss-cn-hangzhou.aliyuncs.com/share/aigc3d/data/for_lingteng/LHM/LHM_prior_model.tar
+wget https://virutalbuy-public.oss-cn-hangzhou.aliyuncs.com/share/aigc3d/data/LHM/LHM_prior_model.tar
tar -xvf LHM_prior_model.tar
```
### Data Motion Preparation
-We provide the test motion examples, we will update the procssing scripts ASAP :).
+We provide the test motion examples, we will update the processing scripts ASAP :).
```bash
# Download prior model weights
-wget https://virutalbuy-public.oss-cn-hangzhou.aliyuncs.com/share/aigc3d/data/for_lingteng/LHM/motion_video.tar
+wget https://virutalbuy-public.oss-cn-hangzhou.aliyuncs.com/share/aigc3d/data/LHM/motion_video.tar
tar -xvf ./motion_video.tar
```
@@ -127,18 +224,60 @@ After downloading weights and data, the folder of the project structure seems li
├── requirements.txt
```
+### 💻 Local Gradio Run
+```bash
+python ./app.py
+```
+
### 🏃 Inference Pipeline
```bash
-# bash ./inference.sh ./configs/inference/human-lrm-500M.yaml ./exps/releases/video_human_benchmark/human-lrm-500M/step_060000/ ./train_data/example_imgs/ ./train_data/motion_video/mimo1/smplx_params
-# bash ./inference.sh ./configs/inference/human-lrm-1B.yaml ./exps/releases/video_human_benchmark/human-lrm-1B/step_060000/ ./train_data/example_imgs/ ./train_data/motion_video/mimo1/smplx_params
+# MODEL_NAME={LHM-500M, LHM-1B}
+# bash ./inference.sh ./configs/inference/human-lrm-500M.yaml LHM-500M ./train_data/example_imgs/ ./train_data/motion_video/mimo1/smplx_params
+# bash ./inference.sh ./configs/inference/human-lrm-1B.yaml LHM-1B ./train_data/example_imgs/ ./train_data/motion_video/mimo1/smplx_params
+
+# animation
bash inference.sh ${CONFIG} ${MODEL_NAME} ${IMAGE_PATH_OR_FOLDER} ${MOTION_SEQ}
+
+# export mesh
+bash ./inference_mesh.sh ${CONFIG} ${MODEL_NAME}
```
+### Custom Video Motion Processing
+
+- Download model weights for motion processing.
+ ```bash
+ wget -P ./pretrained_models/human_model_files/pose_estimate https://virutalbuy-public.oss-cn-hangzhou.aliyuncs.com/share/aigc3d/data/LHM/yolov8x.pt
+ wget -P ./pretrained_models/human_model_files/pose_estimate https://virutalbuy-public.oss-cn-hangzhou.aliyuncs.com/share/aigc3d/data/LHM/vitpose-h-wholebody.pth
+ ```
+
+- Install extra dependencies.
+ ```bash
+ cd ./engine/pose_estimation
+ pip install -v -e third-party/ViTPose
+ pip install ultralytics
+ ```
+
+- Run the script.
+ ```bash
+ # python ./engine/pose_estimation/video2motion.py --video_path ./train_data/demo.mp4 --output_path ./train_data/custom_motion
+
+ python ./engine/pose_estimation/video2motion.py --video_path ${VIDEO_PATH} --output_path ${OUTPUT_PATH}
+ ```
+
+- Use the motion to drive the avatar.
+ ```bash
+ # if not sam2? pip install rembg.
+ # bash ./inference.sh ./configs/inference/human-lrm-500M.yaml LHM-500M ./train_data/example_imgs/ ./train_data/custom_motion/demo/smplx_params
+ # bash ./inference.sh ./configs/inference/human-lrm-1B.yaml LHM-1B ./train_data/example_imgs/ ./train_data/custom_motion/demo/smplx_params
+
+ bash inference.sh ${CONFIG} ${MODEL_NAME} ${IMAGE_PATH_OR_FOLDER} ${OUTPUT_PATH}/${VIDEO_NAME}/smplx_params
+ ```
+
## Compute Metric
-We provide some simple script to compute the metrics.
+We provide some simple scripts to compute the metrics.
```bash
# download pretrain model into ./pretrained_models/
-wget https://virutalbuy-public.oss-cn-hangzhou.aliyuncs.com/share/aigc3d/data/for_lingteng/arcface_resnet18.pth
+wget https://virutalbuy-public.oss-cn-hangzhou.aliyuncs.com/share/aigc3d/data/LHM/arcface_resnet18.pth
# Face Similarity
python ./tools/metrics/compute_facesimilarity.py -f1 ${gt_folder} -f2 ${results_folder}
# PSNR
@@ -147,6 +286,9 @@ python ./tools/metrics/compute_psnr.py -f1 ${gt_folder} -f2 ${results_folder}
python ./tools/metrics/compute_ssim_lpips.py -f1 ${gt_folder} -f2 ${results_folder}
```
+## ✅ ComfyUI Wrapper Implemented
+
+The ComfyUI wrapper for LHM has been implemented in this repository! See the documentation at the top of this README.
## Acknowledgement
This work is built on many amazing research works and open-source projects:
@@ -156,15 +298,21 @@ This work is built on many amazing research works and open-source projects:
Thanks for their excellent works and great contribution to 3D generation and 3D digital human area.
+We would like to express our sincere gratitude to [站长推荐推荐](https://space.bilibili.com/175365958?spm_id_from=333.337.0.0) for the installation tutorial video on bilibili.
+
+## ✨ Star History
+
+[](https://star-history.com/#aigc3d/LHM&Date)
+
## Citation
```
@inproceedings{qiu2025LHM,
- title={LHM: Large Animatable Human Reconstruction Model from a Single Image in One Second},
+ title={LHM: Large Animatable Human Reconstruction Model from a Single Image in Seconds},
author={Lingteng Qiu and Xiaodong Gu and Peihao Li and Qi Zuo
and Weichao Shen and Junfei Zhang and Kejie Qiu and Weihao Yuan
and Guanying Chen and Zilong Dong and Liefeng Bo
},
- booktitle={arXiv preprint arXiv:xxxxx},
+ booktitle={arXiv preprint arXiv:2503.10625},
year={2025}
}
-```
\ No newline at end of file
+```
diff --git a/README_CN.md b/README_CN.md
new file mode 100755
index 0000000..a0fdd99
--- /dev/null
+++ b/README_CN.md
@@ -0,0 +1,236 @@
+# - 官方 PyTorch 实现
+
+####
', '', f'## {titlecase(dataset)} Dataset', ''
+ ]
+ for keyword, info in keywords.items():
+ keyword_strs = [titlecase(x.replace('_', ' ')) for x in keyword]
+ lines += [
+ ' ', '',
+ (f'### {" + ".join(keyword_strs)}'
+ f' on {titlecase(dataset)}'), '', info, ''
+ ]
+
+ with open(f'topics/{subtopic.lower()}.md', 'w') as f:
+ f.write('\n'.join(lines))
+
+# Step 5: write files by paper
+allfiles = [x.split('/')[-2:] for x in glob('../en/papers/*/*.md')]
+sections = sorted(list(set(x[0] for x in allfiles)))
+for section in sections:
+ lines = [f'# {titlecase(section)}', '']
+ files = [f for s, f in allfiles if s == section]
+ for file in files:
+ with open(f'../en/papers/{section}/{file}', 'r') as f:
+ keyline = [
+ line for line in f.readlines() if line.startswith('', '', keyline).strip()
+ paperlines = []
+ for subtopic, datasets in contents.items():
+ for dataset, keywords in datasets.items():
+ keywords = {k: v for k, v in keywords.items() if keyline in v}
+ if len(keywords) == 0:
+ continue
+ for keyword, info in keywords.items():
+ keyword_strs = [
+ titlecase(x.replace('_', ' ')) for x in keyword
+ ]
+ paperlines += [
+ ' ', '',
+ (f'### {" + ".join(keyword_strs)}'
+ f' on {titlecase(dataset)}'), '', info, ''
+ ]
+ if len(paperlines) > 0:
+ lines += ['', '
', '', f'## {papername}', '']
+ lines += paperlines
+
+ with open(f'papers/{section}.md', 'w') as f:
+ f.write('\n'.join(lines))
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/conf.py b/engine/pose_estimation/third-party/ViTPose/docs/en/conf.py
new file mode 100644
index 0000000..10efef6
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/conf.py
@@ -0,0 +1,116 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+# Configuration file for the Sphinx documentation builder.
+#
+# This file only contains a selection of the most common options. For a full
+# list see the documentation:
+# https://www.sphinx-doc.org/en/master/usage/configuration.html
+
+# -- Path setup --------------------------------------------------------------
+
+# If extensions (or modules to document with autodoc) are in another directory,
+# add these directories to sys.path here. If the directory is relative to the
+# documentation root, use os.path.abspath to make it absolute, like shown here.
+#
+import os
+import subprocess
+import sys
+
+import pytorch_sphinx_theme
+
+sys.path.insert(0, os.path.abspath('../..'))
+
+# -- Project information -----------------------------------------------------
+
+project = 'MMPose'
+copyright = '2020-2021, OpenMMLab'
+author = 'MMPose Authors'
+
+# The full version, including alpha/beta/rc tags
+version_file = '../../mmpose/version.py'
+
+
+def get_version():
+ with open(version_file, 'r') as f:
+ exec(compile(f.read(), version_file, 'exec'))
+ return locals()['__version__']
+
+
+release = get_version()
+
+# -- General configuration ---------------------------------------------------
+
+# Add any Sphinx extension module names here, as strings. They can be
+# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
+# ones.
+extensions = [
+ 'sphinx.ext.autodoc', 'sphinx.ext.napoleon', 'sphinx.ext.viewcode',
+ 'sphinx_markdown_tables', 'sphinx_copybutton', 'myst_parser'
+]
+
+autodoc_mock_imports = ['json_tricks', 'mmpose.version']
+
+# Ignore >>> when copying code
+copybutton_prompt_text = r'>>> |\.\.\. '
+copybutton_prompt_is_regexp = True
+
+# Add any paths that contain templates here, relative to this directory.
+templates_path = ['_templates']
+
+# List of patterns, relative to source directory, that match files and
+# directories to ignore when looking for source files.
+# This pattern also affects html_static_path and html_extra_path.
+exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']
+
+# -- Options for HTML output -------------------------------------------------
+source_suffix = {
+ '.rst': 'restructuredtext',
+ '.md': 'markdown',
+}
+
+# The theme to use for HTML and HTML Help pages. See the documentation for
+# a list of builtin themes.
+#
+html_theme = 'pytorch_sphinx_theme'
+html_theme_path = [pytorch_sphinx_theme.get_html_theme_path()]
+html_theme_options = {
+ 'menu': [
+ {
+ 'name':
+ 'Tutorial',
+ 'url':
+ 'https://colab.research.google.com/github/'
+ 'open-mmlab/mmpose/blob/master/demo/MMPose_Tutorial.ipynb'
+ },
+ {
+ 'name': 'GitHub',
+ 'url': 'https://github.com/open-mmlab/mmpose'
+ },
+ ],
+ # Specify the language of the shared menu
+ 'menu_lang':
+ 'en'
+}
+
+# Add any paths that contain custom static files (such as style sheets) here,
+# relative to this directory. They are copied after the builtin static files,
+# so a file named "default.css" will overwrite the builtin "default.css".
+
+language = 'en'
+
+html_static_path = ['_static']
+html_css_files = ['css/readthedocs.css']
+
+# Enable ::: for my_st
+myst_enable_extensions = ['colon_fence']
+
+master_doc = 'index'
+
+
+def builder_inited_handler(app):
+ subprocess.run(['./collect.py'])
+ subprocess.run(['./merge_docs.sh'])
+ subprocess.run(['./stats.py'])
+
+
+def setup(app):
+ app.connect('builder-inited', builder_inited_handler)
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/data_preparation.md b/engine/pose_estimation/third-party/ViTPose/docs/en/data_preparation.md
new file mode 100644
index 0000000..0c691f5
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/data_preparation.md
@@ -0,0 +1,13 @@
+# Prepare Datasets
+
+MMPose supports multiple tasks. Please follow the corresponding guidelines for data preparation.
+
+- [2D Body Keypoint](tasks/2d_body_keypoint.md)
+- [3D Body Keypoint](tasks/3d_body_keypoint.md)
+- [3D Body Mesh Recovery](tasks/3d_body_mesh.md)
+- [2D Hand Keypoint](tasks/2d_hand_keypoint.md)
+- [3D Hand Keypoint](tasks/3d_hand_keypoint.md)
+- [2D Face Keypoint](tasks/2d_face_keypoint.md)
+- [2D WholeBody Keypoint](tasks/2d_wholebody_keypoint.md)
+- [2D Fashion Landmark](tasks/2d_fashion_landmark.md)
+- [2D Animal Keypoint](tasks/2d_animal_keypoint.md)
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/faq.md b/engine/pose_estimation/third-party/ViTPose/docs/en/faq.md
new file mode 100644
index 0000000..277885f
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/faq.md
@@ -0,0 +1,135 @@
+# FAQ
+
+We list some common issues faced by many users and their corresponding solutions here.
+Feel free to enrich the list if you find any frequent issues and have ways to help others to solve them.
+If the contents here do not cover your issue, please create an issue using the [provided templates](/.github/ISSUE_TEMPLATE/error-report.md) and make sure you fill in all required information in the template.
+
+## Installation
+
+- **Unable to install xtcocotools**
+
+ 1. Try to install it using pypi manually `pip install xtcocotools`.
+ 1. If step1 does not work. Try to install it from [source](https://github.com/jin-s13/xtcocoapi).
+
+ ```
+ git clone https://github.com/jin-s13/xtcocoapi
+ cd xtcocoapi
+ python setup.py install
+ ```
+
+- **No matching distribution found for xtcocotools>=1.6**
+
+ 1. Install cython by `pip install cython`.
+ 1. Install xtcocotools from [source](https://github.com/jin-s13/xtcocoapi).
+
+ ```
+ git clone https://github.com/jin-s13/xtcocoapi
+ cd xtcocoapi
+ python setup.py install
+ ```
+
+- **"No module named 'mmcv.ops'"; "No module named 'mmcv._ext'"**
+
+ 1. Uninstall existing mmcv in the environment using `pip uninstall mmcv`.
+ 1. Install mmcv-full following the [installation instruction](https://mmcv.readthedocs.io/en/latest/#installation).
+
+## Data
+
+- **How to convert my 2d keypoint dataset to coco-type?**
+
+ You may refer to this conversion [tool](https://github.com/open-mmlab/mmpose/blob/master/tools/dataset/parse_macaquepose_dataset.py) to prepare your data.
+ Here is an [example](https://github.com/open-mmlab/mmpose/blob/master/tests/data/macaque/test_macaque.json) of the coco-type json.
+ In the coco-type json, we need "categories", "annotations" and "images". "categories" contain some basic information of the dataset, e.g. class name and keypoint names.
+ "images" contain image-level information. We need "id", "file_name", "height", "width". Others are optional.
+ Note: (1) It is okay that "id"s are not continuous or not sorted (e.g. 1000, 40, 352, 333 ...).
+
+ "annotations" contain instance-level information. We need "image_id", "id", "keypoints", "num_keypoints", "bbox", "iscrowd", "area", "category_id". Others are optional.
+ Note: (1) "num_keypoints" means the number of visible keypoints. (2) By default, please set "iscrowd: 0". (3) "area" can be calculated using the bbox (area = w * h) (4) Simply set "category_id: 1". (5) The "image_id" in "annotations" should match the "id" in "images".
+
+- **What if my custom dataset does not have bounding box label?**
+
+ We can estimate the bounding box of a person as the minimal box that tightly bounds all the keypoints.
+
+- **What if my custom dataset does not have segmentation label?**
+
+ Just set the `area` of the person as the area of the bounding boxes. During evaluation, please set `use_area=False` as in this [example](https://github.com/open-mmlab/mmpose/blob/a82dd486853a8a471522ac06b8b9356db61f8547/mmpose/datasets/datasets/top_down/topdown_aic_dataset.py#L113).
+
+- **What is `COCO_val2017_detections_AP_H_56_person.json`? Can I train pose models without it?**
+
+ "COCO_val2017_detections_AP_H_56_person.json" contains the "detected" human bounding boxes for COCO validation set, which are generated by FasterRCNN.
+ One can choose to use gt bounding boxes to evaluate models, by setting `use_gt_bbox=True` and `bbox_file=''`. Or one can use detected boxes to evaluate
+ the generalizability of models, by setting `use_gt_bbox=False` and `bbox_file='COCO_val2017_detections_AP_H_56_person.json'`.
+
+## Training
+
+- **RuntimeError: Address already in use**
+
+ Set the environment variables `MASTER_PORT=XXX`. For example,
+ `MASTER_PORT=29517 GPUS=16 GPUS_PER_NODE=8 CPUS_PER_TASK=2 ./tools/slurm_train.sh Test res50 configs/body/2D_Kpt_SV_RGB_Img/topdown_hm/coco/res50_coco_256x192.py work_dirs/res50_coco_256x192`
+
+- **"Unexpected keys in source state dict" when loading pre-trained weights**
+
+ It's normal that some layers in the pretrained model are not used in the pose model. ImageNet-pretrained classification network and the pose network may have different architectures (e.g. no classification head). So some unexpected keys in source state dict is actually expected.
+
+- **How to use trained models for backbone pre-training ?**
+
+ Refer to [Use Pre-Trained Model](/docs/en/tutorials/1_finetune.md#use-pre-trained-model),
+ in order to use the pre-trained model for the whole network (backbone + head), the new config adds the link of pre-trained models in the `load_from`.
+
+ And to use backbone for pre-training, you can change `pretrained` value in the backbone dict of config files to the checkpoint path / url.
+ When training, the unexpected keys will be ignored.
+
+- **How to visualize the training accuracy/loss curves in real-time ?**
+
+ Use `TensorboardLoggerHook` in `log_config` like
+
+ ```python
+ log_config=dict(interval=20, hooks=[dict(type='TensorboardLoggerHook')])
+ ```
+
+ You can refer to [tutorials/6_customize_runtime.md](/tutorials/6_customize_runtime.md#log-config) and the example [config](https://github.com/open-mmlab/mmpose/tree/e1ec589884235bee875c89102170439a991f8450/configs/top_down/resnet/coco/res50_coco_256x192.py#L26).
+
+- **Log info is NOT printed**
+
+ Use smaller log interval. For example, change `interval=50` to `interval=1` in the [config](https://github.com/open-mmlab/mmpose/tree/e1ec589884235bee875c89102170439a991f8450/configs/top_down/resnet/coco/res50_coco_256x192.py#L23).
+
+- **How to fix stages of backbone when finetuning a model ?**
+
+ You can refer to [`def _freeze_stages()`](https://github.com/open-mmlab/mmpose/blob/d026725554f9dc08e8708bd9da8678f794a7c9a6/mmpose/models/backbones/resnet.py#L618) and [`frozen_stages`](https://github.com/open-mmlab/mmpose/blob/d026725554f9dc08e8708bd9da8678f794a7c9a6/mmpose/models/backbones/resnet.py#L498),
+ reminding to set `find_unused_parameters = True` in config files for distributed training or testing.
+
+## Evaluation
+
+- **How to evaluate on MPII test dataset?**
+ Since we do not have the ground-truth for test dataset, we cannot evaluate it 'locally'.
+ If you would like to evaluate the performance on test set, you have to upload the pred.mat (which is generated during testing) to the official server via email, according to [the MPII guideline](http://human-pose.mpi-inf.mpg.de/#evaluation).
+
+- **For top-down 2d pose estimation, why predicted joint coordinates can be out of the bounding box (bbox)?**
+ We do not directly use the bbox to crop the image. bbox will be first transformed to center & scale, and the scale will be multiplied by a factor (1.25) to include some context. If the ratio of width/height is different from that of model input (possibly 192/256), we will adjust the bbox.
+
+## Inference
+
+- **How to run mmpose on CPU?**
+
+ Run demos with `--device=cpu`.
+
+- **How to speed up inference?**
+
+ For top-down models, try to edit the config file. For example,
+
+ 1. set `flip_test=False` in [topdown-res50](https://github.com/open-mmlab/mmpose/tree/e1ec589884235bee875c89102170439a991f8450/configs/top_down/resnet/coco/res50_coco_256x192.py#L51).
+ 1. set `post_process='default'` in [topdown-res50](https://github.com/open-mmlab/mmpose/tree/e1ec589884235bee875c89102170439a991f8450/configs/top_down/resnet/coco/res50_coco_256x192.py#L54).
+ 1. use faster human bounding box detector, see [MMDetection](https://mmdetection.readthedocs.io/en/latest/model_zoo.html).
+
+ For bottom-up models, try to edit the config file. For example,
+
+ 1. set `flip_test=False` in [AE-res50](https://github.com/open-mmlab/mmpose/tree/e1ec589884235bee875c89102170439a991f8450/configs/bottom_up/resnet/coco/res50_coco_512x512.py#L91).
+ 1. set `adjust=False` in [AE-res50](https://github.com/open-mmlab/mmpose/tree/e1ec589884235bee875c89102170439a991f8450/configs/bottom_up/resnet/coco/res50_coco_512x512.py#L89).
+ 1. set `refine=False` in [AE-res50](https://github.com/open-mmlab/mmpose/tree/e1ec589884235bee875c89102170439a991f8450/configs/bottom_up/resnet/coco/res50_coco_512x512.py#L90).
+ 1. use smaller input image size in [AE-res50](https://github.com/open-mmlab/mmpose/tree/e1ec589884235bee875c89102170439a991f8450/configs/bottom_up/resnet/coco/res50_coco_512x512.py#L39).
+
+## Deployment
+
+- **Why is the onnx model converted by mmpose throwing error when converting to other frameworks such as TensorRT?**
+
+ For now, we can only make sure that models in mmpose are onnx-compatible. However, some operations in onnx may be unsupported by your target framework for deployment, e.g. TensorRT in [this issue](https://github.com/open-mmlab/mmaction2/issues/414). When such situation occurs, we suggest you raise an issue and ask the community to help as long as `pytorch2onnx.py` works well and is verified numerically.
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/getting_started.md b/engine/pose_estimation/third-party/ViTPose/docs/en/getting_started.md
new file mode 100644
index 0000000..d7cfea3
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/getting_started.md
@@ -0,0 +1,283 @@
+# Getting Started
+
+This page provides basic tutorials about the usage of MMPose.
+For installation instructions, please see [install.md](install.md).
+
+
+
+- [Prepare Datasets](#prepare-datasets)
+- [Inference with Pre-Trained Models](#inference-with-pre-trained-models)
+ - [Test a dataset](#test-a-dataset)
+ - [Run demos](#run-demos)
+- [Train a Model](#train-a-model)
+ - [Train with a single GPU](#train-with-a-single-gpu)
+ - [Train with CPU](#train-with-cpu)
+ - [Train with multiple GPUs](#train-with-multiple-gpus)
+ - [Train with multiple machines](#train-with-multiple-machines)
+ - [Launch multiple jobs on a single machine](#launch-multiple-jobs-on-a-single-machine)
+- [Benchmark](#benchmark)
+- [Tutorials](#tutorials)
+
+
+
+## Prepare Datasets
+
+MMPose supports multiple tasks. Please follow the corresponding guidelines for data preparation.
+
+- [2D Body Keypoint Detection](/docs/en/tasks/2d_body_keypoint.md)
+- [3D Body Keypoint Detection](/docs/en/tasks/3d_body_keypoint.md)
+- [3D Body Mesh Recovery](/docs/en/tasks/3d_body_mesh.md)
+- [2D Hand Keypoint Detection](/docs/en/tasks/2d_hand_keypoint.md)
+- [3D Hand Keypoint Detection](/docs/en/tasks/3d_hand_keypoint.md)
+- [2D Face Keypoint Detection](/docs/en/tasks/2d_face_keypoint.md)
+- [2D WholeBody Keypoint Detection](/docs/en/tasks/2d_wholebody_keypoint.md)
+- [2D Fashion Landmark Detection](/docs/en/tasks/2d_fashion_landmark.md)
+- [2D Animal Keypoint Detection](/docs/en/tasks/2d_animal_keypoint.md)
+
+## Inference with Pre-trained Models
+
+We provide testing scripts to evaluate a whole dataset (COCO, MPII etc.),
+and provide some high-level apis for easier integration to other OpenMMLab projects.
+
+### Test a dataset
+
+- [x] single GPU
+- [x] CPU
+- [x] single node multiple GPUs
+- [x] multiple node
+
+You can use the following commands to test a dataset.
+
+```shell
+# single-gpu testing
+python tools/test.py ${CONFIG_FILE} ${CHECKPOINT_FILE} [--out ${RESULT_FILE}] [--fuse-conv-bn] \
+ [--eval ${EVAL_METRICS}] [--gpu_collect] [--tmpdir ${TMPDIR}] [--cfg-options ${CFG_OPTIONS}] \
+ [--launcher ${JOB_LAUNCHER}] [--local_rank ${LOCAL_RANK}]
+
+# CPU: disable GPUs and run single-gpu testing script
+export CUDA_VISIBLE_DEVICES=-1
+python tools/test.py ${CONFIG_FILE} ${CHECKPOINT_FILE} [--out ${RESULT_FILE}] \
+ [--eval ${EVAL_METRICS}]
+
+# multi-gpu testing
+./tools/dist_test.sh ${CONFIG_FILE} ${CHECKPOINT_FILE} ${GPU_NUM} [--out ${RESULT_FILE}] [--fuse-conv-bn] \
+ [--eval ${EVAL_METRIC}] [--gpu_collect] [--tmpdir ${TMPDIR}] [--cfg-options ${CFG_OPTIONS}] \
+ [--launcher ${JOB_LAUNCHER}] [--local_rank ${LOCAL_RANK}]
+```
+
+Note that the provided `CHECKPOINT_FILE` is either the path to the model checkpoint file downloaded in advance, or the url link to the model checkpoint.
+
+Optional arguments:
+
+- `RESULT_FILE`: Filename of the output results. If not specified, the results will not be saved to a file.
+- `--fuse-conv-bn`: Whether to fuse conv and bn, this will slightly increase the inference speed.
+- `EVAL_METRICS`: Items to be evaluated on the results. Allowed values depend on the dataset.
+- `--gpu_collect`: If specified, recognition results will be collected using gpu communication. Otherwise, it will save the results on different gpus to `TMPDIR` and collect them by the rank 0 worker.
+- `TMPDIR`: Temporary directory used for collecting results from multiple workers, available when `--gpu_collect` is not specified.
+- `CFG_OPTIONS`: Override some settings in the used config, the key-value pair in xxx=yyy format will be merged into config file. For example, '--cfg-options model.backbone.depth=18 model.backbone.with_cp=True'.
+- `JOB_LAUNCHER`: Items for distributed job initialization launcher. Allowed choices are `none`, `pytorch`, `slurm`, `mpi`. Especially, if set to none, it will test in a non-distributed mode.
+- `LOCAL_RANK`: ID for local rank. If not specified, it will be set to 0.
+
+Examples:
+
+Assume that you have already downloaded the checkpoints to the directory `checkpoints/`.
+
+1. Test ResNet50 on COCO (without saving the test results) and evaluate the mAP.
+
+ ```shell
+ ./tools/dist_test.sh configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/res50_coco_256x192.py \
+ checkpoints/SOME_CHECKPOINT.pth 1 \
+ --eval mAP
+ ```
+
+1. Test ResNet50 on COCO with 8 GPUS. Download the checkpoint via url, and evaluate the mAP.
+
+ ```shell
+ ./tools/dist_test.sh configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/res50_coco_256x192.py \
+ https://download.openmmlab.com/mmpose/top_down/resnet/res50_coco_256x192-ec54d7f3_20200709.pth 8 \
+ --eval mAP
+ ```
+
+1. Test ResNet50 on COCO in slurm environment and evaluate the mAP.
+
+ ```shell
+ ./tools/slurm_test.sh slurm_partition test_job \
+ configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/res50_coco_256x192.py \
+ checkpoints/SOME_CHECKPOINT.pth \
+ --eval mAP
+ ```
+
+### Run demos
+
+We also provide scripts to run demos.
+Here is an example of running top-down human pose demos using ground-truth bounding boxes.
+
+```shell
+python demo/top_down_img_demo.py \
+ ${MMPOSE_CONFIG_FILE} ${MMPOSE_CHECKPOINT_FILE} \
+ --img-root ${IMG_ROOT} --json-file ${JSON_FILE} \
+ --out-img-root ${OUTPUT_DIR} \
+ [--show --device ${GPU_ID}] \
+ [--kpt-thr ${KPT_SCORE_THR}]
+```
+
+Examples:
+
+```shell
+python demo/top_down_img_demo.py \
+ configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/hrnet_w48_coco_256x192.py \
+ https://download.openmmlab.com/mmpose/top_down/hrnet/hrnet_w48_coco_256x192-b9e0b3ab_20200708.pth \
+ --img-root tests/data/coco/ --json-file tests/data/coco/test_coco.json \
+ --out-img-root vis_results
+```
+
+More examples and details can be found in the [demo folder](/demo) and the [demo docs](https://mmpose.readthedocs.io/en/latest/demo.html).
+
+## Train a model
+
+MMPose implements distributed training and non-distributed training,
+which uses `MMDistributedDataParallel` and `MMDataParallel` respectively.
+
+We adopt distributed training for both single machine and multiple machines. Supposing that the server has 8 GPUs, 8 processes will be started and each process runs on a single GPU.
+
+Each process keeps an isolated model, data loader, and optimizer. Model parameters are only synchronized once at the beginning. After a forward and backward pass, gradients will be allreduced among all GPUs, and the optimizer will update model parameters. Since the gradients are allreduced, the model parameter stays the same for all processes after the iteration.
+
+### Training setting
+
+All outputs (log files and checkpoints) will be saved to the working directory,
+which is specified by `work_dir` in the config file.
+
+By default we evaluate the model on the validation set after each epoch, you can change the evaluation interval by modifying the interval argument in the training config
+
+```python
+evaluation = dict(interval=5) # This evaluate the model per 5 epoch.
+```
+
+According to the [Linear Scaling Rule](https://arxiv.org/abs/1706.02677), you need to set the learning rate proportional to the batch size if you use different GPUs or videos per GPU, e.g., lr=0.01 for 4 GPUs x 2 video/gpu and lr=0.08 for 16 GPUs x 4 video/gpu.
+
+### Train with a single GPU
+
+```shell
+python tools/train.py ${CONFIG_FILE} [optional arguments]
+```
+
+If you want to specify the working directory in the command, you can add an argument `--work-dir ${YOUR_WORK_DIR}`.
+
+### Train with CPU
+
+The process of training on the CPU is consistent with single GPU training. We just need to disable GPUs before the training process.
+
+```shell
+export CUDA_VISIBLE_DEVICES=-1
+```
+
+And then run the script [above](#training-on-a-single-GPU).
+
+**Note**:
+
+We do not recommend users to use CPU for training because it is too slow. We support this feature to allow users to debug on machines without GPU for convenience.
+
+### Train with multiple GPUs
+
+```shell
+./tools/dist_train.sh ${CONFIG_FILE} ${GPU_NUM} [optional arguments]
+```
+
+Optional arguments are:
+
+- `--work-dir ${WORK_DIR}`: Override the working directory specified in the config file.
+- `--resume-from ${CHECKPOINT_FILE}`: Resume from a previous checkpoint file.
+- `--no-validate`: Whether not to evaluate the checkpoint during training.
+- `--gpus ${GPU_NUM}`: Number of gpus to use, which is only applicable to non-distributed training.
+- `--gpu-ids ${GPU_IDS}`: IDs of gpus to use, which is only applicable to non-distributed training.
+- `--seed ${SEED}`: Seed id for random state in python, numpy and pytorch to generate random numbers.
+- `--deterministic`: If specified, it will set deterministic options for CUDNN backend.
+- `--cfg-options CFG_OPTIONS`: Override some settings in the used config, the key-value pair in xxx=yyy format will be merged into config file. For example, '--cfg-options model.backbone.depth=18 model.backbone.with_cp=True'.
+- `--launcher ${JOB_LAUNCHER}`: Items for distributed job initialization launcher. Allowed choices are `none`, `pytorch`, `slurm`, `mpi`. Especially, if set to none, it will test in a non-distributed mode.
+- `--autoscale-lr`: If specified, it will automatically scale lr with the number of gpus by [Linear Scaling Rule](https://arxiv.org/abs/1706.02677).
+- `LOCAL_RANK`: ID for local rank. If not specified, it will be set to 0.
+
+Difference between `resume-from` and `load-from`:
+`resume-from` loads both the model weights and optimizer status, and the epoch is also inherited from the specified checkpoint. It is usually used for resuming the training process that is interrupted accidentally.
+`load-from` only loads the model weights and the training epoch starts from 0. It is usually used for finetuning.
+
+Here is an example of using 8 GPUs to load ResNet50 checkpoint.
+
+```shell
+./tools/dist_train.sh configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/res50_coco_256x192.py 8 --resume_from work_dirs/res50_coco_256x192/latest.pth
+```
+
+### Train with multiple machines
+
+If you can run MMPose on a cluster managed with [slurm](https://slurm.schedmd.com/), you can use the script `slurm_train.sh`. (This script also supports single machine training.)
+
+```shell
+./tools/slurm_train.sh ${PARTITION} ${JOB_NAME} ${CONFIG_FILE} ${WORK_DIR}
+```
+
+Here is an example of using 16 GPUs to train ResNet50 on the dev partition in a slurm cluster.
+(Use `GPUS_PER_NODE=8` to specify a single slurm cluster node with 8 GPUs, `CPUS_PER_TASK=2` to use 2 cpus per task.
+Assume that `Test` is a valid ${PARTITION} name.)
+
+```shell
+GPUS=16 GPUS_PER_NODE=8 CPUS_PER_TASK=2 ./tools/slurm_train.sh Test res50 configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/res50_coco_256x192.py work_dirs/res50_coco_256x192
+```
+
+You can check [slurm_train.sh](/tools/slurm_train.sh) for full arguments and environment variables.
+
+If you have just multiple machines connected with ethernet, you can refer to
+pytorch [launch utility](https://pytorch.org/docs/en/stable/distributed_deprecated.html#launch-utility).
+Usually it is slow if you do not have high speed networking like InfiniBand.
+
+### Launch multiple jobs on a single machine
+
+If you launch multiple jobs on a single machine, e.g., 2 jobs of 4-GPU training on a machine with 8 GPUs,
+you need to specify different ports (29500 by default) for each job to avoid communication conflict.
+
+If you use `dist_train.sh` to launch training jobs, you can set the port in commands.
+
+```shell
+CUDA_VISIBLE_DEVICES=0,1,2,3 PORT=29500 ./tools/dist_train.sh ${CONFIG_FILE} 4
+CUDA_VISIBLE_DEVICES=4,5,6,7 PORT=29501 ./tools/dist_train.sh ${CONFIG_FILE} 4
+```
+
+If you use launch training jobs with slurm, you need to modify the config files (usually the 4th line in config files) to set different communication ports.
+
+In `config1.py`,
+
+```python
+dist_params = dict(backend='nccl', port=29500)
+```
+
+In `config2.py`,
+
+```python
+dist_params = dict(backend='nccl', port=29501)
+```
+
+Then you can launch two jobs with `config1.py` ang `config2.py`.
+
+```shell
+CUDA_VISIBLE_DEVICES=0,1,2,3 ./tools/slurm_train.sh ${PARTITION} ${JOB_NAME} config1.py ${WORK_DIR} 4
+CUDA_VISIBLE_DEVICES=4,5,6,7 ./tools/slurm_train.sh ${PARTITION} ${JOB_NAME} config2.py ${WORK_DIR} 4
+```
+
+## Benchmark
+
+You can get average inference speed using the following script. Note that it does not include the IO time and the pre-processing time.
+
+```shell
+python tools/analysis/benchmark_inference.py ${MMPOSE_CONFIG_FILE}
+```
+
+## Tutorials
+
+We provide some tutorials for users:
+
+- [learn about configs](tutorials/0_config.md)
+- [finetune model](tutorials/1_finetune.md)
+- [add new dataset](tutorials/2_new_dataset.md)
+- [customize data pipelines](tutorials/3_data_pipeline.md)
+- [add new modules](tutorials/4_new_modules.md)
+- [export a model to ONNX](tutorials/5_export_model.md)
+- [customize runtime settings](tutorials/6_customize_runtime.md).
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/index.rst b/engine/pose_estimation/third-party/ViTPose/docs/en/index.rst
new file mode 100644
index 0000000..a562822
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/index.rst
@@ -0,0 +1,99 @@
+Welcome to MMPose's documentation!
+==================================
+
+You can change the documentation language at the lower-left corner of the page.
+
+您可以在页面左下角切换文档语言。
+
+.. toctree::
+ :maxdepth: 2
+
+ install.md
+ getting_started.md
+ demo.md
+ benchmark.md
+ inference_speed_summary.md
+
+.. toctree::
+ :maxdepth: 2
+ :caption: Datasets
+
+ datasets.md
+ tasks/2d_body_keypoint.md
+ tasks/2d_wholebody_keypoint.md
+ tasks/2d_face_keypoint.md
+ tasks/2d_hand_keypoint.md
+ tasks/2d_fashion_landmark.md
+ tasks/2d_animal_keypoint.md
+ tasks/3d_body_keypoint.md
+ tasks/3d_body_mesh.md
+ tasks/3d_hand_keypoint.md
+
+.. toctree::
+ :maxdepth: 2
+ :caption: Model Zoo
+
+ modelzoo.md
+ topics/animal.md
+ topics/body(2d,kpt,sview,img).md
+ topics/body(2d,kpt,sview,vid).md
+ topics/body(3d,kpt,sview,img).md
+ topics/body(3d,kpt,sview,vid).md
+ topics/body(3d,kpt,mview,img).md
+ topics/body(3d,mesh,sview,img).md
+ topics/face.md
+ topics/fashion.md
+ topics/hand(2d).md
+ topics/hand(3d).md
+ topics/wholebody.md
+
+.. toctree::
+ :maxdepth: 2
+ :caption: Model Zoo (by paper)
+
+ papers/algorithms.md
+ papers/backbones.md
+ papers/datasets.md
+ papers/techniques.md
+
+.. toctree::
+ :maxdepth: 2
+ :caption: Tutorials
+
+ tutorials/0_config.md
+ tutorials/1_finetune.md
+ tutorials/2_new_dataset.md
+ tutorials/3_data_pipeline.md
+ tutorials/4_new_modules.md
+ tutorials/5_export_model.md
+ tutorials/6_customize_runtime.md
+
+.. toctree::
+ :maxdepth: 2
+ :caption: Useful Tools and Scripts
+
+ useful_tools.md
+
+.. toctree::
+ :maxdepth: 2
+ :caption: Notes
+
+ changelog.md
+ faq.md
+
+.. toctree::
+ :caption: API Reference
+
+ api.rst
+
+.. toctree::
+ :caption: Languages
+
+ language.md
+
+
+Indices and tables
+==================
+
+* :ref:`genindex`
+* :ref:`search`
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/inference_speed_summary.md b/engine/pose_estimation/third-party/ViTPose/docs/en/inference_speed_summary.md
new file mode 100644
index 0000000..9d165ec
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/inference_speed_summary.md
@@ -0,0 +1,114 @@
+# Inference Speed
+
+We summarize the model complexity and inference speed of major models in MMPose, including FLOPs, parameter counts and inference speeds on both CPU and GPU devices with different batch sizes. We also compare the mAP of different models on COCO human keypoint dataset, showing the trade-off between model performance and model complexity.
+
+## Comparison Rules
+
+To ensure the fairness of the comparison, the comparison experiments are conducted under the same hardware and software environment using the same dataset. We also list the mAP (mean average precision) on COCO human keypoint dataset of the models along with the corresponding config files.
+
+For model complexity information measurement, we calculate the FLOPs and parameter counts of a model with corresponding input shape. Note that some layers or ops are currently not supported, for example, `DeformConv2d`, so you may need to check if all ops are supported and verify that the flops and parameter counts computation is correct.
+
+For inference speed, we omit the time for data pre-processing and only measure the time for model forwarding and data post-processing. For each model setting, we keep the same data pre-processing methods to make sure the same feature input. We measure the inference speed on both CPU and GPU devices. For topdown heatmap models, we also test the case when the batch size is larger, e.g., 10, to test model performance in crowded scenes.
+
+The inference speed is measured with frames per second (FPS), namely the average iterations per second, which can show how fast the model can handle an input. The higher, the faster, the better.
+
+### Hardware
+
+- GPU: GeForce GTX 1660 SUPER
+- CPU: Intel(R) Core(TM) i7-8700 CPU @ 3.20GHz
+
+### Software Environment
+
+- Ubuntu 16.04
+- Python 3.8
+- PyTorch 1.10
+- CUDA 10.2
+- mmcv-full 1.3.17
+- mmpose 0.20.0
+
+## Model complexity information and inference speed results of major models in MMPose
+
+| Algorithm | Model | config | Input size | mAP | Flops (GFLOPs) | Params (M) | GPU Inference Speed (FPS)1 | GPU Inference Speed (FPS, bs=10)2 | CPU Inference Speed (FPS) | CPU Inference Speed (FPS, bs=10) |
+| :--- | :---------------: | :-----------------: |:--------------------: | :----------------------------: | :-----------------: | :---------------: |:--------------------: | :----------------------------: | :-----------------: | :-----------------: |
+| topdown_heatmap | Alexnet | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/alexnet_coco_256x192.py) | (3, 192, 256) | 0.397 | 1.42 | 5.62 | 229.21 ± 16.91 | 33.52 ± 1.14 | 13.92 ± 0.60 | 1.38 ± 0.02 |
+| topdown_heatmap | CPM | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/cpm_coco_256x192.py) | (3, 192, 256) | 0.623 | 63.81 | 31.3 | 11.35 ± 0.22 | 3.87 ± 0.07 | 0.31 ± 0.01 | 0.03 ± 0.00 |
+| topdown_heatmap | CPM | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/cpm_coco_384x288.py) | (3, 288, 384) | 0.65 | 143.57 | 31.3 | 7.09 ± 0.14 | 2.10 ± 0.05 | 0.14 ± 0.00 | 0.01 ± 0.00 |
+| topdown_heatmap | Hourglass-52 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/hourglass52_coco_256x256.py) | (3, 256, 256) | 0.726 | 28.67 | 94.85 | 25.50 ± 1.68 | 3.99 ± 0.07 | 0.92 ± 0.03 | 0.09 ± 0.00 |
+| topdown_heatmap | Hourglass-52 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/hourglass52_coco_384x384.py) | (3, 384, 384) | 0.746 | 64.5 | 94.85 | 14.74 ± 0.8 | 1.86 ± 0.06 | 0.43 ± 0.03 | 0.04 ± 0.00 |
+| topdown_heatmap | HRNet-W32 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/hrnet_w32_coco_256x192.py) | (3, 192, 256) | 0.746 | 7.7 | 28.54 | 22.73 ± 1.12 | 6.60 ± 0.14 | 2.73 ± 0.11 | 0.32 ± 0.00 |
+| topdown_heatmap | HRNet-W32 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/hrnet_w32_coco_384x288.py) | (3, 288, 384) | 0.76 | 17.33 | 28.54 | 22.78 ± 1.21 | 3.28 ± 0.08 | 1.35 ± 0.05 | 0.14 ± 0.00 |
+| topdown_heatmap | HRNet-W48 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/hrnet_w48_coco_256x192.py) | (3, 192, 256) | 0.756 | 15.77 | 63.6 | 22.01 ± 1.10 | 3.74 ± 0.10 | 1.46 ± 0.05 | 0.16 ± 0.00 |
+| topdown_heatmap | HRNet-W48 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/hrnet_w48_coco_384x288.py) | (3, 288, 384) | 0.767 | 35.48 | 63.6 | 15.03 ± 1.03 | 1.80 ± 0.03 | 0.68 ± 0.02 | 0.07 ± 0.00 |
+| topdown_heatmap | LiteHRNet-30 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/litehrnet_30_coco_256x192.py) | (3, 192, 256) | 0.675 | 0.42 | 1.76 | 11.86 ± 0.38 | 9.77 ± 0.23 | 5.84 ± 0.39 | 0.80 ± 0.00 |
+| topdown_heatmap | LiteHRNet-30 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/litehrnet_30_coco_384x288.py) | (3, 288, 384) | 0.7 | 0.95 | 1.76 | 11.52 ± 0.39 | 5.18 ± 0.11 | 3.45 ± 0.22 | 0.37 ± 0.00 |
+| topdown_heatmap | MobilenetV2 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/mobilenetv2_coco_256x192.py) | (3, 192, 256) | 0.646 | 1.59 | 9.57 | 91.82 ± 10.98 | 17.85 ± 0.32 | 10.44 ± 0.80 | 1.05 ± 0.01 |
+| topdown_heatmap | MobilenetV2 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/mobilenetv2_coco_384x288.py) | (3, 288, 384) | 0.673 | 3.57 | 9.57 | 71.27 ± 6.82 | 8.00 ± 0.15 | 5.01 ± 0.32 | 0.46 ± 0.00 |
+| topdown_heatmap | MSPN-50 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/mspn50_coco_256x192.py) | (3, 192, 256) | 0.723 | 5.11 | 25.11 | 59.65 ± 3.74 | 9.51 ± 0.15 | 3.98 ± 0.21 | 0.43 ± 0.00 |
+| topdown_heatmap | 2xMSPN-50 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/2xmspn50_coco_256x192.py) | (3, 192, 256) | 0.754 | 11.35 | 56.8 | 30.64 ± 2.61 | 4.74 ± 0.12 | 1.85 ± 0.08 | 0.20 ± 0.00 |
+| topdown_heatmap | 3xMSPN-50 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/3xmspn50_coco_256x192.py) | (3, 192, 256) | 0.758 | 17.59 | 88.49 | 20.90 ± 1.82 | 3.22 ± 0.08 | 1.23 ± 0.04 | 0.13 ± 0.00 |
+| topdown_heatmap | 4xMSPN-50 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/4xmspn50_coco_256x192.py) | (3, 192, 256) | 0.764 | 23.82 | 120.18 | 15.79 ± 1.14 | 2.45 ± 0.05 | 0.90 ± 0.03 | 0.10 ± 0.00 |
+| topdown_heatmap | ResNest-50 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/resnest50_coco_256x192.py) | (3, 192, 256) | 0.721 | 6.73 | 35.93 | 48.36 ± 4.12 | 7.48 ± 0.13 | 3.00 ± 0.13 | 0.33 ± 0.00 |
+| topdown_heatmap | ResNest-50 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/resnest50_coco_384x288.py) | (3, 288, 384) | 0.737 | 15.14 | 35.93 | 30.30 ± 2.30 | 3.62 ± 0.09 | 1.43 ± 0.05 | 0.13 ± 0.00 |
+| topdown_heatmap | ResNest-101 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/resnest101_coco_256x192.py) | (3, 192, 256) | 0.725 | 10.38 | 56.61 | 29.21 ± 1.98 | 5.30 ± 0.12 | 2.01 ± 0.08 | 0.22 ± 0.00 |
+| topdown_heatmap | ResNest-101 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/resnest101_coco_384x288.py) | (3, 288, 384) | 0.746 | 23.36 | 56.61 | 19.02 ± 1.40 | 2.59 ± 0.05 | 0.97 ± 0.03 | 0.09 ± 0.00 |
+| topdown_heatmap | ResNest-200 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/resnest200_coco_256x192.py) | (3, 192, 256) | 0.732 | 17.5 | 78.54 | 16.11 ± 0.71 | 3.29 ± 0.07 | 1.33 ± 0.02 | 0.14 ± 0.00 |
+| topdown_heatmap | ResNest-200 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/resnest200_coco_384x288.py) | (3, 288, 384) | 0.754 | 39.37 | 78.54 | 11.48 ± 0.68 | 1.58 ± 0.02 | 0.63 ± 0.01 | 0.06 ± 0.00 |
+| topdown_heatmap | ResNest-269 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/resnest269_coco_256x192.py) | (3, 192, 256) | 0.738 | 22.45 | 119.27 | 12.02 ± 0.47 | 2.60 ± 0.05 | 1.03 ± 0.01 | 0.11 ± 0.00 |
+| topdown_heatmap | ResNest-269 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/resnest269_coco_384x288.py) | (3, 288, 384) | 0.755 | 50.5 | 119.27 | 8.82 ± 0.42 | 1.24 ± 0.02 | 0.49 ± 0.01 | 0.05 ± 0.00 |
+| topdown_heatmap | ResNet-50 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/res50_coco_256x192.py) | (3, 192, 256) | 0.718 | 5.46 | 34 | 64.23 ± 6.05 | 9.33 ± 0.21 | 4.00 ± 0.10 | 0.41 ± 0.00 |
+| topdown_heatmap | ResNet-50 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/res50_coco_384x288.py) | (3, 288, 384) | 0.731 | 12.29 | 34 | 36.78 ± 3.05 | 4.48 ± 0.12 | 1.92 ± 0.04 | 0.19 ± 0.00 |
+| topdown_heatmap | ResNet-101 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/res101_coco_256x192.py) | (3, 192, 256) | 0.726 | 9.11 | 52.99 | 43.35 ± 4.36 | 6.44 ± 0.14 | 2.57 ± 0.05 | 0.27 ± 0.00 |
+| topdown_heatmap | ResNet-101 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/res101_coco_384x288.py) | (3, 288, 384) | 0.748 | 20.5 | 52.99 | 23.29 ± 1.83 | 3.12 ± 0.09 | 1.23 ± 0.03 | 0.11 ± 0.00 |
+| topdown_heatmap | ResNet-152 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/res152_coco_256x192.py) | (3, 192, 256) | 0.735 | 12.77 | 68.64 | 32.31 ± 2.84 | 4.88 ± 0.17 | 1.89 ± 0.03 | 0.20 ± 0.00 |
+| topdown_heatmap | ResNet-152 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/res152_coco_384x288.py) | (3, 288, 384) | 0.75 | 28.73 | 68.64 | 17.32 ± 1.17 | 2.40 ± 0.04 | 0.91 ± 0.01 | 0.08 ± 0.00 |
+| topdown_heatmap | ResNetV1d-50 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/resnetv1d50_coco_256x192.py) | (3, 192, 256) | 0.722 | 5.7 | 34.02 | 63.44 ± 6.09 | 9.09 ± 0.10 | 3.82 ± 0.10 | 0.39 ± 0.00 |
+| topdown_heatmap | ResNetV1d-50 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/resnetv1d50_coco_384x288.py) | (3, 288, 384) | 0.73 | 12.82 | 34.02 | 36.21 ± 3.10 | 4.30 ± 0.12 | 1.82 ± 0.04 | 0.16 ± 0.00 |
+| topdown_heatmap | ResNetV1d-101 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/resnetv1d101_coco_256x192.py) | (3, 192, 256) | 0.731 | 9.35 | 53.01 | 41.48 ± 3.76 | 6.33 ± 0.15 | 2.48 ± 0.05 | 0.26 ± 0.00 |
+| topdown_heatmap | ResNetV1d-101 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/resnetv1d101_coco_384x288.py) | (3, 288, 384) | 0.748 | 21.04 | 53.01 | 23.49 ± 1.76 | 3.07 ± 0.07 | 1.19 ± 0.02 | 0.11 ± 0.00 |
+| topdown_heatmap | ResNetV1d-152 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/resnetv1d152_coco_256x192.py) | (3, 192, 256) | 0.737 | 13.01 | 68.65 | 31.96 ± 2.87 | 4.69 ± 0.18 | 1.87 ± 0.02 | 0.19 ± 0.00 |
+| topdown_heatmap | ResNetV1d-152 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/resnetv1d152_coco_384x288.py) | (3, 288, 384) | 0.752 | 29.26 | 68.65 | 17.31 ± 1.13 | 2.32 ± 0.04 | 0.88 ± 0.01 | 0.08 ± 0.00 |
+| topdown_heatmap | ResNext-50 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/resnext50_coco_256x192.py) | (3, 192, 256) | 0.714 | 5.61 | 33.47 | 48.34 ± 3.85 | 7.66 ± 0.13 | 3.71 ± 0.10 | 0.37 ± 0.00 |
+| topdown_heatmap | ResNext-50 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/resnext50_coco_384x288.py) | (3, 288, 384) | 0.724 | 12.62 | 33.47 | 30.66 ± 2.38 | 3.64 ± 0.11 | 1.73 ± 0.03 | 0.15 ± 0.00 |
+| topdown_heatmap | ResNext-101 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/resnext101_coco_256x192.py) | (3, 192, 256) | 0.726 | 9.29 | 52.62 | 27.33 ± 2.35 | 5.09 ± 0.13 | 2.45 ± 0.04 | 0.25 ± 0.00 |
+| topdown_heatmap | ResNext-101 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/resnext101_coco_384x288.py) | (3, 288, 384) | 0.743 | 20.91 | 52.62 | 18.19 ± 1.38 | 2.42 ± 0.04 | 1.15 ± 0.01 | 0.10 ± 0.00 |
+| topdown_heatmap | ResNext-152 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/resnext152_coco_256x192.py) | (3, 192, 256) | 0.73 | 12.98 | 68.39 | 19.61 ± 1.61 | 3.80 ± 0.13 | 1.83 ± 0.02 | 0.18 ± 0.00 |
+| topdown_heatmap | ResNext-152 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/resnext152_coco_384x288.py) | (3, 288, 384) | 0.742 | 29.21 | 68.39 | 13.14 ± 0.75 | 1.82 ± 0.03 | 0.85 ± 0.01 | 0.08 ± 0.00 |
+| topdown_heatmap | RSN-18 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/rsn18_coco_256x192.py) | (3, 192, 256) | 0.704 | 2.27 | 9.14 | 47.80 ± 4.50 | 13.68 ± 0.25 | 6.70 ± 0.28 | 0.70 ± 0.00 |
+| topdown_heatmap | RSN-50 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/rsn50_coco_256x192.py) | (3, 192, 256) | 0.723 | 4.11 | 19.33 | 27.22 ± 1.61 | 8.81 ± 0.13 | 3.98 ± 0.12 | 0.45 ± 0.00 |
+| topdown_heatmap | 2xRSN-50 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/2xrsn50_coco_256x192.py) | (3, 192, 256) | 0.745 | 8.29 | 39.26 | 13.88 ± 0.64 | 4.78 ± 0.13 | 2.02 ± 0.04 | 0.23 ± 0.00 |
+| topdown_heatmap | 3xRSN-50 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/3xrsn50_coco_256x192.py) | (3, 192, 256) | 0.75 | 12.47 | 59.2 | 9.40 ± 0.32 | 3.37 ± 0.09 | 1.34 ± 0.03 | 0.15 ± 0.00 |
+| topdown_heatmap | SCNet-50 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/scnet50_coco_256x192.py) | (3, 192, 256) | 0.728 | 5.31 | 34.01 | 40.76 ± 3.08 | 8.35 ± 0.19 | 3.82 ± 0.08 | 0.40 ± 0.00 |
+| topdown_heatmap | SCNet-50 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/scnet50_coco_384x288.py) | (3, 288, 384) | 0.751 | 11.94 | 34.01 | 32.61 ± 2.97 | 4.19 ± 0.10 | 1.85 ± 0.03 | 0.17 ± 0.00 |
+| topdown_heatmap | SCNet-101 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/scnet101_coco_256x192.py) | (3, 192, 256) | 0.733 | 8.51 | 53.01 | 24.28 ± 1.19 | 5.80 ± 0.13 | 2.49 ± 0.05 | 0.27 ± 0.00 |
+| topdown_heatmap | SCNet-101 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/scnet101_coco_384x288.py) | (3, 288, 384) | 0.752 | 19.14 | 53.01 | 20.43 ± 1.76 | 2.91 ± 0.06 | 1.23 ± 0.02 | 0.12 ± 0.00 |
+| topdown_heatmap | SeresNet-50 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/seresnet50_coco_256x192.py) | (3, 192, 256) | 0.728 | 5.47 | 36.53 | 54.83 ± 4.94 | 8.80 ± 0.12 | 3.85 ± 0.10 | 0.40 ± 0.00 |
+| topdown_heatmap | SeresNet-50 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/seresnet50_coco_384x288.py) | (3, 288, 384) | 0.748 | 12.3 | 36.53 | 33.00 ± 2.67 | 4.26 ± 0.12 | 1.86 ± 0.04 | 0.17 ± 0.00 |
+| topdown_heatmap | SeresNet-101 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/seresnet101_coco_256x192.py) | (3, 192, 256) | 0.734 | 9.13 | 57.77 | 33.90 ± 2.65 | 6.01 ± 0.13 | 2.48 ± 0.05 | 0.26 ± 0.00 |
+| topdown_heatmap | SeresNet-101 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/seresnet101_coco_384x288.py) | (3, 288, 384) | 0.753 | 20.53 | 57.77 | 20.57 ± 1.57 | 2.96 ± 0.07 | 1.20 ± 0.02 | 0.11 ± 0.00 |
+| topdown_heatmap | SeresNet-152 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/seresnet152_coco_256x192.py) | (3, 192, 256) | 0.73 | 12.79 | 75.26 | 24.25 ± 1.95 | 4.45 ± 0.10 | 1.82 ± 0.02 | 0.19 ± 0.00 |
+| topdown_heatmap | SeresNet-152 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/seresnet152_coco_384x288.py) | (3, 288, 384) | 0.753 | 28.76 | 75.26 | 15.11 ± 0.99 | 2.25 ± 0.04 | 0.88 ± 0.01 | 0.08 ± 0.00 |
+| topdown_heatmap | ShuffleNetV1 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/shufflenetv1_coco_256x192.py) | (3, 192, 256) | 0.585 | 1.35 | 6.94 | 80.79 ± 8.95 | 21.91 ± 0.46 | 11.84 ± 0.59 | 1.25 ± 0.01 |
+| topdown_heatmap | ShuffleNetV1 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/shufflenetv1_coco_384x288.py) | (3, 288, 384) | 0.622 | 3.05 | 6.94 | 63.45 ± 5.21 | 9.84 ± 0.10 | 6.01 ± 0.31 | 0.57 ± 0.00 |
+| topdown_heatmap | ShuffleNetV2 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/shufflenetv2_coco_256x192.py) | (3, 192, 256) | 0.599 | 1.37 | 7.55 | 82.36 ± 7.30 | 22.68 ± 0.53 | 12.40 ± 0.66 | 1.34 ± 0.02 |
+| topdown_heatmap | ShuffleNetV2 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/shufflenetv2_coco_384x288.py) | (3, 288, 384) | 0.636 | 3.08 | 7.55 | 63.63 ± 5.72 | 10.47 ± 0.16 | 6.32 ± 0.28 | 0.63 ± 0.01 |
+| topdown_heatmap | VGG16 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/vgg16_bn_coco_256x192.py) | (3, 192, 256) | 0.698 | 16.22 | 18.92 | 51.91 ± 2.98 | 6.18 ± 0.13 | 1.64 ± 0.03 | 0.15 ± 0.00 |
+| topdown_heatmap | VIPNAS + ResNet-50 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/vipnas_res50_coco_256x192.py) | (3, 192, 256) | 0.711 | 1.49 | 7.29 | 34.88 ± 2.45 | 10.29 ± 0.13 | 6.51 ± 0.17 | 0.65 ± 0.00 |
+| topdown_heatmap | VIPNAS + MobileNetV3 | [config](/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/vipnas_mbv3_coco_256x192.py) | (3, 192, 256) | 0.7 | 0.76 | 5.9 | 53.62 ± 6.59 | 11.54 ± 0.18 | 1.26 ± 0.02 | 0.13 ± 0.00 |
+| Associative Embedding | HigherHRNet-W32 | [config](/configs/body/2d_kpt_sview_rgb_img/associative_embedding/coco/higherhrnet_w32_coco_512x512.py) | (3, 512, 512) | 0.677 | 46.58 | 28.65 | 7.80 ± 0.67 | / | 0.28 ± 0.02 | / |
+| Associative Embedding | HigherHRNet-W32 | [config](/configs/body/2d_kpt_sview_rgb_img/associative_embedding/coco/higherhrnet_w32_coco_640x640.py) | (3, 640, 640) | 0.686 | 72.77 | 28.65 | 5.30 ± 0.37 | / | 0.17 ± 0.01 | / |
+| Associative Embedding | HigherHRNet-W48 | [config](/configs/body/2d_kpt_sview_rgb_img/associative_embedding/coco/higherhrnet_w48_coco_512x512.py) | (3, 512, 512) | 0.686 | 96.17 | 63.83 | 4.55 ± 0.35 | / | 0.15 ± 0.01 | / |
+| Associative Embedding | Hourglass-AE | [config](/configs/body/2d_kpt_sview_rgb_img/associative_embedding/coco/hourglass_ae_coco_512x512.py) | (3, 512, 512) | 0.613 | 221.58 | 138.86 | 3.55 ± 0.24 | / | 0.08 ± 0.00 | / |
+| Associative Embedding | HRNet-W32 | [config](/configs/body/2d_kpt_sview_rgb_img/associative_embedding/coco/hrnet_w32_coco_512x512.py) | (3, 512, 512) | 0.654 | 41.1 | 28.54 | 8.93 ± 0.76 | / | 0.33 ± 0.02 | / |
+| Associative Embedding | HRNet-W48 | [config](/configs/body/2d_kpt_sview_rgb_img/associative_embedding/coco/hrnet_w48_coco_512x512.py) | (3, 512, 512) | 0.665 | 84.12 | 63.6 | 5.27 ± 0.43 | / | 0.18 ± 0.01 | / |
+| Associative Embedding | MobilenetV2 | [config](/configs/body/2d_kpt_sview_rgb_img/associative_embedding/coco/mobilenetv2_coco_512x512.py) | (3, 512, 512) | 0.38 | 8.54 | 9.57 | 21.24 ± 1.34 | / | 0.81 ± 0.06 | / |
+| Associative Embedding | ResNet-50 | [config](/configs/body/2d_kpt_sview_rgb_img/associative_embedding/coco/res50_coco_512x512.py) | (3, 512, 512) | 0.466 | 29.2 | 34 | 11.71 ± 0.97 | / | 0.41 ± 0.02 | / |
+| Associative Embedding | ResNet-50 | [config](/configs/body/2d_kpt_sview_rgb_img/associative_embedding/coco/res50_coco_640x640.py) | (3, 640, 640) | 0.479 | 45.62 | 34 | 8.20 ± 0.58 | / | 0.26 ± 0.02 | / |
+| Associative Embedding | ResNet-101 | [config](/configs/body/2d_kpt_sview_rgb_img/associative_embedding/coco/res101_coco_512x512.py) | (3, 512, 512) | 0.554 | 48.67 | 53 | 8.26 ± 0.68 | / | 0.28 ± 0.02 | / |
+| Associative Embedding | ResNet-101 | [config](/configs/body/2d_kpt_sview_rgb_img/associative_embedding/coco/res152_coco_512x512.py) | (3, 512, 512) | 0.595 | 68.17 | 68.64 | 6.25 ± 0.53 | / | 0.21 ± 0.01 | / |
+| DeepPose | ResNet-50 | [config](/configs/body/2d_kpt_sview_rgb_img/deeppose/coco/res50_coco_256x192.py) | (3, 192, 256) | 0.526 | 4.04 | 23.58 | 82.20 ± 7.54 | / | 5.50 ± 0.18 | / |
+| DeepPose | ResNet-101 | [config](/configs/body/2d_kpt_sview_rgb_img/deeppose/coco/res101_coco_256x192.py) | (3, 192, 256) | 0.56 | 7.69 | 42.57 | 48.93 ± 4.02 | / | 3.10 ± 0.07 | / |
+| DeepPose | ResNet-152 | [config](/configs/body/2d_kpt_sview_rgb_img/deeppose/coco/res152_coco_256x192.py) | (3, 192, 256) | 0.583 | 11.34 | 58.21 | 35.06 ± 3.50 | / | 2.19 ± 0.04 | / |
+
+1 Note that we run multiple iterations and record the time of each iteration, and the mean and standard deviation value of FPS are both shown.
+
+2 The FPS is defined as the average iterations per second, regardless of the batch size in this iteration.
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/install.md b/engine/pose_estimation/third-party/ViTPose/docs/en/install.md
new file mode 100644
index 0000000..a668b23
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/install.md
@@ -0,0 +1,202 @@
+# Installation
+
+
+
+- [Requirements](#requirements)
+- [Prepare Environment](#prepare-environment)
+- [Install MMPose](#install-mmpose)
+- [Install with CPU only](#install-with-cpu-only)
+- [A from-scratch setup script](#a-from-scratch-setup-script)
+- [Another option: Docker Image](#another-option-docker-image)
+- [Developing with multiple MMPose versions](#developing-with-multiple-mmpose-versions)
+
+
+
+## Requirements
+
+- Linux (Windows is not officially supported)
+- Python 3.6+
+- PyTorch 1.3+
+- CUDA 9.2+ (If you build PyTorch from source, CUDA 9.0 is also compatible)
+- GCC 5+
+- [mmcv](https://github.com/open-mmlab/mmcv) (Please install the latest version of mmcv-full)
+- Numpy
+- cv2
+- json_tricks
+- [xtcocotools](https://github.com/jin-s13/xtcocoapi)
+
+Optional:
+
+- [mmdet](https://github.com/open-mmlab/mmdetection) (to run pose demos)
+- [mmtrack](https://github.com/open-mmlab/mmtracking) (to run pose tracking demos)
+- [pyrender](https://pyrender.readthedocs.io/en/latest/install/index.html) (to run 3d mesh demos)
+- [smplx](https://github.com/vchoutas/smplx) (to run 3d mesh demos)
+
+## Prepare environment
+
+a. Create a conda virtual environment and activate it.
+
+```shell
+conda create -n open-mmlab python=3.7 -y
+conda activate open-mmlab
+```
+
+b. Install PyTorch and torchvision following the [official instructions](https://pytorch.org/), e.g.,
+
+```shell
+conda install pytorch torchvision -c pytorch
+```
+
+```{note}
+Make sure that your compilation CUDA version and runtime CUDA version match.
+```
+
+You can check the supported CUDA version for precompiled packages on the [PyTorch website](https://pytorch.org/).
+
+`E.g.1` If you have CUDA 10.2 installed under `/usr/local/cuda` and would like to install PyTorch 1.8.0,
+you need to install the prebuilt PyTorch with CUDA 10.2.
+
+```shell
+conda install pytorch==1.8.0 torchvision==0.9.0 cudatoolkit=10.2 -c pytorch
+```
+
+`E.g.2` If you have CUDA 9.2 installed under `/usr/local/cuda` and would like to install PyTorch 1.7.0.,
+you need to install the prebuilt PyTorch with CUDA 9.2.
+
+```shell
+conda install pytorch==1.7.0 torchvision==0.8.0 cudatoolkit=9.2 -c pytorch
+```
+
+If you build PyTorch from source instead of installing the pre-built package, you can use more CUDA versions such as 9.0.
+
+## Install MMPose
+
+a. Install mmcv, we recommend you to install the pre-built mmcv as below.
+
+```shell
+# pip install mmcv-full -f https://download.openmmlab.com/mmcv/dist/{cu_version}/{torch_version}/index.html
+pip install mmcv-full -f https://download.openmmlab.com/mmcv/dist/cu102/torch1.9.0/index.html
+# We can ignore the micro version of PyTorch
+pip install mmcv-full -f https://download.openmmlab.com/mmcv/dist/cu102/torch1.9/index.html
+```
+
+mmcv-full is only compiled on PyTorch 1.x.0 because the compatibility usually holds between 1.x.0 and 1.x.1. If your PyTorch version is 1.x.1, you can install mmcv-full compiled with PyTorch 1.x.0 and it usually works well.
+
+See [here](https://github.com/open-mmlab/mmcv#installation) for different versions of MMCV compatible to different PyTorch and CUDA versions.
+
+Optionally you can choose to compile mmcv from source by the following command
+
+```shell
+git clone https://github.com/open-mmlab/mmcv.git
+cd mmcv
+MMCV_WITH_OPS=1 pip install -e . # package mmcv-full, which contains cuda ops, will be installed after this step
+# OR pip install -e . # package mmcv, which contains no cuda ops, will be installed after this step
+cd ..
+```
+
+**Important:** You need to run `pip uninstall mmcv` first if you have mmcv installed. If mmcv and mmcv-full are both installed, there will be `ModuleNotFoundError`.
+
+b. Clone the mmpose repository
+
+```shell
+git clone git@github.com:open-mmlab/mmpose.git # or git clone https://github.com/open-mmlab/mmpose
+cd mmpose
+```
+
+c. Install build requirements and then install mmpose
+
+```shell
+pip install -r requirements.txt
+pip install -v -e . # or "python setup.py develop"
+```
+
+If you build MMPose on macOS, replace the last command with
+
+```shell
+CC=clang CXX=clang++ CFLAGS='-stdlib=libc++' pip install -e .
+```
+
+d. Install optional modules
+
+- [mmdet](https://github.com/open-mmlab/mmdetection) (to run pose demos)
+- [mmtrack](https://github.com/open-mmlab/mmtracking) (to run pose tracking demos)
+- [pyrender](https://pyrender.readthedocs.io/en/latest/install/index.html) (to run 3d mesh demos)
+- [smplx](https://github.com/vchoutas/smplx) (to run 3d mesh demos)
+
+```{note}
+1. The git commit id will be written to the version number with step c, e.g. 0.6.0+2e7045c. The version will also be saved in trained models.
+ It is recommended that you run step d each time you pull some updates from github. If C++/CUDA codes are modified, then this step is compulsory.
+
+1. Following the above instructions, mmpose is installed on `dev` mode, any local modifications made to the code will take effect without the need to reinstall it (unless you submit some commits and want to update the version number).
+
+1. If you would like to use `opencv-python-headless` instead of `opencv-python`,
+ you can install it before installing MMCV.
+
+1. If you have `mmcv` installed, you need to firstly uninstall `mmcv`, and then install `mmcv-full`.
+
+1. Some dependencies are optional. Running `python setup.py develop` will only install the minimum runtime requirements.
+ To use optional dependencies like `smplx`, either install them with `pip install -r requirements/optional.txt`
+ or specify desired extras when calling `pip` (e.g. `pip install -v -e .[optional]`,
+ valid keys for the `[optional]` field are `all`, `tests`, `build`, and `optional`) like `pip install -v -e .[tests,build]`.
+```
+
+## Install with CPU only
+
+The code can be built for CPU only environment (where CUDA isn't available).
+
+In CPU mode you can run the demo/demo.py for example.
+
+## A from-scratch setup script
+
+Here is a full script for setting up mmpose with conda and link the dataset path (supposing that your COCO dataset path is $COCO_ROOT).
+
+```shell
+conda create -n open-mmlab python=3.7 -y
+conda activate open-mmlab
+
+# install latest pytorch prebuilt with the default prebuilt CUDA version (usually the latest)
+conda install -c pytorch pytorch torchvision -y
+
+# install the latest mmcv-full
+# Please replace ``{cu_version}`` and ``{torch_version}`` in the url to your desired one.
+# See [here](https://github.com/open-mmlab/mmcv#installation) for different versions of MMCV compatible to different PyTorch and CUDA versions.
+pip install mmcv-full -f https://download.openmmlab.com/mmcv/dist/{cu_version}/{torch_version}/index.html
+
+# install mmpose
+git clone https://github.com/open-mmlab/mmpose.git
+cd mmpose
+pip install -r requirements.txt
+pip install -v -e .
+
+mkdir data
+ln -s $COCO_ROOT data/coco
+```
+
+## Another option: Docker Image
+
+We provide a [Dockerfile](/docker/Dockerfile) to build an image.
+
+```shell
+# build an image with PyTorch 1.6.0, CUDA 10.1, CUDNN 7.
+docker build -f ./docker/Dockerfile --rm -t mmpose .
+```
+
+**Important:** Make sure you've installed the [nvidia-container-toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html#docker).
+
+Run the following cmd:
+
+```shell
+docker run --gpus all\
+ --shm-size=8g \
+ -it -v {DATA_DIR}:/mmpose/data mmpose
+```
+
+## Developing with multiple MMPose versions
+
+The train and test scripts already modify the `PYTHONPATH` to ensure the script use the MMPose in the current directory.
+
+To use the default MMPose installed in the environment rather than that you are working with, you can remove the following line in those scripts.
+
+```shell
+PYTHONPATH="$(dirname $0)/..":$PYTHONPATH
+```
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/language.md b/engine/pose_estimation/third-party/ViTPose/docs/en/language.md
new file mode 100644
index 0000000..a0a6259
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/language.md
@@ -0,0 +1,3 @@
+## English
+
+## 简体中文
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/make.bat b/engine/pose_estimation/third-party/ViTPose/docs/en/make.bat
new file mode 100644
index 0000000..922152e
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/make.bat
@@ -0,0 +1,35 @@
+@ECHO OFF
+
+pushd %~dp0
+
+REM Command file for Sphinx documentation
+
+if "%SPHINXBUILD%" == "" (
+ set SPHINXBUILD=sphinx-build
+)
+set SOURCEDIR=.
+set BUILDDIR=_build
+
+if "%1" == "" goto help
+
+%SPHINXBUILD% >NUL 2>NUL
+if errorlevel 9009 (
+ echo.
+ echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
+ echo.installed, then set the SPHINXBUILD environment variable to point
+ echo.to the full path of the 'sphinx-build' executable. Alternatively you
+ echo.may add the Sphinx directory to PATH.
+ echo.
+ echo.If you don't have Sphinx installed, grab it from
+ echo.http://sphinx-doc.org/
+ exit /b 1
+)
+
+%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
+goto end
+
+:help
+%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
+
+:end
+popd
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/merge_docs.sh b/engine/pose_estimation/third-party/ViTPose/docs/en/merge_docs.sh
new file mode 100644
index 0000000..6484b78
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/merge_docs.sh
@@ -0,0 +1,28 @@
+#!/usr/bin/env bash
+# Copyright (c) OpenMMLab. All rights reserved.
+
+sed -i '$a\\n' ../../demo/docs/*_demo.md
+cat ../../demo/docs/*_demo.md | sed "s/#/#&/" | sed "s/md###t/html#t/g" | sed '1i\# Demo' | sed 's=](/docs/en/=](/=g' | sed 's=](/=](https://github.com/open-mmlab/mmpose/tree/master/=g' >demo.md
+
+ # remove /docs/ for link used in doc site
+sed -i 's=](/docs/en/=](=g' ./tutorials/*.md
+sed -i 's=](/docs/en/=](=g' ./tasks/*.md
+sed -i 's=](/docs/en/=](=g' ./papers/*.md
+sed -i 's=](/docs/en/=](=g' ./topics/*.md
+sed -i 's=](/docs/en/=](=g' data_preparation.md
+sed -i 's=](/docs/en/=](=g' getting_started.md
+sed -i 's=](/docs/en/=](=g' install.md
+sed -i 's=](/docs/en/=](=g' benchmark.md
+sed -i 's=](/docs/en/=](=g' changelog.md
+sed -i 's=](/docs/en/=](=g' faq.md
+
+sed -i 's=](/=](https://github.com/open-mmlab/mmpose/tree/master/=g' ./tutorials/*.md
+sed -i 's=](/=](https://github.com/open-mmlab/mmpose/tree/master/=g' ./tasks/*.md
+sed -i 's=](/=](https://github.com/open-mmlab/mmpose/tree/master/=g' ./papers/*.md
+sed -i 's=](/=](https://github.com/open-mmlab/mmpose/tree/master/=g' ./topics/*.md
+sed -i 's=](/=](https://github.com/open-mmlab/mmpose/tree/master/=g' data_preparation.md
+sed -i 's=](/=](https://github.com/open-mmlab/mmpose/tree/master/=g' getting_started.md
+sed -i 's=](/=](https://github.com/open-mmlab/mmpose/tree/master/=g' install.md
+sed -i 's=](/=](https://github.com/open-mmlab/mmpose/tree/master/=g' benchmark.md
+sed -i 's=](/=](https://github.com/open-mmlab/mmpose/tree/master/=g' changelog.md
+sed -i 's=](/=](https://github.com/open-mmlab/mmpose/tree/master/=g' faq.md
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/associative_embedding.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/associative_embedding.md
new file mode 100644
index 0000000..3a27267
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/associative_embedding.md
@@ -0,0 +1,30 @@
+# Associative embedding: End-to-end learning for joint detection and grouping (AE)
+
+
+
+
+Associative Embedding (NIPS'2017)
+
+```bibtex
+@inproceedings{newell2017associative,
+ title={Associative embedding: End-to-end learning for joint detection and grouping},
+ author={Newell, Alejandro and Huang, Zhiao and Deng, Jia},
+ booktitle={Advances in neural information processing systems},
+ pages={2277--2287},
+ year={2017}
+}
+```
+
+
+
+## Abstract
+
+
+
+We introduce associative embedding, a novel method for supervising convolutional neural networks for the task of detection and grouping. A number of computer vision problems can be framed in this manner including multi-person pose estimation, instance segmentation, and multi-object tracking. Usually the grouping of detections is achieved with multi-stage pipelines, instead we propose an approach that teaches a network to simultaneously output detections and group assignments. This technique can be easily integrated into any state-of-the-art network architecture that produces pixel-wise predictions. We show how to apply this method to both multi-person pose estimation and instance segmentation and report state-of-the-art performance for multi-person pose on the MPII and MS-COCO datasets.
+
+
+
+
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/awingloss.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/awingloss.md
new file mode 100644
index 0000000..4d4b93a
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/awingloss.md
@@ -0,0 +1,31 @@
+# Adaptive Wing Loss for Robust Face Alignment via Heatmap Regression
+
+
+
+
+AdaptiveWingloss (ICCV'2019)
+
+```bibtex
+@inproceedings{wang2019adaptive,
+ title={Adaptive wing loss for robust face alignment via heatmap regression},
+ author={Wang, Xinyao and Bo, Liefeng and Fuxin, Li},
+ booktitle={Proceedings of the IEEE/CVF international conference on computer vision},
+ pages={6971--6981},
+ year={2019}
+}
+```
+
+
+
+## Abstract
+
+
+
+Heatmap regression with a deep network has become one of the mainstream approaches to localize facial landmarks. However, the loss function for heatmap regression is rarely studied. In this paper, we analyze the ideal loss function properties for heatmap regression in face alignment problems. Then we propose a novel loss function, named Adaptive Wing loss, that is able to adapt its shape to different types of ground truth heatmap pixels. This adaptability penalizes loss more on foreground pixels while less on background pixels. To address the imbalance between foreground and background pixels, we also propose Weighted Loss Map, which assigns high weights on foreground and difficult background pixels to help training process focus more on pixels that are crucial to landmark localization. To further improve face alignment accuracy, we introduce boundary prediction and CoordConv with boundary coordinates. Extensive experiments on different benchmarks, including COFW, 300W and WFLW, show our approach outperforms the state-of-the-art by a significant margin on
+various evaluation metrics. Besides, the Adaptive Wing loss also helps other heatmap regression tasks.
+
+
+
+
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/cpm.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/cpm.md
new file mode 100644
index 0000000..fb5dbfa
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/cpm.md
@@ -0,0 +1,30 @@
+# Convolutional pose machines
+
+
+
+
+CPM (CVPR'2016)
+
+```bibtex
+@inproceedings{wei2016convolutional,
+ title={Convolutional pose machines},
+ author={Wei, Shih-En and Ramakrishna, Varun and Kanade, Takeo and Sheikh, Yaser},
+ booktitle={Proceedings of the IEEE conference on Computer Vision and Pattern Recognition},
+ pages={4724--4732},
+ year={2016}
+}
+```
+
+
+
+## Abstract
+
+
+
+We introduce associative embedding, a novel method for supervising convolutional neural networks for the task of detection and grouping. A number of computer vision problems can be framed in this manner including multi-person pose estimation, instance segmentation, and multi-object tracking. Usually the grouping of detections is achieved with multi-stage pipelines, instead we propose an approach that teaches a network to simultaneously output detections and group assignments. This technique can be easily integrated into any state-of-the-art network architecture that produces pixel-wise predictions. We show how to apply this method to both multi-person pose estimation and instance segmentation and report state-of-the-art performance for multi-person pose on the MPII and MS-COCO datasets.
+
+
+
+
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/dark.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/dark.md
new file mode 100644
index 0000000..083b759
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/dark.md
@@ -0,0 +1,30 @@
+# Distribution-aware coordinate representation for human pose estimation
+
+
+
+
+DarkPose (CVPR'2020)
+
+```bibtex
+@inproceedings{zhang2020distribution,
+ title={Distribution-aware coordinate representation for human pose estimation},
+ author={Zhang, Feng and Zhu, Xiatian and Dai, Hanbin and Ye, Mao and Zhu, Ce},
+ booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
+ pages={7093--7102},
+ year={2020}
+}
+```
+
+
+
+## Abstract
+
+
+
+While being the de facto standard coordinate representation for human pose estimation, heatmap has not been investigated in-depth. This work fills this gap. For the first time, we find that the process of decoding the predicted heatmaps into the final joint coordinates in the original image space is surprisingly significant for the performance. We further probe the design limitations of the standard coordinate decoding method, and propose a more principled distributionaware decoding method. Also, we improve the standard coordinate encoding process (i.e. transforming ground-truth coordinates to heatmaps) by generating unbiased/accurate heatmaps. Taking the two together, we formulate a novel Distribution-Aware coordinate Representation of Keypoints (DARK) method. Serving as a model-agnostic plug-in, DARK brings about significant performance boost to existing human pose estimation models. Extensive experiments show that DARK yields the best results on two common benchmarks, MPII and COCO. Besides, DARK achieves the 2nd place entry in the ICCV 2019 COCO Keypoints Challenge. The code is available online.
+
+
+
+
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/deeppose.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/deeppose.md
new file mode 100644
index 0000000..24778ba
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/deeppose.md
@@ -0,0 +1,30 @@
+# DeepPose: Human pose estimation via deep neural networks
+
+
+
+
+DeepPose (CVPR'2014)
+
+```bibtex
+@inproceedings{toshev2014deeppose,
+ title={Deeppose: Human pose estimation via deep neural networks},
+ author={Toshev, Alexander and Szegedy, Christian},
+ booktitle={Proceedings of the IEEE conference on computer vision and pattern recognition},
+ pages={1653--1660},
+ year={2014}
+}
+```
+
+
+
+## Abstract
+
+
+
+We propose a method for human pose estimation based on Deep Neural Networks (DNNs). The pose estimation is formulated as a DNN-based regression problem towards body joints. We present a cascade of such DNN regressors which results in high precision pose estimates. The approach has the advantage of reasoning about pose in a holistic fashion and has a simple but yet powerful formulation which capitalizes on recent advances in Deep Learning. We present a detailed empirical analysis with state-of-art or better performance on four academic benchmarks of diverse real-world images.
+
+
+
+
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/higherhrnet.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/higherhrnet.md
new file mode 100644
index 0000000..c1d61c9
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/higherhrnet.md
@@ -0,0 +1,30 @@
+# HigherHRNet: Scale-Aware Representation Learning for Bottom-Up Human Pose Estimation
+
+
+
+
+HigherHRNet (CVPR'2020)
+
+```bibtex
+@inproceedings{cheng2020higherhrnet,
+ title={HigherHRNet: Scale-Aware Representation Learning for Bottom-Up Human Pose Estimation},
+ author={Cheng, Bowen and Xiao, Bin and Wang, Jingdong and Shi, Honghui and Huang, Thomas S and Zhang, Lei},
+ booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
+ pages={5386--5395},
+ year={2020}
+}
+```
+
+
+
+## Abstract
+
+
+
+Bottom-up human pose estimation methods have difficulties in predicting the correct pose for small persons due to challenges in scale variation. In this paper, we present HigherHRNet: a novel bottom-up human pose estimation method for learning scale-aware representations using high-resolution feature pyramids. Equipped with multi-resolution supervision for training and multi-resolution aggregation for inference, the proposed approach is able to solve the scale variation challenge in bottom-up multi-person pose estimation and localize keypoints more precisely, especially for small person. The feature pyramid in HigherHRNet consists of feature map outputs from HRNet and upsampled higher-resolution outputs through a transposed convolution. HigherHRNet outperforms the previous best bottom-up method by 2.5% AP for medium person on COCO test-dev, showing its effectiveness in handling scale variation. Furthermore, HigherHRNet achieves new state-of-the-art result on COCO test-dev (70.5% AP) without using refinement or other post-processing techniques, surpassing all existing bottom-up methods. HigherHRNet even surpasses all top-down methods on CrowdPose test (67.6% AP), suggesting its robustness in crowded scene.
+
+
+
+
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/hmr.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/hmr.md
new file mode 100644
index 0000000..5c90aa4
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/hmr.md
@@ -0,0 +1,32 @@
+# End-to-end Recovery of Human Shape and Pose
+
+
+
+
+HMR (CVPR'2018)
+
+```bibtex
+@inProceedings{kanazawaHMR18,
+ title={End-to-end Recovery of Human Shape and Pose},
+ author = {Angjoo Kanazawa
+ and Michael J. Black
+ and David W. Jacobs
+ and Jitendra Malik},
+ booktitle={Computer Vision and Pattern Recognition (CVPR)},
+ year={2018}
+}
+```
+
+
+
+## Abstract
+
+
+
+We describe Human Mesh Recovery (HMR), an end-to-end framework for reconstructing a full 3D mesh of a human body from a single RGB image. In contrast to most current methods that compute 2D or 3D joint locations, we produce a richer and more useful mesh representation that is parameterized by shape and 3D joint angles. The main objective is to minimize the reprojection loss of keypoints, which allows our model to be trained using in-the-wild images that only have ground truth 2D annotations. However, the reprojection loss alone is highly underconstrained. In this work we address this problem by introducing an adversary trained to tell whether human body shape and pose are real or not using a large database of 3D human meshes. We show that HMR can be trained with and without using any paired 2D-to-3D supervision. We do not rely on intermediate 2D keypoint detections and infer 3D pose and shape parameters directly from image pixels. Our model runs in real-time given a bounding box containing the person. We demonstrate our approach on various images in-the-wild and out-perform previous optimization-based methods that output 3D meshes and show competitive results on tasks such as 3D joint location estimation and part segmentation.
+
+
+
+
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/hourglass.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/hourglass.md
new file mode 100644
index 0000000..7782484
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/hourglass.md
@@ -0,0 +1,31 @@
+# Stacked hourglass networks for human pose estimation
+
+
+
+
+Hourglass (ECCV'2016)
+
+```bibtex
+@inproceedings{newell2016stacked,
+ title={Stacked hourglass networks for human pose estimation},
+ author={Newell, Alejandro and Yang, Kaiyu and Deng, Jia},
+ booktitle={European conference on computer vision},
+ pages={483--499},
+ year={2016},
+ organization={Springer}
+}
+```
+
+
+
+## Abstract
+
+
+
+This work introduces a novel convolutional network architecture for the task of human pose estimation. Features are processed across all scales and consolidated to best capture the various spatial relationships associated with the body. We show how repeated bottom-up, top-down processing used in conjunction with intermediate supervision is critical to improving the performance of the network. We refer to the architecture as a “stacked hourglass” network based on the successive steps of pooling and upsampling that are done to produce a final set of predictions. State-of-the-art results are achieved on the FLIC and MPII benchmarks outcompeting all recent methods.
+
+
+
+
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/hrnet.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/hrnet.md
new file mode 100644
index 0000000..05a46f5
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/hrnet.md
@@ -0,0 +1,32 @@
+# Deep high-resolution representation learning for human pose estimation
+
+
+
+
+HRNet (CVPR'2019)
+
+```bibtex
+@inproceedings{sun2019deep,
+ title={Deep high-resolution representation learning for human pose estimation},
+ author={Sun, Ke and Xiao, Bin and Liu, Dong and Wang, Jingdong},
+ booktitle={Proceedings of the IEEE conference on computer vision and pattern recognition},
+ pages={5693--5703},
+ year={2019}
+}
+```
+
+
+
+## Abstract
+
+
+
+In this paper, we are interested in the human pose estimation problem with a focus on learning reliable highresolution representations. Most existing methods recover high-resolution representations from low-resolution representations produced by a high-to-low resolution network. Instead, our proposed network maintains high-resolution representations through the whole process. We start from a high-resolution subnetwork as the first stage, gradually add high-to-low resolution subnetworks one by one to form more stages, and connect the mutliresolution subnetworks in parallel. We conduct repeated multi-scale fusions such that each of the high-to-low resolution representations receives information from other parallel representations over and over, leading to rich highresolution representations. As a result, the predicted keypoint heatmap is potentially more accurate and spatially more precise. We empirically demonstrate the effectiveness
+of our network through the superior pose estimation results over two benchmark datasets: the COCO keypoint detection
+dataset and the MPII Human Pose dataset. In addition, we show the superiority of our network in pose tracking on the PoseTrack dataset.
+
+
+
+
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/hrnetv2.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/hrnetv2.md
new file mode 100644
index 0000000..f2ed2a9
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/hrnetv2.md
@@ -0,0 +1,31 @@
+# Deep high-resolution representation learning for visual recognition
+
+
+
+
+HRNetv2 (TPAMI'2019)
+
+```bibtex
+@article{WangSCJDZLMTWLX19,
+ title={Deep High-Resolution Representation Learning for Visual Recognition},
+ author={Jingdong Wang and Ke Sun and Tianheng Cheng and
+ Borui Jiang and Chaorui Deng and Yang Zhao and Dong Liu and Yadong Mu and
+ Mingkui Tan and Xinggang Wang and Wenyu Liu and Bin Xiao},
+ journal={TPAMI},
+ year={2019}
+}
+```
+
+
+
+## Abstract
+
+
+
+High-resolution representations are essential for position-sensitive vision problems, such as human pose estimation, semantic segmentation, and object detection. Existing state-of-the-art frameworks first encode the input image as a low-resolution representation through a subnetwork that is formed by connecting high-to-low resolution convolutions in series (e.g., ResNet, VGGNet), and then recover the high-resolution representation from the encoded low-resolution representation. Instead, our proposed network, named as High-Resolution Network (HRNet), maintains high-resolution representations through the whole process. There are two key characteristics: (i) Connect the high-to-low resolution convolution streams in parallel and (ii) repeatedly exchange the information across resolutions. The benefit is that the resulting representation is semantically richer and spatially more precise. We show the superiority of the proposed HRNet in a wide range of applications, including human pose estimation, semantic segmentation, and object detection, suggesting that the HRNet is a stronger backbone for computer vision problems.
+
+
+
+
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/internet.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/internet.md
new file mode 100644
index 0000000..e37ea72
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/internet.md
@@ -0,0 +1,29 @@
+# InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Image
+
+
+
+
+InterNet (ECCV'2020)
+
+```bibtex
+@InProceedings{Moon_2020_ECCV_InterHand2.6M,
+author = {Moon, Gyeongsik and Yu, Shoou-I and Wen, He and Shiratori, Takaaki and Lee, Kyoung Mu},
+title = {InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Image},
+booktitle = {European Conference on Computer Vision (ECCV)},
+year = {2020}
+}
+```
+
+
+
+## Abstract
+
+
+
+Analysis of hand-hand interactions is a crucial step towards better understanding human behavior. However, most researches in 3D hand pose estimation have focused on the isolated single hand case. Therefore, we firstly propose (1) a large-scale dataset, InterHand2.6M, and (2) a baseline network, InterNet, for 3D interacting hand pose estimation from a single RGB image. The proposed InterHand2.6M consists of 2.6 M labeled single and interacting hand frames under various poses from multiple subjects. Our InterNet simultaneously performs 3D single and interacting hand pose estimation. In our experiments, we demonstrate big gains in 3D interacting hand pose estimation accuracy when leveraging the interacting hand data in InterHand2.6M. We also report the accuracy of InterNet on InterHand2.6M, which serves as a strong baseline for this new dataset. Finally, we show 3D interacting hand pose estimation results from general images.
+
+
+
+
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/litehrnet.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/litehrnet.md
new file mode 100644
index 0000000..f446062
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/litehrnet.md
@@ -0,0 +1,30 @@
+# Lite-HRNet: A Lightweight High-Resolution Network
+
+
+
+
+LiteHRNet (CVPR'2021)
+
+```bibtex
+@inproceedings{Yulitehrnet21,
+ title={Lite-HRNet: A Lightweight High-Resolution Network},
+ author={Yu, Changqian and Xiao, Bin and Gao, Changxin and Yuan, Lu and Zhang, Lei and Sang, Nong and Wang, Jingdong},
+ booktitle={CVPR},
+ year={2021}
+}
+```
+
+
+
+## Abstract
+
+
+
+We present an efficient high-resolution network, Lite-HRNet, for human pose estimation. We start by simply applying the efficient shuffle block in ShuffleNet to HRNet (high-resolution network), yielding stronger performance over popular lightweight networks, such as MobileNet, ShuffleNet, and Small HRNet.
+We find that the heavily-used pointwise (1x1) convolutions in shuffle blocks become the computational bottleneck. We introduce a lightweight unit, conditional channel weighting, to replace costly pointwise (1x1) convolutions in shuffle blocks. The complexity of channel weighting is linear w.r.t the number of channels and lower than the quadratic time complexity for pointwise convolutions. Our solution learns the weights from all the channels and over multiple resolutions that are readily available in the parallel branches in HRNet. It uses the weights as the bridge to exchange information across channels and resolutions, compensating the role played by the pointwise (1x1) convolution. Lite-HRNet demonstrates superior results on human pose estimation over popular lightweight networks. Moreover, Lite-HRNet can be easily applied to semantic segmentation task in the same lightweight manner.
+
+
+
+
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/mspn.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/mspn.md
new file mode 100644
index 0000000..1915cd3
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/mspn.md
@@ -0,0 +1,29 @@
+# Rethinking on multi-stage networks for human pose estimation
+
+
+
+
+MSPN (ArXiv'2019)
+
+```bibtex
+@article{li2019rethinking,
+ title={Rethinking on Multi-Stage Networks for Human Pose Estimation},
+ author={Li, Wenbo and Wang, Zhicheng and Yin, Binyi and Peng, Qixiang and Du, Yuming and Xiao, Tianzi and Yu, Gang and Lu, Hongtao and Wei, Yichen and Sun, Jian},
+ journal={arXiv preprint arXiv:1901.00148},
+ year={2019}
+}
+```
+
+
+
+## Abstract
+
+
+
+Existing pose estimation approaches fall into two categories: single-stage and multi-stage methods. While multi-stage methods are seemingly more suited for the task, their performance in current practice is not as good as single-stage methods. This work studies this issue. We argue that the current multi-stage methods' unsatisfactory performance comes from the insufficiency in various design choices. We propose several improvements, including the single-stage module design, cross stage feature aggregation, and coarse-to-fine supervision. The resulting method establishes the new state-of-the-art on both MS COCO and MPII Human Pose dataset, justifying the effectiveness of a multi-stage architecture. The source code is publicly available for further research.
+
+
+
+
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/posewarper.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/posewarper.md
new file mode 100644
index 0000000..285a36c
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/posewarper.md
@@ -0,0 +1,29 @@
+# Learning Temporal Pose Estimation from Sparsely-Labeled Videos
+
+
+
+
+PoseWarper (NeurIPS'2019)
+
+```bibtex
+@inproceedings{NIPS2019_gberta,
+title = {Learning Temporal Pose Estimation from Sparsely Labeled Videos},
+author = {Bertasius, Gedas and Feichtenhofer, Christoph, and Tran, Du and Shi, Jianbo, and Torresani, Lorenzo},
+booktitle = {Advances in Neural Information Processing Systems 33},
+year = {2019},
+}
+```
+
+
+
+## Abstract
+
+
+
+Modern approaches for multi-person pose estimation in video require large amounts of dense annotations. However, labeling every frame in a video is costly and labor intensive. To reduce the need for dense annotations, we propose a PoseWarper network that leverages training videos with sparse annotations (every k frames) to learn to perform dense temporal pose propagation and estimation. Given a pair of video frames---a labeled Frame A and an unlabeled Frame B---we train our model to predict human pose in Frame A using the features from Frame B by means of deformable convolutions to implicitly learn the pose warping between A and B. We demonstrate that we can leverage our trained PoseWarper for several applications. First, at inference time we can reverse the application direction of our network in order to propagate pose information from manually annotated frames to unlabeled frames. This makes it possible to generate pose annotations for the entire video given only a few manually-labeled frames. Compared to modern label propagation methods based on optical flow, our warping mechanism is much more compact (6M vs 39M parameters), and also more accurate (88.7% mAP vs 83.8% mAP). We also show that we can improve the accuracy of a pose estimator by training it on an augmented dataset obtained by adding our propagated poses to the original manual labels. Lastly, we can use our PoseWarper to aggregate temporal pose information from neighboring frames during inference. This allows our system to achieve state-of-the-art pose detection results on the PoseTrack2017 and PoseTrack2018 datasets.
+
+
+
+
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/rsn.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/rsn.md
new file mode 100644
index 0000000..b1fb1ea
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/rsn.md
@@ -0,0 +1,31 @@
+# Learning delicate local representations for multi-person pose estimation
+
+
+
+
+RSN (ECCV'2020)
+
+```bibtex
+@misc{cai2020learning,
+ title={Learning Delicate Local Representations for Multi-Person Pose Estimation},
+ author={Yuanhao Cai and Zhicheng Wang and Zhengxiong Luo and Binyi Yin and Angang Du and Haoqian Wang and Xinyu Zhou and Erjin Zhou and Xiangyu Zhang and Jian Sun},
+ year={2020},
+ eprint={2003.04030},
+ archivePrefix={arXiv},
+ primaryClass={cs.CV}
+}
+```
+
+
+
+## Abstract
+
+
+
+In this paper, we propose a novel method called Residual Steps Network (RSN). RSN aggregates features with the same spatial size (Intra-level features) efficiently to obtain delicate local representations, which retain rich low-level spatial information and result in precise keypoint localization. Additionally, we observe the output features contribute differently to final performance. To tackle this problem, we propose an efficient attention mechanism - Pose Refine Machine (PRM) to make a trade-off between local and global representations in output features and further refine the keypoint locations. Our approach won the 1st place of COCO Keypoint Challenge 2019 and achieves state-of-the-art results on both COCO and MPII benchmarks, without using extra training data and pretrained model. Our single model achieves 78.6 on COCO test-dev, 93.0 on MPII test dataset. Ensembled models achieve 79.2 on COCO test-dev, 77.1 on COCO test-challenge dataset.
+
+
+
+
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/scnet.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/scnet.md
new file mode 100644
index 0000000..043c144
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/scnet.md
@@ -0,0 +1,30 @@
+# Improving Convolutional Networks with Self-Calibrated Convolutions
+
+
+
+
+SCNet (CVPR'2020)
+
+```bibtex
+@inproceedings{liu2020improving,
+ title={Improving Convolutional Networks with Self-Calibrated Convolutions},
+ author={Liu, Jiang-Jiang and Hou, Qibin and Cheng, Ming-Ming and Wang, Changhu and Feng, Jiashi},
+ booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
+ pages={10096--10105},
+ year={2020}
+}
+```
+
+
+
+## Abstract
+
+
+
+Recent advances on CNNs are mostly devoted to designing more complex architectures to enhance their representation learning capacity. In this paper, we consider how to improve the basic convolutional feature transformation process of CNNs without tuning the model architectures. To this end, we present a novel self-calibrated convolutions that explicitly expand fields-of-view of each convolutional layers through internal communications and hence enrich the output features. In particular, unlike the standard convolutions that fuse spatial and channel-wise information using small kernels (e.g., 3x3), self-calibrated convolutions adaptively build long-range spatial and inter-channel dependencies around each spatial location through a novel self-calibration operation. Thus, it can help CNNs generate more discriminative representations by explicitly incorporating richer information. Our self-calibrated convolution design is simple and generic, and can be easily applied to augment standard convolutional layers without introducing extra parameters and complexity. Extensive experiments demonstrate that when applying self-calibrated convolutions into different backbones, our networks can significantly improve the baseline models in a variety of vision tasks, including image recognition, object detection, instance segmentation, and keypoint detection, with no need to change the network architectures. We hope this work could provide a promising way for future research in designing novel convolutional feature transformations for improving convolutional networks. Code is available on the project page.
+
+
+
+
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/simplebaseline2d.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/simplebaseline2d.md
new file mode 100644
index 0000000..026ef92
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/simplebaseline2d.md
@@ -0,0 +1,31 @@
+# Simple baselines for human pose estimation and tracking
+
+
+
+
+SimpleBaseline2D (ECCV'2018)
+
+```bibtex
+@inproceedings{xiao2018simple,
+ title={Simple baselines for human pose estimation and tracking},
+ author={Xiao, Bin and Wu, Haiping and Wei, Yichen},
+ booktitle={Proceedings of the European conference on computer vision (ECCV)},
+ pages={466--481},
+ year={2018}
+}
+```
+
+
+
+## Abstract
+
+
+
+There has been significant progress on pose estimation and increasing interests on pose tracking in recent years. At the same time, the overall algorithm and system complexity increases as well, making the algorithm analysis and comparison more difficult. This work provides simple and effective baseline methods. They are helpful for inspiring and
+evaluating new ideas for the field. State-of-the-art results are achieved on challenging benchmarks.
+
+
+
+
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/simplebaseline3d.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/simplebaseline3d.md
new file mode 100644
index 0000000..ee3c583
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/simplebaseline3d.md
@@ -0,0 +1,29 @@
+# A simple yet effective baseline for 3d human pose estimation
+
+
+
+
+SimpleBaseline3D (ICCV'2017)
+
+```bibtex
+@inproceedings{martinez_2017_3dbaseline,
+ title={A simple yet effective baseline for 3d human pose estimation},
+ author={Martinez, Julieta and Hossain, Rayat and Romero, Javier and Little, James J.},
+ booktitle={ICCV},
+ year={2017}
+}
+```
+
+
+
+## Abstract
+
+
+
+Following the success of deep convolutional networks, state-of-the-art methods for 3d human pose estimation have focused on deep end-to-end systems that predict 3d joint locations given raw image pixels. Despite their excellent performance, it is often not easy to understand whether their remaining error stems from a limited 2d pose (visual) understanding, or from a failure to map 2d poses into 3-dimensional positions. With the goal of understanding these sources of error, we set out to build a system that given 2d joint locations predicts 3d positions. Much to our surprise, we have found that, with current technology, "lifting" ground truth 2d joint locations to 3d space is a task that can be solved with a remarkably low error rate: a relatively simple deep feed-forward network outperforms the best reported result by about 30% on Human3.6M, the largest publicly available 3d pose estimation benchmark. Furthermore, training our system on the output of an off-the-shelf state-of-the-art 2d detector (i.e., using images as input) yields state of the art results -- this includes an array of systems that have been trained end-to-end specifically for this task. Our results indicate that a large portion of the error of modern deep 3d pose estimation systems stems from their visual analysis, and suggests directions to further advance the state of the art in 3d human pose estimation.
+
+
+
+
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/softwingloss.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/softwingloss.md
new file mode 100644
index 0000000..524a608
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/softwingloss.md
@@ -0,0 +1,30 @@
+# Structure-Coherent Deep Feature Learning for Robust Face Alignment
+
+
+
+
+SoftWingloss (TIP'2021)
+
+```bibtex
+@article{lin2021structure,
+ title={Structure-Coherent Deep Feature Learning for Robust Face Alignment},
+ author={Lin, Chunze and Zhu, Beier and Wang, Quan and Liao, Renjie and Qian, Chen and Lu, Jiwen and Zhou, Jie},
+ journal={IEEE Transactions on Image Processing},
+ year={2021},
+ publisher={IEEE}
+}
+```
+
+
+
+## Abstract
+
+
+
+In this paper, we propose a structure-coherent deep feature learning method for face alignment. Unlike most existing face alignment methods which overlook the facial structure cues, we explicitly exploit the relation among facial landmarks to make the detector robust to hard cases such as occlusion and large pose. Specifically, we leverage a landmark-graph relational network to enforce the structural relationships among landmarks. We consider the facial landmarks as structural graph nodes and carefully design the neighborhood to passing features among the most related nodes. Our method dynamically adapts the weights of node neighborhood to eliminate distracted information from noisy nodes, such as occluded landmark point. Moreover, different from most previous works which only tend to penalize the landmarks absolute position during the training, we propose a relative location loss to enhance the information of relative location of landmarks. This relative location supervision further regularizes the facial structure. Our approach considers the interactions among facial landmarks and can be easily implemented on top of any convolutional backbone to boost the performance. Extensive experiments on three popular benchmarks, including WFLW, COFW and 300W, demonstrate the effectiveness of the proposed method. In particular, due to explicit structure modeling, our approach is especially robust to challenging cases resulting in impressive low failure rate on COFW and WFLW datasets.
+
+
+
+
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/udp.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/udp.md
new file mode 100644
index 0000000..bb4aceb
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/udp.md
@@ -0,0 +1,30 @@
+# The Devil is in the Details: Delving into Unbiased Data Processing for Human Pose Estimation
+
+
+
+
+UDP (CVPR'2020)
+
+```bibtex
+@InProceedings{Huang_2020_CVPR,
+ author = {Huang, Junjie and Zhu, Zheng and Guo, Feng and Huang, Guan},
+ title = {The Devil Is in the Details: Delving Into Unbiased Data Processing for Human Pose Estimation},
+ booktitle = {The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
+ month = {June},
+ year = {2020}
+}
+```
+
+
+
+## Abstract
+
+
+
+Recently, the leading performance of human pose estimation is dominated by top-down methods. Being a fundamental component in training and inference, data processing has not been systematically considered in pose estimation community, to the best of our knowledge. In this paper, we focus on this problem and find that the devil of top-down pose estimator is in the biased data processing. Specifically, by investigating the standard data processing in state-of-the-art approaches mainly including data transformation and encoding-decoding, we find that the results obtained by common flipping strategy are unaligned with the original ones in inference. Moreover, there is statistical error in standard encoding-decoding during both training and inference. Two problems couple together and significantly degrade the pose estimation performance. Based on quantitative analyses, we then formulate a principled way to tackle this dilemma. Data is processed in continuous space based on unit length (the intervals between pixels) instead of in discrete space with pixel, and a combined classification and regression approach is adopted to perform encoding-decoding. The Unbiased Data Processing (UDP) for human pose estimation can be achieved by combining the two together. UDP not only boosts the performance of existing methods by a large margin but also plays a important role in result reproducing and future exploration. As a model-agnostic approach, UDP promotes SimpleBaseline-ResNet50-256x192 by 1.5 AP (70.2 to 71.7) and HRNet-W32-256x192 by 1.7 AP (73.5 to 75.2) on COCO test-dev set. The HRNet-W48-384x288 equipped with UDP achieves 76.5 AP and sets a new state-of-the-art for human pose estimation. The source code is publicly available for further research.
+
+
+
+
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/videopose3d.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/videopose3d.md
new file mode 100644
index 0000000..f8647e0
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/videopose3d.md
@@ -0,0 +1,30 @@
+# 3D human pose estimation in video with temporal convolutions and semi-supervised training
+
+
+
+
+VideoPose3D (CVPR'2019)
+
+```bibtex
+@inproceedings{pavllo20193d,
+ title={3d human pose estimation in video with temporal convolutions and semi-supervised training},
+ author={Pavllo, Dario and Feichtenhofer, Christoph and Grangier, David and Auli, Michael},
+ booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
+ pages={7753--7762},
+ year={2019}
+}
+```
+
+
+
+## Abstract
+
+
+
+In this work, we demonstrate that 3D poses in video can be effectively estimated with a fully convolutional model based on dilated temporal convolutions over 2D keypoints. We also introduce back-projection, a simple and effective semi-supervised training method that leverages unlabeled video data. We start with predicted 2D keypoints for unlabeled video, then estimate 3D poses and finally back-project to the input 2D keypoints. In the supervised setting, our fully-convolutional model outperforms the previous best result from the literature by 6 mm mean per-joint position error on Human3.6M, corresponding to an error reduction of 11%, and the model also shows significant improvements on HumanEva-I. Moreover, experiments with back-projection show that it comfortably outperforms previous state-of-the-art results in semi-supervised settings where labeled data is scarce.
+
+
+
+
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/vipnas.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/vipnas.md
new file mode 100644
index 0000000..5f52a8c
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/vipnas.md
@@ -0,0 +1,29 @@
+# ViPNAS: Efficient Video Pose Estimation via Neural Architecture Search
+
+
+
+
+ViPNAS (CVPR'2021)
+
+```bibtex
+@article{xu2021vipnas,
+ title={ViPNAS: Efficient Video Pose Estimation via Neural Architecture Search},
+ author={Xu, Lumin and Guan, Yingda and Jin, Sheng and Liu, Wentao and Qian, Chen and Luo, Ping and Ouyang, Wanli and Wang, Xiaogang},
+ booktitle={Proceedings of the IEEE conference on computer vision and pattern recognition},
+ year={2021}
+}
+```
+
+
+
+## Abstract
+
+
+
+Human pose estimation has achieved significant progress in recent years. However, most of the recent methods focus on improving accuracy using complicated models and ignoring real-time efficiency. To achieve a better trade-off between accuracy and efficiency, we propose a novel neural architecture search (NAS) method, termed ViPNAS, to search networks in both spatial and temporal levels for fast online video pose estimation. In the spatial level, we carefully design the search space with five different dimensions including network depth, width, kernel size, group number, and attentions. In the temporal level, we search from a series of temporal feature fusions to optimize the total accuracy and speed across multiple video frames. To the best of our knowledge, we are the first to search for the temporal feature fusion and automatic computation allocation in videos. Extensive experiments demonstrate the effectiveness of our approach on the challenging COCO2017 and PoseTrack2018 datasets. Our discovered model family, S-ViPNAS and T-ViPNAS, achieve significantly higher inference speed (CPU real-time) without sacrificing the accuracy compared to the previous state-of-the-art methods.
+
+
+
+
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/voxelpose.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/voxelpose.md
new file mode 100644
index 0000000..384f4ca
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/voxelpose.md
@@ -0,0 +1,29 @@
+# VoxelPose: Towards Multi-Camera 3D Human Pose Estimation in Wild Environment
+
+
+
+
+VoxelPose (ECCV'2020)
+
+```bibtex
+@inproceedings{tumultipose,
+ title={VoxelPose: Towards Multi-Camera 3D Human Pose Estimation in Wild Environment},
+ author={Tu, Hanyue and Wang, Chunyu and Zeng, Wenjun},
+ booktitle={ECCV},
+ year={2020}
+}
+```
+
+
+
+## Abstract
+
+
+
+We present VoxelPose to estimate 3D poses of multiple people from multiple camera views. In contrast to the previous efforts which require to establish cross-view correspondence based on noisy and incomplete 2D pose estimates, VoxelPose directly operates in the 3D space therefore avoids making incorrect decisions in each camera view. To achieve this goal, features in all camera views are aggregated in the 3D voxel space and fed into Cuboid Proposal Network (CPN) to localize all people. Then we propose Pose Regression Network (PRN) to estimate a detailed 3D pose for each proposal. The approach is robust to occlusion which occurs frequently in practice. Without bells and whistles, it outperforms the previous methods on several public datasets.
+
+
+
+
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/wingloss.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/wingloss.md
new file mode 100644
index 0000000..2aaa057
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/algorithms/wingloss.md
@@ -0,0 +1,31 @@
+# Wing Loss for Robust Facial Landmark Localisation with Convolutional Neural Networks
+
+
+
+
+Wingloss (CVPR'2018)
+
+```bibtex
+@inproceedings{feng2018wing,
+ title={Wing Loss for Robust Facial Landmark Localisation with Convolutional Neural Networks},
+ author={Feng, Zhen-Hua and Kittler, Josef and Awais, Muhammad and Huber, Patrik and Wu, Xiao-Jun},
+ booktitle={Computer Vision and Pattern Recognition (CVPR), 2018 IEEE Conference on},
+ year={2018},
+ pages ={2235-2245},
+ organization={IEEE}
+}
+```
+
+
+
+## Abstract
+
+
+
+We present a new loss function, namely Wing loss, for robust facial landmark localisation with Convolutional Neural Networks (CNNs). We first compare and analyse different loss functions including L2, L1 and smooth L1. The analysis of these loss functions suggests that, for the training of a CNN-based localisation model, more attention should be paid to small and medium range errors. To this end, we design a piece-wise loss function. The new loss amplifies the impact of errors from the interval (-w, w) by switching from L1 loss to a modified logarithm function. To address the problem of under-representation of samples with large out-of-plane head rotations in the training set, we propose a simple but effective boosting strategy, referred to as pose-based data balancing. In particular, we deal with the data imbalance problem by duplicating the minority training samples and perturbing them by injecting random image rotation, bounding box translation and other data augmentation approaches. Last, the proposed approach is extended to create a two-stage framework for robust facial landmark localisation. The experimental results obtained on AFLW and 300W demonstrate the merits of the Wing loss function, and prove the superiority of the proposed method over the state-of-the-art approaches.
+
+
+
+
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/alexnet.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/alexnet.md
new file mode 100644
index 0000000..9a7d0bb
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/alexnet.md
@@ -0,0 +1,30 @@
+# Imagenet classification with deep convolutional neural networks
+
+
+
+
+AlexNet (NeurIPS'2012)
+
+```bibtex
+@inproceedings{krizhevsky2012imagenet,
+ title={Imagenet classification with deep convolutional neural networks},
+ author={Krizhevsky, Alex and Sutskever, Ilya and Hinton, Geoffrey E},
+ booktitle={Advances in neural information processing systems},
+ pages={1097--1105},
+ year={2012}
+}
+```
+
+
+
+## Abstract
+
+
+
+We trained a large, deep convolutional neural network to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes. On the test data, we achieved top-1 and top-5 error rates of 37.5% and 17.0% which is considerably better than the previous state-of-the-art. The neural network, which has 60 million parameters and 650,000 neurons, consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully-connected layers with a final 1000-way softmax. To make training faster, we used non-saturating neurons and a very efficient GPU implementation of the convolution operation. To reduce overfitting in the fully-connected layers we employed a recently-developed regularization method called “dropout” that proved to be very effective. We also entered a variant of this model in the ILSVRC-2012 competition and achieved a winning top-5 test error rate of 15.3%, compared to 26.2% achieved by the second-best entry
+
+
+
+
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/cpm.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/cpm.md
new file mode 100644
index 0000000..fb5dbfa
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/cpm.md
@@ -0,0 +1,30 @@
+# Convolutional pose machines
+
+
+
+
+CPM (CVPR'2016)
+
+```bibtex
+@inproceedings{wei2016convolutional,
+ title={Convolutional pose machines},
+ author={Wei, Shih-En and Ramakrishna, Varun and Kanade, Takeo and Sheikh, Yaser},
+ booktitle={Proceedings of the IEEE conference on Computer Vision and Pattern Recognition},
+ pages={4724--4732},
+ year={2016}
+}
+```
+
+
+
+## Abstract
+
+
+
+We introduce associative embedding, a novel method for supervising convolutional neural networks for the task of detection and grouping. A number of computer vision problems can be framed in this manner including multi-person pose estimation, instance segmentation, and multi-object tracking. Usually the grouping of detections is achieved with multi-stage pipelines, instead we propose an approach that teaches a network to simultaneously output detections and group assignments. This technique can be easily integrated into any state-of-the-art network architecture that produces pixel-wise predictions. We show how to apply this method to both multi-person pose estimation and instance segmentation and report state-of-the-art performance for multi-person pose on the MPII and MS-COCO datasets.
+
+
+
+
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/higherhrnet.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/higherhrnet.md
new file mode 100644
index 0000000..c1d61c9
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/higherhrnet.md
@@ -0,0 +1,30 @@
+# HigherHRNet: Scale-Aware Representation Learning for Bottom-Up Human Pose Estimation
+
+
+
+
+HigherHRNet (CVPR'2020)
+
+```bibtex
+@inproceedings{cheng2020higherhrnet,
+ title={HigherHRNet: Scale-Aware Representation Learning for Bottom-Up Human Pose Estimation},
+ author={Cheng, Bowen and Xiao, Bin and Wang, Jingdong and Shi, Honghui and Huang, Thomas S and Zhang, Lei},
+ booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
+ pages={5386--5395},
+ year={2020}
+}
+```
+
+
+
+## Abstract
+
+
+
+Bottom-up human pose estimation methods have difficulties in predicting the correct pose for small persons due to challenges in scale variation. In this paper, we present HigherHRNet: a novel bottom-up human pose estimation method for learning scale-aware representations using high-resolution feature pyramids. Equipped with multi-resolution supervision for training and multi-resolution aggregation for inference, the proposed approach is able to solve the scale variation challenge in bottom-up multi-person pose estimation and localize keypoints more precisely, especially for small person. The feature pyramid in HigherHRNet consists of feature map outputs from HRNet and upsampled higher-resolution outputs through a transposed convolution. HigherHRNet outperforms the previous best bottom-up method by 2.5% AP for medium person on COCO test-dev, showing its effectiveness in handling scale variation. Furthermore, HigherHRNet achieves new state-of-the-art result on COCO test-dev (70.5% AP) without using refinement or other post-processing techniques, surpassing all existing bottom-up methods. HigherHRNet even surpasses all top-down methods on CrowdPose test (67.6% AP), suggesting its robustness in crowded scene.
+
+
+
+
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/hourglass.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/hourglass.md
new file mode 100644
index 0000000..7782484
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/hourglass.md
@@ -0,0 +1,31 @@
+# Stacked hourglass networks for human pose estimation
+
+
+
+
+Hourglass (ECCV'2016)
+
+```bibtex
+@inproceedings{newell2016stacked,
+ title={Stacked hourglass networks for human pose estimation},
+ author={Newell, Alejandro and Yang, Kaiyu and Deng, Jia},
+ booktitle={European conference on computer vision},
+ pages={483--499},
+ year={2016},
+ organization={Springer}
+}
+```
+
+
+
+## Abstract
+
+
+
+This work introduces a novel convolutional network architecture for the task of human pose estimation. Features are processed across all scales and consolidated to best capture the various spatial relationships associated with the body. We show how repeated bottom-up, top-down processing used in conjunction with intermediate supervision is critical to improving the performance of the network. We refer to the architecture as a “stacked hourglass” network based on the successive steps of pooling and upsampling that are done to produce a final set of predictions. State-of-the-art results are achieved on the FLIC and MPII benchmarks outcompeting all recent methods.
+
+
+
+
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/hrformer.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/hrformer.md
new file mode 100644
index 0000000..dfa7a13
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/hrformer.md
@@ -0,0 +1,39 @@
+# HRFormer: High-Resolution Vision Transformer for Dense Predict
+
+
+
+
+HRFormer (NIPS'2021)
+
+```bibtex
+@article{yuan2021hrformer,
+ title={HRFormer: High-Resolution Vision Transformer for Dense Predict},
+ author={Yuan, Yuhui and Fu, Rao and Huang, Lang and Lin, Weihong and Zhang, Chao and Chen, Xilin and Wang, Jingdong},
+ journal={Advances in Neural Information Processing Systems},
+ volume={34},
+ year={2021}
+}
+```
+
+
+
+## Abstract
+
+
+
+We present a High-Resolution Transformer (HRFormer) that learns high-resolution representations for dense
+prediction tasks, in contrast to the original Vision Transformer that produces low-resolution representations
+and has high memory and computational cost. We take advantage of the multi-resolution parallel design
+introduced in high-resolution convolutional networks (HRNet), along with local-window self-attention
+that performs self-attention over small non-overlapping image windows, for improving the memory and
+computation efficiency. In addition, we introduce a convolution into the FFN to exchange information
+across the disconnected image windows. We demonstrate the effectiveness of the HighResolution Transformer
+on both human pose estimation and semantic segmentation tasks, e.g., HRFormer outperforms Swin
+transformer by 1.3 AP on COCO pose estimation with 50% fewer parameters and 30% fewer FLOPs.
+Code is available at: https://github.com/HRNet/HRFormer
+
+
+
+
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/hrnet.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/hrnet.md
new file mode 100644
index 0000000..05a46f5
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/hrnet.md
@@ -0,0 +1,32 @@
+# Deep high-resolution representation learning for human pose estimation
+
+
+
+
+HRNet (CVPR'2019)
+
+```bibtex
+@inproceedings{sun2019deep,
+ title={Deep high-resolution representation learning for human pose estimation},
+ author={Sun, Ke and Xiao, Bin and Liu, Dong and Wang, Jingdong},
+ booktitle={Proceedings of the IEEE conference on computer vision and pattern recognition},
+ pages={5693--5703},
+ year={2019}
+}
+```
+
+
+
+## Abstract
+
+
+
+In this paper, we are interested in the human pose estimation problem with a focus on learning reliable highresolution representations. Most existing methods recover high-resolution representations from low-resolution representations produced by a high-to-low resolution network. Instead, our proposed network maintains high-resolution representations through the whole process. We start from a high-resolution subnetwork as the first stage, gradually add high-to-low resolution subnetworks one by one to form more stages, and connect the mutliresolution subnetworks in parallel. We conduct repeated multi-scale fusions such that each of the high-to-low resolution representations receives information from other parallel representations over and over, leading to rich highresolution representations. As a result, the predicted keypoint heatmap is potentially more accurate and spatially more precise. We empirically demonstrate the effectiveness
+of our network through the superior pose estimation results over two benchmark datasets: the COCO keypoint detection
+dataset and the MPII Human Pose dataset. In addition, we show the superiority of our network in pose tracking on the PoseTrack dataset.
+
+
+
+
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/hrnetv2.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/hrnetv2.md
new file mode 100644
index 0000000..f2ed2a9
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/hrnetv2.md
@@ -0,0 +1,31 @@
+# Deep high-resolution representation learning for visual recognition
+
+
+
+
+HRNetv2 (TPAMI'2019)
+
+```bibtex
+@article{WangSCJDZLMTWLX19,
+ title={Deep High-Resolution Representation Learning for Visual Recognition},
+ author={Jingdong Wang and Ke Sun and Tianheng Cheng and
+ Borui Jiang and Chaorui Deng and Yang Zhao and Dong Liu and Yadong Mu and
+ Mingkui Tan and Xinggang Wang and Wenyu Liu and Bin Xiao},
+ journal={TPAMI},
+ year={2019}
+}
+```
+
+
+
+## Abstract
+
+
+
+High-resolution representations are essential for position-sensitive vision problems, such as human pose estimation, semantic segmentation, and object detection. Existing state-of-the-art frameworks first encode the input image as a low-resolution representation through a subnetwork that is formed by connecting high-to-low resolution convolutions in series (e.g., ResNet, VGGNet), and then recover the high-resolution representation from the encoded low-resolution representation. Instead, our proposed network, named as High-Resolution Network (HRNet), maintains high-resolution representations through the whole process. There are two key characteristics: (i) Connect the high-to-low resolution convolution streams in parallel and (ii) repeatedly exchange the information across resolutions. The benefit is that the resulting representation is semantically richer and spatially more precise. We show the superiority of the proposed HRNet in a wide range of applications, including human pose estimation, semantic segmentation, and object detection, suggesting that the HRNet is a stronger backbone for computer vision problems.
+
+
+
+
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/litehrnet.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/litehrnet.md
new file mode 100644
index 0000000..f446062
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/litehrnet.md
@@ -0,0 +1,30 @@
+# Lite-HRNet: A Lightweight High-Resolution Network
+
+
+
+
+LiteHRNet (CVPR'2021)
+
+```bibtex
+@inproceedings{Yulitehrnet21,
+ title={Lite-HRNet: A Lightweight High-Resolution Network},
+ author={Yu, Changqian and Xiao, Bin and Gao, Changxin and Yuan, Lu and Zhang, Lei and Sang, Nong and Wang, Jingdong},
+ booktitle={CVPR},
+ year={2021}
+}
+```
+
+
+
+## Abstract
+
+
+
+We present an efficient high-resolution network, Lite-HRNet, for human pose estimation. We start by simply applying the efficient shuffle block in ShuffleNet to HRNet (high-resolution network), yielding stronger performance over popular lightweight networks, such as MobileNet, ShuffleNet, and Small HRNet.
+We find that the heavily-used pointwise (1x1) convolutions in shuffle blocks become the computational bottleneck. We introduce a lightweight unit, conditional channel weighting, to replace costly pointwise (1x1) convolutions in shuffle blocks. The complexity of channel weighting is linear w.r.t the number of channels and lower than the quadratic time complexity for pointwise convolutions. Our solution learns the weights from all the channels and over multiple resolutions that are readily available in the parallel branches in HRNet. It uses the weights as the bridge to exchange information across channels and resolutions, compensating the role played by the pointwise (1x1) convolution. Lite-HRNet demonstrates superior results on human pose estimation over popular lightweight networks. Moreover, Lite-HRNet can be easily applied to semantic segmentation task in the same lightweight manner.
+
+
+
+
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/mobilenetv2.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/mobilenetv2.md
new file mode 100644
index 0000000..9456520
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/mobilenetv2.md
@@ -0,0 +1,30 @@
+# Mobilenetv2: Inverted residuals and linear bottlenecks
+
+
+
+
+MobilenetV2 (CVPR'2018)
+
+```bibtex
+@inproceedings{sandler2018mobilenetv2,
+ title={Mobilenetv2: Inverted residuals and linear bottlenecks},
+ author={Sandler, Mark and Howard, Andrew and Zhu, Menglong and Zhmoginov, Andrey and Chen, Liang-Chieh},
+ booktitle={Proceedings of the IEEE conference on computer vision and pattern recognition},
+ pages={4510--4520},
+ year={2018}
+}
+```
+
+
+
+## Abstract
+
+
+
+In this paper we describe a new mobile architecture, mbox{MobileNetV2}, that improves the state of the art performance of mobile models on multiple tasks and benchmarks as well as across a spectrum of different model sizes. We also describe efficient ways of applying these mobile models to object detection in a novel framework we call mbox{SSDLite}. Additionally, we demonstrate how to build mobile semantic segmentation models through a reduced form of mbox{DeepLabv3} which we call Mobile mbox{DeepLabv3}. is based on an inverted residual structure where the shortcut connections are between the thin bottleneck layers. The intermediate expansion layer uses lightweight depthwise convolutions to filter features as a source of non-linearity. Additionally, we find that it is important to remove non-linearities in the narrow layers in order to maintain representational power. We demonstrate that this improves performance and provide an intuition that led to this design. Finally, our approach allows decoupling of the input/output domains from the expressiveness of the transformation, which provides a convenient framework for further analysis. We measure our performance on mbox{ImageNet}~cite{Russakovsky:2015:ILS:2846547.2846559} classification, COCO object detection cite{COCO}, VOC image segmentation cite{PASCAL}. We evaluate the trade-offs between accuracy, and number of operations measured by multiply-adds (MAdd), as well as actual latency, and the number of parameters.
+
+
+
+
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/mspn.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/mspn.md
new file mode 100644
index 0000000..1915cd3
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/mspn.md
@@ -0,0 +1,29 @@
+# Rethinking on multi-stage networks for human pose estimation
+
+
+
+
+MSPN (ArXiv'2019)
+
+```bibtex
+@article{li2019rethinking,
+ title={Rethinking on Multi-Stage Networks for Human Pose Estimation},
+ author={Li, Wenbo and Wang, Zhicheng and Yin, Binyi and Peng, Qixiang and Du, Yuming and Xiao, Tianzi and Yu, Gang and Lu, Hongtao and Wei, Yichen and Sun, Jian},
+ journal={arXiv preprint arXiv:1901.00148},
+ year={2019}
+}
+```
+
+
+
+## Abstract
+
+
+
+Existing pose estimation approaches fall into two categories: single-stage and multi-stage methods. While multi-stage methods are seemingly more suited for the task, their performance in current practice is not as good as single-stage methods. This work studies this issue. We argue that the current multi-stage methods' unsatisfactory performance comes from the insufficiency in various design choices. We propose several improvements, including the single-stage module design, cross stage feature aggregation, and coarse-to-fine supervision. The resulting method establishes the new state-of-the-art on both MS COCO and MPII Human Pose dataset, justifying the effectiveness of a multi-stage architecture. The source code is publicly available for further research.
+
+
+
+
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/resnest.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/resnest.md
new file mode 100644
index 0000000..748c947
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/resnest.md
@@ -0,0 +1,29 @@
+# ResNeSt: Split-Attention Networks
+
+
+
+
+ResNeSt (ArXiv'2020)
+
+```bibtex
+@article{zhang2020resnest,
+ title={ResNeSt: Split-Attention Networks},
+ author={Zhang, Hang and Wu, Chongruo and Zhang, Zhongyue and Zhu, Yi and Zhang, Zhi and Lin, Haibin and Sun, Yue and He, Tong and Muller, Jonas and Manmatha, R. and Li, Mu and Smola, Alexander},
+ journal={arXiv preprint arXiv:2004.08955},
+ year={2020}
+}
+```
+
+
+
+## Abstract
+
+
+
+It is well known that featuremap attention and multi-path representation are important for visual recognition. In this paper, we present a modularized architecture, which applies the channel-wise attention on different network branches to leverage their success in capturing cross-feature interactions and learning diverse representations. Our design results in a simple and unified computation block, which can be parameterized using only a few variables. Our model, named ResNeSt, outperforms EfficientNet in accuracy and latency trade-off on image classification. In addition, ResNeSt has achieved superior transfer learning results on several public benchmarks serving as the backbone, and has been adopted by the winning entries of COCO-LVIS challenge. The source code for complete system and pretrained models are publicly available.
+
+
+
+
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/resnet.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/resnet.md
new file mode 100644
index 0000000..86b91ff
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/resnet.md
@@ -0,0 +1,32 @@
+# Deep residual learning for image recognition
+
+
+
+
+ResNet (CVPR'2016)
+
+```bibtex
+@inproceedings{he2016deep,
+ title={Deep residual learning for image recognition},
+ author={He, Kaiming and Zhang, Xiangyu and Ren, Shaoqing and Sun, Jian},
+ booktitle={Proceedings of the IEEE conference on computer vision and pattern recognition},
+ pages={770--778},
+ year={2016}
+}
+```
+
+
+
+## Abstract
+
+
+
+Deeper neural networks are more difficult to train. We present a residual learning framework to ease the training of networks that are substantially deeper than those used previously. We explicitly reformulate the layers as learning residual functions with reference to the layer inputs, instead of learning unreferenced functions. We provide comprehensive empirical evidence showing that these residual networks are easier to optimize, and can gain accuracy from
+considerably increased depth. On the ImageNet dataset we evaluate residual nets with a depth of up to 152 layers—8× deeper than VGG nets but still having lower complexity. An ensemble of these residual nets achieves 3.57% error on the ImageNet test set. This result won the 1st place on the ILSVRC 2015 classification task. We also present analysis on CIFAR-10 with 100 and 1000 layers. The depth of representations is of central importance for many visual recognition tasks. Solely due to our extremely deep representations, we obtain a 28% relative improvement on the COCO object detection dataset. Deep residual nets are foundations of our submissions to ILSVRC
+& COCO 2015 competitions1 , where we also won the 1st places on the tasks of ImageNet detection, ImageNet localization, COCO detection, and COCO segmentation.
+
+
+
+
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/resnetv1d.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/resnetv1d.md
new file mode 100644
index 0000000..ebde554
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/resnetv1d.md
@@ -0,0 +1,31 @@
+# Bag of tricks for image classification with convolutional neural networks
+
+
+
+
+ResNetV1D (CVPR'2019)
+
+```bibtex
+@inproceedings{he2019bag,
+ title={Bag of tricks for image classification with convolutional neural networks},
+ author={He, Tong and Zhang, Zhi and Zhang, Hang and Zhang, Zhongyue and Xie, Junyuan and Li, Mu},
+ booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
+ pages={558--567},
+ year={2019}
+}
+```
+
+
+
+## Abstract
+
+
+
+Much of the recent progress made in image classification research can be credited to training procedure refinements, such as changes in data augmentations and optimization methods. In the literature, however, most refinements are either briefly mentioned as implementation details or only visible in source code. In this paper, we will examine a collection of such refinements and empirically evaluate their impact on the final model accuracy through ablation study. We will show that, by combining these refinements together, we are able to improve various CNN models significantly. For example, we raise ResNet-50’s top-1 validation accuracy from 75.3% to 79.29% on ImageNet. We will also demonstrate that improvement on image classification accuracy leads to better transfer learning performance in other application domains such as object detection and semantic
+segmentation.
+
+
+
+
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/resnext.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/resnext.md
new file mode 100644
index 0000000..9803ee9
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/resnext.md
@@ -0,0 +1,30 @@
+# Aggregated residual transformations for deep neural networks
+
+
+
+
+ResNext (CVPR'2017)
+
+```bibtex
+@inproceedings{xie2017aggregated,
+ title={Aggregated residual transformations for deep neural networks},
+ author={Xie, Saining and Girshick, Ross and Doll{\'a}r, Piotr and Tu, Zhuowen and He, Kaiming},
+ booktitle={Proceedings of the IEEE conference on computer vision and pattern recognition},
+ pages={1492--1500},
+ year={2017}
+}
+```
+
+
+
+## Abstract
+
+
+
+We present a simple, highly modularized network architecture for image classification. Our network is constructed by repeating a building block that aggregates a set of transformations with the same topology. Our simple design results in a homogeneous, multi-branch architecture that has only a few hyper-parameters to set. This strategy exposes a new dimension, which we call "cardinality" (the size of the set of transformations), as an essential factor in addition to the dimensions of depth and width. On the ImageNet-1K dataset, we empirically show that even under the restricted condition of maintaining complexity, increasing cardinality is able to improve classification accuracy. Moreover, increasing cardinality is more effective than going deeper or wider when we increase the capacity. Our models, named ResNeXt, are the foundations of our entry to the ILSVRC 2016 classification task in which we secured 2nd place. We further investigate ResNeXt on an ImageNet-5K set and the COCO detection set, also showing better results than its ResNet counterpart. The code and models are publicly available online.
+
+
+
+
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/rsn.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/rsn.md
new file mode 100644
index 0000000..b1fb1ea
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/rsn.md
@@ -0,0 +1,31 @@
+# Learning delicate local representations for multi-person pose estimation
+
+
+
+
+RSN (ECCV'2020)
+
+```bibtex
+@misc{cai2020learning,
+ title={Learning Delicate Local Representations for Multi-Person Pose Estimation},
+ author={Yuanhao Cai and Zhicheng Wang and Zhengxiong Luo and Binyi Yin and Angang Du and Haoqian Wang and Xinyu Zhou and Erjin Zhou and Xiangyu Zhang and Jian Sun},
+ year={2020},
+ eprint={2003.04030},
+ archivePrefix={arXiv},
+ primaryClass={cs.CV}
+}
+```
+
+
+
+## Abstract
+
+
+
+In this paper, we propose a novel method called Residual Steps Network (RSN). RSN aggregates features with the same spatial size (Intra-level features) efficiently to obtain delicate local representations, which retain rich low-level spatial information and result in precise keypoint localization. Additionally, we observe the output features contribute differently to final performance. To tackle this problem, we propose an efficient attention mechanism - Pose Refine Machine (PRM) to make a trade-off between local and global representations in output features and further refine the keypoint locations. Our approach won the 1st place of COCO Keypoint Challenge 2019 and achieves state-of-the-art results on both COCO and MPII benchmarks, without using extra training data and pretrained model. Our single model achieves 78.6 on COCO test-dev, 93.0 on MPII test dataset. Ensembled models achieve 79.2 on COCO test-dev, 77.1 on COCO test-challenge dataset.
+
+
+
+
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/scnet.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/scnet.md
new file mode 100644
index 0000000..043c144
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/scnet.md
@@ -0,0 +1,30 @@
+# Improving Convolutional Networks with Self-Calibrated Convolutions
+
+
+
+
+SCNet (CVPR'2020)
+
+```bibtex
+@inproceedings{liu2020improving,
+ title={Improving Convolutional Networks with Self-Calibrated Convolutions},
+ author={Liu, Jiang-Jiang and Hou, Qibin and Cheng, Ming-Ming and Wang, Changhu and Feng, Jiashi},
+ booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
+ pages={10096--10105},
+ year={2020}
+}
+```
+
+
+
+## Abstract
+
+
+
+Recent advances on CNNs are mostly devoted to designing more complex architectures to enhance their representation learning capacity. In this paper, we consider how to improve the basic convolutional feature transformation process of CNNs without tuning the model architectures. To this end, we present a novel self-calibrated convolutions that explicitly expand fields-of-view of each convolutional layers through internal communications and hence enrich the output features. In particular, unlike the standard convolutions that fuse spatial and channel-wise information using small kernels (e.g., 3x3), self-calibrated convolutions adaptively build long-range spatial and inter-channel dependencies around each spatial location through a novel self-calibration operation. Thus, it can help CNNs generate more discriminative representations by explicitly incorporating richer information. Our self-calibrated convolution design is simple and generic, and can be easily applied to augment standard convolutional layers without introducing extra parameters and complexity. Extensive experiments demonstrate that when applying self-calibrated convolutions into different backbones, our networks can significantly improve the baseline models in a variety of vision tasks, including image recognition, object detection, instance segmentation, and keypoint detection, with no need to change the network architectures. We hope this work could provide a promising way for future research in designing novel convolutional feature transformations for improving convolutional networks. Code is available on the project page.
+
+
+
+
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/seresnet.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/seresnet.md
new file mode 100644
index 0000000..52178e5
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/seresnet.md
@@ -0,0 +1,30 @@
+# Squeeze-and-excitation networks
+
+
+
+
+SEResNet (CVPR'2018)
+
+```bibtex
+@inproceedings{hu2018squeeze,
+ title={Squeeze-and-excitation networks},
+ author={Hu, Jie and Shen, Li and Sun, Gang},
+ booktitle={Proceedings of the IEEE conference on computer vision and pattern recognition},
+ pages={7132--7141},
+ year={2018}
+}
+```
+
+
+
+## Abstract
+
+
+
+Convolutional neural networks are built upon the convolution operation, which extracts informative features by fusing spatial and channel-wise information together within local receptive fields. In order to boost the representational power of a network, several recent approaches have shown the benefit of enhancing spatial encoding. In this work, we focus on the channel relationship and propose a novel architectural unit, which we term the “Squeeze-and-Excitation” (SE) block, that adaptively recalibrates channel-wise feature responses by explicitly modelling interdependencies between channels. We demonstrate that by stacking these blocks together, we can construct SENet architectures that generalise extremely well across challenging datasets. Crucially, we find that SE blocks produce significant performance improvements for existing state-of-the-art deep architectures at minimal additional computational cost. SENets formed the foundation of our ILSVRC 2017 classification submission which won first place and significantly reduced the top-5 error to 2.251%, achieving a ∼25% relative improvement over the winning entry of 2016. Code and models are available at https: //github.com/hujie-frank/SENet.
+
+
+
+
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/shufflenetv1.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/shufflenetv1.md
new file mode 100644
index 0000000..a314c9b
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/shufflenetv1.md
@@ -0,0 +1,30 @@
+# Shufflenet: An extremely efficient convolutional neural network for mobile devices
+
+
+
+
+ShufflenetV1 (CVPR'2018)
+
+```bibtex
+@inproceedings{zhang2018shufflenet,
+ title={Shufflenet: An extremely efficient convolutional neural network for mobile devices},
+ author={Zhang, Xiangyu and Zhou, Xinyu and Lin, Mengxiao and Sun, Jian},
+ booktitle={Proceedings of the IEEE conference on computer vision and pattern recognition},
+ pages={6848--6856},
+ year={2018}
+}
+```
+
+
+
+## Abstract
+
+
+
+We introduce an extremely computation-efficient CNN architecture named ShuffleNet, which is designed specially for mobile devices with very limited computing power (e.g., 10-150 MFLOPs). The new architecture utilizes two new operations, pointwise group convolution and channel shuffle, to greatly reduce computation cost while maintaining accuracy. Experiments on ImageNet classification and MS COCO object detection demonstrate the superior performance of ShuffleNet over other structures, e.g. lower top-1 error (absolute 7.8%) than recent MobileNet~cite{howard2017mobilenets} on ImageNet classification task, under the computation budget of 40 MFLOPs. On an ARM-based mobile device, ShuffleNet achieves $sim$13$ imes$ actual speedup over AlexNet while maintaining comparable accuracy.
+
+
+
+
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/shufflenetv2.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/shufflenetv2.md
new file mode 100644
index 0000000..834ee38
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/shufflenetv2.md
@@ -0,0 +1,30 @@
+# Shufflenet v2: Practical guidelines for efficient cnn architecture design
+
+
+
+
+ShufflenetV2 (ECCV'2018)
+
+```bibtex
+@inproceedings{ma2018shufflenet,
+ title={Shufflenet v2: Practical guidelines for efficient cnn architecture design},
+ author={Ma, Ningning and Zhang, Xiangyu and Zheng, Hai-Tao and Sun, Jian},
+ booktitle={Proceedings of the European conference on computer vision (ECCV)},
+ pages={116--131},
+ year={2018}
+}
+```
+
+
+
+## Abstract
+
+
+
+Current network architecture design is mostly guided by the indirect metric of computation complexity, i.e., FLOPs. However, the direct metric, such as speed, also depends on the other factors such as memory access cost and platform characterics. Taking these factors into account, this work proposes practical guidelines for efficient network de- sign. Accordingly, a new architecture called ShuffleNet V2 is presented. Comprehensive experiments verify that it is the state-of-the-art in both speed and accuracy.
+
+
+
+
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/vgg.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/vgg.md
new file mode 100644
index 0000000..3a92a46
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/vgg.md
@@ -0,0 +1,29 @@
+# Very Deep Convolutional Networks for Large-Scale Image Recognition
+
+
+
+
+VGG (ICLR'2015)
+
+```bibtex
+@article{simonyan2014very,
+ title={Very deep convolutional networks for large-scale image recognition},
+ author={Simonyan, Karen and Zisserman, Andrew},
+ journal={arXiv preprint arXiv:1409.1556},
+ year={2014}
+}
+```
+
+
+
+## Abstract
+
+
+
+In this work we investigate the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting. Our main contribution is a thorough evaluation of networks of increasing depth using an architecture with very small (3x3) convolution filters, which shows that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 weight layers. These findings were the basis of our ImageNet Challenge 2014 submission, where our team secured the first and the second places in the localisation and classification tracks respectively. We also show that our representations generalise well to other datasets, where they achieve state-of-the-art results. We have made our two best-performing ConvNet models publicly available to facilitate further research on the use of deep visual representations in computer vision.
+
+
+
+
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/vipnas.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/vipnas.md
new file mode 100644
index 0000000..5f52a8c
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/backbones/vipnas.md
@@ -0,0 +1,29 @@
+# ViPNAS: Efficient Video Pose Estimation via Neural Architecture Search
+
+
+
+
+ViPNAS (CVPR'2021)
+
+```bibtex
+@article{xu2021vipnas,
+ title={ViPNAS: Efficient Video Pose Estimation via Neural Architecture Search},
+ author={Xu, Lumin and Guan, Yingda and Jin, Sheng and Liu, Wentao and Qian, Chen and Luo, Ping and Ouyang, Wanli and Wang, Xiaogang},
+ booktitle={Proceedings of the IEEE conference on computer vision and pattern recognition},
+ year={2021}
+}
+```
+
+
+
+## Abstract
+
+
+
+Human pose estimation has achieved significant progress in recent years. However, most of the recent methods focus on improving accuracy using complicated models and ignoring real-time efficiency. To achieve a better trade-off between accuracy and efficiency, we propose a novel neural architecture search (NAS) method, termed ViPNAS, to search networks in both spatial and temporal levels for fast online video pose estimation. In the spatial level, we carefully design the search space with five different dimensions including network depth, width, kernel size, group number, and attentions. In the temporal level, we search from a series of temporal feature fusions to optimize the total accuracy and speed across multiple video frames. To the best of our knowledge, we are the first to search for the temporal feature fusion and automatic computation allocation in videos. Extensive experiments demonstrate the effectiveness of our approach on the challenging COCO2017 and PoseTrack2018 datasets. Our discovered model family, S-ViPNAS and T-ViPNAS, achieve significantly higher inference speed (CPU real-time) without sacrificing the accuracy compared to the previous state-of-the-art methods.
+
+
+
+
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/300w.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/300w.md
new file mode 100644
index 0000000..7af778e
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/300w.md
@@ -0,0 +1,20 @@
+# 300 faces in-the-wild challenge: Database and results
+
+
+
+
+300W (IMAVIS'2016)
+
+```bibtex
+@article{sagonas2016300,
+ title={300 faces in-the-wild challenge: Database and results},
+ author={Sagonas, Christos and Antonakos, Epameinondas and Tzimiropoulos, Georgios and Zafeiriou, Stefanos and Pantic, Maja},
+ journal={Image and vision computing},
+ volume={47},
+ pages={3--18},
+ year={2016},
+ publisher={Elsevier}
+}
+```
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/aflw.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/aflw.md
new file mode 100644
index 0000000..f04f265
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/aflw.md
@@ -0,0 +1,19 @@
+# Annotated facial landmarks in the wild: A large-scale, real-world database for facial landmark localization
+
+
+
+
+AFLW (ICCVW'2011)
+
+```bibtex
+@inproceedings{koestinger2011annotated,
+ title={Annotated facial landmarks in the wild: A large-scale, real-world database for facial landmark localization},
+ author={Koestinger, Martin and Wohlhart, Paul and Roth, Peter M and Bischof, Horst},
+ booktitle={2011 IEEE international conference on computer vision workshops (ICCV workshops)},
+ pages={2144--2151},
+ year={2011},
+ organization={IEEE}
+}
+```
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/aic.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/aic.md
new file mode 100644
index 0000000..5054609
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/aic.md
@@ -0,0 +1,17 @@
+# Ai challenger: A large-scale dataset for going deeper in image understanding
+
+
+
+
+AI Challenger (ArXiv'2017)
+
+```bibtex
+@article{wu2017ai,
+ title={Ai challenger: A large-scale dataset for going deeper in image understanding},
+ author={Wu, Jiahong and Zheng, He and Zhao, Bo and Li, Yixin and Yan, Baoming and Liang, Rui and Wang, Wenjia and Zhou, Shipei and Lin, Guosen and Fu, Yanwei and others},
+ journal={arXiv preprint arXiv:1711.06475},
+ year={2017}
+}
+```
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/animalpose.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/animalpose.md
new file mode 100644
index 0000000..58303b8
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/animalpose.md
@@ -0,0 +1,18 @@
+# Cross-Domain Adaptation for Animal Pose Estimation
+
+
+
+
+Animal-Pose (ICCV'2019)
+
+```bibtex
+@InProceedings{Cao_2019_ICCV,
+ author = {Cao, Jinkun and Tang, Hongyang and Fang, Hao-Shu and Shen, Xiaoyong and Lu, Cewu and Tai, Yu-Wing},
+ title = {Cross-Domain Adaptation for Animal Pose Estimation},
+ booktitle = {The IEEE International Conference on Computer Vision (ICCV)},
+ month = {October},
+ year = {2019}
+}
+```
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/ap10k.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/ap10k.md
new file mode 100644
index 0000000..e36988d
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/ap10k.md
@@ -0,0 +1,19 @@
+# AP-10K: A Benchmark for Animal Pose Estimation in the Wild
+
+
+
+
+AP-10K (NeurIPS'2021)
+
+```bibtex
+@misc{yu2021ap10k,
+ title={AP-10K: A Benchmark for Animal Pose Estimation in the Wild},
+ author={Hang Yu and Yufei Xu and Jing Zhang and Wei Zhao and Ziyu Guan and Dacheng Tao},
+ year={2021},
+ eprint={2108.12617},
+ archivePrefix={arXiv},
+ primaryClass={cs.CV}
+}
+```
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/atrw.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/atrw.md
new file mode 100644
index 0000000..fe83ac0
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/atrw.md
@@ -0,0 +1,18 @@
+# ATRW: A Benchmark for Amur Tiger Re-identification in the Wild
+
+
+
+
+ATRW (ACM MM'2020)
+
+```bibtex
+@inproceedings{li2020atrw,
+ title={ATRW: A Benchmark for Amur Tiger Re-identification in the Wild},
+ author={Li, Shuyuan and Li, Jianguo and Tang, Hanlin and Qian, Rui and Lin, Weiyao},
+ booktitle={Proceedings of the 28th ACM International Conference on Multimedia},
+ pages={2590--2598},
+ year={2020}
+}
+```
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/coco.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/coco.md
new file mode 100644
index 0000000..8051dc7
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/coco.md
@@ -0,0 +1,19 @@
+# Microsoft coco: Common objects in context
+
+
+
+
+COCO (ECCV'2014)
+
+```bibtex
+@inproceedings{lin2014microsoft,
+ title={Microsoft coco: Common objects in context},
+ author={Lin, Tsung-Yi and Maire, Michael and Belongie, Serge and Hays, James and Perona, Pietro and Ramanan, Deva and Doll{\'a}r, Piotr and Zitnick, C Lawrence},
+ booktitle={European conference on computer vision},
+ pages={740--755},
+ year={2014},
+ organization={Springer}
+}
+```
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/coco_wholebody.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/coco_wholebody.md
new file mode 100644
index 0000000..69cb2b9
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/coco_wholebody.md
@@ -0,0 +1,17 @@
+# Whole-Body Human Pose Estimation in the Wild
+
+
+
+
+COCO-WholeBody (ECCV'2020)
+
+```bibtex
+@inproceedings{jin2020whole,
+ title={Whole-Body Human Pose Estimation in the Wild},
+ author={Jin, Sheng and Xu, Lumin and Xu, Jin and Wang, Can and Liu, Wentao and Qian, Chen and Ouyang, Wanli and Luo, Ping},
+ booktitle={Proceedings of the European Conference on Computer Vision (ECCV)},
+ year={2020}
+}
+```
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/coco_wholebody_face.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/coco_wholebody_face.md
new file mode 100644
index 0000000..3e1d3d4
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/coco_wholebody_face.md
@@ -0,0 +1,17 @@
+# Whole-Body Human Pose Estimation in the Wild
+
+
+
+
+COCO-WholeBody-Face (ECCV'2020)
+
+```bibtex
+@inproceedings{jin2020whole,
+ title={Whole-Body Human Pose Estimation in the Wild},
+ author={Jin, Sheng and Xu, Lumin and Xu, Jin and Wang, Can and Liu, Wentao and Qian, Chen and Ouyang, Wanli and Luo, Ping},
+ booktitle={Proceedings of the European Conference on Computer Vision (ECCV)},
+ year={2020}
+}
+```
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/coco_wholebody_hand.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/coco_wholebody_hand.md
new file mode 100644
index 0000000..51e2169
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/coco_wholebody_hand.md
@@ -0,0 +1,17 @@
+# Whole-Body Human Pose Estimation in the Wild
+
+
+
+
+COCO-WholeBody-Hand (ECCV'2020)
+
+```bibtex
+@inproceedings{jin2020whole,
+ title={Whole-Body Human Pose Estimation in the Wild},
+ author={Jin, Sheng and Xu, Lumin and Xu, Jin and Wang, Can and Liu, Wentao and Qian, Chen and Ouyang, Wanli and Luo, Ping},
+ booktitle={Proceedings of the European Conference on Computer Vision (ECCV)},
+ year={2020}
+}
+```
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/cofw.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/cofw.md
new file mode 100644
index 0000000..20d29ac
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/cofw.md
@@ -0,0 +1,18 @@
+# Robust face landmark estimation under occlusion
+
+
+
+
+COFW (ICCV'2013)
+
+```bibtex
+@inproceedings{burgos2013robust,
+ title={Robust face landmark estimation under occlusion},
+ author={Burgos-Artizzu, Xavier P and Perona, Pietro and Doll{\'a}r, Piotr},
+ booktitle={Proceedings of the IEEE international conference on computer vision},
+ pages={1513--1520},
+ year={2013}
+}
+```
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/crowdpose.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/crowdpose.md
new file mode 100644
index 0000000..ee678aa
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/crowdpose.md
@@ -0,0 +1,17 @@
+# CrowdPose: Efficient Crowded Scenes Pose Estimation and A New Benchmark
+
+
+
+
+CrowdPose (CVPR'2019)
+
+```bibtex
+@article{li2018crowdpose,
+ title={CrowdPose: Efficient Crowded Scenes Pose Estimation and A New Benchmark},
+ author={Li, Jiefeng and Wang, Can and Zhu, Hao and Mao, Yihuan and Fang, Hao-Shu and Lu, Cewu},
+ journal={arXiv preprint arXiv:1812.00324},
+ year={2018}
+}
+```
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/deepfashion.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/deepfashion.md
new file mode 100644
index 0000000..3955cf3
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/deepfashion.md
@@ -0,0 +1,35 @@
+# DeepFashion: Powering Robust Clothes Recognition and Retrieval with Rich Annotations
+
+
+
+
+DeepFashion (CVPR'2016)
+
+```bibtex
+@inproceedings{liuLQWTcvpr16DeepFashion,
+ author = {Liu, Ziwei and Luo, Ping and Qiu, Shi and Wang, Xiaogang and Tang, Xiaoou},
+ title = {DeepFashion: Powering Robust Clothes Recognition and Retrieval with Rich Annotations},
+ booktitle = {Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
+ month = {June},
+ year = {2016}
+}
+```
+
+
+
+
+
+
+DeepFashion (ECCV'2016)
+
+```bibtex
+@inproceedings{liuYLWTeccv16FashionLandmark,
+ author = {Liu, Ziwei and Yan, Sijie and Luo, Ping and Wang, Xiaogang and Tang, Xiaoou},
+ title = {Fashion Landmark Detection in the Wild},
+ booktitle = {European Conference on Computer Vision (ECCV)},
+ month = {October},
+ year = {2016}
+ }
+```
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/fly.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/fly.md
new file mode 100644
index 0000000..ed1a9c1
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/fly.md
@@ -0,0 +1,21 @@
+# Fast animal pose estimation using deep neural networks
+
+
+
+
+Vinegar Fly (Nature Methods'2019)
+
+```bibtex
+@article{pereira2019fast,
+ title={Fast animal pose estimation using deep neural networks},
+ author={Pereira, Talmo D and Aldarondo, Diego E and Willmore, Lindsay and Kislin, Mikhail and Wang, Samuel S-H and Murthy, Mala and Shaevitz, Joshua W},
+ journal={Nature methods},
+ volume={16},
+ number={1},
+ pages={117--125},
+ year={2019},
+ publisher={Nature Publishing Group}
+}
+```
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/freihand.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/freihand.md
new file mode 100644
index 0000000..ee08602
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/freihand.md
@@ -0,0 +1,18 @@
+# Freihand: A dataset for markerless capture of hand pose and shape from single rgb images
+
+
+
+
+FreiHand (ICCV'2019)
+
+```bibtex
+@inproceedings{zimmermann2019freihand,
+ title={Freihand: A dataset for markerless capture of hand pose and shape from single rgb images},
+ author={Zimmermann, Christian and Ceylan, Duygu and Yang, Jimei and Russell, Bryan and Argus, Max and Brox, Thomas},
+ booktitle={Proceedings of the IEEE International Conference on Computer Vision},
+ pages={813--822},
+ year={2019}
+}
+```
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/h36m.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/h36m.md
new file mode 100644
index 0000000..143e154
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/h36m.md
@@ -0,0 +1,22 @@
+# Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments
+
+
+
+
+Human3.6M (TPAMI'2014)
+
+```bibtex
+@article{h36m_pami,
+ author = {Ionescu, Catalin and Papava, Dragos and Olaru, Vlad and Sminchisescu, Cristian},
+ title = {Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments},
+ journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
+ publisher = {IEEE Computer Society},
+ volume = {36},
+ number = {7},
+ pages = {1325-1339},
+ month = {jul},
+ year = {2014}
+}
+```
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/halpe.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/halpe.md
new file mode 100644
index 0000000..f71793f
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/halpe.md
@@ -0,0 +1,17 @@
+# PaStaNet: Toward Human Activity Knowledge Engine
+
+
+
+
+Halpe (CVPR'2020)
+
+```bibtex
+@inproceedings{li2020pastanet,
+ title={PaStaNet: Toward Human Activity Knowledge Engine},
+ author={Li, Yong-Lu and Xu, Liang and Liu, Xinpeng and Huang, Xijie and Xu, Yue and Wang, Shiyi and Fang, Hao-Shu and Ma, Ze and Chen, Mingyang and Lu, Cewu},
+ booktitle={CVPR},
+ year={2020}
+}
+```
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/horse10.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/horse10.md
new file mode 100644
index 0000000..94e559d
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/horse10.md
@@ -0,0 +1,18 @@
+# Pretraining boosts out-of-domain robustness for pose estimation
+
+
+
+
+Horse-10 (WACV'2021)
+
+```bibtex
+@inproceedings{mathis2021pretraining,
+ title={Pretraining boosts out-of-domain robustness for pose estimation},
+ author={Mathis, Alexander and Biasi, Thomas and Schneider, Steffen and Yuksekgonul, Mert and Rogers, Byron and Bethge, Matthias and Mathis, Mackenzie W},
+ booktitle={Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision},
+ pages={1859--1868},
+ year={2021}
+}
+```
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/interhand.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/interhand.md
new file mode 100644
index 0000000..6b4458a
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/interhand.md
@@ -0,0 +1,18 @@
+# InterHand2.6M: A dataset and baseline for 3D interacting hand pose estimation from a single RGB image
+
+
+
+
+InterHand2.6M (ECCV'2020)
+
+```bibtex
+@article{moon2020interhand2,
+ title={InterHand2.6M: A dataset and baseline for 3D interacting hand pose estimation from a single RGB image},
+ author={Moon, Gyeongsik and Yu, Shoou-I and Wen, He and Shiratori, Takaaki and Lee, Kyoung Mu},
+ journal={arXiv preprint arXiv:2008.09309},
+ year={2020},
+ publisher={Springer}
+}
+```
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/jhmdb.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/jhmdb.md
new file mode 100644
index 0000000..890d788
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/jhmdb.md
@@ -0,0 +1,19 @@
+# Towards understanding action recognition
+
+
+
+
+JHMDB (ICCV'2013)
+
+```bibtex
+@inproceedings{Jhuang:ICCV:2013,
+ title = {Towards understanding action recognition},
+ author = {H. Jhuang and J. Gall and S. Zuffi and C. Schmid and M. J. Black},
+ booktitle = {International Conf. on Computer Vision (ICCV)},
+ month = Dec,
+ pages = {3192-3199},
+ year = {2013}
+}
+```
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/locust.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/locust.md
new file mode 100644
index 0000000..896ee03
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/locust.md
@@ -0,0 +1,20 @@
+# DeepPoseKit, a software toolkit for fast and robust animal pose estimation using deep learning
+
+
+
+
+Desert Locust (Elife'2019)
+
+```bibtex
+@article{graving2019deepposekit,
+ title={DeepPoseKit, a software toolkit for fast and robust animal pose estimation using deep learning},
+ author={Graving, Jacob M and Chae, Daniel and Naik, Hemal and Li, Liang and Koger, Benjamin and Costelloe, Blair R and Couzin, Iain D},
+ journal={Elife},
+ volume={8},
+ pages={e47994},
+ year={2019},
+ publisher={eLife Sciences Publications Limited}
+}
+```
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/macaque.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/macaque.md
new file mode 100644
index 0000000..be4bec1
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/macaque.md
@@ -0,0 +1,18 @@
+# MacaquePose: A novel ‘in the wild’macaque monkey pose dataset for markerless motion capture
+
+
+
+
+MacaquePose (bioRxiv'2020)
+
+```bibtex
+@article{labuguen2020macaquepose,
+ title={MacaquePose: A novel ‘in the wild’macaque monkey pose dataset for markerless motion capture},
+ author={Labuguen, Rollyn and Matsumoto, Jumpei and Negrete, Salvador and Nishimaru, Hiroshi and Nishijo, Hisao and Takada, Masahiko and Go, Yasuhiro and Inoue, Ken-ichi and Shibata, Tomohiro},
+ journal={bioRxiv},
+ year={2020},
+ publisher={Cold Spring Harbor Laboratory}
+}
+```
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/mhp.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/mhp.md
new file mode 100644
index 0000000..6dc5b17
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/mhp.md
@@ -0,0 +1,18 @@
+# Understanding humans in crowded scenes: Deep nested adversarial learning and a new benchmark for multi-human parsing
+
+
+
+
+MHP (ACM MM'2018)
+
+```bibtex
+@inproceedings{zhao2018understanding,
+ title={Understanding humans in crowded scenes: Deep nested adversarial learning and a new benchmark for multi-human parsing},
+ author={Zhao, Jian and Li, Jianshu and Cheng, Yu and Sim, Terence and Yan, Shuicheng and Feng, Jiashi},
+ booktitle={Proceedings of the 26th ACM international conference on Multimedia},
+ pages={792--800},
+ year={2018}
+}
+```
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/mpi_inf_3dhp.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/mpi_inf_3dhp.md
new file mode 100644
index 0000000..3a26d49
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/mpi_inf_3dhp.md
@@ -0,0 +1,20 @@
+# Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervision
+
+
+
+
+MPI-INF-3DHP (3DV'2017)
+
+```bibtex
+@inproceedings{mono-3dhp2017,
+ author = {Mehta, Dushyant and Rhodin, Helge and Casas, Dan and Fua, Pascal and Sotnychenko, Oleksandr and Xu, Weipeng and Theobalt, Christian},
+ title = {Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervision},
+ booktitle = {3D Vision (3DV), 2017 Fifth International Conference on},
+ url = {http://gvv.mpi-inf.mpg.de/3dhp_dataset},
+ year = {2017},
+ organization={IEEE},
+ doi={10.1109/3dv.2017.00064},
+}
+```
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/mpii.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/mpii.md
new file mode 100644
index 0000000..e2df7cf
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/mpii.md
@@ -0,0 +1,18 @@
+# 2D Human Pose Estimation: New Benchmark and State of the Art Analysis
+
+
+
+
+MPII (CVPR'2014)
+
+```bibtex
+@inproceedings{andriluka14cvpr,
+ author = {Mykhaylo Andriluka and Leonid Pishchulin and Peter Gehler and Schiele, Bernt},
+ title = {2D Human Pose Estimation: New Benchmark and State of the Art Analysis},
+ booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
+ year = {2014},
+ month = {June}
+}
+```
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/mpii_trb.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/mpii_trb.md
new file mode 100644
index 0000000..b3e96a7
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/mpii_trb.md
@@ -0,0 +1,18 @@
+# TRB: A Novel Triplet Representation for Understanding 2D Human Body
+
+
+
+
+MPII-TRB (ICCV'2019)
+
+```bibtex
+@inproceedings{duan2019trb,
+ title={TRB: A Novel Triplet Representation for Understanding 2D Human Body},
+ author={Duan, Haodong and Lin, Kwan-Yee and Jin, Sheng and Liu, Wentao and Qian, Chen and Ouyang, Wanli},
+ booktitle={Proceedings of the IEEE International Conference on Computer Vision},
+ pages={9479--9488},
+ year={2019}
+}
+```
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/ochuman.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/ochuman.md
new file mode 100644
index 0000000..5211c34
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/ochuman.md
@@ -0,0 +1,18 @@
+# Pose2seg: Detection free human instance segmentation
+
+
+
+
+OCHuman (CVPR'2019)
+
+```bibtex
+@inproceedings{zhang2019pose2seg,
+ title={Pose2seg: Detection free human instance segmentation},
+ author={Zhang, Song-Hai and Li, Ruilong and Dong, Xin and Rosin, Paul and Cai, Zixi and Han, Xi and Yang, Dingcheng and Huang, Haozhi and Hu, Shi-Min},
+ booktitle={Proceedings of the IEEE conference on computer vision and pattern recognition},
+ pages={889--898},
+ year={2019}
+}
+```
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/onehand10k.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/onehand10k.md
new file mode 100644
index 0000000..5710fda
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/onehand10k.md
@@ -0,0 +1,21 @@
+# Mask-pose cascaded cnn for 2d hand pose estimation from single color image
+
+
+
+
+OneHand10K (TCSVT'2019)
+
+```bibtex
+@article{wang2018mask,
+ title={Mask-pose cascaded cnn for 2d hand pose estimation from single color image},
+ author={Wang, Yangang and Peng, Cong and Liu, Yebin},
+ journal={IEEE Transactions on Circuits and Systems for Video Technology},
+ volume={29},
+ number={11},
+ pages={3258--3268},
+ year={2018},
+ publisher={IEEE}
+}
+```
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/panoptic.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/panoptic.md
new file mode 100644
index 0000000..60719c4
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/panoptic.md
@@ -0,0 +1,18 @@
+# Hand keypoint detection in single images using multiview bootstrapping
+
+
+
+
+CMU Panoptic HandDB (CVPR'2017)
+
+```bibtex
+@inproceedings{simon2017hand,
+ title={Hand keypoint detection in single images using multiview bootstrapping},
+ author={Simon, Tomas and Joo, Hanbyul and Matthews, Iain and Sheikh, Yaser},
+ booktitle={Proceedings of the IEEE conference on Computer Vision and Pattern Recognition},
+ pages={1145--1153},
+ year={2017}
+}
+```
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/panoptic_body3d.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/panoptic_body3d.md
new file mode 100644
index 0000000..b7f45c8
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/panoptic_body3d.md
@@ -0,0 +1,17 @@
+# Panoptic Studio: A Massively Multiview System for Social Motion Capture
+
+
+
+
+CMU Panoptic (ICCV'2015)
+
+```bibtex
+@Article = {joo_iccv_2015,
+author = {Hanbyul Joo, Hao Liu, Lei Tan, Lin Gui, Bart Nabbe, Iain Matthews, Takeo Kanade, Shohei Nobuhara, and Yaser Sheikh},
+title = {Panoptic Studio: A Massively Multiview System for Social Motion Capture},
+booktitle = {ICCV},
+year = {2015}
+}
+```
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/posetrack18.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/posetrack18.md
new file mode 100644
index 0000000..90cfcb5
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/posetrack18.md
@@ -0,0 +1,18 @@
+# Posetrack: A benchmark for human pose estimation and tracking
+
+
+
+
+PoseTrack18 (CVPR'2018)
+
+```bibtex
+@inproceedings{andriluka2018posetrack,
+ title={Posetrack: A benchmark for human pose estimation and tracking},
+ author={Andriluka, Mykhaylo and Iqbal, Umar and Insafutdinov, Eldar and Pishchulin, Leonid and Milan, Anton and Gall, Juergen and Schiele, Bernt},
+ booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
+ pages={5167--5176},
+ year={2018}
+}
+```
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/rhd.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/rhd.md
new file mode 100644
index 0000000..1855037
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/rhd.md
@@ -0,0 +1,19 @@
+# Learning to Estimate 3D Hand Pose from Single RGB Images
+
+
+
+
+RHD (ICCV'2017)
+
+```bibtex
+@TechReport{zb2017hand,
+ author={Christian Zimmermann and Thomas Brox},
+ title={Learning to Estimate 3D Hand Pose from Single RGB Images},
+ institution={arXiv:1705.01389},
+ year={2017},
+ note="https://arxiv.org/abs/1705.01389",
+ url="https://lmb.informatik.uni-freiburg.de/projects/hand3d/"
+}
+```
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/wflw.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/wflw.md
new file mode 100644
index 0000000..08c3ccc
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/wflw.md
@@ -0,0 +1,18 @@
+# Look at boundary: A boundary-aware face alignment algorithm
+
+
+
+
+WFLW (CVPR'2018)
+
+```bibtex
+@inproceedings{wu2018look,
+ title={Look at boundary: A boundary-aware face alignment algorithm},
+ author={Wu, Wayne and Qian, Chen and Yang, Shuo and Wang, Quan and Cai, Yici and Zhou, Qiang},
+ booktitle={Proceedings of the IEEE conference on computer vision and pattern recognition},
+ pages={2129--2138},
+ year={2018}
+}
+```
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/zebra.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/zebra.md
new file mode 100644
index 0000000..2727e59
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/datasets/zebra.md
@@ -0,0 +1,20 @@
+# DeepPoseKit, a software toolkit for fast and robust animal pose estimation using deep learning
+
+
+
+
+Grévy’s Zebra (Elife'2019)
+
+```bibtex
+@article{graving2019deepposekit,
+ title={DeepPoseKit, a software toolkit for fast and robust animal pose estimation using deep learning},
+ author={Graving, Jacob M and Chae, Daniel and Naik, Hemal and Li, Liang and Koger, Benjamin and Costelloe, Blair R and Couzin, Iain D},
+ journal={Elife},
+ volume={8},
+ pages={e47994},
+ year={2019},
+ publisher={eLife Sciences Publications Limited}
+}
+```
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/techniques/albumentations.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/techniques/albumentations.md
new file mode 100644
index 0000000..9d09a7a
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/techniques/albumentations.md
@@ -0,0 +1,21 @@
+# Albumentations: fast and flexible image augmentations
+
+
+
+
+Albumentations (Information'2020)
+
+```bibtex
+@article{buslaev2020albumentations,
+ title={Albumentations: fast and flexible image augmentations},
+ author={Buslaev, Alexander and Iglovikov, Vladimir I and Khvedchenya, Eugene and Parinov, Alex and Druzhinin, Mikhail and Kalinin, Alexandr A},
+ journal={Information},
+ volume={11},
+ number={2},
+ pages={125},
+ year={2020},
+ publisher={Multidisciplinary Digital Publishing Institute}
+}
+```
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/techniques/awingloss.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/techniques/awingloss.md
new file mode 100644
index 0000000..4d4b93a
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/techniques/awingloss.md
@@ -0,0 +1,31 @@
+# Adaptive Wing Loss for Robust Face Alignment via Heatmap Regression
+
+
+
+
+AdaptiveWingloss (ICCV'2019)
+
+```bibtex
+@inproceedings{wang2019adaptive,
+ title={Adaptive wing loss for robust face alignment via heatmap regression},
+ author={Wang, Xinyao and Bo, Liefeng and Fuxin, Li},
+ booktitle={Proceedings of the IEEE/CVF international conference on computer vision},
+ pages={6971--6981},
+ year={2019}
+}
+```
+
+
+
+## Abstract
+
+
+
+Heatmap regression with a deep network has become one of the mainstream approaches to localize facial landmarks. However, the loss function for heatmap regression is rarely studied. In this paper, we analyze the ideal loss function properties for heatmap regression in face alignment problems. Then we propose a novel loss function, named Adaptive Wing loss, that is able to adapt its shape to different types of ground truth heatmap pixels. This adaptability penalizes loss more on foreground pixels while less on background pixels. To address the imbalance between foreground and background pixels, we also propose Weighted Loss Map, which assigns high weights on foreground and difficult background pixels to help training process focus more on pixels that are crucial to landmark localization. To further improve face alignment accuracy, we introduce boundary prediction and CoordConv with boundary coordinates. Extensive experiments on different benchmarks, including COFW, 300W and WFLW, show our approach outperforms the state-of-the-art by a significant margin on
+various evaluation metrics. Besides, the Adaptive Wing loss also helps other heatmap regression tasks.
+
+
+
+
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/techniques/dark.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/techniques/dark.md
new file mode 100644
index 0000000..083b759
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/techniques/dark.md
@@ -0,0 +1,30 @@
+# Distribution-aware coordinate representation for human pose estimation
+
+
+
+
+DarkPose (CVPR'2020)
+
+```bibtex
+@inproceedings{zhang2020distribution,
+ title={Distribution-aware coordinate representation for human pose estimation},
+ author={Zhang, Feng and Zhu, Xiatian and Dai, Hanbin and Ye, Mao and Zhu, Ce},
+ booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
+ pages={7093--7102},
+ year={2020}
+}
+```
+
+
+
+## Abstract
+
+
+
+While being the de facto standard coordinate representation for human pose estimation, heatmap has not been investigated in-depth. This work fills this gap. For the first time, we find that the process of decoding the predicted heatmaps into the final joint coordinates in the original image space is surprisingly significant for the performance. We further probe the design limitations of the standard coordinate decoding method, and propose a more principled distributionaware decoding method. Also, we improve the standard coordinate encoding process (i.e. transforming ground-truth coordinates to heatmaps) by generating unbiased/accurate heatmaps. Taking the two together, we formulate a novel Distribution-Aware coordinate Representation of Keypoints (DARK) method. Serving as a model-agnostic plug-in, DARK brings about significant performance boost to existing human pose estimation models. Extensive experiments show that DARK yields the best results on two common benchmarks, MPII and COCO. Besides, DARK achieves the 2nd place entry in the ICCV 2019 COCO Keypoints Challenge. The code is available online.
+
+
+
+
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/techniques/fp16.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/techniques/fp16.md
new file mode 100644
index 0000000..7fd7ee0
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/techniques/fp16.md
@@ -0,0 +1,17 @@
+# Mixed Precision Training
+
+
+
+
+FP16 (ArXiv'2017)
+
+```bibtex
+@article{micikevicius2017mixed,
+ title={Mixed precision training},
+ author={Micikevicius, Paulius and Narang, Sharan and Alben, Jonah and Diamos, Gregory and Elsen, Erich and Garcia, David and Ginsburg, Boris and Houston, Michael and Kuchaiev, Oleksii and Venkatesh, Ganesh and others},
+ journal={arXiv preprint arXiv:1710.03740},
+ year={2017}
+}
+```
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/techniques/softwingloss.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/techniques/softwingloss.md
new file mode 100644
index 0000000..524a608
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/techniques/softwingloss.md
@@ -0,0 +1,30 @@
+# Structure-Coherent Deep Feature Learning for Robust Face Alignment
+
+
+
+
+SoftWingloss (TIP'2021)
+
+```bibtex
+@article{lin2021structure,
+ title={Structure-Coherent Deep Feature Learning for Robust Face Alignment},
+ author={Lin, Chunze and Zhu, Beier and Wang, Quan and Liao, Renjie and Qian, Chen and Lu, Jiwen and Zhou, Jie},
+ journal={IEEE Transactions on Image Processing},
+ year={2021},
+ publisher={IEEE}
+}
+```
+
+
+
+## Abstract
+
+
+
+In this paper, we propose a structure-coherent deep feature learning method for face alignment. Unlike most existing face alignment methods which overlook the facial structure cues, we explicitly exploit the relation among facial landmarks to make the detector robust to hard cases such as occlusion and large pose. Specifically, we leverage a landmark-graph relational network to enforce the structural relationships among landmarks. We consider the facial landmarks as structural graph nodes and carefully design the neighborhood to passing features among the most related nodes. Our method dynamically adapts the weights of node neighborhood to eliminate distracted information from noisy nodes, such as occluded landmark point. Moreover, different from most previous works which only tend to penalize the landmarks absolute position during the training, we propose a relative location loss to enhance the information of relative location of landmarks. This relative location supervision further regularizes the facial structure. Our approach considers the interactions among facial landmarks and can be easily implemented on top of any convolutional backbone to boost the performance. Extensive experiments on three popular benchmarks, including WFLW, COFW and 300W, demonstrate the effectiveness of the proposed method. In particular, due to explicit structure modeling, our approach is especially robust to challenging cases resulting in impressive low failure rate on COFW and WFLW datasets.
+
+
+
+
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/techniques/udp.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/techniques/udp.md
new file mode 100644
index 0000000..bb4aceb
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/techniques/udp.md
@@ -0,0 +1,30 @@
+# The Devil is in the Details: Delving into Unbiased Data Processing for Human Pose Estimation
+
+
+
+
+UDP (CVPR'2020)
+
+```bibtex
+@InProceedings{Huang_2020_CVPR,
+ author = {Huang, Junjie and Zhu, Zheng and Guo, Feng and Huang, Guan},
+ title = {The Devil Is in the Details: Delving Into Unbiased Data Processing for Human Pose Estimation},
+ booktitle = {The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
+ month = {June},
+ year = {2020}
+}
+```
+
+
+
+## Abstract
+
+
+
+Recently, the leading performance of human pose estimation is dominated by top-down methods. Being a fundamental component in training and inference, data processing has not been systematically considered in pose estimation community, to the best of our knowledge. In this paper, we focus on this problem and find that the devil of top-down pose estimator is in the biased data processing. Specifically, by investigating the standard data processing in state-of-the-art approaches mainly including data transformation and encoding-decoding, we find that the results obtained by common flipping strategy are unaligned with the original ones in inference. Moreover, there is statistical error in standard encoding-decoding during both training and inference. Two problems couple together and significantly degrade the pose estimation performance. Based on quantitative analyses, we then formulate a principled way to tackle this dilemma. Data is processed in continuous space based on unit length (the intervals between pixels) instead of in discrete space with pixel, and a combined classification and regression approach is adopted to perform encoding-decoding. The Unbiased Data Processing (UDP) for human pose estimation can be achieved by combining the two together. UDP not only boosts the performance of existing methods by a large margin but also plays a important role in result reproducing and future exploration. As a model-agnostic approach, UDP promotes SimpleBaseline-ResNet50-256x192 by 1.5 AP (70.2 to 71.7) and HRNet-W32-256x192 by 1.7 AP (73.5 to 75.2) on COCO test-dev set. The HRNet-W48-384x288 equipped with UDP achieves 76.5 AP and sets a new state-of-the-art for human pose estimation. The source code is publicly available for further research.
+
+
+
+
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/papers/techniques/wingloss.md b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/techniques/wingloss.md
new file mode 100644
index 0000000..2aaa057
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/papers/techniques/wingloss.md
@@ -0,0 +1,31 @@
+# Wing Loss for Robust Facial Landmark Localisation with Convolutional Neural Networks
+
+
+
+
+Wingloss (CVPR'2018)
+
+```bibtex
+@inproceedings{feng2018wing,
+ title={Wing Loss for Robust Facial Landmark Localisation with Convolutional Neural Networks},
+ author={Feng, Zhen-Hua and Kittler, Josef and Awais, Muhammad and Huber, Patrik and Wu, Xiao-Jun},
+ booktitle={Computer Vision and Pattern Recognition (CVPR), 2018 IEEE Conference on},
+ year={2018},
+ pages ={2235-2245},
+ organization={IEEE}
+}
+```
+
+
+
+## Abstract
+
+
+
+We present a new loss function, namely Wing loss, for robust facial landmark localisation with Convolutional Neural Networks (CNNs). We first compare and analyse different loss functions including L2, L1 and smooth L1. The analysis of these loss functions suggests that, for the training of a CNN-based localisation model, more attention should be paid to small and medium range errors. To this end, we design a piece-wise loss function. The new loss amplifies the impact of errors from the interval (-w, w) by switching from L1 loss to a modified logarithm function. To address the problem of under-representation of samples with large out-of-plane head rotations in the training set, we propose a simple but effective boosting strategy, referred to as pose-based data balancing. In particular, we deal with the data imbalance problem by duplicating the minority training samples and perturbing them by injecting random image rotation, bounding box translation and other data augmentation approaches. Last, the proposed approach is extended to create a two-stage framework for robust facial landmark localisation. The experimental results obtained on AFLW and 300W demonstrate the merits of the Wing loss function, and prove the superiority of the proposed method over the state-of-the-art approaches.
+
+
+
+
+
+
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/stats.py b/engine/pose_estimation/third-party/ViTPose/docs/en/stats.py
new file mode 100644
index 0000000..10ce3ab
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/stats.py
@@ -0,0 +1,176 @@
+#!/usr/bin/env python
+# Copyright (c) OpenMMLab. All rights reserved.
+import functools as func
+import glob
+import re
+from os.path import basename, splitext
+
+import numpy as np
+import titlecase
+
+
+def anchor(name):
+ return re.sub(r'-+', '-', re.sub(r'[^a-zA-Z0-9]', '-',
+ name.strip().lower())).strip('-')
+
+
+# Count algorithms
+
+files = sorted(glob.glob('topics/*.md'))
+
+stats = []
+
+for f in files:
+ with open(f, 'r') as content_file:
+ content = content_file.read()
+
+ # title
+ title = content.split('\n')[0].replace('#', '')
+
+ # count papers
+ papers = set(
+ (papertype, titlecase.titlecase(paper.lower().strip()))
+ for (papertype, paper) in re.findall(
+ r'\s*\n.*?\btitle\s*=\s*{(.*?)}',
+ content, re.DOTALL))
+ # paper links
+ revcontent = '\n'.join(list(reversed(content.splitlines())))
+ paperlinks = {}
+ for _, p in papers:
+ print(p)
+ paperlinks[p] = ', '.join(
+ ((f'[{paperlink} ⇨]'
+ f'(topics/{splitext(basename(f))[0]}.html#{anchor(paperlink)})')
+ for paperlink in re.findall(
+ rf'\btitle\s*=\s*{{\s*{p}\s*}}.*?\n### (.*?)\s*[,;]?\s*\n',
+ revcontent, re.DOTALL | re.IGNORECASE)))
+ print(' ', paperlinks[p])
+ paperlist = '\n'.join(
+ sorted(f' - [{t}] {x} ({paperlinks[x]})' for t, x in papers))
+ # count configs
+ configs = set(x.lower().strip()
+ for x in re.findall(r'.*configs/.*\.py', content))
+
+ # count ckpts
+ ckpts = set(x.lower().strip()
+ for x in re.findall(r'https://download.*\.pth', content)
+ if 'mmpose' in x)
+
+ statsmsg = f"""
+## [{title}]({f})
+
+* Number of checkpoints: {len(ckpts)}
+* Number of configs: {len(configs)}
+* Number of papers: {len(papers)}
+{paperlist}
+
+ """
+
+ stats.append((papers, configs, ckpts, statsmsg))
+
+allpapers = func.reduce(lambda a, b: a.union(b), [p for p, _, _, _ in stats])
+allconfigs = func.reduce(lambda a, b: a.union(b), [c for _, c, _, _ in stats])
+allckpts = func.reduce(lambda a, b: a.union(b), [c for _, _, c, _ in stats])
+
+# Summarize
+
+msglist = '\n'.join(x for _, _, _, x in stats)
+papertypes, papercounts = np.unique([t for t, _ in allpapers],
+ return_counts=True)
+countstr = '\n'.join(
+ [f' - {t}: {c}' for t, c in zip(papertypes, papercounts)])
+
+modelzoo = f"""
+# Overview
+
+* Number of checkpoints: {len(allckpts)}
+* Number of configs: {len(allconfigs)}
+* Number of papers: {len(allpapers)}
+{countstr}
+
+For supported datasets, see [datasets overview](datasets.md).
+
+{msglist}
+
+"""
+
+with open('modelzoo.md', 'w') as f:
+ f.write(modelzoo)
+
+# Count datasets
+
+files = sorted(glob.glob('tasks/*.md'))
+# files = sorted(glob.glob('docs/tasks/*.md'))
+
+datastats = []
+
+for f in files:
+ with open(f, 'r') as content_file:
+ content = content_file.read()
+
+ # title
+ title = content.split('\n')[0].replace('#', '')
+
+ # count papers
+ papers = set(
+ (papertype, titlecase.titlecase(paper.lower().strip()))
+ for (papertype, paper) in re.findall(
+ r'\s*\n.*?\btitle\s*=\s*{(.*?)}',
+ content, re.DOTALL))
+ # paper links
+ revcontent = '\n'.join(list(reversed(content.splitlines())))
+ paperlinks = {}
+ for _, p in papers:
+ print(p)
+ paperlinks[p] = ', '.join(
+ (f'[{p} ⇨](tasks/{splitext(basename(f))[0]}.html#{anchor(p)})'
+ for p in re.findall(
+ rf'\btitle\s*=\s*{{\s*{p}\s*}}.*?\n## (.*?)\s*[,;]?\s*\n',
+ revcontent, re.DOTALL | re.IGNORECASE)))
+ print(' ', paperlinks[p])
+ paperlist = '\n'.join(
+ sorted(f' - [{t}] {x} ({paperlinks[x]})' for t, x in papers))
+ # count configs
+ configs = set(x.lower().strip()
+ for x in re.findall(r'https.*configs/.*\.py', content))
+
+ # count ckpts
+ ckpts = set(x.lower().strip()
+ for x in re.findall(r'https://download.*\.pth', content)
+ if 'mmpose' in x)
+
+ statsmsg = f"""
+## [{title}]({f})
+
+* Number of papers: {len(papers)}
+{paperlist}
+
+ """
+
+ datastats.append((papers, configs, ckpts, statsmsg))
+
+alldatapapers = func.reduce(lambda a, b: a.union(b),
+ [p for p, _, _, _ in datastats])
+
+# Summarize
+
+msglist = '\n'.join(x for _, _, _, x in stats)
+datamsglist = '\n'.join(x for _, _, _, x in datastats)
+papertypes, papercounts = np.unique([t for t, _ in alldatapapers],
+ return_counts=True)
+countstr = '\n'.join(
+ [f' - {t}: {c}' for t, c in zip(papertypes, papercounts)])
+
+modelzoo = f"""
+# Overview
+
+* Number of papers: {len(alldatapapers)}
+{countstr}
+
+For supported pose algorithms, see [modelzoo overview](modelzoo.md).
+
+{datamsglist}
+"""
+
+with open('datasets.md', 'w') as f:
+ f.write(modelzoo)
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/tasks/2d_animal_keypoint.md b/engine/pose_estimation/third-party/ViTPose/docs/en/tasks/2d_animal_keypoint.md
new file mode 100644
index 0000000..c33ebb8
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/tasks/2d_animal_keypoint.md
@@ -0,0 +1,448 @@
+# 2D Animal Keypoint Dataset
+
+It is recommended to symlink the dataset root to `$MMPOSE/data`.
+If your folder structure is different, you may need to change the corresponding paths in config files.
+
+MMPose supported datasets:
+
+- [Animal-Pose](#animal-pose) \[ [Homepage](https://sites.google.com/view/animal-pose/) \]
+- [AP-10K](#ap-10k) \[ [Homepage](https://github.com/AlexTheBad/AP-10K/) \]
+- [Horse-10](#horse-10) \[ [Homepage](http://www.mackenziemathislab.org/horse10) \]
+- [MacaquePose](#macaquepose) \[ [Homepage](http://www.pri.kyoto-u.ac.jp/datasets/macaquepose/index.html) \]
+- [Vinegar Fly](#vinegar-fly) \[ [Homepage](https://github.com/jgraving/DeepPoseKit-Data) \]
+- [Desert Locust](#desert-locust) \[ [Homepage](https://github.com/jgraving/DeepPoseKit-Data) \]
+- [Grévy’s Zebra](#grvys-zebra) \[ [Homepage](https://github.com/jgraving/DeepPoseKit-Data) \]
+- [ATRW](#atrw) \[ [Homepage](https://cvwc2019.github.io/challenge.html) \]
+
+## Animal-Pose
+
+
+
+
+Animal-Pose (ICCV'2019)
+
+```bibtex
+@InProceedings{Cao_2019_ICCV,
+ author = {Cao, Jinkun and Tang, Hongyang and Fang, Hao-Shu and Shen, Xiaoyong and Lu, Cewu and Tai, Yu-Wing},
+ title = {Cross-Domain Adaptation for Animal Pose Estimation},
+ booktitle = {The IEEE International Conference on Computer Vision (ICCV)},
+ month = {October},
+ year = {2019}
+}
+```
+
+
+
+For [Animal-Pose](https://sites.google.com/view/animal-pose/) dataset, we prepare the dataset as follows:
+
+1. Download the images of [PASCAL2011](http://www.google.com/url?q=http%3A%2F%2Fhost.robots.ox.ac.uk%2Fpascal%2FVOC%2Fvoc2011%2Findex.html&sa=D&sntz=1&usg=AFQjCNGmiJGkhSSWtShDe7NwRPyyyBUYSQ), especially the five categories (dog, cat, sheep, cow, horse), which we use as trainval dataset.
+1. Download the [test-set](https://drive.google.com/drive/folders/1DwhQobZlGntOXxdm7vQsE4bqbFmN3b9y?usp=sharing) images with raw annotations (1000 images, 5 categories).
+1. We have pre-processed the annotations to make it compatible with MMPose. Please download the annotation files from [annotations](https://download.openmmlab.com/mmpose/datasets/animalpose_annotations.tar). If you would like to generate the annotations by yourself, please check our dataset parsing [codes](/tools/dataset/parse_animalpose_dataset.py).
+
+Extract them under {MMPose}/data, and make them look like this:
+
+```text
+mmpose
+├── mmpose
+├── docs
+├── tests
+├── tools
+├── configs
+`── data
+ │── animalpose
+ │
+ │-- VOC2011
+ │ │-- Annotations
+ │ │-- ImageSets
+ │ │-- JPEGImages
+ │ │-- SegmentationClass
+ │ │-- SegmentationObject
+ │
+ │-- animalpose_image_part2
+ │ │-- cat
+ │ │-- cow
+ │ │-- dog
+ │ │-- horse
+ │ │-- sheep
+ │
+ │-- annotations
+ │ │-- animalpose_train.json
+ │ |-- animalpose_val.json
+ │ |-- animalpose_trainval.json
+ │ │-- animalpose_test.json
+ │
+ │-- PASCAL2011_animal_annotation
+ │ │-- cat
+ │ │ |-- 2007_000528_1.xml
+ │ │ |-- 2007_000549_1.xml
+ │ │ │-- ...
+ │ │-- cow
+ │ │-- dog
+ │ │-- horse
+ │ │-- sheep
+ │
+ │-- annimalpose_anno2
+ │ │-- cat
+ │ │ |-- ca1.xml
+ │ │ |-- ca2.xml
+ │ │ │-- ...
+ │ │-- cow
+ │ │-- dog
+ │ │-- horse
+ │ │-- sheep
+
+```
+
+The official dataset does not provide the official train/val/test set split.
+We choose the images from PascalVOC for train & val. In total, we have 3608 images and 5117 annotations for train+val, where
+2798 images with 4000 annotations are used for training, and 810 images with 1117 annotations are used for validation.
+Those images from other sources (1000 images with 1000 annotations) are used for testing.
+
+## AP-10K
+
+
+
+
+AP-10K (NeurIPS'2021)
+
+```bibtex
+@misc{yu2021ap10k,
+ title={AP-10K: A Benchmark for Animal Pose Estimation in the Wild},
+ author={Hang Yu and Yufei Xu and Jing Zhang and Wei Zhao and Ziyu Guan and Dacheng Tao},
+ year={2021},
+ eprint={2108.12617},
+ archivePrefix={arXiv},
+ primaryClass={cs.CV}
+}
+```
+
+
+
+For [AP-10K](https://github.com/AlexTheBad/AP-10K/) dataset, images and annotations can be downloaded from [download](https://drive.google.com/file/d/1-FNNGcdtAQRehYYkGY1y4wzFNg4iWNad/view?usp=sharing).
+Note, this data and annotation data is for non-commercial use only.
+
+Extract them under {MMPose}/data, and make them look like this:
+
+```text
+mmpose
+├── mmpose
+├── docs
+├── tests
+├── tools
+├── configs
+`── data
+ │── ap10k
+ │-- annotations
+ │ │-- ap10k-train-split1.json
+ │ |-- ap10k-train-split2.json
+ │ |-- ap10k-train-split3.json
+ │ │-- ap10k-val-split1.json
+ │ |-- ap10k-val-split2.json
+ │ |-- ap10k-val-split3.json
+ │ |-- ap10k-test-split1.json
+ │ |-- ap10k-test-split2.json
+ │ |-- ap10k-test-split3.json
+ │-- data
+ │ │-- 000000000001.jpg
+ │ │-- 000000000002.jpg
+ │ │-- ...
+
+```
+
+The annotation files in 'annotation' folder contains 50 labeled animal species. There are total 10,015 labeled images with 13,028 instances in the AP-10K dataset. We randonly split them into train, val, and test set following the ratio of 7:1:2.
+
+## Horse-10
+
+
+
+
+Horse-10 (WACV'2021)
+
+```bibtex
+@inproceedings{mathis2021pretraining,
+ title={Pretraining boosts out-of-domain robustness for pose estimation},
+ author={Mathis, Alexander and Biasi, Thomas and Schneider, Steffen and Yuksekgonul, Mert and Rogers, Byron and Bethge, Matthias and Mathis, Mackenzie W},
+ booktitle={Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision},
+ pages={1859--1868},
+ year={2021}
+}
+```
+
+
+
+For [Horse-10](http://www.mackenziemathislab.org/horse10) dataset, images can be downloaded from [download](http://www.mackenziemathislab.org/horse10).
+Please download the annotation files from [horse10_annotations](https://download.openmmlab.com/mmpose/datasets/horse10_annotations.tar). Note, this data and annotation data is for non-commercial use only, per the authors (see http://horse10.deeplabcut.org for more information).
+Extract them under {MMPose}/data, and make them look like this:
+
+```text
+mmpose
+├── mmpose
+├── docs
+├── tests
+├── tools
+├── configs
+`── data
+ │── horse10
+ │-- annotations
+ │ │-- horse10-train-split1.json
+ │ |-- horse10-train-split2.json
+ │ |-- horse10-train-split3.json
+ │ │-- horse10-test-split1.json
+ │ |-- horse10-test-split2.json
+ │ |-- horse10-test-split3.json
+ │-- labeled-data
+ │ │-- BrownHorseinShadow
+ │ │-- BrownHorseintoshadow
+ │ │-- ...
+
+```
+
+## MacaquePose
+
+
+
+
+MacaquePose (bioRxiv'2020)
+
+```bibtex
+@article{labuguen2020macaquepose,
+ title={MacaquePose: A novel ‘in the wild’macaque monkey pose dataset for markerless motion capture},
+ author={Labuguen, Rollyn and Matsumoto, Jumpei and Negrete, Salvador and Nishimaru, Hiroshi and Nishijo, Hisao and Takada, Masahiko and Go, Yasuhiro and Inoue, Ken-ichi and Shibata, Tomohiro},
+ journal={bioRxiv},
+ year={2020},
+ publisher={Cold Spring Harbor Laboratory}
+}
+```
+
+
+
+For [MacaquePose](http://www.pri.kyoto-u.ac.jp/datasets/macaquepose/index.html) dataset, images can be downloaded from [download](http://www.pri.kyoto-u.ac.jp/datasets/macaquepose/index.html).
+Please download the annotation files from [macaque_annotations](https://download.openmmlab.com/mmpose/datasets/macaque_annotations.tar).
+Extract them under {MMPose}/data, and make them look like this:
+
+```text
+mmpose
+├── mmpose
+├── docs
+├── tests
+├── tools
+├── configs
+`── data
+ │── macaque
+ │-- annotations
+ │ │-- macaque_train.json
+ │ |-- macaque_test.json
+ │-- images
+ │ │-- 01418849d54b3005.jpg
+ │ │-- 0142d1d1a6904a70.jpg
+ │ │-- 01ef2c4c260321b7.jpg
+ │ │-- 020a1c75c8c85238.jpg
+ │ │-- 020b1506eef2557d.jpg
+ │ │-- ...
+
+```
+
+Since the official dataset does not provide the test set, we randomly select 12500 images for training, and the rest for evaluation (see [code](/tools/dataset/parse_macaquepose_dataset.py)).
+
+## Vinegar Fly
+
+
+
+
+Vinegar Fly (Nature Methods'2019)
+
+```bibtex
+@article{pereira2019fast,
+ title={Fast animal pose estimation using deep neural networks},
+ author={Pereira, Talmo D and Aldarondo, Diego E and Willmore, Lindsay and Kislin, Mikhail and Wang, Samuel S-H and Murthy, Mala and Shaevitz, Joshua W},
+ journal={Nature methods},
+ volume={16},
+ number={1},
+ pages={117--125},
+ year={2019},
+ publisher={Nature Publishing Group}
+}
+```
+
+
+
+For [Vinegar Fly](https://github.com/jgraving/DeepPoseKit-Data) dataset, images can be downloaded from [vinegar_fly_images](https://download.openmmlab.com/mmpose/datasets/vinegar_fly_images.tar).
+Please download the annotation files from [vinegar_fly_annotations](https://download.openmmlab.com/mmpose/datasets/vinegar_fly_annotations.tar).
+Extract them under {MMPose}/data, and make them look like this:
+
+```text
+mmpose
+├── mmpose
+├── docs
+├── tests
+├── tools
+├── configs
+`── data
+ │── fly
+ │-- annotations
+ │ │-- fly_train.json
+ │ |-- fly_test.json
+ │-- images
+ │ │-- 0.jpg
+ │ │-- 1.jpg
+ │ │-- 2.jpg
+ │ │-- 3.jpg
+ │ │-- ...
+
+```
+
+Since the official dataset does not provide the test set, we randomly select 90\% images for training, and the rest (10\%) for evaluation (see [code](/tools/dataset/parse_deepposekit_dataset.py)).
+
+## Desert Locust
+
+
+
+
+Desert Locust (Elife'2019)
+
+```bibtex
+@article{graving2019deepposekit,
+ title={DeepPoseKit, a software toolkit for fast and robust animal pose estimation using deep learning},
+ author={Graving, Jacob M and Chae, Daniel and Naik, Hemal and Li, Liang and Koger, Benjamin and Costelloe, Blair R and Couzin, Iain D},
+ journal={Elife},
+ volume={8},
+ pages={e47994},
+ year={2019},
+ publisher={eLife Sciences Publications Limited}
+}
+```
+
+
+
+For [Desert Locust](https://github.com/jgraving/DeepPoseKit-Data) dataset, images can be downloaded from [locust_images](https://download.openmmlab.com/mmpose/datasets/locust_images.tar).
+Please download the annotation files from [locust_annotations](https://download.openmmlab.com/mmpose/datasets/locust_annotations.tar).
+Extract them under {MMPose}/data, and make them look like this:
+
+```text
+mmpose
+├── mmpose
+├── docs
+├── tests
+├── tools
+├── configs
+`── data
+ │── locust
+ │-- annotations
+ │ │-- locust_train.json
+ │ |-- locust_test.json
+ │-- images
+ │ │-- 0.jpg
+ │ │-- 1.jpg
+ │ │-- 2.jpg
+ │ │-- 3.jpg
+ │ │-- ...
+
+```
+
+Since the official dataset does not provide the test set, we randomly select 90\% images for training, and the rest (10\%) for evaluation (see [code](/tools/dataset/parse_deepposekit_dataset.py)).
+
+## Grévy’s Zebra
+
+
+
+
+Grévy’s Zebra (Elife'2019)
+
+```bibtex
+@article{graving2019deepposekit,
+ title={DeepPoseKit, a software toolkit for fast and robust animal pose estimation using deep learning},
+ author={Graving, Jacob M and Chae, Daniel and Naik, Hemal and Li, Liang and Koger, Benjamin and Costelloe, Blair R and Couzin, Iain D},
+ journal={Elife},
+ volume={8},
+ pages={e47994},
+ year={2019},
+ publisher={eLife Sciences Publications Limited}
+}
+```
+
+
+
+For [Grévy’s Zebra](https://github.com/jgraving/DeepPoseKit-Data) dataset, images can be downloaded from [zebra_images](https://download.openmmlab.com/mmpose/datasets/zebra_images.tar).
+Please download the annotation files from [zebra_annotations](https://download.openmmlab.com/mmpose/datasets/zebra_annotations.tar).
+Extract them under {MMPose}/data, and make them look like this:
+
+```text
+mmpose
+├── mmpose
+├── docs
+├── tests
+├── tools
+├── configs
+`── data
+ │── zebra
+ │-- annotations
+ │ │-- zebra_train.json
+ │ |-- zebra_test.json
+ │-- images
+ │ │-- 0.jpg
+ │ │-- 1.jpg
+ │ │-- 2.jpg
+ │ │-- 3.jpg
+ │ │-- ...
+
+```
+
+Since the official dataset does not provide the test set, we randomly select 90\% images for training, and the rest (10\%) for evaluation (see [code](/tools/dataset/parse_deepposekit_dataset.py)).
+
+## ATRW
+
+
+
+
+ATRW (ACM MM'2020)
+
+```bibtex
+@inproceedings{li2020atrw,
+ title={ATRW: A Benchmark for Amur Tiger Re-identification in the Wild},
+ author={Li, Shuyuan and Li, Jianguo and Tang, Hanlin and Qian, Rui and Lin, Weiyao},
+ booktitle={Proceedings of the 28th ACM International Conference on Multimedia},
+ pages={2590--2598},
+ year={2020}
+}
+```
+
+
+
+ATRW captures images of the Amur tiger (also known as Siberian tiger, Northeast-China tiger) in the wild.
+For [ATRW](https://cvwc2019.github.io/challenge.html) dataset, please download images from
+[Pose_train](https://lilablobssc.blob.core.windows.net/cvwc2019/train/atrw_pose_train.tar.gz),
+[Pose_val](https://lilablobssc.blob.core.windows.net/cvwc2019/train/atrw_pose_val.tar.gz), and
+[Pose_test](https://lilablobssc.blob.core.windows.net/cvwc2019/test/atrw_pose_test.tar.gz).
+Note that in the ATRW official annotation files, the key "file_name" is written as "filename". To make it compatible with
+other coco-type json files, we have modified this key.
+Please download the modified annotation files from [atrw_annotations](https://download.openmmlab.com/mmpose/datasets/atrw_annotations.tar).
+Extract them under {MMPose}/data, and make them look like this:
+
+```text
+mmpose
+├── mmpose
+├── docs
+├── tests
+├── tools
+├── configs
+`── data
+ │── atrw
+ │-- annotations
+ │ │-- keypoint_train.json
+ │ │-- keypoint_val.json
+ │ │-- keypoint_trainval.json
+ │-- images
+ │ │-- train
+ │ │ │-- 000002.jpg
+ │ │ │-- 000003.jpg
+ │ │ │-- ...
+ │ │-- val
+ │ │ │-- 000001.jpg
+ │ │ │-- 000013.jpg
+ │ │ │-- ...
+ │ │-- test
+ │ │ │-- 000000.jpg
+ │ │ │-- 000004.jpg
+ │ │ │-- ...
+
+```
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/tasks/2d_body_keypoint.md b/engine/pose_estimation/third-party/ViTPose/docs/en/tasks/2d_body_keypoint.md
new file mode 100644
index 0000000..625e4d5
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/tasks/2d_body_keypoint.md
@@ -0,0 +1,500 @@
+# 2D Body Keypoint Datasets
+
+It is recommended to symlink the dataset root to `$MMPOSE/data`.
+If your folder structure is different, you may need to change the corresponding paths in config files.
+
+MMPose supported datasets:
+
+- Images
+ - [COCO](#coco) \[ [Homepage](http://cocodataset.org/) \]
+ - [MPII](#mpii) \[ [Homepage](http://human-pose.mpi-inf.mpg.de/) \]
+ - [MPII-TRB](#mpii-trb) \[ [Homepage](https://github.com/kennymckormick/Triplet-Representation-of-human-Body) \]
+ - [AI Challenger](#aic) \[ [Homepage](https://github.com/AIChallenger/AI_Challenger_2017) \]
+ - [CrowdPose](#crowdpose) \[ [Homepage](https://github.com/Jeff-sjtu/CrowdPose) \]
+ - [OCHuman](#ochuman) \[ [Homepage](https://github.com/liruilong940607/OCHumanApi) \]
+ - [MHP](#mhp) \[ [Homepage](https://lv-mhp.github.io/dataset) \]
+- Videos
+ - [PoseTrack18](#posetrack18) \[ [Homepage](https://posetrack.net/users/download.php) \]
+ - [sub-JHMDB](#sub-jhmdb-dataset) \[ [Homepage](http://jhmdb.is.tue.mpg.de/dataset) \]
+
+## COCO
+
+
+
+
+COCO (ECCV'2014)
+
+```bibtex
+@inproceedings{lin2014microsoft,
+ title={Microsoft coco: Common objects in context},
+ author={Lin, Tsung-Yi and Maire, Michael and Belongie, Serge and Hays, James and Perona, Pietro and Ramanan, Deva and Doll{\'a}r, Piotr and Zitnick, C Lawrence},
+ booktitle={European conference on computer vision},
+ pages={740--755},
+ year={2014},
+ organization={Springer}
+}
+```
+
+
+
+For [COCO](http://cocodataset.org/) data, please download from [COCO download](http://cocodataset.org/#download), 2017 Train/Val is needed for COCO keypoints training and validation.
+[HRNet-Human-Pose-Estimation](https://github.com/HRNet/HRNet-Human-Pose-Estimation) provides person detection result of COCO val2017 to reproduce our multi-person pose estimation results.
+Please download from [OneDrive](https://1drv.ms/f/s!AhIXJn_J-blWzzDXoz5BeFl8sWM-) or [GoogleDrive](https://drive.google.com/drive/folders/1fRUDNUDxe9fjqcRZ2bnF_TKMlO0nB_dk?usp=sharing).
+Optionally, to evaluate on COCO'2017 test-dev, please download the [image-info](https://download.openmmlab.com/mmpose/datasets/person_keypoints_test-dev-2017.json).
+Download and extract them under $MMPOSE/data, and make them look like this:
+
+```text
+mmpose
+├── mmpose
+├── docs
+├── tests
+├── tools
+├── configs
+`── data
+ │── coco
+ │-- annotations
+ │ │-- person_keypoints_train2017.json
+ │ |-- person_keypoints_val2017.json
+ │ |-- person_keypoints_test-dev-2017.json
+ |-- person_detection_results
+ | |-- COCO_val2017_detections_AP_H_56_person.json
+ | |-- COCO_test-dev2017_detections_AP_H_609_person.json
+ │-- train2017
+ │ │-- 000000000009.jpg
+ │ │-- 000000000025.jpg
+ │ │-- 000000000030.jpg
+ │ │-- ...
+ `-- val2017
+ │-- 000000000139.jpg
+ │-- 000000000285.jpg
+ │-- 000000000632.jpg
+ │-- ...
+
+```
+
+## MPII
+
+
+
+
+MPII (CVPR'2014)
+
+```bibtex
+@inproceedings{andriluka14cvpr,
+ author = {Mykhaylo Andriluka and Leonid Pishchulin and Peter Gehler and Schiele, Bernt},
+ title = {2D Human Pose Estimation: New Benchmark and State of the Art Analysis},
+ booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
+ year = {2014},
+ month = {June}
+}
+```
+
+
+
+For [MPII](http://human-pose.mpi-inf.mpg.de/) data, please download from [MPII Human Pose Dataset](http://human-pose.mpi-inf.mpg.de/).
+We have converted the original annotation files into json format, please download them from [mpii_annotations](https://download.openmmlab.com/mmpose/datasets/mpii_annotations.tar).
+Extract them under {MMPose}/data, and make them look like this:
+
+```text
+mmpose
+├── mmpose
+├── docs
+├── tests
+├── tools
+├── configs
+`── data
+ │── mpii
+ |── annotations
+ | |── mpii_gt_val.mat
+ | |── mpii_test.json
+ | |── mpii_train.json
+ | |── mpii_trainval.json
+ | `── mpii_val.json
+ `── images
+ |── 000001163.jpg
+ |── 000003072.jpg
+
+```
+
+During training and inference, the prediction result will be saved as '.mat' format by default. We also provide a tool to convert this '.mat' to more readable '.json' format.
+
+```shell
+python tools/dataset/mat2json ${PRED_MAT_FILE} ${GT_JSON_FILE} ${OUTPUT_PRED_JSON_FILE}
+```
+
+For example,
+
+```shell
+python tools/dataset/mat2json work_dirs/res50_mpii_256x256/pred.mat data/mpii/annotations/mpii_val.json pred.json
+```
+
+## MPII-TRB
+
+
+
+
+MPII-TRB (ICCV'2019)
+
+```bibtex
+@inproceedings{duan2019trb,
+ title={TRB: A Novel Triplet Representation for Understanding 2D Human Body},
+ author={Duan, Haodong and Lin, Kwan-Yee and Jin, Sheng and Liu, Wentao and Qian, Chen and Ouyang, Wanli},
+ booktitle={Proceedings of the IEEE International Conference on Computer Vision},
+ pages={9479--9488},
+ year={2019}
+}
+```
+
+
+
+For [MPII-TRB](https://github.com/kennymckormick/Triplet-Representation-of-human-Body) data, please download from [MPII Human Pose Dataset](http://human-pose.mpi-inf.mpg.de/).
+Please download the annotation files from [mpii_trb_annotations](https://download.openmmlab.com/mmpose/datasets/mpii_trb_annotations.tar).
+Extract them under {MMPose}/data, and make them look like this:
+
+```text
+mmpose
+├── mmpose
+├── docs
+├── tests
+├── tools
+├── configs
+`── data
+ │── mpii
+ |── annotations
+ | |── mpii_trb_train.json
+ | |── mpii_trb_val.json
+ `── images
+ |── 000001163.jpg
+ |── 000003072.jpg
+
+```
+
+## AIC
+
+
+
+
+AI Challenger (ArXiv'2017)
+
+```bibtex
+@article{wu2017ai,
+ title={Ai challenger: A large-scale dataset for going deeper in image understanding},
+ author={Wu, Jiahong and Zheng, He and Zhao, Bo and Li, Yixin and Yan, Baoming and Liang, Rui and Wang, Wenjia and Zhou, Shipei and Lin, Guosen and Fu, Yanwei and others},
+ journal={arXiv preprint arXiv:1711.06475},
+ year={2017}
+}
+```
+
+
+
+For [AIC](https://github.com/AIChallenger/AI_Challenger_2017) data, please download from [AI Challenger 2017](https://github.com/AIChallenger/AI_Challenger_2017), 2017 Train/Val is needed for keypoints training and validation.
+Please download the annotation files from [aic_annotations](https://download.openmmlab.com/mmpose/datasets/aic_annotations.tar).
+Download and extract them under $MMPOSE/data, and make them look like this:
+
+```text
+mmpose
+├── mmpose
+├── docs
+├── tests
+├── tools
+├── configs
+`── data
+ │── aic
+ │-- annotations
+ │ │-- aic_train.json
+ │ |-- aic_val.json
+ │-- ai_challenger_keypoint_train_20170902
+ │ │-- keypoint_train_images_20170902
+ │ │ │-- 0000252aea98840a550dac9a78c476ecb9f47ffa.jpg
+ │ │ │-- 000050f770985ac9653198495ef9b5c82435d49c.jpg
+ │ │ │-- ...
+ `-- ai_challenger_keypoint_validation_20170911
+ │-- keypoint_validation_images_20170911
+ │-- 0002605c53fb92109a3f2de4fc3ce06425c3b61f.jpg
+ │-- 0003b55a2c991223e6d8b4b820045bd49507bf6d.jpg
+ │-- ...
+```
+
+## CrowdPose
+
+
+
+
+CrowdPose (CVPR'2019)
+
+```bibtex
+@article{li2018crowdpose,
+ title={CrowdPose: Efficient Crowded Scenes Pose Estimation and A New Benchmark},
+ author={Li, Jiefeng and Wang, Can and Zhu, Hao and Mao, Yihuan and Fang, Hao-Shu and Lu, Cewu},
+ journal={arXiv preprint arXiv:1812.00324},
+ year={2018}
+}
+```
+
+
+
+For [CrowdPose](https://github.com/Jeff-sjtu/CrowdPose) data, please download from [CrowdPose](https://github.com/Jeff-sjtu/CrowdPose).
+Please download the annotation files and human detection results from [crowdpose_annotations](https://download.openmmlab.com/mmpose/datasets/crowdpose_annotations.tar).
+For top-down approaches, we follow [CrowdPose](https://arxiv.org/abs/1812.00324) to use the [pre-trained weights](https://pjreddie.com/media/files/yolov3.weights) of [YOLOv3](https://github.com/eriklindernoren/PyTorch-YOLOv3) to generate the detected human bounding boxes.
+For model training, we follow [HigherHRNet](https://github.com/HRNet/HigherHRNet-Human-Pose-Estimation) to train models on CrowdPose train/val dataset, and evaluate models on CrowdPose test dataset.
+Download and extract them under $MMPOSE/data, and make them look like this:
+
+```text
+mmpose
+├── mmpose
+├── docs
+├── tests
+├── tools
+├── configs
+`── data
+ │── crowdpose
+ │-- annotations
+ │ │-- mmpose_crowdpose_train.json
+ │ │-- mmpose_crowdpose_val.json
+ │ │-- mmpose_crowdpose_trainval.json
+ │ │-- mmpose_crowdpose_test.json
+ │ │-- det_for_crowd_test_0.1_0.5.json
+ │-- images
+ │-- 100000.jpg
+ │-- 100001.jpg
+ │-- 100002.jpg
+ │-- ...
+```
+
+## OCHuman
+
+
+
+
+OCHuman (CVPR'2019)
+
+```bibtex
+@inproceedings{zhang2019pose2seg,
+ title={Pose2seg: Detection free human instance segmentation},
+ author={Zhang, Song-Hai and Li, Ruilong and Dong, Xin and Rosin, Paul and Cai, Zixi and Han, Xi and Yang, Dingcheng and Huang, Haozhi and Hu, Shi-Min},
+ booktitle={Proceedings of the IEEE conference on computer vision and pattern recognition},
+ pages={889--898},
+ year={2019}
+}
+```
+
+
+
+For [OCHuman](https://github.com/liruilong940607/OCHumanApi) data, please download the images and annotations from [OCHuman](https://github.com/liruilong940607/OCHumanApi),
+Move them under $MMPOSE/data, and make them look like this:
+
+```text
+mmpose
+├── mmpose
+├── docs
+├── tests
+├── tools
+├── configs
+`── data
+ │── ochuman
+ │-- annotations
+ │ │-- ochuman_coco_format_val_range_0.00_1.00.json
+ │ |-- ochuman_coco_format_test_range_0.00_1.00.json
+ |-- images
+ │-- 000001.jpg
+ │-- 000002.jpg
+ │-- 000003.jpg
+ │-- ...
+
+```
+
+## MHP
+
+
+
+
+MHP (ACM MM'2018)
+
+```bibtex
+@inproceedings{zhao2018understanding,
+ title={Understanding humans in crowded scenes: Deep nested adversarial learning and a new benchmark for multi-human parsing},
+ author={Zhao, Jian and Li, Jianshu and Cheng, Yu and Sim, Terence and Yan, Shuicheng and Feng, Jiashi},
+ booktitle={Proceedings of the 26th ACM international conference on Multimedia},
+ pages={792--800},
+ year={2018}
+}
+```
+
+
+
+For [MHP](https://lv-mhp.github.io/dataset) data, please download from [MHP](https://lv-mhp.github.io/dataset).
+Please download the annotation files from [mhp_annotations](https://download.openmmlab.com/mmpose/datasets/mhp_annotations.tar.gz).
+Please download and extract them under $MMPOSE/data, and make them look like this:
+
+```text
+mmpose
+├── mmpose
+├── docs
+├── tests
+├── tools
+├── configs
+`── data
+ │── mhp
+ │-- annotations
+ │ │-- mhp_train.json
+ │ │-- mhp_val.json
+ │
+ `-- train
+ │ │-- images
+ │ │ │-- 1004.jpg
+ │ │ │-- 10050.jpg
+ │ │ │-- ...
+ │
+ `-- val
+ │ │-- images
+ │ │ │-- 10059.jpg
+ │ │ │-- 10068.jpg
+ │ │ │-- ...
+ │
+ `-- test
+ │ │-- images
+ │ │ │-- 1005.jpg
+ │ │ │-- 10052.jpg
+ │ │ │-- ...~~~~
+```
+
+## PoseTrack18
+
+
+
+
+PoseTrack18 (CVPR'2018)
+
+```bibtex
+@inproceedings{andriluka2018posetrack,
+ title={Posetrack: A benchmark for human pose estimation and tracking},
+ author={Andriluka, Mykhaylo and Iqbal, Umar and Insafutdinov, Eldar and Pishchulin, Leonid and Milan, Anton and Gall, Juergen and Schiele, Bernt},
+ booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
+ pages={5167--5176},
+ year={2018}
+}
+```
+
+
+
+For [PoseTrack18](https://posetrack.net/users/download.php) data, please download from [PoseTrack18](https://posetrack.net/users/download.php).
+Please download the annotation files from [posetrack18_annotations](https://download.openmmlab.com/mmpose/datasets/posetrack18_annotations.tar).
+We have merged the video-wise separated official annotation files into two json files (posetrack18_train & posetrack18_val.json). We also generate the [mask files](https://download.openmmlab.com/mmpose/datasets/posetrack18_mask.tar) to speed up training.
+For top-down approaches, we use [MMDetection](https://github.com/open-mmlab/mmdetection) pre-trained [Cascade R-CNN](https://download.openmmlab.com/mmdetection/v2.0/cascade_rcnn/cascade_rcnn_x101_64x4d_fpn_20e_coco/cascade_rcnn_x101_64x4d_fpn_20e_coco_20200509_224357-051557b1.pth) (X-101-64x4d-FPN) to generate the detected human bounding boxes.
+Please download and extract them under $MMPOSE/data, and make them look like this:
+
+```text
+mmpose
+├── mmpose
+├── docs
+├── tests
+├── tools
+├── configs
+`── data
+ │── posetrack18
+ │-- annotations
+ │ │-- posetrack18_train.json
+ │ │-- posetrack18_val.json
+ │ │-- posetrack18_val_human_detections.json
+ │ │-- train
+ │ │ │-- 000001_bonn_train.json
+ │ │ │-- 000002_bonn_train.json
+ │ │ │-- ...
+ │ │-- val
+ │ │ │-- 000342_mpii_test.json
+ │ │ │-- 000522_mpii_test.json
+ │ │ │-- ...
+ │ `-- test
+ │ │-- 000001_mpiinew_test.json
+ │ │-- 000002_mpiinew_test.json
+ │ │-- ...
+ │
+ `-- images
+ │ │-- train
+ │ │ │-- 000001_bonn_train
+ │ │ │ │-- 000000.jpg
+ │ │ │ │-- 000001.jpg
+ │ │ │ │-- ...
+ │ │ │-- ...
+ │ │-- val
+ │ │ │-- 000342_mpii_test
+ │ │ │ │-- 000000.jpg
+ │ │ │ │-- 000001.jpg
+ │ │ │ │-- ...
+ │ │ │-- ...
+ │ `-- test
+ │ │-- 000001_mpiinew_test
+ │ │ │-- 000000.jpg
+ │ │ │-- 000001.jpg
+ │ │ │-- ...
+ │ │-- ...
+ `-- mask
+ │-- train
+ │ │-- 000002_bonn_train
+ │ │ │-- 000000.jpg
+ │ │ │-- 000001.jpg
+ │ │ │-- ...
+ │ │-- ...
+ `-- val
+ │-- 000522_mpii_test
+ │ │-- 000000.jpg
+ │ │-- 000001.jpg
+ │ │-- ...
+ │-- ...
+```
+
+The official evaluation tool for PoseTrack should be installed from GitHub.
+
+```shell
+pip install git+https://github.com/svenkreiss/poseval.git
+```
+
+## sub-JHMDB dataset
+
+
+
+
+RSN (ECCV'2020)
+
+```bibtex
+@misc{cai2020learning,
+ title={Learning Delicate Local Representations for Multi-Person Pose Estimation},
+ author={Yuanhao Cai and Zhicheng Wang and Zhengxiong Luo and Binyi Yin and Angang Du and Haoqian Wang and Xinyu Zhou and Erjin Zhou and Xiangyu Zhang and Jian Sun},
+ year={2020},
+ eprint={2003.04030},
+ archivePrefix={arXiv},
+ primaryClass={cs.CV}
+}
+```
+
+
+
+For [sub-JHMDB](http://jhmdb.is.tue.mpg.de/dataset) data, please download the [images](<(http://files.is.tue.mpg.de/jhmdb/Rename_Images.tar.gz)>) from [JHMDB](http://jhmdb.is.tue.mpg.de/dataset),
+Please download the annotation files from [jhmdb_annotations](https://download.openmmlab.com/mmpose/datasets/jhmdb_annotations.tar).
+Move them under $MMPOSE/data, and make them look like this:
+
+```text
+mmpose
+├── mmpose
+├── docs
+├── tests
+├── tools
+├── configs
+`── data
+ │── jhmdb
+ │-- annotations
+ │ │-- Sub1_train.json
+ │ |-- Sub1_test.json
+ │ │-- Sub2_train.json
+ │ |-- Sub2_test.json
+ │ │-- Sub3_train.json
+ │ |-- Sub3_test.json
+ |-- Rename_Images
+ │-- brush_hair
+ │ │--April_09_brush_hair_u_nm_np1_ba_goo_0
+ | │ │--00001.png
+ | │ │--00002.png
+ │-- catch
+ │-- ...
+
+```
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/tasks/2d_face_keypoint.md b/engine/pose_estimation/third-party/ViTPose/docs/en/tasks/2d_face_keypoint.md
new file mode 100644
index 0000000..fe71500
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/tasks/2d_face_keypoint.md
@@ -0,0 +1,306 @@
+# 2D Face Keypoint Datasets
+
+It is recommended to symlink the dataset root to `$MMPOSE/data`.
+If your folder structure is different, you may need to change the corresponding paths in config files.
+
+MMPose supported datasets:
+
+- [300W](#300w-dataset) \[ [Homepage](https://ibug.doc.ic.ac.uk/resources/300-W/) \]
+- [WFLW](#wflw-dataset) \[ [Homepage](https://wywu.github.io/projects/LAB/WFLW.html) \]
+- [AFLW](#aflw-dataset) \[ [Homepage](https://www.tugraz.at/institute/icg/research/team-bischof/lrs/downloads/aflw/) \]
+- [COFW](#cofw-dataset) \[ [Homepage](http://www.vision.caltech.edu/xpburgos/ICCV13/) \]
+- [COCO-WholeBody-Face](#coco-wholebody-face) \[ [Homepage](https://github.com/jin-s13/COCO-WholeBody/) \]
+
+## 300W Dataset
+
+
+
+
+300W (IMAVIS'2016)
+
+```bibtex
+@article{sagonas2016300,
+ title={300 faces in-the-wild challenge: Database and results},
+ author={Sagonas, Christos and Antonakos, Epameinondas and Tzimiropoulos, Georgios and Zafeiriou, Stefanos and Pantic, Maja},
+ journal={Image and vision computing},
+ volume={47},
+ pages={3--18},
+ year={2016},
+ publisher={Elsevier}
+}
+```
+
+
+
+For 300W data, please download images from [300W Dataset](https://ibug.doc.ic.ac.uk/resources/300-W/).
+Please download the annotation files from [300w_annotations](https://download.openmmlab.com/mmpose/datasets/300w_annotations.tar).
+Extract them under {MMPose}/data, and make them look like this:
+
+```text
+mmpose
+├── mmpose
+├── docs
+├── tests
+├── tools
+├── configs
+`── data
+ │── 300w
+ |── annotations
+ | |── face_landmarks_300w_train.json
+ | |── face_landmarks_300w_valid.json
+ | |── face_landmarks_300w_valid_common.json
+ | |── face_landmarks_300w_valid_challenge.json
+ | |── face_landmarks_300w_test.json
+ `── images
+ |── afw
+ | |── 1051618982_1.jpg
+ | |── 111076519_1.jpg
+ | ...
+ |── helen
+ | |── trainset
+ | | |── 100032540_1.jpg
+ | | |── 100040721_1.jpg
+ | | ...
+ | |── testset
+ | | |── 296814969_3.jpg
+ | | |── 2968560214_1.jpg
+ | | ...
+ |── ibug
+ | |── image_003_1.jpg
+ | |── image_004_1.jpg
+ | ...
+ |── lfpw
+ | |── trainset
+ | | |── image_0001.png
+ | | |── image_0002.png
+ | | ...
+ | |── testset
+ | | |── image_0001.png
+ | | |── image_0002.png
+ | | ...
+ `── Test
+ |── 01_Indoor
+ | |── indoor_001.png
+ | |── indoor_002.png
+ | ...
+ `── 02_Outdoor
+ |── outdoor_001.png
+ |── outdoor_002.png
+ ...
+```
+
+## WFLW Dataset
+
+
+
+
+WFLW (CVPR'2018)
+
+```bibtex
+@inproceedings{wu2018look,
+ title={Look at boundary: A boundary-aware face alignment algorithm},
+ author={Wu, Wayne and Qian, Chen and Yang, Shuo and Wang, Quan and Cai, Yici and Zhou, Qiang},
+ booktitle={Proceedings of the IEEE conference on computer vision and pattern recognition},
+ pages={2129--2138},
+ year={2018}
+}
+```
+
+
+
+For WFLW data, please download images from [WFLW Dataset](https://wywu.github.io/projects/LAB/WFLW.html).
+Please download the annotation files from [wflw_annotations](https://download.openmmlab.com/mmpose/datasets/wflw_annotations.tar).
+Extract them under {MMPose}/data, and make them look like this:
+
+```text
+mmpose
+├── mmpose
+├── docs
+├── tests
+├── tools
+├── configs
+`── data
+ │── wflw
+ |── annotations
+ | |── face_landmarks_wflw_train.json
+ | |── face_landmarks_wflw_test.json
+ | |── face_landmarks_wflw_test_blur.json
+ | |── face_landmarks_wflw_test_occlusion.json
+ | |── face_landmarks_wflw_test_expression.json
+ | |── face_landmarks_wflw_test_largepose.json
+ | |── face_landmarks_wflw_test_illumination.json
+ | |── face_landmarks_wflw_test_makeup.json
+ |
+ `── images
+ |── 0--Parade
+ | |── 0_Parade_marchingband_1_1015.jpg
+ | |── 0_Parade_marchingband_1_1031.jpg
+ | ...
+ |── 1--Handshaking
+ | |── 1_Handshaking_Handshaking_1_105.jpg
+ | |── 1_Handshaking_Handshaking_1_107.jpg
+ | ...
+ ...
+```
+
+## AFLW Dataset
+
+
+
+
+AFLW (ICCVW'2011)
+
+```bibtex
+@inproceedings{koestinger2011annotated,
+ title={Annotated facial landmarks in the wild: A large-scale, real-world database for facial landmark localization},
+ author={Koestinger, Martin and Wohlhart, Paul and Roth, Peter M and Bischof, Horst},
+ booktitle={2011 IEEE international conference on computer vision workshops (ICCV workshops)},
+ pages={2144--2151},
+ year={2011},
+ organization={IEEE}
+}
+```
+
+
+
+For AFLW data, please download images from [AFLW Dataset](https://www.tugraz.at/institute/icg/research/team-bischof/lrs/downloads/aflw/).
+Please download the annotation files from [aflw_annotations](https://download.openmmlab.com/mmpose/datasets/aflw_annotations.tar).
+Extract them under {MMPose}/data, and make them look like this:
+
+```text
+mmpose
+├── mmpose
+├── docs
+├── tests
+├── tools
+├── configs
+`── data
+ │── aflw
+ |── annotations
+ | |── face_landmarks_aflw_train.json
+ | |── face_landmarks_aflw_test_frontal.json
+ | |── face_landmarks_aflw_test.json
+ `── images
+ |── flickr
+ |── 0
+ | |── image00002.jpg
+ | |── image00013.jpg
+ | ...
+ |── 2
+ | |── image00004.jpg
+ | |── image00006.jpg
+ | ...
+ `── 3
+ |── image00032.jpg
+ |── image00035.jpg
+ ...
+```
+
+## COFW Dataset
+
+
+
+
+COFW (ICCV'2013)
+
+```bibtex
+@inproceedings{burgos2013robust,
+ title={Robust face landmark estimation under occlusion},
+ author={Burgos-Artizzu, Xavier P and Perona, Pietro and Doll{\'a}r, Piotr},
+ booktitle={Proceedings of the IEEE international conference on computer vision},
+ pages={1513--1520},
+ year={2013}
+}
+```
+
+
+
+For COFW data, please download from [COFW Dataset (Color Images)](http://www.vision.caltech.edu/xpburgos/ICCV13/Data/COFW_color.zip).
+Move `COFW_train_color.mat` and `COFW_test_color.mat` to `data/cofw/` and make them look like:
+
+```text
+mmpose
+├── mmpose
+├── docs
+├── tests
+├── tools
+├── configs
+`── data
+ │── cofw
+ |── COFW_train_color.mat
+ |── COFW_test_color.mat
+```
+
+Run the following script under `{MMPose}/data`
+
+`python tools/dataset/parse_cofw_dataset.py`
+
+And you will get
+
+```text
+mmpose
+├── mmpose
+├── docs
+├── tests
+├── tools
+├── configs
+`── data
+ │── cofw
+ |── COFW_train_color.mat
+ |── COFW_test_color.mat
+ |── annotations
+ | |── cofw_train.json
+ | |── cofw_test.json
+ |── images
+ |── 000001.jpg
+ |── 000002.jpg
+```
+
+## COCO-WholeBody (Face)
+
+[DATASET]
+
+```bibtex
+@inproceedings{jin2020whole,
+ title={Whole-Body Human Pose Estimation in the Wild},
+ author={Jin, Sheng and Xu, Lumin and Xu, Jin and Wang, Can and Liu, Wentao and Qian, Chen and Ouyang, Wanli and Luo, Ping},
+ booktitle={Proceedings of the European Conference on Computer Vision (ECCV)},
+ year={2020}
+}
+```
+
+For [COCO-WholeBody](https://github.com/jin-s13/COCO-WholeBody/) dataset, images can be downloaded from [COCO download](http://cocodataset.org/#download), 2017 Train/Val is needed for COCO keypoints training and validation.
+Download COCO-WholeBody annotations for COCO-WholeBody annotations for [Train](https://drive.google.com/file/d/1thErEToRbmM9uLNi1JXXfOsaS5VK2FXf/view?usp=sharing) / [Validation](https://drive.google.com/file/d/1N6VgwKnj8DeyGXCvp1eYgNbRmw6jdfrb/view?usp=sharing) (Google Drive).
+Download person detection result of COCO val2017 from [OneDrive](https://1drv.ms/f/s!AhIXJn_J-blWzzDXoz5BeFl8sWM-) or [GoogleDrive](https://drive.google.com/drive/folders/1fRUDNUDxe9fjqcRZ2bnF_TKMlO0nB_dk?usp=sharing).
+Download and extract them under $MMPOSE/data, and make them look like this:
+
+```text
+mmpose
+├── mmpose
+├── docs
+├── tests
+├── tools
+├── configs
+`── data
+ │── coco
+ │-- annotations
+ │ │-- coco_wholebody_train_v1.0.json
+ │ |-- coco_wholebody_val_v1.0.json
+ |-- person_detection_results
+ | |-- COCO_val2017_detections_AP_H_56_person.json
+ │-- train2017
+ │ │-- 000000000009.jpg
+ │ │-- 000000000025.jpg
+ │ │-- 000000000030.jpg
+ │ │-- ...
+ `-- val2017
+ │-- 000000000139.jpg
+ │-- 000000000285.jpg
+ │-- 000000000632.jpg
+ │-- ...
+
+```
+
+Please also install the latest version of [Extended COCO API](https://github.com/jin-s13/xtcocoapi) to support COCO-WholeBody evaluation:
+
+`pip install xtcocotools`
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/tasks/2d_fashion_landmark.md b/engine/pose_estimation/third-party/ViTPose/docs/en/tasks/2d_fashion_landmark.md
new file mode 100644
index 0000000..c0eb2c8
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/tasks/2d_fashion_landmark.md
@@ -0,0 +1,76 @@
+# 2D Fashion Landmark Dataset
+
+It is recommended to symlink the dataset root to `$MMPOSE/data`.
+If your folder structure is different, you may need to change the corresponding paths in config files.
+
+MMPose supported datasets:
+
+- [DeepFashion](#deepfashion) \[ [Homepage](http://mmlab.ie.cuhk.edu.hk/projects/DeepFashion/LandmarkDetection.html) \]
+
+## DeepFashion (Fashion Landmark Detection, FLD)
+
+
+
+
+DeepFashion (CVPR'2016)
+
+```bibtex
+@inproceedings{liuLQWTcvpr16DeepFashion,
+ author = {Liu, Ziwei and Luo, Ping and Qiu, Shi and Wang, Xiaogang and Tang, Xiaoou},
+ title = {DeepFashion: Powering Robust Clothes Recognition and Retrieval with Rich Annotations},
+ booktitle = {Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
+ month = {June},
+ year = {2016}
+}
+```
+
+
+
+
+
+
+DeepFashion (ECCV'2016)
+
+```bibtex
+@inproceedings{liuYLWTeccv16FashionLandmark,
+ author = {Liu, Ziwei and Yan, Sijie and Luo, Ping and Wang, Xiaogang and Tang, Xiaoou},
+ title = {Fashion Landmark Detection in the Wild},
+ booktitle = {European Conference on Computer Vision (ECCV)},
+ month = {October},
+ year = {2016}
+ }
+```
+
+
+
+For [DeepFashion](http://mmlab.ie.cuhk.edu.hk/projects/DeepFashion/LandmarkDetection.html) dataset, images can be downloaded from [download](http://mmlab.ie.cuhk.edu.hk/projects/DeepFashion/LandmarkDetection.html).
+Please download the annotation files from [fld_annotations](https://download.openmmlab.com/mmpose/datasets/fld_annotations.tar).
+Extract them under {MMPose}/data, and make them look like this:
+
+```text
+mmpose
+├── mmpose
+├── docs
+├── tests
+├── tools
+├── configs
+`── data
+ │── fld
+ │-- annotations
+ │ │-- fld_upper_train.json
+ │ |-- fld_upper_val.json
+ │ |-- fld_upper_test.json
+ │ │-- fld_lower_train.json
+ │ |-- fld_lower_val.json
+ │ |-- fld_lower_test.json
+ │ │-- fld_full_train.json
+ │ |-- fld_full_val.json
+ │ |-- fld_full_test.json
+ │-- img
+ │ │-- img_00000001.jpg
+ │ │-- img_00000002.jpg
+ │ │-- img_00000003.jpg
+ │ │-- img_00000004.jpg
+ │ │-- img_00000005.jpg
+ │ │-- ...
+```
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/tasks/2d_hand_keypoint.md b/engine/pose_estimation/third-party/ViTPose/docs/en/tasks/2d_hand_keypoint.md
new file mode 100644
index 0000000..20f93d4
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/tasks/2d_hand_keypoint.md
@@ -0,0 +1,319 @@
+# 2D Hand Keypoint Datasets
+
+It is recommended to symlink the dataset root to `$MMPOSE/data`.
+If your folder structure is different, you may need to change the corresponding paths in config files.
+
+MMPose supported datasets:
+
+- [OneHand10K](#onehand10k) \[ [Homepage](https://www.yangangwang.com/papers/WANG-MCC-2018-10.html) \]
+- [FreiHand](#freihand-dataset) \[ [Homepage](https://lmb.informatik.uni-freiburg.de/projects/freihand/) \]
+- [CMU Panoptic HandDB](#cmu-panoptic-handdb) \[ [Homepage](http://domedb.perception.cs.cmu.edu/handdb.html) \]
+- [InterHand2.6M](#interhand26m) \[ [Homepage](https://mks0601.github.io/InterHand2.6M/) \]
+- [RHD](#rhd-dataset) \[ [Homepage](https://lmb.informatik.uni-freiburg.de/resources/datasets/RenderedHandposeDataset.en.html) \]
+- [COCO-WholeBody-Hand](#coco-wholebody-hand) \[ [Homepage](https://github.com/jin-s13/COCO-WholeBody/) \]
+
+## OneHand10K
+
+
+
+
+OneHand10K (TCSVT'2019)
+
+```bibtex
+@article{wang2018mask,
+ title={Mask-pose cascaded cnn for 2d hand pose estimation from single color image},
+ author={Wang, Yangang and Peng, Cong and Liu, Yebin},
+ journal={IEEE Transactions on Circuits and Systems for Video Technology},
+ volume={29},
+ number={11},
+ pages={3258--3268},
+ year={2018},
+ publisher={IEEE}
+}
+```
+
+
+
+For [OneHand10K](https://www.yangangwang.com/papers/WANG-MCC-2018-10.html) data, please download from [OneHand10K Dataset](https://www.yangangwang.com/papers/WANG-MCC-2018-10.html).
+Please download the annotation files from [onehand10k_annotations](https://download.openmmlab.com/mmpose/datasets/onehand10k_annotations.tar).
+Extract them under {MMPose}/data, and make them look like this:
+
+```text
+mmpose
+├── mmpose
+├── docs
+├── tests
+├── tools
+├── configs
+`── data
+ │── onehand10k
+ |── annotations
+ | |── onehand10k_train.json
+ | |── onehand10k_test.json
+ `── Train
+ | |── source
+ | |── 0.jpg
+ | |── 1.jpg
+ | ...
+ `── Test
+ |── source
+ |── 0.jpg
+ |── 1.jpg
+
+```
+
+## FreiHAND Dataset
+
+
+
+
+FreiHand (ICCV'2019)
+
+```bibtex
+@inproceedings{zimmermann2019freihand,
+ title={Freihand: A dataset for markerless capture of hand pose and shape from single rgb images},
+ author={Zimmermann, Christian and Ceylan, Duygu and Yang, Jimei and Russell, Bryan and Argus, Max and Brox, Thomas},
+ booktitle={Proceedings of the IEEE International Conference on Computer Vision},
+ pages={813--822},
+ year={2019}
+}
+```
+
+
+
+For [FreiHAND](https://lmb.informatik.uni-freiburg.de/projects/freihand/) data, please download from [FreiHand Dataset](https://lmb.informatik.uni-freiburg.de/resources/datasets/FreihandDataset.en.html).
+Since the official dataset does not provide validation set, we randomly split the training data into 8:1:1 for train/val/test.
+Please download the annotation files from [freihand_annotations](https://download.openmmlab.com/mmpose/datasets/frei_annotations.tar).
+Extract them under {MMPose}/data, and make them look like this:
+
+```text
+mmpose
+├── mmpose
+├── docs
+├── tests
+├── tools
+├── configs
+`── data
+ │── freihand
+ |── annotations
+ | |── freihand_train.json
+ | |── freihand_val.json
+ | |── freihand_test.json
+ `── training
+ |── rgb
+ | |── 00000000.jpg
+ | |── 00000001.jpg
+ | ...
+ |── mask
+ |── 00000000.jpg
+ |── 00000001.jpg
+ ...
+```
+
+## CMU Panoptic HandDB
+
+
+
+
+CMU Panoptic HandDB (CVPR'2017)
+
+```bibtex
+@inproceedings{simon2017hand,
+ title={Hand keypoint detection in single images using multiview bootstrapping},
+ author={Simon, Tomas and Joo, Hanbyul and Matthews, Iain and Sheikh, Yaser},
+ booktitle={Proceedings of the IEEE conference on Computer Vision and Pattern Recognition},
+ pages={1145--1153},
+ year={2017}
+}
+```
+
+
+
+For [CMU Panoptic HandDB](http://domedb.perception.cs.cmu.edu/handdb.html), please download from [CMU Panoptic HandDB](http://domedb.perception.cs.cmu.edu/handdb.html).
+Following [Simon et al](https://arxiv.org/abs/1704.07809), panoptic images (hand143_panopticdb) and MPII & NZSL training sets (manual_train) are used for training, while MPII & NZSL test set (manual_test) for testing.
+Please download the annotation files from [panoptic_annotations](https://download.openmmlab.com/mmpose/datasets/panoptic_annotations.tar).
+Extract them under {MMPose}/data, and make them look like this:
+
+```text
+mmpose
+├── mmpose
+├── docs
+├── tests
+├── tools
+├── configs
+`── data
+ │── panoptic
+ |── annotations
+ | |── panoptic_train.json
+ | |── panoptic_test.json
+ |
+ `── hand143_panopticdb
+ | |── imgs
+ | | |── 00000000.jpg
+ | | |── 00000001.jpg
+ | | ...
+ |
+ `── hand_labels
+ |── manual_train
+ | |── 000015774_01_l.jpg
+ | |── 000015774_01_r.jpg
+ | ...
+ |
+ `── manual_test
+ |── 000648952_02_l.jpg
+ |── 000835470_01_l.jpg
+ ...
+```
+
+## InterHand2.6M
+
+
+
+
+InterHand2.6M (ECCV'2020)
+
+```bibtex
+@InProceedings{Moon_2020_ECCV_InterHand2.6M,
+author = {Moon, Gyeongsik and Yu, Shoou-I and Wen, He and Shiratori, Takaaki and Lee, Kyoung Mu},
+title = {InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Image},
+booktitle = {European Conference on Computer Vision (ECCV)},
+year = {2020}
+}
+```
+
+
+
+For [InterHand2.6M](https://mks0601.github.io/InterHand2.6M/), please download from [InterHand2.6M](https://mks0601.github.io/InterHand2.6M/).
+Please download the annotation files from [annotations](https://drive.google.com/drive/folders/1pWXhdfaka-J0fSAze0MsajN0VpZ8e8tO).
+Extract them under {MMPose}/data, and make them look like this:
+
+```text
+mmpose
+├── mmpose
+├── docs
+├── tests
+├── tools
+├── configs
+`── data
+ │── interhand2.6m
+ |── annotations
+ | |── all
+ | |── human_annot
+ | |── machine_annot
+ | |── skeleton.txt
+ | |── subject.txt
+ |
+ `── images
+ | |── train
+ | | |-- Capture0 ~ Capture26
+ | |── val
+ | | |-- Capture0
+ | |── test
+ | | |-- Capture0 ~ Capture7
+```
+
+## RHD Dataset
+
+
+
+
+RHD (ICCV'2017)
+
+```bibtex
+@TechReport{zb2017hand,
+ author={Christian Zimmermann and Thomas Brox},
+ title={Learning to Estimate 3D Hand Pose from Single RGB Images},
+ institution={arXiv:1705.01389},
+ year={2017},
+ note="https://arxiv.org/abs/1705.01389",
+ url="https://lmb.informatik.uni-freiburg.de/projects/hand3d/"
+}
+```
+
+
+
+For [RHD Dataset](https://lmb.informatik.uni-freiburg.de/resources/datasets/RenderedHandposeDataset.en.html), please download from [RHD Dataset](https://lmb.informatik.uni-freiburg.de/resources/datasets/RenderedHandposeDataset.en.html).
+Please download the annotation files from [rhd_annotations](https://download.openmmlab.com/mmpose/datasets/rhd_annotations.zip).
+Extract them under {MMPose}/data, and make them look like this:
+
+```text
+mmpose
+├── mmpose
+├── docs
+├── tests
+├── tools
+├── configs
+`── data
+ │── rhd
+ |── annotations
+ | |── rhd_train.json
+ | |── rhd_test.json
+ `── training
+ | |── color
+ | | |── 00000.jpg
+ | | |── 00001.jpg
+ | |── depth
+ | | |── 00000.jpg
+ | | |── 00001.jpg
+ | |── mask
+ | | |── 00000.jpg
+ | | |── 00001.jpg
+ `── evaluation
+ | |── color
+ | | |── 00000.jpg
+ | | |── 00001.jpg
+ | |── depth
+ | | |── 00000.jpg
+ | | |── 00001.jpg
+ | |── mask
+ | | |── 00000.jpg
+ | | |── 00001.jpg
+```
+
+## COCO-WholeBody (Hand)
+
+[DATASET]
+
+```bibtex
+@inproceedings{jin2020whole,
+ title={Whole-Body Human Pose Estimation in the Wild},
+ author={Jin, Sheng and Xu, Lumin and Xu, Jin and Wang, Can and Liu, Wentao and Qian, Chen and Ouyang, Wanli and Luo, Ping},
+ booktitle={Proceedings of the European Conference on Computer Vision (ECCV)},
+ year={2020}
+}
+```
+
+For [COCO-WholeBody](https://github.com/jin-s13/COCO-WholeBody/) dataset, images can be downloaded from [COCO download](http://cocodataset.org/#download), 2017 Train/Val is needed for COCO keypoints training and validation.
+Download COCO-WholeBody annotations for COCO-WholeBody annotations for [Train](https://drive.google.com/file/d/1thErEToRbmM9uLNi1JXXfOsaS5VK2FXf/view?usp=sharing) / [Validation](https://drive.google.com/file/d/1N6VgwKnj8DeyGXCvp1eYgNbRmw6jdfrb/view?usp=sharing) (Google Drive).
+Download person detection result of COCO val2017 from [OneDrive](https://1drv.ms/f/s!AhIXJn_J-blWzzDXoz5BeFl8sWM-) or [GoogleDrive](https://drive.google.com/drive/folders/1fRUDNUDxe9fjqcRZ2bnF_TKMlO0nB_dk?usp=sharing).
+Download and extract them under $MMPOSE/data, and make them look like this:
+
+```text
+mmpose
+├── mmpose
+├── docs
+├── tests
+├── tools
+├── configs
+`── data
+ │── coco
+ │-- annotations
+ │ │-- coco_wholebody_train_v1.0.json
+ │ |-- coco_wholebody_val_v1.0.json
+ |-- person_detection_results
+ | |-- COCO_val2017_detections_AP_H_56_person.json
+ │-- train2017
+ │ │-- 000000000009.jpg
+ │ │-- 000000000025.jpg
+ │ │-- 000000000030.jpg
+ │ │-- ...
+ `-- val2017
+ │-- 000000000139.jpg
+ │-- 000000000285.jpg
+ │-- 000000000632.jpg
+ │-- ...
+```
+
+Please also install the latest version of [Extended COCO API](https://github.com/jin-s13/xtcocoapi) to support COCO-WholeBody evaluation:
+
+`pip install xtcocotools`
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/tasks/2d_wholebody_keypoint.md b/engine/pose_estimation/third-party/ViTPose/docs/en/tasks/2d_wholebody_keypoint.md
new file mode 100644
index 0000000..e3d573f
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/tasks/2d_wholebody_keypoint.md
@@ -0,0 +1,125 @@
+# 2D Wholebody Keypoint Datasets
+
+It is recommended to symlink the dataset root to `$MMPOSE/data`.
+If your folder structure is different, you may need to change the corresponding paths in config files.
+
+MMPose supported datasets:
+
+- [COCO-WholeBody](#coco-wholebody) \[ [Homepage](https://github.com/jin-s13/COCO-WholeBody/) \]
+- [Halpe](#halpe) \[ [Homepage](https://github.com/Fang-Haoshu/Halpe-FullBody/) \]
+
+## COCO-WholeBody
+
+
+
+
+COCO-WholeBody (ECCV'2020)
+
+```bibtex
+@inproceedings{jin2020whole,
+ title={Whole-Body Human Pose Estimation in the Wild},
+ author={Jin, Sheng and Xu, Lumin and Xu, Jin and Wang, Can and Liu, Wentao and Qian, Chen and Ouyang, Wanli and Luo, Ping},
+ booktitle={Proceedings of the European Conference on Computer Vision (ECCV)},
+ year={2020}
+}
+```
+
+
+
+For [COCO-WholeBody](https://github.com/jin-s13/COCO-WholeBody/) dataset, images can be downloaded from [COCO download](http://cocodataset.org/#download), 2017 Train/Val is needed for COCO keypoints training and validation.
+Download COCO-WholeBody annotations for COCO-WholeBody annotations for [Train](https://drive.google.com/file/d/1thErEToRbmM9uLNi1JXXfOsaS5VK2FXf/view?usp=sharing) / [Validation](https://drive.google.com/file/d/1N6VgwKnj8DeyGXCvp1eYgNbRmw6jdfrb/view?usp=sharing) (Google Drive).
+Download person detection result of COCO val2017 from [OneDrive](https://1drv.ms/f/s!AhIXJn_J-blWzzDXoz5BeFl8sWM-) or [GoogleDrive](https://drive.google.com/drive/folders/1fRUDNUDxe9fjqcRZ2bnF_TKMlO0nB_dk?usp=sharing).
+Download and extract them under $MMPOSE/data, and make them look like this:
+
+```text
+mmpose
+├── mmpose
+├── docs
+├── tests
+├── tools
+├── configs
+`── data
+ │── coco
+ │-- annotations
+ │ │-- coco_wholebody_train_v1.0.json
+ │ |-- coco_wholebody_val_v1.0.json
+ |-- person_detection_results
+ | |-- COCO_val2017_detections_AP_H_56_person.json
+ │-- train2017
+ │ │-- 000000000009.jpg
+ │ │-- 000000000025.jpg
+ │ │-- 000000000030.jpg
+ │ │-- ...
+ `-- val2017
+ │-- 000000000139.jpg
+ │-- 000000000285.jpg
+ │-- 000000000632.jpg
+ │-- ...
+
+```
+
+Please also install the latest version of [Extended COCO API](https://github.com/jin-s13/xtcocoapi) (version>=1.5) to support COCO-WholeBody evaluation:
+
+`pip install xtcocotools`
+
+## Halpe
+
+
+
+
+Halpe (CVPR'2020)
+
+```bibtex
+@inproceedings{li2020pastanet,
+ title={PaStaNet: Toward Human Activity Knowledge Engine},
+ author={Li, Yong-Lu and Xu, Liang and Liu, Xinpeng and Huang, Xijie and Xu, Yue and Wang, Shiyi and Fang, Hao-Shu and Ma, Ze and Chen, Mingyang and Lu, Cewu},
+ booktitle={CVPR},
+ year={2020}
+}
+```
+
+
+
+For [Halpe](https://github.com/Fang-Haoshu/Halpe-FullBody/) dataset, please download images and annotations from [Halpe download](https://github.com/Fang-Haoshu/Halpe-FullBody).
+The images of the training set are from [HICO-Det](https://drive.google.com/open?id=1QZcJmGVlF9f4h-XLWe9Gkmnmj2z1gSnk) and those of the validation set are from [COCO](http://images.cocodataset.org/zips/val2017.zip).
+Download person detection result of COCO val2017 from [OneDrive](https://1drv.ms/f/s!AhIXJn_J-blWzzDXoz5BeFl8sWM-) or [GoogleDrive](https://drive.google.com/drive/folders/1fRUDNUDxe9fjqcRZ2bnF_TKMlO0nB_dk?usp=sharing).
+Download and extract them under $MMPOSE/data, and make them look like this:
+
+```text
+mmpose
+├── mmpose
+├── docs
+├── tests
+├── tools
+├── configs
+`── data
+ │── halpe
+ │-- annotations
+ │ │-- halpe_train_v1.json
+ │ |-- halpe_val_v1.json
+ |-- person_detection_results
+ | |-- COCO_val2017_detections_AP_H_56_person.json
+ │-- hico_20160224_det
+ │ │-- anno_bbox.mat
+ │ │-- anno.mat
+ │ │-- README
+ │ │-- images
+ │ │ │-- train2015
+ │ │ │ │-- HICO_train2015_00000001.jpg
+ │ │ │ │-- HICO_train2015_00000002.jpg
+ │ │ │ │-- HICO_train2015_00000003.jpg
+ │ │ │ │-- ...
+ │ │ │-- test2015
+ │ │-- tools
+ │ │-- ...
+ `-- val2017
+ │-- 000000000139.jpg
+ │-- 000000000285.jpg
+ │-- 000000000632.jpg
+ │-- ...
+
+```
+
+Please also install the latest version of [Extended COCO API](https://github.com/jin-s13/xtcocoapi) (version>=1.5) to support Halpe evaluation:
+
+`pip install xtcocotools`
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/tasks/3d_body_keypoint.md b/engine/pose_estimation/third-party/ViTPose/docs/en/tasks/3d_body_keypoint.md
new file mode 100644
index 0000000..c5ca2a1
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/tasks/3d_body_keypoint.md
@@ -0,0 +1,120 @@
+# 3D Body Keypoint Datasets
+
+It is recommended to symlink the dataset root to `$MMPOSE/data`.
+If your folder structure is different, you may need to change the corresponding paths in config files.
+
+MMPose supported datasets:
+
+- [Human3.6M](#human36m) \[ [Homepage](http://vision.imar.ro/human3.6m/description.php) \]
+- [CMU Panoptic](#cmu-panoptic) \[ [Homepage](http://domedb.perception.cs.cmu.edu/) \]
+
+## Human3.6M
+
+
+
+
+Human3.6M (TPAMI'2014)
+
+```bibtex
+@article{h36m_pami,
+ author = {Ionescu, Catalin and Papava, Dragos and Olaru, Vlad and Sminchisescu, Cristian},
+ title = {Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments},
+ journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
+ publisher = {IEEE Computer Society},
+ volume = {36},
+ number = {7},
+ pages = {1325-1339},
+ month = {jul},
+ year = {2014}
+}
+```
+
+
+
+For [Human3.6M](http://vision.imar.ro/human3.6m/description.php), please download from the official website and run the [preprocessing script](/tools/dataset/preprocess_h36m.py), which will extract camera parameters and pose annotations at full framerate (50 FPS) and downsampled framerate (10 FPS). The processed data should have the following structure:
+
+```text
+mmpose
+├── mmpose
+├── docs
+├── tests
+├── tools
+├── configs
+`── data
+ ├── h36m
+ ├── annotation_body3d
+ | ├── cameras.pkl
+ | ├── fps50
+ | | ├── h36m_test.npz
+ | | ├── h36m_train.npz
+ | | ├── joint2d_rel_stats.pkl
+ | | ├── joint2d_stats.pkl
+ | | ├── joint3d_rel_stats.pkl
+ | | `── joint3d_stats.pkl
+ | `── fps10
+ | ├── h36m_test.npz
+ | ├── h36m_train.npz
+ | ├── joint2d_rel_stats.pkl
+ | ├── joint2d_stats.pkl
+ | ├── joint3d_rel_stats.pkl
+ | `── joint3d_stats.pkl
+ `── images
+ ├── S1
+ | ├── S1_Directions_1.54138969
+ | | ├── S1_Directions_1.54138969_00001.jpg
+ | | ├── S1_Directions_1.54138969_00002.jpg
+ | | ├── ...
+ | ├── ...
+ ├── S5
+ ├── S6
+ ├── S7
+ ├── S8
+ ├── S9
+ `── S11
+```
+
+Please note that Human3.6M dataset is also used in the [3D_body_mesh](/docs/en/tasks/3d_body_mesh.md) task, where different schemes for data preprocessing and organizing are adopted.
+
+## CMU Panoptic
+
+
+CMU Panoptic (ICCV'2015)
+
+```bibtex
+@Article = {joo_iccv_2015,
+author = {Hanbyul Joo, Hao Liu, Lei Tan, Lin Gui, Bart Nabbe, Iain Matthews, Takeo Kanade, Shohei Nobuhara, and Yaser Sheikh},
+title = {Panoptic Studio: A Massively Multiview System for Social Motion Capture},
+booktitle = {ICCV},
+year = {2015}
+}
+```
+
+
+
+Please follow [voxelpose-pytorch](https://github.com/microsoft/voxelpose-pytorch) to prepare this dataset.
+
+1. Download the dataset by following the instructions in [panoptic-toolbox](https://github.com/CMU-Perceptual-Computing-Lab/panoptic-toolbox) and extract them under `$MMPOSE/data/panoptic`.
+
+2. Only download those sequences that are needed. You can also just download a subset of camera views by specifying the number of views (HD_Video_Number) and changing the camera order in `./scripts/getData.sh`. The used sequences and camera views can be found in [VoxelPose](https://arxiv.org/abs/2004.06239). Note that the sequence "160906_band3" might not be available due to errors on the server of CMU Panoptic.
+
+3. Note that we only use HD videos, calibration data, and 3D Body Keypoint in the codes. You can comment out other irrelevant codes such as downloading 3D Face data in `./scripts/getData.sh`.
+
+The directory tree should be like this:
+
+```text
+mmpose
+├── mmpose
+├── docs
+├── tests
+├── tools
+├── configs
+`── data
+ ├── panoptic
+ ├── 16060224_haggling1
+ | | ├── hdImgs
+ | | ├── hdvideos
+ | | ├── hdPose3d_stage1_coco19
+ | | ├── calibration_160224_haggling1.json
+ ├── 160226_haggling1
+ ├── ...
+```
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/tasks/3d_body_mesh.md b/engine/pose_estimation/third-party/ViTPose/docs/en/tasks/3d_body_mesh.md
new file mode 100644
index 0000000..aced63c
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/tasks/3d_body_mesh.md
@@ -0,0 +1,342 @@
+# 3D Body Mesh Recovery Datasets
+
+It is recommended to symlink the dataset root to `$MMPOSE/data`.
+If your folder structure is different, you may need to change the corresponding paths in config files.
+
+To achieve high-quality human mesh estimation, we use multiple datasets for training.
+The following items should be prepared for human mesh training:
+
+
+
+- [3D Body Mesh Recovery Datasets](#3d-body-mesh-recovery-datasets)
+ - [Notes](#notes)
+ - [Annotation Files for Human Mesh Estimation](#annotation-files-for-human-mesh-estimation)
+ - [SMPL Model](#smpl-model)
+ - [COCO](#coco)
+ - [Human3.6M](#human36m)
+ - [MPI-INF-3DHP](#mpi-inf-3dhp)
+ - [LSP](#lsp)
+ - [LSPET](#lspet)
+ - [CMU MoShed Data](#cmu-moshed-data)
+
+
+
+## Notes
+
+### Annotation Files for Human Mesh Estimation
+
+For human mesh estimation, we use multiple datasets for training.
+The annotation of different datasets are preprocessed to the same format. Please
+follow the [preprocess procedure](https://github.com/nkolot/SPIN/tree/master/datasets/preprocess)
+of SPIN to generate the annotation files or download the processed files from
+[here](https://download.openmmlab.com/mmpose/datasets/mesh_annotation_files.zip),
+and make it look like this:
+
+```text
+mmpose
+├── mmpose
+├── docs
+├── tests
+├── tools
+├── configs
+`── data
+ │── mesh_annotation_files
+ ├── coco_2014_train.npz
+ ├── h36m_valid_protocol1.npz
+ ├── h36m_valid_protocol2.npz
+ ├── hr-lspet_train.npz
+ ├── lsp_dataset_original_train.npz
+ ├── mpi_inf_3dhp_train.npz
+ └── mpii_train.npz
+```
+
+### SMPL Model
+
+```bibtex
+@article{loper2015smpl,
+ title={SMPL: A skinned multi-person linear model},
+ author={Loper, Matthew and Mahmood, Naureen and Romero, Javier and Pons-Moll, Gerard and Black, Michael J},
+ journal={ACM transactions on graphics (TOG)},
+ volume={34},
+ number={6},
+ pages={1--16},
+ year={2015},
+ publisher={ACM New York, NY, USA}
+}
+```
+
+For human mesh estimation, SMPL model is used to generate the human mesh.
+Please download the [gender neutral SMPL model](http://smplify.is.tue.mpg.de/),
+[joints regressor](https://download.openmmlab.com/mmpose/datasets/joints_regressor_cmr.npy)
+and [mean parameters](https://download.openmmlab.com/mmpose/datasets/smpl_mean_params.npz)
+under `$MMPOSE/models/smpl`, and make it look like this:
+
+```text
+mmpose
+├── mmpose
+├── ...
+├── models
+ │── smpl
+ ├── joints_regressor_cmr.npy
+ ├── smpl_mean_params.npz
+ └── SMPL_NEUTRAL.pkl
+```
+
+## COCO
+
+
+
+
+COCO (ECCV'2014)
+
+```bibtex
+@inproceedings{lin2014microsoft,
+ title={Microsoft coco: Common objects in context},
+ author={Lin, Tsung-Yi and Maire, Michael and Belongie, Serge and Hays, James and Perona, Pietro and Ramanan, Deva and Doll{\'a}r, Piotr and Zitnick, C Lawrence},
+ booktitle={European conference on computer vision},
+ pages={740--755},
+ year={2014},
+ organization={Springer}
+}
+```
+
+
+
+For [COCO](http://cocodataset.org/) data, please download from [COCO download](http://cocodataset.org/#download). COCO'2014 Train is needed for human mesh estimation training.
+Download and extract them under $MMPOSE/data, and make them look like this:
+
+```text
+mmpose
+├── mmpose
+├── docs
+├── tests
+├── tools
+├── configs
+`── data
+ │── coco
+ │-- train2014
+ │ ├── COCO_train2014_000000000009.jpg
+ │ ├── COCO_train2014_000000000025.jpg
+ │ ├── COCO_train2014_000000000030.jpg
+ | │-- ...
+
+```
+
+## Human3.6M
+
+
+
+
+Human3.6M (TPAMI'2014)
+
+```bibtex
+@article{h36m_pami,
+ author = {Ionescu, Catalin and Papava, Dragos and Olaru, Vlad and Sminchisescu, Cristian},
+ title = {Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments},
+ journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
+ publisher = {IEEE Computer Society},
+ volume = {36},
+ number = {7},
+ pages = {1325-1339},
+ month = {jul},
+ year = {2014}
+}
+```
+
+
+
+For [Human3.6M](http://vision.imar.ro/human3.6m/description.php), we use the MoShed data provided in [HMR](https://github.com/akanazawa/hmr) for training.
+However, due to license limitations, we are not allowed to redistribute the MoShed data.
+
+For the evaluation on Human3.6M dataset, please follow the
+[preprocess procedure](https://github.com/nkolot/SPIN/tree/master/datasets/preprocess)
+of SPIN to extract test images from
+[Human3.6M](http://vision.imar.ro/human3.6m/description.php) original videos,
+and make it look like this:
+
+```text
+mmpose
+├── mmpose
+├── docs
+├── tests
+├── tools
+├── configs
+`── data
+ │── Human3.6M
+ ├── images
+ ├── S11_Directions_1.54138969_000001.jpg
+ ├── S11_Directions_1.54138969_000006.jpg
+ ├── S11_Directions_1.54138969_000011.jpg
+ ├── ...
+```
+
+The download of Human3.6M dataset is quite difficult, you can also download the
+[zip file](https://drive.google.com/file/d/1WnRJD9FS3NUf7MllwgLRJJC-JgYFr8oi/view?usp=sharing)
+of the test images. However, due to the license limitations, we are not allowed to
+redistribute the images either. So the users need to download the original video and
+extract the images by themselves.
+
+## MPI-INF-3DHP
+
+
+
+```bibtex
+@inproceedings{mono-3dhp2017,
+ author = {Mehta, Dushyant and Rhodin, Helge and Casas, Dan and Fua, Pascal and Sotnychenko, Oleksandr and Xu, Weipeng and Theobalt, Christian},
+ title = {Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervision},
+ booktitle = {3D Vision (3DV), 2017 Fifth International Conference on},
+ url = {http://gvv.mpi-inf.mpg.de/3dhp_dataset},
+ year = {2017},
+ organization={IEEE},
+ doi={10.1109/3dv.2017.00064},
+}
+```
+
+For [MPI-INF-3DHP](http://gvv.mpi-inf.mpg.de/3dhp-dataset/), please follow the
+[preprocess procedure](https://github.com/nkolot/SPIN/tree/master/datasets/preprocess)
+of SPIN to sample images, and make them like this:
+
+```text
+mmpose
+├── mmpose
+├── docs
+├── tests
+├── tools
+├── configs
+`── data
+ ├── mpi_inf_3dhp_test_set
+ │ ├── TS1
+ │ ├── TS2
+ │ ├── TS3
+ │ ├── TS4
+ │ ├── TS5
+ │ └── TS6
+ ├── S1
+ │ ├── Seq1
+ │ └── Seq2
+ ├── S2
+ │ ├── Seq1
+ │ └── Seq2
+ ├── S3
+ │ ├── Seq1
+ │ └── Seq2
+ ├── S4
+ │ ├── Seq1
+ │ └── Seq2
+ ├── S5
+ │ ├── Seq1
+ │ └── Seq2
+ ├── S6
+ │ ├── Seq1
+ │ └── Seq2
+ ├── S7
+ │ ├── Seq1
+ │ └── Seq2
+ └── S8
+ ├── Seq1
+ └── Seq2
+```
+
+## LSP
+
+
+
+```bibtex
+@inproceedings{johnson2010clustered,
+ title={Clustered Pose and Nonlinear Appearance Models for Human Pose Estimation.},
+ author={Johnson, Sam and Everingham, Mark},
+ booktitle={bmvc},
+ volume={2},
+ number={4},
+ pages={5},
+ year={2010},
+ organization={Citeseer}
+}
+```
+
+For [LSP](https://sam.johnson.io/research/lsp.html), please download the high resolution version
+[LSP dataset original](http://sam.johnson.io/research/lsp_dataset_original.zip).
+Extract them under `$MMPOSE/data`, and make them look like this:
+
+```text
+mmpose
+├── mmpose
+├── docs
+├── tests
+├── tools
+├── configs
+`── data
+ │── lsp_dataset_original
+ ├── images
+ ├── im0001.jpg
+ ├── im0002.jpg
+ └── ...
+```
+
+## LSPET
+
+
+
+```bibtex
+@inproceedings{johnson2011learning,
+ title={Learning effective human pose estimation from inaccurate annotation},
+ author={Johnson, Sam and Everingham, Mark},
+ booktitle={CVPR 2011},
+ pages={1465--1472},
+ year={2011},
+ organization={IEEE}
+}
+```
+
+For [LSPET](https://sam.johnson.io/research/lspet.html), please download its high resolution form
+[HR-LSPET](http://datasets.d2.mpi-inf.mpg.de/hr-lspet/hr-lspet.zip).
+Extract them under `$MMPOSE/data`, and make them look like this:
+
+```text
+mmpose
+├── mmpose
+├── docs
+├── tests
+├── tools
+├── configs
+`── data
+ │── lspet_dataset
+ ├── images
+ │ ├── im00001.jpg
+ │ ├── im00002.jpg
+ │ ├── im00003.jpg
+ │ └── ...
+ └── joints.mat
+```
+
+## CMU MoShed Data
+
+
+
+```bibtex
+@inproceedings{kanazawa2018end,
+ title={End-to-end recovery of human shape and pose},
+ author={Kanazawa, Angjoo and Black, Michael J and Jacobs, David W and Malik, Jitendra},
+ booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
+ pages={7122--7131},
+ year={2018}
+}
+```
+
+Real-world SMPL parameters are used for the adversarial training in human mesh estimation.
+The MoShed data provided in [HMR](https://github.com/akanazawa/hmr) is included in this
+[zip file](https://download.openmmlab.com/mmpose/datasets/mesh_annotation_files.zip).
+Please download and extract it under `$MMPOSE/data`, and make it look like this:
+
+```text
+mmpose
+├── mmpose
+├── docs
+├── tests
+├── tools
+├── configs
+`── data
+ │── mesh_annotation_files
+ ├── CMU_mosh.npz
+ └── ...
+```
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/tasks/3d_hand_keypoint.md b/engine/pose_estimation/third-party/ViTPose/docs/en/tasks/3d_hand_keypoint.md
new file mode 100644
index 0000000..17537e4
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/tasks/3d_hand_keypoint.md
@@ -0,0 +1,55 @@
+# 3D Hand Keypoint Datasets
+
+It is recommended to symlink the dataset root to `$MMPOSE/data`.
+If your folder structure is different, you may need to change the corresponding paths in config files.
+
+MMPose supported datasets:
+
+- [InterHand2.6M](#interhand26m) \[ [Homepage](https://mks0601.github.io/InterHand2.6M/) \]
+
+## InterHand2.6M
+
+
+
+
+InterHand2.6M (ECCV'2020)
+
+```bibtex
+@InProceedings{Moon_2020_ECCV_InterHand2.6M,
+author = {Moon, Gyeongsik and Yu, Shoou-I and Wen, He and Shiratori, Takaaki and Lee, Kyoung Mu},
+title = {InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Image},
+booktitle = {European Conference on Computer Vision (ECCV)},
+year = {2020}
+}
+```
+
+
+
+For [InterHand2.6M](https://mks0601.github.io/InterHand2.6M/), please download from [InterHand2.6M](https://mks0601.github.io/InterHand2.6M/).
+Please download the annotation files from [annotations](https://drive.google.com/drive/folders/1pWXhdfaka-J0fSAze0MsajN0VpZ8e8tO).
+Extract them under {MMPose}/data, and make them look like this:
+
+```text
+mmpose
+├── mmpose
+├── docs
+├── tests
+├── tools
+├── configs
+`── data
+ │── interhand2.6m
+ |── annotations
+ | |── all
+ | |── human_annot
+ | |── machine_annot
+ | |── skeleton.txt
+ | |── subject.txt
+ |
+ `── images
+ | |── train
+ | | |-- Capture0 ~ Capture26
+ | |── val
+ | | |-- Capture0
+ | |── test
+ | | |-- Capture0 ~ Capture7
+```
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/tutorials/0_config.md b/engine/pose_estimation/third-party/ViTPose/docs/en/tutorials/0_config.md
new file mode 100644
index 0000000..4ca0780
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/tutorials/0_config.md
@@ -0,0 +1,235 @@
+# Tutorial 0: Learn about Configs
+
+We use python files as configs, incorporate modular and inheritance design into our config system, which is convenient to conduct various experiments.
+You can find all the provided configs under `$MMPose/configs`. If you wish to inspect the config file,
+you may run `python tools/analysis/print_config.py /PATH/TO/CONFIG` to see the complete config.
+
+
+
+- [Modify config through script arguments](#modify-config-through-script-arguments)
+- [Config File Naming Convention](#config-file-naming-convention)
+ - [Config System Example](#config-system-example)
+- [FAQ](#faq)
+ - [Use intermediate variables in configs](#use-intermediate-variables-in-configs)
+
+
+
+## Modify config through script arguments
+
+When submitting jobs using "tools/train.py" or "tools/test.py", you may specify `--cfg-options` to in-place modify the config.
+
+- Update config keys of dict chains.
+
+ The config options can be specified following the order of the dict keys in the original config.
+ For example, `--cfg-options model.backbone.norm_eval=False` changes the all BN modules in model backbones to `train` mode.
+
+- Update keys inside a list of configs.
+
+ Some config dicts are composed as a list in your config. For example, the training pipeline `data.train.pipeline` is normally a list
+ e.g. `[dict(type='LoadImageFromFile'), dict(type='TopDownRandomFlip', flip_prob=0.5), ...]`. If you want to change `'flip_prob=0.5'` to `'flip_prob=0.0'` in the pipeline,
+ you may specify `--cfg-options data.train.pipeline.1.flip_prob=0.0`.
+
+- Update values of list/tuples.
+
+ If the value to be updated is a list or a tuple. For example, the config file normally sets `workflow=[('train', 1)]`. If you want to
+ change this key, you may specify `--cfg-options workflow="[(train,1),(val,1)]"`. Note that the quotation mark \" is necessary to
+ support list/tuple data types, and that **NO** white space is allowed inside the quotation marks in the specified value.
+
+## Config File Naming Convention
+
+We follow the style below to name config files. Contributors are advised to follow the same style.
+
+```
+configs/{topic}/{task}/{algorithm}/{dataset}/{backbone}_[model_setting]_{dataset}_[input_size]_[technique].py
+```
+
+`{xxx}` is required field and `[yyy]` is optional.
+
+- `{topic}`: topic type, e.g. `body`, `face`, `hand`, `animal`, etc.
+- `{task}`: task type, `[2d | 3d]_[kpt | mesh]_[sview | mview]_[rgb | rgbd]_[img | vid]`. The task is categorized in 5: (1) 2D or 3D pose estimation, (2) representation type: keypoint (kpt), mesh, or DensePose (dense). (3) Single-view (sview) or multi-view (mview), (4) RGB or RGBD, and (5) Image (img) or Video (vid). e.g. `2d_kpt_sview_rgb_img`, `3d_kpt_sview_rgb_vid`, etc.
+- `{algorithm}`: algorithm type, e.g. `associative_embedding`, `deeppose`, etc.
+- `{dataset}`: dataset name, e.g. `coco`, etc.
+- `{backbone}`: backbone type, e.g. `res50` (ResNet-50), etc.
+- `[model setting]`: specific setting for some models.
+- `[input_size]`: input size of the model.
+- `[technique]`: some specific techniques, including losses, augmentation and tricks, e.g. `wingloss`, `udp`, `fp16`.
+
+### Config System
+
+- An Example of 2D Top-down Heatmap-based Human Pose Estimation
+
+ To help the users have a basic idea of a complete config structure and the modules in the config system,
+ we make brief comments on 'https://github.com/open-mmlab/mmpose/tree/e1ec589884235bee875c89102170439a991f8450/configs/top_down/resnet/coco/res50_coco_256x192.py' as the following.
+ For more detailed usage and alternative for per parameter in each module, please refer to the API documentation.
+
+ ```python
+ # runtime settings
+ log_level = 'INFO' # The level of logging
+ load_from = None # load models as a pre-trained model from a given path. This will not resume training
+ resume_from = None # Resume checkpoints from a given path, the training will be resumed from the epoch when the checkpoint's is saved
+ dist_params = dict(backend='nccl') # Parameters to setup distributed training, the port can also be set
+ workflow = [('train', 1)] # Workflow for runner. [('train', 1)] means there is only one workflow and the workflow named 'train' is executed once
+ checkpoint_config = dict( # Config to set the checkpoint hook, Refer to https://github.com/open-mmlab/mmcv/blob/master/mmcv/runner/hooks/checkpoint.py for implementation
+ interval=10) # Interval to save checkpoint
+ evaluation = dict( # Config of evaluation during training
+ interval=10, # Interval to perform evaluation
+ metric='mAP', # Metrics to be performed
+ save_best='AP') # set `AP` as key indicator to save best checkpoint
+ # optimizer
+ optimizer = dict(
+ # Config used to build optimizer, support (1). All the optimizers in PyTorch
+ # whose arguments are also the same as those in PyTorch. (2). Custom optimizers
+ # which are builed on `constructor`, referring to "tutorials/4_new_modules.md"
+ # for implementation.
+ type='Adam', # Type of optimizer, refer to https://github.com/open-mmlab/mmcv/blob/master/mmcv/runner/optimizer/default_constructor.py#L13 for more details
+ lr=5e-4, # Learning rate, see detail usages of the parameters in the documentation of PyTorch
+ )
+ optimizer_config = dict(grad_clip=None) # Do not use gradient clip
+ # learning policy
+ lr_config = dict( # Learning rate scheduler config used to register LrUpdater hook
+ policy='step', # Policy of scheduler, also support CosineAnnealing, Cyclic, etc. Refer to details of supported LrUpdater from https://github.com/open-mmlab/mmcv/blob/master/mmcv/runner/hooks/lr_updater.py#L9
+ warmup='linear', # Type of warmup used. It can be None(use no warmup), 'constant', 'linear' or 'exp'.
+ warmup_iters=500, # The number of iterations or epochs that warmup
+ warmup_ratio=0.001, # LR used at the beginning of warmup equals to warmup_ratio * initial_lr
+ step=[170, 200]) # Steps to decay the learning rate
+ total_epochs = 210 # Total epochs to train the model
+ log_config = dict( # Config to register logger hook
+ interval=50, # Interval to print the log
+ hooks=[
+ dict(type='TextLoggerHook'), # The logger used to record the training process
+ # dict(type='TensorboardLoggerHook') # The Tensorboard logger is also supported
+ ])
+
+ channel_cfg = dict(
+ num_output_channels=17, # The output channels of keypoint head
+ dataset_joints=17, # Number of joints in the dataset
+ dataset_channel=[ # Dataset supported channels
+ [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16],
+ ],
+ inference_channel=[ # Channels to output
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16
+ ])
+
+ # model settings
+ model = dict( # Config of the model
+ type='TopDown', # Type of the model
+ pretrained='torchvision://resnet50', # The url/site of the pretrained model
+ backbone=dict( # Dict for backbone
+ type='ResNet', # Name of the backbone
+ depth=50), # Depth of ResNet model
+ keypoint_head=dict( # Dict for keypoint head
+ type='TopdownHeatmapSimpleHead', # Name of keypoint head
+ in_channels=2048, # The input channels of keypoint head
+ out_channels=channel_cfg['num_output_channels'], # The output channels of keypoint head
+ loss_keypoint=dict( # Dict for keypoint loss
+ type='JointsMSELoss', # Name of keypoint loss
+ use_target_weight=True)), # Whether to consider target_weight during loss calculation
+ train_cfg=dict(), # Config of training hyper-parameters
+ test_cfg=dict( # Config of testing hyper-parameters
+ flip_test=True, # Whether to use flip-test during inference
+ post_process='default', # Use 'default' post-processing approach.
+ shift_heatmap=True, # Shift and align the flipped heatmap to achieve higher performance
+ modulate_kernel=11)) # Gaussian kernel size for modulation. Only used for "post_process='unbiased'"
+
+ data_cfg = dict(
+ image_size=[192, 256], # Size of model input resolution
+ heatmap_size=[48, 64], # Size of the output heatmap
+ num_output_channels=channel_cfg['num_output_channels'], # Number of output channels
+ num_joints=channel_cfg['dataset_joints'], # Number of joints
+ dataset_channel=channel_cfg['dataset_channel'], # Dataset supported channels
+ inference_channel=channel_cfg['inference_channel'], # Channels to output
+ soft_nms=False, # Whether to perform soft-nms during inference
+ nms_thr=1.0, # Threshold for non maximum suppression.
+ oks_thr=0.9, # Threshold of oks (object keypoint similarity) score during nms
+ vis_thr=0.2, # Threshold of keypoint visibility
+ use_gt_bbox=False, # Whether to use ground-truth bounding box during testing
+ det_bbox_thr=0.0, # Threshold of detected bounding box score. Used when 'use_gt_bbox=True'
+ bbox_file='data/coco/person_detection_results/' # Path to the bounding box detection file
+ 'COCO_val2017_detections_AP_H_56_person.json',
+ )
+
+ train_pipeline = [
+ dict(type='LoadImageFromFile'), # Loading image from file
+ dict(type='TopDownRandomFlip', # Perform random flip augmentation
+ flip_prob=0.5), # Probability of implementing flip
+ dict(
+ type='TopDownHalfBodyTransform', # Config of TopDownHalfBodyTransform data-augmentation
+ num_joints_half_body=8, # Threshold of performing half-body transform.
+ prob_half_body=0.3), # Probability of implementing half-body transform
+ dict(
+ type='TopDownGetRandomScaleRotation', # Config of TopDownGetRandomScaleRotation
+ rot_factor=40, # Rotating to ``[-2*rot_factor, 2*rot_factor]``.
+ scale_factor=0.5), # Scaling to ``[1-scale_factor, 1+scale_factor]``.
+ dict(type='TopDownAffine', # Affine transform the image to make input.
+ use_udp=False), # Do not use unbiased data processing.
+ dict(type='ToTensor'), # Convert other types to tensor type pipeline
+ dict(
+ type='NormalizeTensor', # Normalize input tensors
+ mean=[0.485, 0.456, 0.406], # Mean values of different channels to normalize
+ std=[0.229, 0.224, 0.225]), # Std values of different channels to normalize
+ dict(type='TopDownGenerateTarget', # Generate heatmap target. Different encoding types supported.
+ sigma=2), # Sigma of heatmap gaussian
+ dict(
+ type='Collect', # Collect pipeline that decides which keys in the data should be passed to the detector
+ keys=['img', 'target', 'target_weight'], # Keys of input
+ meta_keys=[ # Meta keys of input
+ 'image_file', 'joints_3d', 'joints_3d_visible', 'center', 'scale',
+ 'rotation', 'bbox_score', 'flip_pairs'
+ ]),
+ ]
+
+ val_pipeline = [
+ dict(type='LoadImageFromFile'), # Loading image from file
+ dict(type='TopDownAffine'), # Affine transform the image to make input.
+ dict(type='ToTensor'), # Config of ToTensor
+ dict(
+ type='NormalizeTensor',
+ mean=[0.485, 0.456, 0.406], # Mean values of different channels to normalize
+ std=[0.229, 0.224, 0.225]), # Std values of different channels to normalize
+ dict(
+ type='Collect', # Collect pipeline that decides which keys in the data should be passed to the detector
+ keys=['img'], # Keys of input
+ meta_keys=[ # Meta keys of input
+ 'image_file', 'center', 'scale', 'rotation', 'bbox_score',
+ 'flip_pairs'
+ ]),
+ ]
+
+ test_pipeline = val_pipeline
+
+ data_root = 'data/coco' # Root of the dataset
+ data = dict( # Config of data
+ samples_per_gpu=64, # Batch size of each single GPU during training
+ workers_per_gpu=2, # Workers to pre-fetch data for each single GPU
+ val_dataloader=dict(samples_per_gpu=32), # Batch size of each single GPU during validation
+ test_dataloader=dict(samples_per_gpu=32), # Batch size of each single GPU during testing
+ train=dict( # Training dataset config
+ type='TopDownCocoDataset', # Name of dataset
+ ann_file=f'{data_root}/annotations/person_keypoints_train2017.json', # Path to annotation file
+ img_prefix=f'{data_root}/train2017/',
+ data_cfg=data_cfg,
+ pipeline=train_pipeline),
+ val=dict( # Validation dataset config
+ type='TopDownCocoDataset', # Name of dataset
+ ann_file=f'{data_root}/annotations/person_keypoints_val2017.json', # Path to annotation file
+ img_prefix=f'{data_root}/val2017/',
+ data_cfg=data_cfg,
+ pipeline=val_pipeline),
+ test=dict( # Testing dataset config
+ type='TopDownCocoDataset', # Name of dataset
+ ann_file=f'{data_root}/annotations/person_keypoints_val2017.json', # Path to annotation file
+ img_prefix=f'{data_root}/val2017/',
+ data_cfg=data_cfg,
+ pipeline=val_pipeline),
+ )
+
+ ```
+
+## FAQ
+
+### Use intermediate variables in configs
+
+Some intermediate variables are used in the config files, like `train_pipeline`/`val_pipeline`/`test_pipeline` etc.
+
+For Example, we would like to first define `train_pipeline`/`val_pipeline`/`test_pipeline` and pass them into `data`.
+Thus, `train_pipeline`/`val_pipeline`/`test_pipeline` are intermediate variable.
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/tutorials/1_finetune.md b/engine/pose_estimation/third-party/ViTPose/docs/en/tutorials/1_finetune.md
new file mode 100644
index 0000000..7f8ea09
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/tutorials/1_finetune.md
@@ -0,0 +1,153 @@
+# Tutorial 1: Finetuning Models
+
+Detectors pre-trained on the COCO dataset can serve as a good pre-trained model for other datasets, e.g., COCO-WholeBody Dataset.
+This tutorial provides instruction for users to use the models provided in the [Model Zoo](https://mmpose.readthedocs.io/en/latest/modelzoo.html) for other datasets to obtain better performance.
+
+
+
+- [Outline](#outline)
+- [Modify Head](#modify-head)
+- [Modify Dataset](#modify-dataset)
+- [Modify Training Schedule](#modify-training-schedule)
+- [Use Pre-Trained Model](#use-pre-trained-model)
+
+
+
+## Outline
+
+There are two steps to finetune a model on a new dataset.
+
+- Add support for the new dataset following [Tutorial 2: Adding New Dataset](tutorials/../2_new_dataset.md).
+- Modify the configs as will be discussed in this tutorial.
+
+To finetune on the custom datasets, the users need to modify four parts in the config.
+
+## Modify Head
+
+Then the new config needs to modify the model according to the keypoint numbers of the new datasets. By only changing `out_channels` in the keypoint_head.
+For example, we have 133 keypoints for COCO-WholeBody, and we have 17 keypoints for COCO.
+
+```python
+channel_cfg = dict(
+ num_output_channels=133, # changing from 17 to 133
+ dataset_joints=133, # changing from 17 to 133
+ dataset_channel=[
+ list(range(133)), # changing from 17 to 133
+ ],
+ inference_channel=list(range(133))) # changing from 17 to 133
+
+# model settings
+model = dict(
+ type='TopDown',
+ pretrained='https://download.openmmlab.com/mmpose/'
+ 'pretrain_models/hrnet_w48-8ef0771d.pth',
+ backbone=dict(
+ type='HRNet',
+ in_channels=3,
+ extra=dict(
+ stage1=dict(
+ num_modules=1,
+ num_branches=1,
+ block='BOTTLENECK',
+ num_blocks=(4, ),
+ num_channels=(64, )),
+ stage2=dict(
+ num_modules=1,
+ num_branches=2,
+ block='BASIC',
+ num_blocks=(4, 4),
+ num_channels=(48, 96)),
+ stage3=dict(
+ num_modules=4,
+ num_branches=3,
+ block='BASIC',
+ num_blocks=(4, 4, 4),
+ num_channels=(48, 96, 192)),
+ stage4=dict(
+ num_modules=3,
+ num_branches=4,
+ block='BASIC',
+ num_blocks=(4, 4, 4, 4),
+ num_channels=(48, 96, 192, 384))),
+ ),
+ keypoint_head=dict(
+ type='TopdownHeatmapSimpleHead',
+ in_channels=48,
+ out_channels=channel_cfg['num_output_channels'], # modify this
+ num_deconv_layers=0,
+ extra=dict(final_conv_kernel=1, ),
+ loss_keypoint=dict(type='JointsMSELoss', use_target_weight=True)),
+ train_cfg=dict(),
+ test_cfg=dict(
+ flip_test=True,
+ post_process='unbiased',
+ shift_heatmap=True,
+ modulate_kernel=17))
+```
+
+Note that the `pretrained='https://download.openmmlab.com/mmpose/pretrain_models/hrnet_w48-8ef0771d.pth'` setting is used for initializing backbone.
+If you are training a new model from ImageNet-pretrained weights, this is for you.
+However, this setting is not related to our task at hand. What we need is load_from, which will be discussed later.
+
+## Modify dataset
+
+The users may also need to prepare the dataset and write the configs about dataset.
+MMPose supports multiple (10+) dataset, including COCO, COCO-WholeBody and MPII-TRB.
+
+```python
+data_root = 'data/coco'
+data = dict(
+ samples_per_gpu=32,
+ workers_per_gpu=2,
+ val_dataloader=dict(samples_per_gpu=32),
+ test_dataloader=dict(samples_per_gpu=32),
+ train=dict(
+ type='TopDownCocoWholeBodyDataset', # modify the name of the dataset
+ ann_file=f'{data_root}/annotations/coco_wholebody_train_v1.0.json', # modify the path to the annotation file
+ img_prefix=f'{data_root}/train2017/',
+ data_cfg=data_cfg,
+ pipeline=train_pipeline),
+ val=dict(
+ type='TopDownCocoWholeBodyDataset', # modify the name of the dataset
+ ann_file=f'{data_root}/annotations/coco_wholebody_val_v1.0.json', # modify the path to the annotation file
+ img_prefix=f'{data_root}/val2017/',
+ data_cfg=data_cfg,
+ pipeline=val_pipeline),
+ test=dict(
+ type='TopDownCocoWholeBodyDataset', # modify the name of the dataset
+ ann_file=f'{data_root}/annotations/coco_wholebody_val_v1.0.json', # modify the path to the annotation file
+ img_prefix=f'{data_root}/val2017/',
+ data_cfg=data_cfg,
+ pipeline=val_pipeline)
+)
+```
+
+## Modify training schedule
+
+The finetuning hyperparameters vary from the default schedule. It usually requires smaller learning rate and less training epochs
+
+```python
+optimizer = dict(
+ type='Adam',
+ lr=5e-4, # reduce it
+)
+optimizer_config = dict(grad_clip=None)
+# learning policy
+lr_config = dict(
+ policy='step',
+ warmup='linear',
+ warmup_iters=500,
+ warmup_ratio=0.001,
+ step=[170, 200]) # reduce it
+total_epochs = 210 # reduce it
+```
+
+## Use pre-trained model
+
+Users can load a pre-trained model by setting the `load_from` field of the config to the model's path or link.
+The users might need to download the model weights before training to avoid the download time during training.
+
+```python
+# use the pre-trained model for the whole HRNet
+load_from = 'https://download.openmmlab.com/mmpose/top_down/hrnet/hrnet_w48_coco_384x288_dark-741844ba_20200812.pth' # model path can be found in model zoo
+```
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/tutorials/2_new_dataset.md b/engine/pose_estimation/third-party/ViTPose/docs/en/tutorials/2_new_dataset.md
new file mode 100644
index 0000000..de628b4
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/tutorials/2_new_dataset.md
@@ -0,0 +1,318 @@
+# Tutorial 2: Adding New Dataset
+
+## Customize datasets by reorganizing data to COCO format
+
+The simplest way to use the custom dataset is to convert your annotation format to COCO dataset format.
+
+The annotation json files in COCO format has the following necessary keys:
+
+```python
+'images': [
+ {
+ 'file_name': '000000001268.jpg',
+ 'height': 427,
+ 'width': 640,
+ 'id': 1268
+ },
+ ...
+],
+'annotations': [
+ {
+ 'segmentation': [[426.36,
+ ...
+ 424.34,
+ 223.3]],
+ 'keypoints': [0,0,0,
+ 0,0,0,
+ 0,0,0,
+ 427,220,2,
+ 443,222,2,
+ 414,228,2,
+ 449,232,2,
+ 408,248,1,
+ 454,261,2,
+ 0,0,0,
+ 0,0,0,
+ 411,287,2,
+ 431,287,2,
+ 0,0,0,
+ 458,265,2,
+ 0,0,0,
+ 466,300,1],
+ 'num_keypoints': 10,
+ 'area': 3894.5826,
+ 'iscrowd': 0,
+ 'image_id': 1268,
+ 'bbox': [402.34, 205.02, 65.26, 88.45],
+ 'category_id': 1,
+ 'id': 215218
+ },
+ ...
+],
+'categories': [
+ {'id': 1, 'name': 'person'},
+ ]
+```
+
+There are three necessary keys in the json file:
+
+- `images`: contains a list of images with their information like `file_name`, `height`, `width`, and `id`.
+- `annotations`: contains the list of instance annotations.
+- `categories`: contains the category name ('person') and its ID (1).
+
+## Create a custom dataset_info config file for the dataset
+
+Add a new dataset info config file.
+
+```
+configs/_base_/datasets/custom.py
+```
+
+An example of the dataset config is as follows.
+
+`keypoint_info` contains the information about each keypoint.
+
+1. `name`: the keypoint name. The keypoint name must be unique.
+2. `id`: the keypoint id.
+3. `color`: ([B, G, R]) is used for keypoint visualization.
+4. `type`: 'upper' or 'lower', will be used in data augmetation.
+5. `swap`: indicates the 'swap pair' (also known as 'flip pair'). When applying image horizontal flip, the left part will become the right part. We need to flip the keypoints accordingly.
+
+`skeleton_info` contains the information about the keypoint connectivity, which is used for visualization.
+
+`joint_weights` assigns different loss weights to different keypoints.
+
+`sigmas` is used to calculate the OKS score. Please read [keypoints-eval](https://cocodataset.org/#keypoints-eval) to learn more about it.
+
+```
+dataset_info = dict(
+ dataset_name='coco',
+ paper_info=dict(
+ author='Lin, Tsung-Yi and Maire, Michael and '
+ 'Belongie, Serge and Hays, James and '
+ 'Perona, Pietro and Ramanan, Deva and '
+ r'Doll{\'a}r, Piotr and Zitnick, C Lawrence',
+ title='Microsoft coco: Common objects in context',
+ container='European conference on computer vision',
+ year='2014',
+ homepage='http://cocodataset.org/',
+ ),
+ keypoint_info={
+ 0:
+ dict(name='nose', id=0, color=[51, 153, 255], type='upper', swap=''),
+ 1:
+ dict(
+ name='left_eye',
+ id=1,
+ color=[51, 153, 255],
+ type='upper',
+ swap='right_eye'),
+ 2:
+ dict(
+ name='right_eye',
+ id=2,
+ color=[51, 153, 255],
+ type='upper',
+ swap='left_eye'),
+ 3:
+ dict(
+ name='left_ear',
+ id=3,
+ color=[51, 153, 255],
+ type='upper',
+ swap='right_ear'),
+ 4:
+ dict(
+ name='right_ear',
+ id=4,
+ color=[51, 153, 255],
+ type='upper',
+ swap='left_ear'),
+ 5:
+ dict(
+ name='left_shoulder',
+ id=5,
+ color=[0, 255, 0],
+ type='upper',
+ swap='right_shoulder'),
+ 6:
+ dict(
+ name='right_shoulder',
+ id=6,
+ color=[255, 128, 0],
+ type='upper',
+ swap='left_shoulder'),
+ 7:
+ dict(
+ name='left_elbow',
+ id=7,
+ color=[0, 255, 0],
+ type='upper',
+ swap='right_elbow'),
+ 8:
+ dict(
+ name='right_elbow',
+ id=8,
+ color=[255, 128, 0],
+ type='upper',
+ swap='left_elbow'),
+ 9:
+ dict(
+ name='left_wrist',
+ id=9,
+ color=[0, 255, 0],
+ type='upper',
+ swap='right_wrist'),
+ 10:
+ dict(
+ name='right_wrist',
+ id=10,
+ color=[255, 128, 0],
+ type='upper',
+ swap='left_wrist'),
+ 11:
+ dict(
+ name='left_hip',
+ id=11,
+ color=[0, 255, 0],
+ type='lower',
+ swap='right_hip'),
+ 12:
+ dict(
+ name='right_hip',
+ id=12,
+ color=[255, 128, 0],
+ type='lower',
+ swap='left_hip'),
+ 13:
+ dict(
+ name='left_knee',
+ id=13,
+ color=[0, 255, 0],
+ type='lower',
+ swap='right_knee'),
+ 14:
+ dict(
+ name='right_knee',
+ id=14,
+ color=[255, 128, 0],
+ type='lower',
+ swap='left_knee'),
+ 15:
+ dict(
+ name='left_ankle',
+ id=15,
+ color=[0, 255, 0],
+ type='lower',
+ swap='right_ankle'),
+ 16:
+ dict(
+ name='right_ankle',
+ id=16,
+ color=[255, 128, 0],
+ type='lower',
+ swap='left_ankle')
+ },
+ skeleton_info={
+ 0:
+ dict(link=('left_ankle', 'left_knee'), id=0, color=[0, 255, 0]),
+ 1:
+ dict(link=('left_knee', 'left_hip'), id=1, color=[0, 255, 0]),
+ 2:
+ dict(link=('right_ankle', 'right_knee'), id=2, color=[255, 128, 0]),
+ 3:
+ dict(link=('right_knee', 'right_hip'), id=3, color=[255, 128, 0]),
+ 4:
+ dict(link=('left_hip', 'right_hip'), id=4, color=[51, 153, 255]),
+ 5:
+ dict(link=('left_shoulder', 'left_hip'), id=5, color=[51, 153, 255]),
+ 6:
+ dict(link=('right_shoulder', 'right_hip'), id=6, color=[51, 153, 255]),
+ 7:
+ dict(
+ link=('left_shoulder', 'right_shoulder'),
+ id=7,
+ color=[51, 153, 255]),
+ 8:
+ dict(link=('left_shoulder', 'left_elbow'), id=8, color=[0, 255, 0]),
+ 9:
+ dict(
+ link=('right_shoulder', 'right_elbow'), id=9, color=[255, 128, 0]),
+ 10:
+ dict(link=('left_elbow', 'left_wrist'), id=10, color=[0, 255, 0]),
+ 11:
+ dict(link=('right_elbow', 'right_wrist'), id=11, color=[255, 128, 0]),
+ 12:
+ dict(link=('left_eye', 'right_eye'), id=12, color=[51, 153, 255]),
+ 13:
+ dict(link=('nose', 'left_eye'), id=13, color=[51, 153, 255]),
+ 14:
+ dict(link=('nose', 'right_eye'), id=14, color=[51, 153, 255]),
+ 15:
+ dict(link=('left_eye', 'left_ear'), id=15, color=[51, 153, 255]),
+ 16:
+ dict(link=('right_eye', 'right_ear'), id=16, color=[51, 153, 255]),
+ 17:
+ dict(link=('left_ear', 'left_shoulder'), id=17, color=[51, 153, 255]),
+ 18:
+ dict(
+ link=('right_ear', 'right_shoulder'), id=18, color=[51, 153, 255])
+ },
+ joint_weights=[
+ 1., 1., 1., 1., 1., 1., 1., 1.2, 1.2, 1.5, 1.5, 1., 1., 1.2, 1.2, 1.5,
+ 1.5
+ ],
+ sigmas=[
+ 0.026, 0.025, 0.025, 0.035, 0.035, 0.079, 0.079, 0.072, 0.072, 0.062,
+ 0.062, 0.107, 0.107, 0.087, 0.087, 0.089, 0.089
+ ])
+```
+
+## Create a custom dataset class
+
+1. First create a package inside the mmpose/datasets/datasets folder.
+
+2. Create a class definition of your dataset in the package folder and register it in the registry with a name. Without a name, it will keep giving the error. `KeyError: 'XXXXX is not in the dataset registry'`
+
+ ```
+ @DATASETS.register_module(name='MyCustomDataset')
+ class MyCustomDataset(SomeOtherBaseClassAsPerYourNeed):
+ ```
+
+3. Make sure you have updated the `__init__.py` of your package folder
+
+4. Make sure you have updated the `__init__.py` of the dataset package folder.
+
+## Create a custom training config file
+
+Create a custom training config file as per your need and the model/architecture you want to use in the configs folder. You may modify an existing config file to use the new custom dataset.
+
+In `configs/my_custom_config.py`:
+
+```python
+...
+# dataset settings
+dataset_type = 'MyCustomDataset'
+...
+data = dict(
+ samples_per_gpu=2,
+ workers_per_gpu=2,
+ train=dict(
+ type=dataset_type,
+ ann_file='path/to/your/train/json',
+ img_prefix='path/to/your/train/img',
+ ...),
+ val=dict(
+ type=dataset_type,
+ ann_file='path/to/your/val/json',
+ img_prefix='path/to/your/val/img',
+ ...),
+ test=dict(
+ type=dataset_type,
+ ann_file='path/to/your/test/json',
+ img_prefix='path/to/your/test/img',
+ ...))
+...
+```
+
+Make sure you have provided all the paths correctly.
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/tutorials/3_data_pipeline.md b/engine/pose_estimation/third-party/ViTPose/docs/en/tutorials/3_data_pipeline.md
new file mode 100644
index 0000000..a637a8c
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/tutorials/3_data_pipeline.md
@@ -0,0 +1,153 @@
+# Tutorial 3: Custom Data Pipelines
+
+## Design of Data pipelines
+
+Following typical conventions, we use `Dataset` and `DataLoader` for data loading
+with multiple workers. `Dataset` returns a dict of data items corresponding
+the arguments of models' forward method.
+Since the data in pose estimation may not be the same size (image size, gt bbox size, etc.),
+we introduce a new `DataContainer` type in MMCV to help collect and distribute
+data of different size.
+See [here](https://github.com/open-mmlab/mmcv/blob/master/mmcv/parallel/data_container.py) for more details.
+
+The data preparation pipeline and the dataset is decomposed. Usually a dataset
+defines how to process the annotations and a data pipeline defines all the steps to prepare a data dict.
+A pipeline consists of a sequence of operations. Each operation takes a dict as input and also output a dict for the next transform.
+
+The operations are categorized into data loading, pre-processing, formatting, label generating.
+
+Here is an pipeline example for Simple Baseline (ResNet50).
+
+```python
+train_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='TopDownRandomFlip', flip_prob=0.5),
+ dict(type='TopDownHalfBodyTransform', num_joints_half_body=8, prob_half_body=0.3),
+ dict(type='TopDownGetRandomScaleRotation', rot_factor=40, scale_factor=0.5),
+ dict(type='TopDownAffine'),
+ dict(type='ToTensor'),
+ dict(
+ type='NormalizeTensor',
+ mean=[0.485, 0.456, 0.406],
+ std=[0.229, 0.224, 0.225]),
+ dict(type='TopDownGenerateTarget', sigma=2),
+ dict(
+ type='Collect',
+ keys=['img', 'target', 'target_weight'],
+ meta_keys=[
+ 'image_file', 'joints_3d', 'joints_3d_visible', 'center', 'scale',
+ 'rotation', 'bbox_score', 'flip_pairs'
+ ]),
+]
+
+val_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='TopDownAffine'),
+ dict(type='ToTensor'),
+ dict(
+ type='NormalizeTensor',
+ mean=[0.485, 0.456, 0.406],
+ std=[0.229, 0.224, 0.225]),
+ dict(
+ type='Collect',
+ keys=['img'],
+ meta_keys=[
+ 'image_file', 'center', 'scale', 'rotation', 'bbox_score',
+ 'flip_pairs'
+ ]),
+]
+```
+
+For each operation, we list the related dict fields that are added/updated/removed.
+
+### Data loading
+
+`LoadImageFromFile`
+
+- add: img, img_file
+
+### Pre-processing
+
+`TopDownRandomFlip`
+
+- update: img, joints_3d, joints_3d_visible, center
+
+`TopDownHalfBodyTransform`
+
+- update: center, scale
+
+`TopDownGetRandomScaleRotation`
+
+- update: scale, rotation
+
+`TopDownAffine`
+
+- update: img, joints_3d, joints_3d_visible
+
+`NormalizeTensor`
+
+- update: img
+
+### Generating labels
+
+`TopDownGenerateTarget`
+
+- add: target, target_weight
+
+### Formatting
+
+`ToTensor`
+
+- update: 'img'
+
+`Collect`
+
+- add: img_meta (the keys of img_meta is specified by `meta_keys`)
+- remove: all other keys except for those specified by `keys`
+
+## Extend and use custom pipelines
+
+1. Write a new pipeline in any file, e.g., `my_pipeline.py`. It takes a dict as input and return a dict.
+
+ ```python
+ from mmpose.datasets import PIPELINES
+
+ @PIPELINES.register_module()
+ class MyTransform:
+
+ def __call__(self, results):
+ results['dummy'] = True
+ return results
+ ```
+
+1. Import the new class.
+
+ ```python
+ from .my_pipeline import MyTransform
+ ```
+
+1. Use it in config files.
+
+ ```python
+ train_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='TopDownRandomFlip', flip_prob=0.5),
+ dict(type='TopDownHalfBodyTransform', num_joints_half_body=8, prob_half_body=0.3),
+ dict(type='TopDownGetRandomScaleRotation', rot_factor=40, scale_factor=0.5),
+ dict(type='TopDownAffine'),
+ dict(type='MyTransform'),
+ dict(type='ToTensor'),
+ dict(
+ type='NormalizeTensor',
+ mean=[0.485, 0.456, 0.406],
+ std=[0.229, 0.224, 0.225]),
+ dict(type='TopDownGenerateTarget', sigma=2),
+ dict(
+ type='Collect',
+ keys=['img', 'target', 'target_weight'],
+ meta_keys=[
+ 'image_file', 'joints_3d', 'joints_3d_visible', 'center', 'scale',
+ 'rotation', 'bbox_score', 'flip_pairs'
+ ]),
+ ]
+ ```
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/tutorials/4_new_modules.md b/engine/pose_estimation/third-party/ViTPose/docs/en/tutorials/4_new_modules.md
new file mode 100644
index 0000000..e1864b2
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/tutorials/4_new_modules.md
@@ -0,0 +1,213 @@
+# Tutorial 4: Adding New Modules
+
+## Customize optimizer
+
+A customized optimizer could be defined as following.
+Assume you want to add a optimizer named as `MyOptimizer`, which has arguments `a`, `b`, and `c`.
+You need to first implement the new optimizer in a file, e.g., in `mmpose/core/optimizer/my_optimizer.py`:
+
+```python
+from mmcv.runner import OPTIMIZERS
+from torch.optim import Optimizer
+
+
+@OPTIMIZERS.register_module()
+class MyOptimizer(Optimizer):
+
+ def __init__(self, a, b, c)
+
+```
+
+Then add this module in `mmpose/core/optimizer/__init__.py` thus the registry will
+find the new module and add it:
+
+```python
+from .my_optimizer import MyOptimizer
+```
+
+Then you can use `MyOptimizer` in `optimizer` field of config files.
+In the configs, the optimizers are defined by the field `optimizer` like the following:
+
+```python
+optimizer = dict(type='SGD', lr=0.02, momentum=0.9, weight_decay=0.0001)
+```
+
+To use your own optimizer, the field can be changed as
+
+```python
+optimizer = dict(type='MyOptimizer', a=a_value, b=b_value, c=c_value)
+```
+
+We already support to use all the optimizers implemented by PyTorch, and the only modification is to change the `optimizer` field of config files.
+For example, if you want to use `ADAM`, though the performance will drop a lot, the modification could be as the following.
+
+```python
+optimizer = dict(type='Adam', lr=0.0003, weight_decay=0.0001)
+```
+
+The users can directly set arguments following the [API doc](https://pytorch.org/docs/stable/optim.html?highlight=optim#module-torch.optim) of PyTorch.
+
+## Customize optimizer constructor
+
+Some models may have some parameter-specific settings for optimization, e.g. weight decay for BatchNorm layers.
+The users can do those fine-grained parameter tuning through customizing optimizer constructor.
+
+```
+from mmcv.utils import build_from_cfg
+
+from mmcv.runner import OPTIMIZER_BUILDERS, OPTIMIZERS
+from mmpose.utils import get_root_logger
+from .cocktail_optimizer import CocktailOptimizer
+
+
+@OPTIMIZER_BUILDERS.register_module()
+class CocktailOptimizerConstructor:
+
+ def __init__(self, optimizer_cfg, paramwise_cfg=None):
+
+ def __call__(self, model):
+
+ return my_optimizer
+
+```
+
+### Develop new components
+
+We basically categorize model components into 3 types.
+
+- detectors: the whole pose detector model pipeline, usually contains a backbone and keypoint_head.
+- backbone: usually an FCN network to extract feature maps, e.g., ResNet, HRNet.
+- keypoint_head: the component for pose estimation task, usually contains some deconv layers.
+
+1. Create a new file `mmpose/models/backbones/my_model.py`.
+
+```python
+import torch.nn as nn
+
+from ..builder import BACKBONES
+
+@BACKBONES.register_module()
+class MyModel(nn.Module):
+
+ def __init__(self, arg1, arg2):
+ pass
+
+ def forward(self, x): # should return a tuple
+ pass
+
+ def init_weights(self, pretrained=None):
+ pass
+```
+
+2. Import the module in `mmpose/models/backbones/__init__.py`.
+
+```python
+from .my_model import MyModel
+```
+
+3. Create a new file `mmpose/models/keypoint_heads/my_head.py`.
+
+You can write a new keypoint head inherit from `nn.Module`,
+and overwrite `init_weights(self)` and `forward(self, x)` method.
+
+```python
+from ..builder import HEADS
+
+
+@HEADS.register_module()
+class MyHead(nn.Module):
+
+ def __init__(self, arg1, arg2):
+ pass
+
+ def forward(self, x):
+ pass
+
+ def init_weights(self):
+ pass
+```
+
+4. Import the module in `mmpose/models/keypoint_heads/__init__.py`
+
+```python
+from .my_head import MyHead
+```
+
+5. Use it in your config file.
+
+For the top-down 2D pose estimation model, we set the module type as `TopDown`.
+
+```python
+model = dict(
+ type='TopDown',
+ backbone=dict(
+ type='MyModel',
+ arg1=xxx,
+ arg2=xxx),
+ keypoint_head=dict(
+ type='MyHead',
+ arg1=xxx,
+ arg2=xxx))
+```
+
+### Add new loss
+
+Assume you want to add a new loss as `MyLoss`, for keypoints estimation.
+To add a new loss function, the users need implement it in `mmpose/models/losses/my_loss.py`.
+The decorator `weighted_loss` enable the loss to be weighted for each element.
+
+```python
+import torch
+import torch.nn as nn
+
+from mmpose.models import LOSSES
+
+def my_loss(pred, target):
+ assert pred.size() == target.size() and target.numel() > 0
+ loss = torch.abs(pred - target)
+ loss = torch.mean(loss)
+ return loss
+
+@LOSSES.register_module()
+class MyLoss(nn.Module):
+
+ def __init__(self, use_target_weight=False):
+ super(MyLoss, self).__init__()
+ self.criterion = my_loss()
+ self.use_target_weight = use_target_weight
+
+ def forward(self, output, target, target_weight):
+ batch_size = output.size(0)
+ num_joints = output.size(1)
+
+ heatmaps_pred = output.reshape(
+ (batch_size, num_joints, -1)).split(1, 1)
+ heatmaps_gt = target.reshape((batch_size, num_joints, -1)).split(1, 1)
+
+ loss = 0.
+
+ for idx in range(num_joints):
+ heatmap_pred = heatmaps_pred[idx].squeeze(1)
+ heatmap_gt = heatmaps_gt[idx].squeeze(1)
+ if self.use_target_weight:
+ loss += self.criterion(
+ heatmap_pred * target_weight[:, idx],
+ heatmap_gt * target_weight[:, idx])
+ else:
+ loss += self.criterion(heatmap_pred, heatmap_gt)
+
+ return loss / num_joints
+```
+
+Then the users need to add it in the `mmpose/models/losses/__init__.py`.
+
+```python
+from .my_loss import MyLoss, my_loss
+
+```
+
+To use it, modify the `loss_keypoint` field in the model.
+
+```python
+loss_keypoint=dict(type='MyLoss', use_target_weight=False)
+```
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/tutorials/5_export_model.md b/engine/pose_estimation/third-party/ViTPose/docs/en/tutorials/5_export_model.md
new file mode 100644
index 0000000..14d7610
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/tutorials/5_export_model.md
@@ -0,0 +1,48 @@
+# Tutorial 5: Exporting a model to ONNX
+
+Open Neural Network Exchange [(ONNX)](https://onnx.ai/) is an open ecosystem that empowers AI developers to choose the right tools as their project evolves.
+
+
+
+- [Supported Models](#supported-models)
+- [Usage](#usage)
+ - [Prerequisite](#prerequisite)
+
+
+
+## Supported Models
+
+So far, our codebase supports onnx exporting from pytorch models trained with MMPose. The supported models include:
+
+- ResNet
+- HRNet
+- HigherHRNet
+
+## Usage
+
+For simple exporting, you can use the [script](/tools/pytorch2onnx.py) here. Note that the package `onnx` and `onnxruntime` are required for verification after exporting.
+
+### Prerequisite
+
+First, install onnx.
+
+```shell
+pip install onnx onnxruntime
+```
+
+We provide a python script to export the pytorch model trained by MMPose to ONNX.
+
+```shell
+python tools/deployment/pytorch2onnx.py ${CONFIG_FILE} ${CHECKPOINT_FILE} [--shape ${SHAPE}] \
+ [--verify] [--show] [--output-file ${OUTPUT_FILE}] [--opset-version ${VERSION}]
+```
+
+Optional arguments:
+
+- `--shape`: The shape of input tensor to the model. If not specified, it will be set to `1 3 256 192`.
+- `--verify`: Determines whether to verify the exported model, runnably and numerically. If not specified, it will be set to `False`.
+- `--show`: Determines whether to print the architecture of the exported model. If not specified, it will be set to `False`.
+- `--output-file`: The output onnx model name. If not specified, it will be set to `tmp.onnx`.
+- `--opset-version`: Determines the operation set version of onnx, we recommend you to use a higher version such as 11 for compatibility. If not specified, it will be set to `11`.
+
+Please fire an issue if you discover any checkpoints that are not perfectly exported or suffer some loss in accuracy.
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/tutorials/6_customize_runtime.md b/engine/pose_estimation/third-party/ViTPose/docs/en/tutorials/6_customize_runtime.md
new file mode 100644
index 0000000..2803cd5
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/tutorials/6_customize_runtime.md
@@ -0,0 +1,352 @@
+# Tutorial 6: Customize Runtime Settings
+
+In this tutorial, we will introduce some methods about how to customize optimization methods, training schedules, workflow and hooks when running your own settings for the project.
+
+
+
+- [Customize Optimization Methods](#customize-optimization-methods)
+ - [Customize optimizer supported by PyTorch](#customize-optimizer-supported-by-pytorch)
+ - [Customize self-implemented optimizer](#customize-self-implemented-optimizer)
+ - [1. Define a new optimizer](#1-define-a-new-optimizer)
+ - [2. Add the optimizer to registry](#2-add-the-optimizer-to-registry)
+ - [3. Specify the optimizer in the config file](#3-specify-the-optimizer-in-the-config-file)
+ - [Customize optimizer constructor](#customize-optimizer-constructor)
+ - [Additional settings](#additional-settings)
+- [Customize Training Schedules](#customize-training-schedules)
+- [Customize Workflow](#customize-workflow)
+- [Customize Hooks](#customize-hooks)
+ - [Customize self-implemented hooks](#customize-self-implemented-hooks)
+ - [1. Implement a new hook](#1-implement-a-new-hook)
+ - [2. Register the new hook](#2-register-the-new-hook)
+ - [3. Modify the config](#3-modify-the-config)
+ - [Use hooks implemented in MMCV](#use-hooks-implemented-in-mmcv)
+ - [Modify default runtime hooks](#modify-default-runtime-hooks)
+ - [Checkpoint config](#checkpoint-config)
+ - [Log config](#log-config)
+ - [Evaluation config](#evaluation-config)
+
+
+
+## Customize Optimization Methods
+
+### Customize optimizer supported by PyTorch
+
+We already support to use all the optimizers implemented by PyTorch, and the only modification is to change the `optimizer` field of config files.
+For example, if you want to use `Adam`, the modification could be as the following.
+
+```python
+optimizer = dict(type='Adam', lr=0.0003, weight_decay=0.0001)
+```
+
+To modify the learning rate of the model, the users only need to modify the `lr` in the config of optimizer.
+The users can directly set arguments following the [API doc](https://pytorch.org/docs/stable/optim.html?highlight=optim#module-torch.optim) of PyTorch.
+
+For example, if you want to use `Adam` with the setting like `torch.optim.Adam(params, lr=0.001, betas=(0.9, 0.999), eps=1e-08, weight_decay=0, amsgrad=False)` in PyTorch,
+the modification could be set as the following.
+
+```python
+optimizer = dict(type='Adam', lr=0.001, betas=(0.9, 0.999), eps=1e-08, weight_decay=0, amsgrad=False)
+```
+
+### Customize self-implemented optimizer
+
+#### 1. Define a new optimizer
+
+A customized optimizer could be defined as following.
+
+Assume you want to add an optimizer named `MyOptimizer`, which has arguments `a`, `b`, and `c`.
+You need to create a new directory named `mmpose/core/optimizer`.
+And then implement the new optimizer in a file, e.g., in `mmpose/core/optimizer/my_optimizer.py`:
+
+```python
+from .builder import OPTIMIZERS
+from torch.optim import Optimizer
+
+
+@OPTIMIZERS.register_module()
+class MyOptimizer(Optimizer):
+
+ def __init__(self, a, b, c):
+
+```
+
+#### 2. Add the optimizer to registry
+
+To find the above module defined above, this module should be imported into the main namespace at first. There are two ways to achieve it.
+
+- Modify `mmpose/core/optimizer/__init__.py` to import it.
+
+ The newly defined module should be imported in `mmpose/core/optimizer/__init__.py` so that the registry will
+ find the new module and add it:
+
+```python
+from .my_optimizer import MyOptimizer
+```
+
+- Use `custom_imports` in the config to manually import it
+
+```python
+custom_imports = dict(imports=['mmpose.core.optimizer.my_optimizer'], allow_failed_imports=False)
+```
+
+The module `mmpose.core.optimizer.my_optimizer` will be imported at the beginning of the program and the class `MyOptimizer` is then automatically registered.
+Note that only the package containing the class `MyOptimizer` should be imported. `mmpose.core.optimizer.my_optimizer.MyOptimizer` **cannot** be imported directly.
+
+#### 3. Specify the optimizer in the config file
+
+Then you can use `MyOptimizer` in `optimizer` field of config files.
+In the configs, the optimizers are defined by the field `optimizer` like the following:
+
+```python
+optimizer = dict(type='SGD', lr=0.02, momentum=0.9, weight_decay=0.0001)
+```
+
+To use your own optimizer, the field can be changed to
+
+```python
+optimizer = dict(type='MyOptimizer', a=a_value, b=b_value, c=c_value)
+```
+
+### Customize optimizer constructor
+
+Some models may have some parameter-specific settings for optimization, e.g. weight decay for BatchNorm layers.
+The users can do those fine-grained parameter tuning through customizing optimizer constructor.
+
+```python
+from mmcv.utils import build_from_cfg
+
+from mmcv.runner.optimizer import OPTIMIZER_BUILDERS, OPTIMIZERS
+from mmpose.utils import get_root_logger
+from .my_optimizer import MyOptimizer
+
+
+@OPTIMIZER_BUILDERS.register_module()
+class MyOptimizerConstructor:
+
+ def __init__(self, optimizer_cfg, paramwise_cfg=None):
+ pass
+
+ def __call__(self, model):
+
+ return my_optimizer
+```
+
+The default optimizer constructor is implemented [here](https://github.com/open-mmlab/mmcv/blob/9ecd6b0d5ff9d2172c49a182eaa669e9f27bb8e7/mmcv/runner/optimizer/default_constructor.py#L11),
+which could also serve as a template for new optimizer constructor.
+
+### Additional settings
+
+Tricks not implemented by the optimizer should be implemented through optimizer constructor (e.g., set parameter-wise learning rates) or hooks.
+We list some common settings that could stabilize the training or accelerate the training. Feel free to create PR, issue for more settings.
+
+- __Use gradient clip to stabilize training__:
+ Some models need gradient clip to clip the gradients to stabilize the training process. An example is as below:
+
+ ```python
+ optimizer_config = dict(grad_clip=dict(max_norm=35, norm_type=2))
+ ```
+
+- __Use momentum schedule to accelerate model convergence__:
+ We support momentum scheduler to modify model's momentum according to learning rate, which could make the model converge in a faster way.
+ Momentum scheduler is usually used with LR scheduler, for example, the following config is used in 3D detection to accelerate convergence.
+ For more details, please refer to the implementation of [CyclicLrUpdater](https://github.com/open-mmlab/mmcv/blob/f48241a65aebfe07db122e9db320c31b685dc674/mmcv/runner/hooks/lr_updater.py#L327)
+ and [CyclicMomentumUpdater](https://github.com/open-mmlab/mmcv/blob/f48241a65aebfe07db122e9db320c31b685dc674/mmcv/runner/hooks/momentum_updater.py#L130).
+
+ ```python
+ lr_config = dict(
+ policy='cyclic',
+ target_ratio=(10, 1e-4),
+ cyclic_times=1,
+ step_ratio_up=0.4,
+ )
+ momentum_config = dict(
+ policy='cyclic',
+ target_ratio=(0.85 / 0.95, 1),
+ cyclic_times=1,
+ step_ratio_up=0.4,
+ )
+ ```
+
+## Customize Training Schedules
+
+we use step learning rate with default value in config files, this calls [`StepLRHook`](https://github.com/open-mmlab/mmcv/blob/f48241a65aebfe07db122e9db320c31b685dc674/mmcv/runner/hooks/lr_updater.py#L153) in MMCV.
+We support many other learning rate schedule [here](https://github.com/open-mmlab/mmcv/blob/master/mmcv/runner/hooks/lr_updater.py), such as `CosineAnnealing` and `Poly` schedule. Here are some examples
+
+- Poly schedule:
+
+ ```python
+ lr_config = dict(policy='poly', power=0.9, min_lr=1e-4, by_epoch=False)
+ ```
+
+- ConsineAnnealing schedule:
+
+ ```python
+ lr_config = dict(
+ policy='CosineAnnealing',
+ warmup='linear',
+ warmup_iters=1000,
+ warmup_ratio=1.0 / 10,
+ min_lr_ratio=1e-5)
+ ```
+
+## Customize Workflow
+
+By default, we recommend users to use `EpochEvalHook` to do evaluation after training epoch, but they can still use `val` workflow as an alternative.
+
+Workflow is a list of (phase, epochs) to specify the running order and epochs. By default it is set to be
+
+```python
+workflow = [('train', 1)]
+```
+
+which means running 1 epoch for training.
+Sometimes user may want to check some metrics (e.g. loss, accuracy) about the model on the validate set.
+In such case, we can set the workflow as
+
+```python
+[('train', 1), ('val', 1)]
+```
+
+so that 1 epoch for training and 1 epoch for validation will be run iteratively.
+
+```{note}
+1. The parameters of model will not be updated during val epoch.
+1. Keyword `total_epochs` in the config only controls the number of training epochs and will not affect the validation workflow.
+1. Workflows `[('train', 1), ('val', 1)]` and `[('train', 1)]` will not change the behavior of `EpochEvalHook` because `EpochEvalHook` is called by `after_train_epoch` and validation workflow only affect hooks that are called through `after_val_epoch`.
+ Therefore, the only difference between `[('train', 1), ('val', 1)]` and `[('train', 1)]` is that the runner will calculate losses on validation set after each training epoch.
+```
+
+## Customize Hooks
+
+### Customize self-implemented hooks
+
+#### 1. Implement a new hook
+
+Here we give an example of creating a new hook in MMPose and using it in training.
+
+```python
+from mmcv.runner import HOOKS, Hook
+
+
+@HOOKS.register_module()
+class MyHook(Hook):
+
+ def __init__(self, a, b):
+ pass
+
+ def before_run(self, runner):
+ pass
+
+ def after_run(self, runner):
+ pass
+
+ def before_epoch(self, runner):
+ pass
+
+ def after_epoch(self, runner):
+ pass
+
+ def before_iter(self, runner):
+ pass
+
+ def after_iter(self, runner):
+ pass
+```
+
+Depending on the functionality of the hook, the users need to specify what the hook will do at each stage of the training in `before_run`, `after_run`, `before_epoch`, `after_epoch`, `before_iter`, and `after_iter`.
+
+#### 2. Register the new hook
+
+Then we need to make `MyHook` imported. Assuming the file is in `mmpose/core/utils/my_hook.py` there are two ways to do that:
+
+- Modify `mmpose/core/utils/__init__.py` to import it.
+
+ The newly defined module should be imported in `mmpose/core/utils/__init__.py` so that the registry will
+ find the new module and add it:
+
+```python
+from .my_hook import MyHook
+```
+
+- Use `custom_imports` in the config to manually import it
+
+```python
+custom_imports = dict(imports=['mmpose.core.utils.my_hook'], allow_failed_imports=False)
+```
+
+#### 3. Modify the config
+
+```python
+custom_hooks = [
+ dict(type='MyHook', a=a_value, b=b_value)
+]
+```
+
+You can also set the priority of the hook by adding key `priority` to `'NORMAL'` or `'HIGHEST'` as below
+
+```python
+custom_hooks = [
+ dict(type='MyHook', a=a_value, b=b_value, priority='NORMAL')
+]
+```
+
+By default the hook's priority is set as `NORMAL` during registration.
+
+### Use hooks implemented in MMCV
+
+If the hook is already implemented in MMCV, you can directly modify the config to use the hook as below
+
+```python
+mmcv_hooks = [
+ dict(type='MMCVHook', a=a_value, b=b_value, priority='NORMAL')
+]
+```
+
+### Modify default runtime hooks
+
+There are some common hooks that are not registered through `custom_hooks` but has been registered by default when importing MMCV, they are
+
+- log_config
+- checkpoint_config
+- evaluation
+- lr_config
+- optimizer_config
+- momentum_config
+
+In those hooks, only the logger hook has the `VERY_LOW` priority, others' priority are `NORMAL`.
+The above-mentioned tutorials already cover how to modify `optimizer_config`, `momentum_config`, and `lr_config`.
+Here we reveals how what we can do with `log_config`, `checkpoint_config`, and `evaluation`.
+
+#### Checkpoint config
+
+The MMCV runner will use `checkpoint_config` to initialize [`CheckpointHook`](https://github.com/open-mmlab/mmcv/blob/9ecd6b0d5ff9d2172c49a182eaa669e9f27bb8e7/mmcv/runner/hooks/checkpoint.py#L9).
+
+```python
+checkpoint_config = dict(interval=1)
+```
+
+The users could set `max_keep_ckpts` to only save only small number of checkpoints or decide whether to store state dict of optimizer by `save_optimizer`.
+More details of the arguments are [here](https://mmcv.readthedocs.io/en/latest/api.html#mmcv.runner.CheckpointHook)
+
+#### Log config
+
+The `log_config` wraps multiple logger hooks and enables to set intervals. Now MMCV supports `WandbLoggerHook`, `MlflowLoggerHook`, and `TensorboardLoggerHook`.
+The detail usages can be found in the [doc](https://mmcv.readthedocs.io/en/latest/api.html#mmcv.runner.LoggerHook).
+
+```python
+log_config = dict(
+ interval=50,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ dict(type='TensorboardLoggerHook')
+ ])
+```
+
+#### Evaluation config
+
+The config of `evaluation` will be used to initialize the [`EvalHook`](https://github.com/open-mmlab/mmpose/blob/master/mmpose/core/evaluation/eval_hooks.py#L11).
+Except the key `interval`, other arguments such as `metric` will be passed to the `dataset.evaluate()`
+
+```python
+evaluation = dict(interval=1, metric='mAP')
+```
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/en/useful_tools.md b/engine/pose_estimation/third-party/ViTPose/docs/en/useful_tools.md
new file mode 100644
index 0000000..a9d246d
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/en/useful_tools.md
@@ -0,0 +1,232 @@
+# Useful Tools
+
+Apart from training/testing scripts, We provide lots of useful tools under the `tools/` directory.
+
+
+
+- [Log Analysis](#log-analysis)
+- [Model Complexity (experimental)](#model-complexity-experimental)
+- [Model Conversion](#model-conversion)
+ - [MMPose model to ONNX (experimental)](#mmpose-model-to-onnx-experimental)
+ - [Prepare a model for publishing](#prepare-a-model-for-publishing)
+- [Model Serving](#model-serving)
+- [Miscellaneous](#miscellaneous)
+ - [Evaluating a metric](#evaluating-a-metric)
+ - [Print the entire config](#print-the-entire-config)
+
+
+
+## Log Analysis
+
+`tools/analysis/analyze_logs.py` plots loss/pose acc curves given a training log file. Run `pip install seaborn` first to install the dependency.
+
+
+
+```shell
+python tools/analysis/analyze_logs.py plot_curve ${JSON_LOGS} [--keys ${KEYS}] [--title ${TITLE}] [--legend ${LEGEND}] [--backend ${BACKEND}] [--style ${STYLE}] [--out ${OUT_FILE}]
+```
+
+Examples:
+
+- Plot the mse loss of some run.
+
+ ```shell
+ python tools/analysis/analyze_logs.py plot_curve log.json --keys loss --legend loss
+ ```
+
+- Plot the acc of some run, and save the figure to a pdf.
+
+ ```shell
+ python tools/analysis/analyze_logs.py plot_curve log.json --keys acc_pose --out results.pdf
+ ```
+
+- Compare the acc of two runs in the same figure.
+
+ ```shell
+ python tools/analysis/analyze_logs.py plot_curve log1.json log2.json --keys acc_pose --legend run1 run2
+ ```
+
+You can also compute the average training speed.
+
+```shell
+python tools/analysis/analyze_logs.py cal_train_time ${JSON_LOGS} [--include-outliers]
+```
+
+- Compute the average training speed for a config file
+
+ ```shell
+ python tools/analysis/analyze_logs.py cal_train_time log.json
+ ```
+
+ The output is expected to be like the following.
+
+ ```text
+ -----Analyze train time of log.json-----
+ slowest epoch 114, average time is 0.9662
+ fastest epoch 16, average time is 0.7532
+ time std over epochs is 0.0426
+ average iter time: 0.8406 s/iter
+ ```
+
+## Model Complexity (Experimental)
+
+`/tools/analysis/get_flops.py` is a script adapted from [flops-counter.pytorch](https://github.com/sovrasov/flops-counter.pytorch) to compute the FLOPs and params of a given model.
+
+```shell
+python tools/analysis/get_flops.py ${CONFIG_FILE} [--shape ${INPUT_SHAPE}]
+```
+
+We will get the result like this
+
+```text
+
+==============================
+Input shape: (1, 3, 256, 192)
+Flops: 8.9 GMac
+Params: 28.04 M
+==============================
+```
+
+```{note}
+This tool is still experimental and we do not guarantee that the number is absolutely correct.
+```
+
+You may use the result for simple comparisons, but double check it before you adopt it in technical reports or papers.
+
+(1) FLOPs are related to the input shape while parameters are not. The default input shape is (1, 3, 340, 256) for 2D recognizer, (1, 3, 32, 340, 256) for 3D recognizer.
+(2) Some operators are not counted into FLOPs like GN and custom operators. Refer to [`mmcv.cnn.get_model_complexity_info()`](https://github.com/open-mmlab/mmcv/blob/master/mmcv/cnn/utils/flops_counter.py) for details.
+
+## Model Conversion
+
+### MMPose model to ONNX (experimental)
+
+`/tools/deployment/pytorch2onnx.py` is a script to convert model to [ONNX](https://github.com/onnx/onnx) format.
+It also supports comparing the output results between Pytorch and ONNX model for verification.
+Run `pip install onnx onnxruntime` first to install the dependency.
+
+```shell
+python tools/deployment/pytorch2onnx.py $CONFIG_PATH $CHECKPOINT_PATH --shape $SHAPE --verify
+```
+
+### Prepare a model for publishing
+
+`tools/publish_model.py` helps users to prepare their model for publishing.
+
+Before you upload a model to AWS, you may want to:
+
+(1) convert model weights to CPU tensors.
+(2) delete the optimizer states.
+(3) compute the hash of the checkpoint file and append the hash id to the filename.
+
+```shell
+python tools/publish_model.py ${INPUT_FILENAME} ${OUTPUT_FILENAME}
+```
+
+E.g.,
+
+```shell
+python tools/publish_model.py work_dirs/hrnet_w32_coco_256x192/latest.pth hrnet_w32_coco_256x192
+```
+
+The final output filename will be `hrnet_w32_coco_256x192-{hash id}_{time_stamp}.pth`.
+
+## Model Serving
+
+MMPose supports model serving with [`TorchServe`](https://pytorch.org/serve/). You can serve an MMPose model via following steps:
+
+### 1. Install TorchServe
+
+Please follow the official installation guide of TorchServe: https://github.com/pytorch/serve#install-torchserve-and-torch-model-archiver
+
+### 2. Convert model from MMPose to TorchServe
+
+```shell
+python tools/deployment/mmpose2torchserve.py \
+ ${CONFIG_FILE} ${CHECKPOINT_FILE} \
+ --output-folder ${MODEL_STORE} \
+ --model-name ${MODEL_NAME}
+```
+
+**Note**: ${MODEL_STORE} needs to be an absolute path to a folder.
+
+A model file `${MODEL_NAME}.mar` will be generated and placed in the `${MODEL_STORE}` folder.
+
+### 3. Deploy model serving
+
+We introduce following 2 approaches to deploying the model serving.
+
+#### Use TorchServe API
+
+```shell
+torchserve --start \
+ --model-store ${MODEL_STORE} \
+ --models ${MODEL_PATH1} [${MODEL_NAME}=${MODEL_PATH2} ... ]
+```
+
+Example:
+
+```shell
+# serve one model
+torchserve --start --model-store /models --models hrnet=hrnet.mar
+
+# serve all models in model-store
+torchserve --start --model-store /models --models all
+```
+
+After executing the `torchserve` command above, TorchServe runse on your host, listening for inference requests. Check the [official docs](https://github.com/pytorch/serve/blob/master/docs/server.md) for more information.
+
+#### Use `mmpose-serve` docker image
+
+**Build `mmpose-serve` docker image:**
+
+```shell
+docker build -t mmpose-serve:latest docker/serve/
+```
+
+**Run `mmpose-serve`:**
+
+Check the official docs for [running TorchServe with docker](https://github.com/pytorch/serve/blob/master/docker/README.md#running-torchserve-in-a-production-docker-environment).
+
+In order to run in GPU, you need to install [nvidia-docker](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html). You can omit the `--gpus` argument in order to run in CPU.
+
+Example:
+
+```shell
+docker run --rm \
+--cpus 8 \
+--gpus device=0 \
+-p8080:8080 -p8081:8081 -p8082:8082 \
+--mount type=bind,source=$MODEL_STORE,target=/home/model-server/model-store \
+mmpose-serve:latest
+```
+
+[Read the docs](https://github.com/pytorch/serve/blob/072f5d088cce9bb64b2a18af065886c9b01b317b/docs/rest_api.md/) about the Inference (8080), Management (8081) and Metrics (8082) APis
+
+### 4. Test deployment
+
+You can use `tools/deployment/test_torchserver.py` to test the model serving. It will compare and visualize the result of torchserver and pytorch.
+
+```shell
+python tools/deployment/test_torchserver.py ${IMAGE_PAHT} ${CONFIG_PATH} ${CHECKPOINT_PATH} ${MODEL_NAME} --out-dir ${OUT_DIR}
+```
+
+Example:
+
+```shell
+python tools/deployment/test_torchserver.py \
+ ls tests/data/coco/000000000785.jpg \
+ configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/hrnet_w48_coco_256x192.py \
+ https://download.openmmlab.com/mmpose/top_down/hrnet/hrnet_w48_coco_256x192-b9e0b3ab_20200708.pth \
+ hrnet \
+ --out-dir vis_results
+```
+
+## Miscellaneous
+
+### Print the entire config
+
+`tools/analysis/print_config.py` prints the whole config verbatim, expanding all its imports.
+
+```shell
+python tools/print_config.py ${CONFIG} [-h] [--options ${OPTIONS [OPTIONS...]}]
+```
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/zh_cn/Makefile b/engine/pose_estimation/third-party/ViTPose/docs/zh_cn/Makefile
new file mode 100644
index 0000000..d4bb2cb
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/zh_cn/Makefile
@@ -0,0 +1,20 @@
+# Minimal makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line, and also
+# from the environment for the first two.
+SPHINXOPTS ?=
+SPHINXBUILD ?= sphinx-build
+SOURCEDIR = .
+BUILDDIR = _build
+
+# Put it first so that "make" without argument is like "make help".
+help:
+ @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+.PHONY: help Makefile
+
+# Catch-all target: route all unknown targets to Sphinx using the new
+# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
+%: Makefile
+ @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/zh_cn/_static/css/readthedocs.css b/engine/pose_estimation/third-party/ViTPose/docs/zh_cn/_static/css/readthedocs.css
new file mode 100644
index 0000000..efc4b98
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/zh_cn/_static/css/readthedocs.css
@@ -0,0 +1,6 @@
+.header-logo {
+ background-image: url("../images/mmpose-logo.png");
+ background-size: 120px 50px;
+ height: 50px;
+ width: 120px;
+}
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/zh_cn/_static/images/mmpose-logo.png b/engine/pose_estimation/third-party/ViTPose/docs/zh_cn/_static/images/mmpose-logo.png
new file mode 100644
index 0000000..128e171
Binary files /dev/null and b/engine/pose_estimation/third-party/ViTPose/docs/zh_cn/_static/images/mmpose-logo.png differ
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/zh_cn/api.rst b/engine/pose_estimation/third-party/ViTPose/docs/zh_cn/api.rst
new file mode 100644
index 0000000..2856891
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/zh_cn/api.rst
@@ -0,0 +1,109 @@
+mmpose.apis
+-------------
+.. automodule:: mmpose.apis
+ :members:
+
+
+mmpose.core
+-------------
+evaluation
+^^^^^^^^^^^
+.. automodule:: mmpose.core.evaluation
+ :members:
+
+fp16
+^^^^^^^^^^^
+.. automodule:: mmpose.core.fp16
+ :members:
+
+
+utils
+^^^^^^^^^^^
+.. automodule:: mmpose.core.utils
+ :members:
+
+
+post_processing
+^^^^^^^^^^^^^^^^
+.. automodule:: mmpose.core.post_processing
+ :members:
+
+
+mmpose.models
+---------------
+backbones
+^^^^^^^^^^^
+.. automodule:: mmpose.models.backbones
+ :members:
+
+necks
+^^^^^^^^^^^
+.. automodule:: mmpose.models.necks
+ :members:
+
+detectors
+^^^^^^^^^^^
+.. automodule:: mmpose.models.detectors
+ :members:
+
+heads
+^^^^^^^^^^^^^^^
+.. automodule:: mmpose.models.heads
+ :members:
+
+losses
+^^^^^^^^^^^
+.. automodule:: mmpose.models.losses
+ :members:
+
+misc
+^^^^^^^^^^^
+.. automodule:: mmpose.models.misc
+ :members:
+
+mmpose.datasets
+-----------------
+.. automodule:: mmpose.datasets
+ :members:
+
+datasets
+^^^^^^^^^^^
+.. automodule:: mmpose.datasets.datasets.top_down
+ :members:
+
+.. automodule:: mmpose.datasets.datasets.bottom_up
+ :members:
+
+pipelines
+^^^^^^^^^^^
+.. automodule:: mmpose.datasets.pipelines
+ :members:
+
+.. automodule:: mmpose.datasets.pipelines.loading
+ :members:
+
+.. automodule:: mmpose.datasets.pipelines.shared_transform
+ :members:
+
+.. automodule:: mmpose.datasets.pipelines.top_down_transform
+ :members:
+
+.. automodule:: mmpose.datasets.pipelines.bottom_up_transform
+ :members:
+
+.. automodule:: mmpose.datasets.pipelines.mesh_transform
+ :members:
+
+.. automodule:: mmpose.datasets.pipelines.pose3d_transform
+ :members:
+
+samplers
+^^^^^^^^^^^
+.. automodule:: mmpose.datasets.samplers
+ :members:
+
+
+mmpose.utils
+---------------
+.. automodule:: mmpose.utils
+ :members:
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/zh_cn/benchmark.md b/engine/pose_estimation/third-party/ViTPose/docs/zh_cn/benchmark.md
new file mode 100644
index 0000000..0de8844
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/zh_cn/benchmark.md
@@ -0,0 +1,3 @@
+# 基准测试
+
+内容建设中……
diff --git a/engine/pose_estimation/third-party/ViTPose/docs/zh_cn/collect.py b/engine/pose_estimation/third-party/ViTPose/docs/zh_cn/collect.py
new file mode 100644
index 0000000..5f8aede
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/docs/zh_cn/collect.py
@@ -0,0 +1,101 @@
+#!/usr/bin/env python
+# Copyright (c) OpenMMLab. All rights reserved.
+import os
+import re
+from glob import glob
+
+from titlecase import titlecase
+
+os.makedirs('topics', exist_ok=True)
+os.makedirs('papers', exist_ok=True)
+
+# Step 1: get subtopics: a mix of topic and task
+minisections = [
+ x.split('/')[-2:] for x in glob('../../configs/*/*') if '_base_' not in x
+]
+alltopics = sorted(list(set(x[0] for x in minisections)))
+subtopics = []
+for t in alltopics:
+ data = [x[1].split('_') for x in minisections if x[0] == t]
+ valid_ids = []
+ for i in range(len(data[0])):
+ if len(set(x[i] for x in data)) > 1:
+ valid_ids.append(i)
+ if len(valid_ids) > 0:
+ subtopics.extend([
+ f"{titlecase(t)}({','.join([d[i].title() for i in valid_ids])})",
+ t, '_'.join(d)
+ ] for d in data)
+ else:
+ subtopics.append([titlecase(t), t, '_'.join(data[0])])
+
+contents = {}
+for subtopic, topic, task in sorted(subtopics):
+ # Step 2: get all datasets
+ datasets = sorted(
+ list(
+ set(
+ x.split('/')[-2]
+ for x in glob(f'../../configs/{topic}/{task}/*/*/'))))
+ contents[subtopic] = {d: {} for d in datasets}
+ for dataset in datasets:
+ # Step 3: get all settings: algorithm + backbone + trick
+ for file in glob(f'../../configs/{topic}/{task}/*/{dataset}/*.md'):
+ keywords = (file.split('/')[-3],
+ *file.split('/')[-1].split('_')[:-1])
+ with open(file, 'r') as f:
+ contents[subtopic][dataset][keywords] = f.read()
+
+# Step 4: write files by topic
+for subtopic, datasets in contents.items():
+ lines = [f'# {subtopic}', '']
+ for dataset, keywords in datasets.items():
+ if len(keywords) == 0:
+ continue
+ lines += [
+ '', '
', '', f'## {titlecase(dataset)} Dataset', ''
+ ]
+ for keyword, info in keywords.items():
+ keyword_strs = [titlecase(x.replace('_', ' ')) for x in keyword]
+ lines += [
+ ' ', '',
+ (f'### {" + ".join(keyword_strs)}'
+ f' on {titlecase(dataset)}'), '', info, ''
+ ]
+
+ with open(f'topics/{subtopic.lower()}.md', 'w') as f:
+ f.write('\n'.join(lines))
+
+# Step 5: write files by paper
+allfiles = [x.split('/')[-2:] for x in glob('../en/papers/*/*.md')]
+sections = sorted(list(set(x[0] for x in allfiles)))
+for section in sections:
+ lines = [f'# {titlecase(section)}', '']
+ files = [f for s, f in allfiles if s == section]
+ for file in files:
+ with open(f'../en/papers/{section}/{file}', 'r') as f:
+ keyline = [
+ line for line in f.readlines() if line.startswith('', '', keyline).strip()
+ paperlines = []
+ for subtopic, datasets in contents.items():
+ for dataset, keywords in datasets.items():
+ keywords = {k: v for k, v in keywords.items() if keyline in v}
+ if len(keywords) == 0:
+ continue
+ for keyword, info in keywords.items():
+ keyword_strs = [
+ titlecase(x.replace('_', ' ')) for x in keyword
+ ]
+ paperlines += [
+ ' ', '',
+ (f'### {" + ".join(keyword_strs)}'
+ f' on {titlecase(dataset)}'), '', info, ''
+ ]
+ if len(paperlines) > 0:
+ lines += ['', '
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/body3d_two_stage_img_demo.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/body3d_two_stage_img_demo.py
new file mode 100644
index 0000000..3cc6b0d
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/body3d_two_stage_img_demo.py
@@ -0,0 +1,296 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os
+import os.path as osp
+import warnings
+from argparse import ArgumentParser
+
+import mmcv
+import numpy as np
+from xtcocotools.coco import COCO
+
+from mmpose.apis import (inference_pose_lifter_model,
+ inference_top_down_pose_model, vis_3d_pose_result)
+from mmpose.apis.inference import init_pose_model
+from mmpose.core import SimpleCamera
+from mmpose.datasets import DatasetInfo
+
+
+def _keypoint_camera_to_world(keypoints,
+ camera_params,
+ image_name=None,
+ dataset='Body3DH36MDataset'):
+ """Project 3D keypoints from the camera space to the world space.
+
+ Args:
+ keypoints (np.ndarray): 3D keypoints in shape [..., 3]
+ camera_params (dict): Parameters for all cameras.
+ image_name (str): The image name to specify the camera.
+ dataset (str): The dataset type, e.g. Body3DH36MDataset.
+ """
+ cam_key = None
+ if dataset == 'Body3DH36MDataset':
+ subj, rest = osp.basename(image_name).split('_', 1)
+ _, rest = rest.split('.', 1)
+ camera, rest = rest.split('_', 1)
+ cam_key = (subj, camera)
+ else:
+ raise NotImplementedError
+
+ camera = SimpleCamera(camera_params[cam_key])
+ keypoints_world = keypoints.copy()
+ keypoints_world[..., :3] = camera.camera_to_world(keypoints[..., :3])
+
+ return keypoints_world
+
+
+def main():
+ parser = ArgumentParser()
+ parser.add_argument(
+ 'pose_lifter_config',
+ help='Config file for the 2nd stage pose lifter model')
+ parser.add_argument(
+ 'pose_lifter_checkpoint',
+ help='Checkpoint file for the 2nd stage pose lifter model')
+ parser.add_argument(
+ '--pose-detector-config',
+ type=str,
+ default=None,
+ help='Config file for the 1st stage 2D pose detector')
+ parser.add_argument(
+ '--pose-detector-checkpoint',
+ type=str,
+ default=None,
+ help='Checkpoint file for the 1st stage 2D pose detector')
+ parser.add_argument('--img-root', type=str, default='', help='Image root')
+ parser.add_argument(
+ '--json-file',
+ type=str,
+ default=None,
+ help='Json file containing image and bbox information. Optionally,'
+ 'The Json file can also contain 2D pose information. See'
+ '"only-second-stage"')
+ parser.add_argument(
+ '--camera-param-file',
+ type=str,
+ default=None,
+ help='Camera parameter file for converting 3D pose predictions from '
+ ' the camera space to to world space. If None, no conversion will be '
+ 'applied.')
+ parser.add_argument(
+ '--only-second-stage',
+ action='store_true',
+ help='If true, load 2D pose detection result from the Json file and '
+ 'skip the 1st stage. The pose detection model will be ignored.')
+ parser.add_argument(
+ '--rebase-keypoint-height',
+ action='store_true',
+ help='Rebase the predicted 3D pose so its lowest keypoint has a '
+ 'height of 0 (landing on the ground). This is useful for '
+ 'visualization when the model do not predict the global position '
+ 'of the 3D pose.')
+ parser.add_argument(
+ '--show-ground-truth',
+ action='store_true',
+ help='If True, show ground truth if it is available. The ground truth '
+ 'should be contained in the annotations in the Json file with the key '
+ '"keypoints_3d" for each instance.')
+ parser.add_argument(
+ '--show',
+ action='store_true',
+ default=False,
+ help='whether to show img')
+ parser.add_argument(
+ '--out-img-root',
+ type=str,
+ default=None,
+ help='Root of the output visualization images. '
+ 'Default not saving the visualization images.')
+ parser.add_argument(
+ '--device', default='cuda:0', help='Device for inference')
+ parser.add_argument('--kpt-thr', type=float, default=0.3)
+ parser.add_argument(
+ '--radius',
+ type=int,
+ default=4,
+ help='Keypoint radius for visualization')
+ parser.add_argument(
+ '--thickness',
+ type=int,
+ default=1,
+ help='Link thickness for visualization')
+
+ args = parser.parse_args()
+ assert args.show or (args.out_img_root != '')
+
+ coco = COCO(args.json_file)
+
+ # First stage: 2D pose detection
+ pose_det_results_list = []
+ if args.only_second_stage:
+ from mmpose.apis.inference import _xywh2xyxy
+
+ print('Stage 1: load 2D pose results from Json file.')
+ for image_id, image in coco.imgs.items():
+ image_name = osp.join(args.img_root, image['file_name'])
+ ann_ids = coco.getAnnIds(image_id)
+ pose_det_results = []
+ for ann_id in ann_ids:
+ ann = coco.anns[ann_id]
+ keypoints = np.array(ann['keypoints']).reshape(-1, 3)
+ keypoints[..., 2] = keypoints[..., 2] >= 1
+ keypoints_3d = np.array(ann['keypoints_3d']).reshape(-1, 4)
+ keypoints_3d[..., 3] = keypoints_3d[..., 3] >= 1
+ bbox = np.array(ann['bbox']).reshape(1, -1)
+
+ pose_det_result = {
+ 'image_name': image_name,
+ 'bbox': _xywh2xyxy(bbox),
+ 'keypoints': keypoints,
+ 'keypoints_3d': keypoints_3d
+ }
+ pose_det_results.append(pose_det_result)
+ pose_det_results_list.append(pose_det_results)
+
+ else:
+ print('Stage 1: 2D pose detection.')
+
+ pose_det_model = init_pose_model(
+ args.pose_detector_config,
+ args.pose_detector_checkpoint,
+ device=args.device.lower())
+
+ assert pose_det_model.cfg.model.type == 'TopDown', 'Only "TopDown"' \
+ 'model is supported for the 1st stage (2D pose detection)'
+
+ dataset = pose_det_model.cfg.data['test']['type']
+ dataset_info = pose_det_model.cfg.data['test'].get(
+ 'dataset_info', None)
+ if dataset_info is None:
+ warnings.warn(
+ 'Please set `dataset_info` in the config.'
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 '
+ 'for details.', DeprecationWarning)
+ else:
+ dataset_info = DatasetInfo(dataset_info)
+
+ img_keys = list(coco.imgs.keys())
+
+ for i in mmcv.track_iter_progress(range(len(img_keys))):
+ # get bounding box annotations
+ image_id = img_keys[i]
+ image = coco.loadImgs(image_id)[0]
+ image_name = osp.join(args.img_root, image['file_name'])
+ ann_ids = coco.getAnnIds(image_id)
+
+ # make person results for single image
+ person_results = []
+ for ann_id in ann_ids:
+ person = {}
+ ann = coco.anns[ann_id]
+ person['bbox'] = ann['bbox']
+ person_results.append(person)
+
+ pose_det_results, _ = inference_top_down_pose_model(
+ pose_det_model,
+ image_name,
+ person_results,
+ bbox_thr=None,
+ format='xywh',
+ dataset=dataset,
+ dataset_info=dataset_info,
+ return_heatmap=False,
+ outputs=None)
+
+ for res in pose_det_results:
+ res['image_name'] = image_name
+ pose_det_results_list.append(pose_det_results)
+
+ # Second stage: Pose lifting
+ print('Stage 2: 2D-to-3D pose lifting.')
+
+ pose_lift_model = init_pose_model(
+ args.pose_lifter_config,
+ args.pose_lifter_checkpoint,
+ device=args.device.lower())
+
+ assert pose_lift_model.cfg.model.type == 'PoseLifter', 'Only' \
+ '"PoseLifter" model is supported for the 2nd stage ' \
+ '(2D-to-3D lifting)'
+ dataset = pose_lift_model.cfg.data['test']['type']
+ dataset_info = pose_lift_model.cfg.data['test'].get('dataset_info', None)
+ if dataset_info is None:
+ warnings.warn(
+ 'Please set `dataset_info` in the config.'
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 for details.',
+ DeprecationWarning)
+ else:
+ dataset_info = DatasetInfo(dataset_info)
+
+ camera_params = None
+ if args.camera_param_file is not None:
+ camera_params = mmcv.load(args.camera_param_file)
+
+ for i, pose_det_results in enumerate(
+ mmcv.track_iter_progress(pose_det_results_list)):
+ # 2D-to-3D pose lifting
+ # Note that the pose_det_results are regarded as a single-frame pose
+ # sequence
+ pose_lift_results = inference_pose_lifter_model(
+ pose_lift_model,
+ pose_results_2d=[pose_det_results],
+ dataset=dataset,
+ dataset_info=dataset_info,
+ with_track_id=False)
+
+ image_name = pose_det_results[0]['image_name']
+
+ # Pose processing
+ pose_lift_results_vis = []
+ for idx, res in enumerate(pose_lift_results):
+ keypoints_3d = res['keypoints_3d']
+ # project to world space
+ if camera_params is not None:
+ keypoints_3d = _keypoint_camera_to_world(
+ keypoints_3d,
+ camera_params=camera_params,
+ image_name=image_name,
+ dataset=dataset)
+ # rebase height (z-axis)
+ if args.rebase_keypoint_height:
+ keypoints_3d[..., 2] -= np.min(
+ keypoints_3d[..., 2], axis=-1, keepdims=True)
+ res['keypoints_3d'] = keypoints_3d
+ # Add title
+ det_res = pose_det_results[idx]
+ instance_id = det_res.get('track_id', idx)
+ res['title'] = f'Prediction ({instance_id})'
+ pose_lift_results_vis.append(res)
+ # Add ground truth
+ if args.show_ground_truth:
+ if 'keypoints_3d' not in det_res:
+ print('Fail to show ground truth. Please make sure that'
+ ' the instance annotations from the Json file'
+ ' contain "keypoints_3d".')
+ else:
+ gt = res.copy()
+ gt['keypoints_3d'] = det_res['keypoints_3d']
+ gt['title'] = f'Ground truth ({instance_id})'
+ pose_lift_results_vis.append(gt)
+
+ # Visualization
+ if args.out_img_root is None:
+ out_file = None
+ else:
+ os.makedirs(args.out_img_root, exist_ok=True)
+ out_file = osp.join(args.out_img_root, f'vis_{i}.jpg')
+
+ vis_3d_pose_result(
+ pose_lift_model,
+ result=pose_lift_results_vis,
+ img=image_name,
+ dataset_info=dataset_info,
+ out_file=out_file)
+
+
+if __name__ == '__main__':
+ main()
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/body3d_two_stage_video_demo.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/body3d_two_stage_video_demo.py
new file mode 100644
index 0000000..5f47f62
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/body3d_two_stage_video_demo.py
@@ -0,0 +1,307 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import copy
+import os
+import os.path as osp
+from argparse import ArgumentParser
+
+import cv2
+import mmcv
+import numpy as np
+
+from mmpose.apis import (extract_pose_sequence, get_track_id,
+ inference_pose_lifter_model,
+ inference_top_down_pose_model, init_pose_model,
+ process_mmdet_results, vis_3d_pose_result)
+
+try:
+ from mmdet.apis import inference_detector, init_detector
+
+ has_mmdet = True
+except (ImportError, ModuleNotFoundError):
+ has_mmdet = False
+
+
+def covert_keypoint_definition(keypoints, pose_det_dataset, pose_lift_dataset):
+ """Convert pose det dataset keypoints definition to pose lifter dataset
+ keypoints definition.
+
+ Args:
+ keypoints (ndarray[K, 2 or 3]): 2D keypoints to be transformed.
+ pose_det_dataset, (str): Name of the dataset for 2D pose detector.
+ pose_lift_dataset (str): Name of the dataset for pose lifter model.
+ """
+ if pose_det_dataset == 'TopDownH36MDataset' and \
+ pose_lift_dataset == 'Body3DH36MDataset':
+ return keypoints
+ elif pose_det_dataset == 'TopDownCocoDataset' and \
+ pose_lift_dataset == 'Body3DH36MDataset':
+ keypoints_new = np.zeros((17, keypoints.shape[1]))
+ # pelvis is in the middle of l_hip and r_hip
+ keypoints_new[0] = (keypoints[11] + keypoints[12]) / 2
+ # thorax is in the middle of l_shoulder and r_shoulder
+ keypoints_new[8] = (keypoints[5] + keypoints[6]) / 2
+ # head is in the middle of l_eye and r_eye
+ keypoints_new[10] = (keypoints[1] + keypoints[2]) / 2
+ # spine is in the middle of thorax and pelvis
+ keypoints_new[7] = (keypoints_new[0] + keypoints_new[8]) / 2
+ # rearrange other keypoints
+ keypoints_new[[1, 2, 3, 4, 5, 6, 9, 11, 12, 13, 14, 15, 16]] = \
+ keypoints[[12, 14, 16, 11, 13, 15, 0, 5, 7, 9, 6, 8, 10]]
+ return keypoints_new
+ else:
+ raise NotImplementedError
+
+
+def main():
+ parser = ArgumentParser()
+ parser.add_argument('det_config', help='Config file for detection')
+ parser.add_argument('det_checkpoint', help='Checkpoint file for detection')
+ parser.add_argument(
+ 'pose_detector_config',
+ type=str,
+ default=None,
+ help='Config file for the 1st stage 2D pose detector')
+ parser.add_argument(
+ 'pose_detector_checkpoint',
+ type=str,
+ default=None,
+ help='Checkpoint file for the 1st stage 2D pose detector')
+ parser.add_argument(
+ 'pose_lifter_config',
+ help='Config file for the 2nd stage pose lifter model')
+ parser.add_argument(
+ 'pose_lifter_checkpoint',
+ help='Checkpoint file for the 2nd stage pose lifter model')
+ parser.add_argument(
+ '--video-path', type=str, default='', help='Video path')
+ parser.add_argument(
+ '--rebase-keypoint-height',
+ action='store_true',
+ help='Rebase the predicted 3D pose so its lowest keypoint has a '
+ 'height of 0 (landing on the ground). This is useful for '
+ 'visualization when the model do not predict the global position '
+ 'of the 3D pose.')
+ parser.add_argument(
+ '--norm-pose-2d',
+ action='store_true',
+ help='Scale the bbox (along with the 2D pose) to the average bbox '
+ 'scale of the dataset, and move the bbox (along with the 2D pose) to '
+ 'the average bbox center of the dataset. This is useful when bbox '
+ 'is small, especially in multi-person scenarios.')
+ parser.add_argument(
+ '--num-instances',
+ type=int,
+ default=-1,
+ help='The number of 3D poses to be visualized in every frame. If '
+ 'less than 0, it will be set to the number of pose results in the '
+ 'first frame.')
+ parser.add_argument(
+ '--show',
+ action='store_true',
+ default=False,
+ help='whether to show visualizations.')
+ parser.add_argument(
+ '--out-video-root',
+ type=str,
+ default=None,
+ help='Root of the output video file. '
+ 'Default not saving the visualization video.')
+ parser.add_argument(
+ '--device', default='cuda:0', help='Device for inference')
+ parser.add_argument(
+ '--det-cat-id',
+ type=int,
+ default=1,
+ help='Category id for bounding box detection model')
+ parser.add_argument(
+ '--bbox-thr',
+ type=float,
+ default=0.9,
+ help='Bounding box score threshold')
+ parser.add_argument('--kpt-thr', type=float, default=0.3)
+ parser.add_argument(
+ '--use-oks-tracking', action='store_true', help='Using OKS tracking')
+ parser.add_argument(
+ '--tracking-thr', type=float, default=0.3, help='Tracking threshold')
+ parser.add_argument(
+ '--euro',
+ action='store_true',
+ help='Using One_Euro_Filter for smoothing')
+ parser.add_argument(
+ '--radius',
+ type=int,
+ default=8,
+ help='Keypoint radius for visualization')
+ parser.add_argument(
+ '--thickness',
+ type=int,
+ default=2,
+ help='Link thickness for visualization')
+
+ assert has_mmdet, 'Please install mmdet to run the demo.'
+
+ args = parser.parse_args()
+ assert args.show or (args.out_video_root != '')
+ assert args.det_config is not None
+ assert args.det_checkpoint is not None
+
+ video = mmcv.VideoReader(args.video_path)
+ assert video.opened, f'Failed to load video file {args.video_path}'
+
+ # First stage: 2D pose detection
+ print('Stage 1: 2D pose detection.')
+
+ person_det_model = init_detector(
+ args.det_config, args.det_checkpoint, device=args.device.lower())
+
+ pose_det_model = init_pose_model(
+ args.pose_detector_config,
+ args.pose_detector_checkpoint,
+ device=args.device.lower())
+
+ assert pose_det_model.cfg.model.type == 'TopDown', 'Only "TopDown"' \
+ 'model is supported for the 1st stage (2D pose detection)'
+
+ pose_det_dataset = pose_det_model.cfg.data['test']['type']
+
+ pose_det_results_list = []
+ next_id = 0
+ pose_det_results = []
+ for frame in video:
+ pose_det_results_last = pose_det_results
+
+ # test a single image, the resulting box is (x1, y1, x2, y2)
+ mmdet_results = inference_detector(person_det_model, frame)
+
+ # keep the person class bounding boxes.
+ person_det_results = process_mmdet_results(mmdet_results,
+ args.det_cat_id)
+
+ # make person results for single image
+ pose_det_results, _ = inference_top_down_pose_model(
+ pose_det_model,
+ frame,
+ person_det_results,
+ bbox_thr=args.bbox_thr,
+ format='xyxy',
+ dataset=pose_det_dataset,
+ return_heatmap=False,
+ outputs=None)
+
+ # get track id for each person instance
+ pose_det_results, next_id = get_track_id(
+ pose_det_results,
+ pose_det_results_last,
+ next_id,
+ use_oks=args.use_oks_tracking,
+ tracking_thr=args.tracking_thr,
+ use_one_euro=args.euro,
+ fps=video.fps)
+
+ pose_det_results_list.append(copy.deepcopy(pose_det_results))
+
+ # Second stage: Pose lifting
+ print('Stage 2: 2D-to-3D pose lifting.')
+
+ pose_lift_model = init_pose_model(
+ args.pose_lifter_config,
+ args.pose_lifter_checkpoint,
+ device=args.device.lower())
+
+ assert pose_lift_model.cfg.model.type == 'PoseLifter', \
+ 'Only "PoseLifter" model is supported for the 2nd stage ' \
+ '(2D-to-3D lifting)'
+ pose_lift_dataset = pose_lift_model.cfg.data['test']['type']
+
+ if args.out_video_root == '':
+ save_out_video = False
+ else:
+ os.makedirs(args.out_video_root, exist_ok=True)
+ save_out_video = True
+
+ if save_out_video:
+ fourcc = cv2.VideoWriter_fourcc(*'mp4v')
+ fps = video.fps
+ writer = None
+
+ # convert keypoint definition
+ for pose_det_results in pose_det_results_list:
+ for res in pose_det_results:
+ keypoints = res['keypoints']
+ res['keypoints'] = covert_keypoint_definition(
+ keypoints, pose_det_dataset, pose_lift_dataset)
+
+ # load temporal padding config from model.data_cfg
+ if hasattr(pose_lift_model.cfg, 'test_data_cfg'):
+ data_cfg = pose_lift_model.cfg.test_data_cfg
+ else:
+ data_cfg = pose_lift_model.cfg.data_cfg
+
+ num_instances = args.num_instances
+ for i, pose_det_results in enumerate(
+ mmcv.track_iter_progress(pose_det_results_list)):
+ # extract and pad input pose2d sequence
+ pose_results_2d = extract_pose_sequence(
+ pose_det_results_list,
+ frame_idx=i,
+ causal=data_cfg.causal,
+ seq_len=data_cfg.seq_len,
+ step=data_cfg.seq_frame_interval)
+ # 2D-to-3D pose lifting
+ pose_lift_results = inference_pose_lifter_model(
+ pose_lift_model,
+ pose_results_2d=pose_results_2d,
+ dataset=pose_lift_dataset,
+ with_track_id=True,
+ image_size=video.resolution,
+ norm_pose_2d=args.norm_pose_2d)
+
+ # Pose processing
+ pose_lift_results_vis = []
+ for idx, res in enumerate(pose_lift_results):
+ keypoints_3d = res['keypoints_3d']
+ # exchange y,z-axis, and then reverse the direction of x,z-axis
+ keypoints_3d = keypoints_3d[..., [0, 2, 1]]
+ keypoints_3d[..., 0] = -keypoints_3d[..., 0]
+ keypoints_3d[..., 2] = -keypoints_3d[..., 2]
+ # rebase height (z-axis)
+ if args.rebase_keypoint_height:
+ keypoints_3d[..., 2] -= np.min(
+ keypoints_3d[..., 2], axis=-1, keepdims=True)
+ res['keypoints_3d'] = keypoints_3d
+ # add title
+ det_res = pose_det_results[idx]
+ instance_id = det_res['track_id']
+ res['title'] = f'Prediction ({instance_id})'
+ # only visualize the target frame
+ res['keypoints'] = det_res['keypoints']
+ res['bbox'] = det_res['bbox']
+ res['track_id'] = instance_id
+ pose_lift_results_vis.append(res)
+
+ # Visualization
+ if num_instances < 0:
+ num_instances = len(pose_lift_results_vis)
+ img_vis = vis_3d_pose_result(
+ pose_lift_model,
+ result=pose_lift_results_vis,
+ img=video[i],
+ out_file=None,
+ radius=args.radius,
+ thickness=args.thickness,
+ num_instances=num_instances)
+
+ if save_out_video:
+ if writer is None:
+ writer = cv2.VideoWriter(
+ osp.join(args.out_video_root,
+ f'vis_{osp.basename(args.video_path)}'), fourcc,
+ fps, (img_vis.shape[1], img_vis.shape[0]))
+ writer.write(img_vis)
+
+ if save_out_video:
+ writer.release()
+
+
+if __name__ == '__main__':
+ main()
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/bottom_up_img_demo.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/bottom_up_img_demo.py
new file mode 100644
index 0000000..ae343ac
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/bottom_up_img_demo.py
@@ -0,0 +1,127 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os
+import os.path as osp
+import warnings
+from argparse import ArgumentParser
+
+import mmcv
+
+from mmpose.apis import (inference_bottom_up_pose_model, init_pose_model,
+ vis_pose_result)
+from mmpose.datasets import DatasetInfo
+
+
+def main():
+ """Visualize the demo images."""
+ parser = ArgumentParser()
+ parser.add_argument('pose_config', help='Config file for detection')
+ parser.add_argument('pose_checkpoint', help='Checkpoint file')
+ parser.add_argument(
+ '--img-path',
+ type=str,
+ help='Path to an image file or a image folder.')
+ parser.add_argument(
+ '--show',
+ action='store_true',
+ default=False,
+ help='whether to show img')
+ parser.add_argument(
+ '--out-img-root',
+ type=str,
+ default='',
+ help='Root of the output img file. '
+ 'Default not saving the visualization images.')
+ parser.add_argument(
+ '--device', default='cuda:0', help='Device used for inference')
+ parser.add_argument(
+ '--kpt-thr', type=float, default=0.3, help='Keypoint score threshold')
+ parser.add_argument(
+ '--pose-nms-thr',
+ type=float,
+ default=0.9,
+ help='OKS threshold for pose NMS')
+ parser.add_argument(
+ '--radius',
+ type=int,
+ default=4,
+ help='Keypoint radius for visualization')
+ parser.add_argument(
+ '--thickness',
+ type=int,
+ default=1,
+ help='Link thickness for visualization')
+
+ args = parser.parse_args()
+
+ assert args.show or (args.out_img_root != '')
+
+ # prepare image list
+ if osp.isfile(args.img_path):
+ image_list = [args.img_path]
+ elif osp.isdir(args.img_path):
+ image_list = [
+ osp.join(args.img_path, fn) for fn in os.listdir(args.img_path)
+ if fn.lower().endswith(('.png', '.jpg', '.jpeg', '.tiff', '.bmp'))
+ ]
+ else:
+ raise ValueError('Image path should be an image or image folder.'
+ f'Got invalid image path: {args.img_path}')
+
+ # build the pose model from a config file and a checkpoint file
+ pose_model = init_pose_model(
+ args.pose_config, args.pose_checkpoint, device=args.device.lower())
+
+ dataset = pose_model.cfg.data['test']['type']
+ dataset_info = pose_model.cfg.data['test'].get('dataset_info', None)
+ if dataset_info is None:
+ warnings.warn(
+ 'Please set `dataset_info` in the config.'
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 for details.',
+ DeprecationWarning)
+ assert (dataset == 'BottomUpCocoDataset')
+ else:
+ dataset_info = DatasetInfo(dataset_info)
+
+ # optional
+ return_heatmap = False
+
+ # e.g. use ('backbone', ) to return backbone feature
+ output_layer_names = None
+
+ # process each image
+ for image_name in mmcv.track_iter_progress(image_list):
+
+ # test a single image, with a list of bboxes.
+ pose_results, returned_outputs = inference_bottom_up_pose_model(
+ pose_model,
+ image_name,
+ dataset=dataset,
+ dataset_info=dataset_info,
+ pose_nms_thr=args.pose_nms_thr,
+ return_heatmap=return_heatmap,
+ outputs=output_layer_names)
+
+ if args.out_img_root == '':
+ out_file = None
+ else:
+ os.makedirs(args.out_img_root, exist_ok=True)
+ out_file = os.path.join(
+ args.out_img_root,
+ f'vis_{osp.splitext(osp.basename(image_name))[0]}.jpg')
+
+ # show the results
+ vis_pose_result(
+ pose_model,
+ image_name,
+ pose_results,
+ radius=args.radius,
+ thickness=args.thickness,
+ dataset=dataset,
+ dataset_info=dataset_info,
+ kpt_score_thr=args.kpt_thr,
+ show=args.show,
+ out_file=out_file)
+
+
+if __name__ == '__main__':
+ main()
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/bottom_up_pose_tracking_demo.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/bottom_up_pose_tracking_demo.py
new file mode 100644
index 0000000..b79e1f4
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/bottom_up_pose_tracking_demo.py
@@ -0,0 +1,158 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os
+import warnings
+from argparse import ArgumentParser
+
+import cv2
+
+from mmpose.apis import (get_track_id, inference_bottom_up_pose_model,
+ init_pose_model, vis_pose_tracking_result)
+from mmpose.datasets import DatasetInfo
+
+
+def main():
+ """Visualize the demo images."""
+ parser = ArgumentParser()
+ parser.add_argument('pose_config', help='Config file for pose')
+ parser.add_argument('pose_checkpoint', help='Checkpoint file for pose')
+ parser.add_argument('--video-path', type=str, help='Video path')
+ parser.add_argument(
+ '--show',
+ action='store_true',
+ default=False,
+ help='whether to show visualizations.')
+ parser.add_argument(
+ '--out-video-root',
+ default='',
+ help='Root of the output video file. '
+ 'Default not saving the visualization video.')
+ parser.add_argument(
+ '--device', default='cuda:0', help='Device used for inference')
+ parser.add_argument(
+ '--kpt-thr', type=float, default=0.5, help='Keypoint score threshold')
+ parser.add_argument(
+ '--pose-nms-thr',
+ type=float,
+ default=0.9,
+ help='OKS threshold for pose NMS')
+ parser.add_argument(
+ '--use-oks-tracking', action='store_true', help='Using OKS tracking')
+ parser.add_argument(
+ '--tracking-thr', type=float, default=0.3, help='Tracking threshold')
+ parser.add_argument(
+ '--euro',
+ action='store_true',
+ help='Using One_Euro_Filter for smoothing')
+ parser.add_argument(
+ '--radius',
+ type=int,
+ default=4,
+ help='Keypoint radius for visualization')
+ parser.add_argument(
+ '--thickness',
+ type=int,
+ default=1,
+ help='Link thickness for visualization')
+
+ args = parser.parse_args()
+
+ assert args.show or (args.out_video_root != '')
+
+ # build the pose model from a config file and a checkpoint file
+ pose_model = init_pose_model(
+ args.pose_config, args.pose_checkpoint, device=args.device.lower())
+
+ dataset = pose_model.cfg.data['test']['type']
+ dataset_info = pose_model.cfg.data['test'].get('dataset_info', None)
+ if dataset_info is None:
+ warnings.warn(
+ 'Please set `dataset_info` in the config.'
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 for details.',
+ DeprecationWarning)
+ assert (dataset == 'BottomUpCocoDataset')
+ else:
+ dataset_info = DatasetInfo(dataset_info)
+
+ cap = cv2.VideoCapture(args.video_path)
+ fps = None
+
+ assert cap.isOpened(), f'Faild to load video file {args.video_path}'
+
+ if args.out_video_root == '':
+ save_out_video = False
+ else:
+ os.makedirs(args.out_video_root, exist_ok=True)
+ save_out_video = True
+
+ if save_out_video:
+ fps = cap.get(cv2.CAP_PROP_FPS)
+ size = (int(cap.get(cv2.CAP_PROP_FRAME_WIDTH)),
+ int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT)))
+ fourcc = cv2.VideoWriter_fourcc(*'mp4v')
+ videoWriter = cv2.VideoWriter(
+ os.path.join(args.out_video_root,
+ f'vis_{os.path.basename(args.video_path)}'), fourcc,
+ fps, size)
+
+ # optional
+ return_heatmap = False
+
+ # e.g. use ('backbone', ) to return backbone feature
+ output_layer_names = None
+ next_id = 0
+ pose_results = []
+ while (cap.isOpened()):
+ flag, img = cap.read()
+ if not flag:
+ break
+ pose_results_last = pose_results
+
+ pose_results, returned_outputs = inference_bottom_up_pose_model(
+ pose_model,
+ img,
+ dataset=dataset,
+ dataset_info=dataset_info,
+ pose_nms_thr=args.pose_nms_thr,
+ return_heatmap=return_heatmap,
+ outputs=output_layer_names)
+
+ # get track id for each person instance
+ pose_results, next_id = get_track_id(
+ pose_results,
+ pose_results_last,
+ next_id,
+ use_oks=args.use_oks_tracking,
+ tracking_thr=args.tracking_thr,
+ use_one_euro=args.euro,
+ fps=fps)
+
+ # show the results
+ vis_img = vis_pose_tracking_result(
+ pose_model,
+ img,
+ pose_results,
+ radius=args.radius,
+ thickness=args.thickness,
+ dataset=dataset,
+ dataset_info=dataset_info,
+ kpt_score_thr=args.kpt_thr,
+ show=False)
+
+ if args.show:
+ cv2.imshow('Image', vis_img)
+
+ if save_out_video:
+ videoWriter.write(vis_img)
+
+ if args.show and cv2.waitKey(1) & 0xFF == ord('q'):
+ break
+
+ cap.release()
+ if save_out_video:
+ videoWriter.release()
+ if args.show:
+ cv2.destroyAllWindows()
+
+
+if __name__ == '__main__':
+ main()
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/bottom_up_video_demo.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/bottom_up_video_demo.py
new file mode 100644
index 0000000..14785a0
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/bottom_up_video_demo.py
@@ -0,0 +1,135 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os
+import warnings
+from argparse import ArgumentParser
+
+import cv2
+
+from mmpose.apis import (inference_bottom_up_pose_model, init_pose_model,
+ vis_pose_result)
+from mmpose.datasets import DatasetInfo
+
+
+def main():
+ """Visualize the demo images."""
+ parser = ArgumentParser()
+ parser.add_argument('pose_config', help='Config file for pose')
+ parser.add_argument('pose_checkpoint', help='Checkpoint file for pose')
+ parser.add_argument('--video-path', type=str, help='Video path')
+ parser.add_argument(
+ '--show',
+ action='store_true',
+ default=False,
+ help='whether to show visualizations.')
+ parser.add_argument(
+ '--out-video-root',
+ default='',
+ help='Root of the output video file. '
+ 'Default not saving the visualization video.')
+ parser.add_argument(
+ '--device', default='cuda:0', help='Device used for inference')
+ parser.add_argument(
+ '--kpt-thr', type=float, default=0.3, help='Keypoint score threshold')
+ parser.add_argument(
+ '--pose-nms-thr',
+ type=float,
+ default=0.9,
+ help='OKS threshold for pose NMS')
+ parser.add_argument(
+ '--radius',
+ type=int,
+ default=4,
+ help='Keypoint radius for visualization')
+ parser.add_argument(
+ '--thickness',
+ type=int,
+ default=1,
+ help='Link thickness for visualization')
+
+ args = parser.parse_args()
+
+ assert args.show or (args.out_video_root != '')
+
+ # build the pose model from a config file and a checkpoint file
+ pose_model = init_pose_model(
+ args.pose_config, args.pose_checkpoint, device=args.device.lower())
+
+ dataset = pose_model.cfg.data['test']['type']
+ dataset_info = pose_model.cfg.data['test'].get('dataset_info', None)
+ if dataset_info is None:
+ warnings.warn(
+ 'Please set `dataset_info` in the config.'
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 for details.',
+ DeprecationWarning)
+ assert (dataset == 'BottomUpCocoDataset')
+ else:
+ dataset_info = DatasetInfo(dataset_info)
+
+ cap = cv2.VideoCapture(args.video_path)
+
+ if args.out_video_root == '':
+ save_out_video = False
+ else:
+ os.makedirs(args.out_video_root, exist_ok=True)
+ save_out_video = True
+
+ if save_out_video:
+ fps = cap.get(cv2.CAP_PROP_FPS)
+ size = (int(cap.get(cv2.CAP_PROP_FRAME_WIDTH)),
+ int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT)))
+ fourcc = cv2.VideoWriter_fourcc(*'mp4v')
+ videoWriter = cv2.VideoWriter(
+ os.path.join(args.out_video_root,
+ f'vis_{os.path.basename(args.video_path)}'), fourcc,
+ fps, size)
+
+ # optional
+ return_heatmap = False
+
+ # e.g. use ('backbone', ) to return backbone feature
+ output_layer_names = None
+
+ while (cap.isOpened()):
+ flag, img = cap.read()
+ if not flag:
+ break
+
+ pose_results, returned_outputs = inference_bottom_up_pose_model(
+ pose_model,
+ img,
+ dataset=dataset,
+ dataset_info=dataset_info,
+ pose_nms_thr=args.pose_nms_thr,
+ return_heatmap=return_heatmap,
+ outputs=output_layer_names)
+
+ # show the results
+ vis_img = vis_pose_result(
+ pose_model,
+ img,
+ pose_results,
+ radius=args.radius,
+ thickness=args.thickness,
+ dataset=dataset,
+ dataset_info=dataset_info,
+ kpt_score_thr=args.kpt_thr,
+ show=False)
+
+ if args.show:
+ cv2.imshow('Image', vis_img)
+
+ if save_out_video:
+ videoWriter.write(vis_img)
+
+ if args.show and cv2.waitKey(1) & 0xFF == ord('q'):
+ break
+
+ cap.release()
+ if save_out_video:
+ videoWriter.release()
+ if args.show:
+ cv2.destroyAllWindows()
+
+
+if __name__ == '__main__':
+ main()
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/docs/2d_animal_demo.md b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/docs/2d_animal_demo.md
new file mode 100644
index 0000000..bb994e8
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/docs/2d_animal_demo.md
@@ -0,0 +1,148 @@
+## 2D Animal Pose Demo
+
+### 2D Animal Pose Image Demo
+
+#### Using gt hand bounding boxes as input
+
+We provide a demo script to test a single image, given gt json file.
+
+*Pose Model Preparation:*
+The pre-trained pose estimation model can be downloaded from [model zoo](https://mmpose.readthedocs.io/en/latest/topics/animal.html).
+Take [macaque model](https://download.openmmlab.com/mmpose/animal/resnet/res50_macaque_256x192-98f1dd3a_20210407.pth) as an example:
+
+```shell
+python demo/top_down_img_demo.py \
+ ${MMPOSE_CONFIG_FILE} ${MMPOSE_CHECKPOINT_FILE} \
+ --img-root ${IMG_ROOT} --json-file ${JSON_FILE} \
+ --out-img-root ${OUTPUT_DIR} \
+ [--show --device ${GPU_ID or CPU}] \
+ [--kpt-thr ${KPT_SCORE_THR}]
+```
+
+Examples:
+
+```shell
+python demo/top_down_img_demo.py \
+ configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap/macaque/res50_macaque_256x192.py \
+ https://download.openmmlab.com/mmpose/animal/resnet/res50_macaque_256x192-98f1dd3a_20210407.pth \
+ --img-root tests/data/macaque/ --json-file tests/data/macaque/test_macaque.json \
+ --out-img-root vis_results
+```
+
+To run demos on CPU:
+
+```shell
+python demo/top_down_img_demo.py \
+ configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap/macaque/res50_macaque_256x192.py \
+ https://download.openmmlab.com/mmpose/animal/resnet/res50_macaque_256x192-98f1dd3a_20210407.pth \
+ --img-root tests/data/macaque/ --json-file tests/data/macaque/test_macaque.json \
+ --out-img-root vis_results \
+ --device=cpu
+```
+
+### 2D Animal Pose Video Demo
+
+We also provide video demos to illustrate the results.
+
+#### Using the full image as input
+
+If the video is cropped with the object centered in the screen, we can simply use the full image as the model input (without object detection).
+
+```shell
+python demo/top_down_video_demo_full_frame_without_det.py \
+ ${MMPOSE_CONFIG_FILE} ${MMPOSE_CHECKPOINT_FILE} \
+ --video-path ${VIDEO_FILE} \
+ --out-video-root ${OUTPUT_VIDEO_ROOT} \
+ [--show --device ${GPU_ID or CPU}] \
+ [--kpt-thr ${KPT_SCORE_THR}]
+```
+
+Examples:
+
+```shell
+python demo/top_down_video_demo_full_frame_without_det.py \
+ configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap/fly/res152_fly_192x192.py \
+ https://download.openmmlab.com/mmpose/animal/resnet/res152_fly_192x192-fcafbd5a_20210407.pth \
+ --video-path demo/resources/ \
+ --out-video-root vis_results
+```
+
+
+
+#### Using MMDetection to detect animals
+
+Assume that you have already installed [mmdet](https://github.com/open-mmlab/mmdetection).
+
+**COCO-animals**
+
+In COCO dataset, there are 80 object categories, including 10 common `animal` categories (15: 'bird', 16: 'cat', 17: 'dog', 18: 'horse', 19: 'sheep', 20: 'cow', 21: 'elephant', 22: 'bear', 23: 'zebra', 24: 'giraffe')
+For these COCO-animals, please download the COCO pre-trained detection model from [MMDetection Model Zoo](https://mmdetection.readthedocs.io/en/latest/model_zoo.html).
+
+```shell
+python demo/top_down_video_demo_with_mmdet.py \
+ ${MMDET_CONFIG_FILE} ${MMDET_CHECKPOINT_FILE} \
+ ${MMPOSE_CONFIG_FILE} ${MMPOSE_CHECKPOINT_FILE} \
+ --video-path ${VIDEO_FILE} \
+ --out-video-root ${OUTPUT_VIDEO_ROOT} \
+ --det-cat-id ${CATEGORY_ID}
+ [--show --device ${GPU_ID or CPU}] \
+ [--bbox-thr ${BBOX_SCORE_THR} --kpt-thr ${KPT_SCORE_THR}]
+```
+
+Examples:
+
+```shell
+python demo/top_down_video_demo_with_mmdet.py \
+ demo/mmdetection_cfg/faster_rcnn_r50_fpn_coco.py \
+ https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_2x_coco/faster_rcnn_r50_fpn_2x_coco_bbox_mAP-0.384_20200504_210434-a5d8aa15.pth \
+ configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap/horse10/res50_horse10_256x256-split1.py \
+ https://download.openmmlab.com/mmpose/animal/resnet/res50_horse10_256x256_split1-3a3dc37e_20210405.pth \
+ --video-path demo/resources/ \
+ --out-video-root vis_results \
+ --bbox-thr 0.1 \
+ --kpt-thr 0.4 \
+ --det-cat-id 18
+```
+
+
+
+**Other Animals**
+
+For other animals, we have also provided some pre-trained animal detection models (1-class models). Supported models can be found in [det model zoo](/demo/docs/mmdet_modelzoo.md).
+The pre-trained animal pose estimation model can be found in [pose model zoo](https://mmpose.readthedocs.io/en/latest/topics/animal.html).
+
+```shell
+python demo/top_down_video_demo_with_mmdet.py \
+ ${MMDET_CONFIG_FILE} ${MMDET_CHECKPOINT_FILE} \
+ ${MMPOSE_CONFIG_FILE} ${MMPOSE_CHECKPOINT_FILE} \
+ --video-path ${VIDEO_FILE} \
+ --out-video-root ${OUTPUT_VIDEO_ROOT} \
+ [--det-cat-id ${CATEGORY_ID}]
+ [--show --device ${GPU_ID or CPU}] \
+ [--bbox-thr ${BBOX_SCORE_THR} --kpt-thr ${KPT_SCORE_THR}]
+```
+
+Examples:
+
+```shell
+python demo/top_down_video_demo_with_mmdet.py \
+ demo/mmdetection_cfg/cascade_rcnn_x101_64x4d_fpn_1class.py \
+ https://openmmlab.oss-cn-hangzhou.aliyuncs.com/mmpose/mmdet_pretrained/cascade_rcnn_x101_64x4d_fpn_20e_macaque-e45e36f5_20210409.pth \
+ configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap/macaque/res152_macaque_256x192.py \
+ https://download.openmmlab.com/mmpose/animal/resnet/res152_macaque_256x192-c42abc02_20210407.pth \
+ --video-path demo/resources/ \
+ --out-video-root vis_results \
+ --bbox-thr 0.5 \
+ --kpt-thr 0.3 \
+```
+
+
+
+### Speed Up Inference
+
+Some tips to speed up MMPose inference:
+
+For 2D animal pose estimation models, try to edit the config file. For example,
+
+1. set `flip_test=False` in [macaque-res50](https://github.com/open-mmlab/mmpose/tree/e1ec589884235bee875c89102170439a991f8450/configs/animal/resnet/macaque/res50_macaque_256x192.py#L51).
+1. set `post_process='default'` in [macaque-res50](https://github.com/open-mmlab/mmpose/tree/e1ec589884235bee875c89102170439a991f8450/configs/animal/resnet/macaque/res50_macaque_256x192.py#L52).
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/docs/2d_face_demo.md b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/docs/2d_face_demo.md
new file mode 100644
index 0000000..a3b0f83
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/docs/2d_face_demo.md
@@ -0,0 +1,103 @@
+## 2D Face Keypoint Demo
+
+
+
+### 2D Face Image Demo
+
+#### Using gt face bounding boxes as input
+
+We provide a demo script to test a single image, given gt json file.
+
+*Face Keypoint Model Preparation:*
+The pre-trained face keypoint estimation model can be found from [model zoo](https://mmpose.readthedocs.io/en/latest/topics/face.html).
+Take [aflw model](https://download.openmmlab.com/mmpose/face/hrnetv2/hrnetv2_w18_aflw_256x256-f2bbc62b_20210125.pth) as an example:
+
+```shell
+python demo/top_down_img_demo.py \
+ ${MMPOSE_CONFIG_FILE} ${MMPOSE_CHECKPOINT_FILE} \
+ --img-root ${IMG_ROOT} --json-file ${JSON_FILE} \
+ --out-img-root ${OUTPUT_DIR} \
+ [--show --device ${GPU_ID or CPU}] \
+ [--kpt-thr ${KPT_SCORE_THR}]
+```
+
+Examples:
+
+```shell
+python demo/top_down_img_demo.py \
+ configs/face/2d_kpt_sview_rgb_img/topdown_heatmap/aflw/hrnetv2_w18_aflw_256x256.py \
+ https://download.openmmlab.com/mmpose/face/hrnetv2/hrnetv2_w18_aflw_256x256-f2bbc62b_20210125.pth \
+ --img-root tests/data/aflw/ --json-file tests/data/aflw/test_aflw.json \
+ --out-img-root vis_results
+```
+
+To run demos on CPU:
+
+```shell
+python demo/top_down_img_demo.py \
+ configs/face/2d_kpt_sview_rgb_img/topdown_heatmap/aflw/hrnetv2_w18_aflw_256x256.py \
+ https://download.openmmlab.com/mmpose/face/hrnetv2/hrnetv2_w18_aflw_256x256-f2bbc62b_20210125.pth \
+ --img-root tests/data/aflw/ --json-file tests/data/aflw/test_aflw.json \
+ --out-img-root vis_results \
+ --device=cpu
+```
+
+#### Using face bounding box detectors
+
+We provide a demo script to run face detection and face keypoint estimation.
+
+Please install `face_recognition` before running the demo, by `pip install face_recognition`.
+For more details, please refer to https://github.com/ageitgey/face_recognition.
+
+```shell
+python demo/face_img_demo.py \
+ ${MMPOSE_CONFIG_FILE} ${MMPOSE_CHECKPOINT_FILE} \
+ --img-root ${IMG_ROOT} --img ${IMG_FILE} \
+ --out-img-root ${OUTPUT_DIR} \
+ [--show --device ${GPU_ID or CPU}] \
+ [--kpt-thr ${KPT_SCORE_THR}]
+```
+
+```shell
+python demo/face_img_demo.py \
+ configs/face/2d_kpt_sview_rgb_img/topdown_heatmap/aflw/hrnetv2_w18_aflw_256x256.py \
+ https://download.openmmlab.com/mmpose/face/hrnetv2/hrnetv2_w18_aflw_256x256-f2bbc62b_20210125.pth \
+ --img-root tests/data/aflw/ \
+ --img image04476.jpg \
+ --out-img-root vis_results
+```
+
+### 2D Face Video Demo
+
+We also provide a video demo to illustrate the results.
+
+Please install `face_recognition` before running the demo, by `pip install face_recognition`.
+For more details, please refer to https://github.com/ageitgey/face_recognition.
+
+```shell
+python demo/face_video_demo.py \
+ ${MMPOSE_CONFIG_FILE} ${MMPOSE_CHECKPOINT_FILE} \
+ --video-path ${VIDEO_FILE} \
+ --out-video-root ${OUTPUT_VIDEO_ROOT} \
+ [--show --device ${GPU_ID or CPU}] \
+ [--kpt-thr ${KPT_SCORE_THR}]
+```
+
+Examples:
+
+```shell
+python demo/face_video_demo.py \
+ configs/face/2d_kpt_sview_rgb_img/topdown_heatmap/aflw/hrnetv2_w18_aflw_256x256.py \
+ https://download.openmmlab.com/mmpose/face/hrnetv2/hrnetv2_w18_aflw_256x256-f2bbc62b_20210125.pth \
+ --video-path https://user-images.githubusercontent.com/87690686/137441355-ec4da09c-3a8f-421b-bee9-b8b26f8c2dd0.mp4 \
+ --out-video-root vis_results
+```
+
+### Speed Up Inference
+
+Some tips to speed up MMPose inference:
+
+For 2D face keypoint estimation models, try to edit the config file. For example,
+
+1. set `flip_test=False` in [face-hrnetv2_w18](https://github.com/open-mmlab/mmpose/tree/e1ec589884235bee875c89102170439a991f8450/configs/face/hrnetv2/aflw/hrnetv2_w18_aflw_256x256.py#L83).
+1. set `post_process='default'` in [face-hrnetv2_w18](https://github.com/open-mmlab/mmpose/tree/e1ec589884235bee875c89102170439a991f8450/configs/face/hrnetv2/aflw/hrnetv2_w18_aflw_256x256.py#L84).
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/docs/2d_hand_demo.md b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/docs/2d_hand_demo.md
new file mode 100644
index 0000000..14b30f7
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/docs/2d_hand_demo.md
@@ -0,0 +1,113 @@
+## 2D Hand Keypoint Demo
+
+
+
+### 2D Hand Image Demo
+
+#### Using gt hand bounding boxes as input
+
+We provide a demo script to test a single image, given gt json file.
+
+*Hand Pose Model Preparation:*
+The pre-trained hand pose estimation model can be downloaded from [model zoo](https://mmpose.readthedocs.io/en/latest/topics/hand%282d%29.html).
+Take [onehand10k model](https://download.openmmlab.com/mmpose/top_down/resnet/res50_onehand10k_256x256-e67998f6_20200813.pth) as an example:
+
+```shell
+python demo/top_down_img_demo.py \
+ ${MMPOSE_CONFIG_FILE} ${MMPOSE_CHECKPOINT_FILE} \
+ --img-root ${IMG_ROOT} --json-file ${JSON_FILE} \
+ --out-img-root ${OUTPUT_DIR} \
+ [--show --device ${GPU_ID or CPU}] \
+ [--kpt-thr ${KPT_SCORE_THR}]
+```
+
+Examples:
+
+```shell
+python demo/top_down_img_demo.py \
+ configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/onehand10k/res50_onehand10k_256x256.py \
+ https://download.openmmlab.com/mmpose/top_down/resnet/res50_onehand10k_256x256-e67998f6_20200813.pth \
+ --img-root tests/data/onehand10k/ --json-file tests/data/onehand10k/test_onehand10k.json \
+ --out-img-root vis_results
+```
+
+To run demos on CPU:
+
+```shell
+python demo/top_down_img_demo.py \
+ configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/onehand10k/res50_onehand10k_256x256.py \
+ https://download.openmmlab.com/mmpose/top_down/resnet/res50_onehand10k_256x256-e67998f6_20200813.pth \
+ --img-root tests/data/onehand10k/ --json-file tests/data/onehand10k/test_onehand10k.json \
+ --out-img-root vis_results \
+ --device=cpu
+```
+
+#### Using mmdet for hand bounding box detection
+
+We provide a demo script to run mmdet for hand detection, and mmpose for hand pose estimation.
+
+Assume that you have already installed [mmdet](https://github.com/open-mmlab/mmdetection).
+
+*Hand Box Model Preparation:* The pre-trained hand box estimation model can be found in [det model zoo](/demo/docs/mmdet_modelzoo.md).
+
+*Hand Pose Model Preparation:* The pre-trained hand pose estimation model can be downloaded from [pose model zoo](https://mmpose.readthedocs.io/en/latest/topics/hand%282d%29.html).
+
+```shell
+python demo/top_down_img_demo_with_mmdet.py \
+ ${MMDET_CONFIG_FILE} ${MMDET_CHECKPOINT_FILE} \
+ ${MMPOSE_CONFIG_FILE} ${MMPOSE_CHECKPOINT_FILE} \
+ --img-root ${IMG_ROOT} --img ${IMG_FILE} \
+ --out-img-root ${OUTPUT_DIR} \
+ [--show --device ${GPU_ID or CPU}] \
+ [--bbox-thr ${BBOX_SCORE_THR} --kpt-thr ${KPT_SCORE_THR}]
+```
+
+```shell
+python demo/top_down_img_demo_with_mmdet.py demo/mmdetection_cfg/cascade_rcnn_x101_64x4d_fpn_1class.py \
+ https://download.openmmlab.com/mmpose/mmdet_pretrained/cascade_rcnn_x101_64x4d_fpn_20e_onehand10k-dac19597_20201030.pth \
+ configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/onehand10k/res50_onehand10k_256x256.py \
+ https://download.openmmlab.com/mmpose/top_down/resnet/res50_onehand10k_256x256-e67998f6_20200813.pth \
+ --img-root tests/data/onehand10k/ \
+ --img 9.jpg \
+ --out-img-root vis_results
+```
+
+### 2D Hand Video Demo
+
+We also provide a video demo to illustrate the results.
+
+Assume that you have already installed [mmdet](https://github.com/open-mmlab/mmdetection).
+
+*Hand Box Model Preparation:* The pre-trained hand box estimation model can be found in [det model zoo](/demo/docs/mmdet_modelzoo.md).
+
+*Hand Pose Model Preparation:* The pre-trained hand pose estimation model can be found in [pose model zoo](https://mmpose.readthedocs.io/en/latest/topics/hand%282d%29.html).
+
+```shell
+python demo/top_down_video_demo_with_mmdet.py \
+ ${MMDET_CONFIG_FILE} ${MMDET_CHECKPOINT_FILE} \
+ ${MMPOSE_CONFIG_FILE} ${MMPOSE_CHECKPOINT_FILE} \
+ --video-path ${VIDEO_FILE} \
+ --out-video-root ${OUTPUT_VIDEO_ROOT} \
+ [--show --device ${GPU_ID or CPU}] \
+ [--bbox-thr ${BBOX_SCORE_THR} --kpt-thr ${KPT_SCORE_THR}]
+```
+
+Examples:
+
+```shell
+python demo/top_down_video_demo_with_mmdet.py demo/mmdetection_cfg/cascade_rcnn_x101_64x4d_fpn_1class.py \
+ https://download.openmmlab.com/mmpose/mmdet_pretrained/cascade_rcnn_x101_64x4d_fpn_20e_onehand10k-dac19597_20201030.pth \
+ configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/onehand10k/res50_onehand10k_256x256.py \
+ https://download.openmmlab.com/mmpose/top_down/resnet/res50_onehand10k_256x256-e67998f6_20200813.pth \
+ --video-path https://user-images.githubusercontent.com/87690686/137441388-3ea93d26-5445-4184-829e-bf7011def9e4.mp4 \
+ --out-video-root vis_results
+```
+
+### Speed Up Inference
+
+Some tips to speed up MMPose inference:
+
+For 2D hand pose estimation models, try to edit the config file. For example,
+
+1. set `flip_test=False` in [hand-res50](https://github.com/open-mmlab/mmpose/tree/e1ec589884235bee875c89102170439a991f8450/configs/hand/resnet/onehand10k/res50_onehand10k_256x256.py#L56).
+1. set `post_process='default'` in [hand-res50](https://github.com/open-mmlab/mmpose/tree/e1ec589884235bee875c89102170439a991f8450/configs/hand/resnet/onehand10k/res50_onehand10k_256x256.py#L57).
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/docs/2d_human_pose_demo.md b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/docs/2d_human_pose_demo.md
new file mode 100644
index 0000000..fc264a3
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/docs/2d_human_pose_demo.md
@@ -0,0 +1,159 @@
+## 2D Human Pose Demo
+
+
+
+### 2D Human Pose Top-Down Image Demo
+
+#### Using gt human bounding boxes as input
+
+We provide a demo script to test a single image, given gt json file.
+
+```shell
+python demo/top_down_img_demo.py \
+ ${MMPOSE_CONFIG_FILE} ${MMPOSE_CHECKPOINT_FILE} \
+ --img-root ${IMG_ROOT} --json-file ${JSON_FILE} \
+ --out-img-root ${OUTPUT_DIR} \
+ [--show --device ${GPU_ID or CPU}] \
+ [--kpt-thr ${KPT_SCORE_THR}]
+```
+
+Examples:
+
+```shell
+python demo/top_down_img_demo.py \
+ configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/hrnet_w48_coco_256x192.py \
+ https://download.openmmlab.com/mmpose/top_down/hrnet/hrnet_w48_coco_256x192-b9e0b3ab_20200708.pth \
+ --img-root tests/data/coco/ --json-file tests/data/coco/test_coco.json \
+ --out-img-root vis_results
+```
+
+To run demos on CPU:
+
+```shell
+python demo/top_down_img_demo.py \
+ configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/hrnet_w48_coco_256x192.py \
+ https://download.openmmlab.com/mmpose/top_down/hrnet/hrnet_w48_coco_256x192-b9e0b3ab_20200708.pth \
+ --img-root tests/data/coco/ --json-file tests/data/coco/test_coco.json \
+ --out-img-root vis_results \
+ --device=cpu
+```
+
+#### Using mmdet for human bounding box detection
+
+We provide a demo script to run mmdet for human detection, and mmpose for pose estimation.
+
+Assume that you have already installed [mmdet](https://github.com/open-mmlab/mmdetection).
+
+```shell
+python demo/top_down_img_demo_with_mmdet.py \
+ ${MMDET_CONFIG_FILE} ${MMDET_CHECKPOINT_FILE} \
+ ${MMPOSE_CONFIG_FILE} ${MMPOSE_CHECKPOINT_FILE} \
+ --img-root ${IMG_ROOT} --img ${IMG_FILE} \
+ --out-img-root ${OUTPUT_DIR} \
+ [--show --device ${GPU_ID or CPU}] \
+ [--bbox-thr ${BBOX_SCORE_THR} --kpt-thr ${KPT_SCORE_THR}]
+```
+
+Examples:
+
+```shell
+python demo/top_down_img_demo_with_mmdet.py \
+ demo/mmdetection_cfg/faster_rcnn_r50_fpn_coco.py \
+ https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_1x_coco/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth \
+ configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/hrnet_w48_coco_256x192.py \
+ https://download.openmmlab.com/mmpose/top_down/hrnet/hrnet_w48_coco_256x192-b9e0b3ab_20200708.pth \
+ --img-root tests/data/coco/ \
+ --img 000000196141.jpg \
+ --out-img-root vis_results
+```
+
+### 2D Human Pose Top-Down Video Demo
+
+We also provide a video demo to illustrate the results.
+
+Assume that you have already installed [mmdet](https://github.com/open-mmlab/mmdetection).
+
+```shell
+python demo/top_down_video_demo_with_mmdet.py \
+ ${MMDET_CONFIG_FILE} ${MMDET_CHECKPOINT_FILE} \
+ ${MMPOSE_CONFIG_FILE} ${MMPOSE_CHECKPOINT_FILE} \
+ --video-path ${VIDEO_FILE} \
+ --out-video-root ${OUTPUT_VIDEO_ROOT} \
+ [--show --device ${GPU_ID or CPU}] \
+ [--bbox-thr ${BBOX_SCORE_THR} --kpt-thr ${KPT_SCORE_THR}]
+```
+
+Examples:
+
+```shell
+python demo/top_down_video_demo_with_mmdet.py \
+ demo/mmdetection_cfg/faster_rcnn_r50_fpn_coco.py \
+ https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_1x_coco/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth \
+ configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/hrnet_w48_coco_256x192.py \
+ https://download.openmmlab.com/mmpose/top_down/hrnet/hrnet_w48_coco_256x192-b9e0b3ab_20200708.pth \
+ --video-path demo/resources/demo.mp4 \
+ --out-video-root vis_results
+```
+
+### 2D Human Pose Bottom-Up Image Demo
+
+We provide a demo script to test a single image.
+
+```shell
+python demo/bottom_up_img_demo.py \
+ ${MMPOSE_CONFIG_FILE} ${MMPOSE_CHECKPOINT_FILE} \
+ --img-path ${IMG_PATH}\
+ --out-img-root ${OUTPUT_DIR} \
+ [--show --device ${GPU_ID or CPU}] \
+ [--kpt-thr ${KPT_SCORE_THR} --pose-nms-thr ${POSE_NMS_THR}]
+```
+
+Examples:
+
+```shell
+python demo/bottom_up_img_demo.py \
+ configs/body/2d_kpt_sview_rgb_img/associative_embedding/coco/hrnet_w32_coco_512x512.py \
+ https://download.openmmlab.com/mmpose/bottom_up/hrnet_w32_coco_512x512-bcb8c247_20200816.pth \
+ --img-path tests/data/coco/ \
+ --out-img-root vis_results
+```
+
+### 2D Human Pose Bottom-Up Video Demo
+
+We also provide a video demo to illustrate the results.
+
+```shell
+python demo/bottom_up_video_demo.py \
+ ${MMPOSE_CONFIG_FILE} ${MMPOSE_CHECKPOINT_FILE} \
+ --video-path ${VIDEO_FILE} \
+ --out-video-root ${OUTPUT_VIDEO_ROOT} \
+ [--show --device ${GPU_ID or CPU}] \
+ [--kpt-thr ${KPT_SCORE_THR} --pose-nms-thr ${POSE_NMS_THR}]
+```
+
+Examples:
+
+```shell
+python demo/bottom_up_video_demo.py \
+ configs/body/2d_kpt_sview_rgb_img/associative_embedding/coco/hrnet_w32_coco_512x512.py \
+ https://download.openmmlab.com/mmpose/bottom_up/hrnet_w32_coco_512x512-bcb8c247_20200816.pth \
+ --video-path demo/resources/demo.mp4 \
+ --out-video-root vis_results
+```
+
+### Speed Up Inference
+
+Some tips to speed up MMPose inference:
+
+For top-down models, try to edit the config file. For example,
+
+1. set `flip_test=False` in [topdown-res50](https://github.com/open-mmlab/mmpose/tree/e1ec589884235bee875c89102170439a991f8450/configs/top_down/resnet/coco/res50_coco_256x192.py#L51).
+1. set `post_process='default'` in [topdown-res50](https://github.com/open-mmlab/mmpose/tree/e1ec589884235bee875c89102170439a991f8450/configs/top_down/resnet/coco/res50_coco_256x192.py#L52).
+1. use faster human bounding box detector, see [MMDetection](https://mmdetection.readthedocs.io/en/latest/model_zoo.html).
+
+For bottom-up models, try to edit the config file. For example,
+
+1. set `flip_test=False` in [AE-res50](https://github.com/open-mmlab/mmpose/tree/e1ec589884235bee875c89102170439a991f8450/configs/bottom_up/resnet/coco/res50_coco_512x512.py#L80).
+1. set `adjust=False` in [AE-res50](https://github.com/open-mmlab/mmpose/tree/e1ec589884235bee875c89102170439a991f8450/configs/bottom_up/resnet/coco/res50_coco_512x512.py#L78).
+1. set `refine=False` in [AE-res50](https://github.com/open-mmlab/mmpose/tree/e1ec589884235bee875c89102170439a991f8450/configs/bottom_up/resnet/coco/res50_coco_512x512.py#L79).
+1. use smaller input image size in [AE-res50](https://github.com/open-mmlab/mmpose/tree/e1ec589884235bee875c89102170439a991f8450/configs/bottom_up/resnet/coco/res50_coco_512x512.py#L39).
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/docs/2d_pose_tracking_demo.md b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/docs/2d_pose_tracking_demo.md
new file mode 100644
index 0000000..9b29941
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/docs/2d_pose_tracking_demo.md
@@ -0,0 +1,101 @@
+## 2D Pose Tracking Demo
+
+
+
+### 2D Top-Down Video Human Pose Tracking Demo
+
+We provide a video demo to illustrate the pose tracking results.
+
+Assume that you have already installed [mmdet](https://github.com/open-mmlab/mmdetection).
+
+```shell
+python demo/top_down_pose_tracking_demo_with_mmdet.py \
+ ${MMDET_CONFIG_FILE} ${MMDET_CHECKPOINT_FILE} \
+ ${MMPOSE_CONFIG_FILE} ${MMPOSE_CHECKPOINT_FILE} \
+ --video-path ${VIDEO_FILE} \
+ --out-video-root ${OUTPUT_VIDEO_ROOT} \
+ [--show --device ${GPU_ID or CPU}] \
+ [--bbox-thr ${BBOX_SCORE_THR} --kpt-thr ${KPT_SCORE_THR}]
+ [--use-oks-tracking --tracking-thr ${TRACKING_THR} --euro]
+```
+
+Examples:
+
+```shell
+python demo/top_down_pose_tracking_demo_with_mmdet.py \
+ demo/mmdetection_cfg/faster_rcnn_r50_fpn_coco.py \
+ https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_1x_coco/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth \
+ configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/res50_coco_256x192.py \
+ https://download.openmmlab.com/mmpose/top_down/resnet/res50_coco_256x192-ec54d7f3_20200709.pth \
+ --video-path demo/resources/demo.mp4 \
+ --out-video-root vis_results
+```
+
+### 2D Top-Down Video Human Pose Tracking Demo with MMTracking
+
+MMTracking is an open source video perception toolbox based on PyTorch for tracking related tasks.
+Here we show how to utilize MMTracking and MMPose to achieve human pose tracking.
+
+Assume that you have already installed [mmtracking](https://github.com/open-mmlab/mmtracking).
+
+```shell
+python demo/top_down_video_demo_with_mmtracking.py \
+ ${MMTRACKING_CONFIG_FILE} \
+ ${MMPOSE_CONFIG_FILE} ${MMPOSE_CHECKPOINT_FILE} \
+ --video-path ${VIDEO_FILE} \
+ --out-video-root ${OUTPUT_VIDEO_ROOT} \
+ [--show --device ${GPU_ID or CPU}] \
+ [--bbox-thr ${BBOX_SCORE_THR} --kpt-thr ${KPT_SCORE_THR}]
+```
+
+Examples:
+
+```shell
+python demo/top_down_pose_tracking_demo_with_mmtracking.py \
+ demo/mmtracking_cfg/tracktor_faster-rcnn_r50_fpn_4e_mot17-private.py \
+ configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/res50_coco_256x192.py \
+ https://download.openmmlab.com/mmpose/top_down/resnet/res50_coco_256x192-ec54d7f3_20200709.pth \
+ --video-path demo/resources/demo.mp4 \
+ --out-video-root vis_results
+```
+
+### 2D Bottom-Up Video Human Pose Tracking Demo
+
+We also provide a pose tracking demo with bottom-up pose estimation methods.
+
+```shell
+python demo/bottom_up_pose_tracking_demo.py \
+ ${MMPOSE_CONFIG_FILE} ${MMPOSE_CHECKPOINT_FILE} \
+ --video-path ${VIDEO_FILE} \
+ --out-video-root ${OUTPUT_VIDEO_ROOT} \
+ [--show --device ${GPU_ID or CPU}] \
+ [--kpt-thr ${KPT_SCORE_THR} --pose-nms-thr ${POSE_NMS_THR}]
+ [--use-oks-tracking --tracking-thr ${TRACKING_THR} --euro]
+```
+
+Examples:
+
+```shell
+python demo/bottom_up_pose_tracking_demo.py \
+ configs/body/2d_kpt_sview_rgb_img/associative_embedding/coco/hrnet_w32_coco_512x512.py \
+ https://download.openmmlab.com/mmpose/bottom_up/hrnet_w32_coco_512x512-bcb8c247_20200816.pth \
+ --video-path demo/resources/demo.mp4 \
+ --out-video-root vis_results
+```
+
+### Speed Up Inference
+
+Some tips to speed up MMPose inference:
+
+For top-down models, try to edit the config file. For example,
+
+1. set `flip_test=False` in [topdown-res50](https://github.com/open-mmlab/mmpose/tree/e1ec589884235bee875c89102170439a991f8450/configs/top_down/resnet/coco/res50_coco_256x192.py#L51).
+1. set `post_process='default'` in [topdown-res50](https://github.com/open-mmlab/mmpose/tree/e1ec589884235bee875c89102170439a991f8450/configs/top_down/resnet/coco/res50_coco_256x192.py#L52).
+1. use faster human detector or human tracker, see [MMDetection](https://mmdetection.readthedocs.io/en/latest/model_zoo.html) or [MMTracking](https://mmtracking.readthedocs.io/en/latest/model_zoo.html).
+
+For bottom-up models, try to edit the config file. For example,
+
+1. set `flip_test=False` in [AE-res50](https://github.com/open-mmlab/mmpose/tree/e1ec589884235bee875c89102170439a991f8450/configs/bottom_up/resnet/coco/res50_coco_512x512.py#L80).
+1. set `adjust=False` in [AE-res50](https://github.com/open-mmlab/mmpose/tree/e1ec589884235bee875c89102170439a991f8450/configs/bottom_up/resnet/coco/res50_coco_512x512.py#L78).
+1. set `refine=False` in [AE-res50](https://github.com/open-mmlab/mmpose/tree/e1ec589884235bee875c89102170439a991f8450/configs/bottom_up/resnet/coco/res50_coco_512x512.py#L79).
+1. use smaller input image size in [AE-res50](https://github.com/open-mmlab/mmpose/tree/e1ec589884235bee875c89102170439a991f8450/configs/bottom_up/resnet/coco/res50_coco_512x512.py#L39).
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/docs/2d_wholebody_pose_demo.md b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/docs/2d_wholebody_pose_demo.md
new file mode 100644
index 0000000..a2050ea
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/docs/2d_wholebody_pose_demo.md
@@ -0,0 +1,106 @@
+## 2D Human Whole-Body Pose Demo
+
+
+
+### 2D Human Whole-Body Pose Top-Down Image Demo
+
+#### Using gt human bounding boxes as input
+
+We provide a demo script to test a single image, given gt json file.
+
+```shell
+python demo/top_down_img_demo.py \
+ ${MMPOSE_CONFIG_FILE} ${MMPOSE_CHECKPOINT_FILE} \
+ --img-root ${IMG_ROOT} --json-file ${JSON_FILE} \
+ --out-img-root ${OUTPUT_DIR} \
+ [--show --device ${GPU_ID or CPU}] \
+ [--kpt-thr ${KPT_SCORE_THR}]
+```
+
+Examples:
+
+```shell
+python demo/top_down_img_demo.py \
+ configs/wholebody/2d_kpt_sview_rgb_img/topdown_heatmap/coco-wholebody/hrnet_w48_coco_wholebody_384x288_dark_plus.py \
+ https://download.openmmlab.com/mmpose/top_down/hrnet/hrnet_w48_coco_wholebody_384x288_dark-f5726563_20200918.pth \
+ --img-root tests/data/coco/ --json-file tests/data/coco/test_coco.json \
+ --out-img-root vis_results
+```
+
+To run demos on CPU:
+
+```shell
+python demo/top_down_img_demo.py \
+ configs/wholebody/2d_kpt_sview_rgb_img/topdown_heatmap/coco-wholebody/hrnet_w48_coco_wholebody_384x288_dark_plus.py \
+ https://download.openmmlab.com/mmpose/top_down/hrnet/hrnet_w48_coco_wholebody_384x288_dark-f5726563_20200918.pth \
+ --img-root tests/data/coco/ --json-file tests/data/coco/test_coco.json \
+ --out-img-root vis_results \
+ --device=cpu
+```
+
+#### Using mmdet for human bounding box detection
+
+We provide a demo script to run mmdet for human detection, and mmpose for pose estimation.
+
+Assume that you have already installed [mmdet](https://github.com/open-mmlab/mmdetection).
+
+```shell
+python demo/top_down_img_demo_with_mmdet.py \
+ ${MMDET_CONFIG_FILE} ${MMDET_CHECKPOINT_FILE} \
+ ${MMPOSE_CONFIG_FILE} ${MMPOSE_CHECKPOINT_FILE} \
+ --img-root ${IMG_ROOT} --img ${IMG_FILE} \
+ --out-img-root ${OUTPUT_DIR} \
+ [--show --device ${GPU_ID or CPU}] \
+ [--bbox-thr ${BBOX_SCORE_THR} --kpt-thr ${KPT_SCORE_THR}]
+```
+
+Examples:
+
+```shell
+python demo/top_down_img_demo_with_mmdet.py \
+ demo/mmdetection_cfg/faster_rcnn_r50_fpn_coco.py \
+ https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_1x_coco/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth \
+ configs/wholebody/2d_kpt_sview_rgb_img/topdown_heatmap/coco-wholebody/hrnet_w48_coco_wholebody_384x288_dark_plus.py \
+ https://download.openmmlab.com/mmpose/top_down/hrnet/hrnet_w48_coco_wholebody_384x288_dark-f5726563_20200918.pth \
+ --img-root tests/data/coco/ \
+ --img 000000196141.jpg \
+ --out-img-root vis_results
+```
+
+### 2D Human Whole-Body Pose Top-Down Video Demo
+
+We also provide a video demo to illustrate the results.
+
+Assume that you have already installed [mmdet](https://github.com/open-mmlab/mmdetection).
+
+```shell
+python demo/top_down_video_demo_with_mmdet.py \
+ ${MMDET_CONFIG_FILE} ${MMDET_CHECKPOINT_FILE} \
+ ${MMPOSE_CONFIG_FILE} ${MMPOSE_CHECKPOINT_FILE} \
+ --video-path ${VIDEO_FILE} \
+ --out-video-root ${OUTPUT_VIDEO_ROOT} \
+ [--show --device ${GPU_ID or CPU}] \
+ [--bbox-thr ${BBOX_SCORE_THR} --kpt-thr ${KPT_SCORE_THR}]
+```
+
+Examples:
+
+```shell
+python demo/top_down_video_demo_with_mmdet.py \
+ demo/mmdetection_cfg/faster_rcnn_r50_fpn_coco.py \
+ https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_1x_coco/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth \
+ configs/wholebody/2d_kpt_sview_rgb_img/topdown_heatmap/coco-wholebody/hrnet_w48_coco_wholebody_384x288_dark_plus.py \
+ https://download.openmmlab.com/mmpose/top_down/hrnet/hrnet_w48_coco_wholebody_384x288_dark-f5726563_20200918.pth \
+ --video-path https://user-images.githubusercontent.com/87690686/137440639-fb08603d-9a35-474e-b65f-46b5c06b68d6.mp4 \
+ --out-video-root vis_results
+```
+
+### Speed Up Inference
+
+Some tips to speed up MMPose inference:
+
+For top-down models, try to edit the config file. For example,
+
+1. set `flip_test=False` in [pose_hrnet_w48_dark+](https://github.com/open-mmlab/mmpose/tree/e1ec589884235bee875c89102170439a991f8450/configs/wholebody/darkpose/coco-wholebody/hrnet_w48_coco_wholebody_384x288_dark_plus.py#L80).
+1. set `post_process='default'` in [pose_hrnet_w48_dark+](https://github.com/open-mmlab/mmpose/tree/e1ec589884235bee875c89102170439a991f8450/configs/wholebody/darkpose/coco-wholebody/hrnet_w48_coco_wholebody_384x288_dark_plus.py#L81).
+1. use faster human bounding box detector, see [MMDetection](https://mmdetection.readthedocs.io/en/latest/model_zoo.html).
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/docs/3d_body_mesh_demo.md b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/docs/3d_body_mesh_demo.md
new file mode 100644
index 0000000..b1e93db
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/docs/3d_body_mesh_demo.md
@@ -0,0 +1,28 @@
+## 3D Mesh Demo
+
+
+
+### 3D Mesh Recovery Demo
+
+We provide a demo script to recover human 3D mesh from a single image.
+
+```shell
+python demo/mesh_img_demo.py \
+ ${MMPOSE_CONFIG_FILE} ${MMPOSE_CHECKPOINT_FILE} \
+ --json-file ${JSON_FILE} \
+ --img-root ${IMG_ROOT} \
+ [--show] \
+ [--device ${GPU_ID or CPU}] \
+ [--out-img-root ${OUTPUT_DIR}]
+```
+
+Example:
+
+```shell
+python demo/mesh_img_demo.py \
+ configs/body/3d_mesh_sview_rgb_img/hmr/mixed/res50_mixed_224x224.py \
+ https://download.openmmlab.com/mmpose/mesh/hmr/hmr_mesh_224x224-c21e8229_20201015.pth \
+ --json-file tests/data/h36m/h36m_coco.json \
+ --img-root tests/data/h36m \
+ --out-img-root vis_results
+```
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/docs/3d_hand_demo.md b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/docs/3d_hand_demo.md
new file mode 100644
index 0000000..a3204b7
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/docs/3d_hand_demo.md
@@ -0,0 +1,50 @@
+## 3D Hand Demo
+
+
+
+### 3D Hand Estimation Image Demo
+
+#### Using gt hand bounding boxes as input
+
+We provide a demo script to test a single image, given gt json file.
+
+```shell
+python demo/interhand3d_img_demo.py \
+ ${MMPOSE_CONFIG_FILE} ${MMPOSE_CHECKPOINT_FILE} \
+ --json-file ${JSON_FILE} \
+ --img-root ${IMG_ROOT} \
+ [--camera-param-file ${CAMERA_PARAM_FILE}] \
+ [--gt-joints-file ${GT_JOINTS_FILE}]\
+ [--show] \
+ [--device ${GPU_ID or CPU}] \
+ [--out-img-root ${OUTPUT_DIR}] \
+ [--rebase-keypoint-height] \
+ [--show-ground-truth]
+```
+
+Example with gt keypoints and camera parameters:
+
+```shell
+python demo/interhand3d_img_demo.py \
+ configs/hand/3d_kpt_sview_rgb_img/internet/interhand3d/res50_interhand3d_all_256x256.py \
+ https://download.openmmlab.com/mmpose/hand3d/internet/res50_intehand3d_all_256x256-b9c1cf4c_20210506.pth \
+ --json-file tests/data/interhand2.6m/test_interhand2.6m_data.json \
+ --img-root tests/data/interhand2.6m \
+ --camera-param-file tests/data/interhand2.6m/test_interhand2.6m_camera.json \
+ --gt-joints-file tests/data/interhand2.6m/test_interhand2.6m_joint_3d.json \
+ --out-img-root vis_results \
+ --rebase-keypoint-height \
+ --show-ground-truth
+```
+
+Example without gt keypoints and camera parameters:
+
+```shell
+python demo/interhand3d_img_demo.py \
+ configs/hand/3d_kpt_sview_rgb_img/internet/interhand3d/res50_interhand3d_all_256x256.py \
+ https://download.openmmlab.com/mmpose/hand3d/internet/res50_intehand3d_all_256x256-b9c1cf4c_20210506.pth \
+ --json-file tests/data/interhand2.6m/test_interhand2.6m_data.json \
+ --img-root tests/data/interhand2.6m \
+ --out-img-root vis_results \
+ --rebase-keypoint-height
+```
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/docs/3d_human_pose_demo.md b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/docs/3d_human_pose_demo.md
new file mode 100644
index 0000000..4771c69
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/docs/3d_human_pose_demo.md
@@ -0,0 +1,84 @@
+## 3D Human Pose Demo
+
+
+
+### 3D Human Pose Two-stage Estimation Image Demo
+
+#### Using ground truth 2D poses as the 1st stage (pose detection) result, and inference the 2nd stage (2D-to-3D lifting)
+
+We provide a demo script to test on single images with a given ground-truth Json file.
+
+```shell
+python demo/body3d_two_stage_img_demo.py \
+ ${MMPOSE_CONFIG_FILE_3D} \
+ ${MMPOSE_CHECKPOINT_FILE_3D} \
+ --json-file ${JSON_FILE} \
+ --img-root ${IMG_ROOT} \
+ --only-second-stage \
+ [--show] \
+ [--device ${GPU_ID or CPU}] \
+ [--out-img-root ${OUTPUT_DIR}] \
+ [--rebase-keypoint-height] \
+ [--show-ground-truth]
+```
+
+Example:
+
+```shell
+python demo/body3d_two_stage_img_demo.py \
+ configs/body/3d_kpt_sview_rgb_img/pose_lift/h36m/simplebaseline3d_h36m.py \
+ https://download.openmmlab.com/mmpose/body3d/simple_baseline/simple3Dbaseline_h36m-f0ad73a4_20210419.pth \
+ --json-file tests/data/h36m/h36m_coco.json \
+ --img-root tests/data/h36m \
+ --camera-param-file tests/data/h36m/cameras.pkl \
+ --only-second-stage \
+ --out-img-root vis_results \
+ --rebase-keypoint-height \
+ --show-ground-truth
+```
+
+### 3D Human Pose Two-stage Estimation Video Demo
+
+#### Using mmdet for human bounding box detection and top-down model for the 1st stage (2D pose detection), and inference the 2nd stage (2D-to-3D lifting)
+
+Assume that you have already installed [mmdet](https://github.com/open-mmlab/mmdetection).
+
+```shell
+python demo/body3d_two_stage_video_demo.py \
+ ${MMDET_CONFIG_FILE} \
+ ${MMDET_CHECKPOINT_FILE} \
+ ${MMPOSE_CONFIG_FILE_2D} \
+ ${MMPOSE_CHECKPOINT_FILE_2D} \
+ ${MMPOSE_CONFIG_FILE_3D} \
+ ${MMPOSE_CHECKPOINT_FILE_3D} \
+ --video-path ${VIDEO_PATH} \
+ [--rebase-keypoint-height] \
+ [--norm-pose-2d] \
+ [--num-poses-vis NUM_POSES_VIS] \
+ [--show] \
+ [--out-video-root ${OUT_VIDEO_ROOT}] \
+ [--device ${GPU_ID or CPU}] \
+ [--det-cat-id DET_CAT_ID] \
+ [--bbox-thr BBOX_THR] \
+ [--kpt-thr KPT_THR] \
+ [--use-oks-tracking] \
+ [--tracking-thr TRACKING_THR] \
+ [--euro] \
+ [--radius RADIUS] \
+ [--thickness THICKNESS]
+```
+
+Example:
+
+```shell
+python demo/body3d_two_stage_video_demo.py \
+ demo/mmdetection_cfg/faster_rcnn_r50_fpn_coco.py \
+ https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_1x_coco/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth \
+ configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/hrnet_w48_coco_256x192.py \
+ https://download.openmmlab.com/mmpose/top_down/hrnet/hrnet_w48_coco_256x192-b9e0b3ab_20200708.pth \
+ configs/body/3d_kpt_sview_rgb_vid/video_pose_lift/h36m/videopose3d_h36m_243frames_fullconv_supervised_cpn_ft.py \
+ https://download.openmmlab.com/mmpose/body3d/videopose/videopose_h36m_243frames_fullconv_supervised_cpn_ft-88f5abbb_20210527.pth \
+ --video-path demo/resources/.mp4 \
+ --out-video-root vis_results \
+ --rebase-keypoint-height
+```
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/docs/mmdet_modelzoo.md b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/docs/mmdet_modelzoo.md
new file mode 100644
index 0000000..6017fcd
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/docs/mmdet_modelzoo.md
@@ -0,0 +1,30 @@
+## Pre-trained Detection Models
+
+### Human Bounding Box Detection Models
+
+For human bounding box detection models, please download from [MMDetection Model Zoo](https://mmdetection.readthedocs.io/en/latest/model_zoo.html).
+MMDetection provides 80-class COCO-pretrained models, which already includes the `person` category.
+
+### Hand Bounding Box Detection Models
+
+For hand bounding box detection, we simply train our hand box models on onehand10k dataset using MMDetection.
+
+#### Hand detection results on OneHand10K test set
+
+| Arch | Box AP | ckpt | log |
+| :-------------- | :-----------: | :------: | :------: |
+| [Cascade_R-CNN X-101-64x4d-FPN-1class](/demo/mmdetection_cfg/cascade_rcnn_x101_64x4d_fpn_1class.py) | 0.817 | [ckpt](https://download.openmmlab.com/mmpose/mmdet_pretrained/cascade_rcnn_x101_64x4d_fpn_20e_onehand10k-dac19597_20201030.pth) | [log](https://download.openmmlab.com/mmpose/mmdet_pretrained/cascade_rcnn_x101_64x4d_fpn_20e_onehand10k_20201030.log.json) |
+
+### Animal Bounding Box Detection Models
+
+#### COCO animals
+
+In COCO dataset, there are 80 object categories, including 10 common `animal` categories (16: 'bird', 17: 'cat', 18: 'dog', 19: 'horse', 20: 'sheep', 21: 'cow', 22: 'elephant', 23: 'bear', 24: 'zebra', 25: 'giraffe')
+For animals in the categories, please download from [MMDetection Model Zoo](https://mmdetection.readthedocs.io/en/latest/model_zoo.html).
+
+#### Macaque detection results on MacaquePose test set
+
+| Arch | Box AP | ckpt | log |
+| :-------------- | :-----------: | :------: | :------: |
+| [Faster_R-CNN_Res50-FPN-1class](/demo/mmdetection_cfg/faster_rcnn_r50_fpn_1class.py) | 0.840 | [ckpt](https://download.openmmlab.com/mmpose/mmdet_pretrained/faster_rcnn_r50_fpn_1x_macaque-f64f2812_20210409.pth) | [log](https://download.openmmlab.com/mmpose/mmdet_pretrained/faster_rcnn_r50_fpn_1x_macaque_20210409.log.json) |
+| [Cascade_R-CNN X-101-64x4d-FPN-1class](/demo/mmdetection_cfg/cascade_rcnn_x101_64x4d_fpn_1class.py) | 0.879 | [ckpt](https://download.openmmlab.com/mmpose/mmdet_pretrained/cascade_rcnn_x101_64x4d_fpn_20e_macaque-e45e36f5_20210409.pth) | [log](https://download.openmmlab.com/mmpose/mmdet_pretrained/cascade_rcnn_x101_64x4d_fpn_20e_macaque_20210409.log.json) |
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/docs/webcam_demo.md b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/docs/webcam_demo.md
new file mode 100644
index 0000000..a8a82a8
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/docs/webcam_demo.md
@@ -0,0 +1,49 @@
+## Webcam Demo
+
+We provide a webcam demo tool which integrartes detection and 2D pose estimation for humans and animals. You can simply run the following command:
+
+```python
+python demo/webcam_demo.py
+```
+
+It will launch a window to display the webcam video steam with detection and pose estimation results:
+
+
+
+
+
+### Usage Tips
+
+- **Which model is used in the demo tool?**
+
+ Please check the following default arguments in the script. You can also choose other models from the [MMDetection Model Zoo](https://github.com/open-mmlab/mmdetection/blob/master/docs/model_zoo.md) and [MMPose Model Zoo](https://mmpose.readthedocs.io/en/latest/modelzoo.html#) or use your own models.
+
+ | Model | Arguments |
+ | :--: | :-- |
+ | Detection | `--det-config`, `--det-checkpoint` |
+ | Human Pose | `--human-pose-config`, `--human-pose-checkpoint` |
+ | Animal Pose | `--animal-pose-config`, `--animal-pose-checkpoint` |
+
+- **Can this tool run without GPU?**
+
+ Yes, you can set `--device=cpu` and the model inference will be performed on CPU. Of course, this may cause a low inference FPS compared to using GPU devices.
+
+- **Why there is time delay between the pose visualization and the video?**
+
+ The video I/O and model inference are running asynchronously and the latter usually takes more time for a single frame. To allevidate the time delay, you can:
+
+ 1. set `--display-delay=MILLISECONDS` to defer the video stream, according to the inference delay shown at the top left corner. Or,
+
+ 2. set `--synchronous-mode` to force video stream being aligned with inference results. This may reduce the video display FPS.
+
+- **Can this tool process video files?**
+
+ Yes. You can set `--cam-id=VIDEO_FILE_PATH` to run the demo tool in offline mode on a video file. Note that `--synchronous-mode` should be set in this case.
+
+- **How to enable/disable the special effects?**
+
+ The special effects can be enabled/disabled at launch time by setting arguments like `--bugeye`, `--sunglasses`, *etc*. You can also toggle the effects by keyboard shortcuts like `b`, `s` when the tool starts.
+
+- **What if my computer doesn't have a camera?**
+
+ You can use a smart phone as a webcam with apps like [Camo](https://reincubate.com/camo/) or [DroidCam](https://www.dev47apps.com/).
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/face_img_demo.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/face_img_demo.py
new file mode 100644
index 0000000..e94eb08
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/face_img_demo.py
@@ -0,0 +1,140 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os
+import warnings
+from argparse import ArgumentParser
+
+from mmpose.apis import (inference_top_down_pose_model, init_pose_model,
+ vis_pose_result)
+from mmpose.datasets import DatasetInfo
+
+try:
+ import face_recognition
+ has_face_det = True
+except (ImportError, ModuleNotFoundError):
+ has_face_det = False
+
+
+def process_face_det_results(face_det_results):
+ """Process det results, and return a list of bboxes.
+
+ :param face_det_results: (top, right, bottom and left)
+ :return: a list of detected bounding boxes (x,y,x,y)-format
+ """
+
+ person_results = []
+ for bbox in face_det_results:
+ person = {}
+ # left, top, right, bottom
+ person['bbox'] = [bbox[3], bbox[0], bbox[1], bbox[2]]
+ person_results.append(person)
+
+ return person_results
+
+
+def main():
+ """Visualize the demo images.
+
+ Using mmdet to detect the human.
+ """
+ parser = ArgumentParser()
+ parser.add_argument('pose_config', help='Config file for pose')
+ parser.add_argument('pose_checkpoint', help='Checkpoint file for pose')
+ parser.add_argument('--img-root', type=str, default='', help='Image root')
+ parser.add_argument('--img', type=str, default='', help='Image file')
+ parser.add_argument(
+ '--show',
+ action='store_true',
+ default=False,
+ help='whether to show img')
+ parser.add_argument(
+ '--out-img-root',
+ type=str,
+ default='',
+ help='root of the output img file. '
+ 'Default not saving the visualization images.')
+ parser.add_argument(
+ '--device', default='cuda:0', help='Device used for inference')
+ parser.add_argument(
+ '--kpt-thr', type=float, default=0.3, help='Keypoint score threshold')
+ parser.add_argument(
+ '--radius',
+ type=int,
+ default=4,
+ help='Keypoint radius for visualization')
+ parser.add_argument(
+ '--thickness',
+ type=int,
+ default=1,
+ help='Link thickness for visualization')
+
+ assert has_face_det, 'Please install face_recognition to run the demo. ' \
+ '"pip install face_recognition", For more details, ' \
+ 'see https://github.com/ageitgey/face_recognition'
+
+ args = parser.parse_args()
+
+ assert args.show or (args.out_img_root != '')
+ assert args.img != ''
+
+ # build the pose model from a config file and a checkpoint file
+ pose_model = init_pose_model(
+ args.pose_config, args.pose_checkpoint, device=args.device.lower())
+
+ dataset = pose_model.cfg.data['test']['type']
+ dataset_info = pose_model.cfg.data['test'].get('dataset_info', None)
+ if dataset_info is None:
+ warnings.warn(
+ 'Please set `dataset_info` in the config.'
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 for details.',
+ DeprecationWarning)
+ else:
+ dataset_info = DatasetInfo(dataset_info)
+
+ image_name = os.path.join(args.img_root, args.img)
+
+ # test a single image, the resulting box is (top, right, bottom and left)
+ image = face_recognition.load_image_file(image_name)
+ face_det_results = face_recognition.face_locations(image)
+
+ # keep the person class bounding boxes.
+ face_results = process_face_det_results(face_det_results)
+
+ # optional
+ return_heatmap = False
+
+ # e.g. use ('backbone', ) to return backbone feature
+ output_layer_names = None
+
+ pose_results, returned_outputs = inference_top_down_pose_model(
+ pose_model,
+ image_name,
+ face_results,
+ bbox_thr=None,
+ format='xyxy',
+ dataset=dataset,
+ dataset_info=dataset_info,
+ return_heatmap=return_heatmap,
+ outputs=output_layer_names)
+
+ if args.out_img_root == '':
+ out_file = None
+ else:
+ os.makedirs(args.out_img_root, exist_ok=True)
+ out_file = os.path.join(args.out_img_root, f'vis_{args.img}')
+
+ # show the results
+ vis_pose_result(
+ pose_model,
+ image_name,
+ pose_results,
+ radius=args.radius,
+ thickness=args.thickness,
+ dataset=dataset,
+ dataset_info=dataset_info,
+ kpt_score_thr=args.kpt_thr,
+ show=args.show,
+ out_file=out_file)
+
+
+if __name__ == '__main__':
+ main()
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/face_video_demo.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/face_video_demo.py
new file mode 100644
index 0000000..cebe262
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/face_video_demo.py
@@ -0,0 +1,167 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os
+import warnings
+from argparse import ArgumentParser
+
+import cv2
+
+from mmpose.apis import (inference_top_down_pose_model, init_pose_model,
+ vis_pose_result)
+from mmpose.datasets import DatasetInfo
+
+try:
+ import face_recognition
+ has_face_det = True
+except (ImportError, ModuleNotFoundError):
+ has_face_det = False
+
+
+def process_face_det_results(face_det_results):
+ """Process det results, and return a list of bboxes.
+
+ :param face_det_results: (top, right, bottom and left)
+ :return: a list of detected bounding boxes (x,y,x,y)-format
+ """
+
+ person_results = []
+ for bbox in face_det_results:
+ person = {}
+ # left, top, right, bottom
+ person['bbox'] = [bbox[3], bbox[0], bbox[1], bbox[2]]
+ person_results.append(person)
+
+ return person_results
+
+
+def main():
+ """Visualize the demo images.
+
+ Using mmdet to detect the human.
+ """
+ parser = ArgumentParser()
+ parser.add_argument('pose_config', help='Config file for pose')
+ parser.add_argument('pose_checkpoint', help='Checkpoint file for pose')
+ parser.add_argument('--video-path', type=str, help='Video path')
+ parser.add_argument(
+ '--show',
+ action='store_true',
+ default=False,
+ help='whether to show visualizations.')
+ parser.add_argument(
+ '--out-video-root',
+ default='',
+ help='Root of the output video file. '
+ 'Default not saving the visualization video.')
+ parser.add_argument(
+ '--device', default='cuda:0', help='Device used for inference')
+ parser.add_argument(
+ '--kpt-thr', type=float, default=0.3, help='Keypoint score threshold')
+ parser.add_argument(
+ '--radius',
+ type=int,
+ default=4,
+ help='Keypoint radius for visualization')
+ parser.add_argument(
+ '--thickness',
+ type=int,
+ default=1,
+ help='Link thickness for visualization')
+
+ assert has_face_det, 'Please install face_recognition to run the demo. '\
+ '"pip install face_recognition", For more details, '\
+ 'see https://github.com/ageitgey/face_recognition'
+
+ args = parser.parse_args()
+
+ assert args.show or (args.out_video_root != '')
+
+ # build the pose model from a config file and a checkpoint file
+ pose_model = init_pose_model(
+ args.pose_config, args.pose_checkpoint, device=args.device.lower())
+
+ dataset = pose_model.cfg.data['test']['type']
+ dataset_info = pose_model.cfg.data['test'].get('dataset_info', None)
+ if dataset_info is None:
+ warnings.warn(
+ 'Please set `dataset_info` in the config.'
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 for details.',
+ DeprecationWarning)
+ else:
+ dataset_info = DatasetInfo(dataset_info)
+
+ cap = cv2.VideoCapture(args.video_path)
+ assert cap.isOpened(), f'Faild to load video file {args.video_path}'
+
+ if args.out_video_root == '':
+ save_out_video = False
+ else:
+ os.makedirs(args.out_video_root, exist_ok=True)
+ save_out_video = True
+
+ if save_out_video:
+ fps = cap.get(cv2.CAP_PROP_FPS)
+ size = (int(cap.get(cv2.CAP_PROP_FRAME_WIDTH)),
+ int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT)))
+ fourcc = cv2.VideoWriter_fourcc(*'mp4v')
+ videoWriter = cv2.VideoWriter(
+ os.path.join(args.out_video_root,
+ f'vis_{os.path.basename(args.video_path)}'), fourcc,
+ fps, size)
+
+ # optional
+ return_heatmap = False
+
+ # e.g. use ('backbone', ) to return backbone feature
+ output_layer_names = None
+
+ while (cap.isOpened()):
+ flag, img = cap.read()
+ if not flag:
+ break
+
+ face_det_results = face_recognition.face_locations(
+ cv2.cvtColor(img, cv2.COLOR_BGR2RGB))
+ face_results = process_face_det_results(face_det_results)
+
+ # test a single image, with a list of bboxes.
+ pose_results, returned_outputs = inference_top_down_pose_model(
+ pose_model,
+ img,
+ face_results,
+ bbox_thr=None,
+ format='xyxy',
+ dataset=dataset,
+ dataset_info=dataset_info,
+ return_heatmap=return_heatmap,
+ outputs=output_layer_names)
+
+ # show the results
+ vis_img = vis_pose_result(
+ pose_model,
+ img,
+ pose_results,
+ radius=args.radius,
+ thickness=args.thickness,
+ dataset=dataset,
+ dataset_info=dataset_info,
+ kpt_score_thr=args.kpt_thr,
+ show=False)
+
+ if args.show:
+ cv2.imshow('Image', vis_img)
+
+ if save_out_video:
+ videoWriter.write(vis_img)
+
+ if args.show and cv2.waitKey(1) & 0xFF == ord('q'):
+ break
+
+ cap.release()
+ if save_out_video:
+ videoWriter.release()
+ if args.show:
+ cv2.destroyAllWindows()
+
+
+if __name__ == '__main__':
+ main()
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/interhand3d_img_demo.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/interhand3d_img_demo.py
new file mode 100644
index 0000000..a6dbeff
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/interhand3d_img_demo.py
@@ -0,0 +1,258 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os
+import os.path as osp
+from argparse import ArgumentParser
+
+import mmcv
+import numpy as np
+from xtcocotools.coco import COCO
+
+from mmpose.apis import inference_interhand_3d_model, vis_3d_pose_result
+from mmpose.apis.inference import init_pose_model
+from mmpose.core import SimpleCamera
+
+
+def _transform_interhand_camera_param(interhand_camera_param):
+ """Transform the camera parameters in interhand2.6m dataset to the format
+ of SimpleCamera.
+
+ Args:
+ interhand_camera_param (dict): camera parameters including:
+ - camrot: 3x3, camera rotation matrix (world-to-camera)
+ - campos: 3x1, camera location in world space
+ - focal: 2x1, camera focal length
+ - princpt: 2x1, camera center
+
+ Returns:
+ param (dict): camera parameters including:
+ - R: 3x3, camera rotation matrix (camera-to-world)
+ - T: 3x1, camera translation (camera-to-world)
+ - f: 2x1, camera focal length
+ - c: 2x1, camera center
+ """
+ camera_param = {}
+ camera_param['R'] = np.array(interhand_camera_param['camrot']).T
+ camera_param['T'] = np.array(interhand_camera_param['campos'])[:, None]
+ camera_param['f'] = np.array(interhand_camera_param['focal'])[:, None]
+ camera_param['c'] = np.array(interhand_camera_param['princpt'])[:, None]
+ return camera_param
+
+
+def main():
+ parser = ArgumentParser()
+ parser.add_argument('pose_config', help='Config file for pose network')
+ parser.add_argument('pose_checkpoint', help='Checkpoint file')
+ parser.add_argument('--img-root', type=str, default='', help='Image root')
+ parser.add_argument(
+ '--json-file',
+ type=str,
+ default='',
+ help='Json file containing image info.')
+ parser.add_argument(
+ '--camera-param-file',
+ type=str,
+ default=None,
+ help='Camera parameter file for converting 3D pose predictions from '
+ ' the pixel space to camera space. If None, keypoints in pixel space'
+ 'will be visualized')
+ parser.add_argument(
+ '--gt-joints-file',
+ type=str,
+ default=None,
+ help='Optional argument. Ground truth 3D keypoint parameter file. '
+ 'If None, gt keypoints will not be shown and keypoints in pixel '
+ 'space will be visualized.')
+ parser.add_argument(
+ '--rebase-keypoint-height',
+ action='store_true',
+ help='Rebase the predicted 3D pose so its lowest keypoint has a '
+ 'height of 0 (landing on the ground). This is useful for '
+ 'visualization when the model do not predict the global position '
+ 'of the 3D pose.')
+ parser.add_argument(
+ '--show-ground-truth',
+ action='store_true',
+ help='If True, show ground truth keypoint if it is available.')
+ parser.add_argument(
+ '--show',
+ action='store_true',
+ default=False,
+ help='whether to show img')
+ parser.add_argument(
+ '--out-img-root',
+ type=str,
+ default=None,
+ help='Root of the output visualization images. '
+ 'Default not saving the visualization images.')
+ parser.add_argument(
+ '--device', default='cuda:0', help='Device for inference')
+ parser.add_argument(
+ '--kpt-thr', type=float, default=0.3, help='Keypoint score threshold')
+ parser.add_argument(
+ '--radius',
+ type=int,
+ default=4,
+ help='Keypoint radius for visualization')
+ parser.add_argument(
+ '--thickness',
+ type=int,
+ default=1,
+ help='Link thickness for visualization')
+
+ args = parser.parse_args()
+ assert args.show or (args.out_img_root != '')
+
+ coco = COCO(args.json_file)
+
+ # build the pose model from a config file and a checkpoint file
+ pose_model = init_pose_model(
+ args.pose_config, args.pose_checkpoint, device=args.device.lower())
+ dataset = pose_model.cfg.data['test']['type']
+
+ # load camera parameters
+ camera_params = None
+ if args.camera_param_file is not None:
+ camera_params = mmcv.load(args.camera_param_file)
+ # load ground truth joints parameters
+ gt_joint_params = None
+ if args.gt_joints_file is not None:
+ gt_joint_params = mmcv.load(args.gt_joints_file)
+
+ # load hand bounding boxes
+ det_results_list = []
+ for image_id, image in coco.imgs.items():
+ image_name = osp.join(args.img_root, image['file_name'])
+
+ ann_ids = coco.getAnnIds(image_id)
+ det_results = []
+
+ capture_key = str(image['capture'])
+ camera_key = image['camera']
+ frame_idx = image['frame_idx']
+
+ for ann_id in ann_ids:
+ ann = coco.anns[ann_id]
+ if camera_params is not None:
+ camera_param = {
+ key: camera_params[capture_key][key][camera_key]
+ for key in camera_params[capture_key].keys()
+ }
+ camera_param = _transform_interhand_camera_param(camera_param)
+ else:
+ camera_param = None
+ if gt_joint_params is not None:
+ joint_param = gt_joint_params[capture_key][str(frame_idx)]
+ gt_joint = np.concatenate([
+ np.array(joint_param['world_coord']),
+ np.array(joint_param['joint_valid'])
+ ],
+ axis=-1)
+ else:
+ gt_joint = None
+
+ det_result = {
+ 'image_name': image_name,
+ 'bbox': ann['bbox'], # bbox format is 'xywh'
+ 'camera_param': camera_param,
+ 'keypoints_3d_gt': gt_joint
+ }
+ det_results.append(det_result)
+ det_results_list.append(det_results)
+
+ for i, det_results in enumerate(
+ mmcv.track_iter_progress(det_results_list)):
+
+ image_name = det_results[0]['image_name']
+
+ pose_results = inference_interhand_3d_model(
+ pose_model, image_name, det_results, dataset=dataset)
+
+ # Post processing
+ pose_results_vis = []
+ for idx, res in enumerate(pose_results):
+ keypoints_3d = res['keypoints_3d']
+ # normalize kpt score
+ if keypoints_3d[:, 3].max() > 1:
+ keypoints_3d[:, 3] /= 255
+ # get 2D keypoints in pixel space
+ res['keypoints'] = keypoints_3d[:, [0, 1, 3]]
+
+ # For model-predicted keypoints, channel 0 and 1 are coordinates
+ # in pixel space, and channel 2 is the depth (in mm) relative
+ # to root joints.
+ # If both camera parameter and absolute depth of root joints are
+ # provided, we can transform keypoint to camera space for better
+ # visualization.
+ camera_param = res['camera_param']
+ keypoints_3d_gt = res['keypoints_3d_gt']
+ if camera_param is not None and keypoints_3d_gt is not None:
+ # build camera model
+ camera = SimpleCamera(camera_param)
+ # transform gt joints from world space to camera space
+ keypoints_3d_gt[:, :3] = camera.world_to_camera(
+ keypoints_3d_gt[:, :3])
+
+ # transform relative depth to absolute depth
+ keypoints_3d[:21, 2] += keypoints_3d_gt[20, 2]
+ keypoints_3d[21:, 2] += keypoints_3d_gt[41, 2]
+
+ # transform keypoints from pixel space to camera space
+ keypoints_3d[:, :3] = camera.pixel_to_camera(
+ keypoints_3d[:, :3])
+
+ # rotate the keypoint to make z-axis correspondent to height
+ # for better visualization
+ vis_R = np.array([[1, 0, 0], [0, 0, -1], [0, 1, 0]])
+ keypoints_3d[:, :3] = keypoints_3d[:, :3] @ vis_R
+ if keypoints_3d_gt is not None:
+ keypoints_3d_gt[:, :3] = keypoints_3d_gt[:, :3] @ vis_R
+
+ # rebase height (z-axis)
+ if args.rebase_keypoint_height:
+ valid = keypoints_3d[..., 3] > 0
+ keypoints_3d[..., 2] -= np.min(
+ keypoints_3d[valid, 2], axis=-1, keepdims=True)
+ res['keypoints_3d'] = keypoints_3d
+ res['keypoints_3d_gt'] = keypoints_3d_gt
+
+ # Add title
+ instance_id = res.get('track_id', idx)
+ res['title'] = f'Prediction ({instance_id})'
+ pose_results_vis.append(res)
+ # Add ground truth
+ if args.show_ground_truth:
+ if keypoints_3d_gt is None:
+ print('Fail to show ground truth. Please make sure that'
+ ' gt-joints-file is provided.')
+ else:
+ gt = res.copy()
+ if args.rebase_keypoint_height:
+ valid = keypoints_3d_gt[..., 3] > 0
+ keypoints_3d_gt[..., 2] -= np.min(
+ keypoints_3d_gt[valid, 2], axis=-1, keepdims=True)
+ gt['keypoints_3d'] = keypoints_3d_gt
+ gt['title'] = f'Ground truth ({instance_id})'
+ pose_results_vis.append(gt)
+
+ # Visualization
+ if args.out_img_root is None:
+ out_file = None
+ else:
+ os.makedirs(args.out_img_root, exist_ok=True)
+ out_file = osp.join(args.out_img_root, f'vis_{i}.jpg')
+
+ vis_3d_pose_result(
+ pose_model,
+ result=pose_results_vis,
+ img=det_results[0]['image_name'],
+ out_file=out_file,
+ dataset=dataset,
+ show=args.show,
+ kpt_score_thr=args.kpt_thr,
+ radius=args.radius,
+ thickness=args.thickness,
+ )
+
+
+if __name__ == '__main__':
+ main()
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/mesh_img_demo.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/mesh_img_demo.py
new file mode 100644
index 0000000..127ebad
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/mesh_img_demo.py
@@ -0,0 +1,93 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os
+from argparse import ArgumentParser
+
+from xtcocotools.coco import COCO
+
+from mmpose.apis import (inference_mesh_model, init_pose_model,
+ vis_3d_mesh_result)
+
+
+def main():
+ """Visualize the demo images.
+
+ Require the json_file containing boxes.
+ """
+ parser = ArgumentParser()
+ parser.add_argument('pose_config', help='Config file for detection')
+ parser.add_argument('pose_checkpoint', help='Checkpoint file')
+ parser.add_argument('--img-root', type=str, default='', help='Image root')
+ parser.add_argument(
+ '--json-file',
+ type=str,
+ default='',
+ help='Json file containing image info.')
+ parser.add_argument(
+ '--show',
+ action='store_true',
+ default=False,
+ help='whether to show img')
+ parser.add_argument(
+ '--out-img-root',
+ type=str,
+ default='',
+ help='Root of the output img file. '
+ 'Default not saving the visualization images.')
+ parser.add_argument(
+ '--device', default='cuda:0', help='Device used for inference')
+
+ args = parser.parse_args()
+
+ assert args.show or (args.out_img_root != '')
+
+ coco = COCO(args.json_file)
+ # build the pose model from a config file and a checkpoint file
+ pose_model = init_pose_model(
+ args.pose_config, args.pose_checkpoint, device=args.device.lower())
+
+ dataset = pose_model.cfg.data['test']['type']
+
+ img_keys = list(coco.imgs.keys())
+
+ # process each image
+ for i in range(len(img_keys)):
+ # get bounding box annotations
+ image_id = img_keys[i]
+ image = coco.loadImgs(image_id)[0]
+ image_name = os.path.join(args.img_root, image['file_name'])
+ ann_ids = coco.getAnnIds(image_id)
+
+ # make person bounding boxes
+ person_results = []
+ for ann_id in ann_ids:
+ person = {}
+ ann = coco.anns[ann_id]
+ # bbox format is 'xywh'
+ person['bbox'] = ann['bbox']
+ person_results.append(person)
+
+ # test a single image, with a list of bboxes
+ pose_results = inference_mesh_model(
+ pose_model,
+ image_name,
+ person_results,
+ bbox_thr=None,
+ format='xywh',
+ dataset=dataset)
+
+ if args.out_img_root == '':
+ out_file = None
+ else:
+ os.makedirs(args.out_img_root, exist_ok=True)
+ out_file = os.path.join(args.out_img_root, f'vis_{i}.jpg')
+
+ vis_3d_mesh_result(
+ pose_model,
+ pose_results,
+ image_name,
+ show=args.show,
+ out_file=out_file)
+
+
+if __name__ == '__main__':
+ main()
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/mmdetection_cfg/cascade_rcnn_x101_64x4d_fpn_1class.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/mmdetection_cfg/cascade_rcnn_x101_64x4d_fpn_1class.py
new file mode 100644
index 0000000..4e60b6b
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/mmdetection_cfg/cascade_rcnn_x101_64x4d_fpn_1class.py
@@ -0,0 +1,255 @@
+checkpoint_config = dict(interval=1)
+# yapf:disable
+log_config = dict(
+ interval=50,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook')
+ ])
+# yapf:enable
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
+
+# optimizer
+optimizer = dict(type='SGD', lr=0.02, momentum=0.9, weight_decay=0.0001)
+optimizer_config = dict(grad_clip=None)
+# learning policy
+lr_config = dict(
+ policy='step',
+ warmup='linear',
+ warmup_iters=500,
+ warmup_ratio=0.001,
+ step=[16, 19])
+total_epochs = 20
+# model settings
+model = dict(
+ type='CascadeRCNN',
+ pretrained='open-mmlab://resnext101_64x4d',
+ backbone=dict(
+ type='ResNeXt',
+ depth=101,
+ groups=64,
+ base_width=4,
+ num_stages=4,
+ out_indices=(0, 1, 2, 3),
+ frozen_stages=1,
+ norm_cfg=dict(type='BN', requires_grad=True),
+ style='pytorch'),
+ neck=dict(
+ type='FPN',
+ in_channels=[256, 512, 1024, 2048],
+ out_channels=256,
+ num_outs=5),
+ rpn_head=dict(
+ type='RPNHead',
+ in_channels=256,
+ feat_channels=256,
+ anchor_generator=dict(
+ type='AnchorGenerator',
+ scales=[8],
+ ratios=[0.5, 1.0, 2.0],
+ strides=[4, 8, 16, 32, 64]),
+ bbox_coder=dict(
+ type='DeltaXYWHBBoxCoder',
+ target_means=[.0, .0, .0, .0],
+ target_stds=[1.0, 1.0, 1.0, 1.0]),
+ loss_cls=dict(
+ type='CrossEntropyLoss', use_sigmoid=True, loss_weight=1.0),
+ loss_bbox=dict(type='SmoothL1Loss', beta=1.0 / 9.0, loss_weight=1.0)),
+ roi_head=dict(
+ type='CascadeRoIHead',
+ num_stages=3,
+ stage_loss_weights=[1, 0.5, 0.25],
+ bbox_roi_extractor=dict(
+ type='SingleRoIExtractor',
+ roi_layer=dict(type='RoIAlign', output_size=7, sampling_ratio=0),
+ out_channels=256,
+ featmap_strides=[4, 8, 16, 32]),
+ bbox_head=[
+ dict(
+ type='Shared2FCBBoxHead',
+ in_channels=256,
+ fc_out_channels=1024,
+ roi_feat_size=7,
+ num_classes=1,
+ bbox_coder=dict(
+ type='DeltaXYWHBBoxCoder',
+ target_means=[0., 0., 0., 0.],
+ target_stds=[0.1, 0.1, 0.2, 0.2]),
+ reg_class_agnostic=True,
+ loss_cls=dict(
+ type='CrossEntropyLoss',
+ use_sigmoid=False,
+ loss_weight=1.0),
+ loss_bbox=dict(type='SmoothL1Loss', beta=1.0,
+ loss_weight=1.0)),
+ dict(
+ type='Shared2FCBBoxHead',
+ in_channels=256,
+ fc_out_channels=1024,
+ roi_feat_size=7,
+ num_classes=1,
+ bbox_coder=dict(
+ type='DeltaXYWHBBoxCoder',
+ target_means=[0., 0., 0., 0.],
+ target_stds=[0.05, 0.05, 0.1, 0.1]),
+ reg_class_agnostic=True,
+ loss_cls=dict(
+ type='CrossEntropyLoss',
+ use_sigmoid=False,
+ loss_weight=1.0),
+ loss_bbox=dict(type='SmoothL1Loss', beta=1.0,
+ loss_weight=1.0)),
+ dict(
+ type='Shared2FCBBoxHead',
+ in_channels=256,
+ fc_out_channels=1024,
+ roi_feat_size=7,
+ num_classes=1,
+ bbox_coder=dict(
+ type='DeltaXYWHBBoxCoder',
+ target_means=[0., 0., 0., 0.],
+ target_stds=[0.033, 0.033, 0.067, 0.067]),
+ reg_class_agnostic=True,
+ loss_cls=dict(
+ type='CrossEntropyLoss',
+ use_sigmoid=False,
+ loss_weight=1.0),
+ loss_bbox=dict(type='SmoothL1Loss', beta=1.0, loss_weight=1.0))
+ ]),
+ # model training and testing settings
+ train_cfg=dict(
+ rpn=dict(
+ assigner=dict(
+ type='MaxIoUAssigner',
+ pos_iou_thr=0.7,
+ neg_iou_thr=0.3,
+ min_pos_iou=0.3,
+ match_low_quality=True,
+ ignore_iof_thr=-1),
+ sampler=dict(
+ type='RandomSampler',
+ num=256,
+ pos_fraction=0.5,
+ neg_pos_ub=-1,
+ add_gt_as_proposals=False),
+ allowed_border=0,
+ pos_weight=-1,
+ debug=False),
+ rpn_proposal=dict(
+ nms_pre=2000,
+ max_per_img=2000,
+ nms=dict(type='nms', iou_threshold=0.7),
+ min_bbox_size=0),
+ rcnn=[
+ dict(
+ assigner=dict(
+ type='MaxIoUAssigner',
+ pos_iou_thr=0.5,
+ neg_iou_thr=0.5,
+ min_pos_iou=0.5,
+ match_low_quality=False,
+ ignore_iof_thr=-1),
+ sampler=dict(
+ type='RandomSampler',
+ num=512,
+ pos_fraction=0.25,
+ neg_pos_ub=-1,
+ add_gt_as_proposals=True),
+ pos_weight=-1,
+ debug=False),
+ dict(
+ assigner=dict(
+ type='MaxIoUAssigner',
+ pos_iou_thr=0.6,
+ neg_iou_thr=0.6,
+ min_pos_iou=0.6,
+ match_low_quality=False,
+ ignore_iof_thr=-1),
+ sampler=dict(
+ type='RandomSampler',
+ num=512,
+ pos_fraction=0.25,
+ neg_pos_ub=-1,
+ add_gt_as_proposals=True),
+ pos_weight=-1,
+ debug=False),
+ dict(
+ assigner=dict(
+ type='MaxIoUAssigner',
+ pos_iou_thr=0.7,
+ neg_iou_thr=0.7,
+ min_pos_iou=0.7,
+ match_low_quality=False,
+ ignore_iof_thr=-1),
+ sampler=dict(
+ type='RandomSampler',
+ num=512,
+ pos_fraction=0.25,
+ neg_pos_ub=-1,
+ add_gt_as_proposals=True),
+ pos_weight=-1,
+ debug=False)
+ ]),
+ test_cfg=dict(
+ rpn=dict(
+ nms_pre=1000,
+ max_per_img=1000,
+ nms=dict(type='nms', iou_threshold=0.7),
+ min_bbox_size=0),
+ rcnn=dict(
+ score_thr=0.05,
+ nms=dict(type='nms', iou_threshold=0.5),
+ max_per_img=100)))
+
+dataset_type = 'CocoDataset'
+data_root = 'data/coco'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True)
+train_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='LoadAnnotations', with_bbox=True),
+ dict(type='Resize', img_scale=(1333, 800), keep_ratio=True),
+ dict(type='RandomFlip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='Pad', size_divisor=32),
+ dict(type='DefaultFormatBundle'),
+ dict(type='Collect', keys=['img', 'gt_bboxes', 'gt_labels']),
+]
+test_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(
+ type='MultiScaleFlipAug',
+ img_scale=(1333, 800),
+ flip=False,
+ transforms=[
+ dict(type='Resize', keep_ratio=True),
+ dict(type='RandomFlip'),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='Pad', size_divisor=32),
+ dict(type='DefaultFormatBundle'),
+ dict(type='Collect', keys=['img']),
+ ])
+]
+data = dict(
+ samples_per_gpu=2,
+ workers_per_gpu=2,
+ train=dict(
+ type=dataset_type,
+ ann_file=f'{data_root}/annotations/instances_train2017.json',
+ img_prefix=f'{data_root}/train2017/',
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=f'{data_root}/annotations/instances_val2017.json',
+ img_prefix=f'{data_root}/val2017/',
+ pipeline=test_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=f'{data_root}/annotations/instances_val2017.json',
+ img_prefix=f'{data_root}/val2017/',
+ pipeline=test_pipeline))
+evaluation = dict(interval=1, metric='bbox')
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/mmdetection_cfg/cascade_rcnn_x101_64x4d_fpn_coco.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/mmdetection_cfg/cascade_rcnn_x101_64x4d_fpn_coco.py
new file mode 100644
index 0000000..f91bd0d
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/mmdetection_cfg/cascade_rcnn_x101_64x4d_fpn_coco.py
@@ -0,0 +1,256 @@
+checkpoint_config = dict(interval=1)
+# yapf:disable
+log_config = dict(
+ interval=50,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook')
+ ])
+# yapf:enable
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
+
+# optimizer
+optimizer = dict(type='SGD', lr=0.02, momentum=0.9, weight_decay=0.0001)
+optimizer_config = dict(grad_clip=None)
+# learning policy
+lr_config = dict(
+ policy='step',
+ warmup='linear',
+ warmup_iters=500,
+ warmup_ratio=0.001,
+ step=[16, 19])
+total_epochs = 20
+
+# model settings
+model = dict(
+ type='CascadeRCNN',
+ pretrained='open-mmlab://resnext101_64x4d',
+ backbone=dict(
+ type='ResNeXt',
+ depth=101,
+ groups=64,
+ base_width=4,
+ num_stages=4,
+ out_indices=(0, 1, 2, 3),
+ frozen_stages=1,
+ norm_cfg=dict(type='BN', requires_grad=True),
+ style='pytorch'),
+ neck=dict(
+ type='FPN',
+ in_channels=[256, 512, 1024, 2048],
+ out_channels=256,
+ num_outs=5),
+ rpn_head=dict(
+ type='RPNHead',
+ in_channels=256,
+ feat_channels=256,
+ anchor_generator=dict(
+ type='AnchorGenerator',
+ scales=[8],
+ ratios=[0.5, 1.0, 2.0],
+ strides=[4, 8, 16, 32, 64]),
+ bbox_coder=dict(
+ type='DeltaXYWHBBoxCoder',
+ target_means=[.0, .0, .0, .0],
+ target_stds=[1.0, 1.0, 1.0, 1.0]),
+ loss_cls=dict(
+ type='CrossEntropyLoss', use_sigmoid=True, loss_weight=1.0),
+ loss_bbox=dict(type='SmoothL1Loss', beta=1.0 / 9.0, loss_weight=1.0)),
+ roi_head=dict(
+ type='CascadeRoIHead',
+ num_stages=3,
+ stage_loss_weights=[1, 0.5, 0.25],
+ bbox_roi_extractor=dict(
+ type='SingleRoIExtractor',
+ roi_layer=dict(type='RoIAlign', output_size=7, sampling_ratio=0),
+ out_channels=256,
+ featmap_strides=[4, 8, 16, 32]),
+ bbox_head=[
+ dict(
+ type='Shared2FCBBoxHead',
+ in_channels=256,
+ fc_out_channels=1024,
+ roi_feat_size=7,
+ num_classes=80,
+ bbox_coder=dict(
+ type='DeltaXYWHBBoxCoder',
+ target_means=[0., 0., 0., 0.],
+ target_stds=[0.1, 0.1, 0.2, 0.2]),
+ reg_class_agnostic=True,
+ loss_cls=dict(
+ type='CrossEntropyLoss',
+ use_sigmoid=False,
+ loss_weight=1.0),
+ loss_bbox=dict(type='SmoothL1Loss', beta=1.0,
+ loss_weight=1.0)),
+ dict(
+ type='Shared2FCBBoxHead',
+ in_channels=256,
+ fc_out_channels=1024,
+ roi_feat_size=7,
+ num_classes=80,
+ bbox_coder=dict(
+ type='DeltaXYWHBBoxCoder',
+ target_means=[0., 0., 0., 0.],
+ target_stds=[0.05, 0.05, 0.1, 0.1]),
+ reg_class_agnostic=True,
+ loss_cls=dict(
+ type='CrossEntropyLoss',
+ use_sigmoid=False,
+ loss_weight=1.0),
+ loss_bbox=dict(type='SmoothL1Loss', beta=1.0,
+ loss_weight=1.0)),
+ dict(
+ type='Shared2FCBBoxHead',
+ in_channels=256,
+ fc_out_channels=1024,
+ roi_feat_size=7,
+ num_classes=80,
+ bbox_coder=dict(
+ type='DeltaXYWHBBoxCoder',
+ target_means=[0., 0., 0., 0.],
+ target_stds=[0.033, 0.033, 0.067, 0.067]),
+ reg_class_agnostic=True,
+ loss_cls=dict(
+ type='CrossEntropyLoss',
+ use_sigmoid=False,
+ loss_weight=1.0),
+ loss_bbox=dict(type='SmoothL1Loss', beta=1.0, loss_weight=1.0))
+ ]),
+ # model training and testing settings
+ train_cfg=dict(
+ rpn=dict(
+ assigner=dict(
+ type='MaxIoUAssigner',
+ pos_iou_thr=0.7,
+ neg_iou_thr=0.3,
+ min_pos_iou=0.3,
+ match_low_quality=True,
+ ignore_iof_thr=-1),
+ sampler=dict(
+ type='RandomSampler',
+ num=256,
+ pos_fraction=0.5,
+ neg_pos_ub=-1,
+ add_gt_as_proposals=False),
+ allowed_border=0,
+ pos_weight=-1,
+ debug=False),
+ rpn_proposal=dict(
+ nms_pre=2000,
+ max_per_img=2000,
+ nms=dict(type='nms', iou_threshold=0.7),
+ min_bbox_size=0),
+ rcnn=[
+ dict(
+ assigner=dict(
+ type='MaxIoUAssigner',
+ pos_iou_thr=0.5,
+ neg_iou_thr=0.5,
+ min_pos_iou=0.5,
+ match_low_quality=False,
+ ignore_iof_thr=-1),
+ sampler=dict(
+ type='RandomSampler',
+ num=512,
+ pos_fraction=0.25,
+ neg_pos_ub=-1,
+ add_gt_as_proposals=True),
+ pos_weight=-1,
+ debug=False),
+ dict(
+ assigner=dict(
+ type='MaxIoUAssigner',
+ pos_iou_thr=0.6,
+ neg_iou_thr=0.6,
+ min_pos_iou=0.6,
+ match_low_quality=False,
+ ignore_iof_thr=-1),
+ sampler=dict(
+ type='RandomSampler',
+ num=512,
+ pos_fraction=0.25,
+ neg_pos_ub=-1,
+ add_gt_as_proposals=True),
+ pos_weight=-1,
+ debug=False),
+ dict(
+ assigner=dict(
+ type='MaxIoUAssigner',
+ pos_iou_thr=0.7,
+ neg_iou_thr=0.7,
+ min_pos_iou=0.7,
+ match_low_quality=False,
+ ignore_iof_thr=-1),
+ sampler=dict(
+ type='RandomSampler',
+ num=512,
+ pos_fraction=0.25,
+ neg_pos_ub=-1,
+ add_gt_as_proposals=True),
+ pos_weight=-1,
+ debug=False)
+ ]),
+ test_cfg=dict(
+ rpn=dict(
+ nms_pre=1000,
+ max_per_img=1000,
+ nms=dict(type='nms', iou_threshold=0.7),
+ min_bbox_size=0),
+ rcnn=dict(
+ score_thr=0.05,
+ nms=dict(type='nms', iou_threshold=0.5),
+ max_per_img=100)))
+
+dataset_type = 'CocoDataset'
+data_root = 'data/coco'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True)
+train_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='LoadAnnotations', with_bbox=True),
+ dict(type='Resize', img_scale=(1333, 800), keep_ratio=True),
+ dict(type='RandomFlip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='Pad', size_divisor=32),
+ dict(type='DefaultFormatBundle'),
+ dict(type='Collect', keys=['img', 'gt_bboxes', 'gt_labels']),
+]
+test_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(
+ type='MultiScaleFlipAug',
+ img_scale=(1333, 800),
+ flip=False,
+ transforms=[
+ dict(type='Resize', keep_ratio=True),
+ dict(type='RandomFlip'),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='Pad', size_divisor=32),
+ dict(type='DefaultFormatBundle'),
+ dict(type='Collect', keys=['img']),
+ ])
+]
+data = dict(
+ samples_per_gpu=2,
+ workers_per_gpu=2,
+ train=dict(
+ type=dataset_type,
+ ann_file=f'{data_root}/annotations/instances_train2017.json',
+ img_prefix=f'{data_root}/train2017/',
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=f'{data_root}/annotations/instances_val2017.json',
+ img_prefix=f'{data_root}/val2017/',
+ pipeline=test_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=f'{data_root}/annotations/instances_val2017.json',
+ img_prefix=f'{data_root}/val2017/',
+ pipeline=test_pipeline))
+evaluation = dict(interval=1, metric='bbox')
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/mmdetection_cfg/faster_rcnn_r50_fpn_1class.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/mmdetection_cfg/faster_rcnn_r50_fpn_1class.py
new file mode 100644
index 0000000..ee54f5b
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/mmdetection_cfg/faster_rcnn_r50_fpn_1class.py
@@ -0,0 +1,182 @@
+checkpoint_config = dict(interval=1)
+# yapf:disable
+log_config = dict(
+ interval=50,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook')
+ ])
+# yapf:enable
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
+# optimizer
+optimizer = dict(type='SGD', lr=0.02, momentum=0.9, weight_decay=0.0001)
+optimizer_config = dict(grad_clip=None)
+# learning policy
+lr_config = dict(
+ policy='step',
+ warmup='linear',
+ warmup_iters=500,
+ warmup_ratio=0.001,
+ step=[8, 11])
+total_epochs = 12
+
+model = dict(
+ type='FasterRCNN',
+ pretrained='torchvision://resnet50',
+ backbone=dict(
+ type='ResNet',
+ depth=50,
+ num_stages=4,
+ out_indices=(0, 1, 2, 3),
+ frozen_stages=1,
+ norm_cfg=dict(type='BN', requires_grad=True),
+ norm_eval=True,
+ style='pytorch'),
+ neck=dict(
+ type='FPN',
+ in_channels=[256, 512, 1024, 2048],
+ out_channels=256,
+ num_outs=5),
+ rpn_head=dict(
+ type='RPNHead',
+ in_channels=256,
+ feat_channels=256,
+ anchor_generator=dict(
+ type='AnchorGenerator',
+ scales=[8],
+ ratios=[0.5, 1.0, 2.0],
+ strides=[4, 8, 16, 32, 64]),
+ bbox_coder=dict(
+ type='DeltaXYWHBBoxCoder',
+ target_means=[.0, .0, .0, .0],
+ target_stds=[1.0, 1.0, 1.0, 1.0]),
+ loss_cls=dict(
+ type='CrossEntropyLoss', use_sigmoid=True, loss_weight=1.0),
+ loss_bbox=dict(type='L1Loss', loss_weight=1.0)),
+ roi_head=dict(
+ type='StandardRoIHead',
+ bbox_roi_extractor=dict(
+ type='SingleRoIExtractor',
+ roi_layer=dict(type='RoIAlign', output_size=7, sampling_ratio=0),
+ out_channels=256,
+ featmap_strides=[4, 8, 16, 32]),
+ bbox_head=dict(
+ type='Shared2FCBBoxHead',
+ in_channels=256,
+ fc_out_channels=1024,
+ roi_feat_size=7,
+ num_classes=1,
+ bbox_coder=dict(
+ type='DeltaXYWHBBoxCoder',
+ target_means=[0., 0., 0., 0.],
+ target_stds=[0.1, 0.1, 0.2, 0.2]),
+ reg_class_agnostic=False,
+ loss_cls=dict(
+ type='CrossEntropyLoss', use_sigmoid=False, loss_weight=1.0),
+ loss_bbox=dict(type='L1Loss', loss_weight=1.0))),
+ # model training and testing settings
+ train_cfg=dict(
+ rpn=dict(
+ assigner=dict(
+ type='MaxIoUAssigner',
+ pos_iou_thr=0.7,
+ neg_iou_thr=0.3,
+ min_pos_iou=0.3,
+ match_low_quality=True,
+ ignore_iof_thr=-1),
+ sampler=dict(
+ type='RandomSampler',
+ num=256,
+ pos_fraction=0.5,
+ neg_pos_ub=-1,
+ add_gt_as_proposals=False),
+ allowed_border=-1,
+ pos_weight=-1,
+ debug=False),
+ rpn_proposal=dict(
+ nms_pre=2000,
+ max_per_img=1000,
+ nms=dict(type='nms', iou_threshold=0.7),
+ min_bbox_size=0),
+ rcnn=dict(
+ assigner=dict(
+ type='MaxIoUAssigner',
+ pos_iou_thr=0.5,
+ neg_iou_thr=0.5,
+ min_pos_iou=0.5,
+ match_low_quality=False,
+ ignore_iof_thr=-1),
+ sampler=dict(
+ type='RandomSampler',
+ num=512,
+ pos_fraction=0.25,
+ neg_pos_ub=-1,
+ add_gt_as_proposals=True),
+ pos_weight=-1,
+ debug=False)),
+ test_cfg=dict(
+ rpn=dict(
+ nms_pre=1000,
+ max_per_img=1000,
+ nms=dict(type='nms', iou_threshold=0.7),
+ min_bbox_size=0),
+ rcnn=dict(
+ score_thr=0.05,
+ nms=dict(type='nms', iou_threshold=0.5),
+ max_per_img=100)
+ # soft-nms is also supported for rcnn testing
+ # e.g., nms=dict(type='soft_nms', iou_threshold=0.5, min_score=0.05)
+ ))
+
+dataset_type = 'CocoDataset'
+data_root = 'data/coco'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True)
+train_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='LoadAnnotations', with_bbox=True),
+ dict(type='Resize', img_scale=(1333, 800), keep_ratio=True),
+ dict(type='RandomFlip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='Pad', size_divisor=32),
+ dict(type='DefaultFormatBundle'),
+ dict(type='Collect', keys=['img', 'gt_bboxes', 'gt_labels']),
+]
+test_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(
+ type='MultiScaleFlipAug',
+ img_scale=(1333, 800),
+ flip=False,
+ transforms=[
+ dict(type='Resize', keep_ratio=True),
+ dict(type='RandomFlip'),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='Pad', size_divisor=32),
+ dict(type='DefaultFormatBundle'),
+ dict(type='Collect', keys=['img']),
+ ])
+]
+data = dict(
+ samples_per_gpu=2,
+ workers_per_gpu=2,
+ train=dict(
+ type=dataset_type,
+ ann_file=f'{data_root}/annotations/instances_train2017.json',
+ img_prefix=f'{data_root}/train2017/',
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=f'{data_root}/annotations/instances_val2017.json',
+ img_prefix=f'{data_root}/val2017/',
+ pipeline=test_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=f'{data_root}/annotations/instances_val2017.json',
+ img_prefix=f'{data_root}/val2017/',
+ pipeline=test_pipeline))
+evaluation = dict(interval=1, metric='bbox')
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/mmdetection_cfg/faster_rcnn_r50_fpn_coco.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/mmdetection_cfg/faster_rcnn_r50_fpn_coco.py
new file mode 100644
index 0000000..a9ad952
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/mmdetection_cfg/faster_rcnn_r50_fpn_coco.py
@@ -0,0 +1,182 @@
+checkpoint_config = dict(interval=1)
+# yapf:disable
+log_config = dict(
+ interval=50,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook')
+ ])
+# yapf:enable
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
+# optimizer
+optimizer = dict(type='SGD', lr=0.02, momentum=0.9, weight_decay=0.0001)
+optimizer_config = dict(grad_clip=None)
+# learning policy
+lr_config = dict(
+ policy='step',
+ warmup='linear',
+ warmup_iters=500,
+ warmup_ratio=0.001,
+ step=[8, 11])
+total_epochs = 12
+
+model = dict(
+ type='FasterRCNN',
+ pretrained='torchvision://resnet50',
+ backbone=dict(
+ type='ResNet',
+ depth=50,
+ num_stages=4,
+ out_indices=(0, 1, 2, 3),
+ frozen_stages=1,
+ norm_cfg=dict(type='BN', requires_grad=True),
+ norm_eval=True,
+ style='pytorch'),
+ neck=dict(
+ type='FPN',
+ in_channels=[256, 512, 1024, 2048],
+ out_channels=256,
+ num_outs=5),
+ rpn_head=dict(
+ type='RPNHead',
+ in_channels=256,
+ feat_channels=256,
+ anchor_generator=dict(
+ type='AnchorGenerator',
+ scales=[8],
+ ratios=[0.5, 1.0, 2.0],
+ strides=[4, 8, 16, 32, 64]),
+ bbox_coder=dict(
+ type='DeltaXYWHBBoxCoder',
+ target_means=[.0, .0, .0, .0],
+ target_stds=[1.0, 1.0, 1.0, 1.0]),
+ loss_cls=dict(
+ type='CrossEntropyLoss', use_sigmoid=True, loss_weight=1.0),
+ loss_bbox=dict(type='L1Loss', loss_weight=1.0)),
+ roi_head=dict(
+ type='StandardRoIHead',
+ bbox_roi_extractor=dict(
+ type='SingleRoIExtractor',
+ roi_layer=dict(type='RoIAlign', output_size=7, sampling_ratio=0),
+ out_channels=256,
+ featmap_strides=[4, 8, 16, 32]),
+ bbox_head=dict(
+ type='Shared2FCBBoxHead',
+ in_channels=256,
+ fc_out_channels=1024,
+ roi_feat_size=7,
+ num_classes=80,
+ bbox_coder=dict(
+ type='DeltaXYWHBBoxCoder',
+ target_means=[0., 0., 0., 0.],
+ target_stds=[0.1, 0.1, 0.2, 0.2]),
+ reg_class_agnostic=False,
+ loss_cls=dict(
+ type='CrossEntropyLoss', use_sigmoid=False, loss_weight=1.0),
+ loss_bbox=dict(type='L1Loss', loss_weight=1.0))),
+ # model training and testing settings
+ train_cfg=dict(
+ rpn=dict(
+ assigner=dict(
+ type='MaxIoUAssigner',
+ pos_iou_thr=0.7,
+ neg_iou_thr=0.3,
+ min_pos_iou=0.3,
+ match_low_quality=True,
+ ignore_iof_thr=-1),
+ sampler=dict(
+ type='RandomSampler',
+ num=256,
+ pos_fraction=0.5,
+ neg_pos_ub=-1,
+ add_gt_as_proposals=False),
+ allowed_border=-1,
+ pos_weight=-1,
+ debug=False),
+ rpn_proposal=dict(
+ nms_pre=2000,
+ max_per_img=1000,
+ nms=dict(type='nms', iou_threshold=0.7),
+ min_bbox_size=0),
+ rcnn=dict(
+ assigner=dict(
+ type='MaxIoUAssigner',
+ pos_iou_thr=0.5,
+ neg_iou_thr=0.5,
+ min_pos_iou=0.5,
+ match_low_quality=False,
+ ignore_iof_thr=-1),
+ sampler=dict(
+ type='RandomSampler',
+ num=512,
+ pos_fraction=0.25,
+ neg_pos_ub=-1,
+ add_gt_as_proposals=True),
+ pos_weight=-1,
+ debug=False)),
+ test_cfg=dict(
+ rpn=dict(
+ nms_pre=1000,
+ max_per_img=1000,
+ nms=dict(type='nms', iou_threshold=0.7),
+ min_bbox_size=0),
+ rcnn=dict(
+ score_thr=0.05,
+ nms=dict(type='nms', iou_threshold=0.5),
+ max_per_img=100)
+ # soft-nms is also supported for rcnn testing
+ # e.g., nms=dict(type='soft_nms', iou_threshold=0.5, min_score=0.05)
+ ))
+
+dataset_type = 'CocoDataset'
+data_root = 'data/coco'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True)
+train_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='LoadAnnotations', with_bbox=True),
+ dict(type='Resize', img_scale=(1333, 800), keep_ratio=True),
+ dict(type='RandomFlip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='Pad', size_divisor=32),
+ dict(type='DefaultFormatBundle'),
+ dict(type='Collect', keys=['img', 'gt_bboxes', 'gt_labels']),
+]
+test_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(
+ type='MultiScaleFlipAug',
+ img_scale=(1333, 800),
+ flip=False,
+ transforms=[
+ dict(type='Resize', keep_ratio=True),
+ dict(type='RandomFlip'),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='Pad', size_divisor=32),
+ dict(type='DefaultFormatBundle'),
+ dict(type='Collect', keys=['img']),
+ ])
+]
+data = dict(
+ samples_per_gpu=2,
+ workers_per_gpu=2,
+ train=dict(
+ type=dataset_type,
+ ann_file=f'{data_root}/annotations/instances_train2017.json',
+ img_prefix=f'{data_root}/train2017/',
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=f'{data_root}/annotations/instances_val2017.json',
+ img_prefix=f'{data_root}/val2017/',
+ pipeline=test_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=f'{data_root}/annotations/instances_val2017.json',
+ img_prefix=f'{data_root}/val2017/',
+ pipeline=test_pipeline))
+evaluation = dict(interval=1, metric='bbox')
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/mmdetection_cfg/mask_rcnn_r50_fpn_2x_coco.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/mmdetection_cfg/mask_rcnn_r50_fpn_2x_coco.py
new file mode 100644
index 0000000..05d39fa
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/mmdetection_cfg/mask_rcnn_r50_fpn_2x_coco.py
@@ -0,0 +1,242 @@
+model = dict(
+ type='MaskRCNN',
+ backbone=dict(
+ type='ResNet',
+ depth=50,
+ num_stages=4,
+ out_indices=(0, 1, 2, 3),
+ frozen_stages=1,
+ norm_cfg=dict(type='BN', requires_grad=True),
+ norm_eval=True,
+ style='pytorch',
+ init_cfg=dict(type='Pretrained', checkpoint='torchvision://resnet50')),
+ neck=dict(
+ type='FPN',
+ in_channels=[256, 512, 1024, 2048],
+ out_channels=256,
+ num_outs=5),
+ rpn_head=dict(
+ type='RPNHead',
+ in_channels=256,
+ feat_channels=256,
+ anchor_generator=dict(
+ type='AnchorGenerator',
+ scales=[8],
+ ratios=[0.5, 1.0, 2.0],
+ strides=[4, 8, 16, 32, 64]),
+ bbox_coder=dict(
+ type='DeltaXYWHBBoxCoder',
+ target_means=[0.0, 0.0, 0.0, 0.0],
+ target_stds=[1.0, 1.0, 1.0, 1.0]),
+ loss_cls=dict(
+ type='CrossEntropyLoss', use_sigmoid=True, loss_weight=1.0),
+ loss_bbox=dict(type='L1Loss', loss_weight=1.0)),
+ roi_head=dict(
+ type='StandardRoIHead',
+ bbox_roi_extractor=dict(
+ type='SingleRoIExtractor',
+ roi_layer=dict(type='RoIAlign', output_size=7, sampling_ratio=0),
+ out_channels=256,
+ featmap_strides=[4, 8, 16, 32]),
+ bbox_head=dict(
+ type='Shared2FCBBoxHead',
+ in_channels=256,
+ fc_out_channels=1024,
+ roi_feat_size=7,
+ num_classes=80,
+ bbox_coder=dict(
+ type='DeltaXYWHBBoxCoder',
+ target_means=[0.0, 0.0, 0.0, 0.0],
+ target_stds=[0.1, 0.1, 0.2, 0.2]),
+ reg_class_agnostic=False,
+ loss_cls=dict(
+ type='CrossEntropyLoss', use_sigmoid=False, loss_weight=1.0),
+ loss_bbox=dict(type='L1Loss', loss_weight=1.0)),
+ mask_roi_extractor=dict(
+ type='SingleRoIExtractor',
+ roi_layer=dict(type='RoIAlign', output_size=14, sampling_ratio=0),
+ out_channels=256,
+ featmap_strides=[4, 8, 16, 32]),
+ mask_head=dict(
+ type='FCNMaskHead',
+ num_convs=4,
+ in_channels=256,
+ conv_out_channels=256,
+ num_classes=80,
+ loss_mask=dict(
+ type='CrossEntropyLoss', use_mask=True, loss_weight=1.0))),
+ train_cfg=dict(
+ rpn=dict(
+ assigner=dict(
+ type='MaxIoUAssigner',
+ pos_iou_thr=0.7,
+ neg_iou_thr=0.3,
+ min_pos_iou=0.3,
+ match_low_quality=True,
+ ignore_iof_thr=-1),
+ sampler=dict(
+ type='RandomSampler',
+ num=256,
+ pos_fraction=0.5,
+ neg_pos_ub=-1,
+ add_gt_as_proposals=False),
+ allowed_border=-1,
+ pos_weight=-1,
+ debug=False),
+ rpn_proposal=dict(
+ nms_pre=2000,
+ max_per_img=1000,
+ nms=dict(type='nms', iou_threshold=0.7),
+ min_bbox_size=0),
+ rcnn=dict(
+ assigner=dict(
+ type='MaxIoUAssigner',
+ pos_iou_thr=0.5,
+ neg_iou_thr=0.5,
+ min_pos_iou=0.5,
+ match_low_quality=True,
+ ignore_iof_thr=-1),
+ sampler=dict(
+ type='RandomSampler',
+ num=512,
+ pos_fraction=0.25,
+ neg_pos_ub=-1,
+ add_gt_as_proposals=True),
+ mask_size=28,
+ pos_weight=-1,
+ debug=False)),
+ test_cfg=dict(
+ rpn=dict(
+ nms_pre=1000,
+ max_per_img=1000,
+ nms=dict(type='nms', iou_threshold=0.7),
+ min_bbox_size=0),
+ rcnn=dict(
+ score_thr=0.05,
+ nms=dict(type='nms', iou_threshold=0.5),
+ max_per_img=100,
+ mask_thr_binary=0.5)))
+dataset_type = 'CocoDataset'
+data_root = 'data/coco/'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True)
+train_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='LoadAnnotations', with_bbox=True, with_mask=True),
+ dict(type='Resize', img_scale=(1333, 800), keep_ratio=True),
+ dict(type='RandomFlip', flip_ratio=0.5),
+ dict(
+ type='Normalize',
+ mean=[123.675, 116.28, 103.53],
+ std=[58.395, 57.12, 57.375],
+ to_rgb=True),
+ dict(type='Pad', size_divisor=32),
+ dict(type='DefaultFormatBundle'),
+ dict(type='Collect', keys=['img', 'gt_bboxes', 'gt_labels', 'gt_masks'])
+]
+test_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(
+ type='MultiScaleFlipAug',
+ img_scale=(1333, 800),
+ flip=False,
+ transforms=[
+ dict(type='Resize', keep_ratio=True),
+ dict(type='RandomFlip'),
+ dict(
+ type='Normalize',
+ mean=[123.675, 116.28, 103.53],
+ std=[58.395, 57.12, 57.375],
+ to_rgb=True),
+ dict(type='Pad', size_divisor=32),
+ dict(type='ImageToTensor', keys=['img']),
+ dict(type='Collect', keys=['img'])
+ ])
+]
+data = dict(
+ samples_per_gpu=2,
+ workers_per_gpu=2,
+ train=dict(
+ type='CocoDataset',
+ ann_file='data/coco/annotations/instances_train2017.json',
+ img_prefix='data/coco/train2017/',
+ pipeline=[
+ dict(type='LoadImageFromFile'),
+ dict(type='LoadAnnotations', with_bbox=True, with_mask=True),
+ dict(type='Resize', img_scale=(1333, 800), keep_ratio=True),
+ dict(type='RandomFlip', flip_ratio=0.5),
+ dict(
+ type='Normalize',
+ mean=[123.675, 116.28, 103.53],
+ std=[58.395, 57.12, 57.375],
+ to_rgb=True),
+ dict(type='Pad', size_divisor=32),
+ dict(type='DefaultFormatBundle'),
+ dict(
+ type='Collect',
+ keys=['img', 'gt_bboxes', 'gt_labels', 'gt_masks'])
+ ]),
+ val=dict(
+ type='CocoDataset',
+ ann_file='data/coco/annotations/instances_val2017.json',
+ img_prefix='data/coco/val2017/',
+ pipeline=[
+ dict(type='LoadImageFromFile'),
+ dict(
+ type='MultiScaleFlipAug',
+ img_scale=(1333, 800),
+ flip=False,
+ transforms=[
+ dict(type='Resize', keep_ratio=True),
+ dict(type='RandomFlip'),
+ dict(
+ type='Normalize',
+ mean=[123.675, 116.28, 103.53],
+ std=[58.395, 57.12, 57.375],
+ to_rgb=True),
+ dict(type='Pad', size_divisor=32),
+ dict(type='ImageToTensor', keys=['img']),
+ dict(type='Collect', keys=['img'])
+ ])
+ ]),
+ test=dict(
+ type='CocoDataset',
+ ann_file='data/coco/annotations/instances_val2017.json',
+ img_prefix='data/coco/val2017/',
+ pipeline=[
+ dict(type='LoadImageFromFile'),
+ dict(
+ type='MultiScaleFlipAug',
+ img_scale=(1333, 800),
+ flip=False,
+ transforms=[
+ dict(type='Resize', keep_ratio=True),
+ dict(type='RandomFlip'),
+ dict(
+ type='Normalize',
+ mean=[123.675, 116.28, 103.53],
+ std=[58.395, 57.12, 57.375],
+ to_rgb=True),
+ dict(type='Pad', size_divisor=32),
+ dict(type='ImageToTensor', keys=['img']),
+ dict(type='Collect', keys=['img'])
+ ])
+ ]))
+evaluation = dict(metric=['bbox', 'segm'])
+optimizer = dict(type='SGD', lr=0.02, momentum=0.9, weight_decay=0.0001)
+optimizer_config = dict(grad_clip=None)
+lr_config = dict(
+ policy='step',
+ warmup='linear',
+ warmup_iters=500,
+ warmup_ratio=0.001,
+ step=[16, 22])
+runner = dict(type='EpochBasedRunner', max_epochs=24)
+checkpoint_config = dict(interval=1)
+log_config = dict(interval=50, hooks=[dict(type='TextLoggerHook')])
+custom_hooks = [dict(type='NumClassCheckHook')]
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/mmdetection_cfg/ssdlite_mobilenetv2_scratch_600e_coco.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/mmdetection_cfg/ssdlite_mobilenetv2_scratch_600e_coco.py
new file mode 100644
index 0000000..91b9e59
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/mmdetection_cfg/ssdlite_mobilenetv2_scratch_600e_coco.py
@@ -0,0 +1,216 @@
+# =========================================================
+# from 'mmdetection/configs/_base_/default_runtime.py'
+# =========================================================
+checkpoint_config = dict(interval=1)
+# yapf:disable
+log_config = dict(
+ interval=50,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook')
+ ])
+# yapf:enable
+custom_hooks = [dict(type='NumClassCheckHook')]
+# =========================================================
+
+# =========================================================
+# from 'mmdetection/configs/_base_/datasets/coco_detection.py'
+# =========================================================
+# dataset settings
+dataset_type = 'CocoDataset'
+data_root = 'data/coco/'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True)
+train_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='LoadAnnotations', with_bbox=True),
+ dict(type='Resize', img_scale=(1333, 800), keep_ratio=True),
+ dict(type='RandomFlip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='Pad', size_divisor=32),
+ dict(type='DefaultFormatBundle'),
+ dict(type='Collect', keys=['img', 'gt_bboxes', 'gt_labels']),
+]
+test_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(
+ type='MultiScaleFlipAug',
+ img_scale=(1333, 800),
+ flip=False,
+ transforms=[
+ dict(type='Resize', keep_ratio=True),
+ dict(type='RandomFlip'),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='Pad', size_divisor=32),
+ dict(type='ImageToTensor', keys=['img']),
+ dict(type='Collect', keys=['img']),
+ ])
+]
+data = dict(
+ samples_per_gpu=2,
+ workers_per_gpu=2,
+ train=dict(
+ type=dataset_type,
+ ann_file=data_root + 'annotations/instances_train2017.json',
+ img_prefix=data_root + 'train2017/',
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=data_root + 'annotations/instances_val2017.json',
+ img_prefix=data_root + 'val2017/',
+ pipeline=test_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=data_root + 'annotations/instances_val2017.json',
+ img_prefix=data_root + 'val2017/',
+ pipeline=test_pipeline))
+evaluation = dict(interval=1, metric='bbox')
+# =========================================================
+
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
+
+model = dict(
+ type='SingleStageDetector',
+ backbone=dict(
+ type='MobileNetV2',
+ out_indices=(4, 7),
+ norm_cfg=dict(type='BN', eps=0.001, momentum=0.03),
+ init_cfg=dict(type='TruncNormal', layer='Conv2d', std=0.03)),
+ neck=dict(
+ type='SSDNeck',
+ in_channels=(96, 1280),
+ out_channels=(96, 1280, 512, 256, 256, 128),
+ level_strides=(2, 2, 2, 2),
+ level_paddings=(1, 1, 1, 1),
+ l2_norm_scale=None,
+ use_depthwise=True,
+ norm_cfg=dict(type='BN', eps=0.001, momentum=0.03),
+ act_cfg=dict(type='ReLU6'),
+ init_cfg=dict(type='TruncNormal', layer='Conv2d', std=0.03)),
+ bbox_head=dict(
+ type='SSDHead',
+ in_channels=(96, 1280, 512, 256, 256, 128),
+ num_classes=80,
+ use_depthwise=True,
+ norm_cfg=dict(type='BN', eps=0.001, momentum=0.03),
+ act_cfg=dict(type='ReLU6'),
+ init_cfg=dict(type='Normal', layer='Conv2d', std=0.001),
+
+ # set anchor size manually instead of using the predefined
+ # SSD300 setting.
+ anchor_generator=dict(
+ type='SSDAnchorGenerator',
+ scale_major=False,
+ strides=[16, 32, 64, 107, 160, 320],
+ ratios=[[2, 3], [2, 3], [2, 3], [2, 3], [2, 3], [2, 3]],
+ min_sizes=[48, 100, 150, 202, 253, 304],
+ max_sizes=[100, 150, 202, 253, 304, 320]),
+ bbox_coder=dict(
+ type='DeltaXYWHBBoxCoder',
+ target_means=[.0, .0, .0, .0],
+ target_stds=[0.1, 0.1, 0.2, 0.2])),
+ # model training and testing settings
+ train_cfg=dict(
+ assigner=dict(
+ type='MaxIoUAssigner',
+ pos_iou_thr=0.5,
+ neg_iou_thr=0.5,
+ min_pos_iou=0.,
+ ignore_iof_thr=-1,
+ gt_max_assign_all=False),
+ smoothl1_beta=1.,
+ allowed_border=-1,
+ pos_weight=-1,
+ neg_pos_ratio=3,
+ debug=False),
+ test_cfg=dict(
+ nms_pre=1000,
+ nms=dict(type='nms', iou_threshold=0.45),
+ min_bbox_size=0,
+ score_thr=0.02,
+ max_per_img=200))
+cudnn_benchmark = True
+
+# dataset settings
+dataset_type = 'CocoDataset'
+data_root = 'data/coco/'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True)
+train_pipeline = [
+ dict(type='LoadImageFromFile', to_float32=True),
+ dict(type='LoadAnnotations', with_bbox=True),
+ dict(
+ type='PhotoMetricDistortion',
+ brightness_delta=32,
+ contrast_range=(0.5, 1.5),
+ saturation_range=(0.5, 1.5),
+ hue_delta=18),
+ dict(
+ type='Expand',
+ mean=img_norm_cfg['mean'],
+ to_rgb=img_norm_cfg['to_rgb'],
+ ratio_range=(1, 4)),
+ dict(
+ type='MinIoURandomCrop',
+ min_ious=(0.1, 0.3, 0.5, 0.7, 0.9),
+ min_crop_size=0.3),
+ dict(type='Resize', img_scale=(320, 320), keep_ratio=False),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='RandomFlip', flip_ratio=0.5),
+ dict(type='Pad', size_divisor=320),
+ dict(type='DefaultFormatBundle'),
+ dict(type='Collect', keys=['img', 'gt_bboxes', 'gt_labels']),
+]
+test_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(
+ type='MultiScaleFlipAug',
+ img_scale=(320, 320),
+ flip=False,
+ transforms=[
+ dict(type='Resize', keep_ratio=False),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='Pad', size_divisor=320),
+ dict(type='ImageToTensor', keys=['img']),
+ dict(type='Collect', keys=['img']),
+ ])
+]
+data = dict(
+ samples_per_gpu=24,
+ workers_per_gpu=4,
+ train=dict(
+ _delete_=True,
+ type='RepeatDataset', # use RepeatDataset to speed up training
+ times=5,
+ dataset=dict(
+ type=dataset_type,
+ ann_file=data_root + 'annotations/instances_train2017.json',
+ img_prefix=data_root + 'train2017/',
+ pipeline=train_pipeline)),
+ val=dict(pipeline=test_pipeline),
+ test=dict(pipeline=test_pipeline))
+
+# optimizer
+optimizer = dict(type='SGD', lr=0.015, momentum=0.9, weight_decay=4.0e-5)
+optimizer_config = dict(grad_clip=None)
+
+# learning policy
+lr_config = dict(
+ policy='CosineAnnealing',
+ warmup='linear',
+ warmup_iters=500,
+ warmup_ratio=0.001,
+ min_lr=0)
+runner = dict(type='EpochBasedRunner', max_epochs=120)
+
+# Avoid evaluation and saving weights too frequently
+evaluation = dict(interval=5, metric='bbox')
+checkpoint_config = dict(interval=5)
+custom_hooks = [
+ dict(type='NumClassCheckHook'),
+ dict(type='CheckInvalidLossHook', interval=50, priority='VERY_LOW')
+]
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/mmdetection_cfg/yolov3_d53_320_273e_coco.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/mmdetection_cfg/yolov3_d53_320_273e_coco.py
new file mode 100644
index 0000000..d7e9cca
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/mmdetection_cfg/yolov3_d53_320_273e_coco.py
@@ -0,0 +1,140 @@
+# model settings
+model = dict(
+ type='YOLOV3',
+ pretrained='open-mmlab://darknet53',
+ backbone=dict(type='Darknet', depth=53, out_indices=(3, 4, 5)),
+ neck=dict(
+ type='YOLOV3Neck',
+ num_scales=3,
+ in_channels=[1024, 512, 256],
+ out_channels=[512, 256, 128]),
+ bbox_head=dict(
+ type='YOLOV3Head',
+ num_classes=80,
+ in_channels=[512, 256, 128],
+ out_channels=[1024, 512, 256],
+ anchor_generator=dict(
+ type='YOLOAnchorGenerator',
+ base_sizes=[[(116, 90), (156, 198), (373, 326)],
+ [(30, 61), (62, 45), (59, 119)],
+ [(10, 13), (16, 30), (33, 23)]],
+ strides=[32, 16, 8]),
+ bbox_coder=dict(type='YOLOBBoxCoder'),
+ featmap_strides=[32, 16, 8],
+ loss_cls=dict(
+ type='CrossEntropyLoss',
+ use_sigmoid=True,
+ loss_weight=1.0,
+ reduction='sum'),
+ loss_conf=dict(
+ type='CrossEntropyLoss',
+ use_sigmoid=True,
+ loss_weight=1.0,
+ reduction='sum'),
+ loss_xy=dict(
+ type='CrossEntropyLoss',
+ use_sigmoid=True,
+ loss_weight=2.0,
+ reduction='sum'),
+ loss_wh=dict(type='MSELoss', loss_weight=2.0, reduction='sum')),
+ # training and testing settings
+ train_cfg=dict(
+ assigner=dict(
+ type='GridAssigner',
+ pos_iou_thr=0.5,
+ neg_iou_thr=0.5,
+ min_pos_iou=0)),
+ test_cfg=dict(
+ nms_pre=1000,
+ min_bbox_size=0,
+ score_thr=0.05,
+ conf_thr=0.005,
+ nms=dict(type='nms', iou_threshold=0.45),
+ max_per_img=100))
+# dataset settings
+dataset_type = 'CocoDataset'
+data_root = 'data/coco'
+img_norm_cfg = dict(mean=[0, 0, 0], std=[255., 255., 255.], to_rgb=True)
+train_pipeline = [
+ dict(type='LoadImageFromFile', to_float32=True),
+ dict(type='LoadAnnotations', with_bbox=True),
+ dict(type='PhotoMetricDistortion'),
+ dict(
+ type='Expand',
+ mean=img_norm_cfg['mean'],
+ to_rgb=img_norm_cfg['to_rgb'],
+ ratio_range=(1, 2)),
+ dict(
+ type='MinIoURandomCrop',
+ min_ious=(0.4, 0.5, 0.6, 0.7, 0.8, 0.9),
+ min_crop_size=0.3),
+ dict(type='Resize', img_scale=(320, 320), keep_ratio=True),
+ dict(type='RandomFlip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='Pad', size_divisor=32),
+ dict(type='DefaultFormatBundle'),
+ dict(type='Collect', keys=['img', 'gt_bboxes', 'gt_labels'])
+]
+test_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(
+ type='MultiScaleFlipAug',
+ img_scale=(320, 320),
+ flip=False,
+ transforms=[
+ dict(type='Resize', keep_ratio=True),
+ dict(type='RandomFlip'),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='Pad', size_divisor=32),
+ dict(type='DefaultFormatBundle'),
+ dict(type='Collect', keys=['img'])
+ ])
+]
+data = dict(
+ samples_per_gpu=8,
+ workers_per_gpu=4,
+ train=dict(
+ type=dataset_type,
+ ann_file=f'{data_root}/annotations/instances_train2017.json',
+ img_prefix=f'{data_root}/train2017/',
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=f'{data_root}/annotations/instances_val2017.json',
+ img_prefix=f'{data_root}/val2017/',
+ pipeline=test_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=f'{data_root}/annotations/instances_val2017.json',
+ img_prefix=f'{data_root}/val2017/',
+ pipeline=test_pipeline))
+# optimizer
+optimizer = dict(type='SGD', lr=0.001, momentum=0.9, weight_decay=0.0005)
+optimizer_config = dict(grad_clip=dict(max_norm=35, norm_type=2))
+# learning policy
+lr_config = dict(
+ policy='step',
+ warmup='linear',
+ warmup_iters=2000, # same as burn-in in darknet
+ warmup_ratio=0.1,
+ step=[218, 246])
+# runtime settings
+runner = dict(type='EpochBasedRunner', max_epochs=273)
+evaluation = dict(interval=1, metric=['bbox'])
+
+checkpoint_config = dict(interval=1)
+# yapf:disable
+log_config = dict(
+ interval=50,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook')
+ ])
+# yapf:enable
+custom_hooks = [dict(type='NumClassCheckHook')]
+
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/mmtracking_cfg/deepsort_faster-rcnn_fpn_4e_mot17-private-half.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/mmtracking_cfg/deepsort_faster-rcnn_fpn_4e_mot17-private-half.py
new file mode 100644
index 0000000..1d7fccf
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/mmtracking_cfg/deepsort_faster-rcnn_fpn_4e_mot17-private-half.py
@@ -0,0 +1,321 @@
+model = dict(
+ detector=dict(
+ type='FasterRCNN',
+ backbone=dict(
+ type='ResNet',
+ depth=50,
+ num_stages=4,
+ out_indices=(0, 1, 2, 3),
+ frozen_stages=1,
+ norm_cfg=dict(type='BN', requires_grad=True),
+ norm_eval=True,
+ style='pytorch',
+ init_cfg=dict(
+ type='Pretrained', checkpoint='torchvision://resnet50')),
+ neck=dict(
+ type='FPN',
+ in_channels=[256, 512, 1024, 2048],
+ out_channels=256,
+ num_outs=5),
+ rpn_head=dict(
+ type='RPNHead',
+ in_channels=256,
+ feat_channels=256,
+ anchor_generator=dict(
+ type='AnchorGenerator',
+ scales=[8],
+ ratios=[0.5, 1.0, 2.0],
+ strides=[4, 8, 16, 32, 64]),
+ bbox_coder=dict(
+ type='DeltaXYWHBBoxCoder',
+ target_means=[0.0, 0.0, 0.0, 0.0],
+ target_stds=[1.0, 1.0, 1.0, 1.0],
+ clip_border=False),
+ loss_cls=dict(
+ type='CrossEntropyLoss', use_sigmoid=True, loss_weight=1.0),
+ loss_bbox=dict(
+ type='SmoothL1Loss', beta=0.1111111111111111,
+ loss_weight=1.0)),
+ roi_head=dict(
+ type='StandardRoIHead',
+ bbox_roi_extractor=dict(
+ type='SingleRoIExtractor',
+ roi_layer=dict(
+ type='RoIAlign', output_size=7, sampling_ratio=0),
+ out_channels=256,
+ featmap_strides=[4, 8, 16, 32]),
+ bbox_head=dict(
+ type='Shared2FCBBoxHead',
+ in_channels=256,
+ fc_out_channels=1024,
+ roi_feat_size=7,
+ num_classes=1,
+ bbox_coder=dict(
+ type='DeltaXYWHBBoxCoder',
+ target_means=[0.0, 0.0, 0.0, 0.0],
+ target_stds=[0.1, 0.1, 0.2, 0.2],
+ clip_border=False),
+ reg_class_agnostic=False,
+ loss_cls=dict(
+ type='CrossEntropyLoss',
+ use_sigmoid=False,
+ loss_weight=1.0),
+ loss_bbox=dict(type='SmoothL1Loss', loss_weight=1.0))),
+ train_cfg=dict(
+ rpn=dict(
+ assigner=dict(
+ type='MaxIoUAssigner',
+ pos_iou_thr=0.7,
+ neg_iou_thr=0.3,
+ min_pos_iou=0.3,
+ match_low_quality=True,
+ ignore_iof_thr=-1),
+ sampler=dict(
+ type='RandomSampler',
+ num=256,
+ pos_fraction=0.5,
+ neg_pos_ub=-1,
+ add_gt_as_proposals=False),
+ allowed_border=-1,
+ pos_weight=-1,
+ debug=False),
+ rpn_proposal=dict(
+ nms_pre=2000,
+ max_per_img=1000,
+ nms=dict(type='nms', iou_threshold=0.7),
+ min_bbox_size=0),
+ rcnn=dict(
+ assigner=dict(
+ type='MaxIoUAssigner',
+ pos_iou_thr=0.5,
+ neg_iou_thr=0.5,
+ min_pos_iou=0.5,
+ match_low_quality=False,
+ ignore_iof_thr=-1),
+ sampler=dict(
+ type='RandomSampler',
+ num=512,
+ pos_fraction=0.25,
+ neg_pos_ub=-1,
+ add_gt_as_proposals=True),
+ pos_weight=-1,
+ debug=False)),
+ test_cfg=dict(
+ rpn=dict(
+ nms_pre=1000,
+ max_per_img=1000,
+ nms=dict(type='nms', iou_threshold=0.7),
+ min_bbox_size=0),
+ rcnn=dict(
+ score_thr=0.05,
+ nms=dict(type='nms', iou_threshold=0.5),
+ max_per_img=100)),
+ init_cfg=dict(
+ type='Pretrained',
+ checkpoint='https://download.openmmlab.com/mmtracking/'
+ 'mot/faster_rcnn/faster-rcnn_r50_fpn_4e_mot17-half-64ee2ed4.pth')),
+ type='DeepSORT',
+ motion=dict(type='KalmanFilter', center_only=False),
+ reid=dict(
+ type='BaseReID',
+ backbone=dict(
+ type='ResNet',
+ depth=50,
+ num_stages=4,
+ out_indices=(3, ),
+ style='pytorch'),
+ neck=dict(type='GlobalAveragePooling', kernel_size=(8, 4), stride=1),
+ head=dict(
+ type='LinearReIDHead',
+ num_fcs=1,
+ in_channels=2048,
+ fc_channels=1024,
+ out_channels=128,
+ num_classes=380,
+ loss=dict(type='CrossEntropyLoss', loss_weight=1.0),
+ loss_pairwise=dict(
+ type='TripletLoss', margin=0.3, loss_weight=1.0),
+ norm_cfg=dict(type='BN1d'),
+ act_cfg=dict(type='ReLU')),
+ init_cfg=dict(
+ type='Pretrained',
+ checkpoint='https://download.openmmlab.com/mmtracking/'
+ 'mot/reid/tracktor_reid_r50_iter25245-a452f51f.pth')),
+ tracker=dict(
+ type='SortTracker',
+ obj_score_thr=0.5,
+ reid=dict(
+ num_samples=10,
+ img_scale=(256, 128),
+ img_norm_cfg=None,
+ match_score_thr=2.0),
+ match_iou_thr=0.5,
+ momentums=None,
+ num_tentatives=2,
+ num_frames_retain=100))
+dataset_type = 'MOTChallengeDataset'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True)
+train_pipeline = [
+ dict(type='LoadMultiImagesFromFile', to_float32=True),
+ dict(type='SeqLoadAnnotations', with_bbox=True, with_track=True),
+ dict(
+ type='SeqResize',
+ img_scale=(1088, 1088),
+ share_params=True,
+ ratio_range=(0.8, 1.2),
+ keep_ratio=True,
+ bbox_clip_border=False),
+ dict(type='SeqPhotoMetricDistortion', share_params=True),
+ dict(
+ type='SeqRandomCrop',
+ share_params=False,
+ crop_size=(1088, 1088),
+ bbox_clip_border=False),
+ dict(type='SeqRandomFlip', share_params=True, flip_ratio=0.5),
+ dict(
+ type='SeqNormalize',
+ mean=[123.675, 116.28, 103.53],
+ std=[58.395, 57.12, 57.375],
+ to_rgb=True),
+ dict(type='SeqPad', size_divisor=32),
+ dict(type='MatchInstances', skip_nomatch=True),
+ dict(
+ type='VideoCollect',
+ keys=[
+ 'img', 'gt_bboxes', 'gt_labels', 'gt_match_indices',
+ 'gt_instance_ids'
+ ]),
+ dict(type='SeqDefaultFormatBundle', ref_prefix='ref')
+]
+test_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(
+ type='MultiScaleFlipAug',
+ img_scale=(1088, 1088),
+ flip=False,
+ transforms=[
+ dict(type='Resize', keep_ratio=True),
+ dict(type='RandomFlip'),
+ dict(
+ type='Normalize',
+ mean=[123.675, 116.28, 103.53],
+ std=[58.395, 57.12, 57.375],
+ to_rgb=True),
+ dict(type='Pad', size_divisor=32),
+ dict(type='ImageToTensor', keys=['img']),
+ dict(type='VideoCollect', keys=['img'])
+ ])
+]
+data_root = 'data/MOT17/'
+data = dict(
+ samples_per_gpu=2,
+ workers_per_gpu=2,
+ train=dict(
+ type='MOTChallengeDataset',
+ visibility_thr=-1,
+ ann_file='data/MOT17/annotations/half-train_cocoformat.json',
+ img_prefix='data/MOT17/train',
+ ref_img_sampler=dict(
+ num_ref_imgs=1,
+ frame_range=10,
+ filter_key_img=True,
+ method='uniform'),
+ pipeline=[
+ dict(type='LoadMultiImagesFromFile', to_float32=True),
+ dict(type='SeqLoadAnnotations', with_bbox=True, with_track=True),
+ dict(
+ type='SeqResize',
+ img_scale=(1088, 1088),
+ share_params=True,
+ ratio_range=(0.8, 1.2),
+ keep_ratio=True,
+ bbox_clip_border=False),
+ dict(type='SeqPhotoMetricDistortion', share_params=True),
+ dict(
+ type='SeqRandomCrop',
+ share_params=False,
+ crop_size=(1088, 1088),
+ bbox_clip_border=False),
+ dict(type='SeqRandomFlip', share_params=True, flip_ratio=0.5),
+ dict(
+ type='SeqNormalize',
+ mean=[123.675, 116.28, 103.53],
+ std=[58.395, 57.12, 57.375],
+ to_rgb=True),
+ dict(type='SeqPad', size_divisor=32),
+ dict(type='MatchInstances', skip_nomatch=True),
+ dict(
+ type='VideoCollect',
+ keys=[
+ 'img', 'gt_bboxes', 'gt_labels', 'gt_match_indices',
+ 'gt_instance_ids'
+ ]),
+ dict(type='SeqDefaultFormatBundle', ref_prefix='ref')
+ ]),
+ val=dict(
+ type='MOTChallengeDataset',
+ ann_file='data/MOT17/annotations/half-val_cocoformat.json',
+ img_prefix='data/MOT17/train',
+ ref_img_sampler=None,
+ pipeline=[
+ dict(type='LoadImageFromFile'),
+ dict(
+ type='MultiScaleFlipAug',
+ img_scale=(1088, 1088),
+ flip=False,
+ transforms=[
+ dict(type='Resize', keep_ratio=True),
+ dict(type='RandomFlip'),
+ dict(
+ type='Normalize',
+ mean=[123.675, 116.28, 103.53],
+ std=[58.395, 57.12, 57.375],
+ to_rgb=True),
+ dict(type='Pad', size_divisor=32),
+ dict(type='ImageToTensor', keys=['img']),
+ dict(type='VideoCollect', keys=['img'])
+ ])
+ ]),
+ test=dict(
+ type='MOTChallengeDataset',
+ ann_file='data/MOT17/annotations/half-val_cocoformat.json',
+ img_prefix='data/MOT17/train',
+ ref_img_sampler=None,
+ pipeline=[
+ dict(type='LoadImageFromFile'),
+ dict(
+ type='MultiScaleFlipAug',
+ img_scale=(1088, 1088),
+ flip=False,
+ transforms=[
+ dict(type='Resize', keep_ratio=True),
+ dict(type='RandomFlip'),
+ dict(
+ type='Normalize',
+ mean=[123.675, 116.28, 103.53],
+ std=[58.395, 57.12, 57.375],
+ to_rgb=True),
+ dict(type='Pad', size_divisor=32),
+ dict(type='ImageToTensor', keys=['img']),
+ dict(type='VideoCollect', keys=['img'])
+ ])
+ ]))
+optimizer = dict(type='SGD', lr=0.02, momentum=0.9, weight_decay=0.0001)
+optimizer_config = dict(grad_clip=None)
+checkpoint_config = dict(interval=1)
+log_config = dict(interval=50, hooks=[dict(type='TextLoggerHook')])
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
+lr_config = dict(
+ policy='step',
+ warmup='linear',
+ warmup_iters=100,
+ warmup_ratio=0.01,
+ step=[3])
+total_epochs = 4
+evaluation = dict(metric=['bbox', 'track'], interval=1)
+search_metrics = ['MOTA', 'IDF1', 'FN', 'FP', 'IDs', 'MT', 'ML']
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/mmtracking_cfg/tracktor_faster-rcnn_r50_fpn_4e_mot17-private.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/mmtracking_cfg/tracktor_faster-rcnn_r50_fpn_4e_mot17-private.py
new file mode 100644
index 0000000..9736269
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/mmtracking_cfg/tracktor_faster-rcnn_r50_fpn_4e_mot17-private.py
@@ -0,0 +1,325 @@
+model = dict(
+ detector=dict(
+ type='FasterRCNN',
+ pretrained='torchvision://resnet50',
+ backbone=dict(
+ type='ResNet',
+ depth=50,
+ num_stages=4,
+ out_indices=(0, 1, 2, 3),
+ frozen_stages=1,
+ norm_cfg=dict(type='BN', requires_grad=True),
+ norm_eval=True,
+ style='pytorch'),
+ neck=dict(
+ type='FPN',
+ in_channels=[256, 512, 1024, 2048],
+ out_channels=256,
+ num_outs=5),
+ rpn_head=dict(
+ type='RPNHead',
+ in_channels=256,
+ feat_channels=256,
+ anchor_generator=dict(
+ type='AnchorGenerator',
+ scales=[8],
+ ratios=[0.5, 1.0, 2.0],
+ strides=[4, 8, 16, 32, 64]),
+ bbox_coder=dict(
+ type='DeltaXYWHBBoxCoder',
+ target_means=[0.0, 0.0, 0.0, 0.0],
+ target_stds=[1.0, 1.0, 1.0, 1.0],
+ clip_border=False),
+ loss_cls=dict(
+ type='CrossEntropyLoss', use_sigmoid=True, loss_weight=1.0),
+ loss_bbox=dict(
+ type='SmoothL1Loss', beta=0.1111111111111111,
+ loss_weight=1.0)),
+ roi_head=dict(
+ type='StandardRoIHead',
+ bbox_roi_extractor=dict(
+ type='SingleRoIExtractor',
+ roi_layer=dict(
+ type='RoIAlign', output_size=7, sampling_ratio=0),
+ out_channels=256,
+ featmap_strides=[4, 8, 16, 32]),
+ bbox_head=dict(
+ type='Shared2FCBBoxHead',
+ in_channels=256,
+ fc_out_channels=1024,
+ roi_feat_size=7,
+ num_classes=1,
+ bbox_coder=dict(
+ type='DeltaXYWHBBoxCoder',
+ target_means=[0.0, 0.0, 0.0, 0.0],
+ target_stds=[0.1, 0.1, 0.2, 0.2],
+ clip_border=False),
+ reg_class_agnostic=False,
+ loss_cls=dict(
+ type='CrossEntropyLoss',
+ use_sigmoid=False,
+ loss_weight=1.0),
+ loss_bbox=dict(type='SmoothL1Loss', loss_weight=1.0))),
+ train_cfg=dict(
+ rpn=dict(
+ assigner=dict(
+ type='MaxIoUAssigner',
+ pos_iou_thr=0.7,
+ neg_iou_thr=0.3,
+ min_pos_iou=0.3,
+ match_low_quality=True,
+ ignore_iof_thr=-1),
+ sampler=dict(
+ type='RandomSampler',
+ num=256,
+ pos_fraction=0.5,
+ neg_pos_ub=-1,
+ add_gt_as_proposals=False),
+ allowed_border=-1,
+ pos_weight=-1,
+ debug=False),
+ rpn_proposal=dict(
+ nms_pre=2000,
+ max_per_img=1000,
+ nms=dict(type='nms', iou_threshold=0.7),
+ min_bbox_size=0),
+ rcnn=dict(
+ assigner=dict(
+ type='MaxIoUAssigner',
+ pos_iou_thr=0.5,
+ neg_iou_thr=0.5,
+ min_pos_iou=0.5,
+ match_low_quality=False,
+ ignore_iof_thr=-1),
+ sampler=dict(
+ type='RandomSampler',
+ num=512,
+ pos_fraction=0.25,
+ neg_pos_ub=-1,
+ add_gt_as_proposals=True),
+ pos_weight=-1,
+ debug=False)),
+ test_cfg=dict(
+ rpn=dict(
+ nms_pre=1000,
+ max_per_img=1000,
+ nms=dict(type='nms', iou_threshold=0.7),
+ min_bbox_size=0),
+ rcnn=dict(
+ score_thr=0.05,
+ nms=dict(type='nms', iou_threshold=0.5),
+ max_per_img=100))),
+ type='Tracktor',
+ pretrains=dict(
+ detector='https://download.openmmlab.com/mmtracking/'
+ 'mot/faster_rcnn/faster-rcnn_r50_fpn_4e_mot17-ffa52ae7.pth',
+ reid='https://download.openmmlab.com/mmtracking/mot/'
+ 'reid/reid_r50_6e_mot17-4bf6b63d.pth'),
+ reid=dict(
+ type='BaseReID',
+ backbone=dict(
+ type='ResNet',
+ depth=50,
+ num_stages=4,
+ out_indices=(3, ),
+ style='pytorch'),
+ neck=dict(type='GlobalAveragePooling', kernel_size=(8, 4), stride=1),
+ head=dict(
+ type='LinearReIDHead',
+ num_fcs=1,
+ in_channels=2048,
+ fc_channels=1024,
+ out_channels=128,
+ num_classes=378,
+ loss=dict(type='CrossEntropyLoss', loss_weight=1.0),
+ loss_pairwise=dict(
+ type='TripletLoss', margin=0.3, loss_weight=1.0),
+ norm_cfg=dict(type='BN1d'),
+ act_cfg=dict(type='ReLU'))),
+ motion=dict(
+ type='CameraMotionCompensation',
+ warp_mode='cv2.MOTION_EUCLIDEAN',
+ num_iters=100,
+ stop_eps=1e-05),
+ tracker=dict(
+ type='TracktorTracker',
+ obj_score_thr=0.5,
+ regression=dict(
+ obj_score_thr=0.5,
+ nms=dict(type='nms', iou_threshold=0.6),
+ match_iou_thr=0.3),
+ reid=dict(
+ num_samples=10,
+ img_scale=(256, 128),
+ img_norm_cfg=None,
+ match_score_thr=2.0,
+ match_iou_thr=0.2),
+ momentums=None,
+ num_frames_retain=10))
+dataset_type = 'MOTChallengeDataset'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True)
+train_pipeline = [
+ dict(type='LoadMultiImagesFromFile', to_float32=True),
+ dict(type='SeqLoadAnnotations', with_bbox=True, with_track=True),
+ dict(
+ type='SeqResize',
+ img_scale=(1088, 1088),
+ share_params=True,
+ ratio_range=(0.8, 1.2),
+ keep_ratio=True,
+ bbox_clip_border=False),
+ dict(type='SeqPhotoMetricDistortion', share_params=True),
+ dict(
+ type='SeqRandomCrop',
+ share_params=False,
+ crop_size=(1088, 1088),
+ bbox_clip_border=False),
+ dict(type='SeqRandomFlip', share_params=True, flip_ratio=0.5),
+ dict(
+ type='SeqNormalize',
+ mean=[123.675, 116.28, 103.53],
+ std=[58.395, 57.12, 57.375],
+ to_rgb=True),
+ dict(type='SeqPad', size_divisor=32),
+ dict(type='MatchInstances', skip_nomatch=True),
+ dict(
+ type='VideoCollect',
+ keys=[
+ 'img', 'gt_bboxes', 'gt_labels', 'gt_match_indices',
+ 'gt_instance_ids'
+ ]),
+ dict(type='SeqDefaultFormatBundle', ref_prefix='ref')
+]
+test_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(
+ type='MultiScaleFlipAug',
+ img_scale=(1088, 1088),
+ flip=False,
+ transforms=[
+ dict(type='Resize', keep_ratio=True),
+ dict(type='RandomFlip'),
+ dict(
+ type='Normalize',
+ mean=[123.675, 116.28, 103.53],
+ std=[58.395, 57.12, 57.375],
+ to_rgb=True),
+ dict(type='Pad', size_divisor=32),
+ dict(type='ImageToTensor', keys=['img']),
+ dict(type='VideoCollect', keys=['img'])
+ ])
+]
+data_root = 'data/MOT17/'
+data = dict(
+ samples_per_gpu=2,
+ workers_per_gpu=2,
+ train=dict(
+ type='MOTChallengeDataset',
+ visibility_thr=-1,
+ ann_file='data/MOT17/annotations/train_cocoformat.json',
+ img_prefix='data/MOT17/train',
+ ref_img_sampler=dict(
+ num_ref_imgs=1,
+ frame_range=10,
+ filter_key_img=True,
+ method='uniform'),
+ pipeline=[
+ dict(type='LoadMultiImagesFromFile', to_float32=True),
+ dict(type='SeqLoadAnnotations', with_bbox=True, with_track=True),
+ dict(
+ type='SeqResize',
+ img_scale=(1088, 1088),
+ share_params=True,
+ ratio_range=(0.8, 1.2),
+ keep_ratio=True,
+ bbox_clip_border=False),
+ dict(type='SeqPhotoMetricDistortion', share_params=True),
+ dict(
+ type='SeqRandomCrop',
+ share_params=False,
+ crop_size=(1088, 1088),
+ bbox_clip_border=False),
+ dict(type='SeqRandomFlip', share_params=True, flip_ratio=0.5),
+ dict(
+ type='SeqNormalize',
+ mean=[123.675, 116.28, 103.53],
+ std=[58.395, 57.12, 57.375],
+ to_rgb=True),
+ dict(type='SeqPad', size_divisor=32),
+ dict(type='MatchInstances', skip_nomatch=True),
+ dict(
+ type='VideoCollect',
+ keys=[
+ 'img', 'gt_bboxes', 'gt_labels', 'gt_match_indices',
+ 'gt_instance_ids'
+ ]),
+ dict(type='SeqDefaultFormatBundle', ref_prefix='ref')
+ ]),
+ val=dict(
+ type='MOTChallengeDataset',
+ ann_file='data/MOT17/annotations/train_cocoformat.json',
+ img_prefix='data/MOT17/train',
+ ref_img_sampler=None,
+ pipeline=[
+ dict(type='LoadImageFromFile'),
+ dict(
+ type='MultiScaleFlipAug',
+ img_scale=(1088, 1088),
+ flip=False,
+ transforms=[
+ dict(type='Resize', keep_ratio=True),
+ dict(type='RandomFlip'),
+ dict(
+ type='Normalize',
+ mean=[123.675, 116.28, 103.53],
+ std=[58.395, 57.12, 57.375],
+ to_rgb=True),
+ dict(type='Pad', size_divisor=32),
+ dict(type='ImageToTensor', keys=['img']),
+ dict(type='VideoCollect', keys=['img'])
+ ])
+ ]),
+ test=dict(
+ type='MOTChallengeDataset',
+ ann_file='data/MOT17/annotations/train_cocoformat.json',
+ img_prefix='data/MOT17/train',
+ ref_img_sampler=None,
+ pipeline=[
+ dict(type='LoadImageFromFile'),
+ dict(
+ type='MultiScaleFlipAug',
+ img_scale=(1088, 1088),
+ flip=False,
+ transforms=[
+ dict(type='Resize', keep_ratio=True),
+ dict(type='RandomFlip'),
+ dict(
+ type='Normalize',
+ mean=[123.675, 116.28, 103.53],
+ std=[58.395, 57.12, 57.375],
+ to_rgb=True),
+ dict(type='Pad', size_divisor=32),
+ dict(type='ImageToTensor', keys=['img']),
+ dict(type='VideoCollect', keys=['img'])
+ ])
+ ]))
+optimizer = dict(type='SGD', lr=0.02, momentum=0.9, weight_decay=0.0001)
+optimizer_config = dict(grad_clip=None)
+checkpoint_config = dict(interval=1)
+log_config = dict(interval=50, hooks=[dict(type='TextLoggerHook')])
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
+lr_config = dict(
+ policy='step',
+ warmup='linear',
+ warmup_iters=100,
+ warmup_ratio=0.01,
+ step=[3])
+total_epochs = 4
+evaluation = dict(metric=['bbox', 'track'], interval=1)
+search_metrics = ['MOTA', 'IDF1', 'FN', 'FP', 'IDs', 'MT', 'ML']
+test_set = 'train'
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/resources/demo.mp4 b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/resources/demo.mp4
new file mode 100644
index 0000000..2ba10c2
Binary files /dev/null and b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/resources/demo.mp4 differ
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/resources/demo_coco.gif b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/resources/demo_coco.gif
new file mode 100644
index 0000000..a5488e3
Binary files /dev/null and b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/resources/demo_coco.gif differ
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/top_down_img_demo.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/top_down_img_demo.py
new file mode 100644
index 0000000..da16978
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/top_down_img_demo.py
@@ -0,0 +1,129 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os
+import warnings
+from argparse import ArgumentParser
+
+from xtcocotools.coco import COCO
+
+from mmpose.apis import (inference_top_down_pose_model, init_pose_model,
+ vis_pose_result)
+from mmpose.datasets import DatasetInfo
+
+
+def main():
+ """Visualize the demo images.
+
+ Require the json_file containing boxes.
+ """
+ parser = ArgumentParser()
+ parser.add_argument('pose_config', help='Config file for detection')
+ parser.add_argument('pose_checkpoint', help='Checkpoint file')
+ parser.add_argument('--img-root', type=str, default='', help='Image root')
+ parser.add_argument(
+ '--json-file',
+ type=str,
+ default='',
+ help='Json file containing image info.')
+ parser.add_argument(
+ '--show',
+ action='store_true',
+ default=False,
+ help='whether to show img')
+ parser.add_argument(
+ '--out-img-root',
+ type=str,
+ default='',
+ help='Root of the output img file. '
+ 'Default not saving the visualization images.')
+ parser.add_argument(
+ '--device', default='cuda:0', help='Device used for inference')
+ parser.add_argument(
+ '--kpt-thr', type=float, default=0.3, help='Keypoint score threshold')
+ parser.add_argument(
+ '--radius',
+ type=int,
+ default=4,
+ help='Keypoint radius for visualization')
+ parser.add_argument(
+ '--thickness',
+ type=int,
+ default=1,
+ help='Link thickness for visualization')
+
+ args = parser.parse_args()
+
+ assert args.show or (args.out_img_root != '')
+
+ coco = COCO(args.json_file)
+ # build the pose model from a config file and a checkpoint file
+ pose_model = init_pose_model(
+ args.pose_config, args.pose_checkpoint, device=args.device.lower())
+
+ dataset = pose_model.cfg.data['test']['type']
+ dataset_info = pose_model.cfg.data['test'].get('dataset_info', None)
+ if dataset_info is None:
+ warnings.warn(
+ 'Please set `dataset_info` in the config.'
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 for details.',
+ DeprecationWarning)
+ else:
+ dataset_info = DatasetInfo(dataset_info)
+
+ img_keys = list(coco.imgs.keys())
+
+ # optional
+ return_heatmap = False
+
+ # e.g. use ('backbone', ) to return backbone feature
+ output_layer_names = None
+
+ # process each image
+ for i in range(len(img_keys)):
+ # get bounding box annotations
+ image_id = img_keys[i]
+ image = coco.loadImgs(image_id)[0]
+ image_name = os.path.join(args.img_root, image['file_name'])
+ ann_ids = coco.getAnnIds(image_id)
+
+ # make person bounding boxes
+ person_results = []
+ for ann_id in ann_ids:
+ person = {}
+ ann = coco.anns[ann_id]
+ # bbox format is 'xywh'
+ person['bbox'] = ann['bbox']
+ person_results.append(person)
+
+ # test a single image, with a list of bboxes
+ pose_results, returned_outputs = inference_top_down_pose_model(
+ pose_model,
+ image_name,
+ person_results,
+ bbox_thr=None,
+ format='xywh',
+ dataset=dataset,
+ dataset_info=dataset_info,
+ return_heatmap=return_heatmap,
+ outputs=output_layer_names)
+
+ if args.out_img_root == '':
+ out_file = None
+ else:
+ os.makedirs(args.out_img_root, exist_ok=True)
+ out_file = os.path.join(args.out_img_root, f'vis_{i}.jpg')
+
+ vis_pose_result(
+ pose_model,
+ image_name,
+ pose_results,
+ dataset=dataset,
+ dataset_info=dataset_info,
+ kpt_score_thr=args.kpt_thr,
+ radius=args.radius,
+ thickness=args.thickness,
+ show=args.show,
+ out_file=out_file)
+
+
+if __name__ == '__main__':
+ main()
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/top_down_img_demo_with_mmdet.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/top_down_img_demo_with_mmdet.py
new file mode 100644
index 0000000..227f44b
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/top_down_img_demo_with_mmdet.py
@@ -0,0 +1,138 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os
+import warnings
+from argparse import ArgumentParser
+
+from mmpose.apis import (inference_top_down_pose_model, init_pose_model,
+ process_mmdet_results, vis_pose_result)
+from mmpose.datasets import DatasetInfo
+
+try:
+ from mmdet.apis import inference_detector, init_detector
+ has_mmdet = True
+except (ImportError, ModuleNotFoundError):
+ has_mmdet = False
+
+
+def main():
+ """Visualize the demo images.
+
+ Using mmdet to detect the human.
+ """
+ parser = ArgumentParser()
+ parser.add_argument('det_config', help='Config file for detection')
+ parser.add_argument('det_checkpoint', help='Checkpoint file for detection')
+ parser.add_argument('pose_config', help='Config file for pose')
+ parser.add_argument('pose_checkpoint', help='Checkpoint file for pose')
+ parser.add_argument('--img-root', type=str, default='', help='Image root')
+ parser.add_argument('--img', type=str, default='', help='Image file')
+ parser.add_argument(
+ '--show',
+ action='store_true',
+ default=False,
+ help='whether to show img')
+ parser.add_argument(
+ '--out-img-root',
+ type=str,
+ default='',
+ help='root of the output img file. '
+ 'Default not saving the visualization images.')
+ parser.add_argument(
+ '--device', default='cuda:0', help='Device used for inference')
+ parser.add_argument(
+ '--det-cat-id',
+ type=int,
+ default=1,
+ help='Category id for bounding box detection model')
+ parser.add_argument(
+ '--bbox-thr',
+ type=float,
+ default=0.3,
+ help='Bounding box score threshold')
+ parser.add_argument(
+ '--kpt-thr', type=float, default=0.3, help='Keypoint score threshold')
+ parser.add_argument(
+ '--radius',
+ type=int,
+ default=4,
+ help='Keypoint radius for visualization')
+ parser.add_argument(
+ '--thickness',
+ type=int,
+ default=1,
+ help='Link thickness for visualization')
+
+ assert has_mmdet, 'Please install mmdet to run the demo.'
+
+ args = parser.parse_args()
+
+ assert args.show or (args.out_img_root != '')
+ assert args.img != ''
+ assert args.det_config is not None
+ assert args.det_checkpoint is not None
+
+ det_model = init_detector(
+ args.det_config, args.det_checkpoint, device=args.device.lower())
+ # build the pose model from a config file and a checkpoint file
+ pose_model = init_pose_model(
+ args.pose_config, args.pose_checkpoint, device=args.device.lower())
+
+ dataset = pose_model.cfg.data['test']['type']
+ dataset_info = pose_model.cfg.data['test'].get('dataset_info', None)
+ if dataset_info is None:
+ warnings.warn(
+ 'Please set `dataset_info` in the config.'
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 for details.',
+ DeprecationWarning)
+ else:
+ dataset_info = DatasetInfo(dataset_info)
+
+ image_name = os.path.join(args.img_root, args.img)
+
+ # test a single image, the resulting box is (x1, y1, x2, y2)
+ mmdet_results = inference_detector(det_model, image_name)
+
+ # keep the person class bounding boxes.
+ person_results = process_mmdet_results(mmdet_results, args.det_cat_id)
+
+ # test a single image, with a list of bboxes.
+
+ # optional
+ return_heatmap = False
+
+ # e.g. use ('backbone', ) to return backbone feature
+ output_layer_names = None
+
+ pose_results, returned_outputs = inference_top_down_pose_model(
+ pose_model,
+ image_name,
+ person_results,
+ bbox_thr=args.bbox_thr,
+ format='xyxy',
+ dataset=dataset,
+ dataset_info=dataset_info,
+ return_heatmap=return_heatmap,
+ outputs=output_layer_names)
+
+ if args.out_img_root == '':
+ out_file = None
+ else:
+ os.makedirs(args.out_img_root, exist_ok=True)
+ out_file = os.path.join(args.out_img_root, f'vis_{args.img}')
+
+ # show the results
+ vis_pose_result(
+ pose_model,
+ image_name,
+ pose_results,
+ dataset=dataset,
+ dataset_info=dataset_info,
+ kpt_score_thr=args.kpt_thr,
+ radius=args.radius,
+ thickness=args.thickness,
+ show=args.show,
+ out_file=out_file)
+
+
+if __name__ == '__main__':
+ main()
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/top_down_pose_tracking_demo_with_mmdet.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/top_down_pose_tracking_demo_with_mmdet.py
new file mode 100644
index 0000000..5ddcd93
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/top_down_pose_tracking_demo_with_mmdet.py
@@ -0,0 +1,190 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os
+import warnings
+from argparse import ArgumentParser
+
+import cv2
+
+from mmpose.apis import (get_track_id, inference_top_down_pose_model,
+ init_pose_model, process_mmdet_results,
+ vis_pose_tracking_result)
+from mmpose.datasets import DatasetInfo
+
+try:
+ from mmdet.apis import inference_detector, init_detector
+ has_mmdet = True
+except (ImportError, ModuleNotFoundError):
+ has_mmdet = False
+
+
+def main():
+ """Visualize the demo images.
+
+ Using mmdet to detect the human.
+ """
+ parser = ArgumentParser()
+ parser.add_argument('det_config', help='Config file for detection')
+ parser.add_argument('det_checkpoint', help='Checkpoint file for detection')
+ parser.add_argument('pose_config', help='Config file for pose')
+ parser.add_argument('pose_checkpoint', help='Checkpoint file for pose')
+ parser.add_argument('--video-path', type=str, help='Video path')
+ parser.add_argument(
+ '--show',
+ action='store_true',
+ default=False,
+ help='whether to show visualizations.')
+ parser.add_argument(
+ '--out-video-root',
+ default='',
+ help='Root of the output video file. '
+ 'Default not saving the visualization video.')
+ parser.add_argument(
+ '--device', default='cuda:0', help='Device used for inference')
+ parser.add_argument(
+ '--det-cat-id',
+ type=int,
+ default=1,
+ help='Category id for bounding box detection model')
+ parser.add_argument(
+ '--bbox-thr',
+ type=float,
+ default=0.3,
+ help='Bounding box score threshold')
+ parser.add_argument(
+ '--kpt-thr', type=float, default=0.3, help='Keypoint score threshold')
+ parser.add_argument(
+ '--use-oks-tracking', action='store_true', help='Using OKS tracking')
+ parser.add_argument(
+ '--tracking-thr', type=float, default=0.3, help='Tracking threshold')
+ parser.add_argument(
+ '--euro',
+ action='store_true',
+ help='Using One_Euro_Filter for smoothing')
+ parser.add_argument(
+ '--radius',
+ type=int,
+ default=4,
+ help='Keypoint radius for visualization')
+ parser.add_argument(
+ '--thickness',
+ type=int,
+ default=1,
+ help='Link thickness for visualization')
+
+ assert has_mmdet, 'Please install mmdet to run the demo.'
+
+ args = parser.parse_args()
+
+ assert args.show or (args.out_video_root != '')
+ assert args.det_config is not None
+ assert args.det_checkpoint is not None
+
+ det_model = init_detector(
+ args.det_config, args.det_checkpoint, device=args.device.lower())
+ # build the pose model from a config file and a checkpoint file
+ pose_model = init_pose_model(
+ args.pose_config, args.pose_checkpoint, device=args.device.lower())
+
+ dataset = pose_model.cfg.data['test']['type']
+ dataset_info = pose_model.cfg.data['test'].get('dataset_info', None)
+ if dataset_info is None:
+ warnings.warn(
+ 'Please set `dataset_info` in the config.'
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 for details.',
+ DeprecationWarning)
+ else:
+ dataset_info = DatasetInfo(dataset_info)
+
+ cap = cv2.VideoCapture(args.video_path)
+ fps = None
+
+ assert cap.isOpened(), f'Faild to load video file {args.video_path}'
+
+ if args.out_video_root == '':
+ save_out_video = False
+ else:
+ os.makedirs(args.out_video_root, exist_ok=True)
+ save_out_video = True
+
+ if save_out_video:
+ fps = cap.get(cv2.CAP_PROP_FPS)
+ size = (int(cap.get(cv2.CAP_PROP_FRAME_WIDTH)),
+ int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT)))
+ fourcc = cv2.VideoWriter_fourcc(*'mp4v')
+ videoWriter = cv2.VideoWriter(
+ os.path.join(args.out_video_root,
+ f'vis_{os.path.basename(args.video_path)}'), fourcc,
+ fps, size)
+
+ # optional
+ return_heatmap = False
+
+ # e.g. use ('backbone', ) to return backbone feature
+ output_layer_names = None
+
+ next_id = 0
+ pose_results = []
+ while (cap.isOpened()):
+ pose_results_last = pose_results
+
+ flag, img = cap.read()
+ if not flag:
+ break
+ # test a single image, the resulting box is (x1, y1, x2, y2)
+ mmdet_results = inference_detector(det_model, img)
+
+ # keep the person class bounding boxes.
+ person_results = process_mmdet_results(mmdet_results, args.det_cat_id)
+
+ # test a single image, with a list of bboxes.
+ pose_results, returned_outputs = inference_top_down_pose_model(
+ pose_model,
+ img,
+ person_results,
+ bbox_thr=args.bbox_thr,
+ format='xyxy',
+ dataset=dataset,
+ dataset_info=dataset_info,
+ return_heatmap=return_heatmap,
+ outputs=output_layer_names)
+
+ # get track id for each person instance
+ pose_results, next_id = get_track_id(
+ pose_results,
+ pose_results_last,
+ next_id,
+ use_oks=args.use_oks_tracking,
+ tracking_thr=args.tracking_thr,
+ use_one_euro=args.euro,
+ fps=fps)
+
+ # show the results
+ vis_img = vis_pose_tracking_result(
+ pose_model,
+ img,
+ pose_results,
+ radius=args.radius,
+ thickness=args.thickness,
+ dataset=dataset,
+ dataset_info=dataset_info,
+ kpt_score_thr=args.kpt_thr,
+ show=False)
+
+ if args.show:
+ cv2.imshow('Image', vis_img)
+
+ if save_out_video:
+ videoWriter.write(vis_img)
+
+ if args.show and cv2.waitKey(1) & 0xFF == ord('q'):
+ break
+
+ cap.release()
+ if save_out_video:
+ videoWriter.release()
+ if args.show:
+ cv2.destroyAllWindows()
+
+
+if __name__ == '__main__':
+ main()
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/top_down_pose_tracking_demo_with_mmtracking.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/top_down_pose_tracking_demo_with_mmtracking.py
new file mode 100644
index 0000000..9902e06
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/top_down_pose_tracking_demo_with_mmtracking.py
@@ -0,0 +1,185 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os
+import warnings
+from argparse import ArgumentParser
+
+import cv2
+
+from mmpose.apis import (inference_top_down_pose_model, init_pose_model,
+ vis_pose_tracking_result)
+from mmpose.datasets import DatasetInfo
+
+try:
+ from mmtrack.apis import inference_mot
+ from mmtrack.apis import init_model as init_tracking_model
+ has_mmtrack = True
+except (ImportError, ModuleNotFoundError):
+ has_mmtrack = False
+
+
+def process_mmtracking_results(mmtracking_results):
+ """Process mmtracking results.
+
+ :param mmtracking_results:
+ :return: a list of tracked bounding boxes
+ """
+ person_results = []
+ # 'track_results' is changed to 'track_bboxes'
+ # in https://github.com/open-mmlab/mmtracking/pull/300
+ if 'track_bboxes' in mmtracking_results:
+ tracking_results = mmtracking_results['track_bboxes'][0]
+ elif 'track_results' in mmtracking_results:
+ tracking_results = mmtracking_results['track_results'][0]
+
+ for track in tracking_results:
+ person = {}
+ person['track_id'] = int(track[0])
+ person['bbox'] = track[1:]
+ person_results.append(person)
+ return person_results
+
+
+def main():
+ """Visualize the demo images.
+
+ Using mmdet to detect the human.
+ """
+ parser = ArgumentParser()
+ parser.add_argument('tracking_config', help='Config file for tracking')
+ parser.add_argument('pose_config', help='Config file for pose')
+ parser.add_argument('pose_checkpoint', help='Checkpoint file for pose')
+ parser.add_argument('--video-path', type=str, help='Video path')
+ parser.add_argument(
+ '--show',
+ action='store_true',
+ default=False,
+ help='whether to show visualizations.')
+ parser.add_argument(
+ '--out-video-root',
+ default='',
+ help='Root of the output video file. '
+ 'Default not saving the visualization video.')
+ parser.add_argument(
+ '--device', default='cuda:0', help='Device used for inference')
+ parser.add_argument(
+ '--bbox-thr',
+ type=float,
+ default=0.3,
+ help='Bounding box score threshold')
+ parser.add_argument(
+ '--kpt-thr', type=float, default=0.3, help='Keypoint score threshold')
+ parser.add_argument(
+ '--radius',
+ type=int,
+ default=4,
+ help='Keypoint radius for visualization')
+ parser.add_argument(
+ '--thickness',
+ type=int,
+ default=1,
+ help='Link thickness for visualization')
+
+ assert has_mmtrack, 'Please install mmtrack to run the demo.'
+
+ args = parser.parse_args()
+
+ assert args.show or (args.out_video_root != '')
+ assert args.tracking_config is not None
+
+ tracking_model = init_tracking_model(
+ args.tracking_config, None, device=args.device.lower())
+ # build the pose model from a config file and a checkpoint file
+ pose_model = init_pose_model(
+ args.pose_config, args.pose_checkpoint, device=args.device.lower())
+
+ dataset = pose_model.cfg.data['test']['type']
+ dataset_info = pose_model.cfg.data['test'].get('dataset_info', None)
+ if dataset_info is None:
+ warnings.warn(
+ 'Please set `dataset_info` in the config.'
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 for details.',
+ DeprecationWarning)
+ else:
+ dataset_info = DatasetInfo(dataset_info)
+
+ cap = cv2.VideoCapture(args.video_path)
+ assert cap.isOpened(), f'Faild to load video file {args.video_path}'
+
+ if args.out_video_root == '':
+ save_out_video = False
+ else:
+ os.makedirs(args.out_video_root, exist_ok=True)
+ save_out_video = True
+
+ if save_out_video:
+ fps = cap.get(cv2.CAP_PROP_FPS)
+ size = (int(cap.get(cv2.CAP_PROP_FRAME_WIDTH)),
+ int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT)))
+ fourcc = cv2.VideoWriter_fourcc(*'mp4v')
+ videoWriter = cv2.VideoWriter(
+ os.path.join(args.out_video_root,
+ f'vis_{os.path.basename(args.video_path)}'), fourcc,
+ fps, size)
+
+ # optional
+ return_heatmap = False
+
+ # e.g. use ('backbone', ) to return backbone feature
+ output_layer_names = None
+
+ frame_id = 0
+ while (cap.isOpened()):
+ flag, img = cap.read()
+ if not flag:
+ break
+
+ mmtracking_results = inference_mot(
+ tracking_model, img, frame_id=frame_id)
+
+ # keep the person class bounding boxes.
+ person_results = process_mmtracking_results(mmtracking_results)
+
+ # test a single image, with a list of bboxes.
+ pose_results, returned_outputs = inference_top_down_pose_model(
+ pose_model,
+ img,
+ person_results,
+ bbox_thr=args.bbox_thr,
+ format='xyxy',
+ dataset=dataset,
+ dataset_info=dataset_info,
+ return_heatmap=return_heatmap,
+ outputs=output_layer_names)
+
+ # show the results
+ vis_img = vis_pose_tracking_result(
+ pose_model,
+ img,
+ pose_results,
+ radius=args.radius,
+ thickness=args.thickness,
+ dataset=dataset,
+ dataset_info=dataset_info,
+ kpt_score_thr=args.kpt_thr,
+ show=False)
+
+ if args.show:
+ cv2.imshow('Image', vis_img)
+
+ if save_out_video:
+ videoWriter.write(vis_img)
+
+ if args.show and cv2.waitKey(1) & 0xFF == ord('q'):
+ break
+
+ frame_id += 1
+
+ cap.release()
+ if save_out_video:
+ videoWriter.release()
+ if args.show:
+ cv2.destroyAllWindows()
+
+
+if __name__ == '__main__':
+ main()
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/top_down_video_demo_full_frame_without_det.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/top_down_video_demo_full_frame_without_det.py
new file mode 100644
index 0000000..2d81810
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/top_down_video_demo_full_frame_without_det.py
@@ -0,0 +1,139 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os
+import warnings
+from argparse import ArgumentParser
+
+import cv2
+import numpy as np
+
+from mmpose.apis import (inference_top_down_pose_model, init_pose_model,
+ vis_pose_result)
+from mmpose.datasets import DatasetInfo
+
+
+def main():
+ """Visualize the demo images.
+
+ Using mmdet to detect the human.
+ """
+ parser = ArgumentParser()
+ parser.add_argument('pose_config', help='Config file for pose')
+ parser.add_argument('pose_checkpoint', help='Checkpoint file for pose')
+ parser.add_argument('--video-path', type=str, help='Video path')
+ parser.add_argument(
+ '--show',
+ action='store_true',
+ default=False,
+ help='whether to show visualizations.')
+ parser.add_argument(
+ '--out-video-root',
+ default='',
+ help='Root of the output video file. '
+ 'Default not saving the visualization video.')
+ parser.add_argument(
+ '--device', default='cuda:0', help='Device used for inference')
+ parser.add_argument(
+ '--kpt-thr', type=float, default=0.3, help='Keypoint score threshold')
+ parser.add_argument(
+ '--radius',
+ type=int,
+ default=4,
+ help='Keypoint radius for visualization')
+ parser.add_argument(
+ '--thickness',
+ type=int,
+ default=1,
+ help='Link thickness for visualization')
+
+ args = parser.parse_args()
+
+ assert args.show or (args.out_video_root != '')
+ # build the pose model from a config file and a checkpoint file
+ pose_model = init_pose_model(
+ args.pose_config, args.pose_checkpoint, device=args.device.lower())
+
+ dataset = pose_model.cfg.data['test']['type']
+ dataset_info = pose_model.cfg.data['test'].get('dataset_info', None)
+ if dataset_info is None:
+ warnings.warn(
+ 'Please set `dataset_info` in the config.'
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 for details.',
+ DeprecationWarning)
+ else:
+ dataset_info = DatasetInfo(dataset_info)
+
+ cap = cv2.VideoCapture(args.video_path)
+ assert cap.isOpened(), f'Faild to load video file {args.video_path}'
+
+ fps = cap.get(cv2.CAP_PROP_FPS)
+ size = (int(cap.get(cv2.CAP_PROP_FRAME_WIDTH)),
+ int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT)))
+
+ if args.out_video_root == '':
+ save_out_video = False
+ else:
+ os.makedirs(args.out_video_root, exist_ok=True)
+ save_out_video = True
+
+ if save_out_video:
+ fourcc = cv2.VideoWriter_fourcc(*'mp4v')
+ videoWriter = cv2.VideoWriter(
+ os.path.join(args.out_video_root,
+ f'vis_{os.path.basename(args.video_path)}'), fourcc,
+ fps, size)
+
+ # optional
+ return_heatmap = False
+
+ # e.g. use ('backbone', ) to return backbone feature
+ output_layer_names = None
+
+ while (cap.isOpened()):
+ flag, img = cap.read()
+ if not flag:
+ break
+
+ # keep the person class bounding boxes.
+ person_results = [{'bbox': np.array([0, 0, size[0], size[1]])}]
+
+ # test a single image, with a list of bboxes.
+ pose_results, returned_outputs = inference_top_down_pose_model(
+ pose_model,
+ img,
+ person_results,
+ format='xyxy',
+ dataset=dataset,
+ dataset_info=dataset_info,
+ return_heatmap=return_heatmap,
+ outputs=output_layer_names)
+
+ # show the results
+ vis_img = vis_pose_result(
+ pose_model,
+ img,
+ pose_results,
+ radius=args.radius,
+ thickness=args.thickness,
+ dataset=dataset,
+ dataset_info=dataset_info,
+ kpt_score_thr=args.kpt_thr,
+ show=False)
+
+ if args.show:
+ cv2.imshow('Image', vis_img)
+
+ if save_out_video:
+ videoWriter.write(vis_img)
+
+ if args.show and cv2.waitKey(1) & 0xFF == ord('q'):
+ break
+
+ cap.release()
+ if save_out_video:
+ videoWriter.release()
+ if args.show:
+ cv2.destroyAllWindows()
+
+
+if __name__ == '__main__':
+ main()
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/top_down_video_demo_with_mmdet.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/top_down_video_demo_with_mmdet.py
new file mode 100644
index 0000000..7831c76
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/top_down_video_demo_with_mmdet.py
@@ -0,0 +1,165 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os
+import warnings
+from argparse import ArgumentParser
+
+import cv2
+
+from mmpose.apis import (inference_top_down_pose_model, init_pose_model,
+ process_mmdet_results, vis_pose_result)
+from mmpose.datasets import DatasetInfo
+
+try:
+ from mmdet.apis import inference_detector, init_detector
+ has_mmdet = True
+except (ImportError, ModuleNotFoundError):
+ has_mmdet = False
+
+
+def main():
+ """Visualize the demo images.
+
+ Using mmdet to detect the human.
+ """
+ parser = ArgumentParser()
+ parser.add_argument('det_config', help='Config file for detection')
+ parser.add_argument('det_checkpoint', help='Checkpoint file for detection')
+ parser.add_argument('pose_config', help='Config file for pose')
+ parser.add_argument('pose_checkpoint', help='Checkpoint file for pose')
+ parser.add_argument('--video-path', type=str, help='Video path')
+ parser.add_argument(
+ '--show',
+ action='store_true',
+ default=False,
+ help='whether to show visualizations.')
+ parser.add_argument(
+ '--out-video-root',
+ default='',
+ help='Root of the output video file. '
+ 'Default not saving the visualization video.')
+ parser.add_argument(
+ '--device', default='cuda:0', help='Device used for inference')
+ parser.add_argument(
+ '--det-cat-id',
+ type=int,
+ default=1,
+ help='Category id for bounding box detection model')
+ parser.add_argument(
+ '--bbox-thr',
+ type=float,
+ default=0.3,
+ help='Bounding box score threshold')
+ parser.add_argument(
+ '--kpt-thr', type=float, default=0.3, help='Keypoint score threshold')
+ parser.add_argument(
+ '--radius',
+ type=int,
+ default=4,
+ help='Keypoint radius for visualization')
+ parser.add_argument(
+ '--thickness',
+ type=int,
+ default=1,
+ help='Link thickness for visualization')
+
+ assert has_mmdet, 'Please install mmdet to run the demo.'
+
+ args = parser.parse_args()
+
+ assert args.show or (args.out_video_root != '')
+ assert args.det_config is not None
+ assert args.det_checkpoint is not None
+
+ det_model = init_detector(
+ args.det_config, args.det_checkpoint, device=args.device.lower())
+ # build the pose model from a config file and a checkpoint file
+ pose_model = init_pose_model(
+ args.pose_config, args.pose_checkpoint, device=args.device.lower())
+
+ dataset = pose_model.cfg.data['test']['type']
+ dataset_info = pose_model.cfg.data['test'].get('dataset_info', None)
+ if dataset_info is None:
+ warnings.warn(
+ 'Please set `dataset_info` in the config.'
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 for details.',
+ DeprecationWarning)
+ else:
+ dataset_info = DatasetInfo(dataset_info)
+
+ cap = cv2.VideoCapture(args.video_path)
+ assert cap.isOpened(), f'Faild to load video file {args.video_path}'
+
+ if args.out_video_root == '':
+ save_out_video = False
+ else:
+ os.makedirs(args.out_video_root, exist_ok=True)
+ save_out_video = True
+
+ if save_out_video:
+ fps = cap.get(cv2.CAP_PROP_FPS)
+ size = (int(cap.get(cv2.CAP_PROP_FRAME_WIDTH)),
+ int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT)))
+ fourcc = cv2.VideoWriter_fourcc(*'mp4v')
+ videoWriter = cv2.VideoWriter(
+ os.path.join(args.out_video_root,
+ f'vis_{os.path.basename(args.video_path)}'), fourcc,
+ fps, size)
+
+ # optional
+ return_heatmap = False
+
+ # e.g. use ('backbone', ) to return backbone feature
+ output_layer_names = None
+
+ while (cap.isOpened()):
+ flag, img = cap.read()
+ if not flag:
+ break
+ # test a single image, the resulting box is (x1, y1, x2, y2)
+ mmdet_results = inference_detector(det_model, img)
+ print(mmdet_results)
+ # keep the person class bounding boxes.
+ person_results = process_mmdet_results(mmdet_results, args.det_cat_id)
+
+ # test a single image, with a list of bboxes.
+ pose_results, returned_outputs = inference_top_down_pose_model(
+ pose_model,
+ img,
+ person_results,
+ bbox_thr=args.bbox_thr,
+ format='xyxy',
+ dataset=dataset,
+ dataset_info=dataset_info,
+ return_heatmap=return_heatmap,
+ outputs=output_layer_names)
+
+ # show the results
+ vis_img = vis_pose_result(
+ pose_model,
+ img,
+ pose_results,
+ dataset=dataset,
+ dataset_info=dataset_info,
+ kpt_score_thr=args.kpt_thr,
+ radius=args.radius,
+ thickness=args.thickness,
+ show=False)
+
+ if args.show:
+ cv2.imshow('Image', vis_img)
+
+ if save_out_video:
+ videoWriter.write(vis_img)
+
+ if args.show and cv2.waitKey(1) & 0xFF == ord('q'):
+ break
+
+ cap.release()
+ if save_out_video:
+ videoWriter.release()
+ if args.show:
+ cv2.destroyAllWindows()
+
+
+if __name__ == '__main__':
+ main()
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/webcam_demo.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/webcam_demo.py
new file mode 100644
index 0000000..bff3001
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/demo/webcam_demo.py
@@ -0,0 +1,585 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import argparse
+import time
+from collections import deque
+from queue import Queue
+from threading import Event, Lock, Thread
+
+import cv2
+import numpy as np
+
+from mmpose.apis import (get_track_id, inference_top_down_pose_model,
+ init_pose_model, vis_pose_result)
+from mmpose.core import apply_bugeye_effect, apply_sunglasses_effect
+from mmpose.utils import StopWatch
+
+try:
+ from mmdet.apis import inference_detector, init_detector
+ has_mmdet = True
+except (ImportError, ModuleNotFoundError):
+ has_mmdet = False
+
+try:
+ import psutil
+ psutil_proc = psutil.Process()
+except (ImportError, ModuleNotFoundError):
+ psutil_proc = None
+
+
+def parse_args():
+ parser = argparse.ArgumentParser()
+ parser.add_argument('--cam-id', type=str, default='0')
+ parser.add_argument(
+ '--det-config',
+ type=str,
+ default='demo/mmdetection_cfg/'
+ 'ssdlite_mobilenetv2_scratch_600e_coco.py',
+ help='Config file for detection')
+ parser.add_argument(
+ '--det-checkpoint',
+ type=str,
+ default='https://download.openmmlab.com/mmdetection/v2.0/ssd/'
+ 'ssdlite_mobilenetv2_scratch_600e_coco/ssdlite_mobilenetv2_'
+ 'scratch_600e_coco_20210629_110627-974d9307.pth',
+ help='Checkpoint file for detection')
+ parser.add_argument(
+ '--enable-human-pose',
+ type=int,
+ default=1,
+ help='Enable human pose estimation')
+ parser.add_argument(
+ '--enable-animal-pose',
+ type=int,
+ default=0,
+ help='Enable animal pose estimation')
+ parser.add_argument(
+ '--human-pose-config',
+ type=str,
+ default='configs/wholebody/2d_kpt_sview_rgb_img/topdown_heatmap/'
+ 'coco-wholebody/vipnas_res50_coco_wholebody_256x192_dark.py',
+ help='Config file for human pose')
+ parser.add_argument(
+ '--human-pose-checkpoint',
+ type=str,
+ default='https://download.openmmlab.com/'
+ 'mmpose/top_down/vipnas/'
+ 'vipnas_res50_wholebody_256x192_dark-67c0ce35_20211112.pth',
+ help='Checkpoint file for human pose')
+ parser.add_argument(
+ '--human-det-ids',
+ type=int,
+ default=[1],
+ nargs='+',
+ help='Object category label of human in detection results.'
+ 'Default is [1(person)], following COCO definition.')
+ parser.add_argument(
+ '--animal-pose-config',
+ type=str,
+ default='configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap/'
+ 'animalpose/hrnet_w32_animalpose_256x256.py',
+ help='Config file for animal pose')
+ parser.add_argument(
+ '--animal-pose-checkpoint',
+ type=str,
+ default='https://download.openmmlab.com/mmpose/animal/hrnet/'
+ 'hrnet_w32_animalpose_256x256-1aa7f075_20210426.pth',
+ help='Checkpoint file for animal pose')
+ parser.add_argument(
+ '--animal-det-ids',
+ type=int,
+ default=[16, 17, 18, 19, 20],
+ nargs='+',
+ help='Object category label of animals in detection results'
+ 'Default is [16(cat), 17(dog), 18(horse), 19(sheep), 20(cow)], '
+ 'following COCO definition.')
+ parser.add_argument(
+ '--device', default='cuda:0', help='Device used for inference')
+ parser.add_argument(
+ '--det-score-thr',
+ type=float,
+ default=0.5,
+ help='bbox score threshold')
+ parser.add_argument(
+ '--kpt-thr', type=float, default=0.3, help='bbox score threshold')
+ parser.add_argument(
+ '--vis-mode',
+ type=int,
+ default=2,
+ help='0-none. 1-detection only. 2-detection and pose.')
+ parser.add_argument(
+ '--sunglasses', action='store_true', help='Apply `sunglasses` effect.')
+ parser.add_argument(
+ '--bugeye', action='store_true', help='Apply `bug-eye` effect.')
+
+ parser.add_argument(
+ '--out-video-file',
+ type=str,
+ default=None,
+ help='Record the video into a file. This may reduce the frame rate')
+
+ parser.add_argument(
+ '--out-video-fps',
+ type=int,
+ default=20,
+ help='Set the FPS of the output video file.')
+
+ parser.add_argument(
+ '--buffer-size',
+ type=int,
+ default=-1,
+ help='Frame buffer size. If set -1, the buffer size will be '
+ 'automatically inferred from the display delay time. Default: -1')
+
+ parser.add_argument(
+ '--inference-fps',
+ type=int,
+ default=10,
+ help='Maximum inference FPS. This is to limit the resource consuming '
+ 'especially when the detection and pose model are lightweight and '
+ 'very fast. Default: 10.')
+
+ parser.add_argument(
+ '--display-delay',
+ type=int,
+ default=0,
+ help='Delay the output video in milliseconds. This can be used to '
+ 'align the output video and inference results. The delay can be '
+ 'disabled by setting a non-positive delay time. Default: 0')
+
+ parser.add_argument(
+ '--synchronous-mode',
+ action='store_true',
+ help='Enable synchronous mode that video I/O and inference will be '
+ 'temporally aligned. Note that this will reduce the display FPS.')
+
+ return parser.parse_args()
+
+
+def process_mmdet_results(mmdet_results, class_names=None, cat_ids=1):
+ """Process mmdet results to mmpose input format.
+
+ Args:
+ mmdet_results: raw output of mmdet model
+ class_names: class names of mmdet model
+ cat_ids (int or List[int]): category id list that will be preserved
+ Returns:
+ List[Dict]: detection results for mmpose input
+ """
+ if isinstance(mmdet_results, tuple):
+ mmdet_results = mmdet_results[0]
+
+ if not isinstance(cat_ids, (list, tuple)):
+ cat_ids = [cat_ids]
+
+ # only keep bboxes of interested classes
+ bbox_results = [mmdet_results[i - 1] for i in cat_ids]
+ bboxes = np.vstack(bbox_results)
+
+ # get textual labels of classes
+ labels = np.concatenate([
+ np.full(bbox.shape[0], i - 1, dtype=np.int32)
+ for i, bbox in zip(cat_ids, bbox_results)
+ ])
+ if class_names is None:
+ labels = [f'class: {i}' for i in labels]
+ else:
+ labels = [class_names[i] for i in labels]
+
+ det_results = []
+ for bbox, label in zip(bboxes, labels):
+ det_result = dict(bbox=bbox, label=label)
+ det_results.append(det_result)
+ return det_results
+
+
+def read_camera():
+ # init video reader
+ print('Thread "input" started')
+ cam_id = args.cam_id
+ if cam_id.isdigit():
+ cam_id = int(cam_id)
+ vid_cap = cv2.VideoCapture(cam_id)
+ if not vid_cap.isOpened():
+ print(f'Cannot open camera (ID={cam_id})')
+ exit()
+
+ while not event_exit.is_set():
+ # capture a camera frame
+ ret_val, frame = vid_cap.read()
+ if ret_val:
+ ts_input = time.time()
+
+ event_inference_done.clear()
+ with input_queue_mutex:
+ input_queue.append((ts_input, frame))
+
+ if args.synchronous_mode:
+ event_inference_done.wait()
+
+ frame_buffer.put((ts_input, frame))
+ else:
+ # input ending signal
+ frame_buffer.put((None, None))
+ break
+
+ vid_cap.release()
+
+
+def inference_detection():
+ print('Thread "det" started')
+ stop_watch = StopWatch(window=10)
+ min_interval = 1.0 / args.inference_fps
+ _ts_last = None # timestamp when last inference was done
+
+ while True:
+ while len(input_queue) < 1:
+ time.sleep(0.001)
+ with input_queue_mutex:
+ ts_input, frame = input_queue.popleft()
+ # inference detection
+ with stop_watch.timeit('Det'):
+ mmdet_results = inference_detector(det_model, frame)
+
+ t_info = stop_watch.report_strings()
+ with det_result_queue_mutex:
+ det_result_queue.append((ts_input, frame, t_info, mmdet_results))
+
+ # limit the inference FPS
+ _ts = time.time()
+ if _ts_last is not None and _ts - _ts_last < min_interval:
+ time.sleep(min_interval - _ts + _ts_last)
+ _ts_last = time.time()
+
+
+def inference_pose():
+ print('Thread "pose" started')
+ stop_watch = StopWatch(window=10)
+
+ while True:
+ while len(det_result_queue) < 1:
+ time.sleep(0.001)
+ with det_result_queue_mutex:
+ ts_input, frame, t_info, mmdet_results = det_result_queue.popleft()
+
+ pose_results_list = []
+ for model_info, pose_history in zip(pose_model_list,
+ pose_history_list):
+ model_name = model_info['name']
+ pose_model = model_info['model']
+ cat_ids = model_info['cat_ids']
+ pose_results_last = pose_history['pose_results_last']
+ next_id = pose_history['next_id']
+
+ with stop_watch.timeit(model_name):
+ # process mmdet results
+ det_results = process_mmdet_results(
+ mmdet_results,
+ class_names=det_model.CLASSES,
+ cat_ids=cat_ids)
+
+ # inference pose model
+ dataset_name = pose_model.cfg.data['test']['type']
+ pose_results, _ = inference_top_down_pose_model(
+ pose_model,
+ frame,
+ det_results,
+ bbox_thr=args.det_score_thr,
+ format='xyxy',
+ dataset=dataset_name)
+
+ pose_results, next_id = get_track_id(
+ pose_results,
+ pose_results_last,
+ next_id,
+ use_oks=False,
+ tracking_thr=0.3,
+ use_one_euro=True,
+ fps=None)
+
+ pose_results_list.append(pose_results)
+
+ # update pose history
+ pose_history['pose_results_last'] = pose_results
+ pose_history['next_id'] = next_id
+
+ t_info += stop_watch.report_strings()
+ with pose_result_queue_mutex:
+ pose_result_queue.append((ts_input, t_info, pose_results_list))
+
+ event_inference_done.set()
+
+
+def display():
+ print('Thread "display" started')
+ stop_watch = StopWatch(window=10)
+
+ # initialize result status
+ ts_inference = None # timestamp of the latest inference result
+ fps_inference = 0. # infenrece FPS
+ t_delay_inference = 0. # inference result time delay
+ pose_results_list = None # latest inference result
+ t_info = [] # upstream time information (list[str])
+
+ # initialize visualization and output
+ sunglasses_img = None # resource image for sunglasses effect
+ text_color = (228, 183, 61) # text color to show time/system information
+ vid_out = None # video writer
+
+ # show instructions
+ print('Keyboard shortcuts: ')
+ print('"v": Toggle the visualization of bounding boxes and poses.')
+ print('"s": Toggle the sunglasses effect.')
+ print('"b": Toggle the bug-eye effect.')
+ print('"Q", "q" or Esc: Exit.')
+
+ while True:
+ with stop_watch.timeit('_FPS_'):
+ # acquire a frame from buffer
+ ts_input, frame = frame_buffer.get()
+ # input ending signal
+ if ts_input is None:
+ break
+
+ img = frame
+
+ # get pose estimation results
+ if len(pose_result_queue) > 0:
+ with pose_result_queue_mutex:
+ _result = pose_result_queue.popleft()
+ _ts_input, t_info, pose_results_list = _result
+
+ _ts = time.time()
+ if ts_inference is not None:
+ fps_inference = 1.0 / (_ts - ts_inference)
+ ts_inference = _ts
+ t_delay_inference = (_ts - _ts_input) * 1000
+
+ # visualize detection and pose results
+ if pose_results_list is not None:
+ for model_info, pose_results in zip(pose_model_list,
+ pose_results_list):
+ pose_model = model_info['model']
+ bbox_color = model_info['bbox_color']
+
+ dataset_name = pose_model.cfg.data['test']['type']
+
+ # show pose results
+ if args.vis_mode == 1:
+ img = vis_pose_result(
+ pose_model,
+ img,
+ pose_results,
+ radius=4,
+ thickness=2,
+ dataset=dataset_name,
+ kpt_score_thr=1e7,
+ bbox_color=bbox_color)
+ elif args.vis_mode == 2:
+ img = vis_pose_result(
+ pose_model,
+ img,
+ pose_results,
+ radius=4,
+ thickness=2,
+ dataset=dataset_name,
+ kpt_score_thr=args.kpt_thr,
+ bbox_color=bbox_color)
+
+ # sunglasses effect
+ if args.sunglasses:
+ if dataset_name in {
+ 'TopDownCocoDataset',
+ 'TopDownCocoWholeBodyDataset'
+ }:
+ left_eye_idx = 1
+ right_eye_idx = 2
+ elif dataset_name == 'AnimalPoseDataset':
+ left_eye_idx = 0
+ right_eye_idx = 1
+ else:
+ raise ValueError(
+ 'Sunglasses effect does not support'
+ f'{dataset_name}')
+ if sunglasses_img is None:
+ # The image attributes to:
+ # https://www.vecteezy.com/free-vector/glass
+ # Glass Vectors by Vecteezy
+ sunglasses_img = cv2.imread(
+ 'demo/resources/sunglasses.jpg')
+ img = apply_sunglasses_effect(img, pose_results,
+ sunglasses_img,
+ left_eye_idx,
+ right_eye_idx)
+ # bug-eye effect
+ if args.bugeye:
+ if dataset_name in {
+ 'TopDownCocoDataset',
+ 'TopDownCocoWholeBodyDataset'
+ }:
+ left_eye_idx = 1
+ right_eye_idx = 2
+ elif dataset_name == 'AnimalPoseDataset':
+ left_eye_idx = 0
+ right_eye_idx = 1
+ else:
+ raise ValueError('Bug-eye effect does not support'
+ f'{dataset_name}')
+ img = apply_bugeye_effect(img, pose_results,
+ left_eye_idx, right_eye_idx)
+
+ # delay control
+ if args.display_delay > 0:
+ t_sleep = args.display_delay * 0.001 - (time.time() - ts_input)
+ if t_sleep > 0:
+ time.sleep(t_sleep)
+ t_delay = (time.time() - ts_input) * 1000
+
+ # show time information
+ t_info_display = stop_watch.report_strings() # display fps
+ t_info_display.append(f'Inference FPS: {fps_inference:>5.1f}')
+ t_info_display.append(f'Delay: {t_delay:>3.0f}')
+ t_info_display.append(
+ f'Inference Delay: {t_delay_inference:>3.0f}')
+ t_info_str = ' | '.join(t_info_display + t_info)
+ cv2.putText(img, t_info_str, (20, 20), cv2.FONT_HERSHEY_DUPLEX,
+ 0.3, text_color, 1)
+ # collect system information
+ sys_info = [
+ f'RES: {img.shape[1]}x{img.shape[0]}',
+ f'Buffer: {frame_buffer.qsize()}/{frame_buffer.maxsize}'
+ ]
+ if psutil_proc is not None:
+ sys_info += [
+ f'CPU: {psutil_proc.cpu_percent():.1f}%',
+ f'MEM: {psutil_proc.memory_percent():.1f}%'
+ ]
+ sys_info_str = ' | '.join(sys_info)
+ cv2.putText(img, sys_info_str, (20, 40), cv2.FONT_HERSHEY_DUPLEX,
+ 0.3, text_color, 1)
+
+ # save the output video frame
+ if args.out_video_file is not None:
+ if vid_out is None:
+ fourcc = cv2.VideoWriter_fourcc(*'mp4v')
+ fps = args.out_video_fps
+ frame_size = (img.shape[1], img.shape[0])
+ vid_out = cv2.VideoWriter(args.out_video_file, fourcc, fps,
+ frame_size)
+
+ vid_out.write(img)
+
+ # display
+ cv2.imshow('mmpose webcam demo', img)
+ keyboard_input = cv2.waitKey(1)
+ if keyboard_input in (27, ord('q'), ord('Q')):
+ break
+ elif keyboard_input == ord('s'):
+ args.sunglasses = not args.sunglasses
+ elif keyboard_input == ord('b'):
+ args.bugeye = not args.bugeye
+ elif keyboard_input == ord('v'):
+ args.vis_mode = (args.vis_mode + 1) % 3
+
+ cv2.destroyAllWindows()
+ if vid_out is not None:
+ vid_out.release()
+ event_exit.set()
+
+
+def main():
+ global args
+ global frame_buffer
+ global input_queue, input_queue_mutex
+ global det_result_queue, det_result_queue_mutex
+ global pose_result_queue, pose_result_queue_mutex
+ global det_model, pose_model_list, pose_history_list
+ global event_exit, event_inference_done
+
+ args = parse_args()
+
+ assert has_mmdet, 'Please install mmdet to run the demo.'
+ assert args.det_config is not None
+ assert args.det_checkpoint is not None
+
+ # build detection model
+ det_model = init_detector(
+ args.det_config, args.det_checkpoint, device=args.device.lower())
+
+ # build pose models
+ pose_model_list = []
+ if args.enable_human_pose:
+ pose_model = init_pose_model(
+ args.human_pose_config,
+ args.human_pose_checkpoint,
+ device=args.device.lower())
+ model_info = {
+ 'name': 'HumanPose',
+ 'model': pose_model,
+ 'cat_ids': args.human_det_ids,
+ 'bbox_color': (148, 139, 255),
+ }
+ pose_model_list.append(model_info)
+ if args.enable_animal_pose:
+ pose_model = init_pose_model(
+ args.animal_pose_config,
+ args.animal_pose_checkpoint,
+ device=args.device.lower())
+ model_info = {
+ 'name': 'AnimalPose',
+ 'model': pose_model,
+ 'cat_ids': args.animal_det_ids,
+ 'bbox_color': 'cyan',
+ }
+ pose_model_list.append(model_info)
+
+ # store pose history for pose tracking
+ pose_history_list = []
+ for _ in range(len(pose_model_list)):
+ pose_history_list.append({'pose_results_last': [], 'next_id': 0})
+
+ # frame buffer
+ if args.buffer_size > 0:
+ buffer_size = args.buffer_size
+ else:
+ # infer buffer size from the display delay time
+ # assume that the maximum video fps is 30
+ buffer_size = round(30 * (1 + max(args.display_delay, 0) / 1000.))
+ frame_buffer = Queue(maxsize=buffer_size)
+
+ # queue of input frames
+ # element: (timestamp, frame)
+ input_queue = deque(maxlen=1)
+ input_queue_mutex = Lock()
+
+ # queue of detection results
+ # element: tuple(timestamp, frame, time_info, det_results)
+ det_result_queue = deque(maxlen=1)
+ det_result_queue_mutex = Lock()
+
+ # queue of detection/pose results
+ # element: (timestamp, time_info, pose_results_list)
+ pose_result_queue = deque(maxlen=1)
+ pose_result_queue_mutex = Lock()
+
+ try:
+ event_exit = Event()
+ event_inference_done = Event()
+ t_input = Thread(target=read_camera, args=())
+ t_det = Thread(target=inference_detection, args=(), daemon=True)
+ t_pose = Thread(target=inference_pose, args=(), daemon=True)
+
+ t_input.start()
+ t_det.start()
+ t_pose.start()
+
+ # run display in the main thread
+ display()
+ # join the input thread (non-daemon)
+ t_input.join()
+
+ except KeyboardInterrupt:
+ pass
+
+
+if __name__ == '__main__':
+ main()
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/model-index.yml b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/model-index.yml
new file mode 100644
index 0000000..c5522f6
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/model-index.yml
@@ -0,0 +1,139 @@
+Import:
+- configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap/animalpose/hrnet_animalpose.yml
+- configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap/animalpose/resnet_animalpose.yml
+- configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap/ap10k/hrnet_ap10k.yml
+- configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap/ap10k/resnet_ap10k.yml
+- configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap/atrw/hrnet_atrw.yml
+- configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap/atrw/resnet_atrw.yml
+- configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap/fly/resnet_fly.yml
+- configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap/horse10/hrnet_horse10.yml
+- configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap/horse10/resnet_horse10.yml
+- configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap/locust/resnet_locust.yml
+- configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap/macaque/hrnet_macaque.yml
+- configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap/macaque/resnet_macaque.yml
+- configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap/zebra/resnet_zebra.yml
+- configs/body/2d_kpt_sview_rgb_img/associative_embedding/aic/higherhrnet_aic.yml
+- configs/body/2d_kpt_sview_rgb_img/associative_embedding/aic/hrnet_aic.yml
+- configs/body/2d_kpt_sview_rgb_img/associative_embedding/coco/higherhrnet_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/associative_embedding/coco/higherhrnet_udp_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/associative_embedding/coco/hourglass_ae_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/associative_embedding/coco/hrnet_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/associative_embedding/coco/hrnet_udp_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/associative_embedding/coco/mobilenetv2_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/associative_embedding/coco/resnet_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/associative_embedding/crowdpose/higherhrnet_crowdpose.yml
+- configs/body/2d_kpt_sview_rgb_img/associative_embedding/mhp/hrnet_mhp.yml
+- configs/body/2d_kpt_sview_rgb_img/deeppose/coco/resnet_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/deeppose/mpii/resnet_mpii.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/aic/hrnet_aic.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/aic/resnet_aic.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/alexnet_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/cpm_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/hourglass_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/hrformer_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/hrnet_augmentation_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/hrnet_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/hrnet_dark_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/hrnet_fp16_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/hrnet_udp_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/litehrnet_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/mobilenetv2_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/mspn_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/resnest_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/resnet_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/resnet_dark_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/resnet_fp16_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/resnetv1d_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/resnext_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/rsn_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/scnet_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/seresnet_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/shufflenetv1_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/shufflenetv2_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/vgg_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/vipnas_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/crowdpose/hrnet_crowdpose.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/crowdpose/resnet_crowdpose.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/h36m/hrnet_h36m.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/jhmdb/cpm_jhmdb.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/jhmdb/resnet_jhmdb.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/mhp/resnet_mhp.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/mpii/cpm_mpii.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/mpii/hourglass_mpii.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/mpii/hrnet_dark_mpii.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/mpii/hrnet_mpii.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/mpii/litehrnet_mpii.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/mpii/mobilenetv2_mpii.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/mpii/resnet_mpii.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/mpii/resnetv1d_mpii.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/mpii/resnext_mpii.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/mpii/scnet_mpii.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/mpii/seresnet_mpii.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/mpii/shufflenetv1_mpii.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/mpii/shufflenetv2_mpii.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/mpii_trb/resnet_mpii_trb.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/ochuman/hrnet_ochuman.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/ochuman/resnet_ochuman.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/posetrack18/hrnet_posetrack18.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/posetrack18/resnet_posetrack18.yml
+- configs/body/2d_kpt_sview_rgb_vid/posewarper/posetrack18/hrnet_posetrack18_posewarper.yml
+- configs/body/3d_kpt_mview_rgb_img/voxelpose/panoptic/voxelpose_prn64x64x64_cpn80x80x20_panoptic_cam5.yml
+- configs/body/3d_kpt_sview_rgb_img/pose_lift/h36m/simplebaseline3d_h36m.yml
+- configs/body/3d_kpt_sview_rgb_img/pose_lift/mpi_inf_3dhp/simplebaseline3d_mpi-inf-3dhp.yml
+- configs/body/3d_kpt_sview_rgb_vid/video_pose_lift/h36m/videopose3d_h36m.yml
+- configs/body/3d_kpt_sview_rgb_vid/video_pose_lift/mpi_inf_3dhp/videopose3d_mpi-inf-3dhp.yml
+- configs/body/3d_mesh_sview_rgb_img/hmr/mixed/resnet_mixed.yml
+- configs/face/2d_kpt_sview_rgb_img/deeppose/wflw/resnet_softwingloss_wflw.yml
+- configs/face/2d_kpt_sview_rgb_img/deeppose/wflw/resnet_wflw.yml
+- configs/face/2d_kpt_sview_rgb_img/deeppose/wflw/resnet_wingloss_wflw.yml
+- configs/face/2d_kpt_sview_rgb_img/topdown_heatmap/300w/hrnetv2_300w.yml
+- configs/face/2d_kpt_sview_rgb_img/topdown_heatmap/aflw/hrnetv2_aflw.yml
+- configs/face/2d_kpt_sview_rgb_img/topdown_heatmap/aflw/hrnetv2_dark_aflw.yml
+- configs/face/2d_kpt_sview_rgb_img/topdown_heatmap/coco_wholebody_face/hourglass_coco_wholebody_face.yml
+- configs/face/2d_kpt_sview_rgb_img/topdown_heatmap/coco_wholebody_face/hrnetv2_coco_wholebody_face.yml
+- configs/face/2d_kpt_sview_rgb_img/topdown_heatmap/coco_wholebody_face/hrnetv2_dark_coco_wholebody_face.yml
+- configs/face/2d_kpt_sview_rgb_img/topdown_heatmap/coco_wholebody_face/mobilenetv2_coco_wholebody_face.yml
+- configs/face/2d_kpt_sview_rgb_img/topdown_heatmap/coco_wholebody_face/resnet_coco_wholebody_face.yml
+- configs/face/2d_kpt_sview_rgb_img/topdown_heatmap/coco_wholebody_face/scnet_coco_wholebody_face.yml
+- configs/face/2d_kpt_sview_rgb_img/topdown_heatmap/cofw/hrnetv2_cofw.yml
+- configs/face/2d_kpt_sview_rgb_img/topdown_heatmap/wflw/hrnetv2_awing_wflw.yml
+- configs/face/2d_kpt_sview_rgb_img/topdown_heatmap/wflw/hrnetv2_dark_wflw.yml
+- configs/face/2d_kpt_sview_rgb_img/topdown_heatmap/wflw/hrnetv2_wflw.yml
+- configs/fashion/2d_kpt_sview_rgb_img/deeppose/deepfashion/resnet_deepfashion.yml
+- configs/fashion/2d_kpt_sview_rgb_img/topdown_heatmap/deepfashion/resnet_deepfashion.yml
+- configs/hand/2d_kpt_sview_rgb_img/deeppose/onehand10k/resnet_onehand10k.yml
+- configs/hand/2d_kpt_sview_rgb_img/deeppose/panoptic2d/resnet_panoptic2d.yml
+- configs/hand/2d_kpt_sview_rgb_img/deeppose/rhd2d/resnet_rhd2d.yml
+- configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/coco_wholebody_hand/hourglass_coco_wholebody_hand.yml
+- configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/coco_wholebody_hand/hrnetv2_coco_wholebody_hand.yml
+- configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/coco_wholebody_hand/hrnetv2_dark_coco_wholebody_hand.yml
+- configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/coco_wholebody_hand/litehrnet_coco_wholebody_hand.yml
+- configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/coco_wholebody_hand/mobilenetv2_coco_wholebody_hand.yml
+- configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/coco_wholebody_hand/resnet_coco_wholebody_hand.yml
+- configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/coco_wholebody_hand/scnet_coco_wholebody_hand.yml
+- configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/freihand2d/resnet_freihand2d.yml
+- configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/interhand2d/resnet_interhand2d.yml
+- configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/onehand10k/hrnetv2_dark_onehand10k.yml
+- configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/onehand10k/hrnetv2_onehand10k.yml
+- configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/onehand10k/hrnetv2_udp_onehand10k.yml
+- configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/onehand10k/mobilenetv2_onehand10k.yml
+- configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/onehand10k/resnet_onehand10k.yml
+- configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/panoptic2d/hrnetv2_dark_panoptic2d.yml
+- configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/panoptic2d/hrnetv2_panoptic2d.yml
+- configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/panoptic2d/hrnetv2_udp_panoptic2d.yml
+- configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/panoptic2d/mobilenetv2_panoptic2d.yml
+- configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/panoptic2d/resnet_panoptic2d.yml
+- configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/rhd2d/hrnetv2_dark_rhd2d.yml
+- configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/rhd2d/hrnetv2_rhd2d.yml
+- configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/rhd2d/hrnetv2_udp_rhd2d.yml
+- configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/rhd2d/mobilenetv2_rhd2d.yml
+- configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/rhd2d/resnet_rhd2d.yml
+- configs/hand/3d_kpt_sview_rgb_img/internet/interhand3d/internet_interhand3d.yml
+- configs/wholebody/2d_kpt_sview_rgb_img/associative_embedding/coco-wholebody/higherhrnet_coco-wholebody.yml
+- configs/wholebody/2d_kpt_sview_rgb_img/associative_embedding/coco-wholebody/hrnet_coco-wholebody.yml
+- configs/wholebody/2d_kpt_sview_rgb_img/topdown_heatmap/coco-wholebody/hrnet_coco-wholebody.yml
+- configs/wholebody/2d_kpt_sview_rgb_img/topdown_heatmap/coco-wholebody/hrnet_dark_coco-wholebody.yml
+- configs/wholebody/2d_kpt_sview_rgb_img/topdown_heatmap/coco-wholebody/resnet_coco-wholebody.yml
+- configs/wholebody/2d_kpt_sview_rgb_img/topdown_heatmap/coco-wholebody/vipnas_coco-wholebody.yml
+- configs/wholebody/2d_kpt_sview_rgb_img/topdown_heatmap/coco-wholebody/vipnas_dark_coco-wholebody.yml
+- configs/wholebody/2d_kpt_sview_rgb_img/topdown_heatmap/halpe/hrnet_dark_halpe.yml
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/analysis/analyze_logs.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/analysis/analyze_logs.py
new file mode 100644
index 0000000..d0e1a02
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/analysis/analyze_logs.py
@@ -0,0 +1,167 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import argparse
+import json
+from collections import defaultdict
+
+import matplotlib.pyplot as plt
+import numpy as np
+import seaborn as sns
+
+
+def cal_train_time(log_dicts, args):
+ for i, log_dict in enumerate(log_dicts):
+ print(f'{"-" * 5}Analyze train time of {args.json_logs[i]}{"-" * 5}')
+ all_times = []
+ for epoch in log_dict.keys():
+ if args.include_outliers:
+ all_times.append(log_dict[epoch]['time'])
+ else:
+ all_times.append(log_dict[epoch]['time'][1:])
+ all_times = np.array(all_times)
+ epoch_ave_time = all_times.mean(-1)
+ slowest_epoch = epoch_ave_time.argmax()
+ fastest_epoch = epoch_ave_time.argmin()
+ std_over_epoch = epoch_ave_time.std()
+ print(f'slowest epoch {slowest_epoch + 1}, '
+ f'average time is {epoch_ave_time[slowest_epoch]:.4f}')
+ print(f'fastest epoch {fastest_epoch + 1}, '
+ f'average time is {epoch_ave_time[fastest_epoch]:.4f}')
+ print(f'time std over epochs is {std_over_epoch:.4f}')
+ print(f'average iter time: {np.mean(all_times):.4f} s/iter')
+ print()
+
+
+def plot_curve(log_dicts, args):
+ if args.backend is not None:
+ plt.switch_backend(args.backend)
+ sns.set_style(args.style)
+ # if legend is None, use {filename}_{key} as legend
+ legend = args.legend
+ if legend is None:
+ legend = []
+ for json_log in args.json_logs:
+ for metric in args.keys:
+ legend.append(f'{json_log}_{metric}')
+ assert len(legend) == (len(args.json_logs) * len(args.keys))
+ metrics = args.keys
+
+ num_metrics = len(metrics)
+ for i, log_dict in enumerate(log_dicts):
+ epochs = list(log_dict.keys())
+ for j, metric in enumerate(metrics):
+ print(f'plot curve of {args.json_logs[i]}, metric is {metric}')
+ if metric not in log_dict[epochs[0]]:
+ raise KeyError(
+ f'{args.json_logs[i]} does not contain metric {metric}')
+ xs = []
+ ys = []
+ num_iters_per_epoch = log_dict[epochs[0]]['iter'][-1]
+ for epoch in epochs:
+ iters = log_dict[epoch]['iter']
+ if log_dict[epoch]['mode'][-1] == 'val':
+ iters = iters[:-1]
+ xs.append(np.array(iters) + (epoch - 1) * num_iters_per_epoch)
+ ys.append(np.array(log_dict[epoch][metric][:len(iters)]))
+ xs = np.concatenate(xs)
+ ys = np.concatenate(ys)
+ plt.xlabel('iter')
+ plt.plot(xs, ys, label=legend[i * num_metrics + j], linewidth=0.5)
+ plt.legend()
+ if args.title is not None:
+ plt.title(args.title)
+ if args.out is None:
+ plt.show()
+ else:
+ print(f'save curve to: {args.out}')
+ plt.savefig(args.out)
+ plt.cla()
+
+
+def add_plot_parser(subparsers):
+ parser_plt = subparsers.add_parser(
+ 'plot_curve', help='parser for plotting curves')
+ parser_plt.add_argument(
+ 'json_logs',
+ type=str,
+ nargs='+',
+ help='path of train log in json format')
+ parser_plt.add_argument(
+ '--keys',
+ type=str,
+ nargs='+',
+ default=['top1_acc'],
+ help='the metric that you want to plot')
+ parser_plt.add_argument('--title', type=str, help='title of figure')
+ parser_plt.add_argument(
+ '--legend',
+ type=str,
+ nargs='+',
+ default=None,
+ help='legend of each plot')
+ parser_plt.add_argument(
+ '--backend', type=str, default=None, help='backend of plt')
+ parser_plt.add_argument(
+ '--style', type=str, default='dark', help='style of plt')
+ parser_plt.add_argument('--out', type=str, default=None)
+
+
+def add_time_parser(subparsers):
+ parser_time = subparsers.add_parser(
+ 'cal_train_time',
+ help='parser for computing the average time per training iteration')
+ parser_time.add_argument(
+ 'json_logs',
+ type=str,
+ nargs='+',
+ help='path of train log in json format')
+ parser_time.add_argument(
+ '--include-outliers',
+ action='store_true',
+ help='include the first value of every epoch when computing '
+ 'the average time')
+
+
+def parse_args():
+ parser = argparse.ArgumentParser(description='Analyze Json Log')
+ # currently only support plot curve and calculate average train time
+ subparsers = parser.add_subparsers(dest='task', help='task parser')
+ add_plot_parser(subparsers)
+ add_time_parser(subparsers)
+ args = parser.parse_args()
+ return args
+
+
+def load_json_logs(json_logs):
+ # load and convert json_logs to log_dict, key is epoch, value is a sub dict
+ # keys of sub dict is different metrics, e.g. memory, top1_acc
+ # value of sub dict is a list of corresponding values of all iterations
+ log_dicts = [dict() for _ in json_logs]
+ for json_log, log_dict in zip(json_logs, log_dicts):
+ with open(json_log, 'r') as log_file:
+ for line in log_file:
+ log = json.loads(line.strip())
+ # skip lines without `epoch` field
+ if 'epoch' not in log:
+ continue
+ epoch = log.pop('epoch')
+ if epoch not in log_dict:
+ log_dict[epoch] = defaultdict(list)
+ for k, v in log.items():
+ log_dict[epoch][k].append(v)
+ return log_dicts
+
+
+def main():
+ args = parse_args()
+
+ json_logs = args.json_logs
+ for json_log in json_logs:
+ assert json_log.endswith('.json')
+
+ log_dicts = load_json_logs(json_logs)
+
+ eval(args.task)(log_dicts, args)
+
+
+if __name__ == '__main__':
+ main()
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/analysis/benchmark_inference.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/analysis/benchmark_inference.py
new file mode 100644
index 0000000..14c0736
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/analysis/benchmark_inference.py
@@ -0,0 +1,82 @@
+#!/usr/bin/env bash
+# Copyright (c) OpenMMLab. All rights reserved.
+import argparse
+import time
+
+import torch
+from mmcv import Config
+from mmcv.cnn import fuse_conv_bn
+from mmcv.parallel import MMDataParallel
+from mmcv.runner.fp16_utils import wrap_fp16_model
+
+from mmpose.datasets import build_dataloader, build_dataset
+from mmpose.models import build_posenet
+
+
+def parse_args():
+ parser = argparse.ArgumentParser(
+ description='MMPose benchmark a recognizer')
+ parser.add_argument('config', help='test config file path')
+ parser.add_argument(
+ '--log-interval', default=10, help='interval of logging')
+ parser.add_argument(
+ '--fuse-conv-bn',
+ action='store_true',
+ help='Whether to fuse conv and bn, this will slightly increase'
+ 'the inference speed')
+ args = parser.parse_args()
+ return args
+
+
+def main():
+ args = parse_args()
+
+ cfg = Config.fromfile(args.config)
+ # set cudnn_benchmark
+ if cfg.get('cudnn_benchmark', False):
+ torch.backends.cudnn.benchmark = True
+
+ # build the dataloader
+ dataset = build_dataset(cfg.data.val)
+ data_loader = build_dataloader(
+ dataset,
+ samples_per_gpu=1,
+ workers_per_gpu=cfg.data.workers_per_gpu,
+ dist=False,
+ shuffle=False)
+
+ # build the model and load checkpoint
+ model = build_posenet(cfg.model)
+ fp16_cfg = cfg.get('fp16', None)
+ if fp16_cfg is not None:
+ wrap_fp16_model(model)
+ if args.fuse_conv_bn:
+ model = fuse_conv_bn(model)
+ model = MMDataParallel(model, device_ids=[0])
+
+ # the first several iterations may be very slow so skip them
+ num_warmup = 5
+ pure_inf_time = 0
+
+ # benchmark with total batch and take the average
+ for i, data in enumerate(data_loader):
+
+ torch.cuda.synchronize()
+ start_time = time.perf_counter()
+ with torch.no_grad():
+ model(return_loss=False, **data)
+
+ torch.cuda.synchronize()
+ elapsed = time.perf_counter() - start_time
+
+ if i >= num_warmup:
+ pure_inf_time += elapsed
+ if (i + 1) % args.log_interval == 0:
+ its = (i + 1 - num_warmup) / pure_inf_time
+ print(f'Done item [{i + 1:<3}], {its:.2f} items / s')
+ print(f'Overall average: {its:.2f} items / s')
+ print(f'Total time: {pure_inf_time:.2f} s')
+
+
+if __name__ == '__main__':
+ main()
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/analysis/benchmark_processing.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/analysis/benchmark_processing.py
new file mode 100644
index 0000000..d326f3d
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/analysis/benchmark_processing.py
@@ -0,0 +1,58 @@
+#!/usr/bin/env bash
+# Copyright (c) OpenMMLab. All rights reserved.
+"""This file is for benchmark data loading process. It can also be used to
+refresh the memcached cache. The command line to run this file is:
+
+$ python -m cProfile -o program.prof tools/analysis/benchmark_processing.py
+configs/task/method/[config filename]
+
+Note: When debugging, the `workers_per_gpu` in the config should be set to 0
+during benchmark.
+
+It use cProfile to record cpu running time and output to program.prof
+To visualize cProfile output program.prof, use Snakeviz and run:
+$ snakeviz program.prof
+"""
+import argparse
+
+import mmcv
+from mmcv import Config
+
+from mmpose import __version__
+from mmpose.datasets import build_dataloader, build_dataset
+from mmpose.utils import get_root_logger
+
+
+def main():
+ parser = argparse.ArgumentParser(description='Benchmark data loading')
+ parser.add_argument('config', help='train config file path')
+ args = parser.parse_args()
+ cfg = Config.fromfile(args.config)
+
+ # init logger before other steps
+ logger = get_root_logger()
+ logger.info(f'MMPose Version: {__version__}')
+ logger.info(f'Config: {cfg.text}')
+
+ dataset = build_dataset(cfg.data.train)
+ data_loader = build_dataloader(
+ dataset,
+ samples_per_gpu=1,
+ workers_per_gpu=cfg.data.workers_per_gpu,
+ dist=False,
+ shuffle=False)
+
+ # Start progress bar after first 5 batches
+ prog_bar = mmcv.ProgressBar(
+ len(dataset) - 5 * cfg.data.samples_per_gpu, start=False)
+ for i, data in enumerate(data_loader):
+ if i == 5:
+ prog_bar.start()
+ for _ in data['img']:
+ if i < 5:
+ continue
+ prog_bar.update()
+
+
+if __name__ == '__main__':
+ main()
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/analysis/get_flops.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/analysis/get_flops.py
new file mode 100644
index 0000000..f492a87
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/analysis/get_flops.py
@@ -0,0 +1,103 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import argparse
+from functools import partial
+
+import torch
+
+from mmpose.apis.inference import init_pose_model
+
+try:
+ from mmcv.cnn import get_model_complexity_info
+except ImportError:
+ raise ImportError('Please upgrade mmcv to >0.6.2')
+
+
+def parse_args():
+ parser = argparse.ArgumentParser(description='Train a recognizer')
+ parser.add_argument('config', help='train config file path')
+ parser.add_argument(
+ '--shape',
+ type=int,
+ nargs='+',
+ default=[256, 192],
+ help='input image size')
+ parser.add_argument(
+ '--input-constructor',
+ '-c',
+ type=str,
+ choices=['none', 'batch'],
+ default='none',
+ help='If specified, it takes a callable method that generates '
+ 'input. Otherwise, it will generate a random tensor with '
+ 'input shape to calculate FLOPs.')
+ parser.add_argument(
+ '--batch-size', '-b', type=int, default=1, help='input batch size')
+ parser.add_argument(
+ '--not-print-per-layer-stat',
+ '-n',
+ action='store_true',
+ help='Whether to print complexity information'
+ 'for each layer in a model')
+ args = parser.parse_args()
+ return args
+
+
+def batch_constructor(flops_model, batch_size, input_shape):
+ """Generate a batch of tensors to the model."""
+ batch = {}
+
+ img = torch.ones(()).new_empty(
+ (batch_size, *input_shape),
+ dtype=next(flops_model.parameters()).dtype,
+ device=next(flops_model.parameters()).device)
+
+ batch['img'] = img
+ return batch
+
+
+def main():
+
+ args = parse_args()
+
+ if len(args.shape) == 1:
+ input_shape = (3, args.shape[0], args.shape[0])
+ elif len(args.shape) == 2:
+ input_shape = (3, ) + tuple(args.shape)
+ else:
+ raise ValueError('invalid input shape')
+
+ model = init_pose_model(args.config)
+
+ if args.input_constructor == 'batch':
+ input_constructor = partial(batch_constructor, model, args.batch_size)
+ else:
+ input_constructor = None
+
+ if args.input_constructor == 'batch':
+ input_constructor = partial(batch_constructor, model, args.batch_size)
+ else:
+ input_constructor = None
+
+ if hasattr(model, 'forward_dummy'):
+ model.forward = model.forward_dummy
+ else:
+ raise NotImplementedError(
+ 'FLOPs counter is currently not currently supported with {}'.
+ format(model.__class__.__name__))
+
+ flops, params = get_model_complexity_info(
+ model,
+ input_shape,
+ input_constructor=input_constructor,
+ print_per_layer_stat=(not args.not_print_per_layer_stat))
+ split_line = '=' * 30
+ input_shape = (args.batch_size, ) + input_shape
+ print(f'{split_line}\nInput shape: {input_shape}\n'
+ f'Flops: {flops}\nParams: {params}\n{split_line}')
+ print('!!!Please be cautious if you use the results in papers. '
+ 'You may need to check if all ops are supported and verify that the '
+ 'flops computation is correct.')
+
+
+if __name__ == '__main__':
+ main()
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/analysis/print_config.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/analysis/print_config.py
new file mode 100644
index 0000000..c3538ef
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/analysis/print_config.py
@@ -0,0 +1,27 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import argparse
+
+from mmcv import Config, DictAction
+
+
+def parse_args():
+ parser = argparse.ArgumentParser(description='Print the whole config')
+ parser.add_argument('config', help='config file path')
+ parser.add_argument(
+ '--options', nargs='+', action=DictAction, help='arguments in dict')
+ args = parser.parse_args()
+
+ return args
+
+
+def main():
+ args = parse_args()
+
+ cfg = Config.fromfile(args.config)
+ if args.options is not None:
+ cfg.merge_from_dict(args.options)
+ print(f'Config:\n{cfg.pretty_text}')
+
+
+if __name__ == '__main__':
+ main()
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/analysis/speed_test.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/analysis/speed_test.py
new file mode 100644
index 0000000..fef9e2d
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/analysis/speed_test.py
@@ -0,0 +1,86 @@
+#!/usr/bin/env bash
+# Copyright (c) OpenMMLab. All rights reserved.
+import argparse
+import time
+
+import torch
+from mmcv import Config
+from mmcv.cnn import fuse_conv_bn
+from mmcv.parallel import MMDataParallel
+from mmcv.runner.fp16_utils import wrap_fp16_model
+
+from mmpose.datasets import build_dataloader, build_dataset
+from mmpose.models import build_posenet
+
+
+def parse_args():
+ parser = argparse.ArgumentParser(
+ description='MMPose benchmark a recognizer')
+ parser.add_argument('config', help='test config file path')
+ parser.add_argument('--bz', default=32, type=int, help='test config file path')
+ args = parser.parse_args()
+ return args
+
+
+def main():
+ args = parse_args()
+
+ cfg = Config.fromfile(args.config)
+
+ # Since we only care about the forward speed of the network
+ cfg.model.pretrained=None
+ cfg.model.test_cfg.flip_test=False
+ cfg.model.test_cfg.use_udp=False
+ cfg.model.test_cfg.post_process='none'
+
+ # set cudnn_benchmark
+ if cfg.get('cudnn_benchmark', False):
+ torch.backends.cudnn.benchmark = True
+
+ # build the dataloader
+ dataset = build_dataset(cfg.data.val)
+ data_loader = build_dataloader(
+ dataset,
+ samples_per_gpu=args.bz,
+ workers_per_gpu=cfg.data.workers_per_gpu,
+ dist=False,
+ shuffle=False)
+
+ # build the model and load checkpoint
+ model = build_posenet(cfg.model)
+ model = MMDataParallel(model, device_ids=[0])
+ model.eval()
+
+ # get the example data
+ for i, data in enumerate(data_loader):
+ break
+
+ # the first several iterations may be very slow so skip them
+ num_warmup = 100
+ inference_times = 100
+
+ with torch.no_grad():
+ start_time = time.perf_counter()
+
+ for i in range(num_warmup):
+ torch.cuda.synchronize()
+ model(return_loss=False, **data)
+ torch.cuda.synchronize()
+
+ elapsed = time.perf_counter() - start_time
+ print(f'warmup cost {elapsed} time')
+
+ start_time = time.perf_counter()
+
+ for i in range(inference_times):
+ torch.cuda.synchronize()
+ model(return_loss=False, **data)
+ torch.cuda.synchronize()
+
+ elapsed = time.perf_counter() - start_time
+ fps = args.bz * inference_times / elapsed
+ print(f'the fps is {fps}')
+
+
+if __name__ == '__main__':
+ main()
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/deployment/mmpose2torchserve.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/deployment/mmpose2torchserve.py
new file mode 100644
index 0000000..492a45b
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/deployment/mmpose2torchserve.py
@@ -0,0 +1,135 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os.path as osp
+import warnings
+from argparse import ArgumentParser, Namespace
+from tempfile import TemporaryDirectory
+
+import mmcv
+import torch
+from mmcv.runner import CheckpointLoader
+
+try:
+ from model_archiver.model_packaging import package_model
+ from model_archiver.model_packaging_utils import ModelExportUtils
+except ImportError:
+ package_model = None
+
+
+def mmpose2torchserve(config_file: str,
+ checkpoint_file: str,
+ output_folder: str,
+ model_name: str,
+ model_version: str = '1.0',
+ force: bool = False):
+ """Converts MMPose model (config + checkpoint) to TorchServe `.mar`.
+
+ Args:
+ config_file:
+ In MMPose config format.
+ The contents vary for each task repository.
+ checkpoint_file:
+ In MMPose checkpoint format.
+ The contents vary for each task repository.
+ output_folder:
+ Folder where `{model_name}.mar` will be created.
+ The file created will be in TorchServe archive format.
+ model_name:
+ If not None, used for naming the `{model_name}.mar` file
+ that will be created under `output_folder`.
+ If None, `{Path(checkpoint_file).stem}` will be used.
+ model_version:
+ Model's version.
+ force:
+ If True, if there is an existing `{model_name}.mar`
+ file under `output_folder` it will be overwritten.
+ """
+
+ mmcv.mkdir_or_exist(output_folder)
+
+ config = mmcv.Config.fromfile(config_file)
+
+ with TemporaryDirectory() as tmpdir:
+ model_file = osp.join(tmpdir, 'config.py')
+ config.dump(model_file)
+ handler_path = osp.join(osp.dirname(__file__), 'mmpose_handler.py')
+ model_name = model_name or osp.splitext(
+ osp.basename(checkpoint_file))[0]
+
+ # use mmcv CheckpointLoader if checkpoint is not from a local file
+ if not osp.isfile(checkpoint_file):
+ ckpt = CheckpointLoader.load_checkpoint(checkpoint_file)
+ checkpoint_file = osp.join(tmpdir, 'checkpoint.pth')
+ with open(checkpoint_file, 'wb') as f:
+ torch.save(ckpt, f)
+
+ args = Namespace(
+ **{
+ 'model_file': model_file,
+ 'serialized_file': checkpoint_file,
+ 'handler': handler_path,
+ 'model_name': model_name,
+ 'version': model_version,
+ 'export_path': output_folder,
+ 'force': force,
+ 'requirements_file': None,
+ 'extra_files': None,
+ 'runtime': 'python',
+ 'archive_format': 'default'
+ })
+ manifest = ModelExportUtils.generate_manifest_json(args)
+ package_model(args, manifest)
+
+
+def parse_args():
+ parser = ArgumentParser(
+ description='Convert MMPose models to TorchServe `.mar` format.')
+ parser.add_argument('config', type=str, help='config file path')
+ parser.add_argument('checkpoint', type=str, help='checkpoint file path')
+ parser.add_argument(
+ '--output-folder',
+ type=str,
+ required=True,
+ help='Folder where `{model_name}.mar` will be created.')
+ parser.add_argument(
+ '--model-name',
+ type=str,
+ default=None,
+ help='If not None, used for naming the `{model_name}.mar`'
+ 'file that will be created under `output_folder`.'
+ 'If None, `{Path(checkpoint_file).stem}` will be used.')
+ parser.add_argument(
+ '--model-version',
+ type=str,
+ default='1.0',
+ help='Number used for versioning.')
+ parser.add_argument(
+ '-f',
+ '--force',
+ action='store_true',
+ help='overwrite the existing `{model_name}.mar`')
+ args = parser.parse_args()
+
+ return args
+
+
+if __name__ == '__main__':
+ args = parse_args()
+
+ # Following strings of text style are from colorama package
+ bright_style, reset_style = '\x1b[1m', '\x1b[0m'
+ red_text, blue_text = '\x1b[31m', '\x1b[34m'
+ white_background = '\x1b[107m'
+
+ msg = white_background + bright_style + red_text
+ msg += 'DeprecationWarning: This tool will be deprecated in future. '
+ msg += blue_text + 'Welcome to use the unified model deployment toolbox '
+ msg += 'MMDeploy: https://github.com/open-mmlab/mmdeploy'
+ msg += reset_style
+ warnings.warn(msg)
+
+ if package_model is None:
+ raise ImportError('`torch-model-archiver` is required.'
+ 'Try: pip install torch-model-archiver')
+
+ mmpose2torchserve(args.config, args.checkpoint, args.output_folder,
+ args.model_name, args.model_version, args.force)
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/deployment/mmpose_handler.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/deployment/mmpose_handler.py
new file mode 100644
index 0000000..d7da881
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/deployment/mmpose_handler.py
@@ -0,0 +1,80 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import base64
+import os
+
+import mmcv
+import torch
+
+from mmpose.apis import (inference_bottom_up_pose_model,
+ inference_top_down_pose_model, init_pose_model)
+from mmpose.models.detectors import AssociativeEmbedding, TopDown
+
+try:
+ from ts.torch_handler.base_handler import BaseHandler
+except ImportError:
+ raise ImportError('Please install torchserve.')
+
+
+class MMPoseHandler(BaseHandler):
+
+ def initialize(self, context):
+ properties = context.system_properties
+ self.map_location = 'cuda' if torch.cuda.is_available() else 'cpu'
+ self.device = torch.device(self.map_location + ':' +
+ str(properties.get('gpu_id')) if torch.cuda.
+ is_available() else self.map_location)
+ self.manifest = context.manifest
+
+ model_dir = properties.get('model_dir')
+ serialized_file = self.manifest['model']['serializedFile']
+ checkpoint = os.path.join(model_dir, serialized_file)
+ self.config_file = os.path.join(model_dir, 'config.py')
+
+ self.model = init_pose_model(self.config_file, checkpoint, self.device)
+ self.initialized = True
+
+ def preprocess(self, data):
+ images = []
+
+ for row in data:
+ image = row.get('data') or row.get('body')
+ if isinstance(image, str):
+ image = base64.b64decode(image)
+ image = mmcv.imfrombytes(image)
+ images.append(image)
+
+ return images
+
+ def inference(self, data, *args, **kwargs):
+ if isinstance(self.model, TopDown):
+ results = self._inference_top_down_pose_model(data)
+ elif isinstance(self.model, (AssociativeEmbedding, )):
+ results = self._inference_bottom_up_pose_model(data)
+ else:
+ raise NotImplementedError(
+ f'Model type {type(self.model)} is not supported.')
+
+ return results
+
+ def _inference_top_down_pose_model(self, data):
+ results = []
+ for image in data:
+ # use dummy person bounding box
+ preds, _ = inference_top_down_pose_model(
+ self.model, image, person_results=None)
+ results.append(preds)
+ return results
+
+ def _inference_bottom_up_pose_model(self, data):
+ results = []
+ for image in data:
+ preds, _ = inference_bottom_up_pose_model(self.model, image)
+ results.append(preds)
+ return results
+
+ def postprocess(self, data):
+ output = [[{
+ 'keypoints': pred['keypoints'].tolist()
+ } for pred in preds] for preds in data]
+
+ return output
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/deployment/pytorch2onnx.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/deployment/pytorch2onnx.py
new file mode 100644
index 0000000..5caff6e
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/deployment/pytorch2onnx.py
@@ -0,0 +1,165 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import argparse
+import warnings
+
+import numpy as np
+import torch
+
+from mmpose.apis import init_pose_model
+
+try:
+ import onnx
+ import onnxruntime as rt
+except ImportError as e:
+ raise ImportError(f'Please install onnx and onnxruntime first. {e}')
+
+try:
+ from mmcv.onnx.symbolic import register_extra_symbolics
+except ModuleNotFoundError:
+ raise NotImplementedError('please update mmcv to version>=1.0.4')
+
+
+def _convert_batchnorm(module):
+ """Convert the syncBNs into normal BN3ds."""
+ module_output = module
+ if isinstance(module, torch.nn.SyncBatchNorm):
+ module_output = torch.nn.BatchNorm3d(module.num_features, module.eps,
+ module.momentum, module.affine,
+ module.track_running_stats)
+ if module.affine:
+ module_output.weight.data = module.weight.data.clone().detach()
+ module_output.bias.data = module.bias.data.clone().detach()
+ # keep requires_grad unchanged
+ module_output.weight.requires_grad = module.weight.requires_grad
+ module_output.bias.requires_grad = module.bias.requires_grad
+ module_output.running_mean = module.running_mean
+ module_output.running_var = module.running_var
+ module_output.num_batches_tracked = module.num_batches_tracked
+ for name, child in module.named_children():
+ module_output.add_module(name, _convert_batchnorm(child))
+ del module
+ return module_output
+
+
+def pytorch2onnx(model,
+ input_shape,
+ opset_version=11,
+ show=False,
+ output_file='tmp.onnx',
+ verify=False):
+ """Convert pytorch model to onnx model.
+
+ Args:
+ model (:obj:`nn.Module`): The pytorch model to be exported.
+ input_shape (tuple[int]): The input tensor shape of the model.
+ opset_version (int): Opset version of onnx used. Default: 11.
+ show (bool): Determines whether to print the onnx model architecture.
+ Default: False.
+ output_file (str): Output onnx model name. Default: 'tmp.onnx'.
+ verify (bool): Determines whether to verify the onnx model.
+ Default: False.
+ """
+ model.cpu().eval()
+
+ one_img = torch.randn(input_shape)
+
+ register_extra_symbolics(opset_version)
+ torch.onnx.export(
+ model,
+ one_img,
+ output_file,
+ export_params=True,
+ keep_initializers_as_inputs=True,
+ verbose=show,
+ opset_version=opset_version)
+
+ print(f'Successfully exported ONNX model: {output_file}')
+ if verify:
+ # check by onnx
+ onnx_model = onnx.load(output_file)
+ onnx.checker.check_model(onnx_model)
+
+ # check the numerical value
+ # get pytorch output
+ pytorch_results = model(one_img)
+ if not isinstance(pytorch_results, (list, tuple)):
+ assert isinstance(pytorch_results, torch.Tensor)
+ pytorch_results = [pytorch_results]
+
+ # get onnx output
+ input_all = [node.name for node in onnx_model.graph.input]
+ input_initializer = [
+ node.name for node in onnx_model.graph.initializer
+ ]
+ net_feed_input = list(set(input_all) - set(input_initializer))
+ assert len(net_feed_input) == 1
+ sess = rt.InferenceSession(output_file)
+ onnx_results = sess.run(None,
+ {net_feed_input[0]: one_img.detach().numpy()})
+
+ # compare results
+ assert len(pytorch_results) == len(onnx_results)
+ for pt_result, onnx_result in zip(pytorch_results, onnx_results):
+ assert np.allclose(
+ pt_result.detach().cpu(), onnx_result, atol=1.e-5
+ ), 'The outputs are different between Pytorch and ONNX'
+ print('The numerical values are same between Pytorch and ONNX')
+
+
+def parse_args():
+ parser = argparse.ArgumentParser(
+ description='Convert MMPose models to ONNX')
+ parser.add_argument('config', help='test config file path')
+ parser.add_argument('checkpoint', help='checkpoint file')
+ parser.add_argument('--show', action='store_true', help='show onnx graph')
+ parser.add_argument('--output-file', type=str, default='tmp.onnx')
+ parser.add_argument('--opset-version', type=int, default=11)
+ parser.add_argument(
+ '--verify',
+ action='store_true',
+ help='verify the onnx model output against pytorch output')
+ parser.add_argument(
+ '--shape',
+ type=int,
+ nargs='+',
+ default=[1, 3, 256, 192],
+ help='input size')
+ args = parser.parse_args()
+ return args
+
+
+if __name__ == '__main__':
+ args = parse_args()
+
+ assert args.opset_version == 11, 'MMPose only supports opset 11 now'
+
+ # Following strings of text style are from colorama package
+ bright_style, reset_style = '\x1b[1m', '\x1b[0m'
+ red_text, blue_text = '\x1b[31m', '\x1b[34m'
+ white_background = '\x1b[107m'
+
+ msg = white_background + bright_style + red_text
+ msg += 'DeprecationWarning: This tool will be deprecated in future. '
+ msg += blue_text + 'Welcome to use the unified model deployment toolbox '
+ msg += 'MMDeploy: https://github.com/open-mmlab/mmdeploy'
+ msg += reset_style
+ warnings.warn(msg)
+
+ model = init_pose_model(args.config, args.checkpoint, device='cpu')
+ model = _convert_batchnorm(model)
+
+ # onnx.export does not support kwargs
+ if hasattr(model, 'forward_dummy'):
+ model.forward = model.forward_dummy
+ else:
+ raise NotImplementedError(
+ 'Please implement the forward method for exporting.')
+
+ # convert model to onnx file
+ pytorch2onnx(
+ model,
+ args.shape,
+ opset_version=args.opset_version,
+ show=args.show,
+ output_file=args.output_file,
+ verify=args.verify)
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/deployment/test_torchserver.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/deployment/test_torchserver.py
new file mode 100644
index 0000000..70e27c5
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/deployment/test_torchserver.py
@@ -0,0 +1,79 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os
+import os.path as osp
+import warnings
+from argparse import ArgumentParser
+
+import requests
+
+from mmpose.apis import (inference_bottom_up_pose_model,
+ inference_top_down_pose_model, init_pose_model,
+ vis_pose_result)
+from mmpose.models import AssociativeEmbedding, TopDown
+
+
+def parse_args():
+ parser = ArgumentParser()
+ parser.add_argument('img', help='Image file')
+ parser.add_argument('config', help='Config file')
+ parser.add_argument('checkpoint', help='Checkpoint file')
+ parser.add_argument('model_name', help='The model name in the server')
+ parser.add_argument(
+ '--inference-addr',
+ default='127.0.0.1:8080',
+ help='Address and port of the inference server')
+ parser.add_argument(
+ '--device', default='cuda:0', help='Device used for inference')
+ parser.add_argument(
+ '--out-dir', default='vis_results', help='Visualization output path')
+ args = parser.parse_args()
+ return args
+
+
+def main(args):
+ os.makedirs(args.out_dir, exist_ok=True)
+
+ # Inference single image by native apis.
+ model = init_pose_model(args.config, args.checkpoint, device=args.device)
+ if isinstance(model, TopDown):
+ pytorch_result, _ = inference_top_down_pose_model(
+ model, args.img, person_results=None)
+ elif isinstance(model, (AssociativeEmbedding, )):
+ pytorch_result, _ = inference_bottom_up_pose_model(model, args.img)
+ else:
+ raise NotImplementedError()
+
+ vis_pose_result(
+ model,
+ args.img,
+ pytorch_result,
+ out_file=osp.join(args.out_dir, 'pytorch_result.png'))
+
+ # Inference single image by torchserve engine.
+ url = 'http://' + args.inference_addr + '/predictions/' + args.model_name
+ with open(args.img, 'rb') as image:
+ response = requests.post(url, image)
+ server_result = response.json()
+
+ vis_pose_result(
+ model,
+ args.img,
+ server_result,
+ out_file=osp.join(args.out_dir, 'torchserve_result.png'))
+
+
+if __name__ == '__main__':
+ args = parse_args()
+ main(args)
+
+ # Following strings of text style are from colorama package
+ bright_style, reset_style = '\x1b[1m', '\x1b[0m'
+ red_text, blue_text = '\x1b[31m', '\x1b[34m'
+ white_background = '\x1b[107m'
+
+ msg = white_background + bright_style + red_text
+ msg += 'DeprecationWarning: This tool will be deprecated in future. '
+ msg += blue_text + 'Welcome to use the unified model deployment toolbox '
+ msg += 'MMDeploy: https://github.com/open-mmlab/mmdeploy'
+ msg += reset_style
+ warnings.warn(msg)
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/dist_test.sh b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/dist_test.sh
new file mode 100644
index 0000000..9dcb885
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/dist_test.sh
@@ -0,0 +1,11 @@
+#!/usr/bin/env bash
+# Copyright (c) OpenMMLab. All rights reserved.
+
+CONFIG=$1
+CHECKPOINT=$2
+GPUS=$3
+PORT=${PORT:-29500}
+
+PYTHONPATH="$(dirname $0)/..":$PYTHONPATH \
+python -m torch.distributed.launch --nproc_per_node=$GPUS --master_port=$PORT \
+ $(dirname "$0")/test.py $CONFIG $CHECKPOINT --launcher pytorch ${@:4}
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/dist_train.sh b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/dist_train.sh
new file mode 100644
index 0000000..9727f53
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/dist_train.sh
@@ -0,0 +1,10 @@
+#!/usr/bin/env bash
+# Copyright (c) OpenMMLab. All rights reserved.
+
+CONFIG=$1
+GPUS=$2
+PORT=${PORT:-29500}
+
+PYTHONPATH="$(dirname $0)/..":$PYTHONPATH \
+python -m torch.distributed.launch --nproc_per_node=$GPUS --master_port=$PORT \
+ $(dirname "$0")/train.py $CONFIG --launcher pytorch ${@:3}
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/misc/keypoints2coco_without_mmdet.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/misc/keypoints2coco_without_mmdet.py
new file mode 100644
index 0000000..63220fc
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/misc/keypoints2coco_without_mmdet.py
@@ -0,0 +1,146 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import json
+import os
+from argparse import ArgumentParser
+
+from mmcv import track_iter_progress
+from PIL import Image
+from xtcocotools.coco import COCO
+
+from mmpose.apis import inference_top_down_pose_model, init_pose_model
+
+
+def main():
+ """Visualize the demo images.
+
+ pose_keypoints require the json_file containing boxes.
+ """
+ parser = ArgumentParser()
+ parser.add_argument('pose_config', help='Config file for detection')
+ parser.add_argument('pose_checkpoint', help='Checkpoint file')
+ parser.add_argument('--img-root', type=str, default='', help='Image root')
+ parser.add_argument(
+ '--json-file',
+ type=str,
+ default='',
+ help='Json file containing image person bboxes in COCO format.')
+ parser.add_argument(
+ '--out-json-file',
+ type=str,
+ default='',
+ help='Output json contains pseudolabeled annotation')
+ parser.add_argument(
+ '--show',
+ action='store_true',
+ default=False,
+ help='whether to show img')
+ parser.add_argument(
+ '--device', default='cuda:0', help='Device used for inference')
+ parser.add_argument(
+ '--kpt-thr', type=float, default=0.3, help='Keypoint score threshold')
+
+ args = parser.parse_args()
+
+ coco = COCO(args.json_file)
+ # build the pose model from a config file and a checkpoint file
+ pose_model = init_pose_model(
+ args.pose_config, args.pose_checkpoint, device=args.device.lower())
+
+ dataset = pose_model.cfg.data['test']['type']
+
+ img_keys = list(coco.imgs.keys())
+
+ # optional
+ return_heatmap = False
+
+ # e.g. use ('backbone', ) to return backbone feature
+ output_layer_names = None
+
+ categories = [{'id': 1, 'name': 'person'}]
+ img_anno_dict = {'images': [], 'annotations': [], 'categories': categories}
+
+ # process each image
+ ann_uniq_id = int(0)
+ for i in track_iter_progress(range(len(img_keys))):
+ # get bounding box annotations
+ image_id = img_keys[i]
+ image = coco.loadImgs(image_id)[0]
+ image_name = os.path.join(args.img_root, image['file_name'])
+
+ width, height = Image.open(image_name).size
+ ann_ids = coco.getAnnIds(image_id)
+
+ # make person bounding boxes
+ person_results = []
+ for ann_id in ann_ids:
+ person = {}
+ ann = coco.anns[ann_id]
+ # bbox format is 'xywh'
+ person['bbox'] = ann['bbox']
+ person_results.append(person)
+
+ pose_results, returned_outputs = inference_top_down_pose_model(
+ pose_model,
+ image_name,
+ person_results,
+ bbox_thr=None,
+ format='xywh',
+ dataset=dataset,
+ return_heatmap=return_heatmap,
+ outputs=output_layer_names)
+
+ # add output of model and bboxes to dict
+ for indx, i in enumerate(pose_results):
+ pose_results[indx]['keypoints'][
+ pose_results[indx]['keypoints'][:, 2] < args.kpt_thr, :3] = 0
+ pose_results[indx]['keypoints'][
+ pose_results[indx]['keypoints'][:, 2] >= args.kpt_thr, 2] = 2
+ x = int(pose_results[indx]['bbox'][0])
+ y = int(pose_results[indx]['bbox'][1])
+ w = int(pose_results[indx]['bbox'][2] -
+ pose_results[indx]['bbox'][0])
+ h = int(pose_results[indx]['bbox'][3] -
+ pose_results[indx]['bbox'][1])
+ bbox = [x, y, w, h]
+ area = round((w * h), 0)
+
+ images = {
+ 'file_name': image_name.split('/')[-1],
+ 'height': height,
+ 'width': width,
+ 'id': int(image_id)
+ }
+
+ annotations = {
+ 'keypoints': [
+ int(i) for i in pose_results[indx]['keypoints'].reshape(
+ -1).tolist()
+ ],
+ 'num_keypoints':
+ len(pose_results[indx]['keypoints']),
+ 'area':
+ area,
+ 'iscrowd':
+ 0,
+ 'image_id':
+ int(image_id),
+ 'bbox':
+ bbox,
+ 'category_id':
+ 1,
+ 'id':
+ ann_uniq_id,
+ }
+
+ img_anno_dict['annotations'].append(annotations)
+ ann_uniq_id += 1
+
+ img_anno_dict['images'].append(images)
+
+ # create json
+ with open(args.out_json_file, 'w') as outfile:
+ json.dump(img_anno_dict, outfile, indent=2)
+
+
+if __name__ == '__main__':
+ main()
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/misc/publish_model.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/misc/publish_model.py
new file mode 100644
index 0000000..393721a
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/misc/publish_model.py
@@ -0,0 +1,43 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import argparse
+import subprocess
+from datetime import date
+
+import torch
+
+
+def parse_args():
+ parser = argparse.ArgumentParser(
+ description='Process a checkpoint to be published')
+ parser.add_argument('in_file', help='input checkpoint filename')
+ parser.add_argument('out_file', help='output checkpoint filename')
+ args = parser.parse_args()
+ return args
+
+
+def process_checkpoint(in_file, out_file):
+ checkpoint = torch.load(in_file, map_location='cpu')
+ # remove optimizer for smaller file size
+ if 'optimizer' in checkpoint:
+ del checkpoint['optimizer']
+ # if it is necessary to remove some sensitive data in checkpoint['meta'],
+ # add the code here.
+ torch.save(checkpoint, out_file)
+ sha = subprocess.check_output(['sha256sum', out_file]).decode()
+ if out_file.endswith('.pth'):
+ out_file_name = out_file[:-4]
+ else:
+ out_file_name = out_file
+
+ date_now = date.today().strftime('%Y%m%d')
+ final_file = out_file_name + f'-{sha[:8]}_{date_now}.pth'
+ subprocess.Popen(['mv', out_file, final_file])
+
+
+def main():
+ args = parse_args()
+ process_checkpoint(args.in_file, args.out_file)
+
+
+if __name__ == '__main__':
+ main()
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/model_split.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/model_split.py
new file mode 100644
index 0000000..928380a
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/model_split.py
@@ -0,0 +1,104 @@
+import torch
+import os
+import argparse
+import copy
+
+def parse_args():
+ parser = argparse.ArgumentParser()
+ parser.add_argument('--source', type=str)
+ parser.add_argument('--target', type=str, default=None)
+ args = parser.parse_args()
+ return args
+
+def main():
+
+ args = parse_args()
+
+ if args.target is None:
+ args.target = '/'.join(args.source.split('/')[:-1])
+
+ ckpt = torch.load(args.source, map_location='cpu')
+
+ experts = dict()
+
+ new_ckpt = copy.deepcopy(ckpt)
+
+ state_dict = new_ckpt['state_dict']
+
+ for key, value in state_dict.items():
+ if 'mlp.experts' in key:
+ experts[key] = value
+
+ keys = ckpt['state_dict'].keys()
+
+ target_expert = 0
+ new_ckpt = copy.deepcopy(ckpt)
+
+ for key in keys:
+ if 'mlp.fc2' in key:
+ value = new_ckpt['state_dict'][key]
+ value = torch.cat([value, experts[key.replace('fc2.', f'experts.{target_expert}.')]], dim=0)
+ new_ckpt['state_dict'][key] = value
+
+ torch.save(new_ckpt, os.path.join(args.target, 'coco.pth'))
+
+ names = ['aic', 'mpii', 'ap10k', 'apt36k','wholebody']
+ num_keypoints = [14, 16, 17, 17, 133]
+ weight_names = ['keypoint_head.deconv_layers.0.weight',
+ 'keypoint_head.deconv_layers.1.weight',
+ 'keypoint_head.deconv_layers.1.bias',
+ 'keypoint_head.deconv_layers.1.running_mean',
+ 'keypoint_head.deconv_layers.1.running_var',
+ 'keypoint_head.deconv_layers.1.num_batches_tracked',
+ 'keypoint_head.deconv_layers.3.weight',
+ 'keypoint_head.deconv_layers.4.weight',
+ 'keypoint_head.deconv_layers.4.bias',
+ 'keypoint_head.deconv_layers.4.running_mean',
+ 'keypoint_head.deconv_layers.4.running_var',
+ 'keypoint_head.deconv_layers.4.num_batches_tracked',
+ 'keypoint_head.final_layer.weight',
+ 'keypoint_head.final_layer.bias']
+
+ exist_range = True
+
+ for i in range(5):
+
+ new_ckpt = copy.deepcopy(ckpt)
+
+ target_expert = i + 1
+
+ for key in keys:
+ if 'mlp.fc2' in key:
+ expert_key = key.replace('fc2.', f'experts.{target_expert}.')
+ if expert_key in experts:
+ value = new_ckpt['state_dict'][key]
+ value = torch.cat([value, experts[expert_key]], dim=0)
+ else:
+ exist_range = False
+
+ new_ckpt['state_dict'][key] = value
+
+ if not exist_range:
+ break
+
+ for tensor_name in weight_names:
+ new_ckpt['state_dict'][tensor_name] = new_ckpt['state_dict'][tensor_name.replace('keypoint_head', f'associate_keypoint_heads.{i}')]
+
+ for tensor_name in ['keypoint_head.final_layer.weight', 'keypoint_head.final_layer.bias']:
+ new_ckpt['state_dict'][tensor_name] = new_ckpt['state_dict'][tensor_name][:num_keypoints[i]]
+
+ # remove unnecessary part in the state dict
+ for j in range(5):
+ # remove associate part
+ for tensor_name in weight_names:
+ new_ckpt['state_dict'].pop(tensor_name.replace('keypoint_head', f'associate_keypoint_heads.{j}'))
+ # remove expert part
+ keys = new_ckpt['state_dict'].keys()
+ for key in list(keys):
+ if 'expert' in keys:
+ new_ckpt['state_dict'].pop(key)
+
+ torch.save(new_ckpt, os.path.join(args.target, f'{names[i]}.pth'))
+
+if __name__ == '__main__':
+ main()
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/slurm_test.sh b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/slurm_test.sh
new file mode 100644
index 0000000..c528dc9
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/slurm_test.sh
@@ -0,0 +1,25 @@
+#!/usr/bin/env bash
+# Copyright (c) OpenMMLab. All rights reserved.
+
+set -x
+
+PARTITION=$1
+JOB_NAME=$2
+CONFIG=$3
+CHECKPOINT=$4
+GPUS=${GPUS:-8}
+GPUS_PER_NODE=${GPUS_PER_NODE:-8}
+CPUS_PER_TASK=${CPUS_PER_TASK:-5}
+PY_ARGS=${@:5}
+SRUN_ARGS=${SRUN_ARGS:-""}
+
+PYTHONPATH="$(dirname $0)/..":$PYTHONPATH \
+srun -p ${PARTITION} \
+ --job-name=${JOB_NAME} \
+ --gres=gpu:${GPUS_PER_NODE} \
+ --ntasks=${GPUS} \
+ --ntasks-per-node=${GPUS_PER_NODE} \
+ --cpus-per-task=${CPUS_PER_TASK} \
+ --kill-on-bad-exit=1 \
+ ${SRUN_ARGS} \
+ python -u tools/test.py ${CONFIG} ${CHECKPOINT} --launcher="slurm" ${PY_ARGS}
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/slurm_train.sh b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/slurm_train.sh
new file mode 100644
index 0000000..c3b6549
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/slurm_train.sh
@@ -0,0 +1,25 @@
+#!/usr/bin/env bash
+# Copyright (c) OpenMMLab. All rights reserved.
+
+set -x
+
+PARTITION=$1
+JOB_NAME=$2
+CONFIG=$3
+WORK_DIR=$4
+GPUS=${GPUS:-8}
+GPUS_PER_NODE=${GPUS_PER_NODE:-8}
+CPUS_PER_TASK=${CPUS_PER_TASK:-5}
+SRUN_ARGS=${SRUN_ARGS:-""}
+PY_ARGS=${@:5}
+
+PYTHONPATH="$(dirname $0)/..":$PYTHONPATH \
+srun -p ${PARTITION} \
+ --job-name=${JOB_NAME} \
+ --gres=gpu:${GPUS_PER_NODE} \
+ --ntasks=${GPUS} \
+ --ntasks-per-node=${GPUS_PER_NODE} \
+ --cpus-per-task=${CPUS_PER_TASK} \
+ --kill-on-bad-exit=1 \
+ ${SRUN_ARGS} \
+ python -u tools/train.py ${CONFIG} --work-dir=${WORK_DIR} --launcher="slurm" ${PY_ARGS}
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/test.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/test.py
new file mode 100644
index 0000000..d153992
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/test.py
@@ -0,0 +1,184 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import argparse
+import os
+import os.path as osp
+import warnings
+
+import mmcv
+import torch
+from mmcv import Config, DictAction
+from mmcv.cnn import fuse_conv_bn
+from mmcv.parallel import MMDataParallel, MMDistributedDataParallel
+from mmcv.runner import get_dist_info, init_dist, load_checkpoint
+
+from mmpose.apis import multi_gpu_test, single_gpu_test
+from mmpose.datasets import build_dataloader, build_dataset
+from mmpose.models import build_posenet
+from mmpose.utils import setup_multi_processes
+
+try:
+ from mmcv.runner import wrap_fp16_model
+except ImportError:
+ warnings.warn('auto_fp16 from mmpose will be deprecated from v0.15.0'
+ 'Please install mmcv>=1.1.4')
+ from mmpose.core import wrap_fp16_model
+
+
+def parse_args():
+ parser = argparse.ArgumentParser(description='mmpose test model')
+ parser.add_argument('config', help='test config file path')
+ parser.add_argument('checkpoint', help='checkpoint file')
+ parser.add_argument('--out', help='output result file')
+ parser.add_argument(
+ '--work-dir', help='the dir to save evaluation results')
+ parser.add_argument(
+ '--fuse-conv-bn',
+ action='store_true',
+ help='Whether to fuse conv and bn, this will slightly increase'
+ 'the inference speed')
+ parser.add_argument(
+ '--gpu-id',
+ type=int,
+ default=0,
+ help='id of gpu to use '
+ '(only applicable to non-distributed testing)')
+ parser.add_argument(
+ '--eval',
+ default=None,
+ nargs='+',
+ help='evaluation metric, which depends on the dataset,'
+ ' e.g., "mAP" for MSCOCO')
+ parser.add_argument(
+ '--gpu_collect',
+ action='store_true',
+ help='whether to use gpu to collect results')
+ parser.add_argument('--tmpdir', help='tmp dir for writing some results')
+ parser.add_argument(
+ '--cfg-options',
+ nargs='+',
+ action=DictAction,
+ default={},
+ help='override some settings in the used config, the key-value pair '
+ 'in xxx=yyy format will be merged into config file. For example, '
+ "'--cfg-options model.backbone.depth=18 model.backbone.with_cp=True'")
+ parser.add_argument(
+ '--launcher',
+ choices=['none', 'pytorch', 'slurm', 'mpi'],
+ default='none',
+ help='job launcher')
+ parser.add_argument('--local_rank', type=int, default=0)
+ args = parser.parse_args()
+ if 'LOCAL_RANK' not in os.environ:
+ os.environ['LOCAL_RANK'] = str(args.local_rank)
+ return args
+
+
+def merge_configs(cfg1, cfg2):
+ # Merge cfg2 into cfg1
+ # Overwrite cfg1 if repeated, ignore if value is None.
+ cfg1 = {} if cfg1 is None else cfg1.copy()
+ cfg2 = {} if cfg2 is None else cfg2
+ for k, v in cfg2.items():
+ if v:
+ cfg1[k] = v
+ return cfg1
+
+
+def main():
+ args = parse_args()
+
+ cfg = Config.fromfile(args.config)
+
+ if args.cfg_options is not None:
+ cfg.merge_from_dict(args.cfg_options)
+
+ # set multi-process settings
+ setup_multi_processes(cfg)
+
+ # set cudnn_benchmark
+ if cfg.get('cudnn_benchmark', False):
+ torch.backends.cudnn.benchmark = True
+ cfg.model.pretrained = None
+ cfg.data.test.test_mode = True
+
+ # work_dir is determined in this priority: CLI > segment in file > filename
+ if args.work_dir is not None:
+ # update configs according to CLI args if args.work_dir is not None
+ cfg.work_dir = args.work_dir
+ elif cfg.get('work_dir', None) is None:
+ # use config filename as default work_dir if cfg.work_dir is None
+ cfg.work_dir = osp.join('./work_dirs',
+ osp.splitext(osp.basename(args.config))[0])
+
+ mmcv.mkdir_or_exist(osp.abspath(cfg.work_dir))
+
+ # init distributed env first, since logger depends on the dist info.
+ if args.launcher == 'none':
+ distributed = False
+ else:
+ distributed = True
+ init_dist(args.launcher, **cfg.dist_params)
+
+ # build the dataloader
+ dataset = build_dataset(cfg.data.test, dict(test_mode=True))
+ # step 1: give default values and override (if exist) from cfg.data
+ loader_cfg = {
+ **dict(seed=cfg.get('seed'), drop_last=False, dist=distributed),
+ **({} if torch.__version__ != 'parrots' else dict(
+ prefetch_num=2,
+ pin_memory=False,
+ )),
+ **dict((k, cfg.data[k]) for k in [
+ 'seed',
+ 'prefetch_num',
+ 'pin_memory',
+ 'persistent_workers',
+ ] if k in cfg.data)
+ }
+ # step2: cfg.data.test_dataloader has higher priority
+ test_loader_cfg = {
+ **loader_cfg,
+ **dict(shuffle=False, drop_last=False),
+ **dict(workers_per_gpu=cfg.data.get('workers_per_gpu', 1)),
+ **dict(samples_per_gpu=cfg.data.get('samples_per_gpu', 1)),
+ **cfg.data.get('test_dataloader', {})
+ }
+ data_loader = build_dataloader(dataset, **test_loader_cfg)
+
+ # build the model and load checkpoint
+ model = build_posenet(cfg.model)
+ fp16_cfg = cfg.get('fp16', None)
+ if fp16_cfg is not None:
+ wrap_fp16_model(model)
+ load_checkpoint(model, args.checkpoint, map_location='cpu')
+
+ if args.fuse_conv_bn:
+ model = fuse_conv_bn(model)
+
+ if not distributed:
+ model = MMDataParallel(model, device_ids=[args.gpu_id])
+ outputs = single_gpu_test(model, data_loader)
+ else:
+ model = MMDistributedDataParallel(
+ model.cuda(),
+ device_ids=[torch.cuda.current_device()],
+ broadcast_buffers=False)
+ outputs = multi_gpu_test(model, data_loader, args.tmpdir,
+ args.gpu_collect)
+
+ rank, _ = get_dist_info()
+ eval_config = cfg.get('evaluation', {})
+ eval_config = merge_configs(eval_config, dict(metric=args.eval))
+
+ if rank == 0:
+ if args.out:
+ print(f'\nwriting results to {args.out}')
+ mmcv.dump(outputs, args.out)
+
+ results = dataset.evaluate(outputs, cfg.work_dir, **eval_config)
+ for k, v in sorted(results.items()):
+ print(f'{k}: {v}')
+
+
+if __name__ == '__main__':
+ main()
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/train.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/train.py
new file mode 100644
index 0000000..2e1f707
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/train.py
@@ -0,0 +1,195 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import argparse
+import copy
+import os
+import os.path as osp
+import time
+import warnings
+
+import mmcv
+import torch
+from mmcv import Config, DictAction
+from mmcv.runner import get_dist_info, init_dist, set_random_seed
+from mmcv.utils import get_git_hash
+
+from mmpose import __version__
+from mmpose.apis import init_random_seed, train_model
+from mmpose.datasets import build_dataset
+from mmpose.models import build_posenet
+from mmpose.utils import collect_env, get_root_logger, setup_multi_processes
+import mmcv_custom
+
+def parse_args():
+ parser = argparse.ArgumentParser(description='Train a pose model')
+ parser.add_argument('config', help='train config file path')
+ parser.add_argument('--work-dir', help='the dir to save logs and models')
+ parser.add_argument(
+ '--resume-from', help='the checkpoint file to resume from')
+ parser.add_argument(
+ '--no-validate',
+ action='store_true',
+ help='whether not to evaluate the checkpoint during training')
+ group_gpus = parser.add_mutually_exclusive_group()
+ group_gpus.add_argument(
+ '--gpus',
+ type=int,
+ help='(Deprecated, please use --gpu-id) number of gpus to use '
+ '(only applicable to non-distributed training)')
+ group_gpus.add_argument(
+ '--gpu-ids',
+ type=int,
+ nargs='+',
+ help='(Deprecated, please use --gpu-id) ids of gpus to use '
+ '(only applicable to non-distributed training)')
+ group_gpus.add_argument(
+ '--gpu-id',
+ type=int,
+ default=0,
+ help='id of gpu to use '
+ '(only applicable to non-distributed training)')
+ parser.add_argument('--seed', type=int, default=None, help='random seed')
+ parser.add_argument(
+ '--deterministic',
+ action='store_true',
+ help='whether to set deterministic options for CUDNN backend.')
+ parser.add_argument(
+ '--cfg-options',
+ nargs='+',
+ action=DictAction,
+ default={},
+ help='override some settings in the used config, the key-value pair '
+ 'in xxx=yyy format will be merged into config file. For example, '
+ "'--cfg-options model.backbone.depth=18 model.backbone.with_cp=True'")
+ parser.add_argument(
+ '--launcher',
+ choices=['none', 'pytorch', 'slurm', 'mpi'],
+ default='none',
+ help='job launcher')
+ parser.add_argument('--local_rank', type=int, default=0)
+ parser.add_argument(
+ '--autoscale-lr',
+ action='store_true',
+ help='automatically scale lr with the number of gpus')
+ args = parser.parse_args()
+ if 'LOCAL_RANK' not in os.environ:
+ os.environ['LOCAL_RANK'] = str(args.local_rank)
+
+ return args
+
+
+def main():
+ args = parse_args()
+
+ cfg = Config.fromfile(args.config)
+
+ if args.cfg_options is not None:
+ cfg.merge_from_dict(args.cfg_options)
+
+ # set multi-process settings
+ setup_multi_processes(cfg)
+
+ # set cudnn_benchmark
+ if cfg.get('cudnn_benchmark', False):
+ torch.backends.cudnn.benchmark = True
+
+ # work_dir is determined in this priority: CLI > segment in file > filename
+ if args.work_dir is not None:
+ # update configs according to CLI args if args.work_dir is not None
+ cfg.work_dir = args.work_dir
+ elif cfg.get('work_dir', None) is None:
+ # use config filename as default work_dir if cfg.work_dir is None
+ cfg.work_dir = osp.join('./work_dirs',
+ osp.splitext(osp.basename(args.config))[0])
+ if args.resume_from is not None:
+ cfg.resume_from = args.resume_from
+ if args.gpus is not None:
+ cfg.gpu_ids = range(1)
+ warnings.warn('`--gpus` is deprecated because we only support '
+ 'single GPU mode in non-distributed training. '
+ 'Use `gpus=1` now.')
+ if args.gpu_ids is not None:
+ cfg.gpu_ids = args.gpu_ids[0:1]
+ warnings.warn('`--gpu-ids` is deprecated, please use `--gpu-id`. '
+ 'Because we only support single GPU mode in '
+ 'non-distributed training. Use the first GPU '
+ 'in `gpu_ids` now.')
+ if args.gpus is None and args.gpu_ids is None:
+ cfg.gpu_ids = [args.gpu_id]
+
+ if args.autoscale_lr:
+ # apply the linear scaling rule (https://arxiv.org/abs/1706.02677)
+ cfg.optimizer['lr'] = cfg.optimizer['lr'] * len(cfg.gpu_ids) / 8
+
+ # init distributed env first, since logger depends on the dist info.
+ if args.launcher == 'none':
+ distributed = False
+ if len(cfg.gpu_ids) > 1:
+ warnings.warn(
+ f'We treat {cfg.gpu_ids} as gpu-ids, and reset to '
+ f'{cfg.gpu_ids[0:1]} as gpu-ids to avoid potential error in '
+ 'non-distribute training time.')
+ cfg.gpu_ids = cfg.gpu_ids[0:1]
+ else:
+ distributed = True
+ init_dist(args.launcher, **cfg.dist_params)
+ # re-set gpu_ids with distributed training mode
+ _, world_size = get_dist_info()
+ cfg.gpu_ids = range(world_size)
+
+ # create work_dir
+ mmcv.mkdir_or_exist(osp.abspath(cfg.work_dir))
+ # init the logger before other steps
+ timestamp = time.strftime('%Y%m%d_%H%M%S', time.localtime())
+ log_file = osp.join(cfg.work_dir, f'{timestamp}.log')
+ logger = get_root_logger(log_file=log_file, log_level=cfg.log_level)
+
+ # init the meta dict to record some important information such as
+ # environment info and seed, which will be logged
+ meta = dict()
+ # log env info
+ env_info_dict = collect_env()
+ env_info = '\n'.join([(f'{k}: {v}') for k, v in env_info_dict.items()])
+ dash_line = '-' * 60 + '\n'
+ logger.info('Environment info:\n' + dash_line + env_info + '\n' +
+ dash_line)
+ meta['env_info'] = env_info
+
+ # log some basic info
+ logger.info(f'Distributed training: {distributed}')
+ logger.info(f'Config:\n{cfg.pretty_text}')
+
+ # set random seeds
+ seed = init_random_seed(args.seed)
+ logger.info(f'Set random seed to {seed}, '
+ f'deterministic: {args.deterministic}')
+ set_random_seed(seed, deterministic=args.deterministic)
+ cfg.seed = seed
+ meta['seed'] = seed
+
+ model = build_posenet(cfg.model)
+ datasets = [build_dataset(cfg.data.train)]
+
+ if len(cfg.workflow) == 2:
+ val_dataset = copy.deepcopy(cfg.data.val)
+ val_dataset.pipeline = cfg.data.train.pipeline
+ datasets.append(build_dataset(val_dataset))
+
+ if cfg.checkpoint_config is not None:
+ # save mmpose version, config file content
+ # checkpoints as meta data
+ cfg.checkpoint_config.meta = dict(
+ mmpose_version=__version__ + get_git_hash(digits=7),
+ config=cfg.pretty_text,
+ )
+ train_model(
+ model,
+ datasets,
+ cfg,
+ distributed=distributed,
+ validate=(not args.no_validate),
+ timestamp=timestamp,
+ meta=meta)
+
+
+if __name__ == '__main__':
+ main()
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/README.md b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/README.md
new file mode 100644
index 0000000..30960fd
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/README.md
@@ -0,0 +1,28 @@
+# MMPose Webcam API
+
+MMPose Webcam API is a handy tool to develop interactive webcam applications with MMPose functions.
+
+
+
+MMPose Webcam API Overview
+
+
+## Requirements
+
+* Python >= 3.7.0
+* MMPose >= 0.23.0
+* MMDetection >= 2.21.0
+
+## Tutorials
+
+* [Get started with MMPose Webcam API (Chinese)](/tools/webcam/docs/get_started_cn.md)
+* [Build a Webcam App: A Step-by-step Instruction (Chinese)](/tools/webcam/docs/example_cn.md)
+
+## Examples
+
+* [Pose Estimation](/tools/webcam/configs/examples/): A simple example to estimate and visualize human/animal pose.
+* [Eye Effects](/tools/webcam/configs/eyes/): Apply sunglasses and bug-eye effects.
+* [Face Swap](/tools/webcam/configs/face_swap/): Everybody gets someone else's face.
+* [Meow Dwen Dwen](/tools/webcam/configs/meow_dwen_dwen/): Dress up your cat in Bing Dwen Dwen costume.
+* [Super Saiyan](/tools/webcam/configs/supersaiyan/): Super Saiyan transformation!
+* [New Year](/tools/webcam/configs/newyear/): Set off some firecrackers to celebrate Chinese New Year.
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/configs/background/README.md b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/configs/background/README.md
new file mode 100644
index 0000000..7be8782
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/configs/background/README.md
@@ -0,0 +1,73 @@
+# Matting Effects
+
+We can apply background matting to the videos.
+
+## Instruction
+
+### Get started
+
+Launch the demo from the mmpose root directory:
+
+```shell
+python tools/webcam/run_webcam.py --config tools/webcam/configs/background/background.py
+```
+
+### Hotkeys
+
+| Hotkey | Function |
+| -- | -- |
+| b | Toggle the background matting effect on/off. |
+| h | Show help information. |
+| m | Show the monitoring information. |
+| q | Exit. |
+
+Note that the demo will automatically save the output video into a file `record.mp4`.
+
+### Configuration
+
+- **Choose a detection model**
+
+Users can choose detection models from the [MMDetection Model Zoo](https://mmdetection.readthedocs.io/en/v2.20.0/model_zoo.html). Just set the `model_config` and `model_checkpoint` in the detector node accordingly, and the model will be automatically downloaded and loaded.
+Note that in order to perform background matting, the model should be able to produce segmentation masks.
+
+```python
+# 'DetectorNode':
+# This node performs object detection from the frame image using an
+# MMDetection model.
+dict(
+ type='DetectorNode',
+ name='Detector',
+ model_config='demo/mmdetection_cfg/mask_rcnn_r50_fpn_2x_coco.py',
+ model_checkpoint='https://download.openmmlab.com/'
+ 'mmdetection/v2.0/mask_rcnn/mask_rcnn_r50_fpn_2x_coco/'
+ 'mask_rcnn_r50_fpn_2x_coco_bbox_mAP-0.392'
+ '__segm_mAP-0.354_20200505_003907-3e542a40.pth',
+ input_buffer='_input_', # `_input_` is a runner-reserved buffer
+ output_buffer='det_result'),
+```
+
+- **Run the demo without GPU**
+
+If you don't have GPU and CUDA in your device, the demo can run with only CPU by setting `device='cpu'` in all model nodes. For example:
+
+```python
+dict(
+ type='DetectorNode',
+ name='Detector',
+ model_config='demo/mmdetection_cfg/mask_rcnn_r50_fpn_2x_coco.py',
+ model_checkpoint='https://download.openmmlab.com/'
+ 'mmdetection/v2.0/mask_rcnn/mask_rcnn_r50_fpn_2x_coco/'
+ 'mask_rcnn_r50_fpn_2x_coco_bbox_mAP-0.392'
+ '__segm_mAP-0.354_20200505_003907-3e542a40.pth',
+ device='cpu',
+ input_buffer='_input_', # `_input_` is a runner-reserved buffer
+ output_buffer='det_result'),
+```
+
+- **Debug webcam and display**
+
+You can launch the webcam runner with a debug config:
+
+```shell
+python tools/webcam/run_webcam.py --config tools/webcam/configs/examples/test_camera.py
+```
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/configs/background/background.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/configs/background/background.py
new file mode 100644
index 0000000..fb9f4d6
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/configs/background/background.py
@@ -0,0 +1,93 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+runner = dict(
+ # Basic configurations of the runner
+ name='Matting Effects',
+ camera_id=0,
+ camera_fps=10,
+ synchronous=False,
+ # Define nodes.
+ # The configuration of a node usually includes:
+ # 1. 'type': Node class name
+ # 2. 'name': Node name
+ # 3. I/O buffers (e.g. 'input_buffer', 'output_buffer'): specify the
+ # input and output buffer names. This may depend on the node class.
+ # 4. 'enable_key': assign a hot-key to toggle enable/disable this node.
+ # This may depend on the node class.
+ # 5. Other class-specific arguments
+ nodes=[
+ # 'DetectorNode':
+ # This node performs object detection from the frame image using an
+ # MMDetection model.
+ dict(
+ type='DetectorNode',
+ name='Detector',
+ model_config='demo/mmdetection_cfg/mask_rcnn_r50_fpn_2x_coco.py',
+ model_checkpoint='https://download.openmmlab.com/'
+ 'mmdetection/v2.0/mask_rcnn/mask_rcnn_r50_fpn_2x_coco/'
+ 'mask_rcnn_r50_fpn_2x_coco_bbox_mAP-0.392'
+ '__segm_mAP-0.354_20200505_003907-3e542a40.pth',
+ input_buffer='_input_', # `_input_` is a runner-reserved buffer
+ output_buffer='det_result'),
+ # 'TopDownPoseEstimatorNode':
+ # This node performs keypoint detection from the frame image using an
+ # MMPose top-down model. Detection results is needed.
+ dict(
+ type='TopDownPoseEstimatorNode',
+ name='Human Pose Estimator',
+ model_config='configs/wholebody/2d_kpt_sview_rgb_img/'
+ 'topdown_heatmap/coco-wholebody/'
+ 'vipnas_mbv3_coco_wholebody_256x192_dark.py',
+ model_checkpoint='https://openmmlab-share.oss-cn-hangz'
+ 'hou.aliyuncs.com/mmpose/top_down/vipnas/vipnas_mbv3_co'
+ 'co_wholebody_256x192_dark-e2158108_20211205.pth',
+ cls_names=['person'],
+ input_buffer='det_result',
+ output_buffer='human_pose'),
+ # 'ModelResultBindingNode':
+ # This node binds the latest model inference result with the current
+ # frame. (This means the frame image and inference result may be
+ # asynchronous).
+ dict(
+ type='ModelResultBindingNode',
+ name='ResultBinder',
+ frame_buffer='_frame_', # `_frame_` is a runner-reserved buffer
+ result_buffer='human_pose',
+ output_buffer='frame'),
+ # 'MattingNode':
+ # This node draw the matting visualization result in the frame image.
+ # mask results is needed.
+ dict(
+ type='BackgroundNode',
+ name='Visualizer',
+ enable_key='b',
+ enable=True,
+ frame_buffer='frame',
+ output_buffer='vis_bg',
+ cls_names=['person']),
+ # 'NoticeBoardNode':
+ # This node show a notice board with given content, e.g. help
+ # information.
+ dict(
+ type='NoticeBoardNode',
+ name='Helper',
+ enable_key='h',
+ frame_buffer='vis_bg',
+ output_buffer='vis',
+ content_lines=[
+ 'This is a demo for background changing effects. Have fun!',
+ '', 'Hot-keys:', '"b": Change background',
+ '"h": Show help information',
+ '"m": Show diagnostic information', '"q": Exit'
+ ],
+ ),
+ # 'MonitorNode':
+ # This node show diagnostic information in the frame image. It can
+ # be used for debugging or monitoring system resource status.
+ dict(
+ type='MonitorNode',
+ name='Monitor',
+ enable_key='m',
+ enable=False,
+ frame_buffer='vis',
+ output_buffer='_display_') # `_frame_` is a runner-reserved buffer
+ ])
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/configs/examples/README.md b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/configs/examples/README.md
new file mode 100644
index 0000000..ec9b961
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/configs/examples/README.md
@@ -0,0 +1,110 @@
+# Pose Estimation Demo
+
+This demo performs human bounding box and keypoint detection, and visualizes results.
+
+
+
+
+
+## Instruction
+
+### Get started
+
+Launch the demo from the mmpose root directory:
+
+```shell
+python tools/webcam/run_webcam.py --config tools/webcam/configs/examples/pose_estimation.py
+```
+
+### Hotkeys
+
+| Hotkey | Function |
+| -- | -- |
+| v | Toggle the pose visualization on/off. |
+| h | Show help information. |
+| m | Show the monitoring information. |
+| q | Exit. |
+
+Note that the demo will automatically save the output video into a file `record.mp4`.
+
+### Configuration
+
+- **Choose a detection model**
+
+Users can choose detection models from the [MMDetection Model Zoo](https://mmdetection.readthedocs.io/en/v2.20.0/model_zoo.html). Just set the `model_config` and `model_checkpoint` in the detector node accordingly, and the model will be automatically downloaded and loaded.
+
+```python
+# 'DetectorNode':
+ # This node performs object detection from the frame image using an
+ # MMDetection model.
+dict(
+ type='DetectorNode',
+ name='Detector',
+ model_config='demo/mmdetection_cfg/'
+ 'ssdlite_mobilenetv2_scratch_600e_coco.py',
+ model_checkpoint='https://download.openmmlab.com'
+ '/mmdetection/v2.0/ssd/'
+ 'ssdlite_mobilenetv2_scratch_600e_coco/ssdlite_mobilenetv2_'
+ 'scratch_600e_coco_20210629_110627-974d9307.pth',
+ input_buffer='_input_',
+ output_buffer='det_result')
+```
+
+- **Choose a or more pose models**
+
+In this demo we use two [top-down](https://github.com/open-mmlab/mmpose/tree/master/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap) pose estimation models for humans and animals respectively. Users can choose models from the [MMPose Model Zoo](https://mmpose.readthedocs.io/en/latest/modelzoo.html). To apply different pose models on different instance types, you can add multiple pose estimator nodes with `cls_names` set accordingly.
+
+```python
+# 'TopDownPoseEstimatorNode':
+# This node performs keypoint detection from the frame image using an
+# MMPose top-down model. Detection results is needed.
+dict(
+ type='TopDownPoseEstimatorNode',
+ name='Human Pose Estimator',
+ model_config='configs/wholebody/2d_kpt_sview_rgb_img/'
+ 'topdown_heatmap/coco-wholebody/'
+ 'vipnas_mbv3_coco_wholebody_256x192_dark.py',
+ model_checkpoint='https://openmmlab-share.oss-cn-hangz'
+ 'hou.aliyuncs.com/mmpose/top_down/vipnas/vipnas_mbv3_co'
+ 'co_wholebody_256x192_dark-e2158108_20211205.pth',
+ cls_names=['person'],
+ input_buffer='det_result',
+ output_buffer='human_pose'),
+dict(
+ type='TopDownPoseEstimatorNode',
+ name='Animal Pose Estimator',
+ model_config='configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap'
+ '/animalpose/hrnet_w32_animalpose_256x256.py',
+ model_checkpoint='https://download.openmmlab.com/mmpose/animal/'
+ 'hrnet/hrnet_w32_animalpose_256x256-1aa7f075_20210426.pth',
+ cls_names=['cat', 'dog', 'horse', 'sheep', 'cow'],
+ input_buffer='human_pose',
+ output_buffer='animal_pose')
+```
+
+- **Run the demo without GPU**
+
+If you don't have GPU and CUDA in your device, the demo can run with only CPU by setting `device='cpu'` in all model nodes. For example:
+
+```python
+dict(
+ type='DetectorNode',
+ name='Detector',
+ model_config='demo/mmdetection_cfg/'
+ 'ssdlite_mobilenetv2_scratch_600e_coco.py',
+ model_checkpoint='https://download.openmmlab.com'
+ '/mmdetection/v2.0/ssd/'
+ 'ssdlite_mobilenetv2_scratch_600e_coco/ssdlite_mobilenetv2_'
+ 'scratch_600e_coco_20210629_110627-974d9307.pth',
+ device='cpu',
+ input_buffer='_input_',
+ output_buffer='det_result')
+```
+
+- **Debug webcam and display**
+
+You can lanch the webcam runner with a debug config:
+
+```shell
+python tools/webcam/run_webcam.py --config tools/webcam/configs/examples/test_camera.py
+```
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/configs/examples/pose_estimation.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/configs/examples/pose_estimation.py
new file mode 100644
index 0000000..471333a
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/configs/examples/pose_estimation.py
@@ -0,0 +1,115 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+runner = dict(
+ # Basic configurations of the runner
+ name='Pose Estimation',
+ camera_id=0,
+ camera_fps=20,
+ synchronous=False,
+ # Define nodes.
+ # The configuration of a node usually includes:
+ # 1. 'type': Node class name
+ # 2. 'name': Node name
+ # 3. I/O buffers (e.g. 'input_buffer', 'output_buffer'): specify the
+ # input and output buffer names. This may depend on the node class.
+ # 4. 'enable_key': assign a hot-key to toggle enable/disable this node.
+ # This may depend on the node class.
+ # 5. Other class-specific arguments
+ nodes=[
+ # 'DetectorNode':
+ # This node performs object detection from the frame image using an
+ # MMDetection model.
+ dict(
+ type='DetectorNode',
+ name='Detector',
+ model_config='demo/mmdetection_cfg/'
+ 'ssdlite_mobilenetv2_scratch_600e_coco.py',
+ model_checkpoint='https://download.openmmlab.com'
+ '/mmdetection/v2.0/ssd/'
+ 'ssdlite_mobilenetv2_scratch_600e_coco/ssdlite_mobilenetv2_'
+ 'scratch_600e_coco_20210629_110627-974d9307.pth',
+ input_buffer='_input_', # `_input_` is a runner-reserved buffer
+ output_buffer='det_result'),
+ # 'TopDownPoseEstimatorNode':
+ # This node performs keypoint detection from the frame image using an
+ # MMPose top-down model. Detection results is needed.
+ dict(
+ type='TopDownPoseEstimatorNode',
+ name='Human Pose Estimator',
+ model_config='configs/wholebody/2d_kpt_sview_rgb_img/'
+ 'topdown_heatmap/coco-wholebody/'
+ 'vipnas_mbv3_coco_wholebody_256x192_dark.py',
+ model_checkpoint='https://download.openmmlab.com/mmpose/top_down/'
+ 'vipnas/vipnas_mbv3_coco_wholebody_256x192_dark'
+ '-e2158108_20211205.pth',
+ cls_names=['person'],
+ input_buffer='det_result',
+ output_buffer='human_pose'),
+ dict(
+ type='TopDownPoseEstimatorNode',
+ name='Animal Pose Estimator',
+ model_config='configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap'
+ '/animalpose/hrnet_w32_animalpose_256x256.py',
+ model_checkpoint='https://download.openmmlab.com/mmpose/animal/'
+ 'hrnet/hrnet_w32_animalpose_256x256-1aa7f075_20210426.pth',
+ cls_names=['cat', 'dog', 'horse', 'sheep', 'cow'],
+ input_buffer='human_pose',
+ output_buffer='animal_pose'),
+ # 'ModelResultBindingNode':
+ # This node binds the latest model inference result with the current
+ # frame. (This means the frame image and inference result may be
+ # asynchronous).
+ dict(
+ type='ModelResultBindingNode',
+ name='ResultBinder',
+ frame_buffer='_frame_', # `_frame_` is a runner-reserved buffer
+ result_buffer='animal_pose',
+ output_buffer='frame'),
+ # 'PoseVisualizerNode':
+ # This node draw the pose visualization result in the frame image.
+ # Pose results is needed.
+ dict(
+ type='PoseVisualizerNode',
+ name='Visualizer',
+ enable_key='v',
+ frame_buffer='frame',
+ output_buffer='vis'),
+ # 'NoticeBoardNode':
+ # This node show a notice board with given content, e.g. help
+ # information.
+ dict(
+ type='NoticeBoardNode',
+ name='Helper',
+ enable_key='h',
+ enable=True,
+ frame_buffer='vis',
+ output_buffer='vis_notice',
+ content_lines=[
+ 'This is a demo for pose visualization and simple image '
+ 'effects. Have fun!', '', 'Hot-keys:',
+ '"v": Pose estimation result visualization',
+ '"s": Sunglasses effect B-)', '"b": Bug-eye effect 0_0',
+ '"h": Show help information',
+ '"m": Show diagnostic information', '"q": Exit'
+ ],
+ ),
+ # 'MonitorNode':
+ # This node show diagnostic information in the frame image. It can
+ # be used for debugging or monitoring system resource status.
+ dict(
+ type='MonitorNode',
+ name='Monitor',
+ enable_key='m',
+ enable=False,
+ frame_buffer='vis_notice',
+ output_buffer='display'),
+ # 'RecorderNode':
+ # This node save the output video into a file.
+ dict(
+ type='RecorderNode',
+ name='Recorder',
+ out_video_file='record.mp4',
+ frame_buffer='display',
+ output_buffer='_display_'
+ # `_display_` is a runner-reserved buffer
+ )
+ ])
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/configs/examples/test_camera.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/configs/examples/test_camera.py
new file mode 100644
index 0000000..c0c1677
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/configs/examples/test_camera.py
@@ -0,0 +1,19 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+runner = dict(
+ name='Debug CamRunner',
+ camera_id=0,
+ camera_fps=20,
+ nodes=[
+ dict(
+ type='MonitorNode',
+ name='Monitor',
+ enable_key='m',
+ frame_buffer='_frame_',
+ output_buffer='display'),
+ dict(
+ type='RecorderNode',
+ name='Recorder',
+ out_video_file='webcam_output.mp4',
+ frame_buffer='display',
+ output_buffer='_display_')
+ ])
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/configs/eyes/README.md b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/configs/eyes/README.md
new file mode 100644
index 0000000..f9c3769
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/configs/eyes/README.md
@@ -0,0 +1,31 @@
+# Sunglasses and Bug-eye Effects
+
+We can apply fun effects on videos with pose estimation results, like adding sunglasses on the face, or make the eyes look bigger.
+
+
+
+
+
+## Instruction
+
+### Get started
+
+Launch the demo from the mmpose root directory:
+
+```shell
+python tools/webcam/run_webcam.py --config tools/webcam/configs/examples/pose_estimation.py
+```
+
+### Hotkeys
+
+| Hotkey | Function |
+| -- | -- |
+| s | Toggle the sunglasses effect on/off. |
+| b | Toggle the bug-eye effect on/off. |
+| h | Show help information. |
+| m | Show the monitoring information. |
+| q | Exit. |
+
+### Configuration
+
+See the [README](/tools/webcam/configs/examples/README.md#configuration) of pose estimation demo for model configurations.
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/configs/eyes/eyes.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/configs/eyes/eyes.py
new file mode 100644
index 0000000..91bbfba
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/configs/eyes/eyes.py
@@ -0,0 +1,114 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+runner = dict(
+ # Basic configurations of the runner
+ name='Eye Effects',
+ camera_id=0,
+ camera_fps=20,
+ synchronous=False,
+ # Define nodes.
+ # The configuration of a node usually includes:
+ # 1. 'type': Node class name
+ # 2. 'name': Node name
+ # 3. I/O buffers (e.g. 'input_buffer', 'output_buffer'): specify the
+ # input and output buffer names. This may depend on the node class.
+ # 4. 'enable_key': assign a hot-key to toggle enable/disable this node.
+ # This may depend on the node class.
+ # 5. Other class-specific arguments
+ nodes=[
+ # 'DetectorNode':
+ # This node performs object detection from the frame image using an
+ # MMDetection model.
+ dict(
+ type='DetectorNode',
+ name='Detector',
+ model_config='demo/mmdetection_cfg/'
+ 'ssdlite_mobilenetv2_scratch_600e_coco.py',
+ model_checkpoint='https://download.openmmlab.com'
+ '/mmdetection/v2.0/ssd/'
+ 'ssdlite_mobilenetv2_scratch_600e_coco/ssdlite_mobilenetv2_'
+ 'scratch_600e_coco_20210629_110627-974d9307.pth',
+ input_buffer='_input_', # `_input_` is a runner-reserved buffer
+ output_buffer='det_result'),
+ # 'TopDownPoseEstimatorNode':
+ # This node performs keypoint detection from the frame image using an
+ # MMPose top-down model. Detection results is needed.
+ dict(
+ type='TopDownPoseEstimatorNode',
+ name='Human Pose Estimator',
+ model_config='configs/wholebody/2d_kpt_sview_rgb_img/'
+ 'topdown_heatmap/coco-wholebody/'
+ 'vipnas_mbv3_coco_wholebody_256x192_dark.py',
+ model_checkpoint='https://openmmlab-share.oss-cn-hangz'
+ 'hou.aliyuncs.com/mmpose/top_down/vipnas/vipnas_mbv3_co'
+ 'co_wholebody_256x192_dark-e2158108_20211205.pth',
+ cls_names=['person'],
+ input_buffer='det_result',
+ output_buffer='human_pose'),
+ dict(
+ type='TopDownPoseEstimatorNode',
+ name='Animal Pose Estimator',
+ model_config='configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap'
+ '/animalpose/hrnet_w32_animalpose_256x256.py',
+ model_checkpoint='https://download.openmmlab.com/mmpose/animal/'
+ 'hrnet/hrnet_w32_animalpose_256x256-1aa7f075_20210426.pth',
+ cls_names=['cat', 'dog', 'horse', 'sheep', 'cow'],
+ input_buffer='human_pose',
+ output_buffer='animal_pose'),
+ # 'ModelResultBindingNode':
+ # This node binds the latest model inference result with the current
+ # frame. (This means the frame image and inference result may be
+ # asynchronous).
+ dict(
+ type='ModelResultBindingNode',
+ name='ResultBinder',
+ frame_buffer='_frame_', # `_frame_` is a runner-reserved buffer
+ result_buffer='animal_pose',
+ output_buffer='frame'),
+ # 'SunglassesNode':
+ # This node draw the sunglasses effect in the frame image.
+ # Pose results is needed.
+ dict(
+ type='SunglassesNode',
+ name='Visualizer',
+ enable_key='s',
+ enable=True,
+ frame_buffer='frame',
+ output_buffer='vis_sunglasses'),
+ # 'BugEyeNode':
+ # This node draw the bug-eye effetc in the frame image.
+ # Pose results is needed.
+ dict(
+ type='BugEyeNode',
+ name='Visualizer',
+ enable_key='b',
+ enable=False,
+ frame_buffer='vis_sunglasses',
+ output_buffer='vis_bugeye'),
+ # 'NoticeBoardNode':
+ # This node show a notice board with given content, e.g. help
+ # information.
+ dict(
+ type='NoticeBoardNode',
+ name='Helper',
+ enable_key='h',
+ frame_buffer='vis_bugeye',
+ output_buffer='vis',
+ content_lines=[
+ 'This is a demo for pose visualization and simple image '
+ 'effects. Have fun!', '', 'Hot-keys:',
+ '"s": Sunglasses effect B-)', '"b": Bug-eye effect 0_0',
+ '"h": Show help information',
+ '"m": Show diagnostic information', '"q": Exit'
+ ],
+ ),
+ # 'MonitorNode':
+ # This node show diagnostic information in the frame image. It can
+ # be used for debugging or monitoring system resource status.
+ dict(
+ type='MonitorNode',
+ name='Monitor',
+ enable_key='m',
+ enable=False,
+ frame_buffer='vis',
+ output_buffer='_display_') # `_frame_` is a runner-reserved buffer
+ ])
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/configs/face_swap/README.md b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/configs/face_swap/README.md
new file mode 100644
index 0000000..02f4c8a
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/configs/face_swap/README.md
@@ -0,0 +1,31 @@
+# Sunglasses and Bug-eye Effects
+
+Look! Where is my face?:eyes: And whose face is it?:laughing:
+
+
+
+
+
+## Instruction
+
+### Get started
+
+Launch the demo from the mmpose root directory:
+
+```shell
+python tools/webcam/run_webcam.py --config tools/webcam/configs/face_swap/face_swap.py
+```
+
+### Hotkeys
+
+| Hotkey | Function |
+| -- | -- |
+| s | Switch between modes
Shuffle: Randomly shuffle all faces
Clone: Choose one face and clone it for everyone
None: Nothing happens and everyone is safe :)
|
+| v | Toggle the pose visualization on/off. |
+| h | Show help information. |
+| m | Show diagnostic information. |
+| q | Exit. |
+
+### Configuration
+
+See the [README](/tools/webcam/configs/examples/README.md#configuration) of pose estimation demo for model configurations.
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/configs/face_swap/face_swap.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/configs/face_swap/face_swap.py
new file mode 100644
index 0000000..403eaae
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/configs/face_swap/face_swap.py
@@ -0,0 +1,79 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+runner = dict(
+ name='FaceSwap',
+ camera_id=0,
+ camera_fps=20,
+ synchronous=False,
+ nodes=[
+ dict(
+ type='DetectorNode',
+ name='Detector',
+ model_config='demo/mmdetection_cfg/'
+ 'ssdlite_mobilenetv2_scratch_600e_coco.py',
+ model_checkpoint='https://download.openmmlab.com'
+ '/mmdetection/v2.0/ssd/'
+ 'ssdlite_mobilenetv2_scratch_600e_coco/ssdlite_mobilenetv2_'
+ 'scratch_600e_coco_20210629_110627-974d9307.pth',
+ device='cpu',
+ input_buffer='_input_', # `_input_` is a runner-reserved buffer
+ output_buffer='det_result'),
+ dict(
+ type='TopDownPoseEstimatorNode',
+ name='TopDown Pose Estimator',
+ model_config='configs/wholebody/2d_kpt_sview_rgb_img/'
+ 'topdown_heatmap/coco-wholebody/'
+ 'vipnas_res50_coco_wholebody_256x192_dark.py',
+ model_checkpoint='https://openmmlab-share.oss-cn-hangzhou'
+ '.aliyuncs.com/mmpose/top_down/vipnas/'
+ 'vipnas_res50_wholebody_256x192_dark-67c0ce35_20211112.pth',
+ device='cpu',
+ cls_names=['person'],
+ input_buffer='det_result',
+ output_buffer='pose_result'),
+ dict(
+ type='ModelResultBindingNode',
+ name='ResultBinder',
+ frame_buffer='_frame_', # `_frame_` is a runner-reserved buffer
+ result_buffer='pose_result',
+ output_buffer='frame'),
+ dict(
+ type='FaceSwapNode',
+ name='FaceSwapper',
+ mode_key='s',
+ frame_buffer='frame',
+ output_buffer='face_swap'),
+ dict(
+ type='PoseVisualizerNode',
+ name='Visualizer',
+ enable_key='v',
+ frame_buffer='face_swap',
+ output_buffer='vis_pose'),
+ dict(
+ type='NoticeBoardNode',
+ name='Help Information',
+ enable_key='h',
+ content_lines=[
+ 'Swap your faces! ',
+ 'Hot-keys:',
+ '"v": Toggle the pose visualization on/off.',
+ '"s": Switch between modes: Shuffle, Clone and None',
+ '"h": Show help information',
+ '"m": Show diagnostic information',
+ '"q": Exit',
+ ],
+ frame_buffer='vis_pose',
+ output_buffer='vis_notice'),
+ dict(
+ type='MonitorNode',
+ name='Monitor',
+ enable_key='m',
+ enable=False,
+ frame_buffer='vis_notice',
+ output_buffer='display'),
+ dict(
+ type='RecorderNode',
+ name='Recorder',
+ out_video_file='faceswap_output.mp4',
+ frame_buffer='display',
+ output_buffer='_display_')
+ ])
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/configs/meow_dwen_dwen/README.md b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/configs/meow_dwen_dwen/README.md
new file mode 100644
index 0000000..997ffc1
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/configs/meow_dwen_dwen/README.md
@@ -0,0 +1,44 @@
+# Meow Dwen Dwen
+
+Do you know [Bing DwenDwen (冰墩墩)](https://en.wikipedia.org/wiki/Bing_Dwen_Dwen_and_Shuey_Rhon_Rhon), the mascot of 2022 Beijing Olympic Games?
+
+
+
+
+
+Now you can dress your cat up in this costume and TA-DA! Be prepared for super cute **Meow Dwen Dwen**.
+
+
+
+
+
+You are a dog fan? Hold on, here comes Woof Dwen Dwen.
+
+
+
+
+
+## Instruction
+
+### Get started
+
+Launch the demo from the mmpose root directory:
+
+```shell
+python tools/webcam/run_webcam.py --config tools/webcam/configs/meow_dwen_dwen/meow_dwen_dwen.py
+```
+
+### Hotkeys
+
+| Hotkey | Function |
+| -- | -- |
+| s | Change the background. |
+| h | Show help information. |
+| m | Show diagnostic information. |
+| q | Exit. |
+
+### Configuration
+
+- **Use video input**
+
+As you can see in the config, we set `camera_id` as the path of the input image. You can also set it as a video file path (or url), or a webcam ID number (e.g. `camera_id=0`), to capture the dynamic face from the video input.
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/configs/meow_dwen_dwen/meow_dwen_dwen.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/configs/meow_dwen_dwen/meow_dwen_dwen.py
new file mode 100644
index 0000000..399d01c
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/configs/meow_dwen_dwen/meow_dwen_dwen.py
@@ -0,0 +1,92 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+runner = dict(
+ # Basic configurations of the runner
+ name='Little fans of 2022 Beijing Winter Olympics',
+ # Cat image
+ camera_id='https://user-images.githubusercontent.com/'
+ '15977946/152932036-b5554cf8-24cf-40d6-a358-35a106013f11.jpeg',
+ # Dog image
+ # camera_id='https://user-images.githubusercontent.com/'
+ # '15977946/152932051-cd280b35-8066-45a0-8f52-657c8631aaba.jpg',
+ camera_fps=20,
+ nodes=[
+ dict(
+ type='DetectorNode',
+ name='Detector',
+ model_config='demo/mmdetection_cfg/'
+ 'ssdlite_mobilenetv2_scratch_600e_coco.py',
+ model_checkpoint='https://download.openmmlab.com'
+ '/mmdetection/v2.0/ssd/'
+ 'ssdlite_mobilenetv2_scratch_600e_coco/ssdlite_mobilenetv2_'
+ 'scratch_600e_coco_20210629_110627-974d9307.pth',
+ input_buffer='_input_', # `_input_` is a runner-reserved buffer
+ output_buffer='det_result'),
+ dict(
+ type='TopDownPoseEstimatorNode',
+ name='Animal Pose Estimator',
+ model_config='configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap'
+ '/ap10k/hrnet_w32_ap10k_256x256.py',
+ model_checkpoint='https://download.openmmlab.com/mmpose/animal/'
+ 'hrnet/hrnet_w32_ap10k_256x256-18aac840_20211029.pth',
+ cls_names=['cat', 'dog'],
+ input_buffer='det_result',
+ output_buffer='animal_pose'),
+ dict(
+ type='TopDownPoseEstimatorNode',
+ name='TopDown Pose Estimator',
+ model_config='configs/wholebody/2d_kpt_sview_rgb_img/'
+ 'topdown_heatmap/coco-wholebody/'
+ 'vipnas_res50_coco_wholebody_256x192_dark.py',
+ model_checkpoint='https://openmmlab-share.oss-cn-hangzhou'
+ '.aliyuncs.com/mmpose/top_down/vipnas/'
+ 'vipnas_res50_wholebody_256x192_dark-67c0ce35_20211112.pth',
+ device='cpu',
+ cls_names=['person'],
+ input_buffer='animal_pose',
+ output_buffer='human_pose'),
+ dict(
+ type='ModelResultBindingNode',
+ name='ResultBinder',
+ frame_buffer='_frame_', # `_frame_` is a runner-reserved buffer
+ result_buffer='human_pose',
+ output_buffer='frame'),
+ dict(
+ type='XDwenDwenNode',
+ name='XDwenDwen',
+ mode_key='s',
+ resource_file='tools/webcam/configs/meow_dwen_dwen/'
+ 'resource-info.json',
+ out_shape=(480, 480),
+ frame_buffer='frame',
+ output_buffer='vis'),
+ dict(
+ type='NoticeBoardNode',
+ name='Helper',
+ enable_key='h',
+ enable=False,
+ frame_buffer='vis',
+ output_buffer='vis_notice',
+ content_lines=[
+ 'Let your pet put on a costume of Bing-Dwen-Dwen, '
+ 'the mascot of 2022 Beijing Winter Olympics. Have fun!', '',
+ 'Hot-keys:', '"s": Change the background',
+ '"h": Show help information',
+ '"m": Show diagnostic information', '"q": Exit'
+ ],
+ ),
+ dict(
+ type='MonitorNode',
+ name='Monitor',
+ enable_key='m',
+ enable=False,
+ frame_buffer='vis_notice',
+ output_buffer='display'),
+ dict(
+ type='RecorderNode',
+ name='Recorder',
+ out_video_file='record.mp4',
+ frame_buffer='display',
+ output_buffer='_display_'
+ # `_display_` is a runner-reserved buffer
+ )
+ ])
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/configs/meow_dwen_dwen/resource-info.json b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/configs/meow_dwen_dwen/resource-info.json
new file mode 100644
index 0000000..adb811c
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/configs/meow_dwen_dwen/resource-info.json
@@ -0,0 +1,26 @@
+[
+ {
+ "id": 1,
+ "result": "{\"width\":690,\"height\":713,\"valid\":true,\"rotate\":0,\"step_1\":{\"toolName\":\"pointTool\",\"result\":[{\"x\":374.86387434554973,\"y\":262.8020942408377,\"attribute\":\"\",\"valid\":true,\"id\":\"8SK9cVyu\",\"sourceID\":\"\",\"textAttribute\":\"\",\"order\":2},{\"x\":492.8261780104712,\"y\":285.2,\"attribute\":\"\",\"valid\":true,\"id\":\"qDk54WsI\",\"sourceID\":\"\",\"textAttribute\":\"\",\"order\":1},{\"x\":430.11204188481673,\"y\":318.0502617801047,\"attribute\":\"\",\"valid\":true,\"id\":\"4H80L7lL\",\"sourceID\":\"\",\"textAttribute\":\"\",\"order\":3}]},\"step_2\":{\"dataSourceStep\":0,\"toolName\":\"polygonTool\",\"result\":[{\"id\":\"pwUsrf9u\",\"sourceID\":\"\",\"valid\":true,\"textAttribute\":\"\",\"pointList\":[{\"x\":423.3926701570681,\"y\":191.87539267015708},{\"x\":488.3465968586388,\"y\":209.04712041884818},{\"x\":535.3821989528797,\"y\":248.6167539267016},{\"x\":549.5675392670157,\"y\":306.8513089005236},{\"x\":537.6219895287959,\"y\":349.407329842932},{\"x\":510.74450261780106,\"y\":381.51099476439794},{\"x\":480.1340314136126,\"y\":394.9497382198953},{\"x\":411.4471204188482,\"y\":390.47015706806286},{\"x\":355.45235602094243,\"y\":373.29842931937173},{\"x\":306.17696335078534,\"y\":327.00942408376966},{\"x\":294.97801047120424,\"y\":284.45340314136126},{\"x\":306.9235602094241,\"y\":245.6303664921466},{\"x\":333.8010471204189,\"y\":217.25968586387435},{\"x\":370.3842931937173,\"y\":196.35497382198955}],\"attribute\":\"\",\"order\":1}]}}",
+ "url": "https://user-images.githubusercontent.com/15977946/152742677-35fe8a01-bd06-4a12-a02e-949e7d71f28a.jpg",
+ "fileName": "bing_dwen_dwen1.jpg"
+ },
+ {
+ "id": 2,
+ "result": "{\"width\":690,\"height\":659,\"valid\":true,\"rotate\":0,\"step_1\":{\"dataSourceStep\":0,\"toolName\":\"pointTool\",\"result\":[{\"x\":293.2460732984293,\"y\":242.89842931937173,\"attribute\":\"\",\"valid\":true,\"id\":\"KgPs39bY\",\"sourceID\":\"\",\"textAttribute\":\"\",\"order\":1},{\"x\":170.41675392670155,\"y\":270.50052356020944,\"attribute\":\"\",\"valid\":true,\"id\":\"XwHyoBFU\",\"sourceID\":\"\",\"textAttribute\":\"\",\"order\":2},{\"x\":224.24083769633506,\"y\":308.45340314136126,\"attribute\":\"\",\"valid\":true,\"id\":\"Qfs4YfuB\",\"sourceID\":\"\",\"textAttribute\":\"\",\"order\":3}]},\"step_2\":{\"dataSourceStep\":0,\"toolName\":\"polygonTool\",\"result\":[{\"id\":\"ts5jlJxb\",\"sourceID\":\"\",\"valid\":true,\"textAttribute\":\"\",\"pointList\":[{\"x\":178.69738219895285,\"y\":184.93403141361256},{\"x\":204.91937172774865,\"y\":172.5130890052356},{\"x\":252.5329842931937,\"y\":169.0628272251309},{\"x\":295.3162303664921,\"y\":175.27329842931937},{\"x\":333.95916230366487,\"y\":195.2848167539267},{\"x\":360.18115183246067,\"y\":220.1267015706806},{\"x\":376.0523560209424,\"y\":262.909947643979},{\"x\":373.98219895287957,\"y\":296.0324607329843},{\"x\":344.99999999999994,\"y\":335.365445026178},{\"x\":322.22827225130885,\"y\":355.37696335078533},{\"x\":272.544502617801,\"y\":378.1486910994764},{\"x\":221.48062827225127,\"y\":386.42931937172773},{\"x\":187.6680628272251,\"y\":385.7392670157068},{\"x\":158.68586387434553,\"y\":369.1780104712042},{\"x\":137.98429319371724,\"y\":337.43560209424083},{\"x\":127.63350785340312,\"y\":295.34240837696336},{\"x\":131.0837696335078,\"y\":242.89842931937173},{\"x\":147.64502617801045,\"y\":208.3958115183246}],\"attribute\":\"\",\"order\":1}]}}",
+ "url": "https://user-images.githubusercontent.com/15977946/152742707-c0c51844-e1d0-42d0-9a12-e369002e082f.jpg",
+ "fileName": "bing_dwen_dwen2.jpg"
+ },
+ {
+ "id": 3,
+ "result": "{\"width\":690,\"height\":811,\"valid\":true,\"rotate\":0,\"step_1\":{\"dataSourceStep\":0,\"toolName\":\"pointTool\",\"result\":[{\"x\":361.13507853403144,\"y\":300.62198952879584,\"attribute\":\"\",\"valid\":true,\"id\":\"uAtbXtf2\",\"sourceID\":\"\",\"textAttribute\":\"\",\"order\":1},{\"x\":242.24502617801048,\"y\":317.60628272251313,\"attribute\":\"\",\"valid\":true,\"id\":\"iLtceHMA\",\"sourceID\":\"\",\"textAttribute\":\"\",\"order\":2},{\"x\":302.5392670157068,\"y\":356.67015706806285,\"attribute\":\"\",\"valid\":true,\"id\":\"n9MTlJ6A\",\"sourceID\":\"\",\"textAttribute\":\"\",\"order\":3}]},\"step_2\":{\"dataSourceStep\":0,\"toolName\":\"polygonTool\",\"result\":[{\"id\":\"5sTLU5wF\",\"sourceID\":\"\",\"valid\":true,\"textAttribute\":\"\",\"pointList\":[{\"x\":227.80837696335078,\"y\":247.12146596858642},{\"x\":248.18952879581153,\"y\":235.23246073298432},{\"x\":291.4994764397906,\"y\":225.04188481675394},{\"x\":351.7937172774869,\"y\":229.28795811518327},{\"x\":393.40523560209425,\"y\":245.42303664921468},{\"x\":424.8261780104712,\"y\":272.59790575916236},{\"x\":443.5089005235602,\"y\":298.07434554973827},{\"x\":436.7151832460733,\"y\":345.6303664921466},{\"x\":406.1434554973822,\"y\":382.9958115183247},{\"x\":355.1905759162304,\"y\":408.4722513089006},{\"x\":313.57905759162304,\"y\":419.5120418848168},{\"x\":262.6261780104712,\"y\":417.81361256544506},{\"x\":224.41151832460733,\"y\":399.9801047120419},{\"x\":201.48272251308902,\"y\":364.3130890052356},{\"x\":194.68900523560208,\"y\":315.0586387434555},{\"x\":202.33193717277487,\"y\":272.59790575916236}],\"attribute\":\"\",\"order\":1}]}}",
+ "url": "https://user-images.githubusercontent.com/15977946/152742728-99392ecf-8f5c-46cf-b5c4-fe7fb6b39976.jpg",
+ "fileName": "bing_dwen_dwen3.jpg"
+ },
+ {
+ "id": 4,
+ "result": "{\"width\":690,\"height\":690,\"valid\":true,\"rotate\":0,\"step_1\":{\"dataSourceStep\":0,\"toolName\":\"pointTool\",\"result\":[{\"x\":365.9528795811519,\"y\":464.5759162303665,\"attribute\":\"\",\"valid\":true,\"id\":\"IKprTuHS\",\"sourceID\":\"\",\"textAttribute\":\"\",\"order\":1},{\"x\":470.71727748691103,\"y\":445.06806282722516,\"attribute\":\"\",\"valid\":true,\"id\":\"Z90CWkEI\",\"sourceID\":\"\",\"textAttribute\":\"\",\"order\":2},{\"x\":410.74869109947645,\"y\":395.2146596858639,\"attribute\":\"\",\"valid\":true,\"id\":\"UWRstKZk\",\"sourceID\":\"\",\"textAttribute\":\"\",\"order\":3}]},\"step_2\":{\"dataSourceStep\":0,\"toolName\":\"polygonTool\",\"result\":[{\"id\":\"C30Pc9Ww\",\"sourceID\":\"\",\"valid\":true,\"textAttribute\":\"\",\"pointList\":[{\"x\":412.91623036649213,\"y\":325.85340314136124},{\"x\":468.5497382198953,\"y\":335.9685863874345},{\"x\":501.78534031413614,\"y\":369.2041884816754},{\"x\":514.0680628272252,\"y\":415.44502617801044},{\"x\":504.67539267015707,\"y\":472.5235602094241},{\"x\":484.44502617801044,\"y\":497.0890052356021},{\"x\":443.26178010471205,\"y\":512.9842931937172},{\"x\":389.7958115183246,\"y\":518.7643979057591},{\"x\":336.32984293193715,\"y\":504.31413612565444},{\"x\":302.3717277486911,\"y\":462.40837696335075},{\"x\":298.0366492146597,\"y\":416.89005235602093},{\"x\":318.26701570680626,\"y\":372.0942408376963},{\"x\":363.0628272251309,\"y\":341.0261780104712}],\"attribute\":\"\",\"order\":1}]}}",
+ "url": "https://user-images.githubusercontent.com/15977946/152742755-9dc75f89-4156-4103-9c6d-f35f1f409d11.jpg",
+ "fileName": "bing_dwen_dwen4.jpg"
+ }
+]
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/configs/newyear/README.md b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/configs/newyear/README.md
new file mode 100644
index 0000000..8c655c1
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/configs/newyear/README.md
@@ -0,0 +1,31 @@
+# New Year Hat and Firecracker Effects
+
+This demo provides new year effects with pose estimation results, like adding hat on the head and firecracker in the hands.
+
+
+
+
+
+## Instruction
+
+### Get started
+
+Launch the demo from the mmpose root directory:
+
+```shell
+python tools/webcam/run_webcam.py --config tools/webcam/configs/newyear/new_year.py
+```
+
+### Hotkeys
+
+| Hotkey | Function |
+| -- | -- |
+| t | Toggle the hat effect on/off. |
+| f | Toggle the firecracker effect on/off. |
+| h | Show help information. |
+| m | Show the monitoring information. |
+| q | Exit. |
+
+### Configuration
+
+See the [README](/tools/webcam/configs/examples/README.md#configuration) of pose estimation demo for model configurations.
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/configs/newyear/new_year.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/configs/newyear/new_year.py
new file mode 100644
index 0000000..3551184
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/configs/newyear/new_year.py
@@ -0,0 +1,122 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+runner = dict(
+ # Basic configurations of the runner
+ name='Pose Estimation',
+ camera_id=0,
+ camera_fps=20,
+ synchronous=False,
+ # Define nodes.
+ # The configuration of a node usually includes:
+ # 1. 'type': Node class name
+ # 2. 'name': Node name
+ # 3. I/O buffers (e.g. 'input_buffer', 'output_buffer'): specify the
+ # input and output buffer names. This may depend on the node class.
+ # 4. 'enable_key': assign a hot-key to toggle enable/disable this node.
+ # This may depend on the node class.
+ # 5. Other class-specific arguments
+ nodes=[
+ # 'DetectorNode':
+ # This node performs object detection from the frame image using an
+ # MMDetection model.
+ dict(
+ type='DetectorNode',
+ name='Detector',
+ model_config='demo/mmdetection_cfg/'
+ 'ssdlite_mobilenetv2_scratch_600e_coco.py',
+ model_checkpoint='https://download.openmmlab.com'
+ '/mmdetection/v2.0/ssd/'
+ 'ssdlite_mobilenetv2_scratch_600e_coco/ssdlite_mobilenetv2_'
+ 'scratch_600e_coco_20210629_110627-974d9307.pth',
+ input_buffer='_input_', # `_input_` is a runner-reserved buffer
+ output_buffer='det_result'),
+ # 'TopDownPoseEstimatorNode':
+ # This node performs keypoint detection from the frame image using an
+ # MMPose top-down model. Detection results is needed.
+ dict(
+ type='TopDownPoseEstimatorNode',
+ name='Human Pose Estimator',
+ model_config='configs/wholebody/2d_kpt_sview_rgb_img/'
+ 'topdown_heatmap/coco-wholebody/'
+ 'vipnas_mbv3_coco_wholebody_256x192_dark.py',
+ model_checkpoint='https://openmmlab-share.oss-cn-hangz'
+ 'hou.aliyuncs.com/mmpose/top_down/vipnas/vipnas_mbv3_co'
+ 'co_wholebody_256x192_dark-e2158108_20211205.pth',
+ cls_names=['person'],
+ input_buffer='det_result',
+ output_buffer='human_pose'),
+ dict(
+ type='TopDownPoseEstimatorNode',
+ name='Animal Pose Estimator',
+ model_config='configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap'
+ '/animalpose/hrnet_w32_animalpose_256x256.py',
+ model_checkpoint='https://download.openmmlab.com/mmpose/animal/'
+ 'hrnet/hrnet_w32_animalpose_256x256-1aa7f075_20210426.pth',
+ cls_names=['cat', 'dog', 'horse', 'sheep', 'cow'],
+ input_buffer='human_pose',
+ output_buffer='animal_pose'),
+ # 'ModelResultBindingNode':
+ # This node binds the latest model inference result with the current
+ # frame. (This means the frame image and inference result may be
+ # asynchronous).
+ dict(
+ type='ModelResultBindingNode',
+ name='ResultBinder',
+ frame_buffer='_frame_', # `_frame_` is a runner-reserved buffer
+ result_buffer='animal_pose',
+ output_buffer='frame'),
+ # 'HatNode':
+ # This node draw the hat effect in the frame image.
+ # Pose results is needed.
+ dict(
+ type='HatNode',
+ name='Visualizer',
+ enable_key='t',
+ frame_buffer='frame',
+ output_buffer='vis_hat'),
+ # 'FirecrackerNode':
+ # This node draw the firecracker effect in the frame image.
+ # Pose results is needed.
+ dict(
+ type='FirecrackerNode',
+ name='Visualizer',
+ enable_key='f',
+ frame_buffer='vis_hat',
+ output_buffer='vis_firecracker'),
+ # 'NoticeBoardNode':
+ # This node show a notice board with given content, e.g. help
+ # information.
+ dict(
+ type='NoticeBoardNode',
+ name='Helper',
+ enable_key='h',
+ enable=True,
+ frame_buffer='vis_firecracker',
+ output_buffer='vis_notice',
+ content_lines=[
+ 'This is a demo for pose visualization and simple image '
+ 'effects. Have fun!', '', 'Hot-keys:', '"t": Hat effect',
+ '"f": Firecracker effect', '"h": Show help information',
+ '"m": Show diagnostic information', '"q": Exit'
+ ],
+ ),
+ # 'MonitorNode':
+ # This node show diagnostic information in the frame image. It can
+ # be used for debugging or monitoring system resource status.
+ dict(
+ type='MonitorNode',
+ name='Monitor',
+ enable_key='m',
+ enable=False,
+ frame_buffer='vis_notice',
+ output_buffer='display'),
+ # 'RecorderNode':
+ # This node save the output video into a file.
+ dict(
+ type='RecorderNode',
+ name='Recorder',
+ out_video_file='record.mp4',
+ frame_buffer='display',
+ output_buffer='_display_'
+ # `_display_` is a runner-reserved buffer
+ )
+ ])
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/configs/supersaiyan/README.md b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/configs/supersaiyan/README.md
new file mode 100644
index 0000000..9e9aef1
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/configs/supersaiyan/README.md
@@ -0,0 +1,96 @@
+# Super Saiyan Effects
+
+We can apply fun effects on videos with pose estimation results, like Super Saiyan transformation.
+
+https://user-images.githubusercontent.com/11788150/150138076-2192079f-068a-4d43-bf27-2f1fd708cabc.mp4
+
+## Instruction
+
+### Get started
+
+Launch the demo from the mmpose root directory:
+
+```shell
+python tools/webcam/run_webcam.py --config tools/webcam/configs/supersaiyan/saiyan.py
+```
+
+### Hotkeys
+
+| Hotkey | Function |
+| -- | -- |
+| s | Toggle the Super Saiyan effect on/off. |
+| h | Show help information. |
+| m | Show the monitoring information. |
+| q | Exit. |
+
+Note that the demo will automatically save the output video into a file `record.mp4`.
+
+### Configuration
+
+- **Choose a detection model**
+
+Users can choose detection models from the [MMDetection Model Zoo](https://mmdetection.readthedocs.io/en/v2.20.0/model_zoo.html). Just set the `model_config` and `model_checkpoint` in the detector node accordingly, and the model will be automatically downloaded and loaded.
+
+```python
+# 'DetectorNode':
+# This node performs object detection from the frame image using an
+# MMDetection model.
+dict(
+ type='DetectorNode',
+ name='Detector',
+ model_config='demo/mmdetection_cfg/mask_rcnn_r50_fpn_2x_coco.py',
+ model_checkpoint='https://download.openmmlab.com/'
+ 'mmdetection/v2.0/mask_rcnn/mask_rcnn_r50_fpn_2x_coco/'
+ 'mask_rcnn_r50_fpn_2x_coco_bbox_mAP-0.392'
+ '__segm_mAP-0.354_20200505_003907-3e542a40.pth',
+ input_buffer='_input_', # `_input_` is a runner-reserved buffer
+ output_buffer='det_result'),
+```
+
+- **Choose a or more pose models**
+
+In this demo we use two [top-down](https://github.com/open-mmlab/mmpose/tree/master/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap) pose estimation models for humans and animals respectively. Users can choose models from the [MMPose Model Zoo](https://mmpose.readthedocs.io/en/latest/modelzoo.html). To apply different pose models on different instance types, you can add multiple pose estimator nodes with `cls_names` set accordingly.
+
+```python
+# 'TopDownPoseEstimatorNode':
+# This node performs keypoint detection from the frame image using an
+# MMPose top-down model. Detection results is needed.
+dict(
+ type='TopDownPoseEstimatorNode',
+ name='Human Pose Estimator',
+ model_config='configs/wholebody/2d_kpt_sview_rgb_img/'
+ 'topdown_heatmap/coco-wholebody/'
+ 'vipnas_mbv3_coco_wholebody_256x192_dark.py',
+ model_checkpoint='https://openmmlab-share.oss-cn-hangz'
+ 'hou.aliyuncs.com/mmpose/top_down/vipnas/vipnas_mbv3_co'
+ 'co_wholebody_256x192_dark-e2158108_20211205.pth',
+ cls_names=['person'],
+ input_buffer='det_result',
+ output_buffer='human_pose')
+```
+
+- **Run the demo without GPU**
+
+If you don't have GPU and CUDA in your device, the demo can run with only CPU by setting `device='cpu'` in all model nodes. For example:
+
+```python
+dict(
+ type='DetectorNode',
+ name='Detector',
+ model_config='demo/mmdetection_cfg/mask_rcnn_r50_fpn_2x_coco.py',
+ model_checkpoint='https://download.openmmlab.com/'
+ 'mmdetection/v2.0/mask_rcnn/mask_rcnn_r50_fpn_2x_coco/'
+ 'mask_rcnn_r50_fpn_2x_coco_bbox_mAP-0.392'
+ '__segm_mAP-0.354_20200505_003907-3e542a40.pth',
+ device='cpu',
+ input_buffer='_input_', # `_input_` is a runner-reserved buffer
+ output_buffer='det_result'),
+```
+
+- **Debug webcam and display**
+
+You can launch the webcam runner with a debug config:
+
+```shell
+python tools/webcam/run_webcam.py --config tools/webcam/configs/examples/test_camera.py
+```
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/configs/supersaiyan/saiyan.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/configs/supersaiyan/saiyan.py
new file mode 100644
index 0000000..5a8e7bc
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/configs/supersaiyan/saiyan.py
@@ -0,0 +1,93 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+runner = dict(
+ # Basic configurations of the runner
+ name='Super Saiyan Effects',
+ camera_id=0,
+ camera_fps=30,
+ synchronous=False,
+ # Define nodes.
+ # The configuration of a node usually includes:
+ # 1. 'type': Node class name
+ # 2. 'name': Node name
+ # 3. I/O buffers (e.g. 'input_buffer', 'output_buffer'): specify the
+ # input and output buffer names. This may depend on the node class.
+ # 4. 'enable_key': assign a hot-key to toggle enable/disable this node.
+ # This may depend on the node class.
+ # 5. Other class-specific arguments
+ nodes=[
+ # 'DetectorNode':
+ # This node performs object detection from the frame image using an
+ # MMDetection model.
+ dict(
+ type='DetectorNode',
+ name='Detector',
+ model_config='demo/mmdetection_cfg/mask_rcnn_r50_fpn_2x_coco.py',
+ model_checkpoint='https://download.openmmlab.com/'
+ 'mmdetection/v2.0/mask_rcnn/mask_rcnn_r50_fpn_2x_coco/'
+ 'mask_rcnn_r50_fpn_2x_coco_bbox_mAP-0.392'
+ '__segm_mAP-0.354_20200505_003907-3e542a40.pth',
+ input_buffer='_input_', # `_input_` is a runner-reserved buffer
+ output_buffer='det_result'),
+ # 'TopDownPoseEstimatorNode':
+ # This node performs keypoint detection from the frame image using an
+ # MMPose top-down model. Detection results is needed.
+ dict(
+ type='TopDownPoseEstimatorNode',
+ name='Human Pose Estimator',
+ model_config='configs/wholebody/2d_kpt_sview_rgb_img/'
+ 'topdown_heatmap/coco-wholebody/'
+ 'vipnas_mbv3_coco_wholebody_256x192_dark.py',
+ model_checkpoint='https://openmmlab-share.oss-cn-hangz'
+ 'hou.aliyuncs.com/mmpose/top_down/vipnas/vipnas_mbv3_co'
+ 'co_wholebody_256x192_dark-e2158108_20211205.pth',
+ cls_names=['person'],
+ input_buffer='det_result',
+ output_buffer='human_pose'),
+ # 'ModelResultBindingNode':
+ # This node binds the latest model inference result with the current
+ # frame. (This means the frame image and inference result may be
+ # asynchronous).
+ dict(
+ type='ModelResultBindingNode',
+ name='ResultBinder',
+ frame_buffer='_frame_', # `_frame_` is a runner-reserved buffer
+ result_buffer='human_pose',
+ output_buffer='frame'),
+ # 'SaiyanNode':
+ # This node draw the Super Saiyan effect in the frame image.
+ # Pose results is needed.
+ dict(
+ type='SaiyanNode',
+ name='Visualizer',
+ enable_key='s',
+ cls_names=['person'],
+ enable=True,
+ frame_buffer='frame',
+ output_buffer='vis_saiyan'),
+ # 'NoticeBoardNode':
+ # This node show a notice board with given content, e.g. help
+ # information.
+ dict(
+ type='NoticeBoardNode',
+ name='Helper',
+ enable_key='h',
+ frame_buffer='vis_saiyan',
+ output_buffer='vis',
+ content_lines=[
+ 'This is a demo for super saiyan effects. Have fun!', '',
+ 'Hot-keys:', '"s": Saiyan effect',
+ '"h": Show help information',
+ '"m": Show diagnostic information', '"q": Exit'
+ ],
+ ),
+ # 'MonitorNode':
+ # This node show diagnostic information in the frame image. It can
+ # be used for debugging or monitoring system resource status.
+ dict(
+ type='MonitorNode',
+ name='Monitor',
+ enable_key='m',
+ enable=False,
+ frame_buffer='vis',
+ output_buffer='_display_') # `_frame_` is a runner-reserved buffer
+ ])
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/configs/valentinemagic/README.md b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/configs/valentinemagic/README.md
new file mode 100644
index 0000000..8063d2e
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/configs/valentinemagic/README.md
@@ -0,0 +1,35 @@
+# Valentine Magic
+
+Do you want to show your **love** to your beloved one, especially on Valentine's Day? Express it with your pose using MMPose right away and see the Valentine Magic!
+
+Try to pose a hand heart gesture, and see what will happen?
+
+Prefer a blow kiss? Here comes your flying heart~
+
+
+
+
+
+## Instruction
+
+### Get started
+
+Launch the demo from the mmpose root directory:
+
+```shell
+python tools/webcam/run_webcam.py --config tools/webcam/configs/valentinemagic/valentinemagic.py
+```
+
+### Hotkeys
+
+| Hotkey | Function |
+| -- | -- |
+| l | Toggle the Valentine Magic effect on/off. |
+| v | Toggle the pose visualization on/off. |
+| h | Show help information. |
+| m | Show diagnostic information. |
+| q | Exit. |
+
+### Configuration
+
+See the [README](/tools/webcam/configs/examples/README.md#configuration) of pose estimation demo for model configurations.
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/configs/valentinemagic/valentinemagic.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/configs/valentinemagic/valentinemagic.py
new file mode 100644
index 0000000..5f921b0
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/configs/valentinemagic/valentinemagic.py
@@ -0,0 +1,118 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+runner = dict(
+ # Basic configurations of the runner
+ name='Human Pose and Effects',
+ camera_id=0,
+ camera_fps=30,
+
+ # Define nodes.
+ #
+ # The configuration of a node usually includes:
+ # 1. 'type': Node class name
+ # 2. 'name': Node name
+ # 3. I/O buffers (e.g. 'input_buffer', 'output_buffer'): specify the
+ # input and output buffer names. This may depend on the node class.
+ # 4. 'enable_key': assign a hot-key to toggle enable/disable this node.
+ # This may depend on the node class.
+ # 5. Other class-specific arguments
+ nodes=[
+ # 'DetectorNode':
+ # This node performs object detection from the frame image using an
+ # MMDetection model.
+ dict(
+ type='DetectorNode',
+ name='Detector',
+ model_config='demo/mmdetection_cfg/'
+ 'ssdlite_mobilenetv2_scratch_600e_coco.py',
+ model_checkpoint='https://download.openmmlab.com'
+ '/mmdetection/v2.0/ssd/'
+ 'ssdlite_mobilenetv2_scratch_600e_coco/ssdlite_mobilenetv2_'
+ 'scratch_600e_coco_20210629_110627-974d9307.pth',
+ input_buffer='_input_', # `_input_` is a runner-reserved buffer
+ output_buffer='det_result'),
+ # 'TopDownPoseEstimatorNode':
+ # This node performs keypoint detection from the frame image using an
+ # MMPose top-down model. Detection results is needed.
+ dict(
+ type='TopDownPoseEstimatorNode',
+ name='Human Pose Estimator',
+ model_config='configs/wholebody/2d_kpt_sview_rgb_img/'
+ 'topdown_heatmap/coco-wholebody/'
+ 'vipnas_mbv3_coco_wholebody_256x192_dark.py',
+ model_checkpoint='https://download.openmmlab.com/mmpose/top_down/'
+ 'vipnas/vipnas_mbv3_coco_wholebody_256x192_dark'
+ '-e2158108_20211205.pth',
+ cls_names=['person'],
+ input_buffer='det_result',
+ output_buffer='pose_result'),
+ # 'ModelResultBindingNode':
+ # This node binds the latest model inference result with the current
+ # frame. (This means the frame image and inference result may be
+ # asynchronous).
+ dict(
+ type='ModelResultBindingNode',
+ name='ResultBinder',
+ frame_buffer='_frame_', # `_frame_` is a runner-reserved buffer
+ result_buffer='pose_result',
+ output_buffer='frame'),
+ # 'PoseVisualizerNode':
+ # This node draw the pose visualization result in the frame image.
+ # Pose results is needed.
+ dict(
+ type='PoseVisualizerNode',
+ name='Visualizer',
+ enable_key='v',
+ enable=False,
+ frame_buffer='frame',
+ output_buffer='vis'),
+ # 'ValentineMagicNode':
+ # This node draw heart in the image.
+ # It can launch dynamically expanding heart from the middle of
+ # hands if the persons pose a "hand heart" gesture or blow a kiss.
+ # Only there are two persons in the image can trigger this effect.
+ # Pose results is needed.
+ dict(
+ type='ValentineMagicNode',
+ name='Visualizer',
+ enable_key='l',
+ frame_buffer='vis',
+ output_buffer='vis_heart',
+ ),
+ # 'NoticeBoardNode':
+ # This node show a notice board with given content, e.g. help
+ # information.
+ dict(
+ type='NoticeBoardNode',
+ name='Helper',
+ enable_key='h',
+ enable=False,
+ frame_buffer='vis_heart',
+ output_buffer='vis_notice',
+ content_lines=[
+ 'This is a demo for pose visualization and simple image '
+ 'effects. Have fun!', '', 'Hot-keys:',
+ '"h": Show help information', '"l": LoveHeart Effect',
+ '"v": PoseVisualizer', '"m": Show diagnostic information',
+ '"q": Exit'
+ ],
+ ),
+ # 'MonitorNode':
+ # This node show diagnostic information in the frame image. It can
+ # be used for debugging or monitoring system resource status.
+ dict(
+ type='MonitorNode',
+ name='Monitor',
+ enable_key='m',
+ enable=False,
+ frame_buffer='vis_notice',
+ output_buffer='display'), # `_frame_` is a runner-reserved buffer
+ # 'RecorderNode':
+ # This node record the frames into a local file. It can save the
+ # visualiztion results. Uncommit the following lines to turn it on.
+ dict(
+ type='RecorderNode',
+ name='Recorder',
+ out_video_file='record.mp4',
+ frame_buffer='display',
+ output_buffer='_display_')
+ ])
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/docs/example_cn.md b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/docs/example_cn.md
new file mode 100644
index 0000000..69b9898
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/docs/example_cn.md
@@ -0,0 +1,171 @@
+# 开发示例：给猫咪戴上太阳镜
+
+## 设计思路
+
+在动手之前，我们先考虑如何实现这个功能：
+
+- 首先，要做目标检测，找到图像中的猫咪
+- 接着，要估计猫咪的关键点位置，比如左右眼的位置
+- 最后，把太阳镜素材图片贴在合适的位置，TA-DA！
+
+按照这个思路，下面我们来看如何一步一步实现它。
+
+## Step 1：从一个现成的 Config 开始
+
+在 WebcamAPI 中，已经添加了一些实现常用功能的 Node，并提供了对应的 config 示例。利用这些可以减少用户的开发量。例如，我们可以以上面的姿态估计 demo 为基础。它的 config 位于 `tools/webcam/configs/example/pose_estimation.py`。为了更直观，我们把这个 config 中的功能节点表示成以下流程图：
+
+
+
+图中的每个 Data Buffer 就是一个用来存放数据的容器。用户不需要关注 buffer 的具体细节，只需要将其简单理解成 Node 输入输出的名字即可。用户在 config 中可以任意定义这些名字，不过要注意有以下几个特殊的名字：
+
+- _input_：存放 runner 读入的视频帧，用于模型推理
+- _frame_ ：存放 runner 读入的视频帧，用于可视化
+- _display_：存放经过所以 Node 处理后的结果，用于在屏幕上显示
+
+当一帧视频数据被 runner 读入后，会被放进 _input_ 和 _frame_ 两个 buffer 中，然后按照 config 中定义的 Node 连接关系依次通过各个 Node ，最终到达 _display_ ，并被 runner 读出显示在屏幕上。
+
+#### Get Advanced: 关于 buffer
+
+- Buffer 本质是一个有限长度的队列，在 runner 中会包含一个 BufferManager 实例（见`mmpose/tools/webcam/webcam_apis/buffer.py'）来生成和管理所有 buffer。Node 会按照 config 从对应的 buffer 中读出或写入数据。
+- 当一个 buffer 已满（达到最大长度）时，写入数据的操作通常不会被 block，而是会将 buffer 中已有的最早一条数据“挤出去”。
+- 为什么有_input_和_frame_两个输入呢？因为有些 Node 的操作较为耗时（如目标检测，姿态估计等需要模型推理的 Node）。为了保证显示的流畅，我们通常用_input_来作为这类耗时较大的操作的输入，而用_frame_来实时绘制可视化的结果。因为各个节点是异步运行的，这样就可以保证可视化的实时和流畅。
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/run_webcam.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/run_webcam.py
new file mode 100644
index 0000000..ce8d92e
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/run_webcam.py
@@ -0,0 +1,38 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+
+from argparse import ArgumentParser
+
+from mmcv import Config, DictAction
+from webcam_apis import WebcamRunner
+
+
+def parse_args():
+ parser = ArgumentParser('Lauch webcam runner')
+ parser.add_argument(
+ '--config',
+ type=str,
+ default='tools/webcam/configs/meow_dwen_dwen/meow_dwen_dwen.py')
+
+ parser.add_argument(
+ '--cfg-options',
+ nargs='+',
+ action=DictAction,
+ default={},
+ help='override some settings in the used config, the key-value pair '
+ 'in xxx=yyy format will be merged into config file. For example, '
+ "'--cfg-options runner.camera_id=1 runner.synchronous=True'")
+
+ return parser.parse_args()
+
+
+def launch():
+ args = parse_args()
+ cfg = Config.fromfile(args.config)
+ cfg.merge_from_dict(args.cfg_options)
+
+ runner = WebcamRunner(**cfg.runner)
+ runner.run()
+
+
+if __name__ == '__main__':
+ launch()
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/__init__.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/__init__.py
new file mode 100644
index 0000000..1c8a2f5
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/__init__.py
@@ -0,0 +1,4 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from .webcam_runner import WebcamRunner
+
+__all__ = ['WebcamRunner']
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/nodes/__init__.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/nodes/__init__.py
new file mode 100644
index 0000000..a882030
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/nodes/__init__.py
@@ -0,0 +1,18 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from .builder import NODES
+from .faceswap_node import FaceSwapNode
+from .frame_effect_node import (BackgroundNode, BugEyeNode, MoustacheNode,
+ NoticeBoardNode, PoseVisualizerNode,
+ SaiyanNode, SunglassesNode)
+from .helper_node import ModelResultBindingNode, MonitorNode, RecorderNode
+from .mmdet_node import DetectorNode
+from .mmpose_node import TopDownPoseEstimatorNode
+from .valentinemagic_node import ValentineMagicNode
+from .xdwendwen_node import XDwenDwenNode
+
+__all__ = [
+ 'NODES', 'PoseVisualizerNode', 'DetectorNode', 'TopDownPoseEstimatorNode',
+ 'MonitorNode', 'BugEyeNode', 'SunglassesNode', 'ModelResultBindingNode',
+ 'NoticeBoardNode', 'RecorderNode', 'FaceSwapNode', 'MoustacheNode',
+ 'SaiyanNode', 'BackgroundNode', 'XDwenDwenNode', 'ValentineMagicNode'
+]
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/nodes/builder.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/nodes/builder.py
new file mode 100644
index 0000000..44900b7
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/nodes/builder.py
@@ -0,0 +1,4 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from mmcv.utils import Registry
+
+NODES = Registry('node')
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/nodes/faceswap_node.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/nodes/faceswap_node.py
new file mode 100644
index 0000000..5ac4420
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/nodes/faceswap_node.py
@@ -0,0 +1,254 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from enum import IntEnum
+from typing import List, Union
+
+import cv2
+import numpy as np
+
+from mmpose.datasets import DatasetInfo
+from .builder import NODES
+from .frame_drawing_node import FrameDrawingNode
+
+
+class Mode(IntEnum):
+ NONE = 0,
+ SHUFFLE = 1,
+ CLONE = 2
+
+
+@NODES.register_module()
+class FaceSwapNode(FrameDrawingNode):
+
+ def __init__(
+ self,
+ name: str,
+ frame_buffer: str,
+ output_buffer: Union[str, List[str]],
+ mode_key: Union[str, int],
+ ):
+ super().__init__(name, frame_buffer, output_buffer, enable=True)
+
+ self.mode_key = mode_key
+ self.mode_index = 0
+ self.register_event(
+ self.mode_key, is_keyboard=True, handler_func=self.switch_mode)
+ self.history = dict(mode=None)
+ self._mode = Mode.SHUFFLE
+
+ @property
+ def mode(self):
+ return self._mode
+
+ def switch_mode(self):
+ """Switch modes by updating mode index."""
+ self._mode = Mode((self._mode + 1) % len(Mode))
+
+ def draw(self, frame_msg):
+
+ if self.mode == Mode.NONE:
+ self.history = {'mode': Mode.NONE}
+ return frame_msg.get_image()
+
+ # Init history
+ if self.history['mode'] != self.mode:
+ self.history = {'mode': self.mode, 'target_map': {}}
+
+ # Merge pose results
+ pose_preds = self._merge_pose_results(frame_msg.get_pose_results())
+ num_target = len(pose_preds)
+
+ # Show mode
+ img = frame_msg.get_image()
+ canvas = img.copy()
+ if self.mode == Mode.SHUFFLE:
+ mode_txt = 'Shuffle'
+ else:
+ mode_txt = 'Clone'
+
+ cv2.putText(canvas, mode_txt, (10, 50), cv2.FONT_HERSHEY_DUPLEX, 0.8,
+ (255, 126, 0), 1)
+
+ # Skip if target number is less than 2
+ if num_target >= 2:
+ # Generate new mapping if target number changes
+ if num_target != len(self.history['target_map']):
+ if self.mode == Mode.SHUFFLE:
+ self.history['target_map'] = self._get_swap_map(num_target)
+ else:
+ self.history['target_map'] = np.repeat(
+ np.random.choice(num_target), num_target)
+
+ # # Draw on canvas
+ for tar_idx, src_idx in enumerate(self.history['target_map']):
+ face_src = self._get_face_info(pose_preds[src_idx])
+ face_tar = self._get_face_info(pose_preds[tar_idx])
+ canvas = self._swap_face(img, canvas, face_src, face_tar)
+
+ return canvas
+
+ def _crop_face_by_contour(self, img, contour):
+ mask = np.zeros(img.shape[:2], dtype=np.uint8)
+ cv2.fillPoly(mask, [contour.astype(np.int32)], 1)
+ mask = cv2.dilate(
+ mask, kernel=np.ones((9, 9), dtype=np.uint8), anchor=(4, 0))
+ x1, y1, w, h = cv2.boundingRect(mask)
+ x2 = x1 + w
+ y2 = y1 + h
+ bbox = np.array([x1, y1, x2, y2], dtype=np.int64)
+ patch = img[y1:y2, x1:x2]
+ mask = mask[y1:y2, x1:x2]
+
+ return bbox, patch, mask
+
+ def _swap_face(self, img_src, img_tar, face_src, face_tar):
+
+ if face_src['dataset'] == face_tar['dataset']:
+ # Use full keypoints for face alignment
+ kpts_src = face_src['contour']
+ kpts_tar = face_tar['contour']
+ else:
+ # Use only common landmarks (eyes and nose) for face alignment if
+ # source and target have differenet data type
+ # (e.g. human vs animal)
+ kpts_src = face_src['landmarks']
+ kpts_tar = face_tar['landmarks']
+
+ # Get everything local
+ bbox_src, patch_src, mask_src = self._crop_face_by_contour(
+ img_src, face_src['contour'])
+
+ bbox_tar, _, mask_tar = self._crop_face_by_contour(
+ img_tar, face_tar['contour'])
+
+ kpts_src = kpts_src - bbox_src[:2]
+ kpts_tar = kpts_tar - bbox_tar[:2]
+
+ # Compute affine transformation matrix
+ trans_mat, _ = cv2.estimateAffine2D(
+ kpts_src.astype(np.float32), kpts_tar.astype(np.float32))
+ patch_warp = cv2.warpAffine(
+ patch_src,
+ trans_mat,
+ dsize=tuple(bbox_tar[2:] - bbox_tar[:2]),
+ borderValue=(0, 0, 0))
+ mask_warp = cv2.warpAffine(
+ mask_src,
+ trans_mat,
+ dsize=tuple(bbox_tar[2:] - bbox_tar[:2]),
+ borderValue=(0, 0, 0))
+
+ # Target mask
+ mask_tar = mask_tar & mask_warp
+ mask_tar_soft = cv2.GaussianBlur(mask_tar * 255, (3, 3), 3)
+
+ # Blending
+ center = tuple((0.5 * (bbox_tar[:2] + bbox_tar[2:])).astype(np.int64))
+ img_tar = cv2.seamlessClone(patch_warp, img_tar, mask_tar_soft, center,
+ cv2.NORMAL_CLONE)
+ return img_tar
+
+ @staticmethod
+ def _get_face_info(pose_pred):
+ keypoints = pose_pred['keypoints'][:, :2]
+ model_cfg = pose_pred['model_cfg']
+ dataset_info = DatasetInfo(model_cfg.data.test.dataset_info)
+
+ face_info = {
+ 'dataset': dataset_info.dataset_name,
+ 'landmarks': None, # For alignment
+ 'contour': None, # For mask generation
+ 'bbox': None # For image warping
+ }
+
+ # Fall back to hard coded keypoint id
+
+ if face_info['dataset'] == 'coco':
+ face_info['landmarks'] = np.stack([
+ keypoints[1], # left eye
+ keypoints[2], # right eye
+ keypoints[0], # nose
+ 0.5 * (keypoints[5] + keypoints[6]), # neck (shoulder center)
+ ])
+ elif face_info['dataset'] == 'coco_wholebody':
+ face_info['landmarks'] = np.stack([
+ keypoints[1], # left eye
+ keypoints[2], # right eye
+ keypoints[0], # nose
+ keypoints[32], # chin
+ ])
+ contour_ids = list(range(23, 40)) + list(range(40, 50))[::-1]
+ face_info['contour'] = keypoints[contour_ids]
+ elif face_info['dataset'] == 'ap10k':
+ face_info['landmarks'] = np.stack([
+ keypoints[0], # left eye
+ keypoints[1], # right eye
+ keypoints[2], # nose
+ keypoints[3], # neck
+ ])
+ elif face_info['dataset'] == 'animalpose':
+ face_info['landmarks'] = np.stack([
+ keypoints[0], # left eye
+ keypoints[1], # right eye
+ keypoints[4], # nose
+ keypoints[5], # throat
+ ])
+ elif face_info['dataset'] == 'wflw':
+ face_info['landmarks'] = np.stack([
+ keypoints[97], # left eye
+ keypoints[96], # right eye
+ keypoints[54], # nose
+ keypoints[16], # chine
+ ])
+ contour_ids = list(range(33))[::-1] + list(range(33, 38)) + list(
+ range(42, 47))
+ face_info['contour'] = keypoints[contour_ids]
+ else:
+ raise ValueError('Can not obtain face landmark information'
+ f'from dataset: {face_info["type"]}')
+
+ # Face region
+ if face_info['contour'] is None:
+ # Manually defined counter of face region
+ left_eye, right_eye, nose = face_info['landmarks'][:3]
+ eye_center = 0.5 * (left_eye + right_eye)
+ w_vec = right_eye - left_eye
+ eye_dist = np.linalg.norm(w_vec) + 1e-6
+ w_vec = w_vec / eye_dist
+ h_vec = np.array([w_vec[1], -w_vec[0]], dtype=w_vec.dtype)
+ w = max(0.5 * eye_dist, np.abs(np.dot(nose - eye_center, w_vec)))
+ h = np.abs(np.dot(nose - eye_center, h_vec))
+
+ left_top = eye_center + 1.5 * w * w_vec - 0.5 * h * h_vec
+ right_top = eye_center - 1.5 * w * w_vec - 0.5 * h * h_vec
+ left_bottom = eye_center + 1.5 * w * w_vec + 4 * h * h_vec
+ right_bottom = eye_center - 1.5 * w * w_vec + 4 * h * h_vec
+
+ face_info['contour'] = np.stack(
+ [left_top, right_top, right_bottom, left_bottom])
+
+ # Get tight bbox of face region
+ face_info['bbox'] = np.array([
+ face_info['contour'][:, 0].min(), face_info['contour'][:, 1].min(),
+ face_info['contour'][:, 0].max(), face_info['contour'][:, 1].max()
+ ]).astype(np.int64)
+
+ return face_info
+
+ @staticmethod
+ def _merge_pose_results(pose_results):
+ preds = []
+ if pose_results is not None:
+ for prefix, pose_result in enumerate(pose_results):
+ model_cfg = pose_result['model_cfg']
+ for idx, _pred in enumerate(pose_result['preds']):
+ pred = _pred.copy()
+ pred['id'] = f'{prefix}.{_pred.get("track_id", str(idx))}'
+ pred['model_cfg'] = model_cfg
+ preds.append(pred)
+ return preds
+
+ @staticmethod
+ def _get_swap_map(num_target):
+ ids = np.random.choice(num_target, num_target, replace=False)
+ target_map = ids[(ids + 1) % num_target]
+ return target_map
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/nodes/frame_drawing_node.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/nodes/frame_drawing_node.py
new file mode 100644
index 0000000..cfc3511
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/nodes/frame_drawing_node.py
@@ -0,0 +1,65 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from abc import abstractmethod
+from typing import Dict, List, Optional, Union
+
+import numpy as np
+
+from ..utils import FrameMessage, Message
+from .node import Node
+
+
+class FrameDrawingNode(Node):
+ """Base class for Node that draw on single frame images.
+
+ Args:
+ name (str, optional): The node name (also thread name).
+ frame_buffer (str): The name of the input buffer.
+ output_buffer (str | list): The name(s) of the output buffer(s).
+ enable_key (str | int, optional): Set a hot-key to toggle
+ enable/disable of the node. If an int value is given, it will be
+ treated as an ascii code of a key. Please note:
+ 1. If enable_key is set, the bypass method need to be
+ overridden to define the node behavior when disabled
+ 2. Some hot-key has been use for particular use. For example:
+ 'q', 'Q' and 27 are used for quit
+ Default: None
+ enable (bool): Default enable/disable status. Default: True.
+ """
+
+ def __init__(self,
+ name: str,
+ frame_buffer: str,
+ output_buffer: Union[str, List[str]],
+ enable_key: Optional[Union[str, int]] = None,
+ enable: bool = True):
+
+ super().__init__(name=name, enable_key=enable_key)
+
+ # Register buffers
+ self.register_input_buffer(frame_buffer, 'frame', essential=True)
+ self.register_output_buffer(output_buffer)
+
+ self._enabled = enable
+
+ def process(self, input_msgs: Dict[str, Message]) -> Union[Message, None]:
+ frame_msg = input_msgs['frame']
+
+ img = self.draw(frame_msg)
+ frame_msg.set_image(img)
+
+ return frame_msg
+
+ def bypass(self, input_msgs: Dict[str, Message]) -> Union[Message, None]:
+ return input_msgs['frame']
+
+ @abstractmethod
+ def draw(self, frame_msg: FrameMessage) -> np.ndarray:
+ """Draw on the frame image with information from the single frame.
+
+ Args:
+ frame_meg (FrameMessage): The frame to get information from and
+ draw on.
+
+ Returns:
+ array: The output image
+ """
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/nodes/frame_effect_node.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/nodes/frame_effect_node.py
new file mode 100644
index 0000000..c248c38
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/nodes/frame_effect_node.py
@@ -0,0 +1,917 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from typing import Dict, List, Optional, Tuple, Union
+
+import cv2
+import numpy as np
+from mmcv import color_val
+
+from mmpose.core import (apply_bugeye_effect, apply_sunglasses_effect,
+ imshow_bboxes, imshow_keypoints)
+from mmpose.datasets import DatasetInfo
+from ..utils import (FrameMessage, copy_and_paste, expand_and_clamp,
+ get_cached_file_path, get_eye_keypoint_ids,
+ get_face_keypoint_ids, get_wrist_keypoint_ids,
+ load_image_from_disk_or_url, screen_matting)
+from .builder import NODES
+from .frame_drawing_node import FrameDrawingNode
+
+try:
+ import psutil
+ psutil_proc = psutil.Process()
+except (ImportError, ModuleNotFoundError):
+ psutil_proc = None
+
+
+@NODES.register_module()
+class PoseVisualizerNode(FrameDrawingNode):
+ """Draw the bbox and keypoint detection results.
+
+ Args:
+ name (str, optional): The node name (also thread name).
+ frame_buffer (str): The name of the input buffer.
+ output_buffer (str|list): The name(s) of the output buffer(s).
+ enable_key (str|int, optional): Set a hot-key to toggle enable/disable
+ of the node. If an int value is given, it will be treated as an
+ ascii code of a key. Please note:
+ 1. If enable_key is set, the bypass method need to be
+ overridden to define the node behavior when disabled
+ 2. Some hot-key has been use for particular use. For example:
+ 'q', 'Q' and 27 are used for quit
+ Default: None
+ enable (bool): Default enable/disable status. Default: True.
+ kpt_thr (float): The threshold of keypoint score. Default: 0.3.
+ radius (int): The radius of keypoint. Default: 4.
+ thickness (int): The thickness of skeleton. Default: 2.
+ bbox_color (str|tuple|dict): If a single color (a str like 'green' or
+ a tuple like (0, 255, 0)), it will used to draw the bbox.
+ Optionally, a dict can be given as a map from class labels to
+ colors.
+ """
+
+ default_bbox_color = {
+ 'person': (148, 139, 255),
+ 'cat': (255, 255, 0),
+ 'dog': (255, 255, 0),
+ }
+
+ def __init__(self,
+ name: str,
+ frame_buffer: str,
+ output_buffer: Union[str, List[str]],
+ enable_key: Optional[Union[str, int]] = None,
+ enable: bool = True,
+ kpt_thr: float = 0.3,
+ radius: int = 4,
+ thickness: int = 2,
+ bbox_color: Optional[Union[str, Tuple, Dict]] = None):
+
+ super().__init__(name, frame_buffer, output_buffer, enable_key, enable)
+
+ self.kpt_thr = kpt_thr
+ self.radius = radius
+ self.thickness = thickness
+ if bbox_color is None:
+ self.bbox_color = self.default_bbox_color
+ elif isinstance(bbox_color, dict):
+ self.bbox_color = {k: color_val(v) for k, v in bbox_color.items()}
+ else:
+ self.bbox_color = color_val(bbox_color)
+
+ def draw(self, frame_msg):
+ canvas = frame_msg.get_image()
+ pose_results = frame_msg.get_pose_results()
+
+ if not pose_results:
+ return canvas
+
+ for pose_result in frame_msg.get_pose_results():
+ model_cfg = pose_result['model_cfg']
+ dataset_info = DatasetInfo(model_cfg.dataset_info)
+
+ # Extract bboxes and poses
+ bbox_preds = []
+ bbox_labels = []
+ pose_preds = []
+ for pred in pose_result['preds']:
+ if 'bbox' in pred:
+ bbox_preds.append(pred['bbox'])
+ bbox_labels.append(pred.get('label', None))
+ pose_preds.append(pred['keypoints'])
+
+ # Get bbox colors
+ if isinstance(self.bbox_color, dict):
+ bbox_colors = [
+ self.bbox_color.get(label, (0, 255, 0))
+ for label in bbox_labels
+ ]
+ else:
+ bbox_labels = self.bbox_color
+
+ # Draw bboxes
+ if bbox_preds:
+ bboxes = np.vstack(bbox_preds)
+
+ imshow_bboxes(
+ canvas,
+ bboxes,
+ labels=bbox_labels,
+ colors=bbox_colors,
+ text_color='white',
+ font_scale=0.5,
+ show=False)
+
+ # Draw poses
+ if pose_preds:
+ imshow_keypoints(
+ canvas,
+ pose_preds,
+ skeleton=dataset_info.skeleton,
+ kpt_score_thr=0.3,
+ pose_kpt_color=dataset_info.pose_kpt_color,
+ pose_link_color=dataset_info.pose_link_color,
+ radius=self.radius,
+ thickness=self.thickness)
+
+ return canvas
+
+
+@NODES.register_module()
+class SunglassesNode(FrameDrawingNode):
+
+ def __init__(self,
+ name: str,
+ frame_buffer: str,
+ output_buffer: Union[str, List[str]],
+ enable_key: Optional[Union[str, int]] = None,
+ enable: bool = True,
+ src_img_path: Optional[str] = None):
+
+ super().__init__(name, frame_buffer, output_buffer, enable_key, enable)
+
+ if src_img_path is None:
+ # The image attributes to:
+ # https://www.vecteezy.com/free-vector/glass
+ # Glass Vectors by Vecteezy
+ src_img_path = 'demo/resources/sunglasses.jpg'
+ self.src_img = load_image_from_disk_or_url(src_img_path)
+
+ def draw(self, frame_msg):
+ canvas = frame_msg.get_image()
+ pose_results = frame_msg.get_pose_results()
+ if not pose_results:
+ return canvas
+ for pose_result in pose_results:
+ model_cfg = pose_result['model_cfg']
+ preds = pose_result['preds']
+ left_eye_idx, right_eye_idx = get_eye_keypoint_ids(model_cfg)
+
+ canvas = apply_sunglasses_effect(canvas, preds, self.src_img,
+ left_eye_idx, right_eye_idx)
+ return canvas
+
+
+@NODES.register_module()
+class SpriteNode(FrameDrawingNode):
+
+ def __init__(self,
+ name: str,
+ frame_buffer: str,
+ output_buffer: Union[str, List[str]],
+ enable_key: Optional[Union[str, int]] = None,
+ enable: bool = True,
+ src_img_path: Optional[str] = None):
+
+ super().__init__(name, frame_buffer, output_buffer, enable_key, enable)
+
+ if src_img_path is None:
+ # Sprites of Touhou characters :)
+ # Come from https://www.deviantart.com/shadowbendy/art/Touhou-rpg-maker-vx-Sprite-1-812746920 # noqa: E501
+ src_img_path = (
+ 'https://user-images.githubusercontent.com/'
+ '26739999/151532276-33f968d9-917f-45e3-8a99-ebde60be83bb.png')
+ self.src_img = load_image_from_disk_or_url(
+ src_img_path, cv2.IMREAD_UNCHANGED)[:144, :108]
+ tmp = np.array(np.split(self.src_img, range(36, 144, 36), axis=0))
+ tmp = np.array(np.split(tmp, range(36, 108, 36), axis=2))
+ self.sprites = tmp
+ self.pos = None
+ self.anime_frame = 0
+
+ def apply_sprite_effect(self,
+ img,
+ pose_results,
+ left_hand_index,
+ right_hand_index,
+ kpt_thr=0.5):
+ """Apply sprite effect.
+
+ Args:
+ img (np.ndarray): Image data.
+ pose_results (list[dict]): The pose estimation results containing:
+ - "keypoints" ([K,3]): detection result in [x, y, score]
+ left_hand_index (int): Keypoint index of left hand
+ right_hand_index (int): Keypoint index of right hand
+ kpt_thr (float): The score threshold of required keypoints.
+ """
+
+ hm, wm = self.sprites.shape[2:4]
+ # anchor points in the sunglasses mask
+ if self.pos is None:
+ self.pos = [img.shape[0] // 2, img.shape[1] // 2]
+
+ if len(pose_results) == 0:
+ return img
+
+ kpts = pose_results[0]['keypoints']
+
+ if kpts[left_hand_index, 2] < kpt_thr and kpts[right_hand_index,
+ 2] < kpt_thr:
+ aim = self.pos
+ else:
+ kpt_lhand = kpts[left_hand_index, :2][::-1]
+ kpt_rhand = kpts[right_hand_index, :2][::-1]
+
+ def distance(a, b):
+ return (a[0] - b[0])**2 + (a[1] - b[1])**2
+
+ # Go to the nearest hand
+ if distance(kpt_lhand, self.pos) < distance(kpt_rhand, self.pos):
+ aim = kpt_lhand
+ else:
+ aim = kpt_rhand
+
+ pos_thr = 15
+ if aim[0] < self.pos[0] - pos_thr:
+ # Go down
+ sprite = self.sprites[self.anime_frame][3]
+ self.pos[0] -= 1
+ elif aim[0] > self.pos[0] + pos_thr:
+ # Go up
+ sprite = self.sprites[self.anime_frame][0]
+ self.pos[0] += 1
+ elif aim[1] < self.pos[1] - pos_thr:
+ # Go right
+ sprite = self.sprites[self.anime_frame][1]
+ self.pos[1] -= 1
+ elif aim[1] > self.pos[1] + pos_thr:
+ # Go left
+ sprite = self.sprites[self.anime_frame][2]
+ self.pos[1] += 1
+ else:
+ # Stay
+ self.anime_frame = 0
+ sprite = self.sprites[self.anime_frame][0]
+
+ if self.anime_frame < 2:
+ self.anime_frame += 1
+ else:
+ self.anime_frame = 0
+
+ x = self.pos[0] - hm // 2
+ y = self.pos[1] - wm // 2
+ x = max(0, min(x, img.shape[0] - hm))
+ y = max(0, min(y, img.shape[0] - wm))
+
+ # Overlay image with transparent
+ img[x:x + hm, y:y +
+ wm] = (img[x:x + hm, y:y + wm] * (1 - sprite[:, :, 3:] / 255) +
+ sprite[:, :, :3] * (sprite[:, :, 3:] / 255)).astype('uint8')
+
+ return img
+
+ def draw(self, frame_msg):
+ canvas = frame_msg.get_image()
+ pose_results = frame_msg.get_pose_results()
+ if not pose_results:
+ return canvas
+ for pose_result in pose_results:
+ model_cfg = pose_result['model_cfg']
+ preds = pose_result['preds']
+ # left_hand_idx, right_hand_idx = get_wrist_keypoint_ids(model_cfg) # noqa: E501
+ left_hand_idx, right_hand_idx = get_eye_keypoint_ids(model_cfg)
+
+ canvas = self.apply_sprite_effect(canvas, preds, left_hand_idx,
+ right_hand_idx)
+ return canvas
+
+
+@NODES.register_module()
+class BackgroundNode(FrameDrawingNode):
+
+ def __init__(self,
+ name: str,
+ frame_buffer: str,
+ output_buffer: Union[str, List[str]],
+ enable_key: Optional[Union[str, int]] = None,
+ enable: bool = True,
+ src_img_path: Optional[str] = None,
+ cls_ids: Optional[List] = None,
+ cls_names: Optional[List] = None):
+
+ super().__init__(name, frame_buffer, output_buffer, enable_key, enable)
+
+ self.cls_ids = cls_ids
+ self.cls_names = cls_names
+
+ if src_img_path is None:
+ src_img_path = 'https://user-images.githubusercontent.com/'\
+ '11788150/149731957-abd5c908-9c7f-45b2-b7bf-'\
+ '821ab30c6a3e.jpg'
+ self.src_img = load_image_from_disk_or_url(src_img_path)
+
+ def apply_background_effect(self,
+ img,
+ det_results,
+ background_img,
+ effect_region=(0.2, 0.2, 0.8, 0.8)):
+ """Change background.
+
+ Args:
+ img (np.ndarray): Image data.
+ det_results (list[dict]): The detection results containing:
+
+ - "cls_id" (int): Class index.
+ - "label" (str): Class label (e.g. 'person').
+ - "bbox" (ndarray:(5, )): bounding box result
+ [x, y, w, h, score].
+ - "mask" (ndarray:(w, h)): instance segmentation result.
+ background_img (np.ndarray): Background image.
+ effect_region (tuple(4, )): The region to apply mask,
+ the coordinates are normalized (x1, y1, x2, y2).
+ """
+ if len(det_results) > 0:
+ # Choose the one with the highest score.
+ det_result = det_results[0]
+ bbox = det_result['bbox']
+ mask = det_result['mask'].astype(np.uint8)
+ img = copy_and_paste(img, background_img, mask, bbox,
+ effect_region)
+ return img
+ else:
+ return background_img
+
+ def draw(self, frame_msg):
+ canvas = frame_msg.get_image()
+ if canvas.shape != self.src_img.shape:
+ self.src_img = cv2.resize(self.src_img, canvas.shape[:2])
+ det_results = frame_msg.get_detection_results()
+ if not det_results:
+ return canvas
+
+ full_preds = []
+ for det_result in det_results:
+ preds = det_result['preds']
+ if self.cls_ids:
+ # Filter results by class ID
+ filtered_preds = [
+ p for p in preds if p['cls_id'] in self.cls_ids
+ ]
+ elif self.cls_names:
+ # Filter results by class name
+ filtered_preds = [
+ p for p in preds if p['label'] in self.cls_names
+ ]
+ else:
+ filtered_preds = preds
+ full_preds.extend(filtered_preds)
+
+ canvas = self.apply_background_effect(canvas, full_preds, self.src_img)
+
+ return canvas
+
+
+@NODES.register_module()
+class SaiyanNode(FrameDrawingNode):
+
+ def __init__(self,
+ name: str,
+ frame_buffer: str,
+ output_buffer: Union[str, List[str]],
+ enable_key: Optional[Union[str, int]] = None,
+ enable: bool = True,
+ hair_img_path: Optional[str] = None,
+ light_video_path: Optional[str] = None,
+ cls_ids: Optional[List] = None,
+ cls_names: Optional[List] = None):
+
+ super().__init__(name, frame_buffer, output_buffer, enable_key, enable)
+
+ self.cls_ids = cls_ids
+ self.cls_names = cls_names
+
+ if hair_img_path is None:
+ hair_img_path = 'https://user-images.githubusercontent.com/'\
+ '11788150/149732117-fcd2d804-dc2c-426c-bee7-'\
+ '94be6146e05c.png'
+ self.hair_img = load_image_from_disk_or_url(hair_img_path)
+
+ if light_video_path is None:
+ light_video_path = get_cached_file_path(
+ 'https://'
+ 'user-images.githubusercontent.com/11788150/149732080'
+ '-ea6cfeda-0dc5-4bbb-892a-3831e5580520.mp4')
+ self.light_video_path = light_video_path
+ self.light_video = cv2.VideoCapture(self.light_video_path)
+
+ def apply_saiyan_effect(self,
+ img,
+ pose_results,
+ saiyan_img,
+ light_frame,
+ face_indices,
+ bbox_thr=0.3,
+ kpt_thr=0.5):
+ """Apply saiyan hair effect.
+
+ Args:
+ img (np.ndarray): Image data.
+ pose_results (list[dict]): The pose estimation results containing:
+ - "keypoints" ([K,3]): keypoint detection result
+ in [x, y, score]
+ saiyan_img (np.ndarray): Saiyan image with transparent background.
+ light_frame (np.ndarray): Light image with green screen.
+ face_indices (int): Keypoint index of the face
+ kpt_thr (float): The score threshold of required keypoints.
+ """
+ img = img.copy()
+ im_shape = img.shape
+ # Apply lightning effects.
+ light_mask = screen_matting(light_frame, color='green')
+
+ # anchor points in the mask
+ pts_src = np.array(
+ [
+ [84, 398], # face kpt 0
+ [331, 393], # face kpt 16
+ [84, 145],
+ [331, 140]
+ ],
+ dtype=np.float32)
+
+ for pose in pose_results:
+ bbox = pose['bbox']
+
+ if bbox[-1] < bbox_thr:
+ continue
+
+ mask_inst = pose['mask']
+ # cache
+ fg = img[np.where(mask_inst)]
+
+ bbox = expand_and_clamp(bbox[:4], im_shape, s=3.0)
+ # Apply light effects between fg and bg
+ img = copy_and_paste(
+ light_frame,
+ img,
+ light_mask,
+ effect_region=(bbox[0] / im_shape[1], bbox[1] / im_shape[0],
+ bbox[2] / im_shape[1], bbox[3] / im_shape[0]))
+ # pop
+ img[np.where(mask_inst)] = fg
+
+ # Apply Saiyan hair effects
+ kpts = pose['keypoints']
+ if kpts[face_indices[0], 2] < kpt_thr or kpts[face_indices[16],
+ 2] < kpt_thr:
+ continue
+
+ kpt_0 = kpts[face_indices[0], :2]
+ kpt_16 = kpts[face_indices[16], :2]
+ # orthogonal vector
+ vo = (kpt_0 - kpt_16)[::-1] * [-1, 1]
+
+ # anchor points in the image by eye positions
+ pts_tar = np.vstack([kpt_0, kpt_16, kpt_0 + vo, kpt_16 + vo])
+
+ h_mat, _ = cv2.findHomography(pts_src, pts_tar)
+ patch = cv2.warpPerspective(
+ saiyan_img,
+ h_mat,
+ dsize=(img.shape[1], img.shape[0]),
+ borderValue=(0, 0, 0))
+ mask_patch = cv2.cvtColor(patch, cv2.COLOR_BGR2GRAY)
+ mask_patch = (mask_patch > 1).astype(np.uint8)
+ img = cv2.copyTo(patch, mask_patch, img)
+
+ return img
+
+ def draw(self, frame_msg):
+ canvas = frame_msg.get_image()
+
+ det_results = frame_msg.get_detection_results()
+ if not det_results:
+ return canvas
+
+ pose_results = frame_msg.get_pose_results()
+ if not pose_results:
+ return canvas
+
+ for pose_result in pose_results:
+ model_cfg = pose_result['model_cfg']
+ preds = pose_result['preds']
+ face_indices = get_face_keypoint_ids(model_cfg)
+
+ ret, frame = self.light_video.read()
+ if not ret:
+ self.light_video = cv2.VideoCapture(self.light_video_path)
+ ret, frame = self.light_video.read()
+
+ canvas = self.apply_saiyan_effect(canvas, preds, self.hair_img,
+ frame, face_indices)
+
+ return canvas
+
+
+@NODES.register_module()
+class MoustacheNode(FrameDrawingNode):
+
+ def __init__(self,
+ name: str,
+ frame_buffer: str,
+ output_buffer: Union[str, List[str]],
+ enable_key: Optional[Union[str, int]] = None,
+ enable: bool = True,
+ src_img_path: Optional[str] = None):
+
+ super().__init__(name, frame_buffer, output_buffer, enable_key, enable)
+
+ if src_img_path is None:
+ src_img_path = 'https://user-images.githubusercontent.com/'\
+ '11788150/149732141-3afbab55-252a-428c-b6d8'\
+ '-0e352f432651.jpeg'
+ self.src_img = load_image_from_disk_or_url(src_img_path)
+
+ def apply_moustache_effect(self,
+ img,
+ pose_results,
+ moustache_img,
+ face_indices,
+ kpt_thr=0.5):
+ """Apply moustache effect.
+
+ Args:
+ img (np.ndarray): Image data.
+ pose_results (list[dict]): The pose estimation results containing:
+ - "keypoints" ([K,3]): keypoint detection result
+ in [x, y, score]
+ moustache_img (np.ndarray): Moustache image with white background.
+ left_eye_index (int): Keypoint index of left eye
+ right_eye_index (int): Keypoint index of right eye
+ kpt_thr (float): The score threshold of required keypoints.
+ """
+
+ hm, wm = moustache_img.shape[:2]
+ # anchor points in the moustache mask
+ pts_src = np.array([[1164, 741], [1729, 741], [1164, 1244],
+ [1729, 1244]],
+ dtype=np.float32)
+
+ for pose in pose_results:
+ kpts = pose['keypoints']
+ if kpts[face_indices[32], 2] < kpt_thr \
+ or kpts[face_indices[34], 2] < kpt_thr \
+ or kpts[face_indices[61], 2] < kpt_thr \
+ or kpts[face_indices[63], 2] < kpt_thr:
+ continue
+
+ kpt_32 = kpts[face_indices[32], :2]
+ kpt_34 = kpts[face_indices[34], :2]
+ kpt_61 = kpts[face_indices[61], :2]
+ kpt_63 = kpts[face_indices[63], :2]
+ # anchor points in the image by eye positions
+ pts_tar = np.vstack([kpt_32, kpt_34, kpt_61, kpt_63])
+
+ h_mat, _ = cv2.findHomography(pts_src, pts_tar)
+ patch = cv2.warpPerspective(
+ moustache_img,
+ h_mat,
+ dsize=(img.shape[1], img.shape[0]),
+ borderValue=(255, 255, 255))
+ # mask the white background area in the patch with a threshold 200
+ mask = cv2.cvtColor(patch, cv2.COLOR_BGR2GRAY)
+ mask = (mask < 200).astype(np.uint8)
+ img = cv2.copyTo(patch, mask, img)
+
+ return img
+
+ def draw(self, frame_msg):
+ canvas = frame_msg.get_image()
+ pose_results = frame_msg.get_pose_results()
+ if not pose_results:
+ return canvas
+ for pose_result in pose_results:
+ model_cfg = pose_result['model_cfg']
+ preds = pose_result['preds']
+ face_indices = get_face_keypoint_ids(model_cfg)
+ canvas = self.apply_moustache_effect(canvas, preds, self.src_img,
+ face_indices)
+ return canvas
+
+
+@NODES.register_module()
+class BugEyeNode(FrameDrawingNode):
+
+ def draw(self, frame_msg):
+ canvas = frame_msg.get_image()
+ pose_results = frame_msg.get_pose_results()
+ if not pose_results:
+ return canvas
+ for pose_result in pose_results:
+ model_cfg = pose_result['model_cfg']
+ preds = pose_result['preds']
+ left_eye_idx, right_eye_idx = get_eye_keypoint_ids(model_cfg)
+
+ canvas = apply_bugeye_effect(canvas, preds, left_eye_idx,
+ right_eye_idx)
+ return canvas
+
+
+@NODES.register_module()
+class NoticeBoardNode(FrameDrawingNode):
+
+ default_content_lines = ['This is a notice board!']
+
+ def __init__(
+ self,
+ name: str,
+ frame_buffer: str,
+ output_buffer: Union[str, List[str]],
+ enable_key: Optional[Union[str, int]] = None,
+ enable: bool = True,
+ content_lines: Optional[List[str]] = None,
+ x_offset: int = 20,
+ y_offset: int = 20,
+ y_delta: int = 15,
+ text_color: Union[str, Tuple[int, int, int]] = 'black',
+ background_color: Union[str, Tuple[int, int, int]] = (255, 183, 0),
+ text_scale: float = 0.4,
+ ):
+ super().__init__(name, frame_buffer, output_buffer, enable_key, enable)
+
+ self.x_offset = x_offset
+ self.y_offset = y_offset
+ self.y_delta = y_delta
+ self.text_color = color_val(text_color)
+ self.background_color = color_val(background_color)
+ self.text_scale = text_scale
+
+ if content_lines:
+ self.content_lines = content_lines
+ else:
+ self.content_lines = self.default_content_lines
+
+ def draw(self, frame_msg: FrameMessage) -> np.ndarray:
+ img = frame_msg.get_image()
+ canvas = np.full(img.shape, self.background_color, dtype=img.dtype)
+
+ x = self.x_offset
+ y = self.y_offset
+
+ max_len = max([len(line) for line in self.content_lines])
+
+ def _put_line(line=''):
+ nonlocal y
+ cv2.putText(canvas, line, (x, y), cv2.FONT_HERSHEY_DUPLEX,
+ self.text_scale, self.text_color, 1)
+ y += self.y_delta
+
+ for line in self.content_lines:
+ _put_line(line)
+
+ x1 = max(0, self.x_offset)
+ x2 = min(img.shape[1], int(x + max_len * self.text_scale * 20))
+ y1 = max(0, self.y_offset - self.y_delta)
+ y2 = min(img.shape[0], y)
+
+ src1 = canvas[y1:y2, x1:x2]
+ src2 = img[y1:y2, x1:x2]
+ img[y1:y2, x1:x2] = cv2.addWeighted(src1, 0.5, src2, 0.5, 0)
+
+ return img
+
+
+@NODES.register_module()
+class HatNode(FrameDrawingNode):
+
+ def __init__(self,
+ name: str,
+ frame_buffer: str,
+ output_buffer: Union[str, List[str]],
+ enable_key: Optional[Union[str, int]] = None,
+ src_img_path: Optional[str] = None):
+
+ super().__init__(name, frame_buffer, output_buffer, enable_key)
+
+ if src_img_path is None:
+ # The image attributes to:
+ # http://616pic.com/sucai/1m9i70p52.html
+ src_img_path = 'https://user-images.githubusercontent.' \
+ 'com/28900607/149766271-2f591c19-9b67-4' \
+ 'd92-8f94-c272396ca141.png'
+ self.src_img = load_image_from_disk_or_url(src_img_path,
+ cv2.IMREAD_UNCHANGED)
+
+ @staticmethod
+ def apply_hat_effect(img,
+ pose_results,
+ hat_img,
+ left_eye_index,
+ right_eye_index,
+ kpt_thr=0.5):
+ """Apply hat effect.
+ Args:
+ img (np.ndarray): Image data.
+ pose_results (list[dict]): The pose estimation results containing:
+ - "keypoints" ([K,3]): keypoint detection result in
+ [x, y, score]
+ hat_img (np.ndarray): Hat image with white alpha channel.
+ left_eye_index (int): Keypoint index of left eye
+ right_eye_index (int): Keypoint index of right eye
+ kpt_thr (float): The score threshold of required keypoints.
+ """
+ img_orig = img.copy()
+
+ img = img_orig.copy()
+ hm, wm = hat_img.shape[:2]
+ # anchor points in the sunglasses mask
+ a = 0.3
+ b = 0.7
+ pts_src = np.array([[a * wm, a * hm], [a * wm, b * hm],
+ [b * wm, a * hm], [b * wm, b * hm]],
+ dtype=np.float32)
+
+ for pose in pose_results:
+ kpts = pose['keypoints']
+
+ if kpts[left_eye_index, 2] < kpt_thr or \
+ kpts[right_eye_index, 2] < kpt_thr:
+ continue
+
+ kpt_leye = kpts[left_eye_index, :2]
+ kpt_reye = kpts[right_eye_index, :2]
+ # orthogonal vector to the left-to-right eyes
+ vo = 0.5 * (kpt_reye - kpt_leye)[::-1] * [-1, 1]
+ veye = 0.5 * (kpt_reye - kpt_leye)
+
+ # anchor points in the image by eye positions
+ pts_tar = np.vstack([
+ kpt_reye + 1 * veye + 5 * vo, kpt_reye + 1 * veye + 1 * vo,
+ kpt_leye - 1 * veye + 5 * vo, kpt_leye - 1 * veye + 1 * vo
+ ])
+
+ h_mat, _ = cv2.findHomography(pts_src, pts_tar)
+ patch = cv2.warpPerspective(
+ hat_img,
+ h_mat,
+ dsize=(img.shape[1], img.shape[0]),
+ borderValue=(255, 255, 255))
+ # mask the white background area in the patch with a threshold 200
+ mask = (patch[:, :, -1] > 128)
+ patch = patch[:, :, :-1]
+ mask = mask * (cv2.cvtColor(patch, cv2.COLOR_BGR2GRAY) > 30)
+ mask = mask.astype(np.uint8)
+
+ img = cv2.copyTo(patch, mask, img)
+ return img
+
+ def draw(self, frame_msg):
+ canvas = frame_msg.get_image()
+ pose_results = frame_msg.get_pose_results()
+ if not pose_results:
+ return canvas
+ for pose_result in pose_results:
+ model_cfg = pose_result['model_cfg']
+ preds = pose_result['preds']
+ left_eye_idx, right_eye_idx = get_eye_keypoint_ids(model_cfg)
+
+ canvas = self.apply_hat_effect(canvas, preds, self.src_img,
+ left_eye_idx, right_eye_idx)
+ return canvas
+
+
+@NODES.register_module()
+class FirecrackerNode(FrameDrawingNode):
+
+ def __init__(self,
+ name: str,
+ frame_buffer: str,
+ output_buffer: Union[str, List[str]],
+ enable_key: Optional[Union[str, int]] = None,
+ src_img_path: Optional[str] = None):
+
+ super().__init__(name, frame_buffer, output_buffer, enable_key)
+
+ if src_img_path is None:
+ self.src_img_path = 'https://user-images.githubusercontent' \
+ '.com/28900607/149766281-6376055c-ed8b' \
+ '-472b-991f-60e6ae6ee1da.gif'
+ src_img = cv2.VideoCapture(self.src_img_path)
+
+ self.frame_list = []
+ ret, frame = src_img.read()
+ while frame is not None:
+ self.frame_list.append(frame)
+ ret, frame = src_img.read()
+ self.num_frames = len(self.frame_list)
+ self.frame_idx = 0
+ self.frame_period = 4 # each frame in gif lasts for 4 frames in video
+
+ @staticmethod
+ def apply_firecracker_effect(img,
+ pose_results,
+ firecracker_img,
+ left_wrist_idx,
+ right_wrist_idx,
+ kpt_thr=0.5):
+ """Apply firecracker effect.
+ Args:
+ img (np.ndarray): Image data.
+ pose_results (list[dict]): The pose estimation results containing:
+ - "keypoints" ([K,3]): keypoint detection result in
+ [x, y, score]
+ firecracker_img (np.ndarray): Firecracker image with white
+ background.
+ left_wrist_idx (int): Keypoint index of left wrist
+ right_wrist_idx (int): Keypoint index of right wrist
+ kpt_thr (float): The score threshold of required keypoints.
+ """
+
+ hm, wm = firecracker_img.shape[:2]
+ # anchor points in the firecracker mask
+ pts_src = np.array([[0. * wm, 0. * hm], [0. * wm, 1. * hm],
+ [1. * wm, 0. * hm], [1. * wm, 1. * hm]],
+ dtype=np.float32)
+
+ h, w = img.shape[:2]
+ h_tar = h / 3
+ w_tar = h_tar / hm * wm
+
+ for pose in pose_results:
+ kpts = pose['keypoints']
+
+ if kpts[left_wrist_idx, 2] > kpt_thr:
+ kpt_lwrist = kpts[left_wrist_idx, :2]
+ # anchor points in the image by eye positions
+ pts_tar = np.vstack([
+ kpt_lwrist - [w_tar / 2, 0],
+ kpt_lwrist - [w_tar / 2, -h_tar],
+ kpt_lwrist + [w_tar / 2, 0],
+ kpt_lwrist + [w_tar / 2, h_tar]
+ ])
+
+ h_mat, _ = cv2.findHomography(pts_src, pts_tar)
+ patch = cv2.warpPerspective(
+ firecracker_img,
+ h_mat,
+ dsize=(img.shape[1], img.shape[0]),
+ borderValue=(255, 255, 255))
+ # mask the white background area in the patch with
+ # a threshold 200
+ mask = cv2.cvtColor(patch, cv2.COLOR_BGR2GRAY)
+ mask = (mask < 240).astype(np.uint8)
+ img = cv2.copyTo(patch, mask, img)
+
+ if kpts[right_wrist_idx, 2] > kpt_thr:
+ kpt_rwrist = kpts[right_wrist_idx, :2]
+
+ # anchor points in the image by eye positions
+ pts_tar = np.vstack([
+ kpt_rwrist - [w_tar / 2, 0],
+ kpt_rwrist - [w_tar / 2, -h_tar],
+ kpt_rwrist + [w_tar / 2, 0],
+ kpt_rwrist + [w_tar / 2, h_tar]
+ ])
+
+ h_mat, _ = cv2.findHomography(pts_src, pts_tar)
+ patch = cv2.warpPerspective(
+ firecracker_img,
+ h_mat,
+ dsize=(img.shape[1], img.shape[0]),
+ borderValue=(255, 255, 255))
+ # mask the white background area in the patch with
+ # a threshold 200
+ mask = cv2.cvtColor(patch, cv2.COLOR_BGR2GRAY)
+ mask = (mask < 240).astype(np.uint8)
+ img = cv2.copyTo(patch, mask, img)
+
+ return img
+
+ def draw(self, frame_msg):
+ canvas = frame_msg.get_image()
+ pose_results = frame_msg.get_pose_results()
+ if not pose_results:
+ return canvas
+
+ frame = self.frame_list[self.frame_idx // self.frame_period]
+ for pose_result in pose_results:
+ model_cfg = pose_result['model_cfg']
+ preds = pose_result['preds']
+ left_wrist_idx, right_wrist_idx = get_wrist_keypoint_ids(model_cfg)
+
+ canvas = self.apply_firecracker_effect(canvas, preds, frame,
+ left_wrist_idx,
+ right_wrist_idx)
+ self.frame_idx = (self.frame_idx + 1) % (
+ self.num_frames * self.frame_period)
+
+ return canvas
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/nodes/helper_node.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/nodes/helper_node.py
new file mode 100644
index 0000000..349c4f4
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/nodes/helper_node.py
@@ -0,0 +1,296 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import logging
+import time
+from queue import Full, Queue
+from threading import Thread
+from typing import List, Optional, Union
+
+import cv2
+import numpy as np
+from mmcv import color_val
+
+from mmpose.utils.timer import RunningAverage
+from .builder import NODES
+from .node import Node
+
+try:
+ import psutil
+ psutil_proc = psutil.Process()
+except (ImportError, ModuleNotFoundError):
+ psutil_proc = None
+
+
+@NODES.register_module()
+class ModelResultBindingNode(Node):
+
+ def __init__(self, name: str, frame_buffer: str, result_buffer: str,
+ output_buffer: Union[str, List[str]]):
+ super().__init__(name=name, enable=True)
+ self.synchronous = None
+
+ # Cache the latest model result
+ self.last_result_msg = None
+ self.last_output_msg = None
+
+ # Inference speed analysis
+ self.frame_fps = RunningAverage(window=10)
+ self.frame_lag = RunningAverage(window=10)
+ self.result_fps = RunningAverage(window=10)
+ self.result_lag = RunningAverage(window=10)
+
+ # Register buffers
+ # Note that essential buffers will be set in set_runner() because
+ # it depends on the runner.synchronous attribute.
+ self.register_input_buffer(result_buffer, 'result', essential=False)
+ self.register_input_buffer(frame_buffer, 'frame', essential=False)
+ self.register_output_buffer(output_buffer)
+
+ def set_runner(self, runner):
+ super().set_runner(runner)
+
+ # Set synchronous according to the runner
+ if runner.synchronous:
+ self.synchronous = True
+ essential_input = 'result'
+ else:
+ self.synchronous = False
+ essential_input = 'frame'
+
+ # Set essential input buffer according to the synchronous setting
+ for buffer_info in self._input_buffers:
+ if buffer_info.input_name == essential_input:
+ buffer_info.essential = True
+
+ def process(self, input_msgs):
+ result_msg = input_msgs['result']
+
+ # Update last result
+ if result_msg is not None:
+ # Update result FPS
+ if self.last_result_msg is not None:
+ self.result_fps.update(
+ 1.0 /
+ (result_msg.timestamp - self.last_result_msg.timestamp))
+ # Update inference latency
+ self.result_lag.update(time.time() - result_msg.timestamp)
+ # Update last inference result
+ self.last_result_msg = result_msg
+
+ if not self.synchronous:
+ # Asynchronous mode: Bind the latest result with the current frame.
+ frame_msg = input_msgs['frame']
+
+ self.frame_lag.update(time.time() - frame_msg.timestamp)
+
+ # Bind result to frame
+ if self.last_result_msg is not None:
+ frame_msg.set_full_results(
+ self.last_result_msg.get_full_results())
+ frame_msg.merge_route_info(
+ self.last_result_msg.get_route_info())
+
+ output_msg = frame_msg
+
+ else:
+ # Synchronous mode: Directly output the frame that the model result
+ # was obtained from.
+ self.frame_lag.update(time.time() - result_msg.timestamp)
+ output_msg = result_msg
+
+ # Update frame fps and lag
+ if self.last_output_msg is not None:
+ self.frame_lag.update(time.time() - output_msg.timestamp)
+ self.frame_fps.update(
+ 1.0 / (output_msg.timestamp - self.last_output_msg.timestamp))
+ self.last_output_msg = output_msg
+
+ return output_msg
+
+ def _get_node_info(self):
+ info = super()._get_node_info()
+ info['result_fps'] = self.result_fps.average()
+ info['result_lag (ms)'] = self.result_lag.average() * 1000
+ info['frame_fps'] = self.frame_fps.average()
+ info['frame_lag (ms)'] = self.frame_lag.average() * 1000
+ return info
+
+
+@NODES.register_module()
+class MonitorNode(Node):
+
+ _default_ignore_items = ['timestamp']
+
+ def __init__(self,
+ name: str,
+ frame_buffer: str,
+ output_buffer: Union[str, List[str]],
+ enable_key: Optional[Union[str, int]] = None,
+ enable: bool = False,
+ x_offset=20,
+ y_offset=20,
+ y_delta=15,
+ text_color='black',
+ background_color=(255, 183, 0),
+ text_scale=0.4,
+ ignore_items: Optional[List[str]] = None):
+ super().__init__(name=name, enable_key=enable_key, enable=enable)
+
+ self.x_offset = x_offset
+ self.y_offset = y_offset
+ self.y_delta = y_delta
+ self.text_color = color_val(text_color)
+ self.background_color = color_val(background_color)
+ self.text_scale = text_scale
+ if ignore_items is None:
+ self.ignore_items = self._default_ignore_items
+ else:
+ self.ignore_items = ignore_items
+
+ self.register_input_buffer(frame_buffer, 'frame', essential=True)
+ self.register_output_buffer(output_buffer)
+
+ def process(self, input_msgs):
+ frame_msg = input_msgs['frame']
+
+ frame_msg.update_route_info(
+ node_name='System Info',
+ node_type='dummy',
+ info=self._get_system_info())
+
+ img = frame_msg.get_image()
+ route_info = frame_msg.get_route_info()
+ img = self._show_route_info(img, route_info)
+
+ frame_msg.set_image(img)
+ return frame_msg
+
+ def _get_system_info(self):
+ sys_info = {}
+ if psutil_proc is not None:
+ sys_info['CPU(%)'] = psutil_proc.cpu_percent()
+ sys_info['Memory(%)'] = psutil_proc.memory_percent()
+ return sys_info
+
+ def _show_route_info(self, img, route_info):
+ canvas = np.full(img.shape, self.background_color, dtype=img.dtype)
+
+ x = self.x_offset
+ y = self.y_offset
+
+ max_len = 0
+
+ def _put_line(line=''):
+ nonlocal y, max_len
+ cv2.putText(canvas, line, (x, y), cv2.FONT_HERSHEY_DUPLEX,
+ self.text_scale, self.text_color, 1)
+ y += self.y_delta
+ max_len = max(max_len, len(line))
+
+ for node_info in route_info:
+ title = f'{node_info["node"]}({node_info["node_type"]})'
+ _put_line(title)
+ for k, v in node_info['info'].items():
+ if k in self.ignore_items:
+ continue
+ if isinstance(v, float):
+ v = f'{v:.1f}'
+ _put_line(f' {k}: {v}')
+
+ x1 = max(0, self.x_offset)
+ x2 = min(img.shape[1], int(x + max_len * self.text_scale * 20))
+ y1 = max(0, self.y_offset - self.y_delta)
+ y2 = min(img.shape[0], y)
+
+ src1 = canvas[y1:y2, x1:x2]
+ src2 = img[y1:y2, x1:x2]
+ img[y1:y2, x1:x2] = cv2.addWeighted(src1, 0.5, src2, 0.5, 0)
+
+ return img
+
+ def bypass(self, input_msgs):
+ return input_msgs['frame']
+
+
+@NODES.register_module()
+class RecorderNode(Node):
+ """Record the frames into a local file."""
+
+ def __init__(
+ self,
+ name: str,
+ frame_buffer: str,
+ output_buffer: Union[str, List[str]],
+ out_video_file: str,
+ out_video_fps: int = 30,
+ out_video_codec: str = 'mp4v',
+ buffer_size: int = 30,
+ ):
+ super().__init__(name=name, enable_key=None, enable=True)
+
+ self.queue = Queue(maxsize=buffer_size)
+ self.out_video_file = out_video_file
+ self.out_video_fps = out_video_fps
+ self.out_video_codec = out_video_codec
+ self.vwriter = None
+
+ # Register buffers
+ self.register_input_buffer(frame_buffer, 'frame', essential=True)
+ self.register_output_buffer(output_buffer)
+
+ # Start a new thread to write frame
+ self.t_record = Thread(target=self._record, args=(), daemon=True)
+ self.t_record.start()
+
+ def process(self, input_msgs):
+
+ frame_msg = input_msgs['frame']
+ img = frame_msg.get_image() if frame_msg is not None else None
+ img_queued = False
+
+ while not img_queued:
+ try:
+ self.queue.put(img, timeout=1)
+ img_queued = True
+ logging.info(f'{self.name}: recorder received one frame!')
+ except Full:
+ logging.info(f'{self.name}: recorder jamed!')
+
+ return frame_msg
+
+ def _record(self):
+
+ while True:
+
+ img = self.queue.get()
+
+ if img is None:
+ break
+
+ if self.vwriter is None:
+ fourcc = cv2.VideoWriter_fourcc(*self.out_video_codec)
+ fps = self.out_video_fps
+ frame_size = (img.shape[1], img.shape[0])
+ self.vwriter = cv2.VideoWriter(self.out_video_file, fourcc,
+ fps, frame_size)
+ assert self.vwriter.isOpened()
+
+ self.vwriter.write(img)
+
+ logging.info('Video recorder released!')
+ if self.vwriter is not None:
+ self.vwriter.release()
+
+ def on_exit(self):
+ try:
+ # Try putting a None into the output queue so the self.vwriter will
+ # be released after all queue frames have been written to file.
+ self.queue.put(None, timeout=1)
+ self.t_record.join(timeout=1)
+ except Full:
+ pass
+
+ if self.t_record.is_alive():
+ # Force to release self.vwriter
+ logging.info('Video recorder forced release!')
+ if self.vwriter is not None:
+ self.vwriter.release()
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/nodes/mmdet_node.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/nodes/mmdet_node.py
new file mode 100644
index 0000000..4207647
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/nodes/mmdet_node.py
@@ -0,0 +1,84 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from typing import List, Optional, Union
+
+from .builder import NODES
+from .node import Node
+
+try:
+ from mmdet.apis import inference_detector, init_detector
+ has_mmdet = True
+except (ImportError, ModuleNotFoundError):
+ has_mmdet = False
+
+
+@NODES.register_module()
+class DetectorNode(Node):
+
+ def __init__(self,
+ name: str,
+ model_config: str,
+ model_checkpoint: str,
+ input_buffer: str,
+ output_buffer: Union[str, List[str]],
+ enable_key: Optional[Union[str, int]] = None,
+ device: str = 'cuda:0'):
+ # Check mmdetection is installed
+ assert has_mmdet, 'Please install mmdet to run the demo.'
+ super().__init__(name=name, enable_key=enable_key, enable=True)
+
+ self.model_config = model_config
+ self.model_checkpoint = model_checkpoint
+ self.device = device.lower()
+
+ # Init model
+ self.model = init_detector(
+ self.model_config,
+ self.model_checkpoint,
+ device=self.device.lower())
+
+ # Register buffers
+ self.register_input_buffer(input_buffer, 'input', essential=True)
+ self.register_output_buffer(output_buffer)
+
+ def bypass(self, input_msgs):
+ return input_msgs['input']
+
+ def process(self, input_msgs):
+ input_msg = input_msgs['input']
+
+ img = input_msg.get_image()
+
+ preds = inference_detector(self.model, img)
+ det_result = self._post_process(preds)
+
+ input_msg.add_detection_result(det_result, tag=self.name)
+ return input_msg
+
+ def _post_process(self, preds):
+ if isinstance(preds, tuple):
+ dets = preds[0]
+ segms = preds[1]
+ else:
+ dets = preds
+ segms = [None] * len(dets)
+
+ assert len(dets) == len(self.model.CLASSES)
+ assert len(segms) == len(self.model.CLASSES)
+ result = {'preds': [], 'model_cfg': self.model.cfg.copy()}
+
+ for i, (cls_name, bboxes,
+ masks) in enumerate(zip(self.model.CLASSES, dets, segms)):
+ if masks is None:
+ masks = [None] * len(bboxes)
+ else:
+ assert len(masks) == len(bboxes)
+
+ preds_i = [{
+ 'cls_id': i,
+ 'label': cls_name,
+ 'bbox': bbox,
+ 'mask': mask,
+ } for (bbox, mask) in zip(bboxes, masks)]
+ result['preds'].extend(preds_i)
+
+ return result
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/nodes/mmpose_node.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/nodes/mmpose_node.py
new file mode 100644
index 0000000..167d741
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/nodes/mmpose_node.py
@@ -0,0 +1,122 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import time
+from typing import Dict, List, Optional, Union
+
+from mmpose.apis import (get_track_id, inference_top_down_pose_model,
+ init_pose_model)
+from ..utils import Message
+from .builder import NODES
+from .node import Node
+
+
+@NODES.register_module()
+class TopDownPoseEstimatorNode(Node):
+
+ def __init__(self,
+ name: str,
+ model_config: str,
+ model_checkpoint: str,
+ input_buffer: str,
+ output_buffer: Union[str, List[str]],
+ enable_key: Optional[Union[str, int]] = None,
+ enable: bool = True,
+ device: str = 'cuda:0',
+ cls_ids: Optional[List] = None,
+ cls_names: Optional[List] = None,
+ bbox_thr: float = 0.5):
+ super().__init__(name=name, enable_key=enable_key, enable=enable)
+
+ # Init model
+ self.model_config = model_config
+ self.model_checkpoint = model_checkpoint
+ self.device = device.lower()
+
+ self.cls_ids = cls_ids
+ self.cls_names = cls_names
+ self.bbox_thr = bbox_thr
+
+ # Init model
+ self.model = init_pose_model(
+ self.model_config,
+ self.model_checkpoint,
+ device=self.device.lower())
+
+ # Store history for pose tracking
+ self.track_info = {
+ 'next_id': 0,
+ 'last_pose_preds': [],
+ 'last_time': None
+ }
+
+ # Register buffers
+ self.register_input_buffer(input_buffer, 'input', essential=True)
+ self.register_output_buffer(output_buffer)
+
+ def bypass(self, input_msgs):
+ return input_msgs['input']
+
+ def process(self, input_msgs: Dict[str, Message]) -> Message:
+
+ input_msg = input_msgs['input']
+ img = input_msg.get_image()
+ det_results = input_msg.get_detection_results()
+
+ if det_results is None:
+ raise ValueError(
+ 'No detection results are found in the frame message.'
+ f'{self.__class__.__name__} should be used after a '
+ 'detector node.')
+
+ full_det_preds = []
+ for det_result in det_results:
+ det_preds = det_result['preds']
+ if self.cls_ids:
+ # Filter detection results by class ID
+ det_preds = [
+ p for p in det_preds if p['cls_id'] in self.cls_ids
+ ]
+ elif self.cls_names:
+ # Filter detection results by class name
+ det_preds = [
+ p for p in det_preds if p['label'] in self.cls_names
+ ]
+ full_det_preds.extend(det_preds)
+
+ # Inference pose
+ pose_preds, _ = inference_top_down_pose_model(
+ self.model,
+ img,
+ full_det_preds,
+ bbox_thr=self.bbox_thr,
+ format='xyxy')
+
+ # Pose tracking
+ current_time = time.time()
+ if self.track_info['last_time'] is None:
+ fps = None
+ elif self.track_info['last_time'] >= current_time:
+ fps = None
+ else:
+ fps = 1.0 / (current_time - self.track_info['last_time'])
+
+ pose_preds, next_id = get_track_id(
+ pose_preds,
+ self.track_info['last_pose_preds'],
+ self.track_info['next_id'],
+ use_oks=False,
+ tracking_thr=0.3,
+ use_one_euro=True,
+ fps=fps)
+
+ self.track_info['next_id'] = next_id
+ self.track_info['last_pose_preds'] = pose_preds.copy()
+ self.track_info['last_time'] = current_time
+
+ pose_result = {
+ 'preds': pose_preds,
+ 'model_cfg': self.model.cfg.copy(),
+ }
+
+ input_msg.add_pose_result(pose_result, tag=self.name)
+
+ return input_msg
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/nodes/node.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/nodes/node.py
new file mode 100644
index 0000000..31e48d0
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/nodes/node.py
@@ -0,0 +1,372 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import logging
+import time
+from abc import ABCMeta, abstractmethod
+from dataclasses import dataclass
+from queue import Empty
+from threading import Thread
+from typing import Callable, Dict, List, Optional, Tuple, Union
+
+from mmcv.utils.misc import is_method_overridden
+
+from mmpose.utils import StopWatch
+from ..utils import Message, VideoEndingMessage, limit_max_fps
+
+
+@dataclass
+class BufferInfo():
+ """Dataclass for buffer information."""
+ buffer_name: str
+ input_name: Optional[str] = None
+ essential: bool = False
+
+
+@dataclass
+class EventInfo():
+ """Dataclass for event handler information."""
+ event_name: str
+ is_keyboard: bool = False
+ handler_func: Optional[Callable] = None
+
+
+class Node(Thread, metaclass=ABCMeta):
+ """Base interface of functional module.
+
+ Parameters:
+ name (str, optional): The node name (also thread name).
+ enable_key (str|int, optional): Set a hot-key to toggle enable/disable
+ of the node. If an int value is given, it will be treated as an
+ ascii code of a key. Please note:
+ 1. If enable_key is set, the bypass method need to be
+ overridden to define the node behavior when disabled
+ 2. Some hot-key has been use for particular use. For example:
+ 'q', 'Q' and 27 are used for quit
+ Default: None
+ max_fps (int): Maximum FPS of the node. This is to avoid the node
+ running unrestrictedly and causing large resource consuming.
+ Default: 30
+ input_check_interval (float): Minimum interval (in millisecond) between
+ checking if input is ready. Default: 0.001
+ enable (bool): Default enable/disable status. Default: True.
+ daemon (bool): Whether node is a daemon. Default: True.
+ """
+
+ def __init__(self,
+ name: Optional[str] = None,
+ enable_key: Optional[Union[str, int]] = None,
+ max_fps: int = 30,
+ input_check_interval: float = 0.01,
+ enable: bool = True,
+ daemon=False):
+ super().__init__(name=name, daemon=daemon)
+ self._runner = None
+ self._enabled = enable
+ self.enable_key = enable_key
+ self.max_fps = max_fps
+ self.input_check_interval = input_check_interval
+
+ # A partitioned buffer manager the runner's buffer manager that
+ # only accesses the buffers related to the node
+ self._buffer_manager = None
+
+ # Input/output buffers are a list of registered buffers' information
+ self._input_buffers = []
+ self._output_buffers = []
+
+ # Event manager is a copy of assigned runner's event manager
+ self._event_manager = None
+
+ # A list of registered event information
+ # See register_event() for more information
+ # Note that we recommend to handle events in nodes by registering
+ # handlers, but one can still access the raw event by _event_manager
+ self._registered_events = []
+
+ # A list of (listener_threads, event_info)
+ # See set_runner() for more information
+ self._event_listener_threads = []
+
+ # A timer to calculate node FPS
+ self._timer = StopWatch(window=10)
+
+ # Register enable toggle key
+ if self.enable_key:
+ # If the node allows toggling enable, it should override the
+ # `bypass` method to define the node behavior when disabled.
+ if not is_method_overridden('bypass', Node, self.__class__):
+ raise NotImplementedError(
+ f'The node {self.__class__} does not support toggling'
+ 'enable but got argument `enable_key`. To support toggling'
+ 'enable, please override the `bypass` method of the node.')
+
+ self.register_event(
+ event_name=self.enable_key,
+ is_keyboard=True,
+ handler_func=self._toggle_enable,
+ )
+
+ @property
+ def registered_buffers(self):
+ return self._input_buffers + self._output_buffers
+
+ @property
+ def registered_events(self):
+ return self._registered_events.copy()
+
+ def _toggle_enable(self):
+ self._enabled = not self._enabled
+
+ def register_input_buffer(self,
+ buffer_name: str,
+ input_name: str,
+ essential: bool = False):
+ """Register an input buffer, so that Node can automatically check if
+ data is ready, fetch data from the buffers and format the inputs to
+ feed into `process` method.
+
+ This method can be invoked multiple times to register multiple input
+ buffers.
+
+ The subclass of Node should invoke `register_input_buffer` in its
+ `__init__` method.
+
+ Args:
+ buffer_name (str): The name of the buffer
+ input_name (str): The name of the fetched message from the
+ corresponding buffer
+ essential (bool): An essential input means the node will wait
+ until the input is ready before processing. Otherwise, an
+ inessential input will not block the processing, instead
+ a None will be fetched if the buffer is not ready.
+ """
+ buffer_info = BufferInfo(buffer_name, input_name, essential)
+ self._input_buffers.append(buffer_info)
+
+ def register_output_buffer(self, buffer_name: Union[str, List[str]]):
+ """Register one or multiple output buffers, so that the Node can
+ automatically send the output of the `process` method to these buffers.
+
+ The subclass of Node should invoke `register_output_buffer` in its
+ `__init__` method.
+
+ Args:
+ buffer_name (str|list): The name(s) of the output buffer(s).
+ """
+
+ if not isinstance(buffer_name, list):
+ buffer_name = [buffer_name]
+
+ for name in buffer_name:
+ buffer_info = BufferInfo(name)
+ self._output_buffers.append(buffer_info)
+
+ def register_event(self,
+ event_name: str,
+ is_keyboard: bool = False,
+ handler_func: Optional[Callable] = None):
+ """Register an event. All events used in the node need to be registered
+ in __init__(). If a callable handler is given, a thread will be create
+ to listen and handle the event when the node starts.
+
+ Args:
+ Args:
+ event_name (str|int): The event name. If is_keyboard==True,
+ event_name should be a str (as char) or an int (as ascii)
+ is_keyboard (bool): Indicate whether it is an keyboard
+ event. If True, the argument event_name will be regarded as a
+ key indicator.
+ handler_func (callable, optional): The event handler function,
+ which should be a collable object with no arguments or
+ return values. Default: None.
+ """
+ event_info = EventInfo(event_name, is_keyboard, handler_func)
+ self._registered_events.append(event_info)
+
+ def set_runner(self, runner):
+ # Get partitioned buffer manager
+ buffer_names = [
+ buffer.buffer_name
+ for buffer in self._input_buffers + self._output_buffers
+ ]
+ self._buffer_manager = runner.buffer_manager.get_sub_manager(
+ buffer_names)
+
+ # Get event manager
+ self._event_manager = runner.event_manager
+
+ def _get_input_from_buffer(self) -> Tuple[bool, Optional[Dict]]:
+ """Get and pack input data if it's ready. The function returns a tuple
+ of a status flag and a packed data dictionary. If input_buffer is
+ ready, the status flag will be True, and the packed data is a dict
+ whose items are buffer names and corresponding messages (unready
+ additional buffers will give a `None`). Otherwise, the status flag is
+ False and the packed data is None.
+
+ Returns:
+ bool: status flag
+ dict[str, Message]: the packed inputs where the key is the buffer
+ name and the value is the Message got from the corresponding
+ buffer.
+ """
+ buffer_manager = self._buffer_manager
+
+ if buffer_manager is None:
+ raise ValueError(f'{self.name}: Runner not set!')
+
+ # Check that essential buffers are ready
+ for buffer_info in self._input_buffers:
+ if buffer_info.essential and buffer_manager.is_empty(
+ buffer_info.buffer_name):
+ return False, None
+
+ # Default input
+ result = {
+ buffer_info.input_name: None
+ for buffer_info in self._input_buffers
+ }
+
+ for buffer_info in self._input_buffers:
+ try:
+ result[buffer_info.input_name] = buffer_manager.get(
+ buffer_info.buffer_name, block=False)
+ except Empty:
+ if buffer_info.essential:
+ # Return unsuccessful flag if any
+ # essential input is unready
+ return False, None
+
+ return True, result
+
+ def _send_output_to_buffers(self, output_msg):
+ """Send output of the process method to registered output buffers.
+
+ Args:
+ output_msg (Message): output message
+ force (bool, optional): If True, block until the output message
+ has been put into all output buffers. Default: False
+ """
+ for buffer_info in self._output_buffers:
+ buffer_name = buffer_info.buffer_name
+ self._buffer_manager.put_force(buffer_name, output_msg)
+
+ @abstractmethod
+ def process(self, input_msgs: Dict[str, Message]) -> Union[Message, None]:
+ """The core method that implement the function of the node. This method
+ will be invoked when the node is enabled and the input data is ready.
+
+ All subclasses of Node should override this method.
+
+ Args:
+ input_msgs (dict): The input data collected from the buffers. For
+ each item, the key is the `input_name` of the registered input
+ buffer, while the value is a Message instance fetched from the
+ buffer (or None if the buffer is unessential and not ready).
+
+ Returns:
+ Message: The output message of the node. It will be send to all
+ registered output buffers.
+ """
+
+ def bypass(self, input_msgs: Dict[str, Message]) -> Union[Message, None]:
+ """The method that defines the node behavior when disabled. Note that
+ if the node has an `enable_key`, this method should be override.
+
+ The method input/output is same as it of `process` method.
+
+ Args:
+ input_msgs (dict): The input data collected from the buffers. For
+ each item, the key is the `input_name` of the registered input
+ buffer, while the value is a Message instance fetched from the
+ buffer (or None if the buffer is unessential and not ready).
+
+ Returns:
+ Message: The output message of the node. It will be send to all
+ registered output buffers.
+ """
+ raise NotImplementedError
+
+ def _get_node_info(self):
+ """Get route information of the node."""
+ info = {'fps': self._timer.report('_FPS_'), 'timestamp': time.time()}
+ return info
+
+ def on_exit(self):
+ """This method will be invoked on event `_exit_`.
+
+ Subclasses should override this method to specifying the exiting
+ behavior.
+ """
+
+ def run(self):
+ """Method representing the Node's activity.
+
+ This method override the standard run() method of Thread. Users should
+ not override this method in subclasses.
+ """
+
+ logging.info(f'Node {self.name} starts')
+
+ # Create event listener threads
+ for event_info in self._registered_events:
+
+ if event_info.handler_func is None:
+ continue
+
+ def event_listener():
+ while True:
+ with self._event_manager.wait_and_handle(
+ event_info.event_name, event_info.is_keyboard):
+ event_info.handler_func()
+
+ t_listener = Thread(target=event_listener, args=(), daemon=True)
+ t_listener.start()
+ self._event_listener_threads.append(t_listener)
+
+ # Loop
+ while True:
+ # Exit
+ if self._event_manager.is_set('_exit_'):
+ self.on_exit()
+ break
+
+ # Check if input is ready
+ input_status, input_msgs = self._get_input_from_buffer()
+
+ # Input is not ready
+ if not input_status:
+ time.sleep(self.input_check_interval)
+ continue
+
+ # If a VideoEndingMessage is received, broadcast the signal
+ # without invoking process() or bypass()
+ video_ending = False
+ for _, msg in input_msgs.items():
+ if isinstance(msg, VideoEndingMessage):
+ self._send_output_to_buffers(msg)
+ video_ending = True
+ break
+
+ if video_ending:
+ self.on_exit()
+ break
+
+ # Check if enabled
+ if not self._enabled:
+ # Override bypass method to define node behavior when disabled
+ output_msg = self.bypass(input_msgs)
+ else:
+ with self._timer.timeit():
+ with limit_max_fps(self.max_fps):
+ # Process
+ output_msg = self.process(input_msgs)
+
+ if output_msg:
+ # Update route information
+ node_info = self._get_node_info()
+ output_msg.update_route_info(node=self, info=node_info)
+
+ # Send output message
+ if output_msg is not None:
+ self._send_output_to_buffers(output_msg)
+
+ logging.info(f'{self.name}: process ending.')
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/nodes/valentinemagic_node.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/nodes/valentinemagic_node.py
new file mode 100644
index 0000000..8b1c6a5
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/nodes/valentinemagic_node.py
@@ -0,0 +1,340 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import time
+from dataclasses import dataclass
+from typing import Dict, List, Optional, Tuple, Union
+
+import cv2
+import numpy as np
+
+from ..utils import (FrameMessage, get_eye_keypoint_ids, get_hand_keypoint_ids,
+ get_mouth_keypoint_ids, load_image_from_disk_or_url)
+from .builder import NODES
+from .frame_drawing_node import FrameDrawingNode
+
+
+@dataclass
+class HeartInfo():
+ """Dataclass for heart information."""
+ heart_type: int
+ start_time: float
+ start_pos: Tuple[int, int]
+ end_pos: Tuple[int, int]
+
+
+@NODES.register_module()
+class ValentineMagicNode(FrameDrawingNode):
+
+ def __init__(self,
+ name: str,
+ frame_buffer: str,
+ output_buffer: Union[str, List[str]],
+ enable_key: Optional[Union[str, int]] = None,
+ kpt_vis_thr: float = 0.3,
+ hand_heart_angle_thr: float = 90.0,
+ longest_duration: float = 2.0,
+ largest_ratio: float = 0.25,
+ hand_heart_img_path: Optional[str] = None,
+ flying_heart_img_path: Optional[str] = None,
+ hand_heart_dis_ratio_thr: float = 1.0,
+ flying_heart_dis_ratio_thr: float = 3.5,
+ num_persons: int = 2):
+
+ super().__init__(
+ name, frame_buffer, output_buffer, enable_key=enable_key)
+
+ if hand_heart_img_path is None:
+ hand_heart_img_path = 'https://user-images.githubusercontent.com/'\
+ '87690686/149731850-ea946766-a4e8-4efa-82f5'\
+ '-e2f0515db8ae.png'
+ if flying_heart_img_path is None:
+ flying_heart_img_path = 'https://user-images.githubusercontent.'\
+ 'com/87690686/153554948-937ce496-33dd-4'\
+ '9ab-9829-0433fd7c13c4.png'
+
+ self.hand_heart = load_image_from_disk_or_url(hand_heart_img_path)
+ self.flying_heart = load_image_from_disk_or_url(flying_heart_img_path)
+
+ self.kpt_vis_thr = kpt_vis_thr
+ self.hand_heart_angle_thr = hand_heart_angle_thr
+ self.hand_heart_dis_ratio_thr = hand_heart_dis_ratio_thr
+ self.flying_heart_dis_ratio_thr = flying_heart_dis_ratio_thr
+ self.longest_duration = longest_duration
+ self.largest_ratio = largest_ratio
+ self.num_persons = num_persons
+
+ # record the heart infos for each person
+ self.heart_infos = {}
+
+ def _cal_distance(self, p1: np.ndarray, p2: np.ndarray) -> np.float64:
+ """calculate the distance of points p1 and p2."""
+ return np.sqrt((p1[0] - p2[0])**2 + (p1[1] - p2[1])**2)
+
+ def _cal_angle(self, p1: np.ndarray, p2: np.ndarray, p3: np.ndarray,
+ p4: np.ndarray) -> np.float64:
+ """calculate the angle of vectors v1(constructed by points p2 and p1)
+ and v2(constructed by points p4 and p3)"""
+ v1 = p2 - p1
+ v2 = p4 - p3
+
+ vector_prod = v1[0] * v2[0] + v1[1] * v2[1]
+ length_prod = np.sqrt(pow(v1[0], 2) + pow(v1[1], 2)) * np.sqrt(
+ pow(v2[0], 2) + pow(v2[1], 2))
+ cos = vector_prod * 1.0 / (length_prod * 1.0 + 1e-6)
+
+ return (np.arccos(cos) / np.pi) * 180
+
+ def _check_heart(self, pred: Dict[str,
+ np.ndarray], hand_indices: List[int],
+ mouth_index: int, eye_indices: List[int]) -> int:
+ """Check the type of Valentine Magic based on the pose results and
+ keypoint indices of hand, mouth. and eye.
+
+ Args:
+ pred(dict): The pose estimation results containing:
+ - "keypoints" (np.ndarray[K,3]): keypoint detection result
+ in [x, y, score]
+ hand_indices(list[int]): keypoint indices of hand
+ mouth_index(int): keypoint index of mouth
+ eye_indices(list[int]): keypoint indices of eyes
+
+ Returns:
+ int: a number representing the type of heart pose,
+ 0: None, 1: hand heart, 2: left hand blow kiss,
+ 3: right hand blow kiss
+ """
+ kpts = pred['keypoints']
+
+ left_eye_idx, right_eye_idx = eye_indices
+ left_eye_pos = kpts[left_eye_idx][:2]
+ right_eye_pos = kpts[right_eye_idx][:2]
+ eye_dis = self._cal_distance(left_eye_pos, right_eye_pos)
+
+ # these indices are corresoponding to the following keypoints:
+ # left_hand_root, left_pinky_finger1,
+ # left_pinky_finger3, left_pinky_finger4,
+ # right_hand_root, right_pinky_finger1
+ # right_pinky_finger3, right_pinky_finger4
+
+ both_hands_vis = True
+ for i in [0, 17, 19, 20, 21, 38, 40, 41]:
+ if kpts[hand_indices[i]][2] < self.kpt_vis_thr:
+ both_hands_vis = False
+
+ if both_hands_vis:
+ p1 = kpts[hand_indices[20]][:2]
+ p2 = kpts[hand_indices[19]][:2]
+ p3 = kpts[hand_indices[17]][:2]
+ p4 = kpts[hand_indices[0]][:2]
+ left_angle = self._cal_angle(p1, p2, p3, p4)
+
+ p1 = kpts[hand_indices[41]][:2]
+ p2 = kpts[hand_indices[40]][:2]
+ p3 = kpts[hand_indices[38]][:2]
+ p4 = kpts[hand_indices[21]][:2]
+ right_angle = self._cal_angle(p1, p2, p3, p4)
+
+ hand_dis = self._cal_distance(kpts[hand_indices[20]][:2],
+ kpts[hand_indices[41]][:2])
+
+ if (left_angle < self.hand_heart_angle_thr
+ and right_angle < self.hand_heart_angle_thr
+ and hand_dis / eye_dis < self.hand_heart_dis_ratio_thr):
+ return 1
+
+ # these indices are corresoponding to the following keypoints:
+ # left_middle_finger1, left_middle_finger4,
+ left_hand_vis = True
+ for i in [9, 12]:
+ if kpts[hand_indices[i]][2] < self.kpt_vis_thr:
+ left_hand_vis = False
+ break
+ # right_middle_finger1, right_middle_finger4
+
+ right_hand_vis = True
+ for i in [30, 33]:
+ if kpts[hand_indices[i]][2] < self.kpt_vis_thr:
+ right_hand_vis = False
+ break
+
+ mouth_vis = True
+ if kpts[mouth_index][2] < self.kpt_vis_thr:
+ mouth_vis = False
+
+ if (not left_hand_vis and not right_hand_vis) or not mouth_vis:
+ return 0
+
+ mouth_pos = kpts[mouth_index]
+
+ left_mid_hand_pos = (kpts[hand_indices[9]][:2] +
+ kpts[hand_indices[12]][:2]) / 2
+ lefthand_mouth_dis = self._cal_distance(left_mid_hand_pos, mouth_pos)
+
+ if lefthand_mouth_dis / eye_dis < self.flying_heart_dis_ratio_thr:
+ return 2
+
+ right_mid_hand_pos = (kpts[hand_indices[30]][:2] +
+ kpts[hand_indices[33]][:2]) / 2
+ righthand_mouth_dis = self._cal_distance(right_mid_hand_pos, mouth_pos)
+
+ if righthand_mouth_dis / eye_dis < self.flying_heart_dis_ratio_thr:
+ return 3
+
+ return 0
+
+ def _get_heart_route(self, heart_type: int, cur_pred: Dict[str,
+ np.ndarray],
+ tar_pred: Dict[str,
+ np.ndarray], hand_indices: List[int],
+ mouth_index: int) -> Tuple[int, int]:
+ """get the start and end position of the heart, based on two keypoint
+ results and keypoint indices of hand and mouth.
+
+ Args:
+ cur_pred(dict): The pose estimation results of current person,
+ containing: the following keys:
+ - "keypoints" (np.ndarray[K,3]): keypoint detection result
+ in [x, y, score]
+ tar_pred(dict): The pose estimation results of target person,
+ containing: the following keys:
+ - "keypoints" (np.ndarray[K,3]): keypoint detection result
+ in [x, y, score]
+ hand_indices(list[int]): keypoint indices of hand
+ mouth_index(int): keypoint index of mouth
+
+ Returns:
+ tuple(int): the start position of heart
+ tuple(int): the end position of heart
+ """
+ cur_kpts = cur_pred['keypoints']
+
+ assert heart_type in [1, 2,
+ 3], 'Can not determine the type of heart effect'
+
+ if heart_type == 1:
+ p1 = cur_kpts[hand_indices[20]][:2]
+ p2 = cur_kpts[hand_indices[41]][:2]
+ elif heart_type == 2:
+ p1 = cur_kpts[hand_indices[9]][:2]
+ p2 = cur_kpts[hand_indices[12]][:2]
+ elif heart_type == 3:
+ p1 = cur_kpts[hand_indices[30]][:2]
+ p2 = cur_kpts[hand_indices[33]][:2]
+
+ cur_x, cur_y = (p1 + p2) / 2
+ # the mid point of two fingers
+ start_pos = (int(cur_x), int(cur_y))
+
+ tar_kpts = tar_pred['keypoints']
+ end_pos = tar_kpts[mouth_index][:2]
+
+ return start_pos, end_pos
+
+ def _draw_heart(self, canvas: np.ndarray, heart_info: HeartInfo,
+ t_pass: float) -> np.ndarray:
+ """draw the heart according to heart info and time."""
+ start_x, start_y = heart_info.start_pos
+ end_x, end_y = heart_info.end_pos
+
+ scale = t_pass / self.longest_duration
+
+ max_h, max_w = canvas.shape[:2]
+ hm, wm = self.largest_ratio * max_h, self.largest_ratio * max_h
+ new_h, new_w = int(hm * scale), int(wm * scale)
+
+ x = int(start_x + scale * (end_x - start_x))
+ y = int(start_y + scale * (end_y - start_y))
+
+ y1 = max(0, y - int(new_h / 2))
+ y2 = min(max_h - 1, y + int(new_h / 2))
+
+ x1 = max(0, x - int(new_w / 2))
+ x2 = min(max_w - 1, x + int(new_w / 2))
+
+ target = canvas[y1:y2 + 1, x1:x2 + 1].copy()
+ new_h, new_w = target.shape[:2]
+
+ if new_h == 0 or new_w == 0:
+ return canvas
+
+ assert heart_info.heart_type in [
+ 1, 2, 3
+ ], 'Can not determine the type of heart effect'
+ if heart_info.heart_type == 1: # hand heart
+ patch = self.hand_heart.copy()
+ elif heart_info.heart_type >= 2: # hand blow kiss
+ patch = self.flying_heart.copy()
+ if heart_info.start_pos[0] > heart_info.end_pos[0]:
+ patch = patch[:, ::-1]
+
+ patch = cv2.resize(patch, (new_w, new_h))
+ mask = cv2.cvtColor(patch, cv2.COLOR_BGR2GRAY)
+ mask = (mask < 100)[..., None].astype(np.float32) * 0.8
+
+ canvas[y1:y2 + 1, x1:x2 + 1] = patch * mask + target * (1 - mask)
+
+ return canvas
+
+ def draw(self, frame_msg: FrameMessage) -> np.ndarray:
+ canvas = frame_msg.get_image()
+
+ pose_results = frame_msg.get_pose_results()
+ if not pose_results:
+ return canvas
+
+ for pose_result in pose_results:
+ model_cfg = pose_result['model_cfg']
+
+ preds = [pred.copy() for pred in pose_result['preds']]
+ # if number of persons in the image is less than 2,
+ # no heart effect will be triggered
+ if len(preds) < self.num_persons:
+ continue
+
+ # if number of persons in the image is more than 2,
+ # only use the first two pose results
+ preds = preds[:self.num_persons]
+ ids = [preds[i]['track_id'] for i in range(self.num_persons)]
+
+ for id in self.heart_infos.copy():
+ if id not in ids:
+ # if the id of a person not in previous heart_infos,
+ # delete the corresponding field
+ del self.heart_infos[id]
+
+ for i in range(self.num_persons):
+ id = preds[i]['track_id']
+
+ # if the predicted person in previous heart_infos,
+ # draw the heart
+ if id in self.heart_infos.copy():
+ t_pass = time.time() - self.heart_infos[id].start_time
+
+ # the time passed since last heart pose less than
+ # longest_duration, continue to draw the heart
+ if t_pass < self.longest_duration:
+ canvas = self._draw_heart(canvas, self.heart_infos[id],
+ t_pass)
+ # reset corresponding heart info
+ else:
+ del self.heart_infos[id]
+ else:
+ hand_indices = get_hand_keypoint_ids(model_cfg)
+ mouth_index = get_mouth_keypoint_ids(model_cfg)
+ eye_indices = get_eye_keypoint_ids(model_cfg)
+
+ # check the type of Valentine Magic based on pose results
+ # and keypoint indices of hand and mouth
+ heart_type = self._check_heart(preds[i], hand_indices,
+ mouth_index, eye_indices)
+ # trigger a Valentine Magic effect
+ if heart_type:
+ # get the route of heart
+ start_pos, end_pos = self._get_heart_route(
+ heart_type, preds[i],
+ preds[self.num_persons - 1 - i], hand_indices,
+ mouth_index)
+ start_time = time.time()
+ self.heart_infos[id] = HeartInfo(
+ heart_type, start_time, start_pos, end_pos)
+
+ return canvas
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/nodes/xdwendwen_node.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/nodes/xdwendwen_node.py
new file mode 100644
index 0000000..1a0914d
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/nodes/xdwendwen_node.py
@@ -0,0 +1,240 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import json
+from dataclasses import dataclass
+from typing import List, Tuple, Union
+
+import cv2
+import numpy as np
+
+from mmpose.datasets.dataset_info import DatasetInfo
+from ..utils import load_image_from_disk_or_url
+from .builder import NODES
+from .frame_drawing_node import FrameDrawingNode
+
+
+@dataclass
+class DynamicInfo:
+ pos_curr: Tuple[int, int] = (0, 0)
+ pos_step: Tuple[int, int] = (0, 0)
+ step_curr: int = 0
+
+
+@NODES.register_module()
+class XDwenDwenNode(FrameDrawingNode):
+ """An effect drawing node that captures the face of a cat or dog and blend
+ it into a Bing-Dwen-Dwen (the mascot of 2022 Beijing Winter Olympics).
+
+ Parameters:
+ name (str, optional): The node name (also thread name).
+ frame_buffer (str): The name of the input buffer.
+ output_buffer (str | list): The name(s) of the output buffer(s).
+ mode_key (str | int): A hot key to switch the background image.
+ resource_file (str): The annotation file of resource images, which
+ should be in Labelbee format and contain both facial keypoint and
+ region annotations.
+ out_shape (tuple): The shape of output frame in (width, height).
+ """
+
+ dynamic_scale = 0.15
+ dynamic_max_step = 15
+
+ def __init__(
+ self,
+ name: str,
+ frame_buffer: str,
+ output_buffer: Union[str, List[str]],
+ mode_key: Union[str, int],
+ resource_file: str,
+ out_shape: Tuple[int, int] = (480, 480),
+ rigid_transform: bool = True,
+ ):
+ super().__init__(name, frame_buffer, output_buffer, enable=True)
+
+ self.mode_key = mode_key
+ self.mode_index = 0
+ self.out_shape = out_shape
+ self.rigid = rigid_transform
+
+ self.latest_pred = None
+
+ self.dynamic_info = DynamicInfo()
+
+ self.register_event(
+ self.mode_key, is_keyboard=True, handler_func=self.switch_mode)
+
+ self._init_resource(resource_file)
+
+ def _init_resource(self, resource_file):
+
+ # The resource_file is a JSON file that contains the facial
+ # keypoint and mask annotation information of the resource files.
+ # The annotations should follow the label-bee standard format.
+ # See https://github.com/open-mmlab/labelbee-client for details.
+ with open(resource_file) as f:
+ anns = json.load(f)
+ resource_infos = []
+
+ for ann in anns:
+ # Load image
+ img = load_image_from_disk_or_url(ann['url'])
+ # Load result
+ rst = json.loads(ann['result'])
+
+ # Check facial keypoint information
+ assert rst['step_1']['toolName'] == 'pointTool'
+ assert len(rst['step_1']['result']) == 3
+
+ keypoints = sorted(
+ rst['step_1']['result'], key=lambda x: x['order'])
+ keypoints = np.array([[pt['x'], pt['y']] for pt in keypoints])
+
+ # Check facial mask
+ assert rst['step_2']['toolName'] == 'polygonTool'
+ assert len(rst['step_2']['result']) == 1
+ assert len(rst['step_2']['result'][0]['pointList']) > 2
+
+ mask_pts = np.array(
+ [[pt['x'], pt['y']]
+ for pt in rst['step_2']['result'][0]['pointList']])
+
+ mul = 1.0 + self.dynamic_scale
+
+ w_scale = self.out_shape[0] / img.shape[1] * mul
+ h_scale = self.out_shape[1] / img.shape[0] * mul
+
+ img = cv2.resize(
+ img,
+ dsize=None,
+ fx=w_scale,
+ fy=h_scale,
+ interpolation=cv2.INTER_CUBIC)
+
+ keypoints *= [w_scale, h_scale]
+ mask_pts *= [w_scale, h_scale]
+
+ mask = cv2.fillPoly(
+ np.zeros(img.shape[:2], dtype=np.uint8),
+ [mask_pts.astype(np.int32)],
+ color=1)
+
+ res = {
+ 'img': img,
+ 'keypoints': keypoints,
+ 'mask': mask,
+ }
+ resource_infos.append(res)
+
+ self.resource_infos = resource_infos
+
+ self._reset_dynamic()
+
+ def switch_mode(self):
+ self.mode_index = (self.mode_index + 1) % len(self.resource_infos)
+
+ def _reset_dynamic(self):
+ x_tar = np.random.randint(int(self.out_shape[0] * self.dynamic_scale))
+ y_tar = np.random.randint(int(self.out_shape[1] * self.dynamic_scale))
+
+ x_step = (x_tar -
+ self.dynamic_info.pos_curr[0]) / self.dynamic_max_step
+ y_step = (y_tar -
+ self.dynamic_info.pos_curr[1]) / self.dynamic_max_step
+
+ self.dynamic_info.pos_step = (x_step, y_step)
+ self.dynamic_info.step_curr = 0
+
+ def draw(self, frame_msg):
+
+ full_pose_results = frame_msg.get_pose_results()
+
+ pred = None
+ if full_pose_results:
+ for pose_results in full_pose_results:
+ if not pose_results['preds']:
+ continue
+
+ pred = pose_results['preds'][0].copy()
+ pred['dataset'] = DatasetInfo(pose_results['model_cfg'].data.
+ test.dataset_info).dataset_name
+
+ self.latest_pred = pred
+ break
+
+ # Use the latest pose result if there is none available in
+ # the current frame.
+ if pred is None:
+ pred = self.latest_pred
+
+ # Get the background image and facial annotations
+ res = self.resource_infos[self.mode_index]
+ img = frame_msg.get_image()
+ canvas = res['img'].copy()
+ mask = res['mask']
+ kpts_tar = res['keypoints']
+
+ if pred is not None:
+ if pred['dataset'] == 'ap10k':
+ # left eye: 0, right eye: 1, nose: 2
+ kpts_src = pred['keypoints'][[0, 1, 2], :2]
+ elif pred['dataset'] == 'coco_wholebody':
+ # left eye: 1, right eye 2, nose: 0
+ kpts_src = pred['keypoints'][[1, 2, 0], :2]
+ else:
+ raise ValueError('Can not obtain face landmark information'
+ f'from dataset: {pred["type"]}')
+
+ trans_mat = self._get_transform(kpts_src, kpts_tar)
+
+ warp = cv2.warpAffine(img, trans_mat, dsize=canvas.shape[:2])
+ cv2.copyTo(warp, mask, canvas)
+
+ # Add random movement to the background
+ xc, yc = self.dynamic_info.pos_curr
+ xs, ys = self.dynamic_info.pos_step
+ w, h = self.out_shape
+
+ x = min(max(int(xc), 0), canvas.shape[1] - w + 1)
+ y = min(max(int(yc), 0), canvas.shape[0] - h + 1)
+
+ canvas = canvas[y:y + h, x:x + w]
+
+ self.dynamic_info.pos_curr = (xc + xs, yc + ys)
+ self.dynamic_info.step_curr += 1
+
+ if self.dynamic_info.step_curr == self.dynamic_max_step:
+ self._reset_dynamic()
+
+ return canvas
+
+ def _get_transform(self, kpts_src, kpts_tar):
+ if self.rigid:
+ # rigid transform
+ n = kpts_src.shape[0]
+ X = np.zeros((n * 2, 4), dtype=np.float32)
+ U = np.zeros((n * 2, 1), dtype=np.float32)
+ X[:n, :2] = kpts_src
+ X[:n, 2] = 1
+ X[n:, 0] = kpts_src[:, 1]
+ X[n:, 1] = -kpts_src[:, 0]
+ X[n:, 3] = 1
+
+ U[:n, 0] = kpts_tar[:, 0]
+ U[n:, 0] = kpts_tar[:, 1]
+
+ M = np.linalg.pinv(X).dot(U).flatten()
+
+ trans_mat = np.array([[M[0], M[1], M[2]], [-M[1], M[0], M[3]]],
+ dtype=np.float32)
+
+ else:
+ # normal affine transform
+ # adaptive horizontal flipping
+ if (np.linalg.norm(kpts_tar[0] - kpts_tar[2]) -
+ np.linalg.norm(kpts_tar[1] - kpts_tar[2])) * (
+ np.linalg.norm(kpts_src[0] - kpts_src[2]) -
+ np.linalg.norm(kpts_src[1] - kpts_src[2])) < 0:
+ kpts_src = kpts_src[[1, 0, 2], :]
+ trans_mat, _ = cv2.estimateAffine2D(
+ kpts_src.astype(np.float32), kpts_tar.astype(np.float32))
+
+ return trans_mat
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/utils/__init__.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/utils/__init__.py
new file mode 100644
index 0000000..d906df0
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/utils/__init__.py
@@ -0,0 +1,31 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from .buffer import BufferManager
+from .event import EventManager
+from .message import FrameMessage, Message, VideoEndingMessage
+from .misc import (ImageCapture, copy_and_paste, expand_and_clamp,
+ get_cached_file_path, is_image_file, limit_max_fps,
+ load_image_from_disk_or_url, screen_matting)
+from .pose import (get_eye_keypoint_ids, get_face_keypoint_ids,
+ get_hand_keypoint_ids, get_mouth_keypoint_ids,
+ get_wrist_keypoint_ids)
+
+__all__ = [
+ 'BufferManager',
+ 'EventManager',
+ 'FrameMessage',
+ 'Message',
+ 'limit_max_fps',
+ 'VideoEndingMessage',
+ 'load_image_from_disk_or_url',
+ 'get_cached_file_path',
+ 'screen_matting',
+ 'expand_and_clamp',
+ 'copy_and_paste',
+ 'is_image_file',
+ 'ImageCapture',
+ 'get_eye_keypoint_ids',
+ 'get_face_keypoint_ids',
+ 'get_wrist_keypoint_ids',
+ 'get_mouth_keypoint_ids',
+ 'get_hand_keypoint_ids',
+]
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/utils/buffer.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/utils/buffer.py
new file mode 100644
index 0000000..b9fca4c
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/utils/buffer.py
@@ -0,0 +1,106 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from functools import wraps
+from queue import Queue
+from typing import Dict, List, Optional
+
+from mmcv import is_seq_of
+
+__all__ = ['BufferManager']
+
+
+def check_buffer_registered(exist=True):
+
+ def wrapper(func):
+
+ @wraps(func)
+ def wrapped(manager, name, *args, **kwargs):
+ if exist:
+ # Assert buffer exist
+ if name not in manager:
+ raise ValueError(f'Fail to call {func.__name__}: '
+ f'buffer "{name}" is not registered.')
+ else:
+ # Assert buffer not exist
+ if name in manager:
+ raise ValueError(f'Fail to call {func.__name__}: '
+ f'buffer "{name}" is already registered.')
+ return func(manager, name, *args, **kwargs)
+
+ return wrapped
+
+ return wrapper
+
+
+class Buffer(Queue):
+
+ def put_force(self, item):
+ """Force to put an item into the buffer.
+
+ If the buffer is already full, the earliest item in the buffer will be
+ remove to make room for the incoming item.
+ """
+ with self.mutex:
+ if self.maxsize > 0:
+ while self._qsize() >= self.maxsize:
+ _ = self._get()
+ self.unfinished_tasks -= 1
+
+ self._put(item)
+ self.unfinished_tasks += 1
+ self.not_empty.notify()
+
+
+class BufferManager():
+
+ def __init__(self,
+ buffer_type: type = Buffer,
+ buffers: Optional[Dict] = None):
+ self.buffer_type = buffer_type
+ if buffers is None:
+ self._buffers = {}
+ else:
+ if is_seq_of(list(buffers.values()), buffer_type):
+ self._buffers = buffers.copy()
+ else:
+ raise ValueError('The values of buffers should be instance '
+ f'of {buffer_type}')
+
+ def __contains__(self, name):
+ return name in self._buffers
+
+ @check_buffer_registered(False)
+ def register_buffer(self, name, maxsize=0):
+ self._buffers[name] = self.buffer_type(maxsize)
+
+ @check_buffer_registered()
+ def put(self, name, item, block=True, timeout=None):
+ self._buffers[name].put(item, block, timeout)
+
+ @check_buffer_registered()
+ def put_force(self, name, item):
+ self._buffers[name].put_force(item)
+
+ @check_buffer_registered()
+ def get(self, name, block=True, timeout=None):
+ return self._buffers[name].get(block, timeout)
+
+ @check_buffer_registered()
+ def is_empty(self, name):
+ return self._buffers[name].empty()
+
+ @check_buffer_registered()
+ def is_full(self, name):
+ return self._buffers[name].full()
+
+ def get_sub_manager(self, buffer_names: List[str]):
+ buffers = {name: self._buffers[name] for name in buffer_names}
+ return BufferManager(self.buffer_type, buffers)
+
+ def get_info(self):
+ buffer_info = {}
+ for name, buffer in self._buffers.items():
+ buffer_info[name] = {
+ 'size': buffer.size,
+ 'maxsize': buffer.maxsize
+ }
+ return buffer_info
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/utils/event.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/utils/event.py
new file mode 100644
index 0000000..ceab26f
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/utils/event.py
@@ -0,0 +1,59 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from collections import defaultdict
+from contextlib import contextmanager
+from threading import Event
+from typing import Optional
+
+
+class EventManager():
+
+ def __init__(self):
+ self._events = defaultdict(Event)
+
+ def register_event(self,
+ event_name: str = None,
+ is_keyboard: bool = False):
+ if is_keyboard:
+ event_name = self._get_keyboard_event_name(event_name)
+ self._events[event_name] = Event()
+
+ def set(self, event_name: str = None, is_keyboard: bool = False):
+ if is_keyboard:
+ event_name = self._get_keyboard_event_name(event_name)
+ return self._events[event_name].set()
+
+ def wait(self,
+ event_name: str = None,
+ is_keyboard: Optional[bool] = False,
+ timeout: Optional[float] = None):
+ if is_keyboard:
+ event_name = self._get_keyboard_event_name(event_name)
+ return self._events[event_name].wait(timeout)
+
+ def is_set(self,
+ event_name: str = None,
+ is_keyboard: Optional[bool] = False):
+ if is_keyboard:
+ event_name = self._get_keyboard_event_name(event_name)
+ return self._events[event_name].is_set()
+
+ def clear(self,
+ event_name: str = None,
+ is_keyboard: Optional[bool] = False):
+ if is_keyboard:
+ event_name = self._get_keyboard_event_name(event_name)
+ return self._events[event_name].clear()
+
+ @staticmethod
+ def _get_keyboard_event_name(key):
+ return f'_keyboard_{chr(key) if isinstance(key,int) else key}'
+
+ @contextmanager
+ def wait_and_handle(self,
+ event_name: str = None,
+ is_keyboard: Optional[bool] = False):
+ self.wait(event_name, is_keyboard)
+ try:
+ yield
+ finally:
+ self.clear(event_name, is_keyboard)
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/utils/message.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/utils/message.py
new file mode 100644
index 0000000..d7b1529
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/utils/message.py
@@ -0,0 +1,204 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import time
+import uuid
+import warnings
+from typing import Dict, List, Optional
+
+import numpy as np
+
+
+class Message():
+ """Message base class.
+
+ All message class should inherit this class. The basic use of a Message
+ instance is to carray a piece of text message (self.msg) and a dict that
+ stores structured data (self.data), e.g. frame image, model prediction,
+ et al.
+
+ A message may also hold route information, which is composed of
+ information of all nodes the message has passed through.
+
+ Parameters:
+ msg (str): The text message.
+ data (dict, optional): The structured data.
+ """
+
+ def __init__(self, msg: str = '', data: Optional[Dict] = None):
+ self.msg = msg
+ self.data = data if data else {}
+ self.route_info = []
+ self.timestamp = time.time()
+ self.id = uuid.uuid4()
+
+ def update_route_info(self,
+ node=None,
+ node_name: Optional[str] = None,
+ node_type: Optional[str] = None,
+ info: Optional[Dict] = None):
+ """Append new node information to the route information.
+
+ Args:
+ node (Node, optional): An instance of Node that provides basic
+ information like the node name and type. Default: None.
+ node_name (str, optional): The node name. If node is given,
+ node_name will be ignored. Default: None.
+ node_type (str, optional): The class name of the node. If node
+ is given, node_type will be ignored. Default: None.
+ info (dict, optional): The node information, which is usually
+ given by node.get_node_info(). Default: None.
+ """
+ if node is not None:
+ if node_name is not None or node_type is not None:
+ warnings.warn(
+ '`node_name` and `node_type` will be overridden if node'
+ 'is provided.')
+ node_name = node.name
+ node_type = node.__class__.__name__
+
+ node_info = {'node': node_name, 'node_type': node_type, 'info': info}
+ self.route_info.append(node_info)
+
+ def set_route_info(self, route_info: List):
+ """Directly set the entire route information.
+
+ Args:
+ route_info (list): route information to set to the message.
+ """
+ self.route_info = route_info
+
+ def merge_route_info(self, route_info: List):
+ """Merge the given route information into the original one of the
+ message. This is used for combining route information from multiple
+ messages. The node information in the route will be reordered according
+ to their timestamps.
+
+ Args:
+ route_info (list): route information to merge.
+ """
+ self.route_info += route_info
+ self.route_info.sort(key=lambda x: x.get('timestamp', np.inf))
+
+ def get_route_info(self) -> List:
+ return self.route_info.copy()
+
+
+class VideoEndingMessage(Message):
+ """A special message to indicate the input video is ending."""
+
+
+class FrameMessage(Message):
+ """The message to store information of a video frame.
+
+ A FrameMessage instance usually holds following data in self.data:
+ - image (array): The frame image
+ - detection_results (list): A list to hold detection results of
+ multiple detectors. Each element is a tuple (tag, result)
+ - pose_results (list): A list to hold pose estimation results of
+ multiple pose estimator. Each element is a tuple (tag, result)
+ """
+
+ def __init__(self, img):
+ super().__init__(data=dict(image=img))
+
+ def get_image(self):
+ """Get the frame image.
+
+ Returns:
+ array: The frame image.
+ """
+ return self.data.get('image', None)
+
+ def set_image(self, img):
+ """Set the frame image to the message."""
+ self.data['image'] = img
+
+ def add_detection_result(self, result, tag: str = None):
+ """Add the detection result from one model into the message's
+ detection_results.
+
+ Args:
+ tag (str, optional): Give a tag to the result, which can be used
+ to retrieve specific results.
+ """
+ if 'detection_results' not in self.data:
+ self.data['detection_results'] = []
+ self.data['detection_results'].append((tag, result))
+
+ def get_detection_results(self, tag: str = None):
+ """Get detection results of the message.
+
+ Args:
+ tag (str, optional): If given, only the results with the tag
+ will be retrieved. Otherwise all results will be retrieved.
+ Default: None.
+
+ Returns:
+ list[dict]: The retrieved detection results
+ """
+ if 'detection_results' not in self.data:
+ return None
+ if tag is None:
+ results = [res for _, res in self.data['detection_results']]
+ else:
+ results = [
+ res for _tag, res in self.data['detection_results']
+ if _tag == tag
+ ]
+ return results
+
+ def add_pose_result(self, result, tag=None):
+ """Add the pose estimation result from one model into the message's
+ pose_results.
+
+ Args:
+ tag (str, optional): Give a tag to the result, which can be used
+ to retrieve specific results.
+ """
+ if 'pose_results' not in self.data:
+ self.data['pose_results'] = []
+ self.data['pose_results'].append((tag, result))
+
+ def get_pose_results(self, tag=None):
+ """Get pose estimation results of the message.
+
+ Args:
+ tag (str, optional): If given, only the results with the tag
+ will be retrieved. Otherwise all results will be retrieved.
+ Default: None.
+
+ Returns:
+ list[dict]: The retrieved pose results
+ """
+ if 'pose_results' not in self.data:
+ return None
+ if tag is None:
+ results = [res for _, res in self.data['pose_results']]
+ else:
+ results = [
+ res for _tag, res in self.data['pose_results'] if _tag == tag
+ ]
+ return results
+
+ def get_full_results(self):
+ """Get all model predictions of the message.
+
+ See set_full_results() for inference.
+
+ Returns:
+ dict: All model predictions, including:
+ - detection_results
+ - pose_results
+ """
+ result_keys = ['detection_results', 'pose_results']
+ results = {k: self.data[k] for k in result_keys}
+ return results
+
+ def set_full_results(self, results):
+ """Set full model results directly.
+
+ Args:
+ results (dict): All model predictions including:
+ - detection_results (list): see also add_detection_results()
+ - pose_results (list): see also add_pose_results()
+ """
+ self.data.update(results)
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/utils/misc.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/utils/misc.py
new file mode 100644
index 0000000..c64f417
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/utils/misc.py
@@ -0,0 +1,343 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os
+import os.path as osp
+import sys
+import time
+from contextlib import contextmanager
+from typing import Optional
+from urllib.parse import urlparse
+from urllib.request import urlopen
+
+import cv2
+import numpy as np
+from torch.hub import HASH_REGEX, download_url_to_file
+
+
+@contextmanager
+def limit_max_fps(fps: Optional[float]):
+ t_start = time.time()
+ try:
+ yield
+ finally:
+ t_end = time.time()
+ if fps is not None:
+ t_sleep = 1.0 / fps - t_end + t_start
+ if t_sleep > 0:
+ time.sleep(t_sleep)
+
+
+def _is_url(filename):
+ """Check if the file is a url link.
+
+ Args:
+ filename (str): the file name or url link.
+
+ Returns:
+ bool: is url or not.
+ """
+ prefixes = ['http://', 'https://']
+ for p in prefixes:
+ if filename.startswith(p):
+ return True
+ return False
+
+
+def load_image_from_disk_or_url(filename, readFlag=cv2.IMREAD_COLOR):
+ """Load an image file, from disk or url.
+
+ Args:
+ filename (str): file name on the disk or url link.
+ readFlag (int): readFlag for imdecode.
+
+ Returns:
+ np.ndarray: A loaded image
+ """
+ if _is_url(filename):
+ # download the image, convert it to a NumPy array, and then read
+ # it into OpenCV format
+ resp = urlopen(filename)
+ image = np.asarray(bytearray(resp.read()), dtype='uint8')
+ image = cv2.imdecode(image, readFlag)
+ return image
+ else:
+ image = cv2.imread(filename, readFlag)
+ return image
+
+
+def mkdir_or_exist(dir_name, mode=0o777):
+ if dir_name == '':
+ return
+ dir_name = osp.expanduser(dir_name)
+ os.makedirs(dir_name, mode=mode, exist_ok=True)
+
+
+def get_cached_file_path(url,
+ save_dir=None,
+ progress=True,
+ check_hash=False,
+ file_name=None):
+ r"""Loads the Torch serialized object at the given URL.
+
+ If downloaded file is a zip file, it will be automatically decompressed
+
+ If the object is already present in `model_dir`, it's deserialized and
+ returned.
+ The default value of ``model_dir`` is ``/checkpoints`` where
+ ``hub_dir`` is the directory returned by :func:`~torch.hub.get_dir`.
+
+ Args:
+ url (str): URL of the object to download
+ save_dir (str, optional): directory in which to save the object
+ progress (bool, optional): whether or not to display a progress bar
+ to stderr. Default: True
+ check_hash(bool, optional): If True, the filename part of the URL
+ should follow the naming convention ``filename-.ext``
+ where ```` is the first eight or more digits of the
+ SHA256 hash of the contents of the file. The hash is used to
+ ensure unique names and to verify the contents of the file.
+ Default: False
+ file_name (str, optional): name for the downloaded file. Filename
+ from ``url`` will be used if not set. Default: None.
+ """
+ if save_dir is None:
+ save_dir = os.path.join('webcam_resources')
+
+ mkdir_or_exist(save_dir)
+
+ parts = urlparse(url)
+ filename = os.path.basename(parts.path)
+ if file_name is not None:
+ filename = file_name
+ cached_file = os.path.join(save_dir, filename)
+ if not os.path.exists(cached_file):
+ sys.stderr.write('Downloading: "{}" to {}\n'.format(url, cached_file))
+ hash_prefix = None
+ if check_hash:
+ r = HASH_REGEX.search(filename) # r is Optional[Match[str]]
+ hash_prefix = r.group(1) if r else None
+ download_url_to_file(url, cached_file, hash_prefix, progress=progress)
+ return cached_file
+
+
+def screen_matting(img, color_low=None, color_high=None, color=None):
+ """Screen Matting.
+
+ Args:
+ img (np.ndarray): Image data.
+ color_low (tuple): Lower limit (b, g, r).
+ color_high (tuple): Higher limit (b, g, r).
+ color (str): Support colors include:
+
+ - 'green' or 'g'
+ - 'blue' or 'b'
+ - 'black' or 'k'
+ - 'white' or 'w'
+ """
+
+ if color_high is None or color_low is None:
+ if color is not None:
+ if color.lower() == 'g' or color.lower() == 'green':
+ color_low = (0, 200, 0)
+ color_high = (60, 255, 60)
+ elif color.lower() == 'b' or color.lower() == 'blue':
+ color_low = (230, 0, 0)
+ color_high = (255, 40, 40)
+ elif color.lower() == 'k' or color.lower() == 'black':
+ color_low = (0, 0, 0)
+ color_high = (40, 40, 40)
+ elif color.lower() == 'w' or color.lower() == 'white':
+ color_low = (230, 230, 230)
+ color_high = (255, 255, 255)
+ else:
+ NotImplementedError(f'Not supported color: {color}.')
+ else:
+ ValueError('color or color_high | color_low should be given.')
+
+ mask = cv2.inRange(img, np.array(color_low), np.array(color_high)) == 0
+
+ return mask.astype(np.uint8)
+
+
+def expand_and_clamp(box, im_shape, s=1.25):
+ """Expand the bbox and clip it to fit the image shape.
+
+ Args:
+ box (list): x1, y1, x2, y2
+ im_shape (ndarray): image shape (h, w, c)
+ s (float): expand ratio
+
+ Returns:
+ list: x1, y1, x2, y2
+ """
+
+ x1, y1, x2, y2 = box[:4]
+ w = x2 - x1
+ h = y2 - y1
+ deta_w = w * (s - 1) / 2
+ deta_h = h * (s - 1) / 2
+
+ x1, y1, x2, y2 = x1 - deta_w, y1 - deta_h, x2 + deta_w, y2 + deta_h
+
+ img_h, img_w = im_shape[:2]
+
+ x1 = min(max(0, int(x1)), img_w - 1)
+ y1 = min(max(0, int(y1)), img_h - 1)
+ x2 = min(max(0, int(x2)), img_w - 1)
+ y2 = min(max(0, int(y2)), img_h - 1)
+
+ return [x1, y1, x2, y2]
+
+
+def _find_connected_components(mask):
+ """Find connected components and sort with areas.
+
+ Args:
+ mask (ndarray): instance segmentation result.
+
+ Returns:
+ ndarray (N, 5): Each item contains (x, y, w, h, area).
+ """
+ num, labels, stats, centroids = cv2.connectedComponentsWithStats(mask)
+ stats = stats[stats[:, 4].argsort()]
+ return stats
+
+
+def _find_bbox(mask):
+ """Find the bounding box for the mask.
+
+ Args:
+ mask (ndarray): Mask.
+
+ Returns:
+ list(4, ): Returned box (x1, y1, x2, y2).
+ """
+ mask_shape = mask.shape
+ if len(mask_shape) == 3:
+ assert mask_shape[-1] == 1, 'the channel of the mask should be 1.'
+ elif len(mask_shape) == 2:
+ pass
+ else:
+ NotImplementedError()
+
+ h, w = mask_shape[:2]
+ mask_w = mask.sum(0)
+ mask_h = mask.sum(1)
+
+ left = 0
+ right = w - 1
+ up = 0
+ down = h - 1
+
+ for i in range(w):
+ if mask_w[i] > 0:
+ break
+ left += 1
+
+ for i in range(w - 1, left, -1):
+ if mask_w[i] > 0:
+ break
+ right -= 1
+
+ for i in range(h):
+ if mask_h[i] > 0:
+ break
+ up += 1
+
+ for i in range(h - 1, up, -1):
+ if mask_h[i] > 0:
+ break
+ down -= 1
+
+ return [left, up, right, down]
+
+
+def copy_and_paste(img,
+ background_img,
+ mask,
+ bbox=None,
+ effect_region=(0.2, 0.2, 0.8, 0.8),
+ min_size=(20, 20)):
+ """Copy the image region and paste to the background.
+
+ Args:
+ img (np.ndarray): Image data.
+ background_img (np.ndarray): Background image data.
+ mask (ndarray): instance segmentation result.
+ bbox (ndarray): instance bbox, (x1, y1, x2, y2).
+ effect_region (tuple(4, )): The region to apply mask, the coordinates
+ are normalized (x1, y1, x2, y2).
+ """
+ background_img = background_img.copy()
+ background_h, background_w = background_img.shape[:2]
+ region_h = (effect_region[3] - effect_region[1]) * background_h
+ region_w = (effect_region[2] - effect_region[0]) * background_w
+ region_aspect_ratio = region_w / region_h
+
+ if bbox is None:
+ bbox = _find_bbox(mask)
+ instance_w = bbox[2] - bbox[0]
+ instance_h = bbox[3] - bbox[1]
+
+ if instance_w > min_size[0] and instance_h > min_size[1]:
+ aspect_ratio = instance_w / instance_h
+ if region_aspect_ratio > aspect_ratio:
+ resize_rate = region_h / instance_h
+ else:
+ resize_rate = region_w / instance_w
+
+ mask_inst = mask[int(bbox[1]):int(bbox[3]), int(bbox[0]):int(bbox[2])]
+ img_inst = img[int(bbox[1]):int(bbox[3]), int(bbox[0]):int(bbox[2])]
+ img_inst = cv2.resize(img_inst, (int(
+ resize_rate * instance_w), int(resize_rate * instance_h)))
+ mask_inst = cv2.resize(
+ mask_inst,
+ (int(resize_rate * instance_w), int(resize_rate * instance_h)),
+ interpolation=cv2.INTER_NEAREST)
+
+ mask_ids = list(np.where(mask_inst == 1))
+ mask_ids[1] += int(effect_region[0] * background_w)
+ mask_ids[0] += int(effect_region[1] * background_h)
+
+ background_img[tuple(mask_ids)] = img_inst[np.where(mask_inst == 1)]
+
+ return background_img
+
+
+def is_image_file(path):
+ if isinstance(path, str):
+ if path.lower().endswith(('.png', '.jpg', '.jpeg', '.tiff', '.bmp')):
+ return True
+ return False
+
+
+class ImageCapture:
+ """A mock-up version of cv2.VideoCapture that always return a const image.
+
+ Args:
+ image (str | ndarray): The image or image path
+ """
+
+ def __init__(self, image):
+ if isinstance(image, str):
+ self.image = load_image_from_disk_or_url(image)
+ else:
+ self.image = image
+
+ def isOpened(self):
+ return (self.image is not None)
+
+ def read(self):
+ return True, self.image.copy()
+
+ def release(self):
+ pass
+
+ def get(self, propId):
+ if propId == cv2.CAP_PROP_FRAME_WIDTH:
+ return self.image.shape[1]
+ elif propId == cv2.CAP_PROP_FRAME_HEIGHT:
+ return self.image.shape[0]
+ elif propId == cv2.CAP_PROP_FPS:
+ return np.nan
+ else:
+ raise NotImplementedError()
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/utils/pose.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/utils/pose.py
new file mode 100644
index 0000000..196b40e
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/utils/pose.py
@@ -0,0 +1,226 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from typing import List, Tuple
+
+from mmcv import Config
+
+from mmpose.datasets.dataset_info import DatasetInfo
+
+
+def get_eye_keypoint_ids(model_cfg: Config) -> Tuple[int, int]:
+ """A helpfer function to get the keypoint indices of left and right eyes
+ from the model config.
+
+ Args:
+ model_cfg (Config): pose model config.
+
+ Returns:
+ int: left eye keypoint index.
+ int: right eye keypoint index.
+ """
+ left_eye_idx = None
+ right_eye_idx = None
+
+ # try obtaining eye point ids from dataset_info
+ try:
+ dataset_info = DatasetInfo(model_cfg.data.test.dataset_info)
+ left_eye_idx = dataset_info.keypoint_name2id.get('left_eye', None)
+ right_eye_idx = dataset_info.keypoint_name2id.get('right_eye', None)
+ except AttributeError:
+ left_eye_idx = None
+ right_eye_idx = None
+
+ if left_eye_idx is None or right_eye_idx is None:
+ # Fall back to hard coded keypoint id
+ dataset_name = model_cfg.data.test.type
+ if dataset_name in {
+ 'TopDownCocoDataset', 'TopDownCocoWholeBodyDataset'
+ }:
+ left_eye_idx = 1
+ right_eye_idx = 2
+ elif dataset_name in {'AnimalPoseDataset', 'AnimalAP10KDataset'}:
+ left_eye_idx = 0
+ right_eye_idx = 1
+ else:
+ raise ValueError('Can not determine the eye keypoint id of '
+ f'{dataset_name}')
+
+ return left_eye_idx, right_eye_idx
+
+
+def get_face_keypoint_ids(model_cfg: Config) -> Tuple[int, int]:
+ """A helpfer function to get the keypoint indices of the face from the
+ model config.
+
+ Args:
+ model_cfg (Config): pose model config.
+
+ Returns:
+ list[int]: face keypoint index.
+ """
+ face_indices = None
+
+ # try obtaining nose point ids from dataset_info
+ try:
+ dataset_info = DatasetInfo(model_cfg.data.test.dataset_info)
+ for id in range(68):
+ face_indices.append(
+ dataset_info.keypoint_name2id.get(f'face_{id}', None))
+ except AttributeError:
+ face_indices = None
+
+ if face_indices is None:
+ # Fall back to hard coded keypoint id
+ dataset_name = model_cfg.data.test.type
+ if dataset_name in {'TopDownCocoWholeBodyDataset'}:
+ face_indices = list(range(23, 91))
+ else:
+ raise ValueError('Can not determine the face id of '
+ f'{dataset_name}')
+
+ return face_indices
+
+
+def get_wrist_keypoint_ids(model_cfg: Config) -> Tuple[int, int]:
+ """A helpfer function to get the keypoint indices of left and right wrist
+ from the model config.
+
+ Args:
+ model_cfg (Config): pose model config.
+ Returns:
+ int: left wrist keypoint index.
+ int: right wrist keypoint index.
+ """
+
+ # try obtaining eye point ids from dataset_info
+ try:
+ dataset_info = DatasetInfo(model_cfg.data.test.dataset_info)
+ left_wrist_idx = dataset_info.keypoint_name2id.get('left_wrist', None)
+ right_wrist_idx = dataset_info.keypoint_name2id.get(
+ 'right_wrist', None)
+ except AttributeError:
+ left_wrist_idx = None
+ right_wrist_idx = None
+
+ if left_wrist_idx is None or right_wrist_idx is None:
+ # Fall back to hard coded keypoint id
+ dataset_name = model_cfg.data.test.type
+ if dataset_name in {
+ 'TopDownCocoDataset', 'TopDownCocoWholeBodyDataset'
+ }:
+ left_wrist_idx = 9
+ right_wrist_idx = 10
+ elif dataset_name == 'AnimalPoseDataset':
+ left_wrist_idx = 16
+ right_wrist_idx = 17
+ elif dataset_name == 'AnimalAP10KDataset':
+ left_wrist_idx = 7
+ right_wrist_idx = 10
+ else:
+ raise ValueError('Can not determine the eye keypoint id of '
+ f'{dataset_name}')
+
+ return left_wrist_idx, right_wrist_idx
+
+
+def get_mouth_keypoint_ids(model_cfg: Config) -> Tuple[int, int]:
+ """A helpfer function to get the keypoint indices of the left and right
+ part of mouth from the model config.
+
+ Args:
+ model_cfg (Config): pose model config.
+ Returns:
+ int: left-part mouth keypoint index.
+ int: right-part mouth keypoint index.
+ """
+ # try obtaining mouth point ids from dataset_info
+ try:
+ dataset_info = DatasetInfo(model_cfg.data.test.dataset_info)
+ mouth_index = dataset_info.keypoint_name2id.get('face-62', None)
+ except AttributeError:
+ mouth_index = None
+
+ if mouth_index is None:
+ # Fall back to hard coded keypoint id
+ dataset_name = model_cfg.data.test.type
+ if dataset_name == 'TopDownCocoWholeBodyDataset':
+ mouth_index = 85
+ else:
+ raise ValueError('Can not determine the eye keypoint id of '
+ f'{dataset_name}')
+
+ return mouth_index
+
+
+def get_hand_keypoint_ids(model_cfg: Config) -> List[int]:
+ """A helpfer function to get the keypoint indices of left and right hand
+ from the model config.
+
+ Args:
+ model_cfg (Config): pose model config.
+ Returns:
+ list[int]: hand keypoint indices.
+ """
+ # try obtaining hand keypoint ids from dataset_info
+ try:
+ hand_indices = []
+ dataset_info = DatasetInfo(model_cfg.data.test.dataset_info)
+
+ hand_indices.append(
+ dataset_info.keypoint_name2id.get('left_hand_root', None))
+
+ for id in range(1, 5):
+ hand_indices.append(
+ dataset_info.keypoint_name2id.get(f'left_thumb{id}', None))
+ for id in range(1, 5):
+ hand_indices.append(
+ dataset_info.keypoint_name2id.get(f'left_forefinger{id}',
+ None))
+ for id in range(1, 5):
+ hand_indices.append(
+ dataset_info.keypoint_name2id.get(f'left_middle_finger{id}',
+ None))
+ for id in range(1, 5):
+ hand_indices.append(
+ dataset_info.keypoint_name2id.get(f'left_ring_finger{id}',
+ None))
+ for id in range(1, 5):
+ hand_indices.append(
+ dataset_info.keypoint_name2id.get(f'left_pinky_finger{id}',
+ None))
+
+ hand_indices.append(
+ dataset_info.keypoint_name2id.get('right_hand_root', None))
+
+ for id in range(1, 5):
+ hand_indices.append(
+ dataset_info.keypoint_name2id.get(f'right_thumb{id}', None))
+ for id in range(1, 5):
+ hand_indices.append(
+ dataset_info.keypoint_name2id.get(f'right_forefinger{id}',
+ None))
+ for id in range(1, 5):
+ hand_indices.append(
+ dataset_info.keypoint_name2id.get(f'right_middle_finger{id}',
+ None))
+ for id in range(1, 5):
+ hand_indices.append(
+ dataset_info.keypoint_name2id.get(f'right_ring_finger{id}',
+ None))
+ for id in range(1, 5):
+ hand_indices.append(
+ dataset_info.keypoint_name2id.get(f'right_pinky_finger{id}',
+ None))
+
+ except AttributeError:
+ hand_indices = None
+
+ if hand_indices is None:
+ # Fall back to hard coded keypoint id
+ dataset_name = model_cfg.data.test.type
+ if dataset_name in {'TopDownCocoWholeBodyDataset'}:
+ hand_indices = list(range(91, 133))
+ else:
+ raise ValueError('Can not determine the hand id of '
+ f'{dataset_name}')
+
+ return hand_indices
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/webcam_runner.py b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/webcam_runner.py
new file mode 100644
index 0000000..7843b39
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/.mim/tools/webcam/webcam_apis/webcam_runner.py
@@ -0,0 +1,272 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import logging
+import sys
+import time
+import warnings
+from contextlib import nullcontext
+from threading import Thread
+from typing import Dict, List, Optional, Tuple, Union
+
+import cv2
+
+from .nodes import NODES
+from .utils import (BufferManager, EventManager, FrameMessage, ImageCapture,
+ VideoEndingMessage, is_image_file, limit_max_fps)
+
+DEFAULT_FRAME_BUFFER_SIZE = 1
+DEFAULT_INPUT_BUFFER_SIZE = 1
+DEFAULT_DISPLAY_BUFFER_SIZE = 0
+DEFAULT_USER_BUFFER_SIZE = 1
+
+
+class WebcamRunner():
+ """An interface for building webcam application from config.
+
+ Parameters:
+ name (str): Runner name.
+ camera_id (int | str): The camera ID (usually the ID of the default
+ camera is 0). Alternatively a file path or a URL can be given
+ to load from a video or image file.
+ camera_frame_shape (tuple, optional): Set the frame shape of the
+ camera in (width, height). If not given, the default frame shape
+ will be used. This argument is only valid when using a camera
+ as the input source. Default: None
+ camera_fps (int): Video reading maximum FPS. Default: 30
+ buffer_sizes (dict, optional): A dict to specify buffer sizes. The
+ key is the buffer name and the value is the buffer size.
+ Default: None
+ nodes (list): Node configs.
+ """
+
+ def __init__(self,
+ name: str = 'Default Webcam Runner',
+ camera_id: Union[int, str] = 0,
+ camera_fps: int = 30,
+ camera_frame_shape: Optional[Tuple[int, int]] = None,
+ synchronous: bool = False,
+ buffer_sizes: Optional[Dict[str, int]] = None,
+ nodes: Optional[List[Dict]] = None):
+
+ # Basic parameters
+ self.name = name
+ self.camera_id = camera_id
+ self.camera_fps = camera_fps
+ self.camera_frame_shape = camera_frame_shape
+ self.synchronous = synchronous
+
+ # self.buffer_manager manages data flow between runner and nodes
+ self.buffer_manager = BufferManager()
+ # self.event_manager manages event-based asynchronous communication
+ self.event_manager = EventManager()
+ # self.node_list holds all node instance
+ self.node_list = []
+ # self.vcap is used to read camera frames. It will be built when the
+ # runner starts running
+ self.vcap = None
+
+ # Register runner events
+ self.event_manager.register_event('_exit_', is_keyboard=False)
+ if self.synchronous:
+ self.event_manager.register_event('_idle_', is_keyboard=False)
+
+ # Register nodes
+ if not nodes:
+ raise ValueError('No node is registered to the runner.')
+
+ # Register default buffers
+ if buffer_sizes is None:
+ buffer_sizes = {}
+ # _frame_ buffer
+ frame_buffer_size = buffer_sizes.get('_frame_',
+ DEFAULT_FRAME_BUFFER_SIZE)
+ self.buffer_manager.register_buffer('_frame_', frame_buffer_size)
+ # _input_ buffer
+ input_buffer_size = buffer_sizes.get('_input_',
+ DEFAULT_INPUT_BUFFER_SIZE)
+ self.buffer_manager.register_buffer('_input_', input_buffer_size)
+ # _display_ buffer
+ display_buffer_size = buffer_sizes.get('_display_',
+ DEFAULT_DISPLAY_BUFFER_SIZE)
+ self.buffer_manager.register_buffer('_display_', display_buffer_size)
+
+ # Build all nodes:
+ for node_cfg in nodes:
+ logging.info(f'Create node: {node_cfg.name}({node_cfg.type})')
+ node = NODES.build(node_cfg)
+
+ # Register node
+ self.node_list.append(node)
+
+ # Register buffers
+ for buffer_info in node.registered_buffers:
+ buffer_name = buffer_info.buffer_name
+ if buffer_name in self.buffer_manager:
+ continue
+ buffer_size = buffer_sizes.get(buffer_name,
+ DEFAULT_USER_BUFFER_SIZE)
+ self.buffer_manager.register_buffer(buffer_name, buffer_size)
+ logging.info(
+ f'Register user buffer: {buffer_name}({buffer_size})')
+
+ # Register events
+ for event_info in node.registered_events:
+ self.event_manager.register_event(
+ event_name=event_info.event_name,
+ is_keyboard=event_info.is_keyboard)
+ logging.info(f'Register event: {event_info.event_name}')
+
+ # Set runner for nodes
+ # This step is performed after node building when the runner has
+ # create full buffer/event managers and can
+ for node in self.node_list:
+ logging.info(f'Set runner for node: {node.name})')
+ node.set_runner(self)
+
+ def _read_camera(self):
+ """Continually read video frames and put them into buffers."""
+
+ camera_id = self.camera_id
+ fps = self.camera_fps
+
+ # Build video capture
+ if is_image_file(camera_id):
+ self.vcap = ImageCapture(camera_id)
+ else:
+ self.vcap = cv2.VideoCapture(camera_id)
+ if self.camera_frame_shape is not None:
+ width, height = self.camera_frame_shape
+ self.vcap.set(cv2.CAP_PROP_FRAME_WIDTH, width)
+ self.vcap.set(cv2.CAP_PROP_FRAME_HEIGHT, height)
+
+ if not self.vcap.isOpened():
+ warnings.warn(f'Cannot open camera (ID={camera_id})')
+ sys.exit()
+
+ # Read video frames in a loop
+ first_frame = True
+ while not self.event_manager.is_set('_exit_'):
+ if self.synchronous:
+ if first_frame:
+ cm = nullcontext()
+ else:
+ # Read a new frame until the last frame has been processed
+ cm = self.event_manager.wait_and_handle('_idle_')
+ else:
+ # Read frames with a maximum FPS
+ cm = limit_max_fps(fps)
+
+ first_frame = False
+
+ with cm:
+ # Read a frame
+ ret_val, frame = self.vcap.read()
+ if ret_val:
+ # Put frame message (for display) into buffer `_frame_`
+ frame_msg = FrameMessage(frame)
+ self.buffer_manager.put('_frame_', frame_msg)
+
+ # Put input message (for model inference or other use)
+ # into buffer `_input_`
+ input_msg = FrameMessage(frame.copy())
+ input_msg.update_route_info(
+ node_name='Camera Info',
+ node_type='dummy',
+ info=self._get_camera_info())
+ self.buffer_manager.put_force('_input_', input_msg)
+
+ else:
+ # Put a video ending signal
+ self.buffer_manager.put('_frame_', VideoEndingMessage())
+
+ self.vcap.release()
+
+ def _display(self):
+ """Continually obtain and display output frames."""
+
+ output_msg = None
+
+ while not self.event_manager.is_set('_exit_'):
+ while self.buffer_manager.is_empty('_display_'):
+ time.sleep(0.001)
+
+ # Set _idle_ to allow reading next frame
+ if self.synchronous:
+ self.event_manager.set('_idle_')
+
+ # acquire output from buffer
+ output_msg = self.buffer_manager.get('_display_')
+
+ # None indicates input stream ends
+ if isinstance(output_msg, VideoEndingMessage):
+ self.event_manager.set('_exit_')
+ break
+
+ img = output_msg.get_image()
+
+ # show in a window
+ cv2.imshow(self.name, img)
+
+ # handle keyboard input
+ key = cv2.waitKey(1)
+ if key != -1:
+ self._on_keyboard_input(key)
+
+ cv2.destroyAllWindows()
+
+ def _on_keyboard_input(self, key):
+ """Handle the keyboard input."""
+
+ if key in (27, ord('q'), ord('Q')):
+ logging.info(f'Exit event captured: {key}')
+ self.event_manager.set('_exit_')
+ else:
+ logging.info(f'Keyboard event captured: {key}')
+ self.event_manager.set(key, is_keyboard=True)
+
+ def _get_camera_info(self):
+ """Return the camera information in a dict."""
+
+ frame_width = self.vcap.get(cv2.CAP_PROP_FRAME_WIDTH)
+ frame_height = self.vcap.get(cv2.CAP_PROP_FRAME_HEIGHT)
+ frame_rate = self.vcap.get(cv2.CAP_PROP_FPS)
+
+ cam_info = {
+ 'Camera ID': self.camera_id,
+ 'Source resolution': f'{frame_width}x{frame_height}',
+ 'Source FPS': frame_rate,
+ }
+
+ return cam_info
+
+ def run(self):
+ """Program entry.
+
+ This method starts all nodes as well as video I/O in separate threads.
+ """
+
+ try:
+ # Start node threads
+ non_daemon_nodes = []
+ for node in self.node_list:
+ node.start()
+ if not node.daemon:
+ non_daemon_nodes.append(node)
+
+ # Create a thread to read video frames
+ t_read = Thread(target=self._read_camera, args=())
+ t_read.start()
+
+ # Run display in the main thread
+ self._display()
+ logging.info('Display shut down')
+
+ # joint non-daemon nodes and runner threads
+ logging.info('Camera reading about to join')
+ t_read.join()
+
+ for node in non_daemon_nodes:
+ logging.info(f'Node {node.name} about to join')
+ node.join()
+
+ except KeyboardInterrupt:
+ pass
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/__init__.py b/engine/pose_estimation/third-party/ViTPose/mmpose/__init__.py
new file mode 100644
index 0000000..239d705
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/__init__.py
@@ -0,0 +1,29 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import mmcv
+
+from .version import __version__, short_version
+
+
+def digit_version(version_str):
+ digit_version = []
+ for x in version_str.split('.'):
+ if x.isdigit():
+ digit_version.append(int(x))
+ elif x.find('rc') != -1:
+ patch_version = x.split('rc')
+ digit_version.append(int(patch_version[0]) - 1)
+ digit_version.append(int(patch_version[1]))
+ return digit_version
+
+
+mmcv_minimum_version = '1.3.8'
+mmcv_maximum_version = '2.2.1'
+mmcv_version = digit_version(mmcv.__version__)
+
+
+assert (mmcv_version >= digit_version(mmcv_minimum_version)
+ and mmcv_version <= digit_version(mmcv_maximum_version)), \
+ f'MMCV=={mmcv.__version__} is used but incompatible. ' \
+ f'Please install mmcv>={mmcv_minimum_version}, <={mmcv_maximum_version}.'
+
+__all__ = ['__version__', 'short_version']
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/apis/__init__.py b/engine/pose_estimation/third-party/ViTPose/mmpose/apis/__init__.py
new file mode 100644
index 0000000..0e263ed
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/apis/__init__.py
@@ -0,0 +1,20 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from .inference import (inference_bottom_up_pose_model,
+ inference_top_down_pose_model, init_pose_model,
+ process_mmdet_results, vis_pose_result)
+from .inference_3d import (extract_pose_sequence, inference_interhand_3d_model,
+ inference_mesh_model, inference_pose_lifter_model,
+ vis_3d_mesh_result, vis_3d_pose_result)
+from .inference_tracking import get_track_id, vis_pose_tracking_result
+from .test import multi_gpu_test, single_gpu_test
+from .train import init_random_seed, train_model
+
+__all__ = [
+ 'train_model', 'init_pose_model', 'inference_top_down_pose_model',
+ 'inference_bottom_up_pose_model', 'multi_gpu_test', 'single_gpu_test',
+ 'vis_pose_result', 'get_track_id', 'vis_pose_tracking_result',
+ 'inference_pose_lifter_model', 'vis_3d_pose_result',
+ 'inference_interhand_3d_model', 'extract_pose_sequence',
+ 'inference_mesh_model', 'vis_3d_mesh_result', 'process_mmdet_results',
+ 'init_random_seed'
+]
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/apis/inference.py b/engine/pose_estimation/third-party/ViTPose/mmpose/apis/inference.py
new file mode 100644
index 0000000..64c33e2
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/apis/inference.py
@@ -0,0 +1,961 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os
+import time
+import warnings
+
+import mmcv
+import numpy as np
+import torch
+from mmcv.parallel import collate, scatter
+from mmcv.runner import load_checkpoint
+from PIL import Image
+
+from mmpose.core.post_processing import oks_nms
+from mmpose.datasets.dataset_info import DatasetInfo
+from mmpose.datasets.pipelines import Compose
+from mmpose.models import build_posenet
+from mmpose.utils.hooks import OutputHook
+
+os.environ['KMP_DUPLICATE_LIB_OK'] = 'TRUE'
+
+
+def init_pose_model(config, checkpoint=None, device='cuda:0'):
+ """Initialize a pose model from config file.
+
+ Args:
+ config (str or :obj:`mmcv.Config`): Config file path or the config
+ object.
+ checkpoint (str, optional): Checkpoint path. If left as None, the model
+ will not load any weights.
+
+ Returns:
+ nn.Module: The constructed detector.
+ """
+ if isinstance(config, str):
+ config = mmcv.Config.fromfile(config)
+ elif not isinstance(config, mmcv.Config):
+ raise TypeError('config must be a filename or Config object, '
+ f'but got {type(config)}')
+ config.model.pretrained = None
+ model = build_posenet(config.model)
+ if checkpoint is not None:
+ # load model checkpoint
+ load_checkpoint(model, checkpoint, map_location='cpu')
+ # save the config in the model for convenience
+ model.cfg = config
+ model.to(device)
+ model.eval()
+ return model
+
+
+def _xyxy2xywh(bbox_xyxy):
+ """Transform the bbox format from x1y1x2y2 to xywh.
+
+ Args:
+ bbox_xyxy (np.ndarray): Bounding boxes (with scores), shaped (n, 4) or
+ (n, 5). (left, top, right, bottom, [score])
+
+ Returns:
+ np.ndarray: Bounding boxes (with scores),
+ shaped (n, 4) or (n, 5). (left, top, width, height, [score])
+ """
+ bbox_xywh = bbox_xyxy.copy()
+ bbox_xywh[:, 2] = bbox_xywh[:, 2] - bbox_xywh[:, 0] + 1
+ bbox_xywh[:, 3] = bbox_xywh[:, 3] - bbox_xywh[:, 1] + 1
+
+ return bbox_xywh
+
+
+def _xywh2xyxy(bbox_xywh):
+ """Transform the bbox format from xywh to x1y1x2y2.
+
+ Args:
+ bbox_xywh (ndarray): Bounding boxes (with scores),
+ shaped (n, 4) or (n, 5). (left, top, width, height, [score])
+ Returns:
+ np.ndarray: Bounding boxes (with scores), shaped (n, 4) or
+ (n, 5). (left, top, right, bottom, [score])
+ """
+ bbox_xyxy = bbox_xywh.copy()
+ bbox_xyxy[:, 2] = bbox_xyxy[:, 2] + bbox_xyxy[:, 0] - 1
+ bbox_xyxy[:, 3] = bbox_xyxy[:, 3] + bbox_xyxy[:, 1] - 1
+
+ return bbox_xyxy
+
+
+def _box2cs(cfg, box):
+ """This encodes bbox(x,y,w,h) into (center, scale)
+
+ Args:
+ x, y, w, h
+
+ Returns:
+ tuple: A tuple containing center and scale.
+
+ - np.ndarray[float32](2,): Center of the bbox (x, y).
+ - np.ndarray[float32](2,): Scale of the bbox w & h.
+ """
+
+ x, y, w, h = box[:4]
+ input_size = cfg.data_cfg['image_size']
+ aspect_ratio = input_size[0] / input_size[1]
+ center = np.array([x + w * 0.5, y + h * 0.5], dtype=np.float32)
+
+ if w > aspect_ratio * h:
+ h = w * 1.0 / aspect_ratio
+ elif w < aspect_ratio * h:
+ w = h * aspect_ratio
+
+ # pixel std is 200.0
+ scale = np.array([w / 200.0, h / 200.0], dtype=np.float32)
+ scale = scale * 1.25
+
+ return center, scale
+
+
+
+def _inference_single_pose_model(model,
+ img_or_path,
+ bboxes,
+ dataset='TopDownCocoDataset',
+ dataset_info=None,
+ return_heatmap=False):
+ """Inference human bounding boxes.
+
+ Note:
+ - num_bboxes: N
+ - num_keypoints: K
+
+ Args:
+ model (nn.Module): The loaded pose model.
+ img_or_path (str | np.ndarray): Image filename or loaded image.
+ bboxes (list | np.ndarray): All bounding boxes (with scores),
+ shaped (N, 4) or (N, 5). (left, top, width, height, [score])
+ where N is number of bounding boxes.
+ dataset (str): Dataset name. Deprecated.
+ dataset_info (DatasetInfo): A class containing all dataset info.
+ outputs (list[str] | tuple[str]): Names of layers whose output is
+ to be returned, default: None
+
+ Returns:
+ ndarray[NxKx3]: Predicted pose x, y, score.
+ heatmap[N, K, H, W]: Model output heatmap.
+ """
+
+ cfg = model.cfg
+ device = next(model.parameters()).device
+ if device.type == 'cpu':
+ device = -1
+
+ # build the data pipeline
+
+ test_pipeline = Compose(cfg.test_pipeline)
+
+ assert len(bboxes[0]) in [4, 5]
+
+ if dataset_info is not None:
+ dataset_name = dataset_info.dataset_name
+ flip_pairs = dataset_info.flip_pairs
+ else:
+ warnings.warn(
+ 'dataset is deprecated.'
+ 'Please set `dataset_info` in the config.'
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 for details.',
+ DeprecationWarning)
+ # TODO: These will be removed in the later versions.
+ if dataset in ('TopDownCocoDataset', 'TopDownOCHumanDataset',
+ 'AnimalMacaqueDataset'):
+ flip_pairs = [[1, 2], [3, 4], [5, 6], [7, 8], [9, 10], [11, 12],
+ [13, 14], [15, 16]]
+ elif dataset == 'TopDownCocoWholeBodyDataset':
+ body = [[1, 2], [3, 4], [5, 6], [7, 8], [9, 10], [11, 12],
+ [13, 14], [15, 16]]
+ foot = [[17, 20], [18, 21], [19, 22]]
+
+ face = [[23, 39], [24, 38], [25, 37], [26, 36], [27, 35], [28, 34],
+ [29, 33], [30, 32], [40, 49], [41, 48], [42, 47], [43, 46],
+ [44, 45], [54, 58], [55, 57], [59, 68], [60, 67], [61, 66],
+ [62, 65], [63, 70], [64, 69], [71, 77], [72, 76], [73, 75],
+ [78, 82], [79, 81], [83, 87], [84, 86], [88, 90]]
+
+ hand = [[91, 112], [92, 113], [93, 114], [94, 115], [95, 116],
+ [96, 117], [97, 118], [98, 119], [99, 120], [100, 121],
+ [101, 122], [102, 123], [103, 124], [104, 125], [105, 126],
+ [106, 127], [107, 128], [108, 129], [109, 130], [110, 131],
+ [111, 132]]
+ flip_pairs = body + foot + face + hand
+ elif dataset == 'TopDownAicDataset':
+ flip_pairs = [[0, 3], [1, 4], [2, 5], [6, 9], [7, 10], [8, 11]]
+ elif dataset == 'TopDownMpiiDataset':
+ flip_pairs = [[0, 5], [1, 4], [2, 3], [10, 15], [11, 14], [12, 13]]
+ elif dataset == 'TopDownMpiiTrbDataset':
+ flip_pairs = [[0, 1], [2, 3], [4, 5], [6, 7], [8, 9], [10, 11],
+ [14, 15], [16, 22], [28, 34], [17, 23], [29, 35],
+ [18, 24], [30, 36], [19, 25], [31, 37], [20, 26],
+ [32, 38], [21, 27], [33, 39]]
+ elif dataset in ('OneHand10KDataset', 'FreiHandDataset',
+ 'PanopticDataset', 'InterHand2DDataset'):
+ flip_pairs = []
+ elif dataset in 'Face300WDataset':
+ flip_pairs = [[0, 16], [1, 15], [2, 14], [3, 13], [4, 12], [5, 11],
+ [6, 10], [7, 9], [17, 26], [18, 25], [19, 24],
+ [20, 23], [21, 22], [31, 35], [32, 34], [36, 45],
+ [37, 44], [38, 43], [39, 42], [40, 47], [41, 46],
+ [48, 54], [49, 53], [50, 52], [61, 63], [60, 64],
+ [67, 65], [58, 56], [59, 55]]
+
+ elif dataset in 'FaceAFLWDataset':
+ flip_pairs = [[0, 5], [1, 4], [2, 3], [6, 11], [7, 10], [8, 9],
+ [12, 14], [15, 17]]
+
+ elif dataset in 'FaceCOFWDataset':
+ flip_pairs = [[0, 1], [4, 6], [2, 3], [5, 7], [8, 9], [10, 11],
+ [12, 14], [16, 17], [13, 15], [18, 19], [22, 23]]
+
+ elif dataset in 'FaceWFLWDataset':
+ flip_pairs = [[0, 32], [1, 31], [2, 30], [3, 29], [4, 28], [5, 27],
+ [6, 26], [7, 25], [8, 24], [9, 23], [10, 22],
+ [11, 21], [12, 20], [13, 19], [14, 18], [15, 17],
+ [33, 46], [34, 45], [35, 44], [36, 43], [37, 42],
+ [38, 50], [39, 49], [40, 48], [41, 47], [60, 72],
+ [61, 71], [62, 70], [63, 69], [64, 68], [65, 75],
+ [66, 74], [67, 73], [55, 59], [56, 58], [76, 82],
+ [77, 81], [78, 80], [87, 83], [86, 84], [88, 92],
+ [89, 91], [95, 93], [96, 97]]
+
+ elif dataset in 'AnimalFlyDataset':
+ flip_pairs = [[1, 2], [6, 18], [7, 19], [8, 20], [9, 21], [10, 22],
+ [11, 23], [12, 24], [13, 25], [14, 26], [15, 27],
+ [16, 28], [17, 29], [30, 31]]
+ elif dataset in 'AnimalHorse10Dataset':
+ flip_pairs = []
+
+ elif dataset in 'AnimalLocustDataset':
+ flip_pairs = [[5, 20], [6, 21], [7, 22], [8, 23], [9, 24],
+ [10, 25], [11, 26], [12, 27], [13, 28], [14, 29],
+ [15, 30], [16, 31], [17, 32], [18, 33], [19, 34]]
+
+ elif dataset in 'AnimalZebraDataset':
+ flip_pairs = [[3, 4], [5, 6]]
+
+ elif dataset in 'AnimalPoseDataset':
+ flip_pairs = [[0, 1], [2, 3], [8, 9], [10, 11], [12, 13], [14, 15],
+ [16, 17], [18, 19]]
+ else:
+ raise NotImplementedError()
+ dataset_name = dataset
+
+ batch_data = []
+ for bbox in bboxes:
+ center, scale = _box2cs(cfg, bbox)
+
+ # prepare data
+ data = {
+ 'center':
+ center,
+ 'scale':
+ scale,
+ 'bbox_score':
+ bbox[4] if len(bbox) == 5 else 1,
+ 'bbox_id':
+ 0, # need to be assigned if batch_size > 1
+ 'dataset':
+ dataset_name,
+ 'joints_3d':
+ np.zeros((cfg.data_cfg.num_joints, 3), dtype=np.float32),
+ 'joints_3d_visible':
+ np.zeros((cfg.data_cfg.num_joints, 3), dtype=np.float32),
+ 'rotation':
+ 0,
+ 'ann_info': {
+ 'image_size': np.array(cfg.data_cfg['image_size']),
+ 'num_joints': cfg.data_cfg['num_joints'],
+ 'flip_pairs': flip_pairs
+ }
+ }
+ if isinstance(img_or_path, np.ndarray):
+ data['img'] = img_or_path
+ else:
+ data['image_file'] = img_or_path
+
+ data = test_pipeline(data)
+ batch_data.append(data)
+
+ batch_data = collate(batch_data, samples_per_gpu=len(batch_data))
+ batch_data = scatter(batch_data, [device])[0]
+ # forward the model
+ start = time.time()
+ with torch.no_grad():
+ result = model(
+ img=batch_data['img'],
+ img_metas=batch_data['img_metas'],
+ return_loss=False,
+ return_heatmap=return_heatmap)
+ print(f'model forward time: {time.time() - start}')
+ return result['preds'], result['output_heatmap']
+
+
+def _build_batch_data(model, imgs,
+ bboxes,
+ dataset='TopDownCocoDataset',
+ dataset_info=None):
+ """Inference human bounding boxes.
+
+ Note:
+ - num_bboxes: N
+ - num_keypoints: K
+
+ Args:
+ model (nn.Module): The loaded pose model.
+ img_or_path (str | np.ndarray): Image filename or loaded image.
+ bboxes (list | np.ndarray): All bounding boxes (with scores),
+ shaped (N, 4) or (N, 5). (left, top, width, height, [score])
+ where N is number of bounding boxes.
+ dataset (str): Dataset name. Deprecated.
+ dataset_info (DatasetInfo): A class containing all dataset info.
+ outputs (list[str] | tuple[str]): Names of layers whose output is
+ to be returned, default: None
+
+ Returns:
+ ndarray[NxKx3]: Predicted pose x, y, score.
+ heatmap[N, K, H, W]: Model output heatmap.
+ """
+
+ cfg = model.cfg
+ device = next(model.parameters()).device
+ if device.type == 'cpu':
+ device = -1
+
+ # build the data pipeline
+
+ test_pipeline = Compose(cfg.test_pipeline)
+
+ assert len(bboxes[0]) in [4, 5]
+
+ if dataset_info is not None:
+ dataset_name = dataset_info.dataset_name
+ flip_pairs = dataset_info.flip_pairs
+ else:
+ warnings.warn(
+ 'dataset is deprecated.'
+ 'Please set `dataset_info` in the config.'
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 for details.',
+ DeprecationWarning)
+ # TODO: These will be removed in the later versions.
+ if dataset in ('TopDownCocoDataset', 'TopDownOCHumanDataset',
+ 'AnimalMacaqueDataset'):
+ flip_pairs = [[1, 2], [3, 4], [5, 6], [7, 8], [9, 10], [11, 12],
+ [13, 14], [15, 16]]
+ else:
+ raise NotImplementedError()
+ dataset_name = dataset
+
+ batch_data = []
+ for bbox, img in zip(bboxes, imgs):
+ center, scale = _box2cs(cfg, bbox)
+
+ # prepare data
+ data = {
+ 'center':
+ center,
+ 'scale':
+ scale,
+ 'bbox_score':
+ bbox[4] if len(bbox) == 5 else 1,
+ 'bbox_id':
+ 0, # need to be assigned if batch_size > 1
+ 'dataset':
+ dataset_name,
+ 'joints_3d':
+ np.zeros((cfg.data_cfg.num_joints, 3), dtype=np.float32),
+ 'joints_3d_visible':
+ np.zeros((cfg.data_cfg.num_joints, 3), dtype=np.float32),
+ 'rotation':
+ 0,
+ 'ann_info': {
+ 'image_size': np.array(cfg.data_cfg['image_size']),
+ 'num_joints': cfg.data_cfg['num_joints'],
+ 'flip_pairs': flip_pairs
+ }
+ }
+ if isinstance(img, np.ndarray):
+ data['img'] = img
+
+ data = test_pipeline(data)
+ batch_data.append(data)
+
+ batch_data = collate(batch_data, samples_per_gpu=len(batch_data))
+ batch_data = scatter(batch_data, [device])[0]
+ return batch_data
+
+
+
+def batch_inference_pose_model(model, imgs, bboxes, dataset_info=None, return_heatmap=False):
+ if (dataset_info is None and hasattr(model, 'cfg')
+ and 'dataset_info' in model.cfg):
+ dataset_info = DatasetInfo(model.cfg.dataset_info)
+
+ pose_results = []
+ returned_outputs = []
+
+ if bboxes is None or len(bboxes) < 1:
+ return pose_results, returned_outputs
+
+ bboxes = np.array([box for box in bboxes])
+ bboxes_xyxy = bboxes
+ bboxes_xywh = _xyxy2xywh(bboxes)
+ outputs = None
+ with OutputHook(model, outputs=outputs, as_tensor=False) as h:
+ # poses is results['pred'] # N x 17x 3
+ batch_data = _build_batch_data(model, imgs, bboxes_xywh, 'TopDownCocoDataset', dataset_info)
+ with torch.no_grad():
+ result = model(
+ img=batch_data['img'],
+ img_metas=batch_data['img_metas'],
+ return_loss=False,
+ return_heatmap=return_heatmap)
+ poses = result['preds']
+ returned_outputs.append(h.layer_outputs)
+
+
+ return poses, bboxes_xyxy
+
+def inference_top_down_pose_model(model,
+ img_or_path,
+ person_results=None,
+ bbox_thr=None,
+ format='xywh',
+ dataset='TopDownCocoDataset',
+ dataset_info=None,
+ return_heatmap=False,
+ outputs=None):
+ """Inference a single image with a list of person bounding boxes.
+
+ Note:
+ - num_people: P
+ - num_keypoints: K
+ - bbox height: H
+ - bbox width: W
+
+ Args:
+ model (nn.Module): The loaded pose model.
+ img_or_path (str| np.ndarray): Image filename or loaded image.
+ person_results (list(dict), optional): a list of detected persons that
+ contains ``bbox`` and/or ``track_id``:
+
+ - ``bbox`` (4, ) or (5, ): The person bounding box, which contains
+ 4 box coordinates (and score).
+ - ``track_id`` (int): The unique id for each human instance. If
+ not provided, a dummy person result with a bbox covering
+ the entire image will be used. Default: None.
+ bbox_thr (float | None): Threshold for bounding boxes. Only bboxes
+ with higher scores will be fed into the pose detector.
+ If bbox_thr is None, all boxes will be used.
+ format (str): bbox format ('xyxy' | 'xywh'). Default: 'xywh'.
+
+ - `xyxy` means (left, top, right, bottom),
+ - `xywh` means (left, top, width, height).
+ dataset (str): Dataset name, e.g. 'TopDownCocoDataset'.
+ It is deprecated. Please use dataset_info instead.
+ dataset_info (DatasetInfo): A class containing all dataset info.
+ return_heatmap (bool) : Flag to return heatmap, default: False
+ outputs (list(str) | tuple(str)) : Names of layers whose outputs
+ need to be returned. Default: None.
+
+ Returns:
+ tuple:
+ - pose_results (list[dict]): The bbox & pose info. \
+ Each item in the list is a dictionary, \
+ containing the bbox: (left, top, right, bottom, [score]) \
+ and the pose (ndarray[Kx3]): x, y, score.
+ - returned_outputs (list[dict[np.ndarray[N, K, H, W] | \
+ torch.Tensor[N, K, H, W]]]): \
+ Output feature maps from layers specified in `outputs`. \
+ Includes 'heatmap' if `return_heatmap` is True.
+ """
+ # get dataset info
+ if (dataset_info is None and hasattr(model, 'cfg')
+ and 'dataset_info' in model.cfg):
+ dataset_info = DatasetInfo(model.cfg.dataset_info)
+ if dataset_info is None:
+ warnings.warn(
+ 'dataset is deprecated.'
+ 'Please set `dataset_info` in the config.'
+ 'Check https://github.com/open-mmlab/mmpose/pull/663'
+ ' for details.', DeprecationWarning)
+
+ # only two kinds of bbox format is supported.
+ assert format in ['xyxy', 'xywh']
+
+ pose_results = []
+ returned_outputs = []
+
+ if person_results is None:
+ # create dummy person results
+ if isinstance(img_or_path, str):
+ width, height = Image.open(img_or_path).size
+ else:
+ height, width = img_or_path.shape[:2]
+ person_results = [{'bbox': np.array([0, 0, width, height])}]
+
+ if len(person_results) == 0:
+ return pose_results, returned_outputs
+
+ # Change for-loop preprocess each bbox to preprocess all bboxes at once.
+ bboxes = np.array([box['bbox'] for box in person_results])
+
+ # Select bboxes by score threshold
+ if bbox_thr is not None:
+ assert bboxes.shape[1] == 5
+ valid_idx = np.where(bboxes[:, 4] > bbox_thr)[0]
+ bboxes = bboxes[valid_idx]
+ person_results = [person_results[i] for i in valid_idx]
+
+ if format == 'xyxy':
+ bboxes_xyxy = bboxes
+ bboxes_xywh = _xyxy2xywh(bboxes)
+ else:
+ # format is already 'xywh'
+ bboxes_xywh = bboxes
+ bboxes_xyxy = _xywh2xyxy(bboxes)
+
+ # if bbox_thr remove all bounding box
+ if len(bboxes_xywh) == 0:
+ return [], []
+
+ with OutputHook(model, outputs=outputs, as_tensor=False) as h:
+ # poses is results['pred'] # N x 17x 3
+ poses, heatmap = _inference_single_pose_model(
+ model,
+ img_or_path,
+ bboxes_xywh,
+ dataset=dataset,
+ dataset_info=dataset_info,
+ return_heatmap=return_heatmap)
+
+ if return_heatmap:
+ h.layer_outputs['heatmap'] = heatmap
+
+ returned_outputs.append(h.layer_outputs)
+
+ assert len(poses) == len(person_results), print(
+ len(poses), len(person_results), len(bboxes_xyxy))
+ for pose, person_result, bbox_xyxy in zip(poses, person_results,
+ bboxes_xyxy):
+ pose_result = person_result.copy()
+ pose_result['keypoints'] = pose
+ pose_result['bbox'] = bbox_xyxy
+ pose_results.append(pose_result)
+
+ return pose_results, returned_outputs
+
+
+def inference_bottom_up_pose_model(model,
+ img_or_path,
+ dataset='BottomUpCocoDataset',
+ dataset_info=None,
+ pose_nms_thr=0.9,
+ return_heatmap=False,
+ outputs=None):
+ """Inference a single image with a bottom-up pose model.
+
+ Note:
+ - num_people: P
+ - num_keypoints: K
+ - bbox height: H
+ - bbox width: W
+
+ Args:
+ model (nn.Module): The loaded pose model.
+ img_or_path (str| np.ndarray): Image filename or loaded image.
+ dataset (str): Dataset name, e.g. 'BottomUpCocoDataset'.
+ It is deprecated. Please use dataset_info instead.
+ dataset_info (DatasetInfo): A class containing all dataset info.
+ pose_nms_thr (float): retain oks overlap < pose_nms_thr, default: 0.9.
+ return_heatmap (bool) : Flag to return heatmap, default: False.
+ outputs (list(str) | tuple(str)) : Names of layers whose outputs
+ need to be returned, default: None.
+
+ Returns:
+ tuple:
+ - pose_results (list[np.ndarray]): The predicted pose info. \
+ The length of the list is the number of people (P). \
+ Each item in the list is a ndarray, containing each \
+ person's pose (np.ndarray[Kx3]): x, y, score.
+ - returned_outputs (list[dict[np.ndarray[N, K, H, W] | \
+ torch.Tensor[N, K, H, W]]]): \
+ Output feature maps from layers specified in `outputs`. \
+ Includes 'heatmap' if `return_heatmap` is True.
+ """
+ # get dataset info
+ if (dataset_info is None and hasattr(model, 'cfg')
+ and 'dataset_info' in model.cfg):
+ dataset_info = DatasetInfo(model.cfg.dataset_info)
+
+ if dataset_info is not None:
+ dataset_name = dataset_info.dataset_name
+ flip_index = dataset_info.flip_index
+ sigmas = getattr(dataset_info, 'sigmas', None)
+ else:
+ warnings.warn(
+ 'dataset is deprecated.'
+ 'Please set `dataset_info` in the config.'
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 for details.',
+ DeprecationWarning)
+ assert (dataset == 'BottomUpCocoDataset')
+ dataset_name = dataset
+ flip_index = [0, 2, 1, 4, 3, 6, 5, 8, 7, 10, 9, 12, 11, 14, 13, 16, 15]
+ sigmas = None
+
+ pose_results = []
+ returned_outputs = []
+
+ cfg = model.cfg
+ device = next(model.parameters()).device
+ if device.type == 'cpu':
+ device = -1
+
+ # build the data pipeline
+ test_pipeline = Compose(cfg.test_pipeline)
+
+ # prepare data
+ data = {
+ 'dataset': dataset_name,
+ 'ann_info': {
+ 'image_size': np.array(cfg.data_cfg['image_size']),
+ 'num_joints': cfg.data_cfg['num_joints'],
+ 'flip_index': flip_index,
+ }
+ }
+ if isinstance(img_or_path, np.ndarray):
+ data['img'] = img_or_path
+ else:
+ data['image_file'] = img_or_path
+
+ data = test_pipeline(data)
+ data = collate([data], samples_per_gpu=1)
+ data = scatter(data, [device])[0]
+
+ with OutputHook(model, outputs=outputs, as_tensor=False) as h:
+ # forward the model
+ with torch.no_grad():
+ result = model(
+ img=data['img'],
+ img_metas=data['img_metas'],
+ return_loss=False,
+ return_heatmap=return_heatmap)
+
+ if return_heatmap:
+ h.layer_outputs['heatmap'] = result['output_heatmap']
+
+ returned_outputs.append(h.layer_outputs)
+
+ for idx, pred in enumerate(result['preds']):
+ area = (np.max(pred[:, 0]) - np.min(pred[:, 0])) * (
+ np.max(pred[:, 1]) - np.min(pred[:, 1]))
+ pose_results.append({
+ 'keypoints': pred[:, :3],
+ 'score': result['scores'][idx],
+ 'area': area,
+ })
+
+ # pose nms
+ score_per_joint = cfg.model.test_cfg.get('score_per_joint', False)
+ keep = oks_nms(
+ pose_results,
+ pose_nms_thr,
+ sigmas,
+ score_per_joint=score_per_joint)
+ pose_results = [pose_results[_keep] for _keep in keep]
+
+ return pose_results, returned_outputs
+
+
+def vis_pose_result(model,
+ img,
+ result,
+ radius=4,
+ thickness=1,
+ kpt_score_thr=0.3,
+ bbox_color='green',
+ dataset='TopDownCocoDataset',
+ dataset_info=None,
+ show=False,
+ out_file=None):
+ """Visualize the detection results on the image.
+
+ Args:
+ model (nn.Module): The loaded detector.
+ img (str | np.ndarray): Image filename or loaded image.
+ result (list[dict]): The results to draw over `img`
+ (bbox_result, pose_result).
+ radius (int): Radius of circles.
+ thickness (int): Thickness of lines.
+ kpt_score_thr (float): The threshold to visualize the keypoints.
+ skeleton (list[tuple()]): Default None.
+ show (bool): Whether to show the image. Default True.
+ out_file (str|None): The filename of the output visualization image.
+ """
+
+ # get dataset info
+ if (dataset_info is None and hasattr(model, 'cfg')
+ and 'dataset_info' in model.cfg):
+ dataset_info = DatasetInfo(model.cfg.dataset_info)
+
+ if dataset_info is not None:
+ skeleton = dataset_info.skeleton
+ pose_kpt_color = dataset_info.pose_kpt_color
+ pose_link_color = dataset_info.pose_link_color
+ else:
+ warnings.warn(
+ 'dataset is deprecated.'
+ 'Please set `dataset_info` in the config.'
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 for details.',
+ DeprecationWarning)
+ # TODO: These will be removed in the later versions.
+ palette = np.array([[255, 128, 0], [255, 153, 51], [255, 178, 102],
+ [230, 230, 0], [255, 153, 255], [153, 204, 255],
+ [255, 102, 255], [255, 51, 255], [102, 178, 255],
+ [51, 153, 255], [255, 153, 153], [255, 102, 102],
+ [255, 51, 51], [153, 255, 153], [102, 255, 102],
+ [51, 255, 51], [0, 255, 0], [0, 0, 255],
+ [255, 0, 0], [255, 255, 255]])
+
+ if dataset in ('TopDownCocoDataset', 'BottomUpCocoDataset',
+ 'TopDownOCHumanDataset', 'AnimalMacaqueDataset'):
+ # show the results
+ skeleton = [[15, 13], [13, 11], [16, 14], [14, 12], [11, 12],
+ [5, 11], [6, 12], [5, 6], [5, 7], [6, 8], [7, 9],
+ [8, 10], [1, 2], [0, 1], [0, 2], [1, 3], [2, 4],
+ [3, 5], [4, 6]]
+
+ pose_link_color = palette[[
+ 0, 0, 0, 0, 7, 7, 7, 9, 9, 9, 9, 9, 16, 16, 16, 16, 16, 16, 16
+ ]]
+ pose_kpt_color = palette[[
+ 16, 16, 16, 16, 16, 9, 9, 9, 9, 9, 9, 0, 0, 0, 0, 0, 0
+ ]]
+
+ elif dataset == 'TopDownCocoWholeBodyDataset':
+ # show the results
+ skeleton = [[15, 13], [13, 11], [16, 14], [14, 12], [11, 12],
+ [5, 11], [6, 12], [5, 6], [5, 7], [6, 8], [7, 9],
+ [8, 10], [1, 2], [0, 1], [0, 2],
+ [1, 3], [2, 4], [3, 5], [4, 6], [15, 17], [15, 18],
+ [15, 19], [16, 20], [16, 21], [16, 22], [91, 92],
+ [92, 93], [93, 94], [94, 95], [91, 96], [96, 97],
+ [97, 98], [98, 99], [91, 100], [100, 101], [101, 102],
+ [102, 103], [91, 104], [104, 105], [105, 106],
+ [106, 107], [91, 108], [108, 109], [109, 110],
+ [110, 111], [112, 113], [113, 114], [114, 115],
+ [115, 116], [112, 117], [117, 118], [118, 119],
+ [119, 120], [112, 121], [121, 122], [122, 123],
+ [123, 124], [112, 125], [125, 126], [126, 127],
+ [127, 128], [112, 129], [129, 130], [130, 131],
+ [131, 132]]
+
+ pose_link_color = palette[[
+ 0, 0, 0, 0, 7, 7, 7, 9, 9, 9, 9, 9, 16, 16, 16, 16, 16, 16, 16
+ ] + [16, 16, 16, 16, 16, 16] + [
+ 0, 0, 0, 0, 4, 4, 4, 4, 8, 8, 8, 8, 12, 12, 12, 12, 16, 16, 16,
+ 16
+ ] + [
+ 0, 0, 0, 0, 4, 4, 4, 4, 8, 8, 8, 8, 12, 12, 12, 12, 16, 16, 16,
+ 16
+ ]]
+ pose_kpt_color = palette[
+ [16, 16, 16, 16, 16, 9, 9, 9, 9, 9, 9, 0, 0, 0, 0, 0, 0] +
+ [0, 0, 0, 0, 0, 0] + [19] * (68 + 42)]
+
+ elif dataset == 'TopDownAicDataset':
+ skeleton = [[2, 1], [1, 0], [0, 13], [13, 3], [3, 4], [4, 5],
+ [8, 7], [7, 6], [6, 9], [9, 10], [10, 11], [12, 13],
+ [0, 6], [3, 9]]
+
+ pose_link_color = palette[[
+ 9, 9, 9, 9, 9, 9, 16, 16, 16, 16, 16, 0, 7, 7
+ ]]
+ pose_kpt_color = palette[[
+ 9, 9, 9, 9, 9, 9, 16, 16, 16, 16, 16, 16, 0, 0
+ ]]
+
+ elif dataset == 'TopDownMpiiDataset':
+ skeleton = [[0, 1], [1, 2], [2, 6], [6, 3], [3, 4], [4, 5], [6, 7],
+ [7, 8], [8, 9], [8, 12], [12, 11], [11, 10], [8, 13],
+ [13, 14], [14, 15]]
+
+ pose_link_color = palette[[
+ 16, 16, 16, 16, 16, 16, 7, 7, 0, 9, 9, 9, 9, 9, 9
+ ]]
+ pose_kpt_color = palette[[
+ 16, 16, 16, 16, 16, 16, 7, 7, 0, 0, 9, 9, 9, 9, 9, 9
+ ]]
+
+ elif dataset == 'TopDownMpiiTrbDataset':
+ skeleton = [[12, 13], [13, 0], [13, 1], [0, 2], [1, 3], [2, 4],
+ [3, 5], [0, 6], [1, 7], [6, 7], [6, 8], [7,
+ 9], [8, 10],
+ [9, 11], [14, 15], [16, 17], [18, 19], [20, 21],
+ [22, 23], [24, 25], [26, 27], [28, 29], [30, 31],
+ [32, 33], [34, 35], [36, 37], [38, 39]]
+
+ pose_link_color = palette[[16] * 14 + [19] * 13]
+ pose_kpt_color = palette[[16] * 14 + [0] * 26]
+
+ elif dataset in ('OneHand10KDataset', 'FreiHandDataset',
+ 'PanopticDataset'):
+ skeleton = [[0, 1], [1, 2], [2, 3], [3, 4], [0, 5], [5, 6], [6, 7],
+ [7, 8], [0, 9], [9, 10], [10, 11], [11, 12], [0, 13],
+ [13, 14], [14, 15], [15, 16], [0, 17], [17, 18],
+ [18, 19], [19, 20]]
+
+ pose_link_color = palette[[
+ 0, 0, 0, 0, 4, 4, 4, 4, 8, 8, 8, 8, 12, 12, 12, 12, 16, 16, 16,
+ 16
+ ]]
+ pose_kpt_color = palette[[
+ 0, 0, 0, 0, 0, 4, 4, 4, 4, 8, 8, 8, 8, 12, 12, 12, 12, 16, 16,
+ 16, 16
+ ]]
+
+ elif dataset == 'InterHand2DDataset':
+ skeleton = [[0, 1], [1, 2], [2, 3], [4, 5], [5, 6], [6, 7], [8, 9],
+ [9, 10], [10, 11], [12, 13], [13, 14], [14, 15],
+ [16, 17], [17, 18], [18, 19], [3, 20], [7, 20],
+ [11, 20], [15, 20], [19, 20]]
+
+ pose_link_color = palette[[
+ 0, 0, 0, 4, 4, 4, 8, 8, 8, 12, 12, 12, 16, 16, 16, 0, 4, 8, 12,
+ 16
+ ]]
+ pose_kpt_color = palette[[
+ 0, 0, 0, 0, 4, 4, 4, 4, 8, 8, 8, 8, 12, 12, 12, 12, 16, 16, 16,
+ 16, 0
+ ]]
+
+ elif dataset == 'Face300WDataset':
+ # show the results
+ skeleton = []
+
+ pose_link_color = palette[[]]
+ pose_kpt_color = palette[[19] * 68]
+ kpt_score_thr = 0
+
+ elif dataset == 'FaceAFLWDataset':
+ # show the results
+ skeleton = []
+
+ pose_link_color = palette[[]]
+ pose_kpt_color = palette[[19] * 19]
+ kpt_score_thr = 0
+
+ elif dataset == 'FaceCOFWDataset':
+ # show the results
+ skeleton = []
+
+ pose_link_color = palette[[]]
+ pose_kpt_color = palette[[19] * 29]
+ kpt_score_thr = 0
+
+ elif dataset == 'FaceWFLWDataset':
+ # show the results
+ skeleton = []
+
+ pose_link_color = palette[[]]
+ pose_kpt_color = palette[[19] * 98]
+ kpt_score_thr = 0
+
+ elif dataset == 'AnimalHorse10Dataset':
+ skeleton = [[0, 1], [1, 12], [12, 16], [16, 21], [21, 17],
+ [17, 11], [11, 10], [10, 8], [8, 9], [9, 12], [2, 3],
+ [3, 4], [5, 6], [6, 7], [13, 14], [14, 15], [18, 19],
+ [19, 20]]
+
+ pose_link_color = palette[[4] * 10 + [6] * 2 + [6] * 2 + [7] * 2 +
+ [7] * 2]
+ pose_kpt_color = palette[[
+ 4, 4, 6, 6, 6, 6, 6, 6, 4, 4, 4, 4, 4, 7, 7, 7, 4, 4, 7, 7, 7,
+ 4
+ ]]
+
+ elif dataset == 'AnimalFlyDataset':
+ skeleton = [[1, 0], [2, 0], [3, 0], [4, 3], [5, 4], [7, 6], [8, 7],
+ [9, 8], [11, 10], [12, 11], [13, 12], [15, 14],
+ [16, 15], [17, 16], [19, 18], [20, 19], [21, 20],
+ [23, 22], [24, 23], [25, 24], [27, 26], [28, 27],
+ [29, 28], [30, 3], [31, 3]]
+
+ pose_link_color = palette[[0] * 25]
+ pose_kpt_color = palette[[0] * 32]
+
+ elif dataset == 'AnimalLocustDataset':
+ skeleton = [[1, 0], [2, 1], [3, 2], [4, 3], [6, 5], [7, 6], [9, 8],
+ [10, 9], [11, 10], [13, 12], [14, 13], [15, 14],
+ [17, 16], [18, 17], [19, 18], [21, 20], [22, 21],
+ [24, 23], [25, 24], [26, 25], [28, 27], [29, 28],
+ [30, 29], [32, 31], [33, 32], [34, 33]]
+
+ pose_link_color = palette[[0] * 26]
+ pose_kpt_color = palette[[0] * 35]
+
+ elif dataset == 'AnimalZebraDataset':
+ skeleton = [[1, 0], [2, 1], [3, 2], [4, 2], [5, 7], [6, 7], [7, 2],
+ [8, 7]]
+
+ pose_link_color = palette[[0] * 8]
+ pose_kpt_color = palette[[0] * 9]
+
+ elif dataset in 'AnimalPoseDataset':
+ skeleton = [[0, 1], [0, 2], [1, 3], [0, 4], [1, 4], [4, 5], [5, 7],
+ [6, 7], [5, 8], [8, 12], [12, 16], [5, 9], [9, 13],
+ [13, 17], [6, 10], [10, 14], [14, 18], [6, 11],
+ [11, 15], [15, 19]]
+
+ pose_link_color = palette[[0] * 20]
+ pose_kpt_color = palette[[0] * 20]
+ else:
+ NotImplementedError()
+
+ if hasattr(model, 'module'):
+ model = model.module
+
+ img = model.show_result(
+ img,
+ result,
+ skeleton,
+ radius=radius,
+ thickness=thickness,
+ pose_kpt_color=pose_kpt_color,
+ pose_link_color=pose_link_color,
+ kpt_score_thr=kpt_score_thr,
+ bbox_color=bbox_color,
+ show=show,
+ out_file=out_file)
+
+ return img
+
+
+def process_mmdet_results(mmdet_results, cat_id=1):
+ """Process mmdet results, and return a list of bboxes.
+
+ Args:
+ mmdet_results (list|tuple): mmdet results.
+ cat_id (int): category id (default: 1 for human)
+
+ Returns:
+ person_results (list): a list of detected bounding boxes
+ """
+ if isinstance(mmdet_results, tuple):
+ det_results = mmdet_results[0]
+ else:
+ det_results = mmdet_results
+
+ bboxes = det_results[cat_id - 1]
+
+ person_results = []
+ for bbox in bboxes:
+ person = {}
+ person['bbox'] = bbox
+ person_results.append(person)
+
+ return person_results
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/apis/inference_3d.py b/engine/pose_estimation/third-party/ViTPose/mmpose/apis/inference_3d.py
new file mode 100644
index 0000000..f59f20a
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/apis/inference_3d.py
@@ -0,0 +1,791 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import warnings
+
+import numpy as np
+import torch
+from mmcv.parallel import collate, scatter
+
+from mmpose.datasets.pipelines import Compose
+from .inference import _box2cs, _xywh2xyxy, _xyxy2xywh
+
+
+def extract_pose_sequence(pose_results, frame_idx, causal, seq_len, step=1):
+ """Extract the target frame from 2D pose results, and pad the sequence to a
+ fixed length.
+
+ Args:
+ pose_results (list[list[dict]]): Multi-frame pose detection results
+ stored in a nested list. Each element of the outer list is the
+ pose detection results of a single frame, and each element of the
+ inner list is the pose information of one person, which contains:
+
+ - keypoints (ndarray[K, 2 or 3]): x, y, [score]
+ - track_id (int): unique id of each person, required \
+ when ``with_track_id==True``.
+ - bbox ((4, ) or (5, )): left, right, top, bottom, [score]
+
+ frame_idx (int): The index of the frame in the original video.
+ causal (bool): If True, the target frame is the last frame in
+ a sequence. Otherwise, the target frame is in the middle of
+ a sequence.
+ seq_len (int): The number of frames in the input sequence.
+ step (int): Step size to extract frames from the video.
+
+ Returns:
+ list[list[dict]]: Multi-frame pose detection results stored \
+ in a nested list with a length of seq_len.
+ """
+
+ if causal:
+ frames_left = seq_len - 1
+ frames_right = 0
+ else:
+ frames_left = (seq_len - 1) // 2
+ frames_right = frames_left
+ num_frames = len(pose_results)
+
+ # get the padded sequence
+ pad_left = max(0, frames_left - frame_idx // step)
+ pad_right = max(0, frames_right - (num_frames - 1 - frame_idx) // step)
+ start = max(frame_idx % step, frame_idx - frames_left * step)
+ end = min(num_frames - (num_frames - 1 - frame_idx) % step,
+ frame_idx + frames_right * step + 1)
+ pose_results_seq = [pose_results[0]] * pad_left + \
+ pose_results[start:end:step] + [pose_results[-1]] * pad_right
+ return pose_results_seq
+
+
+def _gather_pose_lifter_inputs(pose_results,
+ bbox_center,
+ bbox_scale,
+ norm_pose_2d=False):
+ """Gather input data (keypoints and track_id) for pose lifter model.
+
+ Note:
+ - The temporal length of the pose detection results: T
+ - The number of the person instances: N
+ - The number of the keypoints: K
+ - The channel number of each keypoint: C
+
+ Args:
+ pose_results (List[List[Dict]]): Multi-frame pose detection results
+ stored in a nested list. Each element of the outer list is the
+ pose detection results of a single frame, and each element of the
+ inner list is the pose information of one person, which contains:
+
+ - keypoints (ndarray[K, 2 or 3]): x, y, [score]
+ - track_id (int): unique id of each person, required when
+ ``with_track_id==True```
+ - bbox ((4, ) or (5, )): left, right, top, bottom, [score]
+
+ bbox_center (ndarray[1, 2]): x, y. The average center coordinate of the
+ bboxes in the dataset.
+ bbox_scale (int|float): The average scale of the bboxes in the dataset.
+ norm_pose_2d (bool): If True, scale the bbox (along with the 2D
+ pose) to bbox_scale, and move the bbox (along with the 2D pose) to
+ bbox_center. Default: False.
+
+ Returns:
+ list[list[dict]]: Multi-frame pose detection results
+ stored in a nested list. Each element of the outer list is the
+ pose detection results of a single frame, and each element of the
+ inner list is the pose information of one person, which contains:
+
+ - keypoints (ndarray[K, 2 or 3]): x, y, [score]
+ - track_id (int): unique id of each person, required when
+ ``with_track_id==True``
+ """
+ sequence_inputs = []
+ for frame in pose_results:
+ frame_inputs = []
+ for res in frame:
+ inputs = dict()
+
+ if norm_pose_2d:
+ bbox = res['bbox']
+ center = np.array([[(bbox[0] + bbox[2]) / 2,
+ (bbox[1] + bbox[3]) / 2]])
+ scale = max(bbox[2] - bbox[0], bbox[3] - bbox[1])
+ inputs['keypoints'] = (res['keypoints'][:, :2] - center) \
+ / scale * bbox_scale + bbox_center
+ else:
+ inputs['keypoints'] = res['keypoints'][:, :2]
+
+ if res['keypoints'].shape[1] == 3:
+ inputs['keypoints'] = np.concatenate(
+ [inputs['keypoints'], res['keypoints'][:, 2:]], axis=1)
+
+ if 'track_id' in res:
+ inputs['track_id'] = res['track_id']
+ frame_inputs.append(inputs)
+ sequence_inputs.append(frame_inputs)
+ return sequence_inputs
+
+
+def _collate_pose_sequence(pose_results, with_track_id=True, target_frame=-1):
+ """Reorganize multi-frame pose detection results into individual pose
+ sequences.
+
+ Note:
+ - The temporal length of the pose detection results: T
+ - The number of the person instances: N
+ - The number of the keypoints: K
+ - The channel number of each keypoint: C
+
+ Args:
+ pose_results (List[List[Dict]]): Multi-frame pose detection results
+ stored in a nested list. Each element of the outer list is the
+ pose detection results of a single frame, and each element of the
+ inner list is the pose information of one person, which contains:
+
+ - keypoints (ndarray[K, 2 or 3]): x, y, [score]
+ - track_id (int): unique id of each person, required when
+ ``with_track_id==True```
+
+ with_track_id (bool): If True, the element in pose_results is expected
+ to contain "track_id", which will be used to gather the pose
+ sequence of a person from multiple frames. Otherwise, the pose
+ results in each frame are expected to have a consistent number and
+ order of identities. Default is True.
+ target_frame (int): The index of the target frame. Default: -1.
+ """
+ T = len(pose_results)
+ assert T > 0
+
+ target_frame = (T + target_frame) % T # convert negative index to positive
+
+ N = len(pose_results[target_frame]) # use identities in the target frame
+ if N == 0:
+ return []
+
+ K, C = pose_results[target_frame][0]['keypoints'].shape
+
+ track_ids = None
+ if with_track_id:
+ track_ids = [res['track_id'] for res in pose_results[target_frame]]
+
+ pose_sequences = []
+ for idx in range(N):
+ pose_seq = dict()
+ # gather static information
+ for k, v in pose_results[target_frame][idx].items():
+ if k != 'keypoints':
+ pose_seq[k] = v
+ # gather keypoints
+ if not with_track_id:
+ pose_seq['keypoints'] = np.stack(
+ [frame[idx]['keypoints'] for frame in pose_results])
+ else:
+ keypoints = np.zeros((T, K, C), dtype=np.float32)
+ keypoints[target_frame] = pose_results[target_frame][idx][
+ 'keypoints']
+ # find the left most frame containing track_ids[idx]
+ for frame_idx in range(target_frame - 1, -1, -1):
+ contains_idx = False
+ for res in pose_results[frame_idx]:
+ if res['track_id'] == track_ids[idx]:
+ keypoints[frame_idx] = res['keypoints']
+ contains_idx = True
+ break
+ if not contains_idx:
+ # replicate the left most frame
+ keypoints[:frame_idx + 1] = keypoints[frame_idx + 1]
+ break
+ # find the right most frame containing track_idx[idx]
+ for frame_idx in range(target_frame + 1, T):
+ contains_idx = False
+ for res in pose_results[frame_idx]:
+ if res['track_id'] == track_ids[idx]:
+ keypoints[frame_idx] = res['keypoints']
+ contains_idx = True
+ break
+ if not contains_idx:
+ # replicate the right most frame
+ keypoints[frame_idx + 1:] = keypoints[frame_idx]
+ break
+ pose_seq['keypoints'] = keypoints
+ pose_sequences.append(pose_seq)
+
+ return pose_sequences
+
+
+def inference_pose_lifter_model(model,
+ pose_results_2d,
+ dataset=None,
+ dataset_info=None,
+ with_track_id=True,
+ image_size=None,
+ norm_pose_2d=False):
+ """Inference 3D pose from 2D pose sequences using a pose lifter model.
+
+ Args:
+ model (nn.Module): The loaded pose lifter model
+ pose_results_2d (list[list[dict]]): The 2D pose sequences stored in a
+ nested list. Each element of the outer list is the 2D pose results
+ of a single frame, and each element of the inner list is the 2D
+ pose of one person, which contains:
+
+ - "keypoints" (ndarray[K, 2 or 3]): x, y, [score]
+ - "track_id" (int)
+ dataset (str): Dataset name, e.g. 'Body3DH36MDataset'
+ with_track_id: If True, the element in pose_results_2d is expected to
+ contain "track_id", which will be used to gather the pose sequence
+ of a person from multiple frames. Otherwise, the pose results in
+ each frame are expected to have a consistent number and order of
+ identities. Default is True.
+ image_size (tuple|list): image width, image height. If None, image size
+ will not be contained in dict ``data``.
+ norm_pose_2d (bool): If True, scale the bbox (along with the 2D
+ pose) to the average bbox scale of the dataset, and move the bbox
+ (along with the 2D pose) to the average bbox center of the dataset.
+
+ Returns:
+ list[dict]: 3D pose inference results. Each element is the result of \
+ an instance, which contains:
+
+ - "keypoints_3d" (ndarray[K, 3]): predicted 3D keypoints
+ - "keypoints" (ndarray[K, 2 or 3]): from the last frame in \
+ ``pose_results_2d``.
+ - "track_id" (int): from the last frame in ``pose_results_2d``. \
+ If there is no valid instance, an empty list will be \
+ returned.
+ """
+ cfg = model.cfg
+ test_pipeline = Compose(cfg.test_pipeline)
+
+ device = next(model.parameters()).device
+ if device.type == 'cpu':
+ device = -1
+
+ if dataset_info is not None:
+ flip_pairs = dataset_info.flip_pairs
+ assert 'stats_info' in dataset_info._dataset_info
+ bbox_center = dataset_info._dataset_info['stats_info']['bbox_center']
+ bbox_scale = dataset_info._dataset_info['stats_info']['bbox_scale']
+ else:
+ warnings.warn(
+ 'dataset is deprecated.'
+ 'Please set `dataset_info` in the config.'
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 for details.',
+ DeprecationWarning)
+ # TODO: These will be removed in the later versions.
+ if dataset == 'Body3DH36MDataset':
+ flip_pairs = [[1, 4], [2, 5], [3, 6], [11, 14], [12, 15], [13, 16]]
+ bbox_center = np.array([[528, 427]], dtype=np.float32)
+ bbox_scale = 400
+ else:
+ raise NotImplementedError()
+
+ target_idx = -1 if model.causal else len(pose_results_2d) // 2
+ pose_lifter_inputs = _gather_pose_lifter_inputs(pose_results_2d,
+ bbox_center, bbox_scale,
+ norm_pose_2d)
+ pose_sequences_2d = _collate_pose_sequence(pose_lifter_inputs,
+ with_track_id, target_idx)
+
+ if not pose_sequences_2d:
+ return []
+
+ batch_data = []
+ for seq in pose_sequences_2d:
+ pose_2d = seq['keypoints'].astype(np.float32)
+ T, K, C = pose_2d.shape
+
+ input_2d = pose_2d[..., :2]
+ input_2d_visible = pose_2d[..., 2:3]
+ if C > 2:
+ input_2d_visible = pose_2d[..., 2:3]
+ else:
+ input_2d_visible = np.ones((T, K, 1), dtype=np.float32)
+
+ # TODO: Will be removed in the later versions
+ # Dummy 3D input
+ # This is for compatibility with configs in mmpose<=v0.14.0, where a
+ # 3D input is required to generate denormalization parameters. This
+ # part will be removed in the future.
+ target = np.zeros((K, 3), dtype=np.float32)
+ target_visible = np.ones((K, 1), dtype=np.float32)
+
+ # Dummy image path
+ # This is for compatibility with configs in mmpose<=v0.14.0, where
+ # target_image_path is required. This part will be removed in the
+ # future.
+ target_image_path = None
+
+ data = {
+ 'input_2d': input_2d,
+ 'input_2d_visible': input_2d_visible,
+ 'target': target,
+ 'target_visible': target_visible,
+ 'target_image_path': target_image_path,
+ 'ann_info': {
+ 'num_joints': K,
+ 'flip_pairs': flip_pairs
+ }
+ }
+
+ if image_size is not None:
+ assert len(image_size) == 2
+ data['image_width'] = image_size[0]
+ data['image_height'] = image_size[1]
+
+ data = test_pipeline(data)
+ batch_data.append(data)
+
+ batch_data = collate(batch_data, samples_per_gpu=len(batch_data))
+ batch_data = scatter(batch_data, target_gpus=[device])[0]
+
+ with torch.no_grad():
+ result = model(
+ input=batch_data['input'],
+ metas=batch_data['metas'],
+ return_loss=False)
+
+ poses_3d = result['preds']
+ if poses_3d.shape[-1] != 4:
+ assert poses_3d.shape[-1] == 3
+ dummy_score = np.ones(
+ poses_3d.shape[:-1] + (1, ), dtype=poses_3d.dtype)
+ poses_3d = np.concatenate((poses_3d, dummy_score), axis=-1)
+ pose_results = []
+ for pose_2d, pose_3d in zip(pose_sequences_2d, poses_3d):
+ pose_result = pose_2d.copy()
+ pose_result['keypoints_3d'] = pose_3d
+ pose_results.append(pose_result)
+
+ return pose_results
+
+
+def vis_3d_pose_result(model,
+ result,
+ img=None,
+ dataset='Body3DH36MDataset',
+ dataset_info=None,
+ kpt_score_thr=0.3,
+ radius=8,
+ thickness=2,
+ num_instances=-1,
+ show=False,
+ out_file=None):
+ """Visualize the 3D pose estimation results.
+
+ Args:
+ model (nn.Module): The loaded model.
+ result (list[dict])
+ """
+
+ if dataset_info is not None:
+ skeleton = dataset_info.skeleton
+ pose_kpt_color = dataset_info.pose_kpt_color
+ pose_link_color = dataset_info.pose_link_color
+ else:
+ warnings.warn(
+ 'dataset is deprecated.'
+ 'Please set `dataset_info` in the config.'
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 for details.',
+ DeprecationWarning)
+ # TODO: These will be removed in the later versions.
+ palette = np.array([[255, 128, 0], [255, 153, 51], [255, 178, 102],
+ [230, 230, 0], [255, 153, 255], [153, 204, 255],
+ [255, 102, 255], [255, 51, 255], [102, 178, 255],
+ [51, 153, 255], [255, 153, 153], [255, 102, 102],
+ [255, 51, 51], [153, 255, 153], [102, 255, 102],
+ [51, 255, 51], [0, 255, 0], [0, 0, 255],
+ [255, 0, 0], [255, 255, 255]])
+
+ if dataset == 'Body3DH36MDataset':
+ skeleton = [[0, 1], [1, 2], [2, 3], [0, 4], [4, 5], [5, 6], [0, 7],
+ [7, 8], [8, 9], [9, 10], [8, 11], [11, 12], [12, 13],
+ [8, 14], [14, 15], [15, 16]]
+
+ pose_kpt_color = palette[[
+ 9, 0, 0, 0, 16, 16, 16, 9, 9, 9, 9, 16, 16, 16, 0, 0, 0
+ ]]
+ pose_link_color = palette[[
+ 0, 0, 0, 16, 16, 16, 9, 9, 9, 9, 16, 16, 16, 0, 0, 0
+ ]]
+
+ elif dataset == 'InterHand3DDataset':
+ skeleton = [[0, 1], [1, 2], [2, 3], [3, 20], [4, 5], [5, 6],
+ [6, 7], [7, 20], [8, 9], [9, 10], [10, 11], [11, 20],
+ [12, 13], [13, 14], [14, 15], [15, 20], [16, 17],
+ [17, 18], [18, 19], [19, 20], [21, 22], [22, 23],
+ [23, 24], [24, 41], [25, 26], [26, 27], [27, 28],
+ [28, 41], [29, 30], [30, 31], [31, 32], [32, 41],
+ [33, 34], [34, 35], [35, 36], [36, 41], [37, 38],
+ [38, 39], [39, 40], [40, 41]]
+
+ pose_kpt_color = [[14, 128, 250], [14, 128, 250], [14, 128, 250],
+ [14, 128, 250], [80, 127, 255], [80, 127, 255],
+ [80, 127, 255], [80, 127, 255], [71, 99, 255],
+ [71, 99, 255], [71, 99, 255], [71, 99, 255],
+ [0, 36, 255], [0, 36, 255], [0, 36, 255],
+ [0, 36, 255], [0, 0, 230], [0, 0, 230],
+ [0, 0, 230], [0, 0, 230], [0, 0, 139],
+ [237, 149, 100], [237, 149, 100],
+ [237, 149, 100], [237, 149, 100], [230, 128, 77],
+ [230, 128, 77], [230, 128, 77], [230, 128, 77],
+ [255, 144, 30], [255, 144, 30], [255, 144, 30],
+ [255, 144, 30], [153, 51, 0], [153, 51, 0],
+ [153, 51, 0], [153, 51, 0], [255, 51, 13],
+ [255, 51, 13], [255, 51, 13], [255, 51, 13],
+ [103, 37, 8]]
+
+ pose_link_color = [[14, 128, 250], [14, 128, 250], [14, 128, 250],
+ [14, 128, 250], [80, 127, 255], [80, 127, 255],
+ [80, 127, 255], [80, 127, 255], [71, 99, 255],
+ [71, 99, 255], [71, 99, 255], [71, 99, 255],
+ [0, 36, 255], [0, 36, 255], [0, 36, 255],
+ [0, 36, 255], [0, 0, 230], [0, 0, 230],
+ [0, 0, 230], [0, 0, 230], [237, 149, 100],
+ [237, 149, 100], [237, 149, 100],
+ [237, 149, 100], [230, 128, 77], [230, 128, 77],
+ [230, 128, 77], [230, 128, 77], [255, 144, 30],
+ [255, 144, 30], [255, 144, 30], [255, 144, 30],
+ [153, 51, 0], [153, 51, 0], [153, 51, 0],
+ [153, 51, 0], [255, 51, 13], [255, 51, 13],
+ [255, 51, 13], [255, 51, 13]]
+ else:
+ raise NotImplementedError
+
+ if hasattr(model, 'module'):
+ model = model.module
+
+ img = model.show_result(
+ result,
+ img,
+ skeleton,
+ radius=radius,
+ thickness=thickness,
+ pose_kpt_color=pose_kpt_color,
+ pose_link_color=pose_link_color,
+ num_instances=num_instances,
+ show=show,
+ out_file=out_file)
+
+ return img
+
+
+def inference_interhand_3d_model(model,
+ img_or_path,
+ det_results,
+ bbox_thr=None,
+ format='xywh',
+ dataset='InterHand3DDataset'):
+ """Inference a single image with a list of hand bounding boxes.
+
+ Note:
+ - num_bboxes: N
+ - num_keypoints: K
+
+ Args:
+ model (nn.Module): The loaded pose model.
+ img_or_path (str | np.ndarray): Image filename or loaded image.
+ det_results (list[dict]): The 2D bbox sequences stored in a list.
+ Each each element of the list is the bbox of one person, whose
+ shape is (ndarray[4 or 5]), containing 4 box coordinates
+ (and score).
+ dataset (str): Dataset name.
+ format: bbox format ('xyxy' | 'xywh'). Default: 'xywh'.
+ 'xyxy' means (left, top, right, bottom),
+ 'xywh' means (left, top, width, height).
+
+ Returns:
+ list[dict]: 3D pose inference results. Each element is the result \
+ of an instance, which contains the predicted 3D keypoints with \
+ shape (ndarray[K,3]). If there is no valid instance, an \
+ empty list will be returned.
+ """
+
+ assert format in ['xyxy', 'xywh']
+
+ pose_results = []
+
+ if len(det_results) == 0:
+ return pose_results
+
+ # Change for-loop preprocess each bbox to preprocess all bboxes at once.
+ bboxes = np.array([box['bbox'] for box in det_results])
+
+ # Select bboxes by score threshold
+ if bbox_thr is not None:
+ assert bboxes.shape[1] == 5
+ valid_idx = np.where(bboxes[:, 4] > bbox_thr)[0]
+ bboxes = bboxes[valid_idx]
+ det_results = [det_results[i] for i in valid_idx]
+
+ if format == 'xyxy':
+ bboxes_xyxy = bboxes
+ bboxes_xywh = _xyxy2xywh(bboxes)
+ else:
+ # format is already 'xywh'
+ bboxes_xywh = bboxes
+ bboxes_xyxy = _xywh2xyxy(bboxes)
+
+ # if bbox_thr remove all bounding box
+ if len(bboxes_xywh) == 0:
+ return []
+
+ cfg = model.cfg
+ device = next(model.parameters()).device
+ if device.type == 'cpu':
+ device = -1
+
+ # build the data pipeline
+ test_pipeline = Compose(cfg.test_pipeline)
+
+ assert len(bboxes[0]) in [4, 5]
+
+ if dataset == 'InterHand3DDataset':
+ flip_pairs = [[i, 21 + i] for i in range(21)]
+ else:
+ raise NotImplementedError()
+
+ batch_data = []
+ for bbox in bboxes:
+ center, scale = _box2cs(cfg, bbox)
+
+ # prepare data
+ data = {
+ 'center':
+ center,
+ 'scale':
+ scale,
+ 'bbox_score':
+ bbox[4] if len(bbox) == 5 else 1,
+ 'bbox_id':
+ 0, # need to be assigned if batch_size > 1
+ 'dataset':
+ dataset,
+ 'joints_3d':
+ np.zeros((cfg.data_cfg.num_joints, 3), dtype=np.float32),
+ 'joints_3d_visible':
+ np.zeros((cfg.data_cfg.num_joints, 3), dtype=np.float32),
+ 'rotation':
+ 0,
+ 'ann_info': {
+ 'image_size': np.array(cfg.data_cfg['image_size']),
+ 'num_joints': cfg.data_cfg['num_joints'],
+ 'flip_pairs': flip_pairs,
+ 'heatmap3d_depth_bound': cfg.data_cfg['heatmap3d_depth_bound'],
+ 'heatmap_size_root': cfg.data_cfg['heatmap_size_root'],
+ 'root_depth_bound': cfg.data_cfg['root_depth_bound']
+ }
+ }
+
+ if isinstance(img_or_path, np.ndarray):
+ data['img'] = img_or_path
+ else:
+ data['image_file'] = img_or_path
+
+ data = test_pipeline(data)
+ batch_data.append(data)
+
+ batch_data = collate(batch_data, samples_per_gpu=len(batch_data))
+ batch_data = scatter(batch_data, [device])[0]
+
+ # forward the model
+ with torch.no_grad():
+ result = model(
+ img=batch_data['img'],
+ img_metas=batch_data['img_metas'],
+ return_loss=False)
+
+ poses_3d = result['preds']
+ rel_root_depth = result['rel_root_depth']
+ hand_type = result['hand_type']
+ if poses_3d.shape[-1] != 4:
+ assert poses_3d.shape[-1] == 3
+ dummy_score = np.ones(
+ poses_3d.shape[:-1] + (1, ), dtype=poses_3d.dtype)
+ poses_3d = np.concatenate((poses_3d, dummy_score), axis=-1)
+
+ # add relative root depth to left hand joints
+ poses_3d[:, 21:, 2] += rel_root_depth
+
+ # set joint scores according to hand type
+ poses_3d[:, :21, 3] *= hand_type[:, [0]]
+ poses_3d[:, 21:, 3] *= hand_type[:, [1]]
+
+ pose_results = []
+ for pose_3d, person_res, bbox_xyxy in zip(poses_3d, det_results,
+ bboxes_xyxy):
+ pose_res = person_res.copy()
+ pose_res['keypoints_3d'] = pose_3d
+ pose_res['bbox'] = bbox_xyxy
+ pose_results.append(pose_res)
+
+ return pose_results
+
+
+def inference_mesh_model(model,
+ img_or_path,
+ det_results,
+ bbox_thr=None,
+ format='xywh',
+ dataset='MeshH36MDataset'):
+ """Inference a single image with a list of bounding boxes.
+
+ Note:
+ - num_bboxes: N
+ - num_keypoints: K
+ - num_vertices: V
+ - num_faces: F
+
+ Args:
+ model (nn.Module): The loaded pose model.
+ img_or_path (str | np.ndarray): Image filename or loaded image.
+ det_results (list[dict]): The 2D bbox sequences stored in a list.
+ Each element of the list is the bbox of one person.
+ "bbox" (ndarray[4 or 5]): The person bounding box,
+ which contains 4 box coordinates (and score).
+ bbox_thr (float | None): Threshold for bounding boxes.
+ Only bboxes with higher scores will be fed into the pose
+ detector. If bbox_thr is None, all boxes will be used.
+ format (str): bbox format ('xyxy' | 'xywh'). Default: 'xywh'.
+
+ - 'xyxy' means (left, top, right, bottom),
+ - 'xywh' means (left, top, width, height).
+ dataset (str): Dataset name.
+
+ Returns:
+ list[dict]: 3D pose inference results. Each element \
+ is the result of an instance, which contains:
+
+ - 'bbox' (ndarray[4]): instance bounding bbox
+ - 'center' (ndarray[2]): bbox center
+ - 'scale' (ndarray[2]): bbox scale
+ - 'keypoints_3d' (ndarray[K,3]): predicted 3D keypoints
+ - 'camera' (ndarray[3]): camera parameters
+ - 'vertices' (ndarray[V, 3]): predicted 3D vertices
+ - 'faces' (ndarray[F, 3]): mesh faces
+
+ If there is no valid instance, an empty list
+ will be returned.
+ """
+
+ assert format in ['xyxy', 'xywh']
+
+ pose_results = []
+
+ if len(det_results) == 0:
+ return pose_results
+
+ # Change for-loop preprocess each bbox to preprocess all bboxes at once.
+ bboxes = np.array([box['bbox'] for box in det_results])
+
+ # Select bboxes by score threshold
+ if bbox_thr is not None:
+ assert bboxes.shape[1] == 5
+ valid_idx = np.where(bboxes[:, 4] > bbox_thr)[0]
+ bboxes = bboxes[valid_idx]
+ det_results = [det_results[i] for i in valid_idx]
+
+ if format == 'xyxy':
+ bboxes_xyxy = bboxes
+ bboxes_xywh = _xyxy2xywh(bboxes)
+ else:
+ # format is already 'xywh'
+ bboxes_xywh = bboxes
+ bboxes_xyxy = _xywh2xyxy(bboxes)
+
+ # if bbox_thr remove all bounding box
+ if len(bboxes_xywh) == 0:
+ return []
+
+ cfg = model.cfg
+ device = next(model.parameters()).device
+ if device.type == 'cpu':
+ device = -1
+
+ # build the data pipeline
+ test_pipeline = Compose(cfg.test_pipeline)
+
+ assert len(bboxes[0]) in [4, 5]
+
+ if dataset == 'MeshH36MDataset':
+ flip_pairs = [[0, 5], [1, 4], [2, 3], [6, 11], [7, 10], [8, 9],
+ [20, 21], [22, 23]]
+ else:
+ raise NotImplementedError()
+
+ batch_data = []
+ for bbox in bboxes:
+ center, scale = _box2cs(cfg, bbox)
+
+ # prepare data
+ data = {
+ 'image_file':
+ img_or_path,
+ 'center':
+ center,
+ 'scale':
+ scale,
+ 'rotation':
+ 0,
+ 'bbox_score':
+ bbox[4] if len(bbox) == 5 else 1,
+ 'dataset':
+ dataset,
+ 'joints_2d':
+ np.zeros((cfg.data_cfg.num_joints, 2), dtype=np.float32),
+ 'joints_2d_visible':
+ np.zeros((cfg.data_cfg.num_joints, 1), dtype=np.float32),
+ 'joints_3d':
+ np.zeros((cfg.data_cfg.num_joints, 3), dtype=np.float32),
+ 'joints_3d_visible':
+ np.zeros((cfg.data_cfg.num_joints, 3), dtype=np.float32),
+ 'pose':
+ np.zeros(72, dtype=np.float32),
+ 'beta':
+ np.zeros(10, dtype=np.float32),
+ 'has_smpl':
+ 0,
+ 'ann_info': {
+ 'image_size': np.array(cfg.data_cfg['image_size']),
+ 'num_joints': cfg.data_cfg['num_joints'],
+ 'flip_pairs': flip_pairs,
+ }
+ }
+
+ data = test_pipeline(data)
+ batch_data.append(data)
+
+ batch_data = collate(batch_data, samples_per_gpu=len(batch_data))
+ batch_data = scatter(batch_data, target_gpus=[device])[0]
+
+ # forward the model
+ with torch.no_grad():
+ preds = model(
+ img=batch_data['img'],
+ img_metas=batch_data['img_metas'],
+ return_loss=False,
+ return_vertices=True,
+ return_faces=True)
+
+ for idx in range(len(det_results)):
+ pose_res = det_results[idx].copy()
+ pose_res['bbox'] = bboxes_xyxy[idx]
+ pose_res['center'] = batch_data['img_metas'][idx]['center']
+ pose_res['scale'] = batch_data['img_metas'][idx]['scale']
+ pose_res['keypoints_3d'] = preds['keypoints_3d'][idx]
+ pose_res['camera'] = preds['camera'][idx]
+ pose_res['vertices'] = preds['vertices'][idx]
+ pose_res['faces'] = preds['faces']
+ pose_results.append(pose_res)
+ return pose_results
+
+
+def vis_3d_mesh_result(model, result, img=None, show=False, out_file=None):
+ """Visualize the 3D mesh estimation results.
+
+ Args:
+ model (nn.Module): The loaded model.
+ result (list[dict]): 3D mesh estimation results.
+ """
+ if hasattr(model, 'module'):
+ model = model.module
+
+ img = model.show_result(result, img, show=show, out_file=out_file)
+
+ return img
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/apis/inference_tracking.py b/engine/pose_estimation/third-party/ViTPose/mmpose/apis/inference_tracking.py
new file mode 100644
index 0000000..d85a5c6
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/apis/inference_tracking.py
@@ -0,0 +1,337 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import warnings
+
+import numpy as np
+
+from mmpose.core import OneEuroFilter, oks_iou
+
+
+def _compute_iou(bboxA, bboxB):
+ """Compute the Intersection over Union (IoU) between two boxes .
+
+ Args:
+ bboxA (list): The first bbox info (left, top, right, bottom, score).
+ bboxB (list): The second bbox info (left, top, right, bottom, score).
+
+ Returns:
+ float: The IoU value.
+ """
+
+ x1 = max(bboxA[0], bboxB[0])
+ y1 = max(bboxA[1], bboxB[1])
+ x2 = min(bboxA[2], bboxB[2])
+ y2 = min(bboxA[3], bboxB[3])
+
+ inter_area = max(0, x2 - x1) * max(0, y2 - y1)
+
+ bboxA_area = (bboxA[2] - bboxA[0]) * (bboxA[3] - bboxA[1])
+ bboxB_area = (bboxB[2] - bboxB[0]) * (bboxB[3] - bboxB[1])
+ union_area = float(bboxA_area + bboxB_area - inter_area)
+ if union_area == 0:
+ union_area = 1e-5
+ warnings.warn('union_area=0 is unexpected')
+
+ iou = inter_area / union_area
+
+ return iou
+
+
+def _track_by_iou(res, results_last, thr):
+ """Get track id using IoU tracking greedily.
+
+ Args:
+ res (dict): The bbox & pose results of the person instance.
+ results_last (list[dict]): The bbox & pose & track_id info of the
+ last frame (bbox_result, pose_result, track_id).
+ thr (float): The threshold for iou tracking.
+
+ Returns:
+ int: The track id for the new person instance.
+ list[dict]: The bbox & pose & track_id info of the persons
+ that have not been matched on the last frame.
+ dict: The matched person instance on the last frame.
+ """
+
+ bbox = list(res['bbox'])
+
+ max_iou_score = -1
+ max_index = -1
+ match_result = {}
+ for index, res_last in enumerate(results_last):
+ bbox_last = list(res_last['bbox'])
+
+ iou_score = _compute_iou(bbox, bbox_last)
+ if iou_score > max_iou_score:
+ max_iou_score = iou_score
+ max_index = index
+
+ if max_iou_score > thr:
+ track_id = results_last[max_index]['track_id']
+ match_result = results_last[max_index]
+ del results_last[max_index]
+ else:
+ track_id = -1
+
+ return track_id, results_last, match_result
+
+
+def _track_by_oks(res, results_last, thr):
+ """Get track id using OKS tracking greedily.
+
+ Args:
+ res (dict): The pose results of the person instance.
+ results_last (list[dict]): The pose & track_id info of the
+ last frame (pose_result, track_id).
+ thr (float): The threshold for oks tracking.
+
+ Returns:
+ int: The track id for the new person instance.
+ list[dict]: The pose & track_id info of the persons
+ that have not been matched on the last frame.
+ dict: The matched person instance on the last frame.
+ """
+ pose = res['keypoints'].reshape((-1))
+ area = res['area']
+ max_index = -1
+ match_result = {}
+
+ if len(results_last) == 0:
+ return -1, results_last, match_result
+
+ pose_last = np.array(
+ [res_last['keypoints'].reshape((-1)) for res_last in results_last])
+ area_last = np.array([res_last['area'] for res_last in results_last])
+
+ oks_score = oks_iou(pose, pose_last, area, area_last)
+
+ max_index = np.argmax(oks_score)
+
+ if oks_score[max_index] > thr:
+ track_id = results_last[max_index]['track_id']
+ match_result = results_last[max_index]
+ del results_last[max_index]
+ else:
+ track_id = -1
+
+ return track_id, results_last, match_result
+
+
+def _get_area(results):
+ """Get bbox for each person instance on the current frame.
+
+ Args:
+ results (list[dict]): The pose results of the current frame
+ (pose_result).
+ Returns:
+ list[dict]: The bbox & pose info of the current frame
+ (bbox_result, pose_result, area).
+ """
+ for result in results:
+ if 'bbox' in result:
+ result['area'] = ((result['bbox'][2] - result['bbox'][0]) *
+ (result['bbox'][3] - result['bbox'][1]))
+ else:
+ xmin = np.min(
+ result['keypoints'][:, 0][result['keypoints'][:, 0] > 0],
+ initial=1e10)
+ xmax = np.max(result['keypoints'][:, 0])
+ ymin = np.min(
+ result['keypoints'][:, 1][result['keypoints'][:, 1] > 0],
+ initial=1e10)
+ ymax = np.max(result['keypoints'][:, 1])
+ result['area'] = (xmax - xmin) * (ymax - ymin)
+ result['bbox'] = np.array([xmin, ymin, xmax, ymax])
+ return results
+
+
+def _temporal_refine(result, match_result, fps=None):
+ """Refine koypoints using tracked person instance on last frame.
+
+ Args:
+ results (dict): The pose results of the current frame
+ (pose_result).
+ match_result (dict): The pose results of the last frame
+ (match_result)
+ Returns:
+ (array): The person keypoints after refine.
+ """
+ if 'one_euro' in match_result:
+ result['keypoints'][:, :2] = match_result['one_euro'](
+ result['keypoints'][:, :2])
+ result['one_euro'] = match_result['one_euro']
+ else:
+ result['one_euro'] = OneEuroFilter(result['keypoints'][:, :2], fps=fps)
+ return result['keypoints']
+
+
+def get_track_id(results,
+ results_last,
+ next_id,
+ min_keypoints=3,
+ use_oks=False,
+ tracking_thr=0.3,
+ use_one_euro=False,
+ fps=None):
+ """Get track id for each person instance on the current frame.
+
+ Args:
+ results (list[dict]): The bbox & pose results of the current frame
+ (bbox_result, pose_result).
+ results_last (list[dict]): The bbox & pose & track_id info of the
+ last frame (bbox_result, pose_result, track_id).
+ next_id (int): The track id for the new person instance.
+ min_keypoints (int): Minimum number of keypoints recognized as person.
+ default: 3.
+ use_oks (bool): Flag to using oks tracking. default: False.
+ tracking_thr (float): The threshold for tracking.
+ use_one_euro (bool): Option to use one-euro-filter. default: False.
+ fps (optional): Parameters that d_cutoff
+ when one-euro-filter is used as a video input
+
+ Returns:
+ tuple:
+ - results (list[dict]): The bbox & pose & track_id info of the \
+ current frame (bbox_result, pose_result, track_id).
+ - next_id (int): The track id for the new person instance.
+ """
+ results = _get_area(results)
+
+ if use_oks:
+ _track = _track_by_oks
+ else:
+ _track = _track_by_iou
+
+ for result in results:
+ track_id, results_last, match_result = _track(result, results_last,
+ tracking_thr)
+ if track_id == -1:
+ result['track_id'] = next_id
+ next_id += 1
+ else:
+ result['track_id'] = track_id
+ del match_result
+
+ return results, next_id
+
+
+def vis_pose_tracking_result(model,
+ img,
+ result,
+ radius=4,
+ thickness=1,
+ kpt_score_thr=0.3,
+ dataset='TopDownCocoDataset',
+ dataset_info=None,
+ show=False,
+ out_file=None):
+ """Visualize the pose tracking results on the image.
+
+ Args:
+ model (nn.Module): The loaded detector.
+ img (str | np.ndarray): Image filename or loaded image.
+ result (list[dict]): The results to draw over `img`
+ (bbox_result, pose_result).
+ radius (int): Radius of circles.
+ thickness (int): Thickness of lines.
+ kpt_score_thr (float): The threshold to visualize the keypoints.
+ skeleton (list[tuple]): Default None.
+ show (bool): Whether to show the image. Default True.
+ out_file (str|None): The filename of the output visualization image.
+ """
+ if hasattr(model, 'module'):
+ model = model.module
+
+ palette = np.array([[255, 128, 0], [255, 153, 51], [255, 178, 102],
+ [230, 230, 0], [255, 153, 255], [153, 204, 255],
+ [255, 102, 255], [255, 51, 255], [102, 178, 255],
+ [51, 153, 255], [255, 153, 153], [255, 102, 102],
+ [255, 51, 51], [153, 255, 153], [102, 255, 102],
+ [51, 255, 51], [0, 255, 0], [0, 0, 255], [255, 0, 0],
+ [255, 255, 255]])
+
+ if dataset_info is None and dataset is not None:
+ warnings.warn(
+ 'dataset is deprecated.'
+ 'Please set `dataset_info` in the config.'
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 for details.',
+ DeprecationWarning)
+ # TODO: These will be removed in the later versions.
+ if dataset in ('TopDownCocoDataset', 'BottomUpCocoDataset',
+ 'TopDownOCHumanDataset'):
+ kpt_num = 17
+ skeleton = [[15, 13], [13, 11], [16, 14], [14, 12], [11, 12],
+ [5, 11], [6, 12], [5, 6], [5, 7], [6, 8], [7, 9],
+ [8, 10], [1, 2], [0, 1], [0, 2], [1, 3], [2, 4],
+ [3, 5], [4, 6]]
+
+ elif dataset == 'TopDownCocoWholeBodyDataset':
+ kpt_num = 133
+ skeleton = [[15, 13], [13, 11], [16, 14], [14, 12], [11, 12],
+ [5, 11], [6, 12], [5, 6], [5, 7], [6, 8], [7, 9],
+ [8, 10], [1, 2], [0, 1], [0, 2],
+ [1, 3], [2, 4], [3, 5], [4, 6], [15, 17], [15, 18],
+ [15, 19], [16, 20], [16, 21], [16, 22], [91, 92],
+ [92, 93], [93, 94], [94, 95], [91, 96], [96, 97],
+ [97, 98], [98, 99], [91, 100], [100, 101], [101, 102],
+ [102, 103], [91, 104], [104, 105], [105, 106],
+ [106, 107], [91, 108], [108, 109], [109, 110],
+ [110, 111], [112, 113], [113, 114], [114, 115],
+ [115, 116], [112, 117], [117, 118], [118, 119],
+ [119, 120], [112, 121], [121, 122], [122, 123],
+ [123, 124], [112, 125], [125, 126], [126, 127],
+ [127, 128], [112, 129], [129, 130], [130, 131],
+ [131, 132]]
+ radius = 1
+
+ elif dataset == 'TopDownAicDataset':
+ kpt_num = 14
+ skeleton = [[2, 1], [1, 0], [0, 13], [13, 3], [3, 4], [4, 5],
+ [8, 7], [7, 6], [6, 9], [9, 10], [10, 11], [12, 13],
+ [0, 6], [3, 9]]
+
+ elif dataset == 'TopDownMpiiDataset':
+ kpt_num = 16
+ skeleton = [[0, 1], [1, 2], [2, 6], [6, 3], [3, 4], [4, 5], [6, 7],
+ [7, 8], [8, 9], [8, 12], [12, 11], [11, 10], [8, 13],
+ [13, 14], [14, 15]]
+
+ elif dataset in ('OneHand10KDataset', 'FreiHandDataset',
+ 'PanopticDataset'):
+ kpt_num = 21
+ skeleton = [[0, 1], [1, 2], [2, 3], [3, 4], [0, 5], [5, 6], [6, 7],
+ [7, 8], [0, 9], [9, 10], [10, 11], [11, 12], [0, 13],
+ [13, 14], [14, 15], [15, 16], [0, 17], [17, 18],
+ [18, 19], [19, 20]]
+
+ elif dataset == 'InterHand2DDataset':
+ kpt_num = 21
+ skeleton = [[0, 1], [1, 2], [2, 3], [4, 5], [5, 6], [6, 7], [8, 9],
+ [9, 10], [10, 11], [12, 13], [13, 14], [14, 15],
+ [16, 17], [17, 18], [18, 19], [3, 20], [7, 20],
+ [11, 20], [15, 20], [19, 20]]
+
+ else:
+ raise NotImplementedError()
+
+ elif dataset_info is not None:
+ kpt_num = dataset_info.keypoint_num
+ skeleton = dataset_info.skeleton
+
+ for res in result:
+ track_id = res['track_id']
+ bbox_color = palette[track_id % len(palette)]
+ pose_kpt_color = palette[[track_id % len(palette)] * kpt_num]
+ pose_link_color = palette[[track_id % len(palette)] * len(skeleton)]
+ img = model.show_result(
+ img, [res],
+ skeleton,
+ radius=radius,
+ thickness=thickness,
+ pose_kpt_color=pose_kpt_color,
+ pose_link_color=pose_link_color,
+ bbox_color=tuple(bbox_color.tolist()),
+ kpt_score_thr=kpt_score_thr,
+ show=show,
+ out_file=out_file)
+
+ return img
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/apis/test.py b/engine/pose_estimation/third-party/ViTPose/mmpose/apis/test.py
new file mode 100644
index 0000000..3843b5a
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/apis/test.py
@@ -0,0 +1,191 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os.path as osp
+import pickle
+import shutil
+import tempfile
+
+import mmcv
+import torch
+import torch.distributed as dist
+from mmcv.runner import get_dist_info
+
+
+def single_gpu_test(model, data_loader):
+ """Test model with a single gpu.
+
+ This method tests model with a single gpu and displays test progress bar.
+
+ Args:
+ model (nn.Module): Model to be tested.
+ data_loader (nn.Dataloader): Pytorch data loader.
+
+
+ Returns:
+ list: The prediction results.
+ """
+
+ model.eval()
+ results = []
+ dataset = data_loader.dataset
+ prog_bar = mmcv.ProgressBar(len(dataset))
+ for data in data_loader:
+ with torch.no_grad():
+ result = model(return_loss=False, **data)
+ results.append(result)
+
+ # use the first key as main key to calculate the batch size
+ batch_size = len(next(iter(data.values())))
+ for _ in range(batch_size):
+ prog_bar.update()
+ return results
+
+
+def multi_gpu_test(model, data_loader, tmpdir=None, gpu_collect=False):
+ """Test model with multiple gpus.
+
+ This method tests model with multiple gpus and collects the results
+ under two different modes: gpu and cpu modes. By setting 'gpu_collect=True'
+ it encodes results to gpu tensors and use gpu communication for results
+ collection. On cpu mode it saves the results on different gpus to 'tmpdir'
+ and collects them by the rank 0 worker.
+
+ Args:
+ model (nn.Module): Model to be tested.
+ data_loader (nn.Dataloader): Pytorch data loader.
+ tmpdir (str): Path of directory to save the temporary results from
+ different gpus under cpu mode.
+ gpu_collect (bool): Option to use either gpu or cpu to collect results.
+
+ Returns:
+ list: The prediction results.
+ """
+ model.eval()
+ results = []
+ dataset = data_loader.dataset
+ rank, world_size = get_dist_info()
+ if rank == 0:
+ prog_bar = mmcv.ProgressBar(len(dataset))
+ for data in data_loader:
+ with torch.no_grad():
+ result = model(return_loss=False, **data)
+ results.append(result)
+
+ if rank == 0:
+ # use the first key as main key to calculate the batch size
+ batch_size = len(next(iter(data.values())))
+ for _ in range(batch_size * world_size):
+ prog_bar.update()
+
+ # collect results from all ranks
+ if gpu_collect:
+ results = collect_results_gpu(results, len(dataset))
+ else:
+ results = collect_results_cpu(results, len(dataset), tmpdir)
+ return results
+
+
+def collect_results_cpu(result_part, size, tmpdir=None):
+ """Collect results in cpu mode.
+
+ It saves the results on different gpus to 'tmpdir' and collects
+ them by the rank 0 worker.
+
+ Args:
+ result_part (list): Results to be collected
+ size (int): Result size.
+ tmpdir (str): Path of directory to save the temporary results from
+ different gpus under cpu mode. Default: None
+
+ Returns:
+ list: Ordered results.
+ """
+ rank, world_size = get_dist_info()
+ # create a tmp dir if it is not specified
+ if tmpdir is None:
+ MAX_LEN = 512
+ # 32 is whitespace
+ dir_tensor = torch.full((MAX_LEN, ),
+ 32,
+ dtype=torch.uint8,
+ device='cuda')
+ if rank == 0:
+ mmcv.mkdir_or_exist('.dist_test')
+ tmpdir = tempfile.mkdtemp(dir='.dist_test')
+ tmpdir = torch.tensor(
+ bytearray(tmpdir.encode()), dtype=torch.uint8, device='cuda')
+ dir_tensor[:len(tmpdir)] = tmpdir
+ dist.broadcast(dir_tensor, 0)
+ tmpdir = dir_tensor.cpu().numpy().tobytes().decode().rstrip()
+ else:
+ mmcv.mkdir_or_exist(tmpdir)
+ # synchronizes all processes to make sure tmpdir exist
+ dist.barrier()
+ # dump the part result to the dir
+ mmcv.dump(result_part, osp.join(tmpdir, f'part_{rank}.pkl'))
+ # synchronizes all processes for loading pickle file
+ dist.barrier()
+ # collect all parts
+ if rank != 0:
+ return None
+
+ # load results of all parts from tmp dir
+ part_list = []
+ for i in range(world_size):
+ part_file = osp.join(tmpdir, f'part_{i}.pkl')
+ part_list.append(mmcv.load(part_file))
+ # sort the results
+ ordered_results = []
+ for res in zip(*part_list):
+ ordered_results.extend(list(res))
+ # the dataloader may pad some samples
+ ordered_results = ordered_results[:size]
+ # remove tmp dir
+ shutil.rmtree(tmpdir)
+ return ordered_results
+
+
+def collect_results_gpu(result_part, size):
+ """Collect results in gpu mode.
+
+ It encodes results to gpu tensors and use gpu communication for results
+ collection.
+
+ Args:
+ result_part (list): Results to be collected
+ size (int): Result size.
+
+ Returns:
+ list: Ordered results.
+ """
+
+ rank, world_size = get_dist_info()
+ # dump result part to tensor with pickle
+ part_tensor = torch.tensor(
+ bytearray(pickle.dumps(result_part)), dtype=torch.uint8, device='cuda')
+ # gather all result part tensor shape
+ shape_tensor = torch.tensor(part_tensor.shape, device='cuda')
+ shape_list = [shape_tensor.clone() for _ in range(world_size)]
+ dist.all_gather(shape_list, shape_tensor)
+ # padding result part tensor to max length
+ shape_max = torch.tensor(shape_list).max()
+ part_send = torch.zeros(shape_max, dtype=torch.uint8, device='cuda')
+ part_send[:shape_tensor[0]] = part_tensor
+ part_recv_list = [
+ part_tensor.new_zeros(shape_max) for _ in range(world_size)
+ ]
+ # gather all result part
+ dist.all_gather(part_recv_list, part_send)
+
+ if rank == 0:
+ part_list = []
+ for recv, shape in zip(part_recv_list, shape_list):
+ part_list.append(
+ pickle.loads(recv[:shape[0]].cpu().numpy().tobytes()))
+ # sort the results
+ ordered_results = []
+ for res in zip(*part_list):
+ ordered_results.extend(list(res))
+ # the dataloader may pad some samples
+ ordered_results = ordered_results[:size]
+ return ordered_results
+ return None
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/apis/train.py b/engine/pose_estimation/third-party/ViTPose/mmpose/apis/train.py
new file mode 100644
index 0000000..7c31f8b
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/apis/train.py
@@ -0,0 +1,200 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import warnings
+
+import mmcv
+import numpy as np
+import torch
+import torch.distributed as dist
+from mmcv.parallel import MMDataParallel, MMDistributedDataParallel
+from mmcv.runner import (DistSamplerSeedHook, EpochBasedRunner, OptimizerHook,
+ get_dist_info)
+from mmcv.utils import digit_version
+
+from mmpose.core import DistEvalHook, EvalHook, build_optimizers
+from mmpose.core.distributed_wrapper import DistributedDataParallelWrapper
+from mmpose.datasets import build_dataloader, build_dataset
+from mmpose.utils import get_root_logger
+
+try:
+ from mmcv.runner import Fp16OptimizerHook
+except ImportError:
+ warnings.warn(
+ 'Fp16OptimizerHook from mmpose will be deprecated from '
+ 'v0.15.0. Please install mmcv>=1.1.4', DeprecationWarning)
+ from mmpose.core import Fp16OptimizerHook
+
+
+def init_random_seed(seed=None, device='cuda'):
+ """Initialize random seed.
+
+ If the seed is not set, the seed will be automatically randomized,
+ and then broadcast to all processes to prevent some potential bugs.
+
+ Args:
+ seed (int, Optional): The seed. Default to None.
+ device (str): The device where the seed will be put on.
+ Default to 'cuda'.
+
+ Returns:
+ int: Seed to be used.
+ """
+ if seed is not None:
+ return seed
+
+ # Make sure all ranks share the same random seed to prevent
+ # some potential bugs. Please refer to
+ # https://github.com/open-mmlab/mmdetection/issues/6339
+ rank, world_size = get_dist_info()
+ seed = np.random.randint(2**31)
+ if world_size == 1:
+ return seed
+
+ if rank == 0:
+ random_num = torch.tensor(seed, dtype=torch.int32, device=device)
+ else:
+ random_num = torch.tensor(0, dtype=torch.int32, device=device)
+ dist.broadcast(random_num, src=0)
+ return random_num.item()
+
+
+def train_model(model,
+ dataset,
+ cfg,
+ distributed=False,
+ validate=False,
+ timestamp=None,
+ meta=None):
+ """Train model entry function.
+
+ Args:
+ model (nn.Module): The model to be trained.
+ dataset (Dataset): Train dataset.
+ cfg (dict): The config dict for training.
+ distributed (bool): Whether to use distributed training.
+ Default: False.
+ validate (bool): Whether to do evaluation. Default: False.
+ timestamp (str | None): Local time for runner. Default: None.
+ meta (dict | None): Meta dict to record some important information.
+ Default: None
+ """
+ logger = get_root_logger(cfg.log_level)
+
+ # prepare data loaders
+ dataset = dataset if isinstance(dataset, (list, tuple)) else [dataset]
+ # step 1: give default values and override (if exist) from cfg.data
+ loader_cfg = {
+ **dict(
+ seed=cfg.get('seed'),
+ drop_last=False,
+ dist=distributed,
+ num_gpus=len(cfg.gpu_ids)),
+ **({} if torch.__version__ != 'parrots' else dict(
+ prefetch_num=2,
+ pin_memory=False,
+ )),
+ **dict((k, cfg.data[k]) for k in [
+ 'samples_per_gpu',
+ 'workers_per_gpu',
+ 'shuffle',
+ 'seed',
+ 'drop_last',
+ 'prefetch_num',
+ 'pin_memory',
+ 'persistent_workers',
+ ] if k in cfg.data)
+ }
+
+ # step 2: cfg.data.train_dataloader has highest priority
+ train_loader_cfg = dict(loader_cfg, **cfg.data.get('train_dataloader', {}))
+
+ data_loaders = [build_dataloader(ds, **train_loader_cfg) for ds in dataset]
+
+ # determine whether use adversarial training precess or not
+ use_adverserial_train = cfg.get('use_adversarial_train', False)
+
+ # put model on gpus
+ if distributed:
+ find_unused_parameters = cfg.get('find_unused_parameters', False)
+ # Sets the `find_unused_parameters` parameter in
+ # torch.nn.parallel.DistributedDataParallel
+
+ if use_adverserial_train:
+ # Use DistributedDataParallelWrapper for adversarial training
+ model = DistributedDataParallelWrapper(
+ model,
+ device_ids=[torch.cuda.current_device()],
+ broadcast_buffers=False,
+ find_unused_parameters=find_unused_parameters)
+ else:
+ model = MMDistributedDataParallel(
+ model.cuda(),
+ device_ids=[torch.cuda.current_device()],
+ broadcast_buffers=False,
+ find_unused_parameters=find_unused_parameters)
+ else:
+ if digit_version(mmcv.__version__) >= digit_version(
+ '1.4.4') or torch.cuda.is_available():
+ model = MMDataParallel(model, device_ids=cfg.gpu_ids)
+ else:
+ warnings.warn(
+ 'We recommend to use MMCV >= 1.4.4 for CPU training. '
+ 'See https://github.com/open-mmlab/mmpose/pull/1157 for '
+ 'details.')
+
+ # build runner
+ optimizer = build_optimizers(model, cfg.optimizer)
+
+ runner = EpochBasedRunner(
+ model,
+ optimizer=optimizer,
+ work_dir=cfg.work_dir,
+ logger=logger,
+ meta=meta)
+ # an ugly workaround to make .log and .log.json filenames the same
+ runner.timestamp = timestamp
+
+ if use_adverserial_train:
+ # The optimizer step process is included in the train_step function
+ # of the model, so the runner should NOT include optimizer hook.
+ optimizer_config = None
+ else:
+ # fp16 setting
+ fp16_cfg = cfg.get('fp16', None)
+ if fp16_cfg is not None:
+ optimizer_config = Fp16OptimizerHook(
+ **cfg.optimizer_config, **fp16_cfg, distributed=distributed)
+ elif distributed and 'type' not in cfg.optimizer_config:
+ optimizer_config = OptimizerHook(**cfg.optimizer_config)
+ else:
+ optimizer_config = cfg.optimizer_config
+
+ # register hooks
+ runner.register_training_hooks(cfg.lr_config, optimizer_config,
+ cfg.checkpoint_config, cfg.log_config,
+ cfg.get('momentum_config', None))
+ if distributed:
+ runner.register_hook(DistSamplerSeedHook())
+
+ # register eval hooks
+ if validate:
+ eval_cfg = cfg.get('evaluation', {})
+ val_dataset = build_dataset(cfg.data.val, dict(test_mode=True))
+ dataloader_setting = dict(
+ samples_per_gpu=1,
+ workers_per_gpu=cfg.data.get('workers_per_gpu', 1),
+ # cfg.gpus will be ignored if distributed
+ num_gpus=len(cfg.gpu_ids),
+ dist=distributed,
+ drop_last=False,
+ shuffle=False)
+ dataloader_setting = dict(dataloader_setting,
+ **cfg.data.get('val_dataloader', {}))
+ val_dataloader = build_dataloader(val_dataset, **dataloader_setting)
+ eval_hook = DistEvalHook if distributed else EvalHook
+ runner.register_hook(eval_hook(val_dataloader, **eval_cfg))
+
+ if cfg.resume_from:
+ runner.resume(cfg.resume_from)
+ elif cfg.load_from:
+ runner.load_checkpoint(cfg.load_from)
+ runner.run(data_loaders, cfg.workflow, cfg.total_epochs)
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/core/__init__.py b/engine/pose_estimation/third-party/ViTPose/mmpose/core/__init__.py
new file mode 100644
index 0000000..66185b7
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/core/__init__.py
@@ -0,0 +1,8 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from .camera import * # noqa: F401, F403
+from .evaluation import * # noqa: F401, F403
+from .fp16 import * # noqa: F401, F403
+from .optimizer import * # noqa: F401, F403
+from .post_processing import * # noqa: F401, F403
+from .utils import * # noqa: F401, F403
+from .visualization import * # noqa: F401, F403
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/core/camera/__init__.py b/engine/pose_estimation/third-party/ViTPose/mmpose/core/camera/__init__.py
new file mode 100644
index 0000000..a4a3c55
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/core/camera/__init__.py
@@ -0,0 +1,6 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from .camera_base import CAMERAS
+from .single_camera import SimpleCamera
+from .single_camera_torch import SimpleCameraTorch
+
+__all__ = ['CAMERAS', 'SimpleCamera', 'SimpleCameraTorch']
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/core/camera/camera_base.py b/engine/pose_estimation/third-party/ViTPose/mmpose/core/camera/camera_base.py
new file mode 100644
index 0000000..28b23e7
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/core/camera/camera_base.py
@@ -0,0 +1,45 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from abc import ABCMeta, abstractmethod
+
+from mmcv.utils import Registry
+
+CAMERAS = Registry('camera')
+
+
+class SingleCameraBase(metaclass=ABCMeta):
+ """Base class for single camera model.
+
+ Args:
+ param (dict): Camera parameters
+
+ Methods:
+ world_to_camera: Project points from world coordinates to camera
+ coordinates
+ camera_to_world: Project points from camera coordinates to world
+ coordinates
+ camera_to_pixel: Project points from camera coordinates to pixel
+ coordinates
+ world_to_pixel: Project points from world coordinates to pixel
+ coordinates
+ """
+
+ @abstractmethod
+ def __init__(self, param):
+ """Load camera parameters and check validity."""
+
+ def world_to_camera(self, X):
+ """Project points from world coordinates to camera coordinates."""
+ raise NotImplementedError
+
+ def camera_to_world(self, X):
+ """Project points from camera coordinates to world coordinates."""
+ raise NotImplementedError
+
+ def camera_to_pixel(self, X):
+ """Project points from camera coordinates to pixel coordinates."""
+ raise NotImplementedError
+
+ def world_to_pixel(self, X):
+ """Project points from world coordinates to pixel coordinates."""
+ _X = self.world_to_camera(X)
+ return self.camera_to_pixel(_X)
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/core/camera/single_camera.py b/engine/pose_estimation/third-party/ViTPose/mmpose/core/camera/single_camera.py
new file mode 100644
index 0000000..cabd799
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/core/camera/single_camera.py
@@ -0,0 +1,123 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import numpy as np
+
+from .camera_base import CAMERAS, SingleCameraBase
+
+
+@CAMERAS.register_module()
+class SimpleCamera(SingleCameraBase):
+ """Camera model to calculate coordinate transformation with given
+ intrinsic/extrinsic camera parameters.
+
+ Note:
+ The keypoint coordinate should be an np.ndarray with a shape of
+ [...,J, C] where J is the keypoint number of an instance, and C is
+ the coordinate dimension. For example:
+
+ [J, C]: shape of joint coordinates of a person with J joints.
+ [N, J, C]: shape of a batch of person joint coordinates.
+ [N, T, J, C]: shape of a batch of pose sequences.
+
+ Args:
+ param (dict): camera parameters including:
+ - R: 3x3, camera rotation matrix (camera-to-world)
+ - T: 3x1, camera translation (camera-to-world)
+ - K: (optional) 2x3, camera intrinsic matrix
+ - k: (optional) nx1, camera radial distortion coefficients
+ - p: (optional) mx1, camera tangential distortion coefficients
+ - f: (optional) 2x1, camera focal length
+ - c: (optional) 2x1, camera center
+ if K is not provided, it will be calculated from f and c.
+
+ Methods:
+ world_to_camera: Project points from world coordinates to camera
+ coordinates
+ camera_to_pixel: Project points from camera coordinates to pixel
+ coordinates
+ world_to_pixel: Project points from world coordinates to pixel
+ coordinates
+ """
+
+ def __init__(self, param):
+
+ self.param = {}
+ # extrinsic param
+ R = np.array(param['R'], dtype=np.float32)
+ T = np.array(param['T'], dtype=np.float32)
+ assert R.shape == (3, 3)
+ assert T.shape == (3, 1)
+ # The camera matrices are transposed in advance because the joint
+ # coordinates are stored as row vectors.
+ self.param['R_c2w'] = R.T
+ self.param['T_c2w'] = T.T
+ self.param['R_w2c'] = R
+ self.param['T_w2c'] = -self.param['T_c2w'] @ self.param['R_w2c']
+
+ # intrinsic param
+ if 'K' in param:
+ K = np.array(param['K'], dtype=np.float32)
+ assert K.shape == (2, 3)
+ self.param['K'] = K.T
+ self.param['f'] = np.array([K[0, 0], K[1, 1]])[:, np.newaxis]
+ self.param['c'] = np.array([K[0, 2], K[1, 2]])[:, np.newaxis]
+ elif 'f' in param and 'c' in param:
+ f = np.array(param['f'], dtype=np.float32)
+ c = np.array(param['c'], dtype=np.float32)
+ assert f.shape == (2, 1)
+ assert c.shape == (2, 1)
+ self.param['K'] = np.concatenate((np.diagflat(f), c), axis=-1).T
+ self.param['f'] = f
+ self.param['c'] = c
+ else:
+ raise ValueError('Camera intrinsic parameters are missing. '
+ 'Either "K" or "f"&"c" should be provided.')
+
+ # distortion param
+ if 'k' in param and 'p' in param:
+ self.undistortion = True
+ self.param['k'] = np.array(param['k'], dtype=np.float32).flatten()
+ self.param['p'] = np.array(param['p'], dtype=np.float32).flatten()
+ assert self.param['k'].size in {3, 6}
+ assert self.param['p'].size == 2
+ else:
+ self.undistortion = False
+
+ def world_to_camera(self, X):
+ assert isinstance(X, np.ndarray)
+ assert X.ndim >= 2 and X.shape[-1] == 3
+ return X @ self.param['R_w2c'] + self.param['T_w2c']
+
+ def camera_to_world(self, X):
+ assert isinstance(X, np.ndarray)
+ assert X.ndim >= 2 and X.shape[-1] == 3
+ return X @ self.param['R_c2w'] + self.param['T_c2w']
+
+ def camera_to_pixel(self, X):
+ assert isinstance(X, np.ndarray)
+ assert X.ndim >= 2 and X.shape[-1] == 3
+
+ _X = X / X[..., 2:]
+
+ if self.undistortion:
+ k = self.param['k']
+ p = self.param['p']
+ _X_2d = _X[..., :2]
+ r2 = (_X_2d**2).sum(-1)
+ radial = 1 + sum(ki * r2**(i + 1) for i, ki in enumerate(k[:3]))
+ if k.size == 6:
+ radial /= 1 + sum(
+ (ki * r2**(i + 1) for i, ki in enumerate(k[3:])))
+
+ tangential = 2 * (p[1] * _X[..., 0] + p[0] * _X[..., 1])
+
+ _X[..., :2] = _X_2d * (radial + tangential)[..., None] + np.outer(
+ r2, p[::-1]).reshape(_X_2d.shape)
+ return _X @ self.param['K']
+
+ def pixel_to_camera(self, X):
+ assert isinstance(X, np.ndarray)
+ assert X.ndim >= 2 and X.shape[-1] == 3
+ _X = X.copy()
+ _X[:, :2] = (X[:, :2] - self.param['c'].T) / self.param['f'].T * X[:,
+ [2]]
+ return _X
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/core/camera/single_camera_torch.py b/engine/pose_estimation/third-party/ViTPose/mmpose/core/camera/single_camera_torch.py
new file mode 100644
index 0000000..22eb72f
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/core/camera/single_camera_torch.py
@@ -0,0 +1,118 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import torch
+
+from .camera_base import CAMERAS, SingleCameraBase
+
+
+@CAMERAS.register_module()
+class SimpleCameraTorch(SingleCameraBase):
+ """Camera model to calculate coordinate transformation with given
+ intrinsic/extrinsic camera parameters.
+
+ Notes:
+ The keypoint coordinate should be an np.ndarray with a shape of
+ [...,J, C] where J is the keypoint number of an instance, and C is
+ the coordinate dimension. For example:
+
+ [J, C]: shape of joint coordinates of a person with J joints.
+ [N, J, C]: shape of a batch of person joint coordinates.
+ [N, T, J, C]: shape of a batch of pose sequences.
+
+ Args:
+ param (dict): camera parameters including:
+ - R: 3x3, camera rotation matrix (camera-to-world)
+ - T: 3x1, camera translation (camera-to-world)
+ - K: (optional) 2x3, camera intrinsic matrix
+ - k: (optional) nx1, camera radial distortion coefficients
+ - p: (optional) mx1, camera tangential distortion coefficients
+ - f: (optional) 2x1, camera focal length
+ - c: (optional) 2x1, camera center
+ if K is not provided, it will be calculated from f and c.
+
+ Methods:
+ world_to_camera: Project points from world coordinates to camera
+ coordinates
+ camera_to_pixel: Project points from camera coordinates to pixel
+ coordinates
+ world_to_pixel: Project points from world coordinates to pixel
+ coordinates
+ """
+
+ def __init__(self, param, device):
+
+ self.param = {}
+ # extrinsic param
+ R = torch.tensor(param['R'], device=device)
+ T = torch.tensor(param['T'], device=device)
+
+ assert R.shape == (3, 3)
+ assert T.shape == (3, 1)
+ # The camera matrices are transposed in advance because the joint
+ # coordinates are stored as row vectors.
+ self.param['R_c2w'] = R.T
+ self.param['T_c2w'] = T.T
+ self.param['R_w2c'] = R
+ self.param['T_w2c'] = -self.param['T_c2w'] @ self.param['R_w2c']
+
+ # intrinsic param
+ if 'K' in param:
+ K = torch.tensor(param['K'], device=device)
+ assert K.shape == (2, 3)
+ self.param['K'] = K.T
+ self.param['f'] = torch.tensor([[K[0, 0]], [K[1, 1]]],
+ device=device)
+ self.param['c'] = torch.tensor([[K[0, 2]], [K[1, 2]]],
+ device=device)
+ elif 'f' in param and 'c' in param:
+ f = torch.tensor(param['f'], device=device)
+ c = torch.tensor(param['c'], device=device)
+ assert f.shape == (2, 1)
+ assert c.shape == (2, 1)
+ self.param['K'] = torch.cat([torch.diagflat(f), c], dim=-1).T
+ self.param['f'] = f
+ self.param['c'] = c
+ else:
+ raise ValueError('Camera intrinsic parameters are missing. '
+ 'Either "K" or "f"&"c" should be provided.')
+
+ # distortion param
+ if 'k' in param and 'p' in param:
+ self.undistortion = True
+ self.param['k'] = torch.tensor(param['k'], device=device).view(-1)
+ self.param['p'] = torch.tensor(param['p'], device=device).view(-1)
+ assert len(self.param['k']) in {3, 6}
+ assert len(self.param['p']) == 2
+ else:
+ self.undistortion = False
+
+ def world_to_camera(self, X):
+ assert isinstance(X, torch.Tensor)
+ assert X.ndim >= 2 and X.shape[-1] == 3
+ return X @ self.param['R_w2c'] + self.param['T_w2c']
+
+ def camera_to_world(self, X):
+ assert isinstance(X, torch.Tensor)
+ assert X.ndim >= 2 and X.shape[-1] == 3
+ return X @ self.param['R_c2w'] + self.param['T_c2w']
+
+ def camera_to_pixel(self, X):
+ assert isinstance(X, torch.Tensor)
+ assert X.ndim >= 2 and X.shape[-1] == 3
+
+ _X = X / X[..., 2:]
+
+ if self.undistortion:
+ k = self.param['k']
+ p = self.param['p']
+ _X_2d = _X[..., :2]
+ r2 = (_X_2d**2).sum(-1)
+ radial = 1 + sum(ki * r2**(i + 1) for i, ki in enumerate(k[:3]))
+ if k.size == 6:
+ radial /= 1 + sum(
+ (ki * r2**(i + 1) for i, ki in enumerate(k[3:])))
+
+ tangential = 2 * (p[1] * _X[..., 0] + p[0] * _X[..., 1])
+
+ _X[..., :2] = _X_2d * (radial + tangential)[..., None] + torch.ger(
+ r2, p.flip([0])).reshape(_X_2d.shape)
+ return _X @ self.param['K']
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/core/distributed_wrapper.py b/engine/pose_estimation/third-party/ViTPose/mmpose/core/distributed_wrapper.py
new file mode 100644
index 0000000..c67acee
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/core/distributed_wrapper.py
@@ -0,0 +1,143 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import torch
+import torch.nn as nn
+from mmcv.parallel import MODULE_WRAPPERS as MMCV_MODULE_WRAPPERS
+from mmcv.parallel import MMDistributedDataParallel
+from mmcv.parallel.scatter_gather import scatter_kwargs
+from mmcv.utils import Registry
+from torch.cuda._utils import _get_device_index
+
+MODULE_WRAPPERS = Registry('module wrapper', parent=MMCV_MODULE_WRAPPERS)
+
+
+@MODULE_WRAPPERS.register_module()
+class DistributedDataParallelWrapper(nn.Module):
+ """A DistributedDataParallel wrapper for models in 3D mesh estimation task.
+
+ In 3D mesh estimation task, there is a need to wrap different modules in
+ the models with separate DistributedDataParallel. Otherwise, it will cause
+ errors for GAN training.
+ More specific, the GAN model, usually has two sub-modules:
+ generator and discriminator. If we wrap both of them in one
+ standard DistributedDataParallel, it will cause errors during training,
+ because when we update the parameters of the generator (or discriminator),
+ the parameters of the discriminator (or generator) is not updated, which is
+ not allowed for DistributedDataParallel.
+ So we design this wrapper to separately wrap DistributedDataParallel
+ for generator and discriminator.
+
+ In this wrapper, we perform two operations:
+ 1. Wrap the modules in the models with separate MMDistributedDataParallel.
+ Note that only modules with parameters will be wrapped.
+ 2. Do scatter operation for 'forward', 'train_step' and 'val_step'.
+
+ Note that the arguments of this wrapper is the same as those in
+ `torch.nn.parallel.distributed.DistributedDataParallel`.
+
+ Args:
+ module (nn.Module): Module that needs to be wrapped.
+ device_ids (list[int | `torch.device`]): Same as that in
+ `torch.nn.parallel.distributed.DistributedDataParallel`.
+ dim (int, optional): Same as that in the official scatter function in
+ pytorch. Defaults to 0.
+ broadcast_buffers (bool): Same as that in
+ `torch.nn.parallel.distributed.DistributedDataParallel`.
+ Defaults to False.
+ find_unused_parameters (bool, optional): Same as that in
+ `torch.nn.parallel.distributed.DistributedDataParallel`.
+ Traverse the autograd graph of all tensors contained in returned
+ value of the wrapped module’s forward function. Defaults to False.
+ kwargs (dict): Other arguments used in
+ `torch.nn.parallel.distributed.DistributedDataParallel`.
+ """
+
+ def __init__(self,
+ module,
+ device_ids,
+ dim=0,
+ broadcast_buffers=False,
+ find_unused_parameters=False,
+ **kwargs):
+ super().__init__()
+ assert len(device_ids) == 1, (
+ 'Currently, DistributedDataParallelWrapper only supports one'
+ 'single CUDA device for each process.'
+ f'The length of device_ids must be 1, but got {len(device_ids)}.')
+ self.module = module
+ self.dim = dim
+ self.to_ddp(
+ device_ids=device_ids,
+ dim=dim,
+ broadcast_buffers=broadcast_buffers,
+ find_unused_parameters=find_unused_parameters,
+ **kwargs)
+ self.output_device = _get_device_index(device_ids[0], True)
+
+ def to_ddp(self, device_ids, dim, broadcast_buffers,
+ find_unused_parameters, **kwargs):
+ """Wrap models with separate MMDistributedDataParallel.
+
+ It only wraps the modules with parameters.
+ """
+ for name, module in self.module._modules.items():
+ if next(module.parameters(), None) is None:
+ module = module.cuda()
+ elif all(not p.requires_grad for p in module.parameters()):
+ module = module.cuda()
+ else:
+ module = MMDistributedDataParallel(
+ module.cuda(),
+ device_ids=device_ids,
+ dim=dim,
+ broadcast_buffers=broadcast_buffers,
+ find_unused_parameters=find_unused_parameters,
+ **kwargs)
+ self.module._modules[name] = module
+
+ def scatter(self, inputs, kwargs, device_ids):
+ """Scatter function.
+
+ Args:
+ inputs (Tensor): Input Tensor.
+ kwargs (dict): Args for
+ ``mmcv.parallel.scatter_gather.scatter_kwargs``.
+ device_ids (int): Device id.
+ """
+ return scatter_kwargs(inputs, kwargs, device_ids, dim=self.dim)
+
+ def forward(self, *inputs, **kwargs):
+ """Forward function.
+
+ Args:
+ inputs (tuple): Input data.
+ kwargs (dict): Args for
+ ``mmcv.parallel.scatter_gather.scatter_kwargs``.
+ """
+ inputs, kwargs = self.scatter(inputs, kwargs,
+ [torch.cuda.current_device()])
+ return self.module(*inputs[0], **kwargs[0])
+
+ def train_step(self, *inputs, **kwargs):
+ """Train step function.
+
+ Args:
+ inputs (Tensor): Input Tensor.
+ kwargs (dict): Args for
+ ``mmcv.parallel.scatter_gather.scatter_kwargs``.
+ """
+ inputs, kwargs = self.scatter(inputs, kwargs,
+ [torch.cuda.current_device()])
+ output = self.module.train_step(*inputs[0], **kwargs[0])
+ return output
+
+ def val_step(self, *inputs, **kwargs):
+ """Validation step function.
+
+ Args:
+ inputs (tuple): Input data.
+ kwargs (dict): Args for ``scatter_kwargs``.
+ """
+ inputs, kwargs = self.scatter(inputs, kwargs,
+ [torch.cuda.current_device()])
+ output = self.module.val_step(*inputs[0], **kwargs[0])
+ return output
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/core/evaluation/__init__.py b/engine/pose_estimation/third-party/ViTPose/mmpose/core/evaluation/__init__.py
new file mode 100644
index 0000000..5f93784
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/core/evaluation/__init__.py
@@ -0,0 +1,22 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from .bottom_up_eval import (aggregate_scale, aggregate_stage_flip,
+ flip_feature_maps, get_group_preds,
+ split_ae_outputs)
+from .eval_hooks import DistEvalHook, EvalHook
+from .mesh_eval import compute_similarity_transform
+from .pose3d_eval import keypoint_3d_auc, keypoint_3d_pck, keypoint_mpjpe
+from .top_down_eval import (keypoint_auc, keypoint_epe, keypoint_pck_accuracy,
+ keypoints_from_heatmaps, keypoints_from_heatmaps3d,
+ keypoints_from_regression,
+ multilabel_classification_accuracy,
+ pose_pck_accuracy, post_dark_udp)
+
+__all__ = [
+ 'EvalHook', 'DistEvalHook', 'pose_pck_accuracy', 'keypoints_from_heatmaps',
+ 'keypoints_from_regression', 'keypoint_pck_accuracy', 'keypoint_3d_pck',
+ 'keypoint_3d_auc', 'keypoint_auc', 'keypoint_epe', 'get_group_preds',
+ 'split_ae_outputs', 'flip_feature_maps', 'aggregate_stage_flip',
+ 'aggregate_scale', 'compute_similarity_transform', 'post_dark_udp',
+ 'keypoint_mpjpe', 'keypoints_from_heatmaps3d',
+ 'multilabel_classification_accuracy'
+]
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/core/evaluation/bottom_up_eval.py b/engine/pose_estimation/third-party/ViTPose/mmpose/core/evaluation/bottom_up_eval.py
new file mode 100644
index 0000000..7b37d7c
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/core/evaluation/bottom_up_eval.py
@@ -0,0 +1,333 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import numpy as np
+import torch
+
+from mmpose.core.post_processing import (get_warp_matrix, transform_preds,
+ warp_affine_joints)
+
+
+def split_ae_outputs(outputs, num_joints, with_heatmaps, with_ae,
+ select_output_index):
+ """Split multi-stage outputs into heatmaps & tags.
+
+ Args:
+ outputs (list(Tensor)): Outputs of network
+ num_joints (int): Number of joints
+ with_heatmaps (list[bool]): Option to output
+ heatmaps for different stages.
+ with_ae (list[bool]): Option to output
+ ae tags for different stages.
+ select_output_index (list[int]): Output keep the selected index
+
+ Returns:
+ tuple: A tuple containing multi-stage outputs.
+
+ - list[Tensor]: multi-stage heatmaps.
+ - list[Tensor]: multi-stage tags.
+ """
+
+ heatmaps = []
+ tags = []
+
+ # aggregate heatmaps from different stages
+ for i, output in enumerate(outputs):
+ if i not in select_output_index:
+ continue
+ # staring index of the associative embeddings
+ offset_feat = num_joints if with_heatmaps[i] else 0
+ if with_heatmaps[i]:
+ heatmaps.append(output[:, :num_joints])
+ if with_ae[i]:
+ tags.append(output[:, offset_feat:])
+
+ return heatmaps, tags
+
+
+def flip_feature_maps(feature_maps, flip_index=None):
+ """Flip the feature maps and swap the channels.
+
+ Args:
+ feature_maps (list[Tensor]): Feature maps.
+ flip_index (list[int] | None): Channel-flip indexes.
+ If None, do not flip channels.
+
+ Returns:
+ list[Tensor]: Flipped feature_maps.
+ """
+ flipped_feature_maps = []
+ for feature_map in feature_maps:
+ feature_map = torch.flip(feature_map, [3])
+ if flip_index is not None:
+ flipped_feature_maps.append(feature_map[:, flip_index, :, :])
+ else:
+ flipped_feature_maps.append(feature_map)
+
+ return flipped_feature_maps
+
+
+def _resize_average(feature_maps, align_corners, index=-1, resize_size=None):
+ """Resize the feature maps and compute the average.
+
+ Args:
+ feature_maps (list[Tensor]): Feature maps.
+ align_corners (bool): Align corners when performing interpolation.
+ index (int): Only used when `resize_size' is None.
+ If `resize_size' is None, the target size is the size
+ of the indexed feature maps.
+ resize_size (list[int, int]): The target size [w, h].
+
+ Returns:
+ list[Tensor]: Averaged feature_maps.
+ """
+
+ if feature_maps is None:
+ return None
+ feature_maps_avg = 0
+
+ feature_map_list = _resize_concate(
+ feature_maps, align_corners, index=index, resize_size=resize_size)
+ for feature_map in feature_map_list:
+ feature_maps_avg += feature_map
+
+ feature_maps_avg /= len(feature_map_list)
+ return [feature_maps_avg]
+
+
+def _resize_unsqueeze_concat(feature_maps,
+ align_corners,
+ index=-1,
+ resize_size=None):
+ """Resize, unsqueeze and concatenate the feature_maps.
+
+ Args:
+ feature_maps (list[Tensor]): Feature maps.
+ align_corners (bool): Align corners when performing interpolation.
+ index (int): Only used when `resize_size' is None.
+ If `resize_size' is None, the target size is the size
+ of the indexed feature maps.
+ resize_size (list[int, int]): The target size [w, h].
+
+ Returns:
+ list[Tensor]: Averaged feature_maps.
+ """
+ if feature_maps is None:
+ return None
+ feature_map_list = _resize_concate(
+ feature_maps, align_corners, index=index, resize_size=resize_size)
+
+ feat_dim = len(feature_map_list[0].shape) - 1
+ output_feature_maps = torch.cat(
+ [torch.unsqueeze(fmap, dim=feat_dim + 1) for fmap in feature_map_list],
+ dim=feat_dim + 1)
+ return [output_feature_maps]
+
+
+def _resize_concate(feature_maps, align_corners, index=-1, resize_size=None):
+ """Resize and concatenate the feature_maps.
+
+ Args:
+ feature_maps (list[Tensor]): Feature maps.
+ align_corners (bool): Align corners when performing interpolation.
+ index (int): Only used when `resize_size' is None.
+ If `resize_size' is None, the target size is the size
+ of the indexed feature maps.
+ resize_size (list[int, int]): The target size [w, h].
+
+ Returns:
+ list[Tensor]: Averaged feature_maps.
+ """
+ if feature_maps is None:
+ return None
+
+ feature_map_list = []
+
+ if index < 0:
+ index += len(feature_maps)
+
+ if resize_size is None:
+ resize_size = (feature_maps[index].size(2),
+ feature_maps[index].size(3))
+
+ for feature_map in feature_maps:
+ ori_size = (feature_map.size(2), feature_map.size(3))
+ if ori_size != resize_size:
+ feature_map = torch.nn.functional.interpolate(
+ feature_map,
+ size=resize_size,
+ mode='bilinear',
+ align_corners=align_corners)
+
+ feature_map_list.append(feature_map)
+
+ return feature_map_list
+
+
+def aggregate_stage_flip(feature_maps,
+ feature_maps_flip,
+ index=-1,
+ project2image=True,
+ size_projected=None,
+ align_corners=False,
+ aggregate_stage='concat',
+ aggregate_flip='average'):
+ """Inference the model to get multi-stage outputs (heatmaps & tags), and
+ resize them to base sizes.
+
+ Args:
+ feature_maps (list[Tensor]): feature_maps can be heatmaps,
+ tags, and pafs.
+ feature_maps_flip (list[Tensor] | None): flipped feature_maps.
+ feature maps can be heatmaps, tags, and pafs.
+ project2image (bool): Option to resize to base scale.
+ size_projected (list[int, int]): Base size of heatmaps [w, h].
+ align_corners (bool): Align corners when performing interpolation.
+ aggregate_stage (str): Methods to aggregate multi-stage feature maps.
+ Options: 'concat', 'average'. Default: 'concat.
+
+ - 'concat': Concatenate the original and the flipped feature maps.
+ - 'average': Get the average of the original and the flipped
+ feature maps.
+ aggregate_flip (str): Methods to aggregate the original and
+ the flipped feature maps. Options: 'concat', 'average', 'none'.
+ Default: 'average.
+
+ - 'concat': Concatenate the original and the flipped feature maps.
+ - 'average': Get the average of the original and the flipped
+ feature maps..
+ - 'none': no flipped feature maps.
+
+ Returns:
+ list[Tensor]: Aggregated feature maps with shape [NxKxWxH].
+ """
+
+ if feature_maps_flip is None:
+ aggregate_flip = 'none'
+
+ output_feature_maps = []
+
+ if aggregate_stage == 'average':
+ _aggregate_stage_func = _resize_average
+ elif aggregate_stage == 'concat':
+ _aggregate_stage_func = _resize_concate
+ else:
+ NotImplementedError()
+
+ if project2image and size_projected:
+ _origin = _aggregate_stage_func(
+ feature_maps,
+ align_corners,
+ index=index,
+ resize_size=(size_projected[1], size_projected[0]))
+
+ _flipped = _aggregate_stage_func(
+ feature_maps_flip,
+ align_corners,
+ index=index,
+ resize_size=(size_projected[1], size_projected[0]))
+ else:
+ _origin = _aggregate_stage_func(
+ feature_maps, align_corners, index=index, resize_size=None)
+ _flipped = _aggregate_stage_func(
+ feature_maps_flip, align_corners, index=index, resize_size=None)
+
+ if aggregate_flip == 'average':
+ assert feature_maps_flip is not None
+ for _ori, _fli in zip(_origin, _flipped):
+ output_feature_maps.append((_ori + _fli) / 2.0)
+
+ elif aggregate_flip == 'concat':
+ assert feature_maps_flip is not None
+ output_feature_maps.append(*_origin)
+ output_feature_maps.append(*_flipped)
+
+ elif aggregate_flip == 'none':
+ if isinstance(_origin, list):
+ output_feature_maps.append(*_origin)
+ else:
+ output_feature_maps.append(_origin)
+ else:
+ NotImplementedError()
+
+ return output_feature_maps
+
+
+def aggregate_scale(feature_maps_list,
+ align_corners=False,
+ aggregate_scale='average'):
+ """Aggregate multi-scale outputs.
+
+ Note:
+ batch size: N
+ keypoints num : K
+ heatmap width: W
+ heatmap height: H
+
+ Args:
+ feature_maps_list (list[Tensor]): Aggregated feature maps.
+ project2image (bool): Option to resize to base scale.
+ align_corners (bool): Align corners when performing interpolation.
+ aggregate_scale (str): Methods to aggregate multi-scale feature maps.
+ Options: 'average', 'unsqueeze_concat'.
+
+ - 'average': Get the average of the feature maps.
+ - 'unsqueeze_concat': Concatenate the feature maps along new axis.
+ Default: 'average.
+
+ Returns:
+ Tensor: Aggregated feature maps.
+ """
+
+ if aggregate_scale == 'average':
+ output_feature_maps = _resize_average(
+ feature_maps_list, align_corners, index=0, resize_size=None)
+
+ elif aggregate_scale == 'unsqueeze_concat':
+ output_feature_maps = _resize_unsqueeze_concat(
+ feature_maps_list, align_corners, index=0, resize_size=None)
+ else:
+ NotImplementedError()
+
+ return output_feature_maps[0]
+
+
+def get_group_preds(grouped_joints,
+ center,
+ scale,
+ heatmap_size,
+ use_udp=False):
+ """Transform the grouped joints back to the image.
+
+ Args:
+ grouped_joints (list): Grouped person joints.
+ center (np.ndarray[2, ]): Center of the bounding box (x, y).
+ scale (np.ndarray[2, ]): Scale of the bounding box
+ wrt [width, height].
+ heatmap_size (np.ndarray[2, ]): Size of the destination heatmaps.
+ use_udp (bool): Unbiased data processing.
+ Paper ref: Huang et al. The Devil is in the Details: Delving into
+ Unbiased Data Processing for Human Pose Estimation (CVPR'2020).
+
+ Returns:
+ list: List of the pose result for each person.
+ """
+ if len(grouped_joints) == 0:
+ return []
+
+ if use_udp:
+ if grouped_joints[0].shape[0] > 0:
+ heatmap_size_t = np.array(heatmap_size, dtype=np.float32) - 1.0
+ trans = get_warp_matrix(
+ theta=0,
+ size_input=heatmap_size_t,
+ size_dst=scale,
+ size_target=heatmap_size_t)
+ grouped_joints[0][..., :2] = \
+ warp_affine_joints(grouped_joints[0][..., :2], trans)
+ results = [person for person in grouped_joints[0]]
+ else:
+ results = []
+ for person in grouped_joints[0]:
+ joints = transform_preds(person, center, scale, heatmap_size)
+ results.append(joints)
+
+ return results
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/core/evaluation/eval_hooks.py b/engine/pose_estimation/third-party/ViTPose/mmpose/core/evaluation/eval_hooks.py
new file mode 100644
index 0000000..cf36a03
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/core/evaluation/eval_hooks.py
@@ -0,0 +1,98 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import warnings
+
+from mmcv.runner import DistEvalHook as _DistEvalHook
+from mmcv.runner import EvalHook as _EvalHook
+
+MMPOSE_GREATER_KEYS = [
+ 'acc', 'ap', 'ar', 'pck', 'auc', '3dpck', 'p-3dpck', '3dauc', 'p-3dauc'
+]
+MMPOSE_LESS_KEYS = ['loss', 'epe', 'nme', 'mpjpe', 'p-mpjpe', 'n-mpjpe']
+
+
+class EvalHook(_EvalHook):
+
+ def __init__(self,
+ dataloader,
+ start=None,
+ interval=1,
+ by_epoch=True,
+ save_best=None,
+ rule=None,
+ test_fn=None,
+ greater_keys=MMPOSE_GREATER_KEYS,
+ less_keys=MMPOSE_LESS_KEYS,
+ **eval_kwargs):
+
+ if test_fn is None:
+ from mmpose.apis import single_gpu_test
+ test_fn = single_gpu_test
+
+ # to be compatible with the config before v0.16.0
+
+ # remove "gpu_collect" from eval_kwargs
+ if 'gpu_collect' in eval_kwargs:
+ warnings.warn(
+ '"gpu_collect" will be deprecated in EvalHook.'
+ 'Please remove it from the config.', DeprecationWarning)
+ _ = eval_kwargs.pop('gpu_collect')
+
+ # update "save_best" according to "key_indicator" and remove the
+ # latter from eval_kwargs
+ if 'key_indicator' in eval_kwargs or isinstance(save_best, bool):
+ warnings.warn(
+ '"key_indicator" will be deprecated in EvalHook.'
+ 'Please use "save_best" to specify the metric key,'
+ 'e.g., save_best="AP".', DeprecationWarning)
+
+ key_indicator = eval_kwargs.pop('key_indicator', 'AP')
+ if save_best is True and key_indicator is None:
+ raise ValueError('key_indicator should not be None, when '
+ 'save_best is set to True.')
+ save_best = key_indicator
+
+ super().__init__(dataloader, start, interval, by_epoch, save_best,
+ rule, test_fn, greater_keys, less_keys, **eval_kwargs)
+
+
+class DistEvalHook(_DistEvalHook):
+
+ def __init__(self,
+ dataloader,
+ start=None,
+ interval=1,
+ by_epoch=True,
+ save_best=None,
+ rule=None,
+ test_fn=None,
+ greater_keys=MMPOSE_GREATER_KEYS,
+ less_keys=MMPOSE_LESS_KEYS,
+ broadcast_bn_buffer=True,
+ tmpdir=None,
+ gpu_collect=False,
+ **eval_kwargs):
+
+ if test_fn is None:
+ from mmpose.apis import multi_gpu_test
+ test_fn = multi_gpu_test
+
+ # to be compatible with the config before v0.16.0
+
+ # update "save_best" according to "key_indicator" and remove the
+ # latter from eval_kwargs
+ if 'key_indicator' in eval_kwargs or isinstance(save_best, bool):
+ warnings.warn(
+ '"key_indicator" will be deprecated in EvalHook.'
+ 'Please use "save_best" to specify the metric key,'
+ 'e.g., save_best="AP".', DeprecationWarning)
+
+ key_indicator = eval_kwargs.pop('key_indicator', 'AP')
+ if save_best is True and key_indicator is None:
+ raise ValueError('key_indicator should not be None, when '
+ 'save_best is set to True.')
+ save_best = key_indicator
+
+ super().__init__(dataloader, start, interval, by_epoch, save_best,
+ rule, test_fn, greater_keys, less_keys,
+ broadcast_bn_buffer, tmpdir, gpu_collect,
+ **eval_kwargs)
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/core/evaluation/mesh_eval.py b/engine/pose_estimation/third-party/ViTPose/mmpose/core/evaluation/mesh_eval.py
new file mode 100644
index 0000000..683b453
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/core/evaluation/mesh_eval.py
@@ -0,0 +1,66 @@
+# ------------------------------------------------------------------------------
+# Adapted from https://github.com/akanazawa/hmr
+# Original licence: Copyright (c) 2018 akanazawa, under the MIT License.
+# ------------------------------------------------------------------------------
+
+import numpy as np
+
+
+def compute_similarity_transform(source_points, target_points):
+ """Computes a similarity transform (sR, t) that takes a set of 3D points
+ source_points (N x 3) closest to a set of 3D points target_points, where R
+ is an 3x3 rotation matrix, t 3x1 translation, s scale. And return the
+ transformed 3D points source_points_hat (N x 3). i.e. solves the orthogonal
+ Procrutes problem.
+
+ Note:
+ Points number: N
+
+ Args:
+ source_points (np.ndarray): Source point set with shape [N, 3].
+ target_points (np.ndarray): Target point set with shape [N, 3].
+
+ Returns:
+ np.ndarray: Transformed source point set with shape [N, 3].
+ """
+
+ assert target_points.shape[0] == source_points.shape[0]
+ assert target_points.shape[1] == 3 and source_points.shape[1] == 3
+
+ source_points = source_points.T
+ target_points = target_points.T
+
+ # 1. Remove mean.
+ mu1 = source_points.mean(axis=1, keepdims=True)
+ mu2 = target_points.mean(axis=1, keepdims=True)
+ X1 = source_points - mu1
+ X2 = target_points - mu2
+
+ # 2. Compute variance of X1 used for scale.
+ var1 = np.sum(X1**2)
+
+ # 3. The outer product of X1 and X2.
+ K = X1.dot(X2.T)
+
+ # 4. Solution that Maximizes trace(R'K) is R=U*V', where U, V are
+ # singular vectors of K.
+ U, _, Vh = np.linalg.svd(K)
+ V = Vh.T
+ # Construct Z that fixes the orientation of R to get det(R)=1.
+ Z = np.eye(U.shape[0])
+ Z[-1, -1] *= np.sign(np.linalg.det(U.dot(V.T)))
+ # Construct R.
+ R = V.dot(Z.dot(U.T))
+
+ # 5. Recover scale.
+ scale = np.trace(R.dot(K)) / var1
+
+ # 6. Recover translation.
+ t = mu2 - scale * (R.dot(mu1))
+
+ # 7. Transform the source points:
+ source_points_hat = scale * R.dot(source_points) + t
+
+ source_points_hat = source_points_hat.T
+
+ return source_points_hat
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/core/evaluation/pose3d_eval.py b/engine/pose_estimation/third-party/ViTPose/mmpose/core/evaluation/pose3d_eval.py
new file mode 100644
index 0000000..545778c
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/core/evaluation/pose3d_eval.py
@@ -0,0 +1,171 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import numpy as np
+
+from .mesh_eval import compute_similarity_transform
+
+
+def keypoint_mpjpe(pred, gt, mask, alignment='none'):
+ """Calculate the mean per-joint position error (MPJPE) and the error after
+ rigid alignment with the ground truth (P-MPJPE).
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+ - keypoint_dims: C
+
+ Args:
+ pred (np.ndarray): Predicted keypoint location with shape [N, K, C].
+ gt (np.ndarray): Groundtruth keypoint location with shape [N, K, C].
+ mask (np.ndarray): Visibility of the target with shape [N, K].
+ False for invisible joints, and True for visible.
+ Invisible joints will be ignored for accuracy calculation.
+ alignment (str, optional): method to align the prediction with the
+ groundtruth. Supported options are:
+
+ - ``'none'``: no alignment will be applied
+ - ``'scale'``: align in the least-square sense in scale
+ - ``'procrustes'``: align in the least-square sense in
+ scale, rotation and translation.
+ Returns:
+ tuple: A tuple containing joint position errors
+
+ - (float | np.ndarray): mean per-joint position error (mpjpe).
+ - (float | np.ndarray): mpjpe after rigid alignment with the
+ ground truth (p-mpjpe).
+ """
+ assert mask.any()
+
+ if alignment == 'none':
+ pass
+ elif alignment == 'procrustes':
+ pred = np.stack([
+ compute_similarity_transform(pred_i, gt_i)
+ for pred_i, gt_i in zip(pred, gt)
+ ])
+ elif alignment == 'scale':
+ pred_dot_pred = np.einsum('nkc,nkc->n', pred, pred)
+ pred_dot_gt = np.einsum('nkc,nkc->n', pred, gt)
+ scale_factor = pred_dot_gt / pred_dot_pred
+ pred = pred * scale_factor[:, None, None]
+ else:
+ raise ValueError(f'Invalid value for alignment: {alignment}')
+
+ error = np.linalg.norm(pred - gt, ord=2, axis=-1)[mask].mean()
+
+ return error
+
+
+def keypoint_3d_pck(pred, gt, mask, alignment='none', threshold=0.15):
+ """Calculate the Percentage of Correct Keypoints (3DPCK) w. or w/o rigid
+ alignment.
+
+ Paper ref: `Monocular 3D Human Pose Estimation In The Wild Using Improved
+ CNN Supervision' 3DV'2017. `__ .
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+ - keypoint_dims: C
+
+ Args:
+ pred (np.ndarray[N, K, C]): Predicted keypoint location.
+ gt (np.ndarray[N, K, C]): Groundtruth keypoint location.
+ mask (np.ndarray[N, K]): Visibility of the target. False for invisible
+ joints, and True for visible. Invisible joints will be ignored for
+ accuracy calculation.
+ alignment (str, optional): method to align the prediction with the
+ groundtruth. Supported options are:
+
+ - ``'none'``: no alignment will be applied
+ - ``'scale'``: align in the least-square sense in scale
+ - ``'procrustes'``: align in the least-square sense in scale,
+ rotation and translation.
+
+ threshold: If L2 distance between the prediction and the groundtruth
+ is less then threshold, the predicted result is considered as
+ correct. Default: 0.15 (m).
+
+ Returns:
+ pck: percentage of correct keypoints.
+ """
+ assert mask.any()
+
+ if alignment == 'none':
+ pass
+ elif alignment == 'procrustes':
+ pred = np.stack([
+ compute_similarity_transform(pred_i, gt_i)
+ for pred_i, gt_i in zip(pred, gt)
+ ])
+ elif alignment == 'scale':
+ pred_dot_pred = np.einsum('nkc,nkc->n', pred, pred)
+ pred_dot_gt = np.einsum('nkc,nkc->n', pred, gt)
+ scale_factor = pred_dot_gt / pred_dot_pred
+ pred = pred * scale_factor[:, None, None]
+ else:
+ raise ValueError(f'Invalid value for alignment: {alignment}')
+
+ error = np.linalg.norm(pred - gt, ord=2, axis=-1)
+ pck = (error < threshold).astype(np.float32)[mask].mean() * 100
+
+ return pck
+
+
+def keypoint_3d_auc(pred, gt, mask, alignment='none'):
+ """Calculate the Area Under the Curve (3DAUC) computed for a range of 3DPCK
+ thresholds.
+
+ Paper ref: `Monocular 3D Human Pose Estimation In The Wild Using Improved
+ CNN Supervision' 3DV'2017. `__ .
+ This implementation is derived from mpii_compute_3d_pck.m, which is
+ provided as part of the MPI-INF-3DHP test data release.
+
+ Note:
+ batch_size: N
+ num_keypoints: K
+ keypoint_dims: C
+
+ Args:
+ pred (np.ndarray[N, K, C]): Predicted keypoint location.
+ gt (np.ndarray[N, K, C]): Groundtruth keypoint location.
+ mask (np.ndarray[N, K]): Visibility of the target. False for invisible
+ joints, and True for visible. Invisible joints will be ignored for
+ accuracy calculation.
+ alignment (str, optional): method to align the prediction with the
+ groundtruth. Supported options are:
+
+ - ``'none'``: no alignment will be applied
+ - ``'scale'``: align in the least-square sense in scale
+ - ``'procrustes'``: align in the least-square sense in scale,
+ rotation and translation.
+
+ Returns:
+ auc: AUC computed for a range of 3DPCK thresholds.
+ """
+ assert mask.any()
+
+ if alignment == 'none':
+ pass
+ elif alignment == 'procrustes':
+ pred = np.stack([
+ compute_similarity_transform(pred_i, gt_i)
+ for pred_i, gt_i in zip(pred, gt)
+ ])
+ elif alignment == 'scale':
+ pred_dot_pred = np.einsum('nkc,nkc->n', pred, pred)
+ pred_dot_gt = np.einsum('nkc,nkc->n', pred, gt)
+ scale_factor = pred_dot_gt / pred_dot_pred
+ pred = pred * scale_factor[:, None, None]
+ else:
+ raise ValueError(f'Invalid value for alignment: {alignment}')
+
+ error = np.linalg.norm(pred - gt, ord=2, axis=-1)
+
+ thresholds = np.linspace(0., 0.15, 31)
+ pck_values = np.zeros(len(thresholds))
+ for i in range(len(thresholds)):
+ pck_values[i] = (error < thresholds[i]).astype(np.float32)[mask].mean()
+
+ auc = pck_values.mean() * 100
+
+ return auc
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/core/evaluation/top_down_eval.py b/engine/pose_estimation/third-party/ViTPose/mmpose/core/evaluation/top_down_eval.py
new file mode 100644
index 0000000..ee6a250
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/core/evaluation/top_down_eval.py
@@ -0,0 +1,684 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import warnings
+
+import cv2
+import numpy as np
+
+from mmpose.core.post_processing import transform_preds
+
+
+def _calc_distances(preds, targets, mask, normalize):
+ """Calculate the normalized distances between preds and target.
+
+ Note:
+ batch_size: N
+ num_keypoints: K
+ dimension of keypoints: D (normally, D=2 or D=3)
+
+ Args:
+ preds (np.ndarray[N, K, D]): Predicted keypoint location.
+ targets (np.ndarray[N, K, D]): Groundtruth keypoint location.
+ mask (np.ndarray[N, K]): Visibility of the target. False for invisible
+ joints, and True for visible. Invisible joints will be ignored for
+ accuracy calculation.
+ normalize (np.ndarray[N, D]): Typical value is heatmap_size
+
+ Returns:
+ np.ndarray[K, N]: The normalized distances. \
+ If target keypoints are missing, the distance is -1.
+ """
+ N, K, _ = preds.shape
+ # set mask=0 when normalize==0
+ _mask = mask.copy()
+ _mask[np.where((normalize == 0).sum(1))[0], :] = False
+ distances = np.full((N, K), -1, dtype=np.float32)
+ # handle invalid values
+ normalize[np.where(normalize <= 0)] = 1e6
+ distances[_mask] = np.linalg.norm(
+ ((preds - targets) / normalize[:, None, :])[_mask], axis=-1)
+ return distances.T
+
+
+def _distance_acc(distances, thr=0.5):
+ """Return the percentage below the distance threshold, while ignoring
+ distances values with -1.
+
+ Note:
+ batch_size: N
+ Args:
+ distances (np.ndarray[N, ]): The normalized distances.
+ thr (float): Threshold of the distances.
+
+ Returns:
+ float: Percentage of distances below the threshold. \
+ If all target keypoints are missing, return -1.
+ """
+ distance_valid = distances != -1
+ num_distance_valid = distance_valid.sum()
+ if num_distance_valid > 0:
+ return (distances[distance_valid] < thr).sum() / num_distance_valid
+ return -1
+
+
+def _get_max_preds(heatmaps):
+ """Get keypoint predictions from score maps.
+
+ Note:
+ batch_size: N
+ num_keypoints: K
+ heatmap height: H
+ heatmap width: W
+
+ Args:
+ heatmaps (np.ndarray[N, K, H, W]): model predicted heatmaps.
+
+ Returns:
+ tuple: A tuple containing aggregated results.
+
+ - preds (np.ndarray[N, K, 2]): Predicted keypoint location.
+ - maxvals (np.ndarray[N, K, 1]): Scores (confidence) of the keypoints.
+ """
+ assert isinstance(heatmaps,
+ np.ndarray), ('heatmaps should be numpy.ndarray')
+ assert heatmaps.ndim == 4, 'batch_images should be 4-ndim'
+
+ N, K, _, W = heatmaps.shape
+ heatmaps_reshaped = heatmaps.reshape((N, K, -1))
+ idx = np.argmax(heatmaps_reshaped, 2).reshape((N, K, 1))
+ maxvals = np.amax(heatmaps_reshaped, 2).reshape((N, K, 1))
+
+ preds = np.tile(idx, (1, 1, 2)).astype(np.float32)
+ preds[:, :, 0] = preds[:, :, 0] % W
+ preds[:, :, 1] = preds[:, :, 1] // W
+
+ preds = np.where(np.tile(maxvals, (1, 1, 2)) > 0.0, preds, -1)
+ return preds, maxvals
+
+
+def _get_max_preds_3d(heatmaps):
+ """Get keypoint predictions from 3D score maps.
+
+ Note:
+ batch size: N
+ num keypoints: K
+ heatmap depth size: D
+ heatmap height: H
+ heatmap width: W
+
+ Args:
+ heatmaps (np.ndarray[N, K, D, H, W]): model predicted heatmaps.
+
+ Returns:
+ tuple: A tuple containing aggregated results.
+
+ - preds (np.ndarray[N, K, 3]): Predicted keypoint location.
+ - maxvals (np.ndarray[N, K, 1]): Scores (confidence) of the keypoints.
+ """
+ assert isinstance(heatmaps, np.ndarray), \
+ ('heatmaps should be numpy.ndarray')
+ assert heatmaps.ndim == 5, 'heatmaps should be 5-ndim'
+
+ N, K, D, H, W = heatmaps.shape
+ heatmaps_reshaped = heatmaps.reshape((N, K, -1))
+ idx = np.argmax(heatmaps_reshaped, 2).reshape((N, K, 1))
+ maxvals = np.amax(heatmaps_reshaped, 2).reshape((N, K, 1))
+
+ preds = np.zeros((N, K, 3), dtype=np.float32)
+ _idx = idx[..., 0]
+ preds[..., 2] = _idx // (H * W)
+ preds[..., 1] = (_idx // W) % H
+ preds[..., 0] = _idx % W
+
+ preds = np.where(maxvals > 0.0, preds, -1)
+ return preds, maxvals
+
+
+def pose_pck_accuracy(output, target, mask, thr=0.05, normalize=None):
+ """Calculate the pose accuracy of PCK for each individual keypoint and the
+ averaged accuracy across all keypoints from heatmaps.
+
+ Note:
+ PCK metric measures accuracy of the localization of the body joints.
+ The distances between predicted positions and the ground-truth ones
+ are typically normalized by the bounding box size.
+ The threshold (thr) of the normalized distance is commonly set
+ as 0.05, 0.1 or 0.2 etc.
+
+ - batch_size: N
+ - num_keypoints: K
+ - heatmap height: H
+ - heatmap width: W
+
+ Args:
+ output (np.ndarray[N, K, H, W]): Model output heatmaps.
+ target (np.ndarray[N, K, H, W]): Groundtruth heatmaps.
+ mask (np.ndarray[N, K]): Visibility of the target. False for invisible
+ joints, and True for visible. Invisible joints will be ignored for
+ accuracy calculation.
+ thr (float): Threshold of PCK calculation. Default 0.05.
+ normalize (np.ndarray[N, 2]): Normalization factor for H&W.
+
+ Returns:
+ tuple: A tuple containing keypoint accuracy.
+
+ - np.ndarray[K]: Accuracy of each keypoint.
+ - float: Averaged accuracy across all keypoints.
+ - int: Number of valid keypoints.
+ """
+ N, K, H, W = output.shape
+ if K == 0:
+ return None, 0, 0
+ if normalize is None:
+ normalize = np.tile(np.array([[H, W]]), (N, 1))
+
+ pred, _ = _get_max_preds(output)
+ gt, _ = _get_max_preds(target)
+ return keypoint_pck_accuracy(pred, gt, mask, thr, normalize)
+
+
+def keypoint_pck_accuracy(pred, gt, mask, thr, normalize):
+ """Calculate the pose accuracy of PCK for each individual keypoint and the
+ averaged accuracy across all keypoints for coordinates.
+
+ Note:
+ PCK metric measures accuracy of the localization of the body joints.
+ The distances between predicted positions and the ground-truth ones
+ are typically normalized by the bounding box size.
+ The threshold (thr) of the normalized distance is commonly set
+ as 0.05, 0.1 or 0.2 etc.
+
+ - batch_size: N
+ - num_keypoints: K
+
+ Args:
+ pred (np.ndarray[N, K, 2]): Predicted keypoint location.
+ gt (np.ndarray[N, K, 2]): Groundtruth keypoint location.
+ mask (np.ndarray[N, K]): Visibility of the target. False for invisible
+ joints, and True for visible. Invisible joints will be ignored for
+ accuracy calculation.
+ thr (float): Threshold of PCK calculation.
+ normalize (np.ndarray[N, 2]): Normalization factor for H&W.
+
+ Returns:
+ tuple: A tuple containing keypoint accuracy.
+
+ - acc (np.ndarray[K]): Accuracy of each keypoint.
+ - avg_acc (float): Averaged accuracy across all keypoints.
+ - cnt (int): Number of valid keypoints.
+ """
+ distances = _calc_distances(pred, gt, mask, normalize)
+
+ acc = np.array([_distance_acc(d, thr) for d in distances])
+ valid_acc = acc[acc >= 0]
+ cnt = len(valid_acc)
+ avg_acc = valid_acc.mean() if cnt > 0 else 0
+ return acc, avg_acc, cnt
+
+
+def keypoint_auc(pred, gt, mask, normalize, num_step=20):
+ """Calculate the pose accuracy of PCK for each individual keypoint and the
+ averaged accuracy across all keypoints for coordinates.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+
+ Args:
+ pred (np.ndarray[N, K, 2]): Predicted keypoint location.
+ gt (np.ndarray[N, K, 2]): Groundtruth keypoint location.
+ mask (np.ndarray[N, K]): Visibility of the target. False for invisible
+ joints, and True for visible. Invisible joints will be ignored for
+ accuracy calculation.
+ normalize (float): Normalization factor.
+
+ Returns:
+ float: Area under curve.
+ """
+ nor = np.tile(np.array([[normalize, normalize]]), (pred.shape[0], 1))
+ x = [1.0 * i / num_step for i in range(num_step)]
+ y = []
+ for thr in x:
+ _, avg_acc, _ = keypoint_pck_accuracy(pred, gt, mask, thr, nor)
+ y.append(avg_acc)
+
+ auc = 0
+ for i in range(num_step):
+ auc += 1.0 / num_step * y[i]
+ return auc
+
+
+def keypoint_nme(pred, gt, mask, normalize_factor):
+ """Calculate the normalized mean error (NME).
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+
+ Args:
+ pred (np.ndarray[N, K, 2]): Predicted keypoint location.
+ gt (np.ndarray[N, K, 2]): Groundtruth keypoint location.
+ mask (np.ndarray[N, K]): Visibility of the target. False for invisible
+ joints, and True for visible. Invisible joints will be ignored for
+ accuracy calculation.
+ normalize_factor (np.ndarray[N, 2]): Normalization factor.
+
+ Returns:
+ float: normalized mean error
+ """
+ distances = _calc_distances(pred, gt, mask, normalize_factor)
+ distance_valid = distances[distances != -1]
+ return distance_valid.sum() / max(1, len(distance_valid))
+
+
+def keypoint_epe(pred, gt, mask):
+ """Calculate the end-point error.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+
+ Args:
+ pred (np.ndarray[N, K, 2]): Predicted keypoint location.
+ gt (np.ndarray[N, K, 2]): Groundtruth keypoint location.
+ mask (np.ndarray[N, K]): Visibility of the target. False for invisible
+ joints, and True for visible. Invisible joints will be ignored for
+ accuracy calculation.
+
+ Returns:
+ float: Average end-point error.
+ """
+
+ distances = _calc_distances(
+ pred, gt, mask,
+ np.ones((pred.shape[0], pred.shape[2]), dtype=np.float32))
+ distance_valid = distances[distances != -1]
+ return distance_valid.sum() / max(1, len(distance_valid))
+
+
+def _taylor(heatmap, coord):
+ """Distribution aware coordinate decoding method.
+
+ Note:
+ - heatmap height: H
+ - heatmap width: W
+
+ Args:
+ heatmap (np.ndarray[H, W]): Heatmap of a particular joint type.
+ coord (np.ndarray[2,]): Coordinates of the predicted keypoints.
+
+ Returns:
+ np.ndarray[2,]: Updated coordinates.
+ """
+ H, W = heatmap.shape[:2]
+ px, py = int(coord[0]), int(coord[1])
+ if 1 < px < W - 2 and 1 < py < H - 2:
+ dx = 0.5 * (heatmap[py][px + 1] - heatmap[py][px - 1])
+ dy = 0.5 * (heatmap[py + 1][px] - heatmap[py - 1][px])
+ dxx = 0.25 * (
+ heatmap[py][px + 2] - 2 * heatmap[py][px] + heatmap[py][px - 2])
+ dxy = 0.25 * (
+ heatmap[py + 1][px + 1] - heatmap[py - 1][px + 1] -
+ heatmap[py + 1][px - 1] + heatmap[py - 1][px - 1])
+ dyy = 0.25 * (
+ heatmap[py + 2 * 1][px] - 2 * heatmap[py][px] +
+ heatmap[py - 2 * 1][px])
+ derivative = np.array([[dx], [dy]])
+ hessian = np.array([[dxx, dxy], [dxy, dyy]])
+ if dxx * dyy - dxy**2 != 0:
+ hessianinv = np.linalg.inv(hessian)
+ offset = -hessianinv @ derivative
+ offset = np.squeeze(np.array(offset.T), axis=0)
+ coord += offset
+ return coord
+
+
+def post_dark_udp(coords, batch_heatmaps, kernel=3):
+ """DARK post-pocessing. Implemented by udp. Paper ref: Huang et al. The
+ Devil is in the Details: Delving into Unbiased Data Processing for Human
+ Pose Estimation (CVPR 2020). Zhang et al. Distribution-Aware Coordinate
+ Representation for Human Pose Estimation (CVPR 2020).
+
+ Note:
+ - batch size: B
+ - num keypoints: K
+ - num persons: N
+ - height of heatmaps: H
+ - width of heatmaps: W
+
+ B=1 for bottom_up paradigm where all persons share the same heatmap.
+ B=N for top_down paradigm where each person has its own heatmaps.
+
+ Args:
+ coords (np.ndarray[N, K, 2]): Initial coordinates of human pose.
+ batch_heatmaps (np.ndarray[B, K, H, W]): batch_heatmaps
+ kernel (int): Gaussian kernel size (K) for modulation.
+
+ Returns:
+ np.ndarray([N, K, 2]): Refined coordinates.
+ """
+ if not isinstance(batch_heatmaps, np.ndarray):
+ batch_heatmaps = batch_heatmaps.cpu().numpy()
+ B, K, H, W = batch_heatmaps.shape
+ N = coords.shape[0]
+ assert (B == 1 or B == N)
+ for heatmaps in batch_heatmaps:
+ for heatmap in heatmaps:
+ cv2.GaussianBlur(heatmap, (kernel, kernel), 0, heatmap)
+ np.clip(batch_heatmaps, 0.001, 50, batch_heatmaps)
+ np.log(batch_heatmaps, batch_heatmaps)
+
+ batch_heatmaps_pad = np.pad(
+ batch_heatmaps, ((0, 0), (0, 0), (1, 1), (1, 1)),
+ mode='edge').flatten()
+
+ index = coords[..., 0] + 1 + (coords[..., 1] + 1) * (W + 2)
+ index += (W + 2) * (H + 2) * np.arange(0, B * K).reshape(-1, K)
+ index = index.astype(int).reshape(-1, 1)
+ i_ = batch_heatmaps_pad[index]
+ ix1 = batch_heatmaps_pad[index + 1]
+ iy1 = batch_heatmaps_pad[index + W + 2]
+ ix1y1 = batch_heatmaps_pad[index + W + 3]
+ ix1_y1_ = batch_heatmaps_pad[index - W - 3]
+ ix1_ = batch_heatmaps_pad[index - 1]
+ iy1_ = batch_heatmaps_pad[index - 2 - W]
+
+ dx = 0.5 * (ix1 - ix1_)
+ dy = 0.5 * (iy1 - iy1_)
+ derivative = np.concatenate([dx, dy], axis=1)
+ derivative = derivative.reshape(N, K, 2, 1)
+ dxx = ix1 - 2 * i_ + ix1_
+ dyy = iy1 - 2 * i_ + iy1_
+ dxy = 0.5 * (ix1y1 - ix1 - iy1 + i_ + i_ - ix1_ - iy1_ + ix1_y1_)
+ hessian = np.concatenate([dxx, dxy, dxy, dyy], axis=1)
+ hessian = hessian.reshape(N, K, 2, 2)
+ hessian = np.linalg.inv(hessian + np.finfo(np.float32).eps * np.eye(2))
+ coords -= np.einsum('ijmn,ijnk->ijmk', hessian, derivative).squeeze()
+ return coords
+
+
+def _gaussian_blur(heatmaps, kernel=11):
+ """Modulate heatmap distribution with Gaussian.
+ sigma = 0.3*((kernel_size-1)*0.5-1)+0.8
+ sigma~=3 if k=17
+ sigma=2 if k=11;
+ sigma~=1.5 if k=7;
+ sigma~=1 if k=3;
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+ - heatmap height: H
+ - heatmap width: W
+
+ Args:
+ heatmaps (np.ndarray[N, K, H, W]): model predicted heatmaps.
+ kernel (int): Gaussian kernel size (K) for modulation, which should
+ match the heatmap gaussian sigma when training.
+ K=17 for sigma=3 and k=11 for sigma=2.
+
+ Returns:
+ np.ndarray ([N, K, H, W]): Modulated heatmap distribution.
+ """
+ assert kernel % 2 == 1
+
+ border = (kernel - 1) // 2
+ batch_size = heatmaps.shape[0]
+ num_joints = heatmaps.shape[1]
+ height = heatmaps.shape[2]
+ width = heatmaps.shape[3]
+ for i in range(batch_size):
+ for j in range(num_joints):
+ origin_max = np.max(heatmaps[i, j])
+ dr = np.zeros((height + 2 * border, width + 2 * border),
+ dtype=np.float32)
+ dr[border:-border, border:-border] = heatmaps[i, j].copy()
+ dr = cv2.GaussianBlur(dr, (kernel, kernel), 0)
+ heatmaps[i, j] = dr[border:-border, border:-border].copy()
+ heatmaps[i, j] *= origin_max / np.max(heatmaps[i, j])
+ return heatmaps
+
+
+def keypoints_from_regression(regression_preds, center, scale, img_size):
+ """Get final keypoint predictions from regression vectors and transform
+ them back to the image.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+
+ Args:
+ regression_preds (np.ndarray[N, K, 2]): model prediction.
+ center (np.ndarray[N, 2]): Center of the bounding box (x, y).
+ scale (np.ndarray[N, 2]): Scale of the bounding box
+ wrt height/width.
+ img_size (list(img_width, img_height)): model input image size.
+
+ Returns:
+ tuple:
+
+ - preds (np.ndarray[N, K, 2]): Predicted keypoint location in images.
+ - maxvals (np.ndarray[N, K, 1]): Scores (confidence) of the keypoints.
+ """
+ N, K, _ = regression_preds.shape
+ preds, maxvals = regression_preds, np.ones((N, K, 1), dtype=np.float32)
+
+ preds = preds * img_size
+
+ # Transform back to the image
+ for i in range(N):
+ preds[i] = transform_preds(preds[i], center[i], scale[i], img_size)
+
+ return preds, maxvals
+
+
+def keypoints_from_heatmaps(heatmaps,
+ center,
+ scale,
+ unbiased=False,
+ post_process='default',
+ kernel=11,
+ valid_radius_factor=0.0546875,
+ use_udp=False,
+ target_type='GaussianHeatmap'):
+ """Get final keypoint predictions from heatmaps and transform them back to
+ the image.
+
+ Note:
+ - batch size: N
+ - num keypoints: K
+ - heatmap height: H
+ - heatmap width: W
+
+ Args:
+ heatmaps (np.ndarray[N, K, H, W]): model predicted heatmaps.
+ center (np.ndarray[N, 2]): Center of the bounding box (x, y).
+ scale (np.ndarray[N, 2]): Scale of the bounding box
+ wrt height/width.
+ post_process (str/None): Choice of methods to post-process
+ heatmaps. Currently supported: None, 'default', 'unbiased',
+ 'megvii'.
+ unbiased (bool): Option to use unbiased decoding. Mutually
+ exclusive with megvii.
+ Note: this arg is deprecated and unbiased=True can be replaced
+ by post_process='unbiased'
+ Paper ref: Zhang et al. Distribution-Aware Coordinate
+ Representation for Human Pose Estimation (CVPR 2020).
+ kernel (int): Gaussian kernel size (K) for modulation, which should
+ match the heatmap gaussian sigma when training.
+ K=17 for sigma=3 and k=11 for sigma=2.
+ valid_radius_factor (float): The radius factor of the positive area
+ in classification heatmap for UDP.
+ use_udp (bool): Use unbiased data processing.
+ target_type (str): 'GaussianHeatmap' or 'CombinedTarget'.
+ GaussianHeatmap: Classification target with gaussian distribution.
+ CombinedTarget: The combination of classification target
+ (response map) and regression target (offset map).
+ Paper ref: Huang et al. The Devil is in the Details: Delving into
+ Unbiased Data Processing for Human Pose Estimation (CVPR 2020).
+
+ Returns:
+ tuple: A tuple containing keypoint predictions and scores.
+
+ - preds (np.ndarray[N, K, 2]): Predicted keypoint location in images.
+ - maxvals (np.ndarray[N, K, 1]): Scores (confidence) of the keypoints.
+ """
+ # Avoid being affected
+ heatmaps = heatmaps.copy()
+
+ # detect conflicts
+ if unbiased:
+ assert post_process not in [False, None, 'megvii']
+ if post_process in ['megvii', 'unbiased']:
+ assert kernel > 0
+ if use_udp:
+ assert not post_process == 'megvii'
+
+ # normalize configs
+ if post_process is False:
+ warnings.warn(
+ 'post_process=False is deprecated, '
+ 'please use post_process=None instead', DeprecationWarning)
+ post_process = None
+ elif post_process is True:
+ if unbiased is True:
+ warnings.warn(
+ 'post_process=True, unbiased=True is deprecated,'
+ " please use post_process='unbiased' instead",
+ DeprecationWarning)
+ post_process = 'unbiased'
+ else:
+ warnings.warn(
+ 'post_process=True, unbiased=False is deprecated, '
+ "please use post_process='default' instead",
+ DeprecationWarning)
+ post_process = 'default'
+ elif post_process == 'default':
+ if unbiased is True:
+ warnings.warn(
+ 'unbiased=True is deprecated, please use '
+ "post_process='unbiased' instead", DeprecationWarning)
+ post_process = 'unbiased'
+
+ # start processing
+ if post_process == 'megvii':
+ heatmaps = _gaussian_blur(heatmaps, kernel=kernel)
+
+ N, K, H, W = heatmaps.shape
+ if use_udp:
+ if target_type.lower() == 'GaussianHeatMap'.lower():
+ preds, maxvals = _get_max_preds(heatmaps)
+ preds = post_dark_udp(preds, heatmaps, kernel=kernel)
+ elif target_type.lower() == 'CombinedTarget'.lower():
+ for person_heatmaps in heatmaps:
+ for i, heatmap in enumerate(person_heatmaps):
+ kt = 2 * kernel + 1 if i % 3 == 0 else kernel
+ cv2.GaussianBlur(heatmap, (kt, kt), 0, heatmap)
+ # valid radius is in direct proportion to the height of heatmap.
+ valid_radius = valid_radius_factor * H
+ offset_x = heatmaps[:, 1::3, :].flatten() * valid_radius
+ offset_y = heatmaps[:, 2::3, :].flatten() * valid_radius
+ heatmaps = heatmaps[:, ::3, :]
+ preds, maxvals = _get_max_preds(heatmaps)
+ index = preds[..., 0] + preds[..., 1] * W
+ index += W * H * np.arange(0, N * K / 3)
+ index = index.astype(int).reshape(N, K // 3, 1)
+ preds += np.concatenate((offset_x[index], offset_y[index]), axis=2)
+ else:
+ raise ValueError('target_type should be either '
+ "'GaussianHeatmap' or 'CombinedTarget'")
+ else:
+ preds, maxvals = _get_max_preds(heatmaps)
+ if post_process == 'unbiased': # alleviate biased coordinate
+ # apply Gaussian distribution modulation.
+ heatmaps = np.log(
+ np.maximum(_gaussian_blur(heatmaps, kernel), 1e-10))
+ for n in range(N):
+ for k in range(K):
+ preds[n][k] = _taylor(heatmaps[n][k], preds[n][k])
+ elif post_process is not None:
+ # add +/-0.25 shift to the predicted locations for higher acc.
+ for n in range(N):
+ for k in range(K):
+ heatmap = heatmaps[n][k]
+ px = int(preds[n][k][0])
+ py = int(preds[n][k][1])
+ if 1 < px < W - 1 and 1 < py < H - 1:
+ diff = np.array([
+ heatmap[py][px + 1] - heatmap[py][px - 1],
+ heatmap[py + 1][px] - heatmap[py - 1][px]
+ ])
+ preds[n][k] += np.sign(diff) * .25
+ if post_process == 'megvii':
+ preds[n][k] += 0.5
+
+ # Transform back to the image
+ for i in range(N):
+ preds[i] = transform_preds(
+ preds[i], center[i], scale[i], [W, H], use_udp=use_udp)
+
+ if post_process == 'megvii':
+ maxvals = maxvals / 255.0 + 0.5
+
+ return preds, maxvals
+
+
+def keypoints_from_heatmaps3d(heatmaps, center, scale):
+ """Get final keypoint predictions from 3d heatmaps and transform them back
+ to the image.
+
+ Note:
+ - batch size: N
+ - num keypoints: K
+ - heatmap depth size: D
+ - heatmap height: H
+ - heatmap width: W
+
+ Args:
+ heatmaps (np.ndarray[N, K, D, H, W]): model predicted heatmaps.
+ center (np.ndarray[N, 2]): Center of the bounding box (x, y).
+ scale (np.ndarray[N, 2]): Scale of the bounding box
+ wrt height/width.
+
+ Returns:
+ tuple: A tuple containing keypoint predictions and scores.
+
+ - preds (np.ndarray[N, K, 3]): Predicted 3d keypoint location \
+ in images.
+ - maxvals (np.ndarray[N, K, 1]): Scores (confidence) of the keypoints.
+ """
+ N, K, D, H, W = heatmaps.shape
+ preds, maxvals = _get_max_preds_3d(heatmaps)
+ # Transform back to the image
+ for i in range(N):
+ preds[i, :, :2] = transform_preds(preds[i, :, :2], center[i], scale[i],
+ [W, H])
+ return preds, maxvals
+
+
+def multilabel_classification_accuracy(pred, gt, mask, thr=0.5):
+ """Get multi-label classification accuracy.
+
+ Note:
+ - batch size: N
+ - label number: L
+
+ Args:
+ pred (np.ndarray[N, L, 2]): model predicted labels.
+ gt (np.ndarray[N, L, 2]): ground-truth labels.
+ mask (np.ndarray[N, 1] or np.ndarray[N, L] ): reliability of
+ ground-truth labels.
+
+ Returns:
+ float: multi-label classification accuracy.
+ """
+ # we only compute accuracy on the samples with ground-truth of all labels.
+ valid = (mask > 0).min(axis=1) if mask.ndim == 2 else (mask > 0)
+ pred, gt = pred[valid], gt[valid]
+
+ if pred.shape[0] == 0:
+ acc = 0.0 # when no sample is with gt labels, set acc to 0.
+ else:
+ # The classification of a sample is regarded as correct
+ # only if it's correct for all labels.
+ acc = (((pred - thr) * (gt - thr)) > 0).all(axis=1).mean()
+ return acc
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/core/fp16/__init__.py b/engine/pose_estimation/third-party/ViTPose/mmpose/core/fp16/__init__.py
new file mode 100644
index 0000000..5cb0548
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/core/fp16/__init__.py
@@ -0,0 +1,9 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from .decorators import auto_fp16, force_fp32
+from .hooks import Fp16OptimizerHook, wrap_fp16_model
+from .utils import cast_tensor_type
+
+__all__ = [
+ 'auto_fp16', 'force_fp32', 'Fp16OptimizerHook', 'wrap_fp16_model',
+ 'cast_tensor_type'
+]
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/core/fp16/decorators.py b/engine/pose_estimation/third-party/ViTPose/mmpose/core/fp16/decorators.py
new file mode 100644
index 0000000..2d70ddf
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/core/fp16/decorators.py
@@ -0,0 +1,175 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import functools
+import warnings
+from inspect import getfullargspec
+
+import torch
+
+from .utils import cast_tensor_type
+
+
+def auto_fp16(apply_to=None, out_fp32=False):
+ """Decorator to enable fp16 training automatically.
+
+ This decorator is useful when you write custom modules and want to support
+ mixed precision training. If inputs arguments are fp32 tensors, they will
+ be converted to fp16 automatically. Arguments other than fp32 tensors are
+ ignored.
+
+ Args:
+ apply_to (Iterable, optional): The argument names to be converted.
+ `None` indicates all arguments.
+ out_fp32 (bool): Whether to convert the output back to fp32.
+
+ Example:
+
+ >>> import torch.nn as nn
+ >>> class MyModule1(nn.Module):
+ >>>
+ >>> # Convert x and y to fp16
+ >>> @auto_fp16()
+ >>> def forward(self, x, y):
+ >>> pass
+
+ >>> import torch.nn as nn
+ >>> class MyModule2(nn.Module):
+ >>>
+ >>> # convert pred to fp16
+ >>> @auto_fp16(apply_to=('pred', ))
+ >>> def do_something(self, pred, others):
+ >>> pass
+ """
+
+ warnings.warn(
+ 'auto_fp16 in mmpose will be deprecated in the next release.'
+ 'Please use mmcv.runner.auto_fp16 instead (mmcv>=1.3.1).',
+ DeprecationWarning)
+
+ def auto_fp16_wrapper(old_func):
+
+ @functools.wraps(old_func)
+ def new_func(*args, **kwargs):
+ # check if the module has set the attribute `fp16_enabled`, if not,
+ # just fallback to the original method.
+ if not isinstance(args[0], torch.nn.Module):
+ raise TypeError('@auto_fp16 can only be used to decorate the '
+ 'method of nn.Module')
+ if not (hasattr(args[0], 'fp16_enabled') and args[0].fp16_enabled):
+ return old_func(*args, **kwargs)
+ # get the arg spec of the decorated method
+ args_info = getfullargspec(old_func)
+ # get the argument names to be casted
+ args_to_cast = args_info.args if apply_to is None else apply_to
+ # convert the args that need to be processed
+ new_args = []
+ # NOTE: default args are not taken into consideration
+ if args:
+ arg_names = args_info.args[:len(args)]
+ for i, arg_name in enumerate(arg_names):
+ if arg_name in args_to_cast:
+ new_args.append(
+ cast_tensor_type(args[i], torch.float, torch.half))
+ else:
+ new_args.append(args[i])
+ # convert the kwargs that need to be processed
+ new_kwargs = {}
+ if kwargs:
+ for arg_name, arg_value in kwargs.items():
+ if arg_name in args_to_cast:
+ new_kwargs[arg_name] = cast_tensor_type(
+ arg_value, torch.float, torch.half)
+ else:
+ new_kwargs[arg_name] = arg_value
+ # apply converted arguments to the decorated method
+ output = old_func(*new_args, **new_kwargs)
+ # cast the results back to fp32 if necessary
+ if out_fp32:
+ output = cast_tensor_type(output, torch.half, torch.float)
+ return output
+
+ return new_func
+
+ return auto_fp16_wrapper
+
+
+def force_fp32(apply_to=None, out_fp16=False):
+ """Decorator to convert input arguments to fp32 in force.
+
+ This decorator is useful when you write custom modules and want to support
+ mixed precision training. If there are some inputs that must be processed
+ in fp32 mode, then this decorator can handle it. If inputs arguments are
+ fp16 tensors, they will be converted to fp32 automatically. Arguments other
+ than fp16 tensors are ignored.
+
+ Args:
+ apply_to (Iterable, optional): The argument names to be converted.
+ `None` indicates all arguments.
+ out_fp16 (bool): Whether to convert the output back to fp16.
+
+ Example:
+
+ >>> import torch.nn as nn
+ >>> class MyModule1(nn.Module):
+ >>>
+ >>> # Convert x and y to fp32
+ >>> @force_fp32()
+ >>> def loss(self, x, y):
+ >>> pass
+
+ >>> import torch.nn as nn
+ >>> class MyModule2(nn.Module):
+ >>>
+ >>> # convert pred to fp32
+ >>> @force_fp32(apply_to=('pred', ))
+ >>> def post_process(self, pred, others):
+ >>> pass
+ """
+ warnings.warn(
+ 'force_fp32 in mmpose will be deprecated in the next release.'
+ 'Please use mmcv.runner.force_fp32 instead (mmcv>=1.3.1).',
+ DeprecationWarning)
+
+ def force_fp32_wrapper(old_func):
+
+ @functools.wraps(old_func)
+ def new_func(*args, **kwargs):
+ # check if the module has set the attribute `fp16_enabled`, if not,
+ # just fallback to the original method.
+ if not isinstance(args[0], torch.nn.Module):
+ raise TypeError('@force_fp32 can only be used to decorate the '
+ 'method of nn.Module')
+ if not (hasattr(args[0], 'fp16_enabled') and args[0].fp16_enabled):
+ return old_func(*args, **kwargs)
+ # get the arg spec of the decorated method
+ args_info = getfullargspec(old_func)
+ # get the argument names to be casted
+ args_to_cast = args_info.args if apply_to is None else apply_to
+ # convert the args that need to be processed
+ new_args = []
+ if args:
+ arg_names = args_info.args[:len(args)]
+ for i, arg_name in enumerate(arg_names):
+ if arg_name in args_to_cast:
+ new_args.append(
+ cast_tensor_type(args[i], torch.half, torch.float))
+ else:
+ new_args.append(args[i])
+ # convert the kwargs that need to be processed
+ new_kwargs = dict()
+ if kwargs:
+ for arg_name, arg_value in kwargs.items():
+ if arg_name in args_to_cast:
+ new_kwargs[arg_name] = cast_tensor_type(
+ arg_value, torch.half, torch.float)
+ else:
+ new_kwargs[arg_name] = arg_value
+ # apply converted arguments to the decorated method
+ output = old_func(*new_args, **new_kwargs)
+ # cast the results back to fp32 if necessary
+ if out_fp16:
+ output = cast_tensor_type(output, torch.float, torch.half)
+ return output
+
+ return new_func
+
+ return force_fp32_wrapper
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/core/fp16/hooks.py b/engine/pose_estimation/third-party/ViTPose/mmpose/core/fp16/hooks.py
new file mode 100644
index 0000000..74081a9
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/core/fp16/hooks.py
@@ -0,0 +1,167 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import copy
+
+import torch
+import torch.nn as nn
+from mmcv.runner import OptimizerHook
+from mmcv.utils import _BatchNorm
+
+from ..utils.dist_utils import allreduce_grads
+from .utils import cast_tensor_type
+
+
+class Fp16OptimizerHook(OptimizerHook):
+ """FP16 optimizer hook.
+
+ The steps of fp16 optimizer is as follows.
+ 1. Scale the loss value.
+ 2. BP in the fp16 model.
+ 2. Copy gradients from fp16 model to fp32 weights.
+ 3. Update fp32 weights.
+ 4. Copy updated parameters from fp32 weights to fp16 model.
+
+ Refer to https://arxiv.org/abs/1710.03740 for more details.
+
+ Args:
+ loss_scale (float): Scale factor multiplied with loss.
+ """
+
+ def __init__(self,
+ grad_clip=None,
+ coalesce=True,
+ bucket_size_mb=-1,
+ loss_scale=512.,
+ distributed=True):
+ self.grad_clip = grad_clip
+ self.coalesce = coalesce
+ self.bucket_size_mb = bucket_size_mb
+ self.loss_scale = loss_scale
+ self.distributed = distributed
+
+ def before_run(self, runner):
+ """Preparing steps before Mixed Precision Training.
+
+ 1. Make a master copy of fp32 weights for optimization.
+ 2. Convert the main model from fp32 to fp16.
+
+ Args:
+ runner (:obj:`mmcv.Runner`): The underlines training runner.
+ """
+ # keep a copy of fp32 weights
+ runner.optimizer.param_groups = copy.deepcopy(
+ runner.optimizer.param_groups)
+ # convert model to fp16
+ wrap_fp16_model(runner.model)
+
+ @staticmethod
+ def copy_grads_to_fp32(fp16_net, fp32_weights):
+ """Copy gradients from fp16 model to fp32 weight copy."""
+ for fp32_param, fp16_param in zip(fp32_weights, fp16_net.parameters()):
+ if fp16_param.grad is not None:
+ if fp32_param.grad is None:
+ fp32_param.grad = fp32_param.data.new(fp32_param.size())
+ fp32_param.grad.copy_(fp16_param.grad)
+
+ @staticmethod
+ def copy_params_to_fp16(fp16_net, fp32_weights):
+ """Copy updated params from fp32 weight copy to fp16 model."""
+ for fp16_param, fp32_param in zip(fp16_net.parameters(), fp32_weights):
+ fp16_param.data.copy_(fp32_param.data)
+
+ def after_train_iter(self, runner):
+ """Backward optimization steps for Mixed Precision Training.
+
+ 1. Scale the loss by a scale factor.
+ 2. Backward the loss to obtain the gradients (fp16).
+ 3. Copy gradients from the model to the fp32 weight copy.
+ 4. Scale the gradients back and update the fp32 weight copy.
+ 5. Copy back the params from fp32 weight copy to the fp16 model.
+
+ Args:
+ runner (:obj:`mmcv.Runner`): The underlines training runner.
+ """
+ # clear grads of last iteration
+ runner.model.zero_grad()
+ runner.optimizer.zero_grad()
+ # scale the loss value
+ scaled_loss = runner.outputs['loss'] * self.loss_scale
+ scaled_loss.backward()
+ # copy fp16 grads in the model to fp32 params in the optimizer
+ fp32_weights = []
+ for param_group in runner.optimizer.param_groups:
+ fp32_weights += param_group['params']
+ self.copy_grads_to_fp32(runner.model, fp32_weights)
+ # allreduce grads
+ if self.distributed:
+ allreduce_grads(fp32_weights, self.coalesce, self.bucket_size_mb)
+ # scale the gradients back
+ for param in fp32_weights:
+ if param.grad is not None:
+ param.grad.div_(self.loss_scale)
+ if self.grad_clip is not None:
+ self.clip_grads(fp32_weights)
+ # update fp32 params
+ runner.optimizer.step()
+ # copy fp32 params to the fp16 model
+ self.copy_params_to_fp16(runner.model, fp32_weights)
+
+
+def wrap_fp16_model(model):
+ """Wrap the FP32 model to FP16.
+
+ 1. Convert FP32 model to FP16.
+ 2. Remain some necessary layers to be FP32, e.g., normalization layers.
+
+ Args:
+ model (nn.Module): Model in FP32.
+ """
+ # convert model to fp16
+ model.half()
+ # patch the normalization layers to make it work in fp32 mode
+ patch_norm_fp32(model)
+ # set `fp16_enabled` flag
+ for m in model.modules():
+ if hasattr(m, 'fp16_enabled'):
+ m.fp16_enabled = True
+
+
+def patch_norm_fp32(module):
+ """Recursively convert normalization layers from FP16 to FP32.
+
+ Args:
+ module (nn.Module): The modules to be converted in FP16.
+
+ Returns:
+ nn.Module: The converted module, the normalization layers have been
+ converted to FP32.
+ """
+ if isinstance(module, (_BatchNorm, nn.GroupNorm)):
+ module.float()
+ module.forward = patch_forward_method(module.forward, torch.half,
+ torch.float)
+ for child in module.children():
+ patch_norm_fp32(child)
+ return module
+
+
+def patch_forward_method(func, src_type, dst_type, convert_output=True):
+ """Patch the forward method of a module.
+
+ Args:
+ func (callable): The original forward method.
+ src_type (torch.dtype): Type of input arguments to be converted from.
+ dst_type (torch.dtype): Type of input arguments to be converted to.
+ convert_output (bool): Whether to convert the output back to src_type.
+
+ Returns:
+ callable: The patched forward method.
+ """
+
+ def new_forward(*args, **kwargs):
+ output = func(*cast_tensor_type(args, src_type, dst_type),
+ **cast_tensor_type(kwargs, src_type, dst_type))
+ if convert_output:
+ output = cast_tensor_type(output, dst_type, src_type)
+ return output
+
+ return new_forward
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/core/fp16/utils.py b/engine/pose_estimation/third-party/ViTPose/mmpose/core/fp16/utils.py
new file mode 100644
index 0000000..f1ec3d3
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/core/fp16/utils.py
@@ -0,0 +1,34 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from collections import abc
+
+import numpy as np
+import torch
+
+
+def cast_tensor_type(inputs, src_type, dst_type):
+ """Recursively convert Tensor in inputs from src_type to dst_type.
+
+ Args:
+ inputs: Inputs that to be casted.
+ src_type (torch.dtype): Source type.
+ dst_type (torch.dtype): Destination type.
+
+ Returns:
+ The same type with inputs, but all contained Tensors have been cast.
+ """
+ if isinstance(inputs, torch.Tensor):
+ return inputs.to(dst_type)
+ elif isinstance(inputs, str):
+ return inputs
+ elif isinstance(inputs, np.ndarray):
+ return inputs
+ elif isinstance(inputs, abc.Mapping):
+ return type(inputs)({
+ k: cast_tensor_type(v, src_type, dst_type)
+ for k, v in inputs.items()
+ })
+ elif isinstance(inputs, abc.Iterable):
+ return type(inputs)(
+ cast_tensor_type(item, src_type, dst_type) for item in inputs)
+
+ return inputs
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/core/optimizer/__init__.py b/engine/pose_estimation/third-party/ViTPose/mmpose/core/optimizer/__init__.py
new file mode 100644
index 0000000..4340ffc
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/core/optimizer/__init__.py
@@ -0,0 +1,4 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from .builder import OPTIMIZERS, build_optimizers
+
+__all__ = ['build_optimizers', 'OPTIMIZERS']
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/core/optimizer/builder.py b/engine/pose_estimation/third-party/ViTPose/mmpose/core/optimizer/builder.py
new file mode 100644
index 0000000..7d6accd
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/core/optimizer/builder.py
@@ -0,0 +1,56 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from mmcv.runner import build_optimizer
+from mmcv.utils import Registry
+
+OPTIMIZERS = Registry('optimizers')
+
+
+def build_optimizers(model, cfgs):
+ """Build multiple optimizers from configs.
+
+ If `cfgs` contains several dicts for optimizers, then a dict for each
+ constructed optimizers will be returned.
+ If `cfgs` only contains one optimizer config, the constructed optimizer
+ itself will be returned.
+
+ For example,
+
+ 1) Multiple optimizer configs:
+
+ .. code-block:: python
+
+ optimizer_cfg = dict(
+ model1=dict(type='SGD', lr=lr),
+ model2=dict(type='SGD', lr=lr))
+
+ The return dict is
+ ``dict('model1': torch.optim.Optimizer, 'model2': torch.optim.Optimizer)``
+
+ 2) Single optimizer config:
+
+ .. code-block:: python
+
+ optimizer_cfg = dict(type='SGD', lr=lr)
+
+ The return is ``torch.optim.Optimizer``.
+
+ Args:
+ model (:obj:`nn.Module`): The model with parameters to be optimized.
+ cfgs (dict): The config dict of the optimizer.
+
+ Returns:
+ dict[:obj:`torch.optim.Optimizer`] | :obj:`torch.optim.Optimizer`:
+ The initialized optimizers.
+ """
+ optimizers = {}
+ if hasattr(model, 'module'):
+ model = model.module
+ # determine whether 'cfgs' has several dicts for optimizers
+ if all(isinstance(v, dict) for v in cfgs.values()):
+ for key, cfg in cfgs.items():
+ cfg_ = cfg.copy()
+ module = getattr(model, key)
+ optimizers[key] = build_optimizer(module, cfg_)
+ return optimizers
+
+ return build_optimizer(model, cfgs)
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/core/post_processing/__init__.py b/engine/pose_estimation/third-party/ViTPose/mmpose/core/post_processing/__init__.py
new file mode 100644
index 0000000..1ee6858
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/core/post_processing/__init__.py
@@ -0,0 +1,14 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from .nms import oks_iou, oks_nms, soft_oks_nms
+from .one_euro_filter import OneEuroFilter
+from .post_transforms import (affine_transform, flip_back, fliplr_joints,
+ fliplr_regression, get_affine_transform,
+ get_warp_matrix, rotate_point, transform_preds,
+ warp_affine_joints)
+
+__all__ = [
+ 'oks_nms', 'soft_oks_nms', 'affine_transform', 'rotate_point', 'flip_back',
+ 'fliplr_joints', 'fliplr_regression', 'transform_preds',
+ 'get_affine_transform', 'get_warp_matrix', 'warp_affine_joints',
+ 'OneEuroFilter', 'oks_iou'
+]
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/core/post_processing/group.py b/engine/pose_estimation/third-party/ViTPose/mmpose/core/post_processing/group.py
new file mode 100644
index 0000000..6235dbc
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/core/post_processing/group.py
@@ -0,0 +1,410 @@
+# ------------------------------------------------------------------------------
+# Adapted from https://github.com/princeton-vl/pose-ae-train/
+# Original licence: Copyright (c) 2017, umich-vl, under BSD 3-Clause License.
+# ------------------------------------------------------------------------------
+
+import numpy as np
+import torch
+from munkres import Munkres
+
+from mmpose.core.evaluation import post_dark_udp
+
+
+def _py_max_match(scores):
+ """Apply munkres algorithm to get the best match.
+
+ Args:
+ scores(np.ndarray): cost matrix.
+
+ Returns:
+ np.ndarray: best match.
+ """
+ m = Munkres()
+ tmp = m.compute(scores)
+ tmp = np.array(tmp).astype(int)
+ return tmp
+
+
+def _match_by_tag(inp, params):
+ """Match joints by tags. Use Munkres algorithm to calculate the best match
+ for keypoints grouping.
+
+ Note:
+ number of keypoints: K
+ max number of people in an image: M (M=30 by default)
+ dim of tags: L
+ If use flip testing, L=2; else L=1.
+
+ Args:
+ inp(tuple):
+ tag_k (np.ndarray[KxMxL]): tag corresponding to the
+ top k values of feature map per keypoint.
+ loc_k (np.ndarray[KxMx2]): top k locations of the
+ feature maps for keypoint.
+ val_k (np.ndarray[KxM]): top k value of the
+ feature maps per keypoint.
+ params(Params): class Params().
+
+ Returns:
+ np.ndarray: result of pose groups.
+ """
+ assert isinstance(params, _Params), 'params should be class _Params()'
+
+ tag_k, loc_k, val_k = inp
+
+ default_ = np.zeros((params.num_joints, 3 + tag_k.shape[2]),
+ dtype=np.float32)
+
+ joint_dict = {}
+ tag_dict = {}
+ for i in range(params.num_joints):
+ idx = params.joint_order[i]
+
+ tags = tag_k[idx]
+ joints = np.concatenate((loc_k[idx], val_k[idx, :, None], tags), 1)
+ mask = joints[:, 2] > params.detection_threshold
+ tags = tags[mask]
+ joints = joints[mask]
+
+ if joints.shape[0] == 0:
+ continue
+
+ if i == 0 or len(joint_dict) == 0:
+ for tag, joint in zip(tags, joints):
+ key = tag[0]
+ joint_dict.setdefault(key, np.copy(default_))[idx] = joint
+ tag_dict[key] = [tag]
+ else:
+ grouped_keys = list(joint_dict.keys())[:params.max_num_people]
+ grouped_tags = [np.mean(tag_dict[i], axis=0) for i in grouped_keys]
+
+ if (params.ignore_too_much
+ and len(grouped_keys) == params.max_num_people):
+ continue
+
+ diff = joints[:, None, 3:] - np.array(grouped_tags)[None, :, :]
+ diff_normed = np.linalg.norm(diff, ord=2, axis=2)
+ diff_saved = np.copy(diff_normed)
+
+ if params.use_detection_val:
+ diff_normed = np.round(diff_normed) * 100 - joints[:, 2:3]
+
+ num_added = diff.shape[0]
+ num_grouped = diff.shape[1]
+
+ if num_added > num_grouped:
+ diff_normed = np.concatenate(
+ (diff_normed,
+ np.zeros((num_added, num_added - num_grouped),
+ dtype=np.float32) + 1e10),
+ axis=1)
+
+ pairs = _py_max_match(diff_normed)
+ for row, col in pairs:
+ if (row < num_added and col < num_grouped
+ and diff_saved[row][col] < params.tag_threshold):
+ key = grouped_keys[col]
+ joint_dict[key][idx] = joints[row]
+ tag_dict[key].append(tags[row])
+ else:
+ key = tags[row][0]
+ joint_dict.setdefault(key, np.copy(default_))[idx] = \
+ joints[row]
+ tag_dict[key] = [tags[row]]
+
+ results = np.array([joint_dict[i] for i in joint_dict]).astype(np.float32)
+ return results
+
+
+class _Params:
+ """A class of parameter.
+
+ Args:
+ cfg(Config): config.
+ """
+
+ def __init__(self, cfg):
+ self.num_joints = cfg['num_joints']
+ self.max_num_people = cfg['max_num_people']
+
+ self.detection_threshold = cfg['detection_threshold']
+ self.tag_threshold = cfg['tag_threshold']
+ self.use_detection_val = cfg['use_detection_val']
+ self.ignore_too_much = cfg['ignore_too_much']
+
+ if self.num_joints == 17:
+ self.joint_order = [
+ i - 1 for i in
+ [1, 2, 3, 4, 5, 6, 7, 12, 13, 8, 9, 10, 11, 14, 15, 16, 17]
+ ]
+ else:
+ self.joint_order = list(np.arange(self.num_joints))
+
+
+class HeatmapParser:
+ """The heatmap parser for post processing."""
+
+ def __init__(self, cfg):
+ self.params = _Params(cfg)
+ self.tag_per_joint = cfg['tag_per_joint']
+ self.pool = torch.nn.MaxPool2d(cfg['nms_kernel'], 1,
+ cfg['nms_padding'])
+ self.use_udp = cfg.get('use_udp', False)
+ self.score_per_joint = cfg.get('score_per_joint', False)
+
+ def nms(self, heatmaps):
+ """Non-Maximum Suppression for heatmaps.
+
+ Args:
+ heatmap(torch.Tensor): Heatmaps before nms.
+
+ Returns:
+ torch.Tensor: Heatmaps after nms.
+ """
+
+ maxm = self.pool(heatmaps)
+ maxm = torch.eq(maxm, heatmaps).float()
+ heatmaps = heatmaps * maxm
+
+ return heatmaps
+
+ def match(self, tag_k, loc_k, val_k):
+ """Group keypoints to human poses in a batch.
+
+ Args:
+ tag_k (np.ndarray[NxKxMxL]): tag corresponding to the
+ top k values of feature map per keypoint.
+ loc_k (np.ndarray[NxKxMx2]): top k locations of the
+ feature maps for keypoint.
+ val_k (np.ndarray[NxKxM]): top k value of the
+ feature maps per keypoint.
+
+ Returns:
+ list
+ """
+
+ def _match(x):
+ return _match_by_tag(x, self.params)
+
+ return list(map(_match, zip(tag_k, loc_k, val_k)))
+
+ def top_k(self, heatmaps, tags):
+ """Find top_k values in an image.
+
+ Note:
+ batch size: N
+ number of keypoints: K
+ heatmap height: H
+ heatmap width: W
+ max number of people: M
+ dim of tags: L
+ If use flip testing, L=2; else L=1.
+
+ Args:
+ heatmaps (torch.Tensor[NxKxHxW])
+ tags (torch.Tensor[NxKxHxWxL])
+
+ Returns:
+ dict: A dict containing top_k values.
+
+ - tag_k (np.ndarray[NxKxMxL]):
+ tag corresponding to the top k values of
+ feature map per keypoint.
+ - loc_k (np.ndarray[NxKxMx2]):
+ top k location of feature map per keypoint.
+ - val_k (np.ndarray[NxKxM]):
+ top k value of feature map per keypoint.
+ """
+ heatmaps = self.nms(heatmaps)
+ N, K, H, W = heatmaps.size()
+ heatmaps = heatmaps.view(N, K, -1)
+ val_k, ind = heatmaps.topk(self.params.max_num_people, dim=2)
+
+ tags = tags.view(tags.size(0), tags.size(1), W * H, -1)
+ if not self.tag_per_joint:
+ tags = tags.expand(-1, self.params.num_joints, -1, -1)
+
+ tag_k = torch.stack(
+ [torch.gather(tags[..., i], 2, ind) for i in range(tags.size(3))],
+ dim=3)
+
+ x = ind % W
+ y = ind // W
+
+ ind_k = torch.stack((x, y), dim=3)
+
+ results = {
+ 'tag_k': tag_k.cpu().numpy(),
+ 'loc_k': ind_k.cpu().numpy(),
+ 'val_k': val_k.cpu().numpy()
+ }
+
+ return results
+
+ @staticmethod
+ def adjust(results, heatmaps):
+ """Adjust the coordinates for better accuracy.
+
+ Note:
+ batch size: N
+ number of keypoints: K
+ heatmap height: H
+ heatmap width: W
+
+ Args:
+ results (list(np.ndarray)): Keypoint predictions.
+ heatmaps (torch.Tensor[NxKxHxW]): Heatmaps.
+ """
+ _, _, H, W = heatmaps.shape
+ for batch_id, people in enumerate(results):
+ for people_id, people_i in enumerate(people):
+ for joint_id, joint in enumerate(people_i):
+ if joint[2] > 0:
+ x, y = joint[0:2]
+ xx, yy = int(x), int(y)
+ tmp = heatmaps[batch_id][joint_id]
+ if tmp[min(H - 1, yy + 1), xx] > tmp[max(0, yy - 1),
+ xx]:
+ y += 0.25
+ else:
+ y -= 0.25
+
+ if tmp[yy, min(W - 1, xx + 1)] > tmp[yy,
+ max(0, xx - 1)]:
+ x += 0.25
+ else:
+ x -= 0.25
+ results[batch_id][people_id, joint_id,
+ 0:2] = (x + 0.5, y + 0.5)
+ return results
+
+ @staticmethod
+ def refine(heatmap, tag, keypoints, use_udp=False):
+ """Given initial keypoint predictions, we identify missing joints.
+
+ Note:
+ number of keypoints: K
+ heatmap height: H
+ heatmap width: W
+ dim of tags: L
+ If use flip testing, L=2; else L=1.
+
+ Args:
+ heatmap: np.ndarray(K, H, W).
+ tag: np.ndarray(K, H, W) | np.ndarray(K, H, W, L)
+ keypoints: np.ndarray of size (K, 3 + L)
+ last dim is (x, y, score, tag).
+ use_udp: bool-unbiased data processing
+
+ Returns:
+ np.ndarray: The refined keypoints.
+ """
+
+ K, H, W = heatmap.shape
+ if len(tag.shape) == 3:
+ tag = tag[..., None]
+
+ tags = []
+ for i in range(K):
+ if keypoints[i, 2] > 0:
+ # save tag value of detected keypoint
+ x, y = keypoints[i][:2].astype(int)
+ x = np.clip(x, 0, W - 1)
+ y = np.clip(y, 0, H - 1)
+ tags.append(tag[i, y, x])
+
+ # mean tag of current detected people
+ prev_tag = np.mean(tags, axis=0)
+ results = []
+
+ for _heatmap, _tag in zip(heatmap, tag):
+ # distance of all tag values with mean tag of
+ # current detected people
+ distance_tag = (((_tag -
+ prev_tag[None, None, :])**2).sum(axis=2)**0.5)
+ norm_heatmap = _heatmap - np.round(distance_tag)
+
+ # find maximum position
+ y, x = np.unravel_index(np.argmax(norm_heatmap), _heatmap.shape)
+ xx = x.copy()
+ yy = y.copy()
+ # detection score at maximum position
+ val = _heatmap[y, x]
+ if not use_udp:
+ # offset by 0.5
+ x += 0.5
+ y += 0.5
+
+ # add a quarter offset
+ if _heatmap[yy, min(W - 1, xx + 1)] > _heatmap[yy, max(0, xx - 1)]:
+ x += 0.25
+ else:
+ x -= 0.25
+
+ if _heatmap[min(H - 1, yy + 1), xx] > _heatmap[max(0, yy - 1), xx]:
+ y += 0.25
+ else:
+ y -= 0.25
+
+ results.append((x, y, val))
+ results = np.array(results)
+
+ if results is not None:
+ for i in range(K):
+ # add keypoint if it is not detected
+ if results[i, 2] > 0 and keypoints[i, 2] == 0:
+ keypoints[i, :3] = results[i, :3]
+
+ return keypoints
+
+ def parse(self, heatmaps, tags, adjust=True, refine=True):
+ """Group keypoints into poses given heatmap and tag.
+
+ Note:
+ batch size: N
+ number of keypoints: K
+ heatmap height: H
+ heatmap width: W
+ dim of tags: L
+ If use flip testing, L=2; else L=1.
+
+ Args:
+ heatmaps (torch.Tensor[NxKxHxW]): model output heatmaps.
+ tags (torch.Tensor[NxKxHxWxL]): model output tagmaps.
+
+ Returns:
+ tuple: A tuple containing keypoint grouping results.
+
+ - results (list(np.ndarray)): Pose results.
+ - scores (list/list(np.ndarray)): Score of people.
+ """
+ results = self.match(**self.top_k(heatmaps, tags))
+
+ if adjust:
+ if self.use_udp:
+ for i in range(len(results)):
+ if results[i].shape[0] > 0:
+ results[i][..., :2] = post_dark_udp(
+ results[i][..., :2].copy(), heatmaps[i:i + 1, :])
+ else:
+ results = self.adjust(results, heatmaps)
+
+ if self.score_per_joint:
+ scores = [i[:, 2] for i in results[0]]
+ else:
+ scores = [i[:, 2].mean() for i in results[0]]
+
+ if refine:
+ results = results[0]
+ # for every detected person
+ for i in range(len(results)):
+ heatmap_numpy = heatmaps[0].cpu().numpy()
+ tag_numpy = tags[0].cpu().numpy()
+ if not self.tag_per_joint:
+ tag_numpy = np.tile(tag_numpy,
+ (self.params.num_joints, 1, 1, 1))
+ results[i] = self.refine(
+ heatmap_numpy, tag_numpy, results[i], use_udp=self.use_udp)
+ results = [results]
+
+ return results, scores
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/core/post_processing/nms.py b/engine/pose_estimation/third-party/ViTPose/mmpose/core/post_processing/nms.py
new file mode 100644
index 0000000..86a0ab3
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/core/post_processing/nms.py
@@ -0,0 +1,207 @@
+# ------------------------------------------------------------------------------
+# Adapted from https://github.com/leoxiaobin/deep-high-resolution-net.pytorch
+# Original licence: Copyright (c) Microsoft, under the MIT License.
+# ------------------------------------------------------------------------------
+
+import numpy as np
+
+
+def nms(dets, thr):
+ """Greedily select boxes with high confidence and overlap <= thr.
+
+ Args:
+ dets: [[x1, y1, x2, y2, score]].
+ thr: Retain overlap < thr.
+
+ Returns:
+ list: Indexes to keep.
+ """
+ if len(dets) == 0:
+ return []
+
+ x1 = dets[:, 0]
+ y1 = dets[:, 1]
+ x2 = dets[:, 2]
+ y2 = dets[:, 3]
+ scores = dets[:, 4]
+
+ areas = (x2 - x1 + 1) * (y2 - y1 + 1)
+ order = scores.argsort()[::-1]
+
+ keep = []
+ while len(order) > 0:
+ i = order[0]
+ keep.append(i)
+ xx1 = np.maximum(x1[i], x1[order[1:]])
+ yy1 = np.maximum(y1[i], y1[order[1:]])
+ xx2 = np.minimum(x2[i], x2[order[1:]])
+ yy2 = np.minimum(y2[i], y2[order[1:]])
+
+ w = np.maximum(0.0, xx2 - xx1 + 1)
+ h = np.maximum(0.0, yy2 - yy1 + 1)
+ inter = w * h
+ ovr = inter / (areas[i] + areas[order[1:]] - inter)
+
+ inds = np.where(ovr <= thr)[0]
+ order = order[inds + 1]
+
+ return keep
+
+
+def oks_iou(g, d, a_g, a_d, sigmas=None, vis_thr=None):
+ """Calculate oks ious.
+
+ Args:
+ g: Ground truth keypoints.
+ d: Detected keypoints.
+ a_g: Area of the ground truth object.
+ a_d: Area of the detected object.
+ sigmas: standard deviation of keypoint labelling.
+ vis_thr: threshold of the keypoint visibility.
+
+ Returns:
+ list: The oks ious.
+ """
+ if sigmas is None:
+ sigmas = np.array([
+ .26, .25, .25, .35, .35, .79, .79, .72, .72, .62, .62, 1.07, 1.07,
+ .87, .87, .89, .89
+ ]) / 10.0
+ vars = (sigmas * 2)**2
+ xg = g[0::3]
+ yg = g[1::3]
+ vg = g[2::3]
+ ious = np.zeros(len(d), dtype=np.float32)
+ for n_d in range(0, len(d)):
+ xd = d[n_d, 0::3]
+ yd = d[n_d, 1::3]
+ vd = d[n_d, 2::3]
+ dx = xd - xg
+ dy = yd - yg
+ e = (dx**2 + dy**2) / vars / ((a_g + a_d[n_d]) / 2 + np.spacing(1)) / 2
+ if vis_thr is not None:
+ ind = list(vg > vis_thr) and list(vd > vis_thr)
+ e = e[ind]
+ ious[n_d] = np.sum(np.exp(-e)) / len(e) if len(e) != 0 else 0.0
+ return ious
+
+
+def oks_nms(kpts_db, thr, sigmas=None, vis_thr=None, score_per_joint=False):
+ """OKS NMS implementations.
+
+ Args:
+ kpts_db: keypoints.
+ thr: Retain overlap < thr.
+ sigmas: standard deviation of keypoint labelling.
+ vis_thr: threshold of the keypoint visibility.
+ score_per_joint: the input scores (in kpts_db) are per joint scores
+
+ Returns:
+ np.ndarray: indexes to keep.
+ """
+ if len(kpts_db) == 0:
+ return []
+
+ if score_per_joint:
+ scores = np.array([k['score'].mean() for k in kpts_db])
+ else:
+ scores = np.array([k['score'] for k in kpts_db])
+
+ kpts = np.array([k['keypoints'].flatten() for k in kpts_db])
+ areas = np.array([k['area'] for k in kpts_db])
+
+ order = scores.argsort()[::-1]
+
+ keep = []
+ while len(order) > 0:
+ i = order[0]
+ keep.append(i)
+
+ oks_ovr = oks_iou(kpts[i], kpts[order[1:]], areas[i], areas[order[1:]],
+ sigmas, vis_thr)
+
+ inds = np.where(oks_ovr <= thr)[0]
+ order = order[inds + 1]
+
+ keep = np.array(keep)
+
+ return keep
+
+
+def _rescore(overlap, scores, thr, type='gaussian'):
+ """Rescoring mechanism gaussian or linear.
+
+ Args:
+ overlap: calculated ious
+ scores: target scores.
+ thr: retain oks overlap < thr.
+ type: 'gaussian' or 'linear'
+
+ Returns:
+ np.ndarray: indexes to keep
+ """
+ assert len(overlap) == len(scores)
+ assert type in ['gaussian', 'linear']
+
+ if type == 'linear':
+ inds = np.where(overlap >= thr)[0]
+ scores[inds] = scores[inds] * (1 - overlap[inds])
+ else:
+ scores = scores * np.exp(-overlap**2 / thr)
+
+ return scores
+
+
+def soft_oks_nms(kpts_db,
+ thr,
+ max_dets=20,
+ sigmas=None,
+ vis_thr=None,
+ score_per_joint=False):
+ """Soft OKS NMS implementations.
+
+ Args:
+ kpts_db
+ thr: retain oks overlap < thr.
+ max_dets: max number of detections to keep.
+ sigmas: Keypoint labelling uncertainty.
+ score_per_joint: the input scores (in kpts_db) are per joint scores
+
+ Returns:
+ np.ndarray: indexes to keep.
+ """
+ if len(kpts_db) == 0:
+ return []
+
+ if score_per_joint:
+ scores = np.array([k['score'].mean() for k in kpts_db])
+ else:
+ scores = np.array([k['score'] for k in kpts_db])
+
+ kpts = np.array([k['keypoints'].flatten() for k in kpts_db])
+ areas = np.array([k['area'] for k in kpts_db])
+
+ order = scores.argsort()[::-1]
+ scores = scores[order]
+
+ keep = np.zeros(max_dets, dtype=np.intp)
+ keep_cnt = 0
+ while len(order) > 0 and keep_cnt < max_dets:
+ i = order[0]
+
+ oks_ovr = oks_iou(kpts[i], kpts[order[1:]], areas[i], areas[order[1:]],
+ sigmas, vis_thr)
+
+ order = order[1:]
+ scores = _rescore(oks_ovr, scores[1:], thr)
+
+ tmp = scores.argsort()[::-1]
+ order = order[tmp]
+ scores = scores[tmp]
+
+ keep[keep_cnt] = i
+ keep_cnt += 1
+
+ keep = keep[:keep_cnt]
+
+ return keep
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/core/post_processing/one_euro_filter.py b/engine/pose_estimation/third-party/ViTPose/mmpose/core/post_processing/one_euro_filter.py
new file mode 100644
index 0000000..01ffa5f
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/core/post_processing/one_euro_filter.py
@@ -0,0 +1,102 @@
+# ------------------------------------------------------------------------------
+# Adapted from https://github.com/HoBeom/OneEuroFilter-Numpy
+# Original licence: Copyright (c) HoBeom Jeon, under the MIT License.
+# ------------------------------------------------------------------------------
+from time import time
+
+import numpy as np
+
+
+def smoothing_factor(t_e, cutoff):
+ r = 2 * np.pi * cutoff * t_e
+ return r / (r + 1)
+
+
+def exponential_smoothing(a, x, x_prev):
+ return a * x + (1 - a) * x_prev
+
+
+class OneEuroFilter:
+
+ def __init__(self,
+ x0,
+ dx0=0.0,
+ min_cutoff=1.7,
+ beta=0.3,
+ d_cutoff=30.0,
+ fps=None):
+ """One Euro Filter for keypoints smoothing.
+
+ Args:
+ x0 (np.ndarray[K, 2]): Initialize keypoints value
+ dx0 (float): 0.0
+ min_cutoff (float): parameter for one euro filter
+ beta (float): parameter for one euro filter
+ d_cutoff (float): Input data FPS
+ fps (float): Video FPS for video inference
+ """
+
+ # The parameters.
+ self.data_shape = x0.shape
+ self.min_cutoff = np.full(x0.shape, min_cutoff)
+ self.beta = np.full(x0.shape, beta)
+ self.d_cutoff = np.full(x0.shape, d_cutoff)
+ # Previous values.
+ self.x_prev = x0.astype(np.float32)
+ self.dx_prev = np.full(x0.shape, dx0)
+ self.mask_prev = np.ma.masked_where(x0 <= 0, x0)
+ self.realtime = True
+ if fps is None:
+ # Using in realtime inference
+ self.t_e = None
+ self.skip_frame_factor = d_cutoff
+ else:
+ # fps using video inference
+ self.realtime = False
+ self.d_cutoff = np.full(x0.shape, float(fps))
+ self.t_prev = time()
+
+ def __call__(self, x, t_e=1.0):
+ """Compute the filtered signal.
+
+ Hyper-parameters (cutoff, beta) are from `VNect
+ `__ .
+
+ Realtime Camera fps (d_cutoff) default 30.0
+
+ Args:
+ x (np.ndarray[K, 2]): keypoints results in frame
+ t_e (Optional): video skip frame count for posetrack
+ evaluation
+ """
+ assert x.shape == self.data_shape
+
+ t = 0
+ if self.realtime:
+ t = time()
+ t_e = (t - self.t_prev) * self.skip_frame_factor
+ t_e = np.full(x.shape, t_e)
+
+ # missing keypoints mask
+ mask = np.ma.masked_where(x <= 0, x)
+
+ # The filtered derivative of the signal.
+ a_d = smoothing_factor(t_e, self.d_cutoff)
+ dx = (x - self.x_prev) / t_e
+ dx_hat = exponential_smoothing(a_d, dx, self.dx_prev)
+
+ # The filtered signal.
+ cutoff = self.min_cutoff + self.beta * np.abs(dx_hat)
+ a = smoothing_factor(t_e, cutoff)
+ x_hat = exponential_smoothing(a, x, self.x_prev)
+
+ # missing keypoints remove
+ np.copyto(x_hat, -10, where=mask.mask)
+
+ # Memorize the previous values.
+ self.x_prev = x_hat
+ self.dx_prev = dx_hat
+ self.t_prev = t
+ self.mask_prev = mask
+
+ return x_hat
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/core/post_processing/post_transforms.py b/engine/pose_estimation/third-party/ViTPose/mmpose/core/post_processing/post_transforms.py
new file mode 100644
index 0000000..93063fb
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/core/post_processing/post_transforms.py
@@ -0,0 +1,366 @@
+# ------------------------------------------------------------------------------
+# Adapted from https://github.com/leoxiaobin/deep-high-resolution-net.pytorch
+# Original licence: Copyright (c) Microsoft, under the MIT License.
+# ------------------------------------------------------------------------------
+
+import math
+
+import cv2
+import numpy as np
+import torch
+
+
+def fliplr_joints(joints_3d, joints_3d_visible, img_width, flip_pairs):
+ """Flip human joints horizontally.
+
+ Note:
+ - num_keypoints: K
+
+ Args:
+ joints_3d (np.ndarray([K, 3])): Coordinates of keypoints.
+ joints_3d_visible (np.ndarray([K, 1])): Visibility of keypoints.
+ img_width (int): Image width.
+ flip_pairs (list[tuple]): Pairs of keypoints which are mirrored
+ (for example, left ear and right ear).
+
+ Returns:
+ tuple: Flipped human joints.
+
+ - joints_3d_flipped (np.ndarray([K, 3])): Flipped joints.
+ - joints_3d_visible_flipped (np.ndarray([K, 1])): Joint visibility.
+ """
+
+ assert len(joints_3d) == len(joints_3d_visible)
+ assert img_width > 0
+
+ joints_3d_flipped = joints_3d.copy()
+ joints_3d_visible_flipped = joints_3d_visible.copy()
+
+ # Swap left-right parts
+ for left, right in flip_pairs:
+ joints_3d_flipped[left, :] = joints_3d[right, :]
+ joints_3d_flipped[right, :] = joints_3d[left, :]
+
+ joints_3d_visible_flipped[left, :] = joints_3d_visible[right, :]
+ joints_3d_visible_flipped[right, :] = joints_3d_visible[left, :]
+
+ # Flip horizontally
+ joints_3d_flipped[:, 0] = img_width - 1 - joints_3d_flipped[:, 0]
+ joints_3d_flipped = joints_3d_flipped * joints_3d_visible_flipped
+
+ return joints_3d_flipped, joints_3d_visible_flipped
+
+
+def fliplr_regression(regression,
+ flip_pairs,
+ center_mode='static',
+ center_x=0.5,
+ center_index=0):
+ """Flip human joints horizontally.
+
+ Note:
+ - batch_size: N
+ - num_keypoint: K
+
+ Args:
+ regression (np.ndarray([..., K, C])): Coordinates of keypoints, where K
+ is the joint number and C is the dimension. Example shapes are:
+
+ - [N, K, C]: a batch of keypoints where N is the batch size.
+ - [N, T, K, C]: a batch of pose sequences, where T is the frame
+ number.
+ flip_pairs (list[tuple()]): Pairs of keypoints which are mirrored
+ (for example, left ear -- right ear).
+ center_mode (str): The mode to set the center location on the x-axis
+ to flip around. Options are:
+
+ - static: use a static x value (see center_x also)
+ - root: use a root joint (see center_index also)
+ center_x (float): Set the x-axis location of the flip center. Only used
+ when center_mode=static.
+ center_index (int): Set the index of the root joint, whose x location
+ will be used as the flip center. Only used when center_mode=root.
+
+ Returns:
+ np.ndarray([..., K, C]): Flipped joints.
+ """
+ assert regression.ndim >= 2, f'Invalid pose shape {regression.shape}'
+
+ allowed_center_mode = {'static', 'root'}
+ assert center_mode in allowed_center_mode, 'Get invalid center_mode ' \
+ f'{center_mode}, allowed choices are {allowed_center_mode}'
+
+ if center_mode == 'static':
+ x_c = center_x
+ elif center_mode == 'root':
+ assert regression.shape[-2] > center_index
+ x_c = regression[..., center_index:center_index + 1, 0]
+
+ regression_flipped = regression.copy()
+ # Swap left-right parts
+ for left, right in flip_pairs:
+ regression_flipped[..., left, :] = regression[..., right, :]
+ regression_flipped[..., right, :] = regression[..., left, :]
+
+ # Flip horizontally
+ regression_flipped[..., 0] = x_c * 2 - regression_flipped[..., 0]
+ return regression_flipped
+
+
+def flip_back(output_flipped, flip_pairs, target_type='GaussianHeatmap'):
+ """Flip the flipped heatmaps back to the original form.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+ - heatmap height: H
+ - heatmap width: W
+
+ Args:
+ output_flipped (np.ndarray[N, K, H, W]): The output heatmaps obtained
+ from the flipped images.
+ flip_pairs (list[tuple()): Pairs of keypoints which are mirrored
+ (for example, left ear -- right ear).
+ target_type (str): GaussianHeatmap or CombinedTarget
+
+ Returns:
+ np.ndarray: heatmaps that flipped back to the original image
+ """
+ assert output_flipped.ndim == 4, \
+ 'output_flipped should be [batch_size, num_keypoints, height, width]'
+ shape_ori = output_flipped.shape
+ channels = 1
+ if target_type.lower() == 'CombinedTarget'.lower():
+ channels = 3
+ output_flipped[:, 1::3, ...] = -output_flipped[:, 1::3, ...]
+ output_flipped = output_flipped.reshape(shape_ori[0], -1, channels,
+ shape_ori[2], shape_ori[3])
+ output_flipped_back = output_flipped.copy()
+
+ # Swap left-right parts
+ for left, right in flip_pairs:
+ output_flipped_back[:, left, ...] = output_flipped[:, right, ...]
+ output_flipped_back[:, right, ...] = output_flipped[:, left, ...]
+ output_flipped_back = output_flipped_back.reshape(shape_ori)
+ # Flip horizontally
+ output_flipped_back = output_flipped_back[..., ::-1]
+ return output_flipped_back
+
+
+def transform_preds(coords, center, scale, output_size, use_udp=False):
+ """Get final keypoint predictions from heatmaps and apply scaling and
+ translation to map them back to the image.
+
+ Note:
+ num_keypoints: K
+
+ Args:
+ coords (np.ndarray[K, ndims]):
+
+ * If ndims=2, corrds are predicted keypoint location.
+ * If ndims=4, corrds are composed of (x, y, scores, tags)
+ * If ndims=5, corrds are composed of (x, y, scores, tags,
+ flipped_tags)
+
+ center (np.ndarray[2, ]): Center of the bounding box (x, y).
+ scale (np.ndarray[2, ]): Scale of the bounding box
+ wrt [width, height].
+ output_size (np.ndarray[2, ] | list(2,)): Size of the
+ destination heatmaps.
+ use_udp (bool): Use unbiased data processing
+
+ Returns:
+ np.ndarray: Predicted coordinates in the images.
+ """
+ assert coords.shape[1] in (2, 4, 5)
+ assert len(center) == 2
+ assert len(scale) == 2
+ assert len(output_size) == 2
+
+ # Recover the scale which is normalized by a factor of 200.
+ scale = scale * 200.0
+
+ if use_udp:
+ scale_x = scale[0] / (output_size[0] - 1.0)
+ scale_y = scale[1] / (output_size[1] - 1.0)
+ else:
+ scale_x = scale[0] / output_size[0]
+ scale_y = scale[1] / output_size[1]
+
+ target_coords = np.ones_like(coords)
+ target_coords[:, 0] = coords[:, 0] * scale_x + center[0] - scale[0] * 0.5
+ target_coords[:, 1] = coords[:, 1] * scale_y + center[1] - scale[1] * 0.5
+
+ return target_coords
+
+
+def get_affine_transform(center,
+ scale,
+ rot,
+ output_size,
+ shift=(0., 0.),
+ inv=False):
+ """Get the affine transform matrix, given the center/scale/rot/output_size.
+
+ Args:
+ center (np.ndarray[2, ]): Center of the bounding box (x, y).
+ scale (np.ndarray[2, ]): Scale of the bounding box
+ wrt [width, height].
+ rot (float): Rotation angle (degree).
+ output_size (np.ndarray[2, ] | list(2,)): Size of the
+ destination heatmaps.
+ shift (0-100%): Shift translation ratio wrt the width/height.
+ Default (0., 0.).
+ inv (bool): Option to inverse the affine transform direction.
+ (inv=False: src->dst or inv=True: dst->src)
+
+ Returns:
+ np.ndarray: The transform matrix.
+ """
+ assert len(center) == 2
+ assert len(scale) == 2
+ assert len(output_size) == 2
+ assert len(shift) == 2
+
+ # pixel_std is 200.
+ scale_tmp = scale * 200.0
+
+ shift = np.array(shift)
+ src_w = scale_tmp[0]
+ dst_w = output_size[0]
+ dst_h = output_size[1]
+
+ rot_rad = np.pi * rot / 180
+ src_dir = rotate_point([0., src_w * -0.5], rot_rad)
+ dst_dir = np.array([0., dst_w * -0.5])
+
+ src = np.zeros((3, 2), dtype=np.float32)
+ src[0, :] = center + scale_tmp * shift
+ src[1, :] = center + src_dir + scale_tmp * shift
+ src[2, :] = _get_3rd_point(src[0, :], src[1, :])
+
+ dst = np.zeros((3, 2), dtype=np.float32)
+ dst[0, :] = [dst_w * 0.5, dst_h * 0.5]
+ dst[1, :] = np.array([dst_w * 0.5, dst_h * 0.5]) + dst_dir
+ dst[2, :] = _get_3rd_point(dst[0, :], dst[1, :])
+
+ if inv:
+ trans = cv2.getAffineTransform(np.float32(dst), np.float32(src))
+ else:
+ trans = cv2.getAffineTransform(np.float32(src), np.float32(dst))
+
+ return trans
+
+
+def affine_transform(pt, trans_mat):
+ """Apply an affine transformation to the points.
+
+ Args:
+ pt (np.ndarray): a 2 dimensional point to be transformed
+ trans_mat (np.ndarray): 2x3 matrix of an affine transform
+
+ Returns:
+ np.ndarray: Transformed points.
+ """
+ assert len(pt) == 2
+ new_pt = np.array(trans_mat) @ np.array([pt[0], pt[1], 1.])
+
+ return new_pt
+
+
+def _get_3rd_point(a, b):
+ """To calculate the affine matrix, three pairs of points are required. This
+ function is used to get the 3rd point, given 2D points a & b.
+
+ The 3rd point is defined by rotating vector `a - b` by 90 degrees
+ anticlockwise, using b as the rotation center.
+
+ Args:
+ a (np.ndarray): point(x,y)
+ b (np.ndarray): point(x,y)
+
+ Returns:
+ np.ndarray: The 3rd point.
+ """
+ assert len(a) == 2
+ assert len(b) == 2
+ direction = a - b
+ third_pt = b + np.array([-direction[1], direction[0]], dtype=np.float32)
+
+ return third_pt
+
+
+def rotate_point(pt, angle_rad):
+ """Rotate a point by an angle.
+
+ Args:
+ pt (list[float]): 2 dimensional point to be rotated
+ angle_rad (float): rotation angle by radian
+
+ Returns:
+ list[float]: Rotated point.
+ """
+ assert len(pt) == 2
+ sn, cs = np.sin(angle_rad), np.cos(angle_rad)
+ new_x = pt[0] * cs - pt[1] * sn
+ new_y = pt[0] * sn + pt[1] * cs
+ rotated_pt = [new_x, new_y]
+
+ return rotated_pt
+
+
+def get_warp_matrix(theta, size_input, size_dst, size_target):
+ """Calculate the transformation matrix under the constraint of unbiased.
+ Paper ref: Huang et al. The Devil is in the Details: Delving into Unbiased
+ Data Processing for Human Pose Estimation (CVPR 2020).
+
+ Args:
+ theta (float): Rotation angle in degrees.
+ size_input (np.ndarray): Size of input image [w, h].
+ size_dst (np.ndarray): Size of output image [w, h].
+ size_target (np.ndarray): Size of ROI in input plane [w, h].
+
+ Returns:
+ np.ndarray: A matrix for transformation.
+ """
+ theta = np.deg2rad(theta)
+ matrix = np.zeros((2, 3), dtype=np.float32)
+ scale_x = size_dst[0] / size_target[0]
+ scale_y = size_dst[1] / size_target[1]
+ matrix[0, 0] = math.cos(theta) * scale_x
+ matrix[0, 1] = -math.sin(theta) * scale_x
+ matrix[0, 2] = scale_x * (-0.5 * size_input[0] * math.cos(theta) +
+ 0.5 * size_input[1] * math.sin(theta) +
+ 0.5 * size_target[0])
+ matrix[1, 0] = math.sin(theta) * scale_y
+ matrix[1, 1] = math.cos(theta) * scale_y
+ matrix[1, 2] = scale_y * (-0.5 * size_input[0] * math.sin(theta) -
+ 0.5 * size_input[1] * math.cos(theta) +
+ 0.5 * size_target[1])
+ return matrix
+
+
+def warp_affine_joints(joints, mat):
+ """Apply affine transformation defined by the transform matrix on the
+ joints.
+
+ Args:
+ joints (np.ndarray[..., 2]): Origin coordinate of joints.
+ mat (np.ndarray[3, 2]): The affine matrix.
+
+ Returns:
+ np.ndarray[..., 2]: Result coordinate of joints.
+ """
+ joints = np.array(joints)
+ shape = joints.shape
+ joints = joints.reshape(-1, 2)
+ return np.dot(
+ np.concatenate((joints, joints[:, 0:1] * 0 + 1), axis=1),
+ mat.T).reshape(shape)
+
+
+def affine_transform_torch(pts, t):
+ npts = pts.shape[0]
+ pts_homo = torch.cat([pts, torch.ones(npts, 1, device=pts.device)], dim=1)
+ out = torch.mm(t, torch.t(pts_homo))
+ return torch.t(out[:2, :])
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/core/utils/__init__.py b/engine/pose_estimation/third-party/ViTPose/mmpose/core/utils/__init__.py
new file mode 100644
index 0000000..bd6c027
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/core/utils/__init__.py
@@ -0,0 +1,5 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from .dist_utils import allreduce_grads
+from .regularizations import WeightNormClipHook
+
+__all__ = ['allreduce_grads', 'WeightNormClipHook']
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/core/utils/dist_utils.py b/engine/pose_estimation/third-party/ViTPose/mmpose/core/utils/dist_utils.py
new file mode 100644
index 0000000..e76e591
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/core/utils/dist_utils.py
@@ -0,0 +1,51 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from collections import OrderedDict
+
+import torch.distributed as dist
+from torch._utils import (_flatten_dense_tensors, _take_tensors,
+ _unflatten_dense_tensors)
+
+
+def _allreduce_coalesced(tensors, world_size, bucket_size_mb=-1):
+ """Allreduce parameters as a whole."""
+ if bucket_size_mb > 0:
+ bucket_size_bytes = bucket_size_mb * 1024 * 1024
+ buckets = _take_tensors(tensors, bucket_size_bytes)
+ else:
+ buckets = OrderedDict()
+ for tensor in tensors:
+ tp = tensor.type()
+ if tp not in buckets:
+ buckets[tp] = []
+ buckets[tp].append(tensor)
+ buckets = buckets.values()
+
+ for bucket in buckets:
+ flat_tensors = _flatten_dense_tensors(bucket)
+ dist.all_reduce(flat_tensors)
+ flat_tensors.div_(world_size)
+ for tensor, synced in zip(
+ bucket, _unflatten_dense_tensors(flat_tensors, bucket)):
+ tensor.copy_(synced)
+
+
+def allreduce_grads(params, coalesce=True, bucket_size_mb=-1):
+ """Allreduce gradients.
+
+ Args:
+ params (list[torch.Parameters]): List of parameters of a model
+ coalesce (bool, optional): Whether allreduce parameters as a whole.
+ Default: True.
+ bucket_size_mb (int, optional): Size of bucket, the unit is MB.
+ Default: -1.
+ """
+ grads = [
+ param.grad.data for param in params
+ if param.requires_grad and param.grad is not None
+ ]
+ world_size = dist.get_world_size()
+ if coalesce:
+ _allreduce_coalesced(grads, world_size, bucket_size_mb)
+ else:
+ for tensor in grads:
+ dist.all_reduce(tensor.div_(world_size))
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/core/utils/regularizations.py b/engine/pose_estimation/third-party/ViTPose/mmpose/core/utils/regularizations.py
new file mode 100644
index 0000000..d8c7449
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/core/utils/regularizations.py
@@ -0,0 +1,86 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from abc import ABCMeta, abstractmethod, abstractproperty
+
+import torch
+
+
+class PytorchModuleHook(metaclass=ABCMeta):
+ """Base class for PyTorch module hook registers.
+
+ An instance of a subclass of PytorchModuleHook can be used to
+ register hook to a pytorch module using the `register` method like:
+ hook_register.register(module)
+
+ Subclasses should add/overwrite the following methods:
+ - __init__
+ - hook
+ - hook_type
+ """
+
+ @abstractmethod
+ def hook(self, *args, **kwargs):
+ """Hook function."""
+
+ @abstractproperty
+ def hook_type(self) -> str:
+ """Hook type Subclasses should overwrite this function to return a
+ string value in.
+
+ {`forward`, `forward_pre`, `backward`}
+ """
+
+ def register(self, module):
+ """Register the hook function to the module.
+
+ Args:
+ module (pytorch module): the module to register the hook.
+
+ Returns:
+ handle (torch.utils.hooks.RemovableHandle): a handle to remove
+ the hook by calling handle.remove()
+ """
+ assert isinstance(module, torch.nn.Module)
+
+ if self.hook_type == 'forward':
+ h = module.register_forward_hook(self.hook)
+ elif self.hook_type == 'forward_pre':
+ h = module.register_forward_pre_hook(self.hook)
+ elif self.hook_type == 'backward':
+ h = module.register_backward_hook(self.hook)
+ else:
+ raise ValueError(f'Invalid hook type {self.hook}')
+
+ return h
+
+
+class WeightNormClipHook(PytorchModuleHook):
+ """Apply weight norm clip regularization.
+
+ The module's parameter will be clip to a given maximum norm before each
+ forward pass.
+
+ Args:
+ max_norm (float): The maximum norm of the parameter.
+ module_param_names (str|list): The parameter name (or name list) to
+ apply weight norm clip.
+ """
+
+ def __init__(self, max_norm=1.0, module_param_names='weight'):
+ self.module_param_names = module_param_names if isinstance(
+ module_param_names, list) else [module_param_names]
+ self.max_norm = max_norm
+
+ @property
+ def hook_type(self):
+ return 'forward_pre'
+
+ def hook(self, module, _input):
+ for name in self.module_param_names:
+ assert name in module._parameters, f'{name} is not a parameter' \
+ f' of the module {type(module)}'
+ param = module._parameters[name]
+
+ with torch.no_grad():
+ m = param.norm().item()
+ if m > self.max_norm:
+ param.mul_(self.max_norm / (m + 1e-6))
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/core/visualization/__init__.py b/engine/pose_estimation/third-party/ViTPose/mmpose/core/visualization/__init__.py
new file mode 100644
index 0000000..9705494
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/core/visualization/__init__.py
@@ -0,0 +1,13 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from .effects import apply_bugeye_effect, apply_sunglasses_effect
+from .image import (imshow_bboxes, imshow_keypoints, imshow_keypoints_3d,
+ imshow_mesh_3d)
+
+__all__ = [
+ 'imshow_keypoints',
+ 'imshow_keypoints_3d',
+ 'imshow_bboxes',
+ 'apply_bugeye_effect',
+ 'apply_sunglasses_effect',
+ 'imshow_mesh_3d',
+]
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/core/visualization/effects.py b/engine/pose_estimation/third-party/ViTPose/mmpose/core/visualization/effects.py
new file mode 100644
index 0000000..d3add7d
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/core/visualization/effects.py
@@ -0,0 +1,111 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import cv2
+import numpy as np
+
+
+def apply_bugeye_effect(img,
+ pose_results,
+ left_eye_index,
+ right_eye_index,
+ kpt_thr=0.5):
+ """Apply bug-eye effect.
+
+ Args:
+ img (np.ndarray): Image data.
+ pose_results (list[dict]): The pose estimation results containing:
+ - "bbox" ([K, 4(or 5)]): detection bbox in
+ [x1, y1, x2, y2, (score)]
+ - "keypoints" ([K,3]): keypoint detection result in [x, y, score]
+ left_eye_index (int): Keypoint index of left eye
+ right_eye_index (int): Keypoint index of right eye
+ kpt_thr (float): The score threshold of required keypoints.
+ """
+
+ xx, yy = np.meshgrid(np.arange(img.shape[1]), np.arange(img.shape[0]))
+ xx = xx.astype(np.float32)
+ yy = yy.astype(np.float32)
+
+ for pose in pose_results:
+ bbox = pose['bbox']
+ kpts = pose['keypoints']
+
+ if kpts[left_eye_index, 2] < kpt_thr or kpts[right_eye_index,
+ 2] < kpt_thr:
+ continue
+
+ kpt_leye = kpts[left_eye_index, :2]
+ kpt_reye = kpts[right_eye_index, :2]
+ for xc, yc in [kpt_leye, kpt_reye]:
+
+ # distortion parameters
+ k1 = 0.001
+ epe = 1e-5
+
+ scale = (bbox[2] - bbox[0])**2 + (bbox[3] - bbox[1])**2
+ r2 = ((xx - xc)**2 + (yy - yc)**2)
+ r2 = (r2 + epe) / scale # normalized by bbox scale
+
+ xx = (xx - xc) / (1 + k1 / r2) + xc
+ yy = (yy - yc) / (1 + k1 / r2) + yc
+
+ img = cv2.remap(
+ img,
+ xx,
+ yy,
+ interpolation=cv2.INTER_AREA,
+ borderMode=cv2.BORDER_REPLICATE)
+ return img
+
+
+def apply_sunglasses_effect(img,
+ pose_results,
+ sunglasses_img,
+ left_eye_index,
+ right_eye_index,
+ kpt_thr=0.5):
+ """Apply sunglasses effect.
+
+ Args:
+ img (np.ndarray): Image data.
+ pose_results (list[dict]): The pose estimation results containing:
+ - "keypoints" ([K,3]): keypoint detection result in [x, y, score]
+ sunglasses_img (np.ndarray): Sunglasses image with white background.
+ left_eye_index (int): Keypoint index of left eye
+ right_eye_index (int): Keypoint index of right eye
+ kpt_thr (float): The score threshold of required keypoints.
+ """
+
+ hm, wm = sunglasses_img.shape[:2]
+ # anchor points in the sunglasses mask
+ pts_src = np.array([[0.3 * wm, 0.3 * hm], [0.3 * wm, 0.7 * hm],
+ [0.7 * wm, 0.3 * hm], [0.7 * wm, 0.7 * hm]],
+ dtype=np.float32)
+
+ for pose in pose_results:
+ kpts = pose['keypoints']
+
+ if kpts[left_eye_index, 2] < kpt_thr or kpts[right_eye_index,
+ 2] < kpt_thr:
+ continue
+
+ kpt_leye = kpts[left_eye_index, :2]
+ kpt_reye = kpts[right_eye_index, :2]
+ # orthogonal vector to the left-to-right eyes
+ vo = 0.5 * (kpt_reye - kpt_leye)[::-1] * [-1, 1]
+
+ # anchor points in the image by eye positions
+ pts_tar = np.vstack(
+ [kpt_reye + vo, kpt_reye - vo, kpt_leye + vo, kpt_leye - vo])
+
+ h_mat, _ = cv2.findHomography(pts_src, pts_tar)
+ patch = cv2.warpPerspective(
+ sunglasses_img,
+ h_mat,
+ dsize=(img.shape[1], img.shape[0]),
+ borderValue=(255, 255, 255))
+ # mask the white background area in the patch with a threshold 200
+ mask = cv2.cvtColor(patch, cv2.COLOR_BGR2GRAY)
+ mask = (mask < 200).astype(np.uint8)
+ img = cv2.copyTo(patch, mask, img)
+
+ return img
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/core/visualization/image.py b/engine/pose_estimation/third-party/ViTPose/mmpose/core/visualization/image.py
new file mode 100644
index 0000000..8acd10b
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/core/visualization/image.py
@@ -0,0 +1,442 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import math
+import os
+import warnings
+
+import cv2
+import mmcv
+import numpy as np
+from matplotlib import pyplot as plt
+from mmcv.utils.misc import deprecated_api_warning
+from mmcv.visualization.color import color_val
+
+try:
+ import trimesh
+ has_trimesh = True
+except (ImportError, ModuleNotFoundError):
+ has_trimesh = False
+
+try:
+ os.environ['PYOPENGL_PLATFORM'] = 'osmesa'
+ import pyrender
+ has_pyrender = True
+except (ImportError, ModuleNotFoundError):
+ has_pyrender = False
+
+
+def imshow_bboxes(img,
+ bboxes,
+ labels=None,
+ colors='green',
+ text_color='white',
+ thickness=1,
+ font_scale=0.5,
+ show=True,
+ win_name='',
+ wait_time=0,
+ out_file=None):
+ """Draw bboxes with labels (optional) on an image. This is a wrapper of
+ mmcv.imshow_bboxes.
+
+ Args:
+ img (str or ndarray): The image to be displayed.
+ bboxes (ndarray): ndarray of shape (k, 4), each row is a bbox in
+ format [x1, y1, x2, y2].
+ labels (str or list[str], optional): labels of each bbox.
+ colors (list[str or tuple or :obj:`Color`]): A list of colors.
+ text_color (str or tuple or :obj:`Color`): Color of texts.
+ thickness (int): Thickness of lines.
+ font_scale (float): Font scales of texts.
+ show (bool): Whether to show the image.
+ win_name (str): The window name.
+ wait_time (int): Value of waitKey param.
+ out_file (str, optional): The filename to write the image.
+
+ Returns:
+ ndarray: The image with bboxes drawn on it.
+ """
+
+ # adapt to mmcv.imshow_bboxes input format
+ bboxes = np.split(
+ bboxes, bboxes.shape[0], axis=0) if bboxes.shape[0] > 0 else []
+ if not isinstance(colors, list):
+ colors = [colors for _ in range(len(bboxes))]
+ colors = [mmcv.color_val(c) for c in colors]
+ assert len(bboxes) == len(colors)
+
+ img = mmcv.imshow_bboxes(
+ img,
+ bboxes,
+ colors,
+ top_k=-1,
+ thickness=thickness,
+ show=False,
+ out_file=None)
+
+ if labels is not None:
+ if not isinstance(labels, list):
+ labels = [labels for _ in range(len(bboxes))]
+ assert len(labels) == len(bboxes)
+
+ for bbox, label, color in zip(bboxes, labels, colors):
+ if label is None:
+ continue
+ bbox_int = bbox[0, :4].astype(np.int32)
+ # roughly estimate the proper font size
+ text_size, text_baseline = cv2.getTextSize(label,
+ cv2.FONT_HERSHEY_DUPLEX,
+ font_scale, thickness)
+ text_x1 = bbox_int[0]
+ text_y1 = max(0, bbox_int[1] - text_size[1] - text_baseline)
+ text_x2 = bbox_int[0] + text_size[0]
+ text_y2 = text_y1 + text_size[1] + text_baseline
+ cv2.rectangle(img, (text_x1, text_y1), (text_x2, text_y2), color,
+ cv2.FILLED)
+ cv2.putText(img, label, (text_x1, text_y2 - text_baseline),
+ cv2.FONT_HERSHEY_DUPLEX, font_scale,
+ mmcv.color_val(text_color), thickness)
+
+ if show:
+ mmcv.imshow(img, win_name, wait_time)
+ if out_file is not None:
+ mmcv.imwrite(img, out_file)
+ return img
+
+
+@deprecated_api_warning({'pose_limb_color': 'pose_link_color'})
+def imshow_keypoints(img,
+ pose_result,
+ skeleton=None,
+ kpt_score_thr=0.3,
+ pose_kpt_color=None,
+ pose_link_color=None,
+ radius=4,
+ thickness=1,
+ show_keypoint_weight=False):
+ """Draw keypoints and links on an image.
+
+ Args:
+ img (str or Tensor): The image to draw poses on. If an image array
+ is given, id will be modified in-place.
+ pose_result (list[kpts]): The poses to draw. Each element kpts is
+ a set of K keypoints as an Kx3 numpy.ndarray, where each
+ keypoint is represented as x, y, score.
+ kpt_score_thr (float, optional): Minimum score of keypoints
+ to be shown. Default: 0.3.
+ pose_kpt_color (np.array[Nx3]`): Color of N keypoints. If None,
+ the keypoint will not be drawn.
+ pose_link_color (np.array[Mx3]): Color of M links. If None, the
+ links will not be drawn.
+ thickness (int): Thickness of lines.
+ """
+
+ img = mmcv.imread(img)
+ img_h, img_w, _ = img.shape
+
+ for kpts in pose_result:
+
+ kpts = np.array(kpts, copy=False)
+
+ # draw each point on image
+ if pose_kpt_color is not None:
+ assert len(pose_kpt_color) == len(kpts)
+ for kid, kpt in enumerate(kpts):
+ x_coord, y_coord, kpt_score = int(kpt[0]), int(kpt[1]), kpt[2]
+ if kpt_score > kpt_score_thr:
+ color = tuple(int(c) for c in pose_kpt_color[kid])
+ if show_keypoint_weight:
+ img_copy = img.copy()
+ cv2.circle(img_copy, (int(x_coord), int(y_coord)),
+ radius, color, -1)
+ transparency = max(0, min(1, kpt_score))
+ cv2.addWeighted(
+ img_copy,
+ transparency,
+ img,
+ 1 - transparency,
+ 0,
+ dst=img)
+ else:
+ cv2.circle(img, (int(x_coord), int(y_coord)), radius,
+ color, -1)
+
+ # draw links
+ if skeleton is not None and pose_link_color is not None:
+ assert len(pose_link_color) == len(skeleton)
+ for sk_id, sk in enumerate(skeleton):
+ pos1 = (int(kpts[sk[0], 0]), int(kpts[sk[0], 1]))
+ pos2 = (int(kpts[sk[1], 0]), int(kpts[sk[1], 1]))
+ if (pos1[0] > 0 and pos1[0] < img_w and pos1[1] > 0
+ and pos1[1] < img_h and pos2[0] > 0 and pos2[0] < img_w
+ and pos2[1] > 0 and pos2[1] < img_h
+ and kpts[sk[0], 2] > kpt_score_thr
+ and kpts[sk[1], 2] > kpt_score_thr):
+ color = tuple(int(c) for c in pose_link_color[sk_id])
+ if show_keypoint_weight:
+ img_copy = img.copy()
+ X = (pos1[0], pos2[0])
+ Y = (pos1[1], pos2[1])
+ mX = np.mean(X)
+ mY = np.mean(Y)
+ length = ((Y[0] - Y[1])**2 + (X[0] - X[1])**2)**0.5
+ angle = math.degrees(
+ math.atan2(Y[0] - Y[1], X[0] - X[1]))
+ stickwidth = 2
+ polygon = cv2.ellipse2Poly(
+ (int(mX), int(mY)),
+ (int(length / 2), int(stickwidth)), int(angle), 0,
+ 360, 1)
+ cv2.fillConvexPoly(img_copy, polygon, color)
+ transparency = max(
+ 0, min(1, 0.5 * (kpts[sk[0], 2] + kpts[sk[1], 2])))
+ cv2.addWeighted(
+ img_copy,
+ transparency,
+ img,
+ 1 - transparency,
+ 0,
+ dst=img)
+ else:
+ cv2.line(img, pos1, pos2, color, thickness=thickness)
+
+ return img
+
+
+def imshow_keypoints_3d(
+ pose_result,
+ img=None,
+ skeleton=None,
+ pose_kpt_color=None,
+ pose_link_color=None,
+ vis_height=400,
+ kpt_score_thr=0.3,
+ num_instances=-1,
+ *,
+ axis_azimuth=70,
+ axis_limit=1.7,
+ axis_dist=10.0,
+ axis_elev=15.0,
+):
+ """Draw 3D keypoints and links in 3D coordinates.
+
+ Args:
+ pose_result (list[dict]): 3D pose results containing:
+ - "keypoints_3d" ([K,4]): 3D keypoints
+ - "title" (str): Optional. A string to specify the title of the
+ visualization of this pose result
+ img (str|np.ndarray): Opptional. The image or image path to show input
+ image and/or 2D pose. Note that the image should be given in BGR
+ channel order.
+ skeleton (list of [idx_i,idx_j]): Skeleton described by a list of
+ links, each is a pair of joint indices.
+ pose_kpt_color (np.ndarray[Nx3]`): Color of N keypoints. If None, do
+ not nddraw keypoints.
+ pose_link_color (np.array[Mx3]): Color of M links. If None, do not
+ draw links.
+ vis_height (int): The image height of the visualization. The width
+ will be N*vis_height depending on the number of visualized
+ items.
+ kpt_score_thr (float): Minimum score of keypoints to be shown.
+ Default: 0.3.
+ num_instances (int): Number of instances to be shown in 3D. If smaller
+ than 0, all the instances in the pose_result will be shown.
+ Otherwise, pad or truncate the pose_result to a length of
+ num_instances.
+ axis_azimuth (float): axis azimuth angle for 3D visualizations.
+ axis_dist (float): axis distance for 3D visualizations.
+ axis_elev (float): axis elevation view angle for 3D visualizations.
+ axis_limit (float): The axis limit to visualize 3d pose. The xyz
+ range will be set as:
+ - x: [x_c - axis_limit/2, x_c + axis_limit/2]
+ - y: [y_c - axis_limit/2, y_c + axis_limit/2]
+ - z: [0, axis_limit]
+ Where x_c, y_c is the mean value of x and y coordinates
+ figsize: (float): figure size in inch.
+ """
+
+ show_img = img is not None
+ if num_instances < 0:
+ num_instances = len(pose_result)
+ else:
+ if len(pose_result) > num_instances:
+ pose_result = pose_result[:num_instances]
+ elif len(pose_result) < num_instances:
+ pose_result += [dict()] * (num_instances - len(pose_result))
+ num_axis = num_instances + 1 if show_img else num_instances
+
+ plt.ioff()
+ fig = plt.figure(figsize=(vis_height * num_axis * 0.01, vis_height * 0.01))
+
+ if show_img:
+ img = mmcv.imread(img, channel_order='bgr')
+ img = mmcv.bgr2rgb(img)
+ img = mmcv.imrescale(img, scale=vis_height / img.shape[0])
+
+ ax_img = fig.add_subplot(1, num_axis, 1)
+ ax_img.get_xaxis().set_visible(False)
+ ax_img.get_yaxis().set_visible(False)
+ ax_img.set_axis_off()
+ ax_img.set_title('Input')
+ ax_img.imshow(img, aspect='equal')
+
+ for idx, res in enumerate(pose_result):
+ dummy = len(res) == 0
+ kpts = np.zeros((1, 3)) if dummy else res['keypoints_3d']
+ if kpts.shape[1] == 3:
+ kpts = np.concatenate([kpts, np.ones((kpts.shape[0], 1))], axis=1)
+ valid = kpts[:, 3] >= kpt_score_thr
+
+ ax_idx = idx + 2 if show_img else idx + 1
+ ax = fig.add_subplot(1, num_axis, ax_idx, projection='3d')
+ ax.view_init(
+ elev=axis_elev,
+ azim=axis_azimuth,
+ )
+ x_c = np.mean(kpts[valid, 0]) if sum(valid) > 0 else 0
+ y_c = np.mean(kpts[valid, 1]) if sum(valid) > 0 else 0
+ ax.set_xlim3d([x_c - axis_limit / 2, x_c + axis_limit / 2])
+ ax.set_ylim3d([y_c - axis_limit / 2, y_c + axis_limit / 2])
+ ax.set_zlim3d([0, axis_limit])
+ ax.set_aspect('auto')
+ ax.set_xticks([])
+ ax.set_yticks([])
+ ax.set_zticks([])
+ ax.set_xticklabels([])
+ ax.set_yticklabels([])
+ ax.set_zticklabels([])
+ ax.dist = axis_dist
+
+ if not dummy and pose_kpt_color is not None:
+ pose_kpt_color = np.array(pose_kpt_color)
+ assert len(pose_kpt_color) == len(kpts)
+ x_3d, y_3d, z_3d = np.split(kpts[:, :3], [1, 2], axis=1)
+ # matplotlib uses RGB color in [0, 1] value range
+ _color = pose_kpt_color[..., ::-1] / 255.
+ ax.scatter(
+ x_3d[valid],
+ y_3d[valid],
+ z_3d[valid],
+ marker='o',
+ color=_color[valid],
+ )
+
+ if not dummy and skeleton is not None and pose_link_color is not None:
+ pose_link_color = np.array(pose_link_color)
+ assert len(pose_link_color) == len(skeleton)
+ for link, link_color in zip(skeleton, pose_link_color):
+ link_indices = [_i for _i in link]
+ xs_3d = kpts[link_indices, 0]
+ ys_3d = kpts[link_indices, 1]
+ zs_3d = kpts[link_indices, 2]
+ kpt_score = kpts[link_indices, 3]
+ if kpt_score.min() > kpt_score_thr:
+ # matplotlib uses RGB color in [0, 1] value range
+ _color = link_color[::-1] / 255.
+ ax.plot(xs_3d, ys_3d, zs_3d, color=_color, zdir='z')
+
+ if 'title' in res:
+ ax.set_title(res['title'])
+
+ # convert figure to numpy array
+ fig.tight_layout()
+ fig.canvas.draw()
+ img_w, img_h = fig.canvas.get_width_height()
+ img_vis = np.frombuffer(
+ fig.canvas.tostring_rgb(), dtype=np.uint8).reshape(img_h, img_w, -1)
+ img_vis = mmcv.rgb2bgr(img_vis)
+
+ plt.close(fig)
+
+ return img_vis
+
+
+def imshow_mesh_3d(img,
+ vertices,
+ faces,
+ camera_center,
+ focal_length,
+ colors=(76, 76, 204)):
+ """Render 3D meshes on background image.
+
+ Args:
+ img(np.ndarray): Background image.
+ vertices (list of np.ndarray): Vetrex coordinates in camera space.
+ faces (list of np.ndarray): Faces of meshes.
+ camera_center ([2]): Center pixel.
+ focal_length ([2]): Focal length of camera.
+ colors (list[str or tuple or Color]): A list of mesh colors.
+ """
+
+ H, W, C = img.shape
+
+ if not has_pyrender:
+ warnings.warn('pyrender package is not installed.')
+ return img
+
+ if not has_trimesh:
+ warnings.warn('trimesh package is not installed.')
+ return img
+
+ try:
+ renderer = pyrender.OffscreenRenderer(
+ viewport_width=W, viewport_height=H)
+ except (ImportError, RuntimeError):
+ warnings.warn('pyrender package is not installed correctly.')
+ return img
+
+ if not isinstance(colors, list):
+ colors = [colors for _ in range(len(vertices))]
+ colors = [color_val(c) for c in colors]
+
+ depth_map = np.ones([H, W]) * np.inf
+ output_img = img
+ for idx in range(len(vertices)):
+ color = colors[idx]
+ color = [c / 255.0 for c in color]
+ color.append(1.0)
+ vert = vertices[idx]
+ face = faces[idx]
+
+ material = pyrender.MetallicRoughnessMaterial(
+ metallicFactor=0.2, alphaMode='OPAQUE', baseColorFactor=color)
+
+ mesh = trimesh.Trimesh(vert, face)
+ rot = trimesh.transformations.rotation_matrix(
+ np.radians(180), [1, 0, 0])
+ mesh.apply_transform(rot)
+ mesh = pyrender.Mesh.from_trimesh(mesh, material=material)
+
+ scene = pyrender.Scene(ambient_light=(0.5, 0.5, 0.5))
+ scene.add(mesh, 'mesh')
+
+ camera_pose = np.eye(4)
+ camera = pyrender.IntrinsicsCamera(
+ fx=focal_length[0],
+ fy=focal_length[1],
+ cx=camera_center[0],
+ cy=camera_center[1],
+ zfar=1e5)
+ scene.add(camera, pose=camera_pose)
+
+ light = pyrender.DirectionalLight(color=[1.0, 1.0, 1.0], intensity=1)
+ light_pose = np.eye(4)
+
+ light_pose[:3, 3] = np.array([0, -1, 1])
+ scene.add(light, pose=light_pose)
+
+ light_pose[:3, 3] = np.array([0, 1, 1])
+ scene.add(light, pose=light_pose)
+
+ light_pose[:3, 3] = np.array([1, 1, 2])
+ scene.add(light, pose=light_pose)
+
+ color, rend_depth = renderer.render(
+ scene, flags=pyrender.RenderFlags.RGBA)
+
+ valid_mask = (rend_depth < depth_map) * (rend_depth > 0)
+ depth_map[valid_mask] = rend_depth[valid_mask]
+ valid_mask = valid_mask[:, :, None]
+ output_img = (
+ valid_mask * color[:, :, :3] + (1 - valid_mask) * output_img)
+
+ return output_img
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/__init__.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/__init__.py
new file mode 100644
index 0000000..1b9e7cf
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/__init__.py
@@ -0,0 +1,42 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from .builder import DATASETS, PIPELINES, build_dataloader, build_dataset
+from .dataset_info import DatasetInfo
+from .pipelines import Compose
+from .samplers import DistributedSampler
+
+from .datasets import ( # isort:skip
+ AnimalATRWDataset, AnimalFlyDataset, AnimalHorse10Dataset,
+ AnimalLocustDataset, AnimalMacaqueDataset, AnimalPoseDataset,
+ AnimalZebraDataset, Body3DH36MDataset, BottomUpAicDataset,
+ BottomUpCocoDataset, BottomUpCocoWholeBodyDataset,
+ BottomUpCrowdPoseDataset, BottomUpMhpDataset, DeepFashionDataset,
+ Face300WDataset, FaceAFLWDataset, FaceCocoWholeBodyDataset,
+ FaceCOFWDataset, FaceWFLWDataset, FreiHandDataset,
+ HandCocoWholeBodyDataset, InterHand2DDataset, InterHand3DDataset,
+ MeshAdversarialDataset, MeshH36MDataset, MeshMixDataset, MoshDataset,
+ OneHand10KDataset, PanopticDataset, TopDownAicDataset, TopDownCocoDataset,
+ TopDownCocoWholeBodyDataset, TopDownCrowdPoseDataset,
+ TopDownFreiHandDataset, TopDownH36MDataset, TopDownJhmdbDataset,
+ TopDownMhpDataset, TopDownMpiiDataset, TopDownMpiiTrbDataset,
+ TopDownOCHumanDataset, TopDownOneHand10KDataset, TopDownPanopticDataset,
+ TopDownPoseTrack18Dataset, TopDownPoseTrack18VideoDataset)
+
+__all__ = [
+ 'TopDownCocoDataset', 'BottomUpCocoDataset', 'BottomUpMhpDataset',
+ 'BottomUpAicDataset', 'BottomUpCocoWholeBodyDataset', 'TopDownMpiiDataset',
+ 'TopDownMpiiTrbDataset', 'OneHand10KDataset', 'PanopticDataset',
+ 'HandCocoWholeBodyDataset', 'FreiHandDataset', 'InterHand2DDataset',
+ 'InterHand3DDataset', 'TopDownOCHumanDataset', 'TopDownAicDataset',
+ 'TopDownCocoWholeBodyDataset', 'MeshH36MDataset', 'MeshMixDataset',
+ 'MoshDataset', 'MeshAdversarialDataset', 'TopDownCrowdPoseDataset',
+ 'BottomUpCrowdPoseDataset', 'TopDownFreiHandDataset',
+ 'TopDownOneHand10KDataset', 'TopDownPanopticDataset',
+ 'TopDownPoseTrack18Dataset', 'TopDownJhmdbDataset', 'TopDownMhpDataset',
+ 'DeepFashionDataset', 'Face300WDataset', 'FaceAFLWDataset',
+ 'FaceWFLWDataset', 'FaceCOFWDataset', 'FaceCocoWholeBodyDataset',
+ 'Body3DH36MDataset', 'AnimalHorse10Dataset', 'AnimalMacaqueDataset',
+ 'AnimalFlyDataset', 'AnimalLocustDataset', 'AnimalZebraDataset',
+ 'AnimalATRWDataset', 'AnimalPoseDataset', 'TopDownH36MDataset',
+ 'TopDownPoseTrack18VideoDataset', 'build_dataloader', 'build_dataset',
+ 'Compose', 'DistributedSampler', 'DATASETS', 'PIPELINES', 'DatasetInfo'
+]
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/builder.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/builder.py
new file mode 100644
index 0000000..990ba85
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/builder.py
@@ -0,0 +1,162 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import copy
+import platform
+import random
+from functools import partial
+
+import numpy as np
+from mmcv.parallel import collate
+from mmcv.runner import get_dist_info
+from mmcv.utils import Registry, build_from_cfg, is_seq_of
+from mmcv.utils.parrots_wrapper import _get_dataloader
+from torch.utils.data.dataset import ConcatDataset
+
+from .samplers import DistributedSampler
+
+if platform.system() != 'Windows':
+ # https://github.com/pytorch/pytorch/issues/973
+ import resource
+ rlimit = resource.getrlimit(resource.RLIMIT_NOFILE)
+ base_soft_limit = rlimit[0]
+ hard_limit = rlimit[1]
+ soft_limit = min(max(4096, base_soft_limit), hard_limit)
+ resource.setrlimit(resource.RLIMIT_NOFILE, (soft_limit, hard_limit))
+
+DATASETS = Registry('dataset')
+PIPELINES = Registry('pipeline')
+
+
+def _concat_dataset(cfg, default_args=None):
+ types = cfg['type']
+ ann_files = cfg['ann_file']
+ img_prefixes = cfg.get('img_prefix', None)
+ dataset_infos = cfg.get('dataset_info', None)
+
+ num_joints = cfg['data_cfg'].get('num_joints', None)
+ dataset_channel = cfg['data_cfg'].get('dataset_channel', None)
+
+ datasets = []
+ num_dset = len(ann_files)
+ for i in range(num_dset):
+ cfg_copy = copy.deepcopy(cfg)
+ cfg_copy['ann_file'] = ann_files[i]
+
+ if isinstance(types, (list, tuple)):
+ cfg_copy['type'] = types[i]
+ if isinstance(img_prefixes, (list, tuple)):
+ cfg_copy['img_prefix'] = img_prefixes[i]
+ if isinstance(dataset_infos, (list, tuple)):
+ cfg_copy['dataset_info'] = dataset_infos[i]
+
+ if isinstance(num_joints, (list, tuple)):
+ cfg_copy['data_cfg']['num_joints'] = num_joints[i]
+
+ if is_seq_of(dataset_channel, list):
+ cfg_copy['data_cfg']['dataset_channel'] = dataset_channel[i]
+
+ datasets.append(build_dataset(cfg_copy, default_args))
+
+ return ConcatDataset(datasets)
+
+
+def build_dataset(cfg, default_args=None):
+ """Build a dataset from config dict.
+
+ Args:
+ cfg (dict): Config dict. It should at least contain the key "type".
+ default_args (dict, optional): Default initialization arguments.
+ Default: None.
+
+ Returns:
+ Dataset: The constructed dataset.
+ """
+ from .dataset_wrappers import RepeatDataset
+
+ if isinstance(cfg, (list, tuple)):
+ dataset = ConcatDataset([build_dataset(c, default_args) for c in cfg])
+ elif cfg['type'] == 'ConcatDataset':
+ dataset = ConcatDataset(
+ [build_dataset(c, default_args) for c in cfg['datasets']])
+ elif cfg['type'] == 'RepeatDataset':
+ dataset = RepeatDataset(
+ build_dataset(cfg['dataset'], default_args), cfg['times'])
+ elif isinstance(cfg.get('ann_file'), (list, tuple)):
+ dataset = _concat_dataset(cfg, default_args)
+ else:
+ dataset = build_from_cfg(cfg, DATASETS, default_args)
+ return dataset
+
+
+def build_dataloader(dataset,
+ samples_per_gpu,
+ workers_per_gpu,
+ num_gpus=1,
+ dist=True,
+ shuffle=True,
+ seed=None,
+ drop_last=True,
+ pin_memory=True,
+ **kwargs):
+ """Build PyTorch DataLoader.
+
+ In distributed training, each GPU/process has a dataloader.
+ In non-distributed training, there is only one dataloader for all GPUs.
+
+ Args:
+ dataset (Dataset): A PyTorch dataset.
+ samples_per_gpu (int): Number of training samples on each GPU, i.e.,
+ batch size of each GPU.
+ workers_per_gpu (int): How many subprocesses to use for data loading
+ for each GPU.
+ num_gpus (int): Number of GPUs. Only used in non-distributed training.
+ dist (bool): Distributed training/test or not. Default: True.
+ shuffle (bool): Whether to shuffle the data at every epoch.
+ Default: True.
+ drop_last (bool): Whether to drop the last incomplete batch in epoch.
+ Default: True
+ pin_memory (bool): Whether to use pin_memory in DataLoader.
+ Default: True
+ kwargs: any keyword argument to be used to initialize DataLoader
+
+ Returns:
+ DataLoader: A PyTorch dataloader.
+ """
+ rank, world_size = get_dist_info()
+ if dist:
+ sampler = DistributedSampler(
+ dataset, world_size, rank, shuffle=shuffle, seed=seed)
+ shuffle = False
+ batch_size = samples_per_gpu
+ num_workers = workers_per_gpu
+ else:
+ sampler = None
+ batch_size = num_gpus * samples_per_gpu
+ num_workers = num_gpus * workers_per_gpu
+
+ init_fn = partial(
+ worker_init_fn, num_workers=num_workers, rank=rank,
+ seed=seed) if seed is not None else None
+
+ _, DataLoader = _get_dataloader()
+ data_loader = DataLoader(
+ dataset,
+ batch_size=batch_size,
+ sampler=sampler,
+ num_workers=num_workers,
+ collate_fn=partial(collate, samples_per_gpu=samples_per_gpu),
+ pin_memory=pin_memory,
+ shuffle=shuffle,
+ worker_init_fn=init_fn,
+ drop_last=drop_last,
+ **kwargs)
+
+ return data_loader
+
+
+def worker_init_fn(worker_id, num_workers, rank, seed):
+ """Init the random seed for various workers."""
+ # The seed of each worker equals to
+ # num_worker * rank + worker_id + user_seed
+ worker_seed = num_workers * rank + worker_id + seed
+ np.random.seed(worker_seed)
+ random.seed(worker_seed)
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/dataset_info.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/dataset_info.py
new file mode 100644
index 0000000..ef0d62e
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/dataset_info.py
@@ -0,0 +1,104 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import numpy as np
+
+
+class DatasetInfo:
+
+ def __init__(self, dataset_info):
+ self._dataset_info = dataset_info
+ self.dataset_name = self._dataset_info['dataset_name']
+ self.paper_info = self._dataset_info['paper_info']
+ self.keypoint_info = self._dataset_info['keypoint_info']
+ self.skeleton_info = self._dataset_info['skeleton_info']
+ self.joint_weights = np.array(
+ self._dataset_info['joint_weights'], dtype=np.float32)[:, None]
+
+ self.sigmas = np.array(self._dataset_info['sigmas'])
+
+ self._parse_keypoint_info()
+ self._parse_skeleton_info()
+
+ def _parse_skeleton_info(self):
+ """Parse skeleton information.
+
+ - link_num (int): number of links.
+ - skeleton (list((2,))): list of links (id).
+ - skeleton_name (list((2,))): list of links (name).
+ - pose_link_color (np.ndarray): the color of the link for
+ visualization.
+ """
+ self.link_num = len(self.skeleton_info.keys())
+ self.pose_link_color = []
+
+ self.skeleton_name = []
+ self.skeleton = []
+ for skid in self.skeleton_info.keys():
+ link = self.skeleton_info[skid]['link']
+ self.skeleton_name.append(link)
+ self.skeleton.append([
+ self.keypoint_name2id[link[0]], self.keypoint_name2id[link[1]]
+ ])
+ self.pose_link_color.append(self.skeleton_info[skid].get(
+ 'color', [255, 128, 0]))
+ self.pose_link_color = np.array(self.pose_link_color)
+
+ def _parse_keypoint_info(self):
+ """Parse keypoint information.
+
+ - keypoint_num (int): number of keypoints.
+ - keypoint_id2name (dict): mapping keypoint id to keypoint name.
+ - keypoint_name2id (dict): mapping keypoint name to keypoint id.
+ - upper_body_ids (list): a list of keypoints that belong to the
+ upper body.
+ - lower_body_ids (list): a list of keypoints that belong to the
+ lower body.
+ - flip_index (list): list of flip index (id)
+ - flip_pairs (list((2,))): list of flip pairs (id)
+ - flip_index_name (list): list of flip index (name)
+ - flip_pairs_name (list((2,))): list of flip pairs (name)
+ - pose_kpt_color (np.ndarray): the color of the keypoint for
+ visualization.
+ """
+
+ self.keypoint_num = len(self.keypoint_info.keys())
+ self.keypoint_id2name = {}
+ self.keypoint_name2id = {}
+
+ self.pose_kpt_color = []
+ self.upper_body_ids = []
+ self.lower_body_ids = []
+
+ self.flip_index_name = []
+ self.flip_pairs_name = []
+
+ for kid in self.keypoint_info.keys():
+
+ keypoint_name = self.keypoint_info[kid]['name']
+ self.keypoint_id2name[kid] = keypoint_name
+ self.keypoint_name2id[keypoint_name] = kid
+ self.pose_kpt_color.append(self.keypoint_info[kid].get(
+ 'color', [255, 128, 0]))
+
+ type = self.keypoint_info[kid].get('type', '')
+ if type == 'upper':
+ self.upper_body_ids.append(kid)
+ elif type == 'lower':
+ self.lower_body_ids.append(kid)
+ else:
+ pass
+
+ swap_keypoint = self.keypoint_info[kid].get('swap', '')
+ if swap_keypoint == keypoint_name or swap_keypoint == '':
+ self.flip_index_name.append(keypoint_name)
+ else:
+ self.flip_index_name.append(swap_keypoint)
+ if [swap_keypoint, keypoint_name] not in self.flip_pairs_name:
+ self.flip_pairs_name.append([keypoint_name, swap_keypoint])
+
+ self.flip_pairs = [[
+ self.keypoint_name2id[pair[0]], self.keypoint_name2id[pair[1]]
+ ] for pair in self.flip_pairs_name]
+ self.flip_index = [
+ self.keypoint_name2id[name] for name in self.flip_index_name
+ ]
+ self.pose_kpt_color = np.array(self.pose_kpt_color)
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/dataset_wrappers.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/dataset_wrappers.py
new file mode 100644
index 0000000..aaaa173
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/dataset_wrappers.py
@@ -0,0 +1,31 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from .builder import DATASETS
+
+
+@DATASETS.register_module()
+class RepeatDataset:
+ """A wrapper of repeated dataset.
+
+ The length of repeated dataset will be `times` larger than the original
+ dataset. This is useful when the data loading time is long but the dataset
+ is small. Using RepeatDataset can reduce the data loading time between
+ epochs.
+
+ Args:
+ dataset (:obj:`Dataset`): The dataset to be repeated.
+ times (int): Repeat times.
+ """
+
+ def __init__(self, dataset, times):
+ self.dataset = dataset
+ self.times = times
+
+ self._ori_len = len(self.dataset)
+
+ def __getitem__(self, idx):
+ """Get data."""
+ return self.dataset[idx % self._ori_len]
+
+ def __len__(self):
+ """Length after repetition."""
+ return self.times * self._ori_len
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/__init__.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/__init__.py
new file mode 100644
index 0000000..f3839e5
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/__init__.py
@@ -0,0 +1,45 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from ...deprecated import (TopDownFreiHandDataset, TopDownOneHand10KDataset,
+ TopDownPanopticDataset)
+from .animal import (AnimalATRWDataset, AnimalFlyDataset, AnimalHorse10Dataset,
+ AnimalLocustDataset, AnimalMacaqueDataset,
+ AnimalPoseDataset, AnimalZebraDataset)
+from .body3d import Body3DH36MDataset, Body3DMviewDirectPanopticDataset
+from .bottom_up import (BottomUpAicDataset, BottomUpCocoDataset,
+ BottomUpCocoWholeBodyDataset, BottomUpCrowdPoseDataset,
+ BottomUpMhpDataset)
+from .face import (Face300WDataset, FaceAFLWDataset, FaceCocoWholeBodyDataset,
+ FaceCOFWDataset, FaceWFLWDataset)
+from .fashion import DeepFashionDataset
+from .hand import (FreiHandDataset, HandCocoWholeBodyDataset,
+ InterHand2DDataset, InterHand3DDataset, OneHand10KDataset,
+ PanopticDataset)
+from .mesh import (MeshAdversarialDataset, MeshH36MDataset, MeshMixDataset,
+ MoshDataset)
+from .top_down import (TopDownAicDataset, TopDownCocoDataset,
+ TopDownCocoWholeBodyDataset, TopDownCrowdPoseDataset,
+ TopDownH36MDataset, TopDownHalpeDataset,
+ TopDownJhmdbDataset, TopDownMhpDataset,
+ TopDownMpiiDataset, TopDownMpiiTrbDataset,
+ TopDownOCHumanDataset, TopDownPoseTrack18Dataset,
+ TopDownPoseTrack18VideoDataset)
+
+__all__ = [
+ 'TopDownCocoDataset', 'BottomUpCocoDataset', 'BottomUpMhpDataset',
+ 'BottomUpAicDataset', 'BottomUpCocoWholeBodyDataset', 'TopDownMpiiDataset',
+ 'TopDownMpiiTrbDataset', 'OneHand10KDataset', 'PanopticDataset',
+ 'HandCocoWholeBodyDataset', 'FreiHandDataset', 'InterHand2DDataset',
+ 'InterHand3DDataset', 'TopDownOCHumanDataset', 'TopDownAicDataset',
+ 'TopDownCocoWholeBodyDataset', 'MeshH36MDataset', 'MeshMixDataset',
+ 'MoshDataset', 'MeshAdversarialDataset', 'TopDownCrowdPoseDataset',
+ 'BottomUpCrowdPoseDataset', 'TopDownFreiHandDataset',
+ 'TopDownOneHand10KDataset', 'TopDownPanopticDataset',
+ 'TopDownPoseTrack18Dataset', 'TopDownJhmdbDataset', 'TopDownMhpDataset',
+ 'DeepFashionDataset', 'Face300WDataset', 'FaceAFLWDataset',
+ 'FaceWFLWDataset', 'FaceCOFWDataset', 'FaceCocoWholeBodyDataset',
+ 'Body3DH36MDataset', 'AnimalHorse10Dataset', 'AnimalMacaqueDataset',
+ 'AnimalFlyDataset', 'AnimalLocustDataset', 'AnimalZebraDataset',
+ 'AnimalATRWDataset', 'AnimalPoseDataset', 'TopDownH36MDataset',
+ 'TopDownHalpeDataset', 'TopDownPoseTrack18VideoDataset',
+ 'Body3DMviewDirectPanopticDataset'
+]
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/animal/__init__.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/animal/__init__.py
new file mode 100644
index 0000000..185b935
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/animal/__init__.py
@@ -0,0 +1,15 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from .animal_ap10k_dataset import AnimalAP10KDataset
+from .animal_atrw_dataset import AnimalATRWDataset
+from .animal_fly_dataset import AnimalFlyDataset
+from .animal_horse10_dataset import AnimalHorse10Dataset
+from .animal_locust_dataset import AnimalLocustDataset
+from .animal_macaque_dataset import AnimalMacaqueDataset
+from .animal_pose_dataset import AnimalPoseDataset
+from .animal_zebra_dataset import AnimalZebraDataset
+
+__all__ = [
+ 'AnimalHorse10Dataset', 'AnimalMacaqueDataset', 'AnimalFlyDataset',
+ 'AnimalLocustDataset', 'AnimalZebraDataset', 'AnimalATRWDataset',
+ 'AnimalPoseDataset', 'AnimalAP10KDataset'
+]
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/animal/animal_ap10k_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/animal/animal_ap10k_dataset.py
new file mode 100644
index 0000000..11a1e73
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/animal/animal_ap10k_dataset.py
@@ -0,0 +1,367 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os.path as osp
+import tempfile
+import warnings
+from collections import OrderedDict, defaultdict
+
+import json_tricks as json
+import numpy as np
+from mmcv import Config, deprecated_api_warning
+from xtcocotools.cocoeval import COCOeval
+
+from ....core.post_processing import oks_nms, soft_oks_nms
+from ...builder import DATASETS
+from ..base import Kpt2dSviewRgbImgTopDownDataset
+
+
+@DATASETS.register_module()
+class AnimalAP10KDataset(Kpt2dSviewRgbImgTopDownDataset):
+ """AP-10K dataset for animal pose estimation.
+
+ "AP-10K: A Benchmark for Animal Pose Estimation in the Wild"
+ Neurips Dataset Track'2021.
+ More details can be found in the `paper
+ `__ .
+
+ The dataset loads raw features and apply specified transforms
+ to return a dict containing the image tensors and other information.
+
+ AP-10K keypoint indexes::
+
+ 0: 'L_Eye',
+ 1: 'R_Eye',
+ 2: 'Nose',
+ 3: 'Neck',
+ 4: 'root of tail',
+ 5: 'L_Shoulder',
+ 6: 'L_Elbow',
+ 7: 'L_F_Paw',
+ 8: 'R_Shoulder',
+ 9: 'R_Elbow',
+ 10: 'R_F_Paw,
+ 11: 'L_Hip',
+ 12: 'L_Knee',
+ 13: 'L_B_Paw',
+ 14: 'R_Hip',
+ 15: 'R_Knee',
+ 16: 'R_B_Paw'
+
+ Args:
+ ann_file (str): Path to the annotation file.
+ img_prefix (str): Path to a directory where images are held.
+ Default: None.
+ data_cfg (dict): config
+ pipeline (list[dict | callable]): A sequence of data transforms.
+ dataset_info (DatasetInfo): A class containing all dataset info.
+ test_mode (bool): Store True when building test or
+ validation dataset. Default: False.
+ """
+
+ def __init__(self,
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=None,
+ test_mode=False):
+
+ if dataset_info is None:
+ warnings.warn(
+ 'dataset_info is missing. '
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 '
+ 'for details.', DeprecationWarning)
+ cfg = Config.fromfile('configs/_base_/datasets/ap10k.py')
+ dataset_info = cfg._cfg_dict['dataset_info']
+
+ super().__init__(
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=dataset_info,
+ test_mode=test_mode)
+
+ self.use_gt_bbox = data_cfg['use_gt_bbox']
+ self.bbox_file = data_cfg['bbox_file']
+ self.det_bbox_thr = data_cfg.get('det_bbox_thr', 0.0)
+
+ self.use_nms = data_cfg.get('use_nms', True)
+ self.soft_nms = data_cfg['soft_nms']
+ self.nms_thr = data_cfg['nms_thr']
+ self.oks_thr = data_cfg['oks_thr']
+ self.vis_thr = data_cfg['vis_thr']
+
+ self.ann_info['use_different_joint_weights'] = False
+ self.db, self.id2Cat = self._get_db()
+
+ print(f'=> num_images: {self.num_images}')
+ print(f'=> load {len(self.db)} samples')
+
+ def _get_db(self):
+ """Load dataset."""
+ assert self.use_gt_bbox
+ gt_db, id2Cat = self._load_coco_keypoint_annotations()
+ return gt_db, id2Cat
+
+ def _load_coco_keypoint_annotations(self):
+ """Ground truth bbox and keypoints."""
+ gt_db, id2Cat = [], dict()
+ for img_id in self.img_ids:
+ db_tmp, id2Cat_tmp = self._load_coco_keypoint_annotation_kernel(
+ img_id)
+ gt_db.extend(db_tmp)
+ id2Cat.update({img_id: id2Cat_tmp})
+ return gt_db, id2Cat
+
+ def _load_coco_keypoint_annotation_kernel(self, img_id):
+ """load annotation from COCOAPI.
+
+ Note:
+ bbox:[x1, y1, w, h]
+ Args:
+ img_id: coco image id
+ Returns:
+ dict: db entry
+ """
+ img_ann = self.coco.loadImgs(img_id)[0]
+ width = img_ann['width']
+ height = img_ann['height']
+ num_joints = self.ann_info['num_joints']
+
+ ann_ids = self.coco.getAnnIds(imgIds=img_id, iscrowd=False)
+ objs = self.coco.loadAnns(ann_ids)
+
+ # sanitize bboxes
+ valid_objs = []
+ for obj in objs:
+ if 'bbox' not in obj:
+ continue
+ x, y, w, h = obj['bbox']
+ x1 = max(0, x)
+ y1 = max(0, y)
+ x2 = min(width - 1, x1 + max(0, w - 1))
+ y2 = min(height - 1, y1 + max(0, h - 1))
+ if ('area' not in obj or obj['area'] > 0) and x2 > x1 and y2 > y1:
+ obj['clean_bbox'] = [x1, y1, x2 - x1, y2 - y1]
+ valid_objs.append(obj)
+ objs = valid_objs
+
+ bbox_id = 0
+ rec = []
+ id2Cat = []
+ for obj in objs:
+ if 'keypoints' not in obj:
+ continue
+ if max(obj['keypoints']) == 0:
+ continue
+ if 'num_keypoints' in obj and obj['num_keypoints'] == 0:
+ continue
+ joints_3d = np.zeros((num_joints, 3), dtype=np.float32)
+ joints_3d_visible = np.zeros((num_joints, 3), dtype=np.float32)
+
+ keypoints = np.array(obj['keypoints']).reshape(-1, 3)
+ joints_3d[:, :2] = keypoints[:, :2]
+ joints_3d_visible[:, :2] = np.minimum(1, keypoints[:, 2:3])
+
+ center, scale = self._xywh2cs(*obj['clean_bbox'][:4])
+
+ image_file = osp.join(self.img_prefix, self.id2name[img_id])
+ rec.append({
+ 'image_file': image_file,
+ 'center': center,
+ 'scale': scale,
+ 'bbox': obj['clean_bbox'][:4],
+ 'rotation': 0,
+ 'joints_3d': joints_3d,
+ 'joints_3d_visible': joints_3d_visible,
+ 'dataset': self.dataset_name,
+ 'bbox_score': 1,
+ 'bbox_id': bbox_id
+ })
+ category = obj['category_id']
+ id2Cat.append({
+ 'image_file': image_file,
+ 'bbox_id': bbox_id,
+ 'category': category,
+ })
+ bbox_id = bbox_id + 1
+
+ return rec, id2Cat
+
+ @deprecated_api_warning(name_dict=dict(outputs='results'))
+ def evaluate(self, results, res_folder=None, metric='mAP', **kwargs):
+ """Evaluate coco keypoint results. The pose prediction results will be
+ saved in ``${res_folder}/result_keypoints.json``.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+ - heatmap height: H
+ - heatmap width: W
+
+ Args:
+ results (list[dict]): Testing results containing the following
+ items:
+
+ - preds (np.ndarray[N,K,3]): The first two dimensions are \
+ coordinates, score is the third dimension of the array.
+ - boxes (np.ndarray[N,6]): [center[0], center[1], scale[0], \
+ scale[1],area, score]
+ - image_paths (list[str]): For example, ['data/coco/val2017\
+ /000000393226.jpg']
+ - heatmap (np.ndarray[N, K, H, W]): model output heatmap
+ - bbox_id (list(int)).
+ res_folder (str, optional): The folder to save the testing
+ results. If not specified, a temp folder will be created.
+ Default: None.
+ metric (str | list[str]): Metric to be performed. Defaults: 'mAP'.
+
+ Returns:
+ dict: Evaluation results for evaluation metric.
+ """
+ metrics = metric if isinstance(metric, list) else [metric]
+ allowed_metrics = ['mAP']
+ for metric in metrics:
+ if metric not in allowed_metrics:
+ raise KeyError(f'metric {metric} is not supported')
+
+ if res_folder is not None:
+ tmp_folder = None
+ res_file = osp.join(res_folder, 'result_keypoints.json')
+ else:
+ tmp_folder = tempfile.TemporaryDirectory()
+ res_file = osp.join(tmp_folder.name, 'result_keypoints.json')
+
+ kpts = defaultdict(list)
+
+ for result in results:
+ preds = result['preds']
+ boxes = result['boxes']
+ image_paths = result['image_paths']
+ bbox_ids = result['bbox_ids']
+
+ batch_size = len(image_paths)
+ for i in range(batch_size):
+ image_id = self.name2id[image_paths[i][len(self.img_prefix):]]
+ cat = self.id2Cat[image_id][bbox_ids[i]]['category']
+ kpts[image_id].append({
+ 'keypoints': preds[i],
+ 'center': boxes[i][0:2],
+ 'scale': boxes[i][2:4],
+ 'area': boxes[i][4],
+ 'score': boxes[i][5],
+ 'image_id': image_id,
+ 'bbox_id': bbox_ids[i],
+ 'category': cat
+ })
+ kpts = self._sort_and_unique_bboxes(kpts)
+
+ # rescoring and oks nms
+ num_joints = self.ann_info['num_joints']
+ vis_thr = self.vis_thr
+ oks_thr = self.oks_thr
+ valid_kpts = []
+ for image_id in kpts.keys():
+ img_kpts = kpts[image_id]
+ for n_p in img_kpts:
+ box_score = n_p['score']
+ kpt_score = 0
+ valid_num = 0
+ for n_jt in range(0, num_joints):
+ t_s = n_p['keypoints'][n_jt][2]
+ if t_s > vis_thr:
+ kpt_score = kpt_score + t_s
+ valid_num = valid_num + 1
+ if valid_num != 0:
+ kpt_score = kpt_score / valid_num
+ # rescoring
+ n_p['score'] = kpt_score * box_score
+
+ if self.use_nms:
+ nms = soft_oks_nms if self.soft_nms else oks_nms
+ keep = nms(list(img_kpts), oks_thr, sigmas=self.sigmas)
+ valid_kpts.append([img_kpts[_keep] for _keep in keep])
+ else:
+ valid_kpts.append(img_kpts)
+
+ self._write_coco_keypoint_results(valid_kpts, res_file)
+
+ info_str = self._do_python_keypoint_eval(res_file)
+ name_value = OrderedDict(info_str)
+
+ if tmp_folder is not None:
+ tmp_folder.cleanup()
+
+ return name_value
+
+ def _write_coco_keypoint_results(self, keypoints, res_file):
+ """Write results into a json file."""
+ data_pack = [{
+ 'cat_id': self._class_to_coco_ind[cls],
+ 'cls_ind': cls_ind,
+ 'cls': cls,
+ 'ann_type': 'keypoints',
+ 'keypoints': keypoints
+ } for cls_ind, cls in enumerate(self.classes)
+ if not cls == '__background__']
+
+ results = self._coco_keypoint_results_one_category_kernel(data_pack[0])
+
+ with open(res_file, 'w') as f:
+ json.dump(results, f, sort_keys=True, indent=4)
+
+ def _coco_keypoint_results_one_category_kernel(self, data_pack):
+ """Get coco keypoint results."""
+ keypoints = data_pack['keypoints']
+ cat_results = []
+
+ for img_kpts in keypoints:
+ if len(img_kpts) == 0:
+ continue
+
+ _key_points = np.array(
+ [img_kpt['keypoints'] for img_kpt in img_kpts])
+ key_points = _key_points.reshape(-1,
+ self.ann_info['num_joints'] * 3)
+
+ result = [{
+ 'image_id': img_kpt['image_id'],
+ 'category_id': img_kpt['category'],
+ 'keypoints': key_point.tolist(),
+ 'score': float(img_kpt['score']),
+ 'center': img_kpt['center'].tolist(),
+ 'scale': img_kpt['scale'].tolist()
+ } for img_kpt, key_point in zip(img_kpts, key_points)]
+
+ cat_results.extend(result)
+
+ return cat_results
+
+ def _do_python_keypoint_eval(self, res_file):
+ """Keypoint evaluation using COCOAPI."""
+ coco_det = self.coco.loadRes(res_file)
+ coco_eval = COCOeval(self.coco, coco_det, 'keypoints', self.sigmas)
+ coco_eval.params.useSegm = None
+ coco_eval.evaluate()
+ coco_eval.accumulate()
+ coco_eval.summarize()
+
+ stats_names = [
+ 'AP', 'AP .5', 'AP .75', 'AP (M)', 'AP (L)', 'AR', 'AR .5',
+ 'AR .75', 'AR (M)', 'AR (L)'
+ ]
+
+ info_str = list(zip(stats_names, coco_eval.stats))
+
+ return info_str
+
+ def _sort_and_unique_bboxes(self, kpts, key='bbox_id'):
+ """sort kpts and remove the repeated ones."""
+ for img_id, persons in kpts.items():
+ num = len(persons)
+ kpts[img_id] = sorted(kpts[img_id], key=lambda x: x[key])
+ for i in range(num - 1, 0, -1):
+ if kpts[img_id][i][key] == kpts[img_id][i - 1][key]:
+ del kpts[img_id][i]
+
+ return kpts
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/animal/animal_atrw_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/animal/animal_atrw_dataset.py
new file mode 100644
index 0000000..edfd3f9
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/animal/animal_atrw_dataset.py
@@ -0,0 +1,353 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os.path as osp
+import tempfile
+import warnings
+from collections import OrderedDict, defaultdict
+
+import json_tricks as json
+import numpy as np
+from mmcv import Config, deprecated_api_warning
+from xtcocotools.cocoeval import COCOeval
+
+from ....core.post_processing import oks_nms, soft_oks_nms
+from ...builder import DATASETS
+from ..base import Kpt2dSviewRgbImgTopDownDataset
+
+
+@DATASETS.register_module()
+class AnimalATRWDataset(Kpt2dSviewRgbImgTopDownDataset):
+ """ATRW dataset for animal pose estimation.
+
+ "ATRW: A Benchmark for Amur Tiger Re-identification in the Wild"
+ ACM MM'2020.
+ More details can be found in the `paper
+ `__ .
+
+ The dataset loads raw features and apply specified transforms
+ to return a dict containing the image tensors and other information.
+
+ ATRW keypoint indexes::
+
+ 0: "left_ear",
+ 1: "right_ear",
+ 2: "nose",
+ 3: "right_shoulder",
+ 4: "right_front_paw",
+ 5: "left_shoulder",
+ 6: "left_front_paw",
+ 7: "right_hip",
+ 8: "right_knee",
+ 9: "right_back_paw",
+ 10: "left_hip",
+ 11: "left_knee",
+ 12: "left_back_paw",
+ 13: "tail",
+ 14: "center"
+
+ Args:
+ ann_file (str): Path to the annotation file.
+ img_prefix (str): Path to a directory where images are held.
+ Default: None.
+ data_cfg (dict): config
+ pipeline (list[dict | callable]): A sequence of data transforms.
+ dataset_info (DatasetInfo): A class containing all dataset info.
+ test_mode (bool): Store True when building test or
+ validation dataset. Default: False.
+ """
+
+ def __init__(self,
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=None,
+ test_mode=False):
+
+ if dataset_info is None:
+ warnings.warn(
+ 'dataset_info is missing. '
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 '
+ 'for details.', DeprecationWarning)
+ cfg = Config.fromfile('configs/_base_/datasets/atrw.py')
+ dataset_info = cfg._cfg_dict['dataset_info']
+
+ super().__init__(
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=dataset_info,
+ test_mode=test_mode)
+
+ self.use_gt_bbox = data_cfg['use_gt_bbox']
+ self.bbox_file = data_cfg['bbox_file']
+ self.det_bbox_thr = data_cfg.get('det_bbox_thr', 0.0)
+ self.use_nms = data_cfg.get('use_nms', True)
+ self.soft_nms = data_cfg['soft_nms']
+ self.nms_thr = data_cfg['nms_thr']
+ self.oks_thr = data_cfg['oks_thr']
+ self.vis_thr = data_cfg['vis_thr']
+
+ self.ann_info['use_different_joint_weights'] = False
+ self.db = self._get_db()
+
+ print(f'=> num_images: {self.num_images}')
+ print(f'=> load {len(self.db)} samples')
+
+ def _get_db(self):
+ """Load dataset."""
+ assert self.use_gt_bbox
+ gt_db = self._load_coco_keypoint_annotations()
+ return gt_db
+
+ def _load_coco_keypoint_annotations(self):
+ """Ground truth bbox and keypoints."""
+ gt_db = []
+ for img_id in self.img_ids:
+ gt_db.extend(self._load_coco_keypoint_annotation_kernel(img_id))
+ return gt_db
+
+ def _load_coco_keypoint_annotation_kernel(self, img_id):
+ """load annotation from COCOAPI.
+
+ Note:
+ bbox:[x1, y1, w, h]
+ Args:
+ img_id: coco image id
+ Returns:
+ dict: db entry
+ """
+ img_ann = self.coco.loadImgs(img_id)[0]
+ width = img_ann['width']
+ height = img_ann['height']
+ num_joints = self.ann_info['num_joints']
+
+ ann_ids = self.coco.getAnnIds(imgIds=img_id, iscrowd=False)
+ objs = self.coco.loadAnns(ann_ids)
+
+ # sanitize bboxes
+ valid_objs = []
+ for obj in objs:
+ if 'bbox' not in obj:
+ continue
+ x, y, w, h = obj['bbox']
+ x1 = max(0, x)
+ y1 = max(0, y)
+ x2 = min(width - 1, x1 + max(0, w - 1))
+ y2 = min(height - 1, y1 + max(0, h - 1))
+ if ('area' not in obj or obj['area'] > 0) and x2 > x1 and y2 > y1:
+ obj['clean_bbox'] = [x1, y1, x2 - x1, y2 - y1]
+ valid_objs.append(obj)
+ objs = valid_objs
+
+ bbox_id = 0
+ rec = []
+ for obj in objs:
+ if 'keypoints' not in obj:
+ continue
+ if max(obj['keypoints']) == 0:
+ continue
+ if 'num_keypoints' in obj and obj['num_keypoints'] == 0:
+ continue
+ joints_3d = np.zeros((num_joints, 3), dtype=np.float32)
+ joints_3d_visible = np.zeros((num_joints, 3), dtype=np.float32)
+
+ keypoints = np.array(obj['keypoints']).reshape(-1, 3)
+ joints_3d[:, :2] = keypoints[:, :2]
+ joints_3d_visible[:, :2] = np.minimum(1, keypoints[:, 2:3])
+
+ center, scale = self._xywh2cs(*obj['clean_bbox'][:4], padding=1.0)
+
+ image_file = osp.join(self.img_prefix, self.id2name[img_id])
+ rec.append({
+ 'image_file': image_file,
+ 'center': center,
+ 'scale': scale,
+ 'bbox': obj['clean_bbox'][:4],
+ 'rotation': 0,
+ 'joints_3d': joints_3d,
+ 'joints_3d_visible': joints_3d_visible,
+ 'dataset': self.dataset_name,
+ 'bbox_score': 1,
+ 'bbox_id': bbox_id
+ })
+ bbox_id = bbox_id + 1
+
+ return rec
+
+ @deprecated_api_warning(name_dict=dict(outputs='results'))
+ def evaluate(self, results, res_folder=None, metric='mAP', **kwargs):
+ """Evaluate coco keypoint results. The pose prediction results will be
+ saved in ``${res_folder}/result_keypoints.json``.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+ - heatmap height: H
+ - heatmap width: W
+
+ Args:
+ results (list[dict]): Testing results containing the following
+ items:
+
+ - preds (np.ndarray[N,K,3]): The first two dimensions are \
+ coordinates, score is the third dimension of the array.
+ - boxes (np.ndarray[N,6]): [center[0], center[1], scale[0], \
+ scale[1],area, score]
+ - image_paths (list[str]): For example, ['data/coco/val2017\
+ /000000393226.jpg']
+ - heatmap (np.ndarray[N, K, H, W]): model output heatmap
+ - bbox_id (list(int)).
+ res_folder (str, optional): The folder to save the testing
+ results. If not specified, a temp folder will be created.
+ Default: None.
+ metric (str | list[str]): Metric to be performed. Defaults: 'mAP'.
+
+ Returns:
+ dict: Evaluation results for evaluation metric.
+ """
+ metrics = metric if isinstance(metric, list) else [metric]
+ allowed_metrics = ['mAP']
+ for metric in metrics:
+ if metric not in allowed_metrics:
+ raise KeyError(f'metric {metric} is not supported')
+
+ if res_folder is not None:
+ tmp_folder = None
+ res_file = osp.join(res_folder, 'result_keypoints.json')
+ else:
+ tmp_folder = tempfile.TemporaryDirectory()
+ res_file = osp.join(tmp_folder.name, 'result_keypoints.json')
+
+ kpts = defaultdict(list)
+
+ for result in results:
+ preds = result['preds']
+ boxes = result['boxes']
+ image_paths = result['image_paths']
+ bbox_ids = result['bbox_ids']
+
+ batch_size = len(image_paths)
+ for i in range(batch_size):
+ image_id = self.name2id[image_paths[i][len(self.img_prefix):]]
+ kpts[image_id].append({
+ 'keypoints': preds[i],
+ 'center': boxes[i][0:2],
+ 'scale': boxes[i][2:4],
+ 'area': boxes[i][4],
+ 'score': boxes[i][5],
+ 'image_id': image_id,
+ 'bbox_id': bbox_ids[i]
+ })
+ kpts = self._sort_and_unique_bboxes(kpts)
+
+ # rescoring and oks nms
+ num_joints = self.ann_info['num_joints']
+ vis_thr = self.vis_thr
+ oks_thr = self.oks_thr
+ valid_kpts = []
+ for image_id in kpts.keys():
+ img_kpts = kpts[image_id]
+ for n_p in img_kpts:
+ box_score = n_p['score']
+ kpt_score = 0
+ valid_num = 0
+ for n_jt in range(0, num_joints):
+ t_s = n_p['keypoints'][n_jt][2]
+ if t_s > vis_thr:
+ kpt_score = kpt_score + t_s
+ valid_num = valid_num + 1
+ if valid_num != 0:
+ kpt_score = kpt_score / valid_num
+ # rescoring
+ n_p['score'] = kpt_score * box_score
+
+ if self.use_nms:
+ nms = soft_oks_nms if self.soft_nms else oks_nms
+ keep = nms(list(img_kpts), oks_thr, sigmas=self.sigmas)
+ valid_kpts.append([img_kpts[_keep] for _keep in keep])
+ else:
+ valid_kpts.append(img_kpts)
+
+ self._write_coco_keypoint_results(valid_kpts, res_file)
+
+ info_str = self._do_python_keypoint_eval(res_file)
+ name_value = OrderedDict(info_str)
+
+ if tmp_folder is not None:
+ tmp_folder.cleanup()
+
+ return name_value
+
+ def _write_coco_keypoint_results(self, keypoints, res_file):
+ """Write results into a json file."""
+ data_pack = [{
+ 'cat_id': self._class_to_coco_ind[cls],
+ 'cls_ind': cls_ind,
+ 'cls': cls,
+ 'ann_type': 'keypoints',
+ 'keypoints': keypoints
+ } for cls_ind, cls in enumerate(self.classes)
+ if not cls == '__background__']
+
+ results = self._coco_keypoint_results_one_category_kernel(data_pack[0])
+
+ with open(res_file, 'w') as f:
+ json.dump(results, f, sort_keys=True, indent=4)
+
+ def _coco_keypoint_results_one_category_kernel(self, data_pack):
+ """Get coco keypoint results."""
+ cat_id = data_pack['cat_id']
+ keypoints = data_pack['keypoints']
+ cat_results = []
+
+ for img_kpts in keypoints:
+ if len(img_kpts) == 0:
+ continue
+
+ _key_points = np.array(
+ [img_kpt['keypoints'] for img_kpt in img_kpts])
+ key_points = _key_points.reshape(-1,
+ self.ann_info['num_joints'] * 3)
+
+ result = [{
+ 'image_id': img_kpt['image_id'],
+ 'category_id': cat_id,
+ 'keypoints': key_point.tolist(),
+ 'score': float(img_kpt['score']),
+ 'center': img_kpt['center'].tolist(),
+ 'scale': img_kpt['scale'].tolist()
+ } for img_kpt, key_point in zip(img_kpts, key_points)]
+
+ cat_results.extend(result)
+
+ return cat_results
+
+ def _do_python_keypoint_eval(self, res_file):
+ """Keypoint evaluation using COCOAPI."""
+ coco_det = self.coco.loadRes(res_file)
+ coco_eval = COCOeval(self.coco, coco_det, 'keypoints', self.sigmas)
+ coco_eval.params.useSegm = None
+ coco_eval.evaluate()
+ coco_eval.accumulate()
+ coco_eval.summarize()
+
+ stats_names = [
+ 'AP', 'AP .5', 'AP .75', 'AP (M)', 'AP (L)', 'AR', 'AR .5',
+ 'AR .75', 'AR (M)', 'AR (L)'
+ ]
+
+ info_str = list(zip(stats_names, coco_eval.stats))
+
+ return info_str
+
+ def _sort_and_unique_bboxes(self, kpts, key='bbox_id'):
+ """sort kpts and remove the repeated ones."""
+ for img_id, persons in kpts.items():
+ num = len(persons)
+ kpts[img_id] = sorted(kpts[img_id], key=lambda x: x[key])
+ for i in range(num - 1, 0, -1):
+ if kpts[img_id][i][key] == kpts[img_id][i - 1][key]:
+ del kpts[img_id][i]
+
+ return kpts
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/animal/animal_base_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/animal/animal_base_dataset.py
new file mode 100644
index 0000000..e191882
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/animal/animal_base_dataset.py
@@ -0,0 +1,16 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from abc import ABCMeta
+
+from torch.utils.data import Dataset
+
+
+class AnimalBaseDataset(Dataset, metaclass=ABCMeta):
+ """This class has been deprecated and replaced by
+ Kpt2dSviewRgbImgTopDownDataset."""
+
+ def __init__(self, *args, **kwargs):
+ raise (ImportError(
+ 'AnimalBaseDataset has been replaced by '
+ 'Kpt2dSviewRgbImgTopDownDataset,'
+ 'check https://github.com/open-mmlab/mmpose/pull/663 for details.')
+ )
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/animal/animal_fly_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/animal/animal_fly_dataset.py
new file mode 100644
index 0000000..f414117
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/animal/animal_fly_dataset.py
@@ -0,0 +1,215 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os.path as osp
+import tempfile
+import warnings
+from collections import OrderedDict
+
+import numpy as np
+from mmcv import Config, deprecated_api_warning
+
+from ...builder import DATASETS
+from ..base import Kpt2dSviewRgbImgTopDownDataset
+
+
+@DATASETS.register_module()
+class AnimalFlyDataset(Kpt2dSviewRgbImgTopDownDataset):
+ """AnimalFlyDataset for animal pose estimation.
+
+ "Fast animal pose estimation using deep neural networks"
+ Nature methods'2019. More details can be found in the `paper
+ `__ .
+
+ The dataset loads raw features and apply specified transforms
+ to return a dict containing the image tensors and other information.
+
+ Vinegar Fly keypoint indexes::
+
+ 0: "head",
+ 1: "eyeL",
+ 2: "eyeR",
+ 3: "neck",
+ 4: "thorax",
+ 5: "abdomen",
+ 6: "forelegR1",
+ 7: "forelegR2",
+ 8: "forelegR3",
+ 9: "forelegR4",
+ 10: "midlegR1",
+ 11: "midlegR2",
+ 12: "midlegR3",
+ 13: "midlegR4",
+ 14: "hindlegR1",
+ 15: "hindlegR2",
+ 16: "hindlegR3",
+ 17: "hindlegR4",
+ 18: "forelegL1",
+ 19: "forelegL2",
+ 20: "forelegL3",
+ 21: "forelegL4",
+ 22: "midlegL1",
+ 23: "midlegL2",
+ 24: "midlegL3",
+ 25: "midlegL4",
+ 26: "hindlegL1",
+ 27: "hindlegL2",
+ 28: "hindlegL3",
+ 29: "hindlegL4",
+ 30: "wingL",
+ 31: "wingR"
+
+ Args:
+ ann_file (str): Path to the annotation file.
+ img_prefix (str): Path to a directory where images are held.
+ Default: None.
+ data_cfg (dict): config
+ pipeline (list[dict | callable]): A sequence of data transforms.
+ dataset_info (DatasetInfo): A class containing all dataset info.
+ test_mode (bool): Store True when building test or
+ validation dataset. Default: False.
+ """
+
+ def __init__(self,
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=None,
+ test_mode=False):
+
+ if dataset_info is None:
+ warnings.warn(
+ 'dataset_info is missing. '
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 '
+ 'for details.', DeprecationWarning)
+ cfg = Config.fromfile('configs/_base_/datasets/fly.py')
+ dataset_info = cfg._cfg_dict['dataset_info']
+
+ super().__init__(
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=dataset_info,
+ test_mode=test_mode)
+
+ self.ann_info['use_different_joint_weights'] = False
+ self.db = self._get_db()
+
+ print(f'=> num_images: {self.num_images}')
+ print(f'=> load {len(self.db)} samples')
+
+ def _get_db(self):
+ """Load dataset."""
+ gt_db = []
+ bbox_id = 0
+ num_joints = self.ann_info['num_joints']
+ for img_id in self.img_ids:
+
+ ann_ids = self.coco.getAnnIds(imgIds=img_id, iscrowd=False)
+ objs = self.coco.loadAnns(ann_ids)
+
+ for obj in objs:
+ if max(obj['keypoints']) == 0:
+ continue
+ joints_3d = np.zeros((num_joints, 3), dtype=np.float32)
+ joints_3d_visible = np.zeros((num_joints, 3), dtype=np.float32)
+
+ keypoints = np.array(obj['keypoints']).reshape(-1, 3)
+ joints_3d[:, :2] = keypoints[:, :2]
+ joints_3d_visible[:, :2] = np.minimum(1, keypoints[:, 2:3])
+
+ # the ori image is 192x192
+ center, scale = self._xywh2cs(0, 0, 192, 192, 0.8)
+
+ image_file = osp.join(self.img_prefix, self.id2name[img_id])
+
+ gt_db.append({
+ 'image_file': image_file,
+ 'center': center,
+ 'scale': scale,
+ 'rotation': 0,
+ 'joints_3d': joints_3d,
+ 'joints_3d_visible': joints_3d_visible,
+ 'dataset': self.dataset_name,
+ 'bbox': obj['bbox'],
+ 'bbox_score': 1,
+ 'bbox_id': bbox_id
+ })
+ bbox_id = bbox_id + 1
+ gt_db = sorted(gt_db, key=lambda x: x['bbox_id'])
+
+ return gt_db
+
+ @deprecated_api_warning(name_dict=dict(outputs='results'))
+ def evaluate(self, results, res_folder=None, metric='PCK', **kwargs):
+ """Evaluate Fly keypoint results. The pose prediction results will be
+ saved in ``${res_folder}/result_keypoints.json``.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+ - heatmap height: H
+ - heatmap width: W
+
+ Args:
+ results (list[dict]): Testing results containing the following
+ items:
+
+ - preds (np.ndarray[N,K,3]): The first two dimensions are \
+ coordinates, score is the third dimension of the array.
+ - boxes (np.ndarray[N,6]): [center[0], center[1], scale[0], \
+ scale[1],area, score]
+ - image_paths (list[str]): For example, ['Test/source/0.jpg']
+ - output_heatmap (np.ndarray[N, K, H, W]): model outputs.
+
+ res_folder (str): Path of directory to save the results.
+ metric (str | list[str]): Metric to be performed.
+ Options: 'PCK', 'AUC', 'EPE'.
+
+ Returns:
+ dict: Evaluation results for evaluation metric.
+ """
+ metrics = metric if isinstance(metric, list) else [metric]
+ allowed_metrics = ['PCK', 'AUC', 'EPE']
+ for metric in metrics:
+ if metric not in allowed_metrics:
+ raise KeyError(f'metric {metric} is not supported')
+
+ if res_folder is not None:
+ tmp_folder = None
+ res_file = osp.join(res_folder, 'result_keypoints.json')
+ else:
+ tmp_folder = tempfile.TemporaryDirectory()
+ res_file = osp.join(tmp_folder.name, 'result_keypoints.json')
+
+ kpts = []
+ for result in results:
+ preds = result['preds']
+ boxes = result['boxes']
+ image_paths = result['image_paths']
+ bbox_ids = result['bbox_ids']
+
+ batch_size = len(image_paths)
+ for i in range(batch_size):
+ image_id = self.name2id[image_paths[i][len(self.img_prefix):]]
+
+ kpts.append({
+ 'keypoints': preds[i].tolist(),
+ 'center': boxes[i][0:2].tolist(),
+ 'scale': boxes[i][2:4].tolist(),
+ 'area': float(boxes[i][4]),
+ 'score': float(boxes[i][5]),
+ 'image_id': image_id,
+ 'bbox_id': bbox_ids[i]
+ })
+ kpts = self._sort_and_unique_bboxes(kpts)
+
+ self._write_keypoint_results(kpts, res_file)
+ info_str = self._report_metric(res_file, metrics)
+ name_value = OrderedDict(info_str)
+
+ if tmp_folder is not None:
+ tmp_folder.cleanup()
+
+ return name_value
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/animal/animal_horse10_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/animal/animal_horse10_dataset.py
new file mode 100644
index 0000000..d2bf198
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/animal/animal_horse10_dataset.py
@@ -0,0 +1,220 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os.path as osp
+import tempfile
+import warnings
+from collections import OrderedDict
+
+import numpy as np
+from mmcv import Config, deprecated_api_warning
+
+from ...builder import DATASETS
+from ..base import Kpt2dSviewRgbImgTopDownDataset
+
+
+@DATASETS.register_module()
+class AnimalHorse10Dataset(Kpt2dSviewRgbImgTopDownDataset):
+ """AnimalHorse10Dataset for animal pose estimation.
+
+ "Pretraining boosts out-of-domain robustness for pose estimation"
+ WACV'2021. More details can be found in the `paper
+ `__ .
+
+ The dataset loads raw features and apply specified transforms
+ to return a dict containing the image tensors and other information.
+
+ Horse-10 keypoint indexes::
+
+ 0: 'Nose',
+ 1: 'Eye',
+ 2: 'Nearknee',
+ 3: 'Nearfrontfetlock',
+ 4: 'Nearfrontfoot',
+ 5: 'Offknee',
+ 6: 'Offfrontfetlock',
+ 7: 'Offfrontfoot',
+ 8: 'Shoulder',
+ 9: 'Midshoulder',
+ 10: 'Elbow',
+ 11: 'Girth',
+ 12: 'Wither',
+ 13: 'Nearhindhock',
+ 14: 'Nearhindfetlock',
+ 15: 'Nearhindfoot',
+ 16: 'Hip',
+ 17: 'Stifle',
+ 18: 'Offhindhock',
+ 19: 'Offhindfetlock',
+ 20: 'Offhindfoot',
+ 21: 'Ischium'
+
+ Args:
+ ann_file (str): Path to the annotation file.
+ img_prefix (str): Path to a directory where images are held.
+ Default: None.
+ data_cfg (dict): config
+ pipeline (list[dict | callable]): A sequence of data transforms.
+ dataset_info (DatasetInfo): A class containing all dataset info.
+ test_mode (bool): Store True when building test or
+ validation dataset. Default: False.
+ """
+
+ def __init__(self,
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=None,
+ test_mode=False):
+
+ if dataset_info is None:
+ warnings.warn(
+ 'dataset_info is missing. '
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 '
+ 'for details.', DeprecationWarning)
+ cfg = Config.fromfile('configs/_base_/datasets/horse10.py')
+ dataset_info = cfg._cfg_dict['dataset_info']
+
+ super().__init__(
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=dataset_info,
+ test_mode=test_mode)
+
+ self.ann_info['use_different_joint_weights'] = False
+ self.db = self._get_db()
+
+ print(f'=> num_images: {self.num_images}')
+ print(f'=> load {len(self.db)} samples')
+
+ def _get_db(self):
+ """Load dataset."""
+ gt_db = []
+ bbox_id = 0
+ num_joints = self.ann_info['num_joints']
+ for img_id in self.img_ids:
+
+ ann_ids = self.coco.getAnnIds(imgIds=img_id, iscrowd=False)
+ objs = self.coco.loadAnns(ann_ids)
+
+ for obj in objs:
+ if max(obj['keypoints']) == 0:
+ continue
+ joints_3d = np.zeros((num_joints, 3), dtype=np.float32)
+ joints_3d_visible = np.zeros((num_joints, 3), dtype=np.float32)
+
+ keypoints = np.array(obj['keypoints']).reshape(-1, 3)
+ joints_3d[:, :2] = keypoints[:, :2]
+ joints_3d_visible[:, :2] = np.minimum(1, keypoints[:, 2:3])
+
+ # use 1.25 padded bbox as input
+ center, scale = self._xywh2cs(*obj['bbox'][:4], 1.25)
+
+ image_file = osp.join(self.img_prefix, self.id2name[img_id])
+
+ gt_db.append({
+ 'image_file': image_file,
+ 'center': center,
+ 'scale': scale,
+ 'rotation': 0,
+ 'joints_3d': joints_3d,
+ 'joints_3d_visible': joints_3d_visible,
+ 'dataset': self.dataset_name,
+ 'bbox': obj['bbox'],
+ 'bbox_score': 1,
+ 'bbox_id': bbox_id
+ })
+ bbox_id = bbox_id + 1
+ gt_db = sorted(gt_db, key=lambda x: x['bbox_id'])
+
+ return gt_db
+
+ def _get_normalize_factor(self, gts):
+ """Get inter-ocular distance as the normalize factor, measured as the
+ Euclidean distance between the outer corners of the eyes.
+
+ Args:
+ gts (np.ndarray[N, K, 2]): Groundtruth keypoint location.
+
+ Returns:
+ np.ndarray[N, 2]: normalized factor
+ """
+
+ interocular = np.linalg.norm(
+ gts[:, 0, :] - gts[:, 1, :], axis=1, keepdims=True)
+ return np.tile(interocular, [1, 2])
+
+ @deprecated_api_warning(name_dict=dict(outputs='results'))
+ def evaluate(self, results, res_folder=None, metric='PCK', **kwargs):
+ """Evaluate horse-10 keypoint results. The pose prediction results will
+ be saved in ``${res_folder}/result_keypoints.json``.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+ - heatmap height: H
+ - heatmap width: W
+
+ Args:
+ results (list[dict]): Testing results containing the following
+ items:
+
+ - preds (np.ndarray[N,K,3]): The first two dimensions are \
+ coordinates, score is the third dimension of the array.
+ - boxes (np.ndarray[N,6]): [center[0], center[1], scale[0], \
+ scale[1],area, score]
+ - image_paths (list[str]): For example, ['Test/source/0.jpg']
+ - output_heatmap (np.ndarray[N, K, H, W]): model outputs.
+ res_folder (str, optional): The folder to save the testing
+ results. If not specified, a temp folder will be created.
+ Default: None.
+ metric (str | list[str]): Metric to be performed.
+ Options: 'PCK', 'NME'.
+
+ Returns:
+ dict: Evaluation results for evaluation metric.
+ """
+ metrics = metric if isinstance(metric, list) else [metric]
+ allowed_metrics = ['PCK', 'NME']
+ for metric in metrics:
+ if metric not in allowed_metrics:
+ raise KeyError(f'metric {metric} is not supported')
+
+ if res_folder is not None:
+ tmp_folder = None
+ res_file = osp.join(res_folder, 'result_keypoints.json')
+ else:
+ tmp_folder = tempfile.TemporaryDirectory()
+ res_file = osp.join(tmp_folder.name, 'result_keypoints.json')
+
+ kpts = []
+ for result in results:
+ preds = result['preds']
+ boxes = result['boxes']
+ image_paths = result['image_paths']
+ bbox_ids = result['bbox_ids']
+
+ batch_size = len(image_paths)
+ for i in range(batch_size):
+ image_id = self.name2id[image_paths[i][len(self.img_prefix):]]
+
+ kpts.append({
+ 'keypoints': preds[i].tolist(),
+ 'center': boxes[i][0:2].tolist(),
+ 'scale': boxes[i][2:4].tolist(),
+ 'area': float(boxes[i][4]),
+ 'score': float(boxes[i][5]),
+ 'image_id': image_id,
+ 'bbox_id': bbox_ids[i]
+ })
+ kpts = self._sort_and_unique_bboxes(kpts)
+
+ self._write_keypoint_results(kpts, res_file)
+ info_str = self._report_metric(res_file, metrics)
+ name_value = OrderedDict(info_str)
+
+ if tmp_folder is not None:
+ tmp_folder.cleanup()
+
+ return name_value
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/animal/animal_locust_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/animal/animal_locust_dataset.py
new file mode 100644
index 0000000..95fb6ac
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/animal/animal_locust_dataset.py
@@ -0,0 +1,218 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os.path as osp
+import tempfile
+import warnings
+from collections import OrderedDict
+
+import numpy as np
+from mmcv import Config, deprecated_api_warning
+
+from ...builder import DATASETS
+from ..base import Kpt2dSviewRgbImgTopDownDataset
+
+
+@DATASETS.register_module()
+class AnimalLocustDataset(Kpt2dSviewRgbImgTopDownDataset):
+ """AnimalLocustDataset for animal pose estimation.
+
+ "DeepPoseKit, a software toolkit for fast and robust animal
+ pose estimation using deep learning" Elife'2019.
+ More details can be found in the paper.
+
+ The dataset loads raw features and apply specified transforms
+ to return a dict containing the image tensors and other information.
+
+ Desert Locust keypoint indexes::
+
+ 0: "head",
+ 1: "neck",
+ 2: "thorax",
+ 3: "abdomen1",
+ 4: "abdomen2",
+ 5: "anttipL",
+ 6: "antbaseL",
+ 7: "eyeL",
+ 8: "forelegL1",
+ 9: "forelegL2",
+ 10: "forelegL3",
+ 11: "forelegL4",
+ 12: "midlegL1",
+ 13: "midlegL2",
+ 14: "midlegL3",
+ 15: "midlegL4",
+ 16: "hindlegL1",
+ 17: "hindlegL2",
+ 18: "hindlegL3",
+ 19: "hindlegL4",
+ 20: "anttipR",
+ 21: "antbaseR",
+ 22: "eyeR",
+ 23: "forelegR1",
+ 24: "forelegR2",
+ 25: "forelegR3",
+ 26: "forelegR4",
+ 27: "midlegR1",
+ 28: "midlegR2",
+ 29: "midlegR3",
+ 30: "midlegR4",
+ 31: "hindlegR1",
+ 32: "hindlegR2",
+ 33: "hindlegR3",
+ 34: "hindlegR4"
+
+ Args:
+ ann_file (str): Path to the annotation file.
+ img_prefix (str): Path to a directory where images are held.
+ Default: None.
+ data_cfg (dict): config
+ pipeline (list[dict | callable]): A sequence of data transforms.
+ dataset_info (DatasetInfo): A class containing all dataset info.
+ test_mode (bool): Store True when building test or
+ validation dataset. Default: False.
+ """
+
+ def __init__(self,
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=None,
+ test_mode=False):
+
+ if dataset_info is None:
+ warnings.warn(
+ 'dataset_info is missing. '
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 '
+ 'for details.', DeprecationWarning)
+ cfg = Config.fromfile('configs/_base_/datasets/locust.py')
+ dataset_info = cfg._cfg_dict['dataset_info']
+
+ super().__init__(
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=dataset_info,
+ test_mode=test_mode)
+
+ self.ann_info['use_different_joint_weights'] = False
+ self.db = self._get_db()
+
+ print(f'=> num_images: {self.num_images}')
+ print(f'=> load {len(self.db)} samples')
+
+ def _get_db(self):
+ """Load dataset."""
+ gt_db = []
+ bbox_id = 0
+ num_joints = self.ann_info['num_joints']
+ for img_id in self.img_ids:
+
+ ann_ids = self.coco.getAnnIds(imgIds=img_id, iscrowd=False)
+ objs = self.coco.loadAnns(ann_ids)
+
+ for obj in objs:
+ if max(obj['keypoints']) == 0:
+ continue
+ joints_3d = np.zeros((num_joints, 3), dtype=np.float32)
+ joints_3d_visible = np.zeros((num_joints, 3), dtype=np.float32)
+
+ keypoints = np.array(obj['keypoints']).reshape(-1, 3)
+ joints_3d[:, :2] = keypoints[:, :2]
+ joints_3d_visible[:, :2] = np.minimum(1, keypoints[:, 2:3])
+
+ # the ori image is 160x160
+ center, scale = self._xywh2cs(0, 0, 160, 160, 0.8)
+
+ image_file = osp.join(self.img_prefix, self.id2name[img_id])
+
+ gt_db.append({
+ 'image_file': image_file,
+ 'center': center,
+ 'scale': scale,
+ 'rotation': 0,
+ 'joints_3d': joints_3d,
+ 'joints_3d_visible': joints_3d_visible,
+ 'dataset': self.dataset_name,
+ 'bbox': obj['bbox'],
+ 'bbox_score': 1,
+ 'bbox_id': bbox_id
+ })
+ bbox_id = bbox_id + 1
+ gt_db = sorted(gt_db, key=lambda x: x['bbox_id'])
+
+ return gt_db
+
+ @deprecated_api_warning(name_dict=dict(outputs='results'))
+ def evaluate(self, results, res_folder=None, metric='PCK', **kwargs):
+ """Evaluate Fly keypoint results. The pose prediction results will be
+ saved in ``${res_folder}/result_keypoints.json``.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+ - heatmap height: H
+ - heatmap width: W
+
+ Args:
+ results (list[dict]): Testing results containing the following
+ items:
+
+ - preds (np.ndarray[N,K,3]): The first two dimensions are \
+ coordinates, score is the third dimension of the array.
+ - boxes (np.ndarray[N,6]): [center[0], center[1], scale[0], \
+ scale[1],area, score]
+ - image_paths (list[str]): For example, ['Test/source/0.jpg']
+ - output_heatmap (np.ndarray[N, K, H, W]): model outputs.
+ res_folder (str, optional): The folder to save the testing
+ results. If not specified, a temp folder will be created.
+ Default: None.
+ metric (str | list[str]): Metric to be performed.
+ Options: 'PCK', 'AUC', 'EPE'.
+
+ Returns:
+ dict: Evaluation results for evaluation metric.
+ """
+ metrics = metric if isinstance(metric, list) else [metric]
+ allowed_metrics = ['PCK', 'AUC', 'EPE']
+ for metric in metrics:
+ if metric not in allowed_metrics:
+ raise KeyError(f'metric {metric} is not supported')
+
+ if res_folder is not None:
+ tmp_folder = None
+ res_file = osp.join(res_folder, 'result_keypoints.json')
+ else:
+ tmp_folder = tempfile.TemporaryDirectory()
+ res_file = osp.join(tmp_folder.name, 'result_keypoints.json')
+
+ kpts = []
+ for result in results:
+ preds = result['preds']
+ boxes = result['boxes']
+ image_paths = result['image_paths']
+ bbox_ids = result['bbox_ids']
+
+ batch_size = len(image_paths)
+ for i in range(batch_size):
+ image_id = self.name2id[image_paths[i][len(self.img_prefix):]]
+
+ kpts.append({
+ 'keypoints': preds[i].tolist(),
+ 'center': boxes[i][0:2].tolist(),
+ 'scale': boxes[i][2:4].tolist(),
+ 'area': float(boxes[i][4]),
+ 'score': float(boxes[i][5]),
+ 'image_id': image_id,
+ 'bbox_id': bbox_ids[i]
+ })
+ kpts = self._sort_and_unique_bboxes(kpts)
+
+ self._write_keypoint_results(kpts, res_file)
+ info_str = self._report_metric(res_file, metrics)
+ name_value = OrderedDict(info_str)
+
+ if tmp_folder is not None:
+ tmp_folder.cleanup()
+
+ return name_value
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/animal/animal_macaque_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/animal/animal_macaque_dataset.py
new file mode 100644
index 0000000..359feca
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/animal/animal_macaque_dataset.py
@@ -0,0 +1,355 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os.path as osp
+import tempfile
+import warnings
+from collections import OrderedDict, defaultdict
+
+import json_tricks as json
+import numpy as np
+from mmcv import Config, deprecated_api_warning
+from xtcocotools.cocoeval import COCOeval
+
+from ....core.post_processing import oks_nms, soft_oks_nms
+from ...builder import DATASETS
+from ..base import Kpt2dSviewRgbImgTopDownDataset
+
+
+@DATASETS.register_module()
+class AnimalMacaqueDataset(Kpt2dSviewRgbImgTopDownDataset):
+ """MacaquePose dataset for animal pose estimation.
+
+ "MacaquePose: A novel ‘in the wild’ macaque monkey pose dataset
+ for markerless motion capture" bioRxiv'2020.
+ More details can be found in the `paper
+ `__ .
+
+ The dataset loads raw features and apply specified transforms
+ to return a dict containing the image tensors and other information.
+
+ Macaque keypoint indexes::
+
+ 0: 'nose',
+ 1: 'left_eye',
+ 2: 'right_eye',
+ 3: 'left_ear',
+ 4: 'right_ear',
+ 5: 'left_shoulder',
+ 6: 'right_shoulder',
+ 7: 'left_elbow',
+ 8: 'right_elbow',
+ 9: 'left_wrist',
+ 10: 'right_wrist',
+ 11: 'left_hip',
+ 12: 'right_hip',
+ 13: 'left_knee',
+ 14: 'right_knee',
+ 15: 'left_ankle',
+ 16: 'right_ankle'
+
+ Args:
+ ann_file (str): Path to the annotation file.
+ img_prefix (str): Path to a directory where images are held.
+ Default: None.
+ data_cfg (dict): config
+ pipeline (list[dict | callable]): A sequence of data transforms.
+ dataset_info (DatasetInfo): A class containing all dataset info.
+ test_mode (bool): Store True when building test or
+ validation dataset. Default: False.
+ """
+
+ def __init__(self,
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=None,
+ test_mode=False):
+
+ if dataset_info is None:
+ warnings.warn(
+ 'dataset_info is missing. '
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 '
+ 'for details.', DeprecationWarning)
+ cfg = Config.fromfile('configs/_base_/datasets/macaque.py')
+ dataset_info = cfg._cfg_dict['dataset_info']
+
+ super().__init__(
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=dataset_info,
+ test_mode=test_mode)
+
+ self.use_gt_bbox = data_cfg['use_gt_bbox']
+ self.bbox_file = data_cfg['bbox_file']
+ self.det_bbox_thr = data_cfg.get('det_bbox_thr', 0.0)
+ self.use_nms = data_cfg.get('use_nms', True)
+ self.soft_nms = data_cfg['soft_nms']
+ self.nms_thr = data_cfg['nms_thr']
+ self.oks_thr = data_cfg['oks_thr']
+ self.vis_thr = data_cfg['vis_thr']
+
+ self.ann_info['use_different_joint_weights'] = False
+ self.db = self._get_db()
+
+ print(f'=> num_images: {self.num_images}')
+ print(f'=> load {len(self.db)} samples')
+
+ def _get_db(self):
+ """Load dataset."""
+ assert self.use_gt_bbox
+ gt_db = self._load_coco_keypoint_annotations()
+ return gt_db
+
+ def _load_coco_keypoint_annotations(self):
+ """Ground truth bbox and keypoints."""
+ gt_db = []
+ for img_id in self.img_ids:
+ gt_db.extend(self._load_coco_keypoint_annotation_kernel(img_id))
+ return gt_db
+
+ def _load_coco_keypoint_annotation_kernel(self, img_id):
+ """load annotation from COCOAPI.
+
+ Note:
+ bbox:[x1, y1, w, h]
+ Args:
+ img_id: coco image id
+ Returns:
+ dict: db entry
+ """
+ img_ann = self.coco.loadImgs(img_id)[0]
+ width = img_ann['width']
+ height = img_ann['height']
+ num_joints = self.ann_info['num_joints']
+
+ ann_ids = self.coco.getAnnIds(imgIds=img_id, iscrowd=False)
+ objs = self.coco.loadAnns(ann_ids)
+
+ # sanitize bboxes
+ valid_objs = []
+ for obj in objs:
+ if 'bbox' not in obj:
+ continue
+ x, y, w, h = obj['bbox']
+ x1 = max(0, x)
+ y1 = max(0, y)
+ x2 = min(width - 1, x1 + max(0, w - 1))
+ y2 = min(height - 1, y1 + max(0, h - 1))
+ if ('area' not in obj or obj['area'] > 0) and x2 > x1 and y2 > y1:
+ obj['clean_bbox'] = [x1, y1, x2 - x1, y2 - y1]
+ valid_objs.append(obj)
+ objs = valid_objs
+
+ bbox_id = 0
+ rec = []
+ for obj in objs:
+ if 'keypoints' not in obj:
+ continue
+ if max(obj['keypoints']) == 0:
+ continue
+ if 'num_keypoints' in obj and obj['num_keypoints'] == 0:
+ continue
+ joints_3d = np.zeros((num_joints, 3), dtype=np.float32)
+ joints_3d_visible = np.zeros((num_joints, 3), dtype=np.float32)
+
+ keypoints = np.array(obj['keypoints']).reshape(-1, 3)
+ joints_3d[:, :2] = keypoints[:, :2]
+ joints_3d_visible[:, :2] = np.minimum(1, keypoints[:, 2:3])
+
+ center, scale = self._xywh2cs(*obj['clean_bbox'][:4])
+
+ image_file = osp.join(self.img_prefix, self.id2name[img_id])
+ rec.append({
+ 'image_file': image_file,
+ 'center': center,
+ 'scale': scale,
+ 'bbox': obj['clean_bbox'][:4],
+ 'rotation': 0,
+ 'joints_3d': joints_3d,
+ 'joints_3d_visible': joints_3d_visible,
+ 'dataset': self.dataset_name,
+ 'bbox_score': 1,
+ 'bbox_id': bbox_id
+ })
+ bbox_id = bbox_id + 1
+
+ return rec
+
+ @deprecated_api_warning(name_dict=dict(outputs='results'))
+ def evaluate(self, results, res_folder=None, metric='mAP', **kwargs):
+ """Evaluate coco keypoint results. The pose prediction results will be
+ saved in ``${res_folder}/result_keypoints.json``.
+
+ Note:
+ batch_size: N
+ num_keypoints: K
+ heatmap height: H
+ heatmap width: W
+
+ Args:
+ results (list[dict]): Testing results containing the following
+ items:
+
+ - preds (np.ndarray[N,K,3]): The first two dimensions are \
+ coordinates, score is the third dimension of the array.
+ - boxes (np.ndarray[N,6]): [center[0], center[1], scale[0], \
+ scale[1],area, score]
+ - image_paths (list[str]): For example, ['data/coco/val2017\
+ /000000393226.jpg']
+ - heatmap (np.ndarray[N, K, H, W]): model output heatmap
+ - bbox_id (list(int)).
+ res_folder (str, optional): The folder to save the testing
+ results. If not specified, a temp folder will be created.
+ Default: None.
+ metric (str | list[str]): Metric to be performed. Defaults: 'mAP'.
+
+ Returns:
+ dict: Evaluation results for evaluation metric.
+ """
+ metrics = metric if isinstance(metric, list) else [metric]
+ allowed_metrics = ['mAP']
+ for metric in metrics:
+ if metric not in allowed_metrics:
+ raise KeyError(f'metric {metric} is not supported')
+
+ if res_folder is not None:
+ tmp_folder = None
+ res_file = osp.join(res_folder, 'result_keypoints.json')
+ else:
+ tmp_folder = tempfile.TemporaryDirectory()
+ res_file = osp.join(tmp_folder.name, 'result_keypoints.json')
+
+ kpts = defaultdict(list)
+
+ for result in results:
+ preds = result['preds']
+ boxes = result['boxes']
+ image_paths = result['image_paths']
+ bbox_ids = result['bbox_ids']
+
+ batch_size = len(image_paths)
+ for i in range(batch_size):
+ image_id = self.name2id[image_paths[i][len(self.img_prefix):]]
+ kpts[image_id].append({
+ 'keypoints': preds[i],
+ 'center': boxes[i][0:2],
+ 'scale': boxes[i][2:4],
+ 'area': boxes[i][4],
+ 'score': boxes[i][5],
+ 'image_id': image_id,
+ 'bbox_id': bbox_ids[i]
+ })
+ kpts = self._sort_and_unique_bboxes(kpts)
+
+ # rescoring and oks nms
+ num_joints = self.ann_info['num_joints']
+ vis_thr = self.vis_thr
+ oks_thr = self.oks_thr
+ valid_kpts = []
+ for image_id in kpts.keys():
+ img_kpts = kpts[image_id]
+ for n_p in img_kpts:
+ box_score = n_p['score']
+ kpt_score = 0
+ valid_num = 0
+ for n_jt in range(0, num_joints):
+ t_s = n_p['keypoints'][n_jt][2]
+ if t_s > vis_thr:
+ kpt_score = kpt_score + t_s
+ valid_num = valid_num + 1
+ if valid_num != 0:
+ kpt_score = kpt_score / valid_num
+ # rescoring
+ n_p['score'] = kpt_score * box_score
+
+ if self.use_nms:
+ nms = soft_oks_nms if self.soft_nms else oks_nms
+ keep = nms(list(img_kpts), oks_thr, sigmas=self.sigmas)
+ valid_kpts.append([img_kpts[_keep] for _keep in keep])
+ else:
+ valid_kpts.append(img_kpts)
+
+ self._write_coco_keypoint_results(valid_kpts, res_file)
+
+ info_str = self._do_python_keypoint_eval(res_file)
+ name_value = OrderedDict(info_str)
+
+ if tmp_folder is not None:
+ tmp_folder.cleanup()
+
+ return name_value
+
+ def _write_coco_keypoint_results(self, keypoints, res_file):
+ """Write results into a json file."""
+ data_pack = [{
+ 'cat_id': self._class_to_coco_ind[cls],
+ 'cls_ind': cls_ind,
+ 'cls': cls,
+ 'ann_type': 'keypoints',
+ 'keypoints': keypoints
+ } for cls_ind, cls in enumerate(self.classes)
+ if not cls == '__background__']
+
+ results = self._coco_keypoint_results_one_category_kernel(data_pack[0])
+
+ with open(res_file, 'w') as f:
+ json.dump(results, f, sort_keys=True, indent=4)
+
+ def _coco_keypoint_results_one_category_kernel(self, data_pack):
+ """Get coco keypoint results."""
+ cat_id = data_pack['cat_id']
+ keypoints = data_pack['keypoints']
+ cat_results = []
+
+ for img_kpts in keypoints:
+ if len(img_kpts) == 0:
+ continue
+
+ _key_points = np.array(
+ [img_kpt['keypoints'] for img_kpt in img_kpts])
+ key_points = _key_points.reshape(-1,
+ self.ann_info['num_joints'] * 3)
+
+ result = [{
+ 'image_id': img_kpt['image_id'],
+ 'category_id': cat_id,
+ 'keypoints': key_point.tolist(),
+ 'score': float(img_kpt['score']),
+ 'center': img_kpt['center'].tolist(),
+ 'scale': img_kpt['scale'].tolist()
+ } for img_kpt, key_point in zip(img_kpts, key_points)]
+
+ cat_results.extend(result)
+
+ return cat_results
+
+ def _do_python_keypoint_eval(self, res_file):
+ """Keypoint evaluation using COCOAPI."""
+ coco_det = self.coco.loadRes(res_file)
+ coco_eval = COCOeval(self.coco, coco_det, 'keypoints', self.sigmas)
+ coco_eval.params.useSegm = None
+ coco_eval.evaluate()
+ coco_eval.accumulate()
+ coco_eval.summarize()
+
+ stats_names = [
+ 'AP', 'AP .5', 'AP .75', 'AP (M)', 'AP (L)', 'AR', 'AR .5',
+ 'AR .75', 'AR (M)', 'AR (L)'
+ ]
+
+ info_str = list(zip(stats_names, coco_eval.stats))
+
+ return info_str
+
+ def _sort_and_unique_bboxes(self, kpts, key='bbox_id'):
+ """sort kpts and remove the repeated ones."""
+ for img_id, persons in kpts.items():
+ num = len(persons)
+ kpts[img_id] = sorted(kpts[img_id], key=lambda x: x[key])
+ for i in range(num - 1, 0, -1):
+ if kpts[img_id][i][key] == kpts[img_id][i - 1][key]:
+ del kpts[img_id][i]
+
+ return kpts
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/animal/animal_pose_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/animal/animal_pose_dataset.py
new file mode 100644
index 0000000..4ced570
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/animal/animal_pose_dataset.py
@@ -0,0 +1,359 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os.path as osp
+import tempfile
+import warnings
+from collections import OrderedDict, defaultdict
+
+import json_tricks as json
+import numpy as np
+from mmcv import Config, deprecated_api_warning
+from xtcocotools.cocoeval import COCOeval
+
+from ....core.post_processing import oks_nms, soft_oks_nms
+from ...builder import DATASETS
+from ..base import Kpt2dSviewRgbImgTopDownDataset
+
+
+@DATASETS.register_module()
+class AnimalPoseDataset(Kpt2dSviewRgbImgTopDownDataset):
+ """Animal-Pose dataset for animal pose estimation.
+
+ "Cross-domain Adaptation For Animal Pose Estimation" ICCV'2019
+ More details can be found in the `paper
+ `__ .
+
+ The dataset loads raw features and apply specified transforms
+ to return a dict containing the image tensors and other information.
+
+ Animal-Pose keypoint indexes::
+
+ 0: 'L_Eye',
+ 1: 'R_Eye',
+ 2: 'L_EarBase',
+ 3: 'R_EarBase',
+ 4: 'Nose',
+ 5: 'Throat',
+ 6: 'TailBase',
+ 7: 'Withers',
+ 8: 'L_F_Elbow',
+ 9: 'R_F_Elbow',
+ 10: 'L_B_Elbow',
+ 11: 'R_B_Elbow',
+ 12: 'L_F_Knee',
+ 13: 'R_F_Knee',
+ 14: 'L_B_Knee',
+ 15: 'R_B_Knee',
+ 16: 'L_F_Paw',
+ 17: 'R_F_Paw',
+ 18: 'L_B_Paw',
+ 19: 'R_B_Paw'
+
+ Args:
+ ann_file (str): Path to the annotation file.
+ img_prefix (str): Path to a directory where images are held.
+ Default: None.
+ data_cfg (dict): config
+ pipeline (list[dict | callable]): A sequence of data transforms.
+ dataset_info (DatasetInfo): A class containing all dataset info.
+ test_mode (bool): Store True when building test or
+ validation dataset. Default: False.
+ """
+
+ def __init__(self,
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=None,
+ test_mode=False):
+
+ if dataset_info is None:
+ warnings.warn(
+ 'dataset_info is missing. '
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 '
+ 'for details.', DeprecationWarning)
+ cfg = Config.fromfile('configs/_base_/datasets/animalpose.py')
+ dataset_info = cfg._cfg_dict['dataset_info']
+
+ super().__init__(
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=dataset_info,
+ test_mode=test_mode)
+
+ self.use_gt_bbox = data_cfg['use_gt_bbox']
+ self.bbox_file = data_cfg['bbox_file']
+ self.det_bbox_thr = data_cfg.get('det_bbox_thr', 0.0)
+ self.use_nms = data_cfg.get('use_nms', True)
+ self.soft_nms = data_cfg['soft_nms']
+ self.nms_thr = data_cfg['nms_thr']
+ self.oks_thr = data_cfg['oks_thr']
+ self.vis_thr = data_cfg['vis_thr']
+
+ self.ann_info['use_different_joint_weights'] = False
+ self.db = self._get_db()
+
+ print(f'=> num_images: {self.num_images}')
+ print(f'=> load {len(self.db)} samples')
+
+ def _get_db(self):
+ """Load dataset."""
+ assert self.use_gt_bbox
+ gt_db = self._load_coco_keypoint_annotations()
+ return gt_db
+
+ def _load_coco_keypoint_annotations(self):
+ """Ground truth bbox and keypoints."""
+ gt_db = []
+ for img_id in self.img_ids:
+ gt_db.extend(self._load_coco_keypoint_annotation_kernel(img_id))
+ return gt_db
+
+ def _load_coco_keypoint_annotation_kernel(self, img_id):
+ """load annotation from COCOAPI.
+
+ Note:
+ bbox:[x1, y1, w, h]
+
+ Args:
+ img_id: coco image id
+
+ Returns:
+ dict: db entry
+ """
+ img_ann = self.coco.loadImgs(img_id)[0]
+ width = img_ann['width']
+ height = img_ann['height']
+ num_joints = self.ann_info['num_joints']
+
+ ann_ids = self.coco.getAnnIds(imgIds=img_id, iscrowd=False)
+ objs = self.coco.loadAnns(ann_ids)
+
+ # sanitize bboxes
+ valid_objs = []
+ for obj in objs:
+ if 'bbox' not in obj:
+ continue
+ x, y, w, h = obj['bbox']
+ x1 = max(0, x)
+ y1 = max(0, y)
+ x2 = min(width - 1, x1 + max(0, w - 1))
+ y2 = min(height - 1, y1 + max(0, h - 1))
+ if ('area' not in obj or obj['area'] > 0) and x2 > x1 and y2 > y1:
+ obj['clean_bbox'] = [x1, y1, x2 - x1, y2 - y1]
+ valid_objs.append(obj)
+ objs = valid_objs
+
+ bbox_id = 0
+ rec = []
+ for obj in objs:
+ if 'keypoints' not in obj:
+ continue
+ if max(obj['keypoints']) == 0:
+ continue
+ if 'num_keypoints' in obj and obj['num_keypoints'] == 0:
+ continue
+ joints_3d = np.zeros((num_joints, 3), dtype=np.float32)
+ joints_3d_visible = np.zeros((num_joints, 3), dtype=np.float32)
+
+ keypoints = np.array(obj['keypoints']).reshape(-1, 3)
+ joints_3d[:, :2] = keypoints[:, :2]
+ joints_3d_visible[:, :2] = np.minimum(1, keypoints[:, 2:3])
+
+ center, scale = self._xywh2cs(*obj['clean_bbox'][:4])
+
+ image_file = osp.join(self.img_prefix, self.id2name[img_id])
+ rec.append({
+ 'image_file': image_file,
+ 'center': center,
+ 'scale': scale,
+ 'bbox': obj['clean_bbox'][:4],
+ 'rotation': 0,
+ 'joints_3d': joints_3d,
+ 'joints_3d_visible': joints_3d_visible,
+ 'dataset': self.dataset_name,
+ 'bbox_score': 1,
+ 'bbox_id': bbox_id
+ })
+ bbox_id = bbox_id + 1
+
+ return rec
+
+ @deprecated_api_warning(name_dict=dict(outputs='results'))
+ def evaluate(self, results, res_folder=None, metric='mAP', **kwargs):
+ """Evaluate coco keypoint results. The pose prediction results will be
+ saved in ``${res_folder}/result_keypoints.json``.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+ - heatmap height: H
+ - heatmap width: W
+
+ Args:
+ results (list[dict]): Testing results containing the following
+ items:
+
+ - preds (np.ndarray[N,K,3]): The first two dimensions are \
+ coordinates, score is the third dimension of the array.
+ - boxes (np.ndarray[N,6]): [center[0], center[1], scale[0], \
+ scale[1],area, score]
+ - image_paths (list[str]): For example, ['data/coco/val2017\
+ /000000393226.jpg']
+ - heatmap (np.ndarray[N, K, H, W]): model output heatmap
+ - bbox_id (list(int)).
+ res_folder (str, optional): The folder to save the testing
+ results. If not specified, a temp folder will be created.
+ Default: None.
+ metric (str | list[str]): Metric to be performed. Defaults: 'mAP'.
+
+ Returns:
+ dict: Evaluation results for evaluation metric.
+ """
+ metrics = metric if isinstance(metric, list) else [metric]
+ allowed_metrics = ['mAP']
+ for metric in metrics:
+ if metric not in allowed_metrics:
+ raise KeyError(f'metric {metric} is not supported')
+
+ if res_folder is not None:
+ tmp_folder = None
+ res_file = osp.join(res_folder, 'result_keypoints.json')
+ else:
+ tmp_folder = tempfile.TemporaryDirectory()
+ res_file = osp.join(tmp_folder.name, 'result_keypoints.json')
+
+ kpts = defaultdict(list)
+
+ for result in results:
+ preds = result['preds']
+ boxes = result['boxes']
+ image_paths = result['image_paths']
+ bbox_ids = result['bbox_ids']
+
+ batch_size = len(image_paths)
+ for i in range(batch_size):
+ image_id = self.name2id[image_paths[i][len(self.img_prefix):]]
+ kpts[image_id].append({
+ 'keypoints': preds[i],
+ 'center': boxes[i][0:2],
+ 'scale': boxes[i][2:4],
+ 'area': boxes[i][4],
+ 'score': boxes[i][5],
+ 'image_id': image_id,
+ 'bbox_id': bbox_ids[i]
+ })
+ kpts = self._sort_and_unique_bboxes(kpts)
+
+ # rescoring and oks nms
+ num_joints = self.ann_info['num_joints']
+ vis_thr = self.vis_thr
+ oks_thr = self.oks_thr
+ valid_kpts = []
+ for image_id in kpts.keys():
+ img_kpts = kpts[image_id]
+ for n_p in img_kpts:
+ box_score = n_p['score']
+ kpt_score = 0
+ valid_num = 0
+ for n_jt in range(0, num_joints):
+ t_s = n_p['keypoints'][n_jt][2]
+ if t_s > vis_thr:
+ kpt_score = kpt_score + t_s
+ valid_num = valid_num + 1
+ if valid_num != 0:
+ kpt_score = kpt_score / valid_num
+ # rescoring
+ n_p['score'] = kpt_score * box_score
+
+ if self.use_nms:
+ nms = soft_oks_nms if self.soft_nms else oks_nms
+ keep = nms(list(img_kpts), oks_thr, sigmas=self.sigmas)
+ valid_kpts.append([img_kpts[_keep] for _keep in keep])
+ else:
+ valid_kpts.append(img_kpts)
+
+ self._write_coco_keypoint_results(valid_kpts, res_file)
+
+ info_str = self._do_python_keypoint_eval(res_file)
+ name_value = OrderedDict(info_str)
+
+ if tmp_folder is not None:
+ tmp_folder.cleanup()
+
+ return name_value
+
+ def _write_coco_keypoint_results(self, keypoints, res_file):
+ """Write results into a json file."""
+ data_pack = [{
+ 'cat_id': self._class_to_coco_ind[cls],
+ 'cls_ind': cls_ind,
+ 'cls': cls,
+ 'ann_type': 'keypoints',
+ 'keypoints': keypoints
+ } for cls_ind, cls in enumerate(self.classes)
+ if not cls == '__background__']
+
+ results = self._coco_keypoint_results_one_category_kernel(data_pack[0])
+
+ with open(res_file, 'w') as f:
+ json.dump(results, f, sort_keys=True, indent=4)
+
+ def _coco_keypoint_results_one_category_kernel(self, data_pack):
+ """Get coco keypoint results."""
+ cat_id = data_pack['cat_id']
+ keypoints = data_pack['keypoints']
+ cat_results = []
+
+ for img_kpts in keypoints:
+ if len(img_kpts) == 0:
+ continue
+
+ _key_points = np.array(
+ [img_kpt['keypoints'] for img_kpt in img_kpts])
+ key_points = _key_points.reshape(-1,
+ self.ann_info['num_joints'] * 3)
+
+ result = [{
+ 'image_id': img_kpt['image_id'],
+ 'category_id': cat_id,
+ 'keypoints': key_point.tolist(),
+ 'score': float(img_kpt['score']),
+ 'center': img_kpt['center'].tolist(),
+ 'scale': img_kpt['scale'].tolist()
+ } for img_kpt, key_point in zip(img_kpts, key_points)]
+
+ cat_results.extend(result)
+
+ return cat_results
+
+ def _do_python_keypoint_eval(self, res_file):
+ """Keypoint evaluation using COCOAPI."""
+ coco_det = self.coco.loadRes(res_file)
+ coco_eval = COCOeval(self.coco, coco_det, 'keypoints', self.sigmas)
+ coco_eval.params.useSegm = None
+ coco_eval.evaluate()
+ coco_eval.accumulate()
+ coco_eval.summarize()
+
+ stats_names = [
+ 'AP', 'AP .5', 'AP .75', 'AP (M)', 'AP (L)', 'AR', 'AR .5',
+ 'AR .75', 'AR (M)', 'AR (L)'
+ ]
+
+ info_str = list(zip(stats_names, coco_eval.stats))
+
+ return info_str
+
+ def _sort_and_unique_bboxes(self, kpts, key='bbox_id'):
+ """sort kpts and remove the repeated ones."""
+ for img_id, persons in kpts.items():
+ num = len(persons)
+ kpts[img_id] = sorted(kpts[img_id], key=lambda x: x[key])
+ for i in range(num - 1, 0, -1):
+ if kpts[img_id][i][key] == kpts[img_id][i - 1][key]:
+ del kpts[img_id][i]
+
+ return kpts
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/animal/animal_zebra_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/animal/animal_zebra_dataset.py
new file mode 100644
index 0000000..9c5e3b7
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/animal/animal_zebra_dataset.py
@@ -0,0 +1,193 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os.path as osp
+import tempfile
+import warnings
+from collections import OrderedDict
+
+import numpy as np
+from mmcv import Config, deprecated_api_warning
+
+from ...builder import DATASETS
+from ..base import Kpt2dSviewRgbImgTopDownDataset
+
+
+@DATASETS.register_module()
+class AnimalZebraDataset(Kpt2dSviewRgbImgTopDownDataset):
+ """AnimalZebraDataset for animal pose estimation.
+
+ "DeepPoseKit, a software toolkit for fast and robust animal
+ pose estimation using deep learning" Elife'2019.
+ More details can be found in the paper.
+
+ The dataset loads raw features and apply specified transforms
+ to return a dict containing the image tensors and other information.
+
+ Desert Locust keypoint indexes::
+
+ 0: "snout",
+ 1: "head",
+ 2: "neck",
+ 3: "forelegL1",
+ 4: "forelegR1",
+ 5: "hindlegL1",
+ 6: "hindlegR1",
+ 7: "tailbase",
+ 8: "tailtip"
+
+ Args:
+ ann_file (str): Path to the annotation file.
+ img_prefix (str): Path to a directory where images are held.
+ Default: None.
+ data_cfg (dict): config
+ pipeline (list[dict | callable]): A sequence of data transforms.
+ dataset_info (DatasetInfo): A class containing all dataset info.
+ test_mode (bool): Store True when building test or
+ validation dataset. Default: False.
+ """
+
+ def __init__(self,
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=None,
+ test_mode=False):
+
+ if dataset_info is None:
+ warnings.warn(
+ 'dataset_info is missing. '
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 '
+ 'for details.', DeprecationWarning)
+ cfg = Config.fromfile('configs/_base_/datasets/zebra.py')
+ dataset_info = cfg._cfg_dict['dataset_info']
+
+ super().__init__(
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=dataset_info,
+ test_mode=test_mode)
+
+ self.ann_info['use_different_joint_weights'] = False
+ self.db = self._get_db()
+
+ print(f'=> num_images: {self.num_images}')
+ print(f'=> load {len(self.db)} samples')
+
+ def _get_db(self):
+ """Load dataset."""
+ gt_db = []
+ bbox_id = 0
+ num_joints = self.ann_info['num_joints']
+ for img_id in self.img_ids:
+
+ ann_ids = self.coco.getAnnIds(imgIds=img_id, iscrowd=False)
+ objs = self.coco.loadAnns(ann_ids)
+
+ for obj in objs:
+ if max(obj['keypoints']) == 0:
+ continue
+ joints_3d = np.zeros((num_joints, 3), dtype=np.float32)
+ joints_3d_visible = np.zeros((num_joints, 3), dtype=np.float32)
+
+ keypoints = np.array(obj['keypoints']).reshape(-1, 3)
+ joints_3d[:, :2] = keypoints[:, :2]
+ joints_3d_visible[:, :2] = np.minimum(1, keypoints[:, 2:3])
+
+ # the ori image is 160x160
+ center, scale = self._xywh2cs(0, 0, 160, 160, 0.8)
+
+ image_file = osp.join(self.img_prefix, self.id2name[img_id])
+
+ gt_db.append({
+ 'image_file': image_file,
+ 'center': center,
+ 'scale': scale,
+ 'rotation': 0,
+ 'joints_3d': joints_3d,
+ 'joints_3d_visible': joints_3d_visible,
+ 'dataset': self.dataset_name,
+ 'bbox': obj['bbox'],
+ 'bbox_score': 1,
+ 'bbox_id': bbox_id
+ })
+ bbox_id = bbox_id + 1
+ gt_db = sorted(gt_db, key=lambda x: x['bbox_id'])
+
+ return gt_db
+
+ @deprecated_api_warning(name_dict=dict(outputs='results'))
+ def evaluate(self, results, res_folder=None, metric='PCK', **kwargs):
+ """Evaluate Fly keypoint results. The pose prediction results will be
+ saved in ``${res_folder}/result_keypoints.json``.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+ - heatmap height: H
+ - heatmap width: W
+
+ Args:
+ results (list[dict]): Testing results containing the following
+ items:
+
+ - preds (np.ndarray[N,K,3]): The first two dimensions are \
+ coordinates, score is the third dimension of the array.
+ - boxes (np.ndarray[N,6]): [center[0], center[1], scale[0], \
+ scale[1],area, score]
+ - image_paths (list[str]): For example, ['Test/source/0.jpg']
+ - output_heatmap (np.ndarray[N, K, H, W]): model outputs.
+
+ res_folder (str, optional): The folder to save the testing
+ results. If not specified, a temp folder will be created.
+ Default: None.
+ metric (str | list[str]): Metric to be performed.
+ Options: 'PCK', 'AUC', 'EPE'.
+
+ Returns:
+ dict: Evaluation results for evaluation metric.
+ """
+ metrics = metric if isinstance(metric, list) else [metric]
+ allowed_metrics = ['PCK', 'AUC', 'EPE']
+ for metric in metrics:
+ if metric not in allowed_metrics:
+ raise KeyError(f'metric {metric} is not supported')
+
+ if res_folder is not None:
+ tmp_folder = None
+ res_file = osp.join(res_folder, 'result_keypoints.json')
+ else:
+ tmp_folder = tempfile.TemporaryDirectory()
+ res_file = osp.join(tmp_folder.name, 'result_keypoints.json')
+
+ kpts = []
+ for result in results:
+ preds = result['preds']
+ boxes = result['boxes']
+ image_paths = result['image_paths']
+ bbox_ids = result['bbox_ids']
+
+ batch_size = len(image_paths)
+ for i in range(batch_size):
+ image_id = self.name2id[image_paths[i][len(self.img_prefix):]]
+
+ kpts.append({
+ 'keypoints': preds[i].tolist(),
+ 'center': boxes[i][0:2].tolist(),
+ 'scale': boxes[i][2:4].tolist(),
+ 'area': float(boxes[i][4]),
+ 'score': float(boxes[i][5]),
+ 'image_id': image_id,
+ 'bbox_id': bbox_ids[i]
+ })
+ kpts = self._sort_and_unique_bboxes(kpts)
+
+ self._write_keypoint_results(kpts, res_file)
+ info_str = self._report_metric(res_file, metrics)
+ name_value = OrderedDict(info_str)
+
+ if tmp_folder is not None:
+ tmp_folder.cleanup()
+
+ return name_value
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/base/__init__.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/base/__init__.py
new file mode 100644
index 0000000..e5f9a08
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/base/__init__.py
@@ -0,0 +1,17 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from .kpt_2d_sview_rgb_img_bottom_up_dataset import \
+ Kpt2dSviewRgbImgBottomUpDataset
+from .kpt_2d_sview_rgb_img_top_down_dataset import \
+ Kpt2dSviewRgbImgTopDownDataset
+from .kpt_2d_sview_rgb_vid_top_down_dataset import \
+ Kpt2dSviewRgbVidTopDownDataset
+from .kpt_3d_mview_rgb_img_direct_dataset import Kpt3dMviewRgbImgDirectDataset
+from .kpt_3d_sview_kpt_2d_dataset import Kpt3dSviewKpt2dDataset
+from .kpt_3d_sview_rgb_img_top_down_dataset import \
+ Kpt3dSviewRgbImgTopDownDataset
+
+__all__ = [
+ 'Kpt3dMviewRgbImgDirectDataset', 'Kpt2dSviewRgbImgTopDownDataset',
+ 'Kpt3dSviewRgbImgTopDownDataset', 'Kpt2dSviewRgbImgBottomUpDataset',
+ 'Kpt3dSviewKpt2dDataset', 'Kpt2dSviewRgbVidTopDownDataset'
+]
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/base/kpt_2d_sview_rgb_img_bottom_up_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/base/kpt_2d_sview_rgb_img_bottom_up_dataset.py
new file mode 100644
index 0000000..9930621
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/base/kpt_2d_sview_rgb_img_bottom_up_dataset.py
@@ -0,0 +1,188 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import copy
+from abc import ABCMeta, abstractmethod
+
+import numpy as np
+import xtcocotools
+from torch.utils.data import Dataset
+from xtcocotools.coco import COCO
+
+from mmpose.datasets import DatasetInfo
+from mmpose.datasets.pipelines import Compose
+
+
+class Kpt2dSviewRgbImgBottomUpDataset(Dataset, metaclass=ABCMeta):
+ """Base class for bottom-up datasets.
+
+ All datasets should subclass it.
+ All subclasses should overwrite:
+ Methods:`_get_single`
+
+ Args:
+ ann_file (str): Path to the annotation file.
+ img_prefix (str): Path to a directory where images are held.
+ Default: None.
+ data_cfg (dict): config
+ pipeline (list[dict | callable]): A sequence of data transforms.
+ dataset_info (DatasetInfo): A class containing all dataset info.
+ coco_style (bool): Whether the annotation json is coco-style.
+ Default: True
+ test_mode (bool): Store True when building test or
+ validation dataset. Default: False.
+ """
+
+ def __init__(self,
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=None,
+ coco_style=True,
+ test_mode=False):
+
+ self.image_info = {}
+ self.ann_info = {}
+
+ self.ann_file = ann_file
+ self.img_prefix = img_prefix
+ self.pipeline = pipeline
+ self.test_mode = test_mode
+
+ # bottom-up
+ self.base_size = data_cfg['base_size']
+ self.base_sigma = data_cfg['base_sigma']
+ self.int_sigma = False
+
+ self.ann_info['image_size'] = np.array(data_cfg['image_size'])
+ self.ann_info['heatmap_size'] = np.array(data_cfg['heatmap_size'])
+ self.ann_info['num_joints'] = data_cfg['num_joints']
+ self.ann_info['num_scales'] = data_cfg['num_scales']
+ self.ann_info['scale_aware_sigma'] = data_cfg['scale_aware_sigma']
+
+ self.ann_info['inference_channel'] = data_cfg['inference_channel']
+ self.ann_info['dataset_channel'] = data_cfg['dataset_channel']
+
+ self.use_nms = data_cfg.get('use_nms', False)
+ self.soft_nms = data_cfg.get('soft_nms', True)
+ self.oks_thr = data_cfg.get('oks_thr', 0.9)
+
+ if dataset_info is None:
+ raise ValueError(
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 '
+ 'for details.')
+
+ dataset_info = DatasetInfo(dataset_info)
+
+ assert self.ann_info['num_joints'] == dataset_info.keypoint_num
+ self.ann_info['flip_pairs'] = dataset_info.flip_pairs
+ self.ann_info['flip_index'] = dataset_info.flip_index
+ self.ann_info['upper_body_ids'] = dataset_info.upper_body_ids
+ self.ann_info['lower_body_ids'] = dataset_info.lower_body_ids
+ self.ann_info['joint_weights'] = dataset_info.joint_weights
+ self.ann_info['skeleton'] = dataset_info.skeleton
+ self.sigmas = dataset_info.sigmas
+ self.dataset_name = dataset_info.dataset_name
+
+ if coco_style:
+ self.coco = COCO(ann_file)
+ if 'categories' in self.coco.dataset:
+ cats = [
+ cat['name']
+ for cat in self.coco.loadCats(self.coco.getCatIds())
+ ]
+ self.classes = ['__background__'] + cats
+ self.num_classes = len(self.classes)
+ self._class_to_ind = dict(
+ zip(self.classes, range(self.num_classes)))
+ self._class_to_coco_ind = dict(
+ zip(cats, self.coco.getCatIds()))
+ self._coco_ind_to_class_ind = dict(
+ (self._class_to_coco_ind[cls], self._class_to_ind[cls])
+ for cls in self.classes[1:])
+ self.img_ids = self.coco.getImgIds()
+ if not test_mode:
+ self.img_ids = [
+ img_id for img_id in self.img_ids if
+ len(self.coco.getAnnIds(imgIds=img_id, iscrowd=None)) > 0
+ ]
+ self.num_images = len(self.img_ids)
+ self.id2name, self.name2id = self._get_mapping_id_name(
+ self.coco.imgs)
+
+ self.pipeline = Compose(self.pipeline)
+
+ @staticmethod
+ def _get_mapping_id_name(imgs):
+ """
+ Args:
+ imgs (dict): dict of image info.
+
+ Returns:
+ tuple: Image name & id mapping dicts.
+
+ - id2name (dict): Mapping image id to name.
+ - name2id (dict): Mapping image name to id.
+ """
+ id2name = {}
+ name2id = {}
+ for image_id, image in imgs.items():
+ file_name = image['file_name']
+ id2name[image_id] = file_name
+ name2id[file_name] = image_id
+
+ return id2name, name2id
+
+ def _get_mask(self, anno, idx):
+ """Get ignore masks to mask out losses."""
+ coco = self.coco
+ img_info = coco.loadImgs(self.img_ids[idx])[0]
+
+ m = np.zeros((img_info['height'], img_info['width']), dtype=np.float32)
+
+ for obj in anno:
+ if 'segmentation' in obj:
+ if obj['iscrowd']:
+ rle = xtcocotools.mask.frPyObjects(obj['segmentation'],
+ img_info['height'],
+ img_info['width'])
+ m += xtcocotools.mask.decode(rle)
+ elif obj['num_keypoints'] == 0:
+ rles = xtcocotools.mask.frPyObjects(
+ obj['segmentation'], img_info['height'],
+ img_info['width'])
+ for rle in rles:
+ m += xtcocotools.mask.decode(rle)
+
+ return m < 0.5
+
+ @abstractmethod
+ def _get_single(self, idx):
+ """Get anno for a single image."""
+ raise NotImplementedError
+
+ @abstractmethod
+ def evaluate(self, results, *args, **kwargs):
+ """Evaluate keypoint results."""
+
+ def prepare_train_img(self, idx):
+ """Prepare image for training given the index."""
+ results = copy.deepcopy(self._get_single(idx))
+ results['ann_info'] = self.ann_info
+ return self.pipeline(results)
+
+ def prepare_test_img(self, idx):
+ """Prepare image for testing given the index."""
+ results = copy.deepcopy(self._get_single(idx))
+ results['ann_info'] = self.ann_info
+ return self.pipeline(results)
+
+ def __len__(self):
+ """Get dataset length."""
+ return len(self.img_ids)
+
+ def __getitem__(self, idx):
+ """Get the sample for either training or testing given index."""
+ if self.test_mode:
+ return self.prepare_test_img(idx)
+
+ return self.prepare_train_img(idx)
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/base/kpt_2d_sview_rgb_img_top_down_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/base/kpt_2d_sview_rgb_img_top_down_dataset.py
new file mode 100644
index 0000000..fb281f1
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/base/kpt_2d_sview_rgb_img_top_down_dataset.py
@@ -0,0 +1,287 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import copy
+from abc import ABCMeta, abstractmethod
+
+import json_tricks as json
+import numpy as np
+from torch.utils.data import Dataset
+from xtcocotools.coco import COCO
+
+from mmpose.core.evaluation.top_down_eval import (keypoint_auc, keypoint_epe,
+ keypoint_nme,
+ keypoint_pck_accuracy)
+from mmpose.datasets import DatasetInfo
+from mmpose.datasets.pipelines import Compose
+
+
+class Kpt2dSviewRgbImgTopDownDataset(Dataset, metaclass=ABCMeta):
+ """Base class for keypoint 2D top-down pose estimation with single-view RGB
+ image as the input.
+
+ All fashion datasets should subclass it.
+ All subclasses should overwrite:
+ Methods:`_get_db`, 'evaluate'
+
+ Args:
+ ann_file (str): Path to the annotation file.
+ img_prefix (str): Path to a directory where images are held.
+ Default: None.
+ data_cfg (dict): config
+ pipeline (list[dict | callable]): A sequence of data transforms.
+ dataset_info (DatasetInfo): A class containing all dataset info.
+ coco_style (bool): Whether the annotation json is coco-style.
+ Default: True
+ test_mode (bool): Store True when building test or
+ validation dataset. Default: False.
+ """
+
+ def __init__(self,
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=None,
+ coco_style=True,
+ test_mode=False):
+
+ self.image_info = {}
+ self.ann_info = {}
+
+ self.ann_file = ann_file
+ self.img_prefix = img_prefix
+ self.pipeline = pipeline
+ self.test_mode = test_mode
+
+ self.ann_info['image_size'] = np.array(data_cfg['image_size'])
+ self.ann_info['heatmap_size'] = np.array(data_cfg['heatmap_size'])
+ self.ann_info['num_joints'] = data_cfg['num_joints']
+
+ self.ann_info['inference_channel'] = data_cfg['inference_channel']
+ self.ann_info['num_output_channels'] = data_cfg['num_output_channels']
+ self.ann_info['dataset_channel'] = data_cfg['dataset_channel']
+
+ self.ann_info['max_num_joints'] = data_cfg.get('max_num_joints', None)
+ self.ann_info['dataset_idx'] = data_cfg.get('dataset_idx', 0)
+
+ self.ann_info['use_different_joint_weights'] = data_cfg.get(
+ 'use_different_joint_weights', False)
+
+ if dataset_info is None:
+ raise ValueError(
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 '
+ 'for details.')
+
+ dataset_info = DatasetInfo(dataset_info)
+
+ assert self.ann_info['num_joints'] == dataset_info.keypoint_num
+ self.ann_info['flip_pairs'] = dataset_info.flip_pairs
+ self.ann_info['flip_index'] = dataset_info.flip_index
+ self.ann_info['upper_body_ids'] = dataset_info.upper_body_ids
+ self.ann_info['lower_body_ids'] = dataset_info.lower_body_ids
+ self.ann_info['joint_weights'] = dataset_info.joint_weights
+ self.ann_info['skeleton'] = dataset_info.skeleton
+ self.sigmas = dataset_info.sigmas
+ self.dataset_name = dataset_info.dataset_name
+
+ if coco_style:
+ self.coco = COCO(ann_file)
+ if 'categories' in self.coco.dataset:
+ cats = [
+ cat['name']
+ for cat in self.coco.loadCats(self.coco.getCatIds())
+ ]
+ self.classes = ['__background__'] + cats
+ self.num_classes = len(self.classes)
+ self._class_to_ind = dict(
+ zip(self.classes, range(self.num_classes)))
+ self._class_to_coco_ind = dict(
+ zip(cats, self.coco.getCatIds()))
+ self._coco_ind_to_class_ind = dict(
+ (self._class_to_coco_ind[cls], self._class_to_ind[cls])
+ for cls in self.classes[1:])
+ self.img_ids = self.coco.getImgIds()
+ self.num_images = len(self.img_ids)
+ self.id2name, self.name2id = self._get_mapping_id_name(
+ self.coco.imgs)
+
+ self.db = []
+
+ self.pipeline = Compose(self.pipeline)
+
+ @staticmethod
+ def _get_mapping_id_name(imgs):
+ """
+ Args:
+ imgs (dict): dict of image info.
+
+ Returns:
+ tuple: Image name & id mapping dicts.
+
+ - id2name (dict): Mapping image id to name.
+ - name2id (dict): Mapping image name to id.
+ """
+ id2name = {}
+ name2id = {}
+ for image_id, image in imgs.items():
+ file_name = image['file_name']
+ id2name[image_id] = file_name
+ name2id[file_name] = image_id
+
+ return id2name, name2id
+
+ def _xywh2cs(self, x, y, w, h, padding=1.25):
+ """This encodes bbox(x,y,w,h) into (center, scale)
+
+ Args:
+ x, y, w, h (float): left, top, width and height
+ padding (float): bounding box padding factor
+
+ Returns:
+ center (np.ndarray[float32](2,)): center of the bbox (x, y).
+ scale (np.ndarray[float32](2,)): scale of the bbox w & h.
+ """
+ aspect_ratio = self.ann_info['image_size'][0] / self.ann_info[
+ 'image_size'][1]
+ center = np.array([x + w * 0.5, y + h * 0.5], dtype=np.float32)
+
+ if (not self.test_mode) and np.random.rand() < 0.3:
+ center += 0.4 * (np.random.rand(2) - 0.5) * [w, h]
+
+ if w > aspect_ratio * h:
+ h = w * 1.0 / aspect_ratio
+ elif w < aspect_ratio * h:
+ w = h * aspect_ratio
+
+ # pixel std is 200.0
+ scale = np.array([w / 200.0, h / 200.0], dtype=np.float32)
+ # padding to include proper amount of context
+ scale = scale * padding
+
+ return center, scale
+
+ def _get_normalize_factor(self, gts, *args, **kwargs):
+ """Get the normalize factor. generally inter-ocular distance measured
+ as the Euclidean distance between the outer corners of the eyes is
+ used. This function should be overrode, to measure NME.
+
+ Args:
+ gts (np.ndarray[N, K, 2]): Groundtruth keypoint location.
+
+ Returns:
+ np.ndarray[N, 2]: normalized factor
+ """
+ return np.ones([gts.shape[0], 2], dtype=np.float32)
+
+ @abstractmethod
+ def _get_db(self):
+ """Load dataset."""
+ raise NotImplementedError
+
+ @abstractmethod
+ def evaluate(self, results, *args, **kwargs):
+ """Evaluate keypoint results."""
+
+ @staticmethod
+ def _write_keypoint_results(keypoints, res_file):
+ """Write results into a json file."""
+
+ with open(res_file, 'w') as f:
+ json.dump(keypoints, f, sort_keys=True, indent=4)
+
+ def _report_metric(self,
+ res_file,
+ metrics,
+ pck_thr=0.2,
+ pckh_thr=0.7,
+ auc_nor=30):
+ """Keypoint evaluation.
+
+ Args:
+ res_file (str): Json file stored prediction results.
+ metrics (str | list[str]): Metric to be performed.
+ Options: 'PCK', 'PCKh', 'AUC', 'EPE', 'NME'.
+ pck_thr (float): PCK threshold, default as 0.2.
+ pckh_thr (float): PCKh threshold, default as 0.7.
+ auc_nor (float): AUC normalization factor, default as 30 pixel.
+
+ Returns:
+ List: Evaluation results for evaluation metric.
+ """
+ info_str = []
+
+ with open(res_file, 'r') as fin:
+ preds = json.load(fin)
+ assert len(preds) == len(self.db)
+
+ outputs = []
+ gts = []
+ masks = []
+ box_sizes = []
+ threshold_bbox = []
+ threshold_head_box = []
+
+ for pred, item in zip(preds, self.db):
+ outputs.append(np.array(pred['keypoints'])[:, :-1])
+ gts.append(np.array(item['joints_3d'])[:, :-1])
+ masks.append((np.array(item['joints_3d_visible'])[:, 0]) > 0)
+ if 'PCK' in metrics:
+ bbox = np.array(item['bbox'])
+ bbox_thr = np.max(bbox[2:])
+ threshold_bbox.append(np.array([bbox_thr, bbox_thr]))
+ if 'PCKh' in metrics:
+ head_box_thr = item['head_size']
+ threshold_head_box.append(
+ np.array([head_box_thr, head_box_thr]))
+ box_sizes.append(item.get('box_size', 1))
+
+ outputs = np.array(outputs)
+ gts = np.array(gts)
+ masks = np.array(masks)
+ threshold_bbox = np.array(threshold_bbox)
+ threshold_head_box = np.array(threshold_head_box)
+ box_sizes = np.array(box_sizes).reshape([-1, 1])
+
+ if 'PCK' in metrics:
+ _, pck, _ = keypoint_pck_accuracy(outputs, gts, masks, pck_thr,
+ threshold_bbox)
+ info_str.append(('PCK', pck))
+
+ if 'PCKh' in metrics:
+ _, pckh, _ = keypoint_pck_accuracy(outputs, gts, masks, pckh_thr,
+ threshold_head_box)
+ info_str.append(('PCKh', pckh))
+
+ if 'AUC' in metrics:
+ info_str.append(('AUC', keypoint_auc(outputs, gts, masks,
+ auc_nor)))
+
+ if 'EPE' in metrics:
+ info_str.append(('EPE', keypoint_epe(outputs, gts, masks)))
+
+ if 'NME' in metrics:
+ normalize_factor = self._get_normalize_factor(
+ gts=gts, box_sizes=box_sizes)
+ info_str.append(
+ ('NME', keypoint_nme(outputs, gts, masks, normalize_factor)))
+
+ return info_str
+
+ def __len__(self):
+ """Get the size of the dataset."""
+ return len(self.db)
+
+ def __getitem__(self, idx):
+ """Get the sample given index."""
+ results = copy.deepcopy(self.db[idx])
+ results['ann_info'] = self.ann_info
+ return self.pipeline(results)
+
+ def _sort_and_unique_bboxes(self, kpts, key='bbox_id'):
+ """sort kpts and remove the repeated ones."""
+ kpts = sorted(kpts, key=lambda x: x[key])
+ num = len(kpts)
+ for i in range(num - 1, 0, -1):
+ if kpts[i][key] == kpts[i - 1][key]:
+ del kpts[i]
+
+ return kpts
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/base/kpt_2d_sview_rgb_vid_top_down_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/base/kpt_2d_sview_rgb_vid_top_down_dataset.py
new file mode 100644
index 0000000..e529270
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/base/kpt_2d_sview_rgb_vid_top_down_dataset.py
@@ -0,0 +1,200 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import copy
+from abc import ABCMeta, abstractmethod
+
+import numpy as np
+from torch.utils.data import Dataset
+from xtcocotools.coco import COCO
+
+from mmpose.datasets import DatasetInfo
+from mmpose.datasets.pipelines import Compose
+
+
+class Kpt2dSviewRgbVidTopDownDataset(Dataset, metaclass=ABCMeta):
+ """Base class for keypoint 2D top-down pose estimation with single-view RGB
+ video as the input.
+
+ All fashion datasets should subclass it.
+ All subclasses should overwrite:
+ Methods:`_get_db`, 'evaluate'
+
+ Args:
+ ann_file (str): Path to the annotation file.
+ img_prefix (str): Path to a directory where videos/images are held.
+ Default: None.
+ data_cfg (dict): config
+ pipeline (list[dict | callable]): A sequence of data transforms.
+ dataset_info (DatasetInfo): A class containing all dataset info.
+ coco_style (bool): Whether the annotation json is coco-style.
+ Default: True
+ test_mode (bool): Store True when building test or
+ validation dataset. Default: False.
+ """
+
+ def __init__(self,
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=None,
+ coco_style=True,
+ test_mode=False):
+
+ self.image_info = {}
+ self.ann_info = {}
+
+ self.ann_file = ann_file
+ self.img_prefix = img_prefix
+ self.pipeline = pipeline
+ self.test_mode = test_mode
+
+ self.ann_info['image_size'] = np.array(data_cfg['image_size'])
+ self.ann_info['heatmap_size'] = np.array(data_cfg['heatmap_size'])
+ self.ann_info['num_joints'] = data_cfg['num_joints']
+
+ self.ann_info['inference_channel'] = data_cfg['inference_channel']
+ self.ann_info['num_output_channels'] = data_cfg['num_output_channels']
+ self.ann_info['dataset_channel'] = data_cfg['dataset_channel']
+
+ self.ann_info['use_different_joint_weights'] = data_cfg.get(
+ 'use_different_joint_weights', False)
+
+ if dataset_info is None:
+ raise ValueError(
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 '
+ 'for details.')
+
+ dataset_info = DatasetInfo(dataset_info)
+
+ assert self.ann_info['num_joints'] == dataset_info.keypoint_num
+ self.ann_info['flip_pairs'] = dataset_info.flip_pairs
+ self.ann_info['flip_index'] = dataset_info.flip_index
+ self.ann_info['upper_body_ids'] = dataset_info.upper_body_ids
+ self.ann_info['lower_body_ids'] = dataset_info.lower_body_ids
+ self.ann_info['joint_weights'] = dataset_info.joint_weights
+ self.ann_info['skeleton'] = dataset_info.skeleton
+ self.sigmas = dataset_info.sigmas
+ self.dataset_name = dataset_info.dataset_name
+
+ if coco_style:
+ self.coco = COCO(ann_file)
+ if 'categories' in self.coco.dataset:
+ cats = [
+ cat['name']
+ for cat in self.coco.loadCats(self.coco.getCatIds())
+ ]
+ self.classes = ['__background__'] + cats
+ self.num_classes = len(self.classes)
+ self._class_to_ind = dict(
+ zip(self.classes, range(self.num_classes)))
+ self._class_to_coco_ind = dict(
+ zip(cats, self.coco.getCatIds()))
+ self._coco_ind_to_class_ind = dict(
+ (self._class_to_coco_ind[cls], self._class_to_ind[cls])
+ for cls in self.classes[1:])
+ self.img_ids = self.coco.getImgIds()
+ self.num_images = len(self.img_ids)
+ self.id2name, self.name2id = self._get_mapping_id_name(
+ self.coco.imgs)
+
+ self.db = []
+
+ self.pipeline = Compose(self.pipeline)
+
+ @staticmethod
+ def _get_mapping_id_name(imgs):
+ """
+ Args:
+ imgs (dict): dict of image info.
+
+ Returns:
+ tuple: Image name & id mapping dicts.
+
+ - id2name (dict): Mapping image id to name.
+ - name2id (dict): Mapping image name to id.
+ """
+ id2name = {}
+ name2id = {}
+ for image_id, image in imgs.items():
+ file_name = image['file_name']
+ id2name[image_id] = file_name
+ name2id[file_name] = image_id
+
+ return id2name, name2id
+
+ def _xywh2cs(self, x, y, w, h, padding=1.25):
+ """This encodes bbox(x,y,w,h) into (center, scale)
+
+ Args:
+ x, y, w, h (float): left, top, width and height
+ padding (float): bounding box padding factor
+
+ Returns:
+ center (np.ndarray[float32](2,)): center of the bbox (x, y).
+ scale (np.ndarray[float32](2,)): scale of the bbox w & h.
+ """
+ aspect_ratio = self.ann_info['image_size'][0] / self.ann_info[
+ 'image_size'][1]
+ center = np.array([x + w * 0.5, y + h * 0.5], dtype=np.float32)
+
+ if (not self.test_mode) and np.random.rand() < 0.3:
+ center += 0.4 * (np.random.rand(2) - 0.5) * [w, h]
+
+ if w > aspect_ratio * h:
+ h = w * 1.0 / aspect_ratio
+ elif w < aspect_ratio * h:
+ w = h * aspect_ratio
+
+ # pixel std is 200.0
+ scale = np.array([w / 200.0, h / 200.0], dtype=np.float32)
+ # padding to include proper amount of context
+ scale = scale * padding
+
+ return center, scale
+
+ @abstractmethod
+ def _get_db(self):
+ """Load dataset."""
+
+ @abstractmethod
+ def evaluate(self, results, *args, **kwargs):
+ """Evaluate keypoint results."""
+
+ @staticmethod
+ @abstractmethod
+ def _write_keypoint_results(keypoint_results, gt_folder, pred_folder):
+ """Write results into a json file."""
+
+ @abstractmethod
+ def _do_keypoint_eval(self, gt_folder, pred_folder):
+ """Keypoint evaluation.
+ Args:
+ gt_folder (str): The folder of the json files storing
+ ground truth keypoint annotations.
+ pred_folder (str): The folder of the json files storing
+ prediction results.
+
+ Returns:
+ List: Evaluation results for evaluation metric.
+ """
+
+ def __len__(self):
+ """Get the size of the dataset."""
+ return len(self.db)
+
+ def __getitem__(self, idx):
+ """Get the sample given index."""
+ results = copy.deepcopy(self.db[idx])
+ results['ann_info'] = self.ann_info
+ return self.pipeline(results)
+
+ def _sort_and_unique_bboxes(self, kpts, key='bbox_id'):
+ """sort kpts and remove the repeated ones."""
+ for img_id, persons in kpts.items():
+ num = len(persons)
+ kpts[img_id] = sorted(kpts[img_id], key=lambda x: x[key])
+ for i in range(num - 1, 0, -1):
+ if kpts[img_id][i][key] == kpts[img_id][i - 1][key]:
+ del kpts[img_id][i]
+
+ return kpts
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/base/kpt_3d_mview_rgb_img_direct_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/base/kpt_3d_mview_rgb_img_direct_dataset.py
new file mode 100644
index 0000000..94cc1c2
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/base/kpt_3d_mview_rgb_img_direct_dataset.py
@@ -0,0 +1,143 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import copy
+from abc import ABCMeta, abstractmethod
+
+import json_tricks as json
+import numpy as np
+from torch.utils.data import Dataset
+
+from mmpose.datasets import DatasetInfo
+from mmpose.datasets.pipelines import Compose
+
+
+class Kpt3dMviewRgbImgDirectDataset(Dataset, metaclass=ABCMeta):
+ """Base class for keypoint 3D top-down pose estimation with multi-view RGB
+ images as the input.
+
+ All subclasses should overwrite:
+ Methods:`_get_db`, 'evaluate'
+
+ Args:
+ ann_file (str): Path to the annotation file.
+ img_prefix (str): Path to a directory where images are held.
+ Default: None.
+ data_cfg (dict): config
+ pipeline (list[dict | callable]): A sequence of data transforms.
+ dataset_info (DatasetInfo): A class containing all dataset info.
+ test_mode (bool): Store True when building test or
+ validation dataset. Default: False.
+ """
+
+ def __init__(self,
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=None,
+ test_mode=False):
+
+ self.image_info = {}
+ self.ann_info = {}
+
+ self.ann_file = ann_file
+ self.img_prefix = img_prefix
+ self.pipeline = pipeline
+ self.test_mode = test_mode
+
+ self.ann_info['image_size'] = np.array(data_cfg['image_size'])
+ self.ann_info['heatmap_size'] = np.array(data_cfg['heatmap_size'])
+ self.ann_info['num_joints'] = data_cfg['num_joints']
+
+ self.ann_info['space_size'] = data_cfg['space_size']
+ self.ann_info['space_center'] = data_cfg['space_center']
+ self.ann_info['cube_size'] = data_cfg['cube_size']
+ self.ann_info['scale_aware_sigma'] = data_cfg.get(
+ 'scale_aware_sigma', False)
+
+ if dataset_info is None:
+ raise ValueError(
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 '
+ 'for details.')
+
+ dataset_info = DatasetInfo(dataset_info)
+
+ assert self.ann_info['num_joints'] <= dataset_info.keypoint_num
+ self.ann_info['flip_pairs'] = dataset_info.flip_pairs
+ self.ann_info['num_scales'] = 1
+ self.ann_info['flip_index'] = dataset_info.flip_index
+ self.ann_info['upper_body_ids'] = dataset_info.upper_body_ids
+ self.ann_info['lower_body_ids'] = dataset_info.lower_body_ids
+ self.ann_info['joint_weights'] = dataset_info.joint_weights
+ self.ann_info['skeleton'] = dataset_info.skeleton
+ self.sigmas = dataset_info.sigmas
+ self.dataset_name = dataset_info.dataset_name
+
+ self.load_config(data_cfg)
+
+ self.db = []
+
+ self.pipeline = Compose(self.pipeline)
+
+ def load_config(self, data_cfg):
+ """Initialize dataset attributes according to the config.
+
+ Override this method to set dataset specific attributes.
+ """
+ self.num_joints = data_cfg['num_joints']
+ self.num_cameras = data_cfg['num_cameras']
+ self.seq_frame_interval = data_cfg.get('seq_frame_interval', 1)
+ self.subset = data_cfg.get('subset', 'train')
+ self.need_2d_label = data_cfg.get('need_2d_label', False)
+ self.need_camera_param = True
+
+ @staticmethod
+ def _get_mapping_id_name(imgs):
+ """
+ Args:
+ imgs (dict): dict of image info.
+
+ Returns:
+ tuple: Image name & id mapping dicts.
+
+ - id2name (dict): Mapping image id to name.
+ - name2id (dict): Mapping image name to id.
+ """
+ id2name = {}
+ name2id = {}
+ for image_id, image in imgs.items():
+ file_name = image['file_name']
+ id2name[image_id] = file_name
+ name2id[file_name] = image_id
+
+ return id2name, name2id
+
+ @abstractmethod
+ def _get_db(self):
+ """Load dataset."""
+ raise NotImplementedError
+
+ @abstractmethod
+ def evaluate(self, results, *args, **kwargs):
+ """Evaluate keypoint results."""
+
+ @staticmethod
+ def _write_keypoint_results(keypoints, res_file):
+ """Write results into a json file."""
+
+ with open(res_file, 'w') as f:
+ json.dump(keypoints, f, sort_keys=True, indent=4)
+
+ def __len__(self):
+ """Get the size of the dataset."""
+ return len(self.db) // self.num_cameras
+
+ def __getitem__(self, idx):
+ """Get the sample given index."""
+ results = {}
+ # return self.pipeline(results)
+ for c in range(self.num_cameras):
+ result = copy.deepcopy(self.db[self.num_cameras * idx + c])
+ result['ann_info'] = self.ann_info
+ results[c] = result
+
+ return self.pipeline(results)
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/base/kpt_3d_sview_kpt_2d_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/base/kpt_3d_sview_kpt_2d_dataset.py
new file mode 100644
index 0000000..dbdb998
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/base/kpt_3d_sview_kpt_2d_dataset.py
@@ -0,0 +1,226 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import copy
+from abc import ABCMeta, abstractmethod
+
+import numpy as np
+from torch.utils.data import Dataset
+
+from mmpose.datasets import DatasetInfo
+from mmpose.datasets.pipelines import Compose
+
+
+class Kpt3dSviewKpt2dDataset(Dataset, metaclass=ABCMeta):
+ """Base class for 3D human pose datasets.
+
+ Subclasses should consider overwriting following methods:
+ - load_config
+ - load_annotations
+ - build_sample_indices
+ - evaluate
+
+ Args:
+ ann_file (str): Path to the annotation file.
+ img_prefix (str): Path to a directory where images are held.
+ Default: None.
+ data_cfg (dict): config
+ - num_joints: Number of joints.
+ - seq_len: Number of frames in a sequence. Default: 1.
+ - seq_frame_interval: Extract frames from the video at certain
+ intervals. Default: 1.
+ - causal: If set to True, the rightmost input frame will be the
+ target frame. Otherwise, the middle input frame will be the
+ target frame. Default: True.
+ - temporal_padding: Whether to pad the video so that poses will be
+ predicted for every frame in the video. Default: False
+ - subset: Reduce dataset size by fraction. Default: 1.
+ - need_2d_label: Whether need 2D joint labels or not.
+ Default: False.
+
+ pipeline (list[dict | callable]): A sequence of data transforms.
+ dataset_info (DatasetInfo): A class containing all dataset info.
+ test_mode (bool): Store True when building test or
+ validation dataset. Default: False.
+ """
+
+ def __init__(self,
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=None,
+ test_mode=False):
+
+ self.ann_file = ann_file
+ self.img_prefix = img_prefix
+ self.data_cfg = copy.deepcopy(data_cfg)
+ self.pipeline = pipeline
+ self.test_mode = test_mode
+ self.ann_info = {}
+
+ if dataset_info is None:
+ raise ValueError(
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 '
+ 'for details.')
+
+ dataset_info = DatasetInfo(dataset_info)
+
+ self.load_config(self.data_cfg)
+
+ self.ann_info['num_joints'] = data_cfg['num_joints']
+ assert self.ann_info['num_joints'] == dataset_info.keypoint_num
+ self.ann_info['flip_pairs'] = dataset_info.flip_pairs
+ self.ann_info['upper_body_ids'] = dataset_info.upper_body_ids
+ self.ann_info['lower_body_ids'] = dataset_info.lower_body_ids
+ self.ann_info['joint_weights'] = dataset_info.joint_weights
+ self.ann_info['skeleton'] = dataset_info.skeleton
+ self.sigmas = dataset_info.sigmas
+ self.dataset_name = dataset_info.dataset_name
+
+ self.data_info = self.load_annotations()
+ self.sample_indices = self.build_sample_indices()
+ self.pipeline = Compose(pipeline)
+
+ self.name2id = {
+ name: i
+ for i, name in enumerate(self.data_info['imgnames'])
+ }
+
+ def load_config(self, data_cfg):
+ """Initialize dataset attributes according to the config.
+
+ Override this method to set dataset specific attributes.
+ """
+
+ self.num_joints = data_cfg['num_joints']
+ self.seq_len = data_cfg.get('seq_len', 1)
+ self.seq_frame_interval = data_cfg.get('seq_frame_interval', 1)
+ self.causal = data_cfg.get('causal', True)
+ self.temporal_padding = data_cfg.get('temporal_padding', False)
+ self.subset = data_cfg.get('subset', 1)
+ self.need_2d_label = data_cfg.get('need_2d_label', False)
+ self.need_camera_param = False
+
+ def load_annotations(self):
+ """Load data annotation."""
+ data = np.load(self.ann_file)
+
+ # get image info
+ _imgnames = data['imgname']
+ num_imgs = len(_imgnames)
+ num_joints = self.ann_info['num_joints']
+
+ if 'scale' in data:
+ _scales = data['scale'].astype(np.float32)
+ else:
+ _scales = np.zeros(num_imgs, dtype=np.float32)
+
+ if 'center' in data:
+ _centers = data['center'].astype(np.float32)
+ else:
+ _centers = np.zeros((num_imgs, 2), dtype=np.float32)
+
+ # get 3D pose
+ if 'S' in data.keys():
+ _joints_3d = data['S'].astype(np.float32)
+ else:
+ _joints_3d = np.zeros((num_imgs, num_joints, 4), dtype=np.float32)
+
+ # get 2D pose
+ if 'part' in data.keys():
+ _joints_2d = data['part'].astype(np.float32)
+ else:
+ _joints_2d = np.zeros((num_imgs, num_joints, 3), dtype=np.float32)
+
+ data_info = {
+ 'imgnames': _imgnames,
+ 'joints_3d': _joints_3d,
+ 'joints_2d': _joints_2d,
+ 'scales': _scales,
+ 'centers': _centers,
+ }
+
+ return data_info
+
+ def build_sample_indices(self):
+ """Build sample indices.
+
+ The default method creates sample indices that each sample is a single
+ frame (i.e. seq_len=1). Override this method in the subclass to define
+ how frames are sampled to form data samples.
+
+ Outputs:
+ sample_indices [list(tuple)]: the frame indices of each sample.
+ For a sample, all frames will be treated as an input sequence,
+ and the ground-truth pose of the last frame will be the target.
+ """
+ sample_indices = []
+ if self.seq_len == 1:
+ num_imgs = len(self.ann_info['imgnames'])
+ sample_indices = [(idx, ) for idx in range(num_imgs)]
+ else:
+ raise NotImplementedError('Multi-frame data sample unsupported!')
+ return sample_indices
+
+ @abstractmethod
+ def evaluate(self, results, *args, **kwargs):
+ """Evaluate keypoint results."""
+
+ def prepare_data(self, idx):
+ """Get data sample."""
+ data = self.data_info
+
+ frame_ids = self.sample_indices[idx]
+ assert len(frame_ids) == self.seq_len
+
+ # get the 3D/2D pose sequence
+ _joints_3d = data['joints_3d'][frame_ids]
+ _joints_2d = data['joints_2d'][frame_ids]
+
+ # get the image info
+ _imgnames = data['imgnames'][frame_ids]
+ _centers = data['centers'][frame_ids]
+ _scales = data['scales'][frame_ids]
+ if _scales.ndim == 1:
+ _scales = np.stack([_scales, _scales], axis=1)
+
+ target_idx = -1 if self.causal else int(self.seq_len) // 2
+
+ results = {
+ 'input_2d': _joints_2d[:, :, :2],
+ 'input_2d_visible': _joints_2d[:, :, -1:],
+ 'input_3d': _joints_3d[:, :, :3],
+ 'input_3d_visible': _joints_3d[:, :, -1:],
+ 'target': _joints_3d[target_idx, :, :3],
+ 'target_visible': _joints_3d[target_idx, :, -1:],
+ 'image_paths': _imgnames,
+ 'target_image_path': _imgnames[target_idx],
+ 'scales': _scales,
+ 'centers': _centers,
+ }
+
+ if self.need_2d_label:
+ results['target_2d'] = _joints_2d[target_idx, :, :2]
+
+ if self.need_camera_param:
+ _cam_param = self.get_camera_param(_imgnames[0])
+ results['camera_param'] = _cam_param
+ # get image size from camera parameters
+ if 'w' in _cam_param and 'h' in _cam_param:
+ results['image_width'] = _cam_param['w']
+ results['image_height'] = _cam_param['h']
+
+ return results
+
+ def __len__(self):
+ """Get the size of the dataset."""
+ return len(self.sample_indices)
+
+ def __getitem__(self, idx):
+ """Get a sample with given index."""
+ results = copy.deepcopy(self.prepare_data(idx))
+ results['ann_info'] = self.ann_info
+ return self.pipeline(results)
+
+ def get_camera_param(self, imgname):
+ """Get camera parameters of a frame by its image name."""
+ raise NotImplementedError
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/base/kpt_3d_sview_rgb_img_top_down_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/base/kpt_3d_sview_rgb_img_top_down_dataset.py
new file mode 100644
index 0000000..af01e81
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/base/kpt_3d_sview_rgb_img_top_down_dataset.py
@@ -0,0 +1,256 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import copy
+from abc import ABCMeta, abstractmethod
+
+import json_tricks as json
+import numpy as np
+from torch.utils.data import Dataset
+from xtcocotools.coco import COCO
+
+from mmpose.datasets import DatasetInfo
+from mmpose.datasets.pipelines import Compose
+
+
+class Kpt3dSviewRgbImgTopDownDataset(Dataset, metaclass=ABCMeta):
+ """Base class for keypoint 3D top-down pose estimation with single-view RGB
+ image as the input.
+
+ All fashion datasets should subclass it.
+ All subclasses should overwrite:
+ Methods:`_get_db`, 'evaluate'
+
+ Args:
+ ann_file (str): Path to the annotation file.
+ img_prefix (str): Path to a directory where images are held.
+ Default: None.
+ data_cfg (dict): config
+ pipeline (list[dict | callable]): A sequence of data transforms.
+ dataset_info (DatasetInfo): A class containing all dataset info.
+ coco_style (bool): Whether the annotation json is coco-style.
+ Default: True
+ test_mode (bool): Store True when building test or
+ validation dataset. Default: False.
+ """
+
+ def __init__(self,
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=None,
+ coco_style=True,
+ test_mode=False):
+
+ self.image_info = {}
+ self.ann_info = {}
+
+ self.ann_file = ann_file
+ self.img_prefix = img_prefix
+ self.pipeline = pipeline
+ self.test_mode = test_mode
+
+ self.ann_info['image_size'] = np.array(data_cfg['image_size'])
+ self.ann_info['heatmap_size'] = np.array(data_cfg['heatmap_size'])
+ self.ann_info['num_joints'] = data_cfg['num_joints']
+
+ self.ann_info['inference_channel'] = data_cfg['inference_channel']
+ self.ann_info['num_output_channels'] = data_cfg['num_output_channels']
+ self.ann_info['dataset_channel'] = data_cfg['dataset_channel']
+
+ if dataset_info is None:
+ raise ValueError(
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 '
+ 'for details.')
+
+ dataset_info = DatasetInfo(dataset_info)
+
+ assert self.ann_info['num_joints'] == dataset_info.keypoint_num
+ self.ann_info['flip_pairs'] = dataset_info.flip_pairs
+ self.ann_info['flip_index'] = dataset_info.flip_index
+ self.ann_info['upper_body_ids'] = dataset_info.upper_body_ids
+ self.ann_info['lower_body_ids'] = dataset_info.lower_body_ids
+ self.ann_info['joint_weights'] = dataset_info.joint_weights
+ self.ann_info['skeleton'] = dataset_info.skeleton
+ self.sigmas = dataset_info.sigmas
+ self.dataset_name = dataset_info.dataset_name
+
+ if coco_style:
+ self.coco = COCO(ann_file)
+ if 'categories' in self.coco.dataset:
+ cats = [
+ cat['name']
+ for cat in self.coco.loadCats(self.coco.getCatIds())
+ ]
+ self.classes = ['__background__'] + cats
+ self.num_classes = len(self.classes)
+ self._class_to_ind = dict(
+ zip(self.classes, range(self.num_classes)))
+ self._class_to_coco_ind = dict(
+ zip(cats, self.coco.getCatIds()))
+ self._coco_ind_to_class_ind = dict(
+ (self._class_to_coco_ind[cls], self._class_to_ind[cls])
+ for cls in self.classes[1:])
+ self.img_ids = self.coco.getImgIds()
+ self.num_images = len(self.img_ids)
+ self.id2name, self.name2id = self._get_mapping_id_name(
+ self.coco.imgs)
+
+ self.db = []
+
+ self.pipeline = Compose(self.pipeline)
+
+ @staticmethod
+ def _cam2pixel(cam_coord, f, c):
+ """Transform the joints from their camera coordinates to their pixel
+ coordinates.
+
+ Note:
+ N: number of joints
+
+ Args:
+ cam_coord (ndarray[N, 3]): 3D joints coordinates
+ in the camera coordinate system
+ f (ndarray[2]): focal length of x and y axis
+ c (ndarray[2]): principal point of x and y axis
+
+ Returns:
+ img_coord (ndarray[N, 3]): the coordinates (x, y, 0)
+ in the image plane.
+ """
+ x = cam_coord[:, 0] / (cam_coord[:, 2] + 1e-8) * f[0] + c[0]
+ y = cam_coord[:, 1] / (cam_coord[:, 2] + 1e-8) * f[1] + c[1]
+ z = np.zeros_like(x)
+ img_coord = np.concatenate((x[:, None], y[:, None], z[:, None]), 1)
+ return img_coord
+
+ @staticmethod
+ def _world2cam(world_coord, R, T):
+ """Transform the joints from their world coordinates to their camera
+ coordinates.
+
+ Note:
+ N: number of joints
+
+ Args:
+ world_coord (ndarray[3, N]): 3D joints coordinates
+ in the world coordinate system
+ R (ndarray[3, 3]): camera rotation matrix
+ T (ndarray[3, 1]): camera position (x, y, z)
+
+ Returns:
+ cam_coord (ndarray[3, N]): 3D joints coordinates
+ in the camera coordinate system
+ """
+ cam_coord = np.dot(R, world_coord - T)
+ return cam_coord
+
+ @staticmethod
+ def _pixel2cam(pixel_coord, f, c):
+ """Transform the joints from their pixel coordinates to their camera
+ coordinates.
+
+ Note:
+ N: number of joints
+
+ Args:
+ pixel_coord (ndarray[N, 3]): 3D joints coordinates
+ in the pixel coordinate system
+ f (ndarray[2]): focal length of x and y axis
+ c (ndarray[2]): principal point of x and y axis
+
+ Returns:
+ cam_coord (ndarray[N, 3]): 3D joints coordinates
+ in the camera coordinate system
+ """
+ x = (pixel_coord[:, 0] - c[0]) / f[0] * pixel_coord[:, 2]
+ y = (pixel_coord[:, 1] - c[1]) / f[1] * pixel_coord[:, 2]
+ z = pixel_coord[:, 2]
+ cam_coord = np.concatenate((x[:, None], y[:, None], z[:, None]), 1)
+ return cam_coord
+
+ @staticmethod
+ def _get_mapping_id_name(imgs):
+ """
+ Args:
+ imgs (dict): dict of image info.
+
+ Returns:
+ tuple: Image name & id mapping dicts.
+
+ - id2name (dict): Mapping image id to name.
+ - name2id (dict): Mapping image name to id.
+ """
+ id2name = {}
+ name2id = {}
+ for image_id, image in imgs.items():
+ file_name = image['file_name']
+ id2name[image_id] = file_name
+ name2id[file_name] = image_id
+
+ return id2name, name2id
+
+ def _xywh2cs(self, x, y, w, h, padding=1.25):
+ """This encodes bbox(x,y,w,h) into (center, scale)
+
+ Args:
+ x, y, w, h (float): left, top, width and height
+ padding (float): bounding box padding factor
+
+ Returns:
+ center (np.ndarray[float32](2,)): center of the bbox (x, y).
+ scale (np.ndarray[float32](2,)): scale of the bbox w & h.
+ """
+ aspect_ratio = self.ann_info['image_size'][0] / self.ann_info[
+ 'image_size'][1]
+ center = np.array([x + w * 0.5, y + h * 0.5], dtype=np.float32)
+
+ if (not self.test_mode) and np.random.rand() < 0.3:
+ center += 0.4 * (np.random.rand(2) - 0.5) * [w, h]
+
+ if w > aspect_ratio * h:
+ h = w * 1.0 / aspect_ratio
+ elif w < aspect_ratio * h:
+ w = h * aspect_ratio
+
+ # pixel std is 200.0
+ scale = np.array([w / 200.0, h / 200.0], dtype=np.float32)
+ # padding to include proper amount of context
+ scale = scale * padding
+
+ return center, scale
+
+ @abstractmethod
+ def _get_db(self):
+ """Load dataset."""
+ raise NotImplementedError
+
+ @abstractmethod
+ def evaluate(self, results, *args, **kwargs):
+ """Evaluate keypoint results."""
+
+ @staticmethod
+ def _write_keypoint_results(keypoints, res_file):
+ """Write results into a json file."""
+
+ with open(res_file, 'w') as f:
+ json.dump(keypoints, f, sort_keys=True, indent=4)
+
+ def __len__(self):
+ """Get the size of the dataset."""
+ return len(self.db)
+
+ def __getitem__(self, idx):
+ """Get the sample given index."""
+ results = copy.deepcopy(self.db[idx])
+ results['ann_info'] = self.ann_info
+ return self.pipeline(results)
+
+ def _sort_and_unique_bboxes(self, kpts, key='bbox_id'):
+ """sort kpts and remove the repeated ones."""
+ kpts = sorted(kpts, key=lambda x: x[key])
+ num = len(kpts)
+ for i in range(num - 1, 0, -1):
+ if kpts[i][key] == kpts[i - 1][key]:
+ del kpts[i]
+
+ return kpts
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/body3d/__init__.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/body3d/__init__.py
new file mode 100644
index 0000000..5bc25a9
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/body3d/__init__.py
@@ -0,0 +1,11 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from .body3d_h36m_dataset import Body3DH36MDataset
+from .body3d_mpi_inf_3dhp_dataset import Body3DMpiInf3dhpDataset
+from .body3d_mview_direct_panoptic_dataset import \
+ Body3DMviewDirectPanopticDataset
+from .body3d_semi_supervision_dataset import Body3DSemiSupervisionDataset
+
+__all__ = [
+ 'Body3DH36MDataset', 'Body3DSemiSupervisionDataset',
+ 'Body3DMpiInf3dhpDataset', 'Body3DMviewDirectPanopticDataset'
+]
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/body3d/body3d_base_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/body3d/body3d_base_dataset.py
new file mode 100644
index 0000000..10c2923
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/body3d/body3d_base_dataset.py
@@ -0,0 +1,16 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from abc import ABCMeta
+
+from torch.utils.data import Dataset
+
+
+class Body3DBaseDataset(Dataset, metaclass=ABCMeta):
+ """This class has been deprecated and replaced by
+ Kpt3dSviewKpt2dDataset."""
+
+ def __init__(self, *args, **kwargs):
+ raise (ImportError(
+ 'Body3DBaseDataset has been replaced by '
+ 'Kpt3dSviewKpt2dDataset'
+ 'check https://github.com/open-mmlab/mmpose/pull/663 for details.')
+ )
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/body3d/body3d_h36m_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/body3d/body3d_h36m_dataset.py
new file mode 100644
index 0000000..ae4949d
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/body3d/body3d_h36m_dataset.py
@@ -0,0 +1,343 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os.path as osp
+import tempfile
+import warnings
+from collections import OrderedDict, defaultdict
+
+import mmcv
+import numpy as np
+from mmcv import Config, deprecated_api_warning
+
+from mmpose.core.evaluation import keypoint_mpjpe
+from mmpose.datasets.datasets.base import Kpt3dSviewKpt2dDataset
+from ...builder import DATASETS
+
+
+@DATASETS.register_module()
+class Body3DH36MDataset(Kpt3dSviewKpt2dDataset):
+ """Human3.6M dataset for 3D human pose estimation.
+
+ "Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human
+ Sensing in Natural Environments", TPAMI`2014.
+ More details can be found in the `paper
+ `__.
+
+ Human3.6M keypoint indexes::
+
+ 0: 'root (pelvis)',
+ 1: 'right_hip',
+ 2: 'right_knee',
+ 3: 'right_foot',
+ 4: 'left_hip',
+ 5: 'left_knee',
+ 6: 'left_foot',
+ 7: 'spine',
+ 8: 'thorax',
+ 9: 'neck_base',
+ 10: 'head',
+ 11: 'left_shoulder',
+ 12: 'left_elbow',
+ 13: 'left_wrist',
+ 14: 'right_shoulder',
+ 15: 'right_elbow',
+ 16: 'right_wrist'
+
+
+ Args:
+ ann_file (str): Path to the annotation file.
+ img_prefix (str): Path to a directory where images are held.
+ Default: None.
+ data_cfg (dict): config
+ pipeline (list[dict | callable]): A sequence of data transforms.
+ dataset_info (DatasetInfo): A class containing all dataset info.
+ test_mode (bool): Store True when building test or
+ validation dataset. Default: False.
+ """
+
+ JOINT_NAMES = [
+ 'Root', 'RHip', 'RKnee', 'RFoot', 'LHip', 'LKnee', 'LFoot', 'Spine',
+ 'Thorax', 'NeckBase', 'Head', 'LShoulder', 'LElbow', 'LWrist',
+ 'RShoulder', 'RElbow', 'RWrist'
+ ]
+
+ # 2D joint source options:
+ # "gt": from the annotation file
+ # "detection": from a detection result file of 2D keypoint
+ # "pipeline": will be generate by the pipeline
+ SUPPORTED_JOINT_2D_SRC = {'gt', 'detection', 'pipeline'}
+
+ # metric
+ ALLOWED_METRICS = {'mpjpe', 'p-mpjpe', 'n-mpjpe'}
+
+ def __init__(self,
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=None,
+ test_mode=False):
+
+ if dataset_info is None:
+ warnings.warn(
+ 'dataset_info is missing. '
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 '
+ 'for details.', DeprecationWarning)
+ cfg = Config.fromfile('configs/_base_/datasets/h36m.py')
+ dataset_info = cfg._cfg_dict['dataset_info']
+
+ super().__init__(
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=dataset_info,
+ test_mode=test_mode)
+
+ def load_config(self, data_cfg):
+ super().load_config(data_cfg)
+ # h36m specific attributes
+ self.joint_2d_src = data_cfg.get('joint_2d_src', 'gt')
+ if self.joint_2d_src not in self.SUPPORTED_JOINT_2D_SRC:
+ raise ValueError(
+ f'Unsupported joint_2d_src "{self.joint_2d_src}". '
+ f'Supported options are {self.SUPPORTED_JOINT_2D_SRC}')
+
+ self.joint_2d_det_file = data_cfg.get('joint_2d_det_file', None)
+
+ self.need_camera_param = data_cfg.get('need_camera_param', False)
+ if self.need_camera_param:
+ assert 'camera_param_file' in data_cfg
+ self.camera_param = self._load_camera_param(
+ data_cfg['camera_param_file'])
+
+ # h36m specific annotation info
+ ann_info = {}
+ ann_info['use_different_joint_weights'] = False
+ # action filter
+ actions = data_cfg.get('actions', '_all_')
+ self.actions = set(
+ actions if isinstance(actions, (list, tuple)) else [actions])
+
+ # subject filter
+ subjects = data_cfg.get('subjects', '_all_')
+ self.subjects = set(
+ subjects if isinstance(subjects, (list, tuple)) else [subjects])
+
+ self.ann_info.update(ann_info)
+
+ def load_annotations(self):
+ data_info = super().load_annotations()
+
+ # get 2D joints
+ if self.joint_2d_src == 'gt':
+ data_info['joints_2d'] = data_info['joints_2d']
+ elif self.joint_2d_src == 'detection':
+ data_info['joints_2d'] = self._load_joint_2d_detection(
+ self.joint_2d_det_file)
+ assert data_info['joints_2d'].shape[0] == data_info[
+ 'joints_3d'].shape[0]
+ assert data_info['joints_2d'].shape[2] == 3
+ elif self.joint_2d_src == 'pipeline':
+ # joint_2d will be generated in the pipeline
+ pass
+ else:
+ raise NotImplementedError(
+ f'Unhandled joint_2d_src option {self.joint_2d_src}')
+
+ return data_info
+
+ @staticmethod
+ def _parse_h36m_imgname(imgname):
+ """Parse imgname to get information of subject, action and camera.
+
+ A typical h36m image filename is like:
+ S1_Directions_1.54138969_000001.jpg
+ """
+ subj, rest = osp.basename(imgname).split('_', 1)
+ action, rest = rest.split('.', 1)
+ camera, rest = rest.split('_', 1)
+
+ return subj, action, camera
+
+ def build_sample_indices(self):
+ """Split original videos into sequences and build frame indices.
+
+ This method overrides the default one in the base class.
+ """
+
+ # Group frames into videos. Assume that self.data_info is
+ # chronological.
+ video_frames = defaultdict(list)
+ for idx, imgname in enumerate(self.data_info['imgnames']):
+ subj, action, camera = self._parse_h36m_imgname(imgname)
+
+ if '_all_' not in self.actions and action not in self.actions:
+ continue
+
+ if '_all_' not in self.subjects and subj not in self.subjects:
+ continue
+
+ video_frames[(subj, action, camera)].append(idx)
+
+ # build sample indices
+ sample_indices = []
+ _len = (self.seq_len - 1) * self.seq_frame_interval + 1
+ _step = self.seq_frame_interval
+ for _, _indices in sorted(video_frames.items()):
+ n_frame = len(_indices)
+
+ if self.temporal_padding:
+ # Pad the sequence so that every frame in the sequence will be
+ # predicted.
+ if self.causal:
+ frames_left = self.seq_len - 1
+ frames_right = 0
+ else:
+ frames_left = (self.seq_len - 1) // 2
+ frames_right = frames_left
+ for i in range(n_frame):
+ pad_left = max(0, frames_left - i // _step)
+ pad_right = max(0,
+ frames_right - (n_frame - 1 - i) // _step)
+ start = max(i % _step, i - frames_left * _step)
+ end = min(n_frame - (n_frame - 1 - i) % _step,
+ i + frames_right * _step + 1)
+ sample_indices.append([_indices[0]] * pad_left +
+ _indices[start:end:_step] +
+ [_indices[-1]] * pad_right)
+ else:
+ seqs_from_video = [
+ _indices[i:(i + _len):_step]
+ for i in range(0, n_frame - _len + 1)
+ ]
+ sample_indices.extend(seqs_from_video)
+
+ # reduce dataset size if self.subset < 1
+ assert 0 < self.subset <= 1
+ subset_size = int(len(sample_indices) * self.subset)
+ start = np.random.randint(0, len(sample_indices) - subset_size + 1)
+ end = start + subset_size
+
+ return sample_indices[start:end]
+
+ def _load_joint_2d_detection(self, det_file):
+ """"Load 2D joint detection results from file."""
+ joints_2d = np.load(det_file).astype(np.float32)
+
+ return joints_2d
+
+ @deprecated_api_warning(name_dict=dict(outputs='results'))
+ def evaluate(self, results, res_folder=None, metric='mpjpe', **kwargs):
+ metrics = metric if isinstance(metric, list) else [metric]
+ for _metric in metrics:
+ if _metric not in self.ALLOWED_METRICS:
+ raise ValueError(
+ f'Unsupported metric "{_metric}" for human3.6 dataset.'
+ f'Supported metrics are {self.ALLOWED_METRICS}')
+
+ if res_folder is not None:
+ tmp_folder = None
+ res_file = osp.join(res_folder, 'result_keypoints.json')
+ else:
+ tmp_folder = tempfile.TemporaryDirectory()
+ res_file = osp.join(tmp_folder.name, 'result_keypoints.json')
+
+ kpts = []
+ for result in results:
+ preds = result['preds']
+ image_paths = result['target_image_paths']
+ batch_size = len(image_paths)
+ for i in range(batch_size):
+ target_id = self.name2id[image_paths[i]]
+ kpts.append({
+ 'keypoints': preds[i],
+ 'target_id': target_id,
+ })
+
+ mmcv.dump(kpts, res_file)
+
+ name_value_tuples = []
+ for _metric in metrics:
+ if _metric == 'mpjpe':
+ _nv_tuples = self._report_mpjpe(kpts)
+ elif _metric == 'p-mpjpe':
+ _nv_tuples = self._report_mpjpe(kpts, mode='p-mpjpe')
+ elif _metric == 'n-mpjpe':
+ _nv_tuples = self._report_mpjpe(kpts, mode='n-mpjpe')
+ else:
+ raise NotImplementedError
+ name_value_tuples.extend(_nv_tuples)
+
+ if tmp_folder is not None:
+ tmp_folder.cleanup()
+
+ return OrderedDict(name_value_tuples)
+
+ def _report_mpjpe(self, keypoint_results, mode='mpjpe'):
+ """Cauculate mean per joint position error (MPJPE) or its variants like
+ P-MPJPE or N-MPJPE.
+
+ Args:
+ keypoint_results (list): Keypoint predictions. See
+ 'Body3DH36MDataset.evaluate' for details.
+ mode (str): Specify mpjpe variants. Supported options are:
+
+ - ``'mpjpe'``: Standard MPJPE.
+ - ``'p-mpjpe'``: MPJPE after aligning prediction to groundtruth
+ via a rigid transformation (scale, rotation and
+ translation).
+ - ``'n-mpjpe'``: MPJPE after aligning prediction to groundtruth
+ in scale only.
+ """
+
+ preds = []
+ gts = []
+ masks = []
+ action_category_indices = defaultdict(list)
+ for idx, result in enumerate(keypoint_results):
+ pred = result['keypoints']
+ target_id = result['target_id']
+ gt, gt_visible = np.split(
+ self.data_info['joints_3d'][target_id], [3], axis=-1)
+ preds.append(pred)
+ gts.append(gt)
+ masks.append(gt_visible)
+
+ action = self._parse_h36m_imgname(
+ self.data_info['imgnames'][target_id])[1]
+ action_category = action.split('_')[0]
+ action_category_indices[action_category].append(idx)
+
+ preds = np.stack(preds)
+ gts = np.stack(gts)
+ masks = np.stack(masks).squeeze(-1) > 0
+
+ err_name = mode.upper()
+ if mode == 'mpjpe':
+ alignment = 'none'
+ elif mode == 'p-mpjpe':
+ alignment = 'procrustes'
+ elif mode == 'n-mpjpe':
+ alignment = 'scale'
+ else:
+ raise ValueError(f'Invalid mode: {mode}')
+
+ error = keypoint_mpjpe(preds, gts, masks, alignment)
+ name_value_tuples = [(err_name, error)]
+
+ for action_category, indices in action_category_indices.items():
+ _error = keypoint_mpjpe(preds[indices], gts[indices],
+ masks[indices])
+ name_value_tuples.append((f'{err_name}_{action_category}', _error))
+
+ return name_value_tuples
+
+ def _load_camera_param(self, camera_param_file):
+ """Load camera parameters from file."""
+ return mmcv.load(camera_param_file)
+
+ def get_camera_param(self, imgname):
+ """Get camera parameters of a frame by its image name."""
+ assert hasattr(self, 'camera_param')
+ subj, _, camera = self._parse_h36m_imgname(imgname)
+ return self.camera_param[(subj, camera)]
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/body3d/body3d_mpi_inf_3dhp_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/body3d/body3d_mpi_inf_3dhp_dataset.py
new file mode 100644
index 0000000..4d06fcd
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/body3d/body3d_mpi_inf_3dhp_dataset.py
@@ -0,0 +1,417 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os.path as osp
+import tempfile
+import warnings
+from collections import OrderedDict, defaultdict
+
+import mmcv
+import numpy as np
+from mmcv import Config, deprecated_api_warning
+
+from mmpose.core.evaluation import (keypoint_3d_auc, keypoint_3d_pck,
+ keypoint_mpjpe)
+from mmpose.datasets.datasets.base import Kpt3dSviewKpt2dDataset
+from ...builder import DATASETS
+
+
+@DATASETS.register_module()
+class Body3DMpiInf3dhpDataset(Kpt3dSviewKpt2dDataset):
+ """MPI-INF-3DHP dataset for 3D human pose estimation.
+
+ "Monocular 3D Human Pose Estimation In The Wild Using Improved CNN
+ Supervision", 3DV'2017.
+ More details can be found in the `paper
+ `__.
+
+ MPI-INF-3DHP keypoint indexes:
+
+ 0: 'head_top',
+ 1: 'neck',
+ 2: 'right_shoulder',
+ 3: 'right_elbow',
+ 4: 'right_wrist',
+ 5: 'left_shoulder;,
+ 6: 'left_elbow',
+ 7: 'left_wrist',
+ 8: 'right_hip',
+ 9: 'right_knee',
+ 10: 'right_ankle',
+ 11: 'left_hip',
+ 12: 'left_knee',
+ 13: 'left_ankle',
+ 14: 'root (pelvis)',
+ 15: 'spine',
+ 16: 'head'
+
+ Args:
+ ann_file (str): Path to the annotation file.
+ img_prefix (str): Path to a directory where images are held.
+ Default: None.
+ data_cfg (dict): Data configurations. Please refer to the docstring of
+ Body3DBaseDataset for common data attributes. Here are MPI-INF-3DHP
+ specific attributes.
+ - joint_2d_src: 2D joint source. Options include:
+ "gt": from the annotation file
+ "detection": from a detection result file of 2D keypoint
+ "pipeline": will be generate by the pipeline
+ Default: "gt".
+ - joint_2d_det_file: Path to the detection result file of 2D
+ keypoint. Only used when joint_2d_src == "detection".
+ - need_camera_param: Whether need camera parameters or not.
+ Default: False.
+ pipeline (list[dict | callable]): A sequence of data transforms.
+ dataset_info (DatasetInfo): A class containing all dataset info.
+ test_mode (bool): Store True when building test or
+ validation dataset. Default: False.
+ """
+
+ JOINT_NAMES = [
+ 'HeadTop', 'Neck', 'RShoulder', 'RElbow', 'RWrist', 'LShoulder',
+ 'LElbow', 'LWrist', 'RHip', 'RKnee', 'RAnkle', 'LHip', 'LKnee',
+ 'LAnkle', 'Root', 'Spine', 'Head'
+ ]
+
+ # 2D joint source options:
+ # "gt": from the annotation file
+ # "detection": from a detection result file of 2D keypoint
+ # "pipeline": will be generate by the pipeline
+ SUPPORTED_JOINT_2D_SRC = {'gt', 'detection', 'pipeline'}
+
+ # metric
+ ALLOWED_METRICS = {
+ 'mpjpe', 'p-mpjpe', '3dpck', 'p-3dpck', '3dauc', 'p-3dauc'
+ }
+
+ def __init__(self,
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=None,
+ test_mode=False):
+
+ if dataset_info is None:
+ warnings.warn(
+ 'dataset_info is missing. '
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 '
+ 'for details.', DeprecationWarning)
+ cfg = Config.fromfile('configs/_base_/datasets/mpi_inf_3dhp.py')
+ dataset_info = cfg._cfg_dict['dataset_info']
+
+ super().__init__(
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=dataset_info,
+ test_mode=test_mode)
+
+ def load_config(self, data_cfg):
+ super().load_config(data_cfg)
+ # mpi-inf-3dhp specific attributes
+ self.joint_2d_src = data_cfg.get('joint_2d_src', 'gt')
+ if self.joint_2d_src not in self.SUPPORTED_JOINT_2D_SRC:
+ raise ValueError(
+ f'Unsupported joint_2d_src "{self.joint_2d_src}". '
+ f'Supported options are {self.SUPPORTED_JOINT_2D_SRC}')
+
+ self.joint_2d_det_file = data_cfg.get('joint_2d_det_file', None)
+
+ self.need_camera_param = data_cfg.get('need_camera_param', False)
+ if self.need_camera_param:
+ assert 'camera_param_file' in data_cfg
+ self.camera_param = self._load_camera_param(
+ data_cfg['camera_param_file'])
+
+ # mpi-inf-3dhp specific annotation info
+ ann_info = {}
+ ann_info['use_different_joint_weights'] = False
+
+ self.ann_info.update(ann_info)
+
+ def load_annotations(self):
+ data_info = super().load_annotations()
+
+ # get 2D joints
+ if self.joint_2d_src == 'gt':
+ data_info['joints_2d'] = data_info['joints_2d']
+ elif self.joint_2d_src == 'detection':
+ data_info['joints_2d'] = self._load_joint_2d_detection(
+ self.joint_2d_det_file)
+ assert data_info['joints_2d'].shape[0] == data_info[
+ 'joints_3d'].shape[0]
+ assert data_info['joints_2d'].shape[2] == 3
+ elif self.joint_2d_src == 'pipeline':
+ # joint_2d will be generated in the pipeline
+ pass
+ else:
+ raise NotImplementedError(
+ f'Unhandled joint_2d_src option {self.joint_2d_src}')
+
+ return data_info
+
+ @staticmethod
+ def _parse_mpi_inf_3dhp_imgname(imgname):
+ """Parse imgname to get information of subject, sequence and camera.
+
+ A typical mpi-inf-3dhp training image filename is like:
+ S1_Seq1_Cam0_000001.jpg. A typical mpi-inf-3dhp testing image filename
+ is like: TS1_000001.jpg
+ """
+ if imgname[0] == 'S':
+ subj, rest = imgname.split('_', 1)
+ seq, rest = rest.split('_', 1)
+ camera, rest = rest.split('_', 1)
+ return subj, seq, camera
+ else:
+ subj, rest = imgname.split('_', 1)
+ return subj, None, None
+
+ def build_sample_indices(self):
+ """Split original videos into sequences and build frame indices.
+
+ This method overrides the default one in the base class.
+ """
+
+ # Group frames into videos. Assume that self.data_info is
+ # chronological.
+ video_frames = defaultdict(list)
+ for idx, imgname in enumerate(self.data_info['imgnames']):
+ subj, seq, camera = self._parse_mpi_inf_3dhp_imgname(imgname)
+ if seq is not None:
+ video_frames[(subj, seq, camera)].append(idx)
+ else:
+ video_frames[subj].append(idx)
+
+ # build sample indices
+ sample_indices = []
+ _len = (self.seq_len - 1) * self.seq_frame_interval + 1
+ _step = self.seq_frame_interval
+ for _, _indices in sorted(video_frames.items()):
+ n_frame = len(_indices)
+
+ if self.temporal_padding:
+ # Pad the sequence so that every frame in the sequence will be
+ # predicted.
+ if self.causal:
+ frames_left = self.seq_len - 1
+ frames_right = 0
+ else:
+ frames_left = (self.seq_len - 1) // 2
+ frames_right = frames_left
+ for i in range(n_frame):
+ pad_left = max(0, frames_left - i // _step)
+ pad_right = max(0,
+ frames_right - (n_frame - 1 - i) // _step)
+ start = max(i % _step, i - frames_left * _step)
+ end = min(n_frame - (n_frame - 1 - i) % _step,
+ i + frames_right * _step + 1)
+ sample_indices.append([_indices[0]] * pad_left +
+ _indices[start:end:_step] +
+ [_indices[-1]] * pad_right)
+ else:
+ seqs_from_video = [
+ _indices[i:(i + _len):_step]
+ for i in range(0, n_frame - _len + 1)
+ ]
+ sample_indices.extend(seqs_from_video)
+
+ # reduce dataset size if self.subset < 1
+ assert 0 < self.subset <= 1
+ subset_size = int(len(sample_indices) * self.subset)
+ start = np.random.randint(0, len(sample_indices) - subset_size + 1)
+ end = start + subset_size
+
+ return sample_indices[start:end]
+
+ def _load_joint_2d_detection(self, det_file):
+ """"Load 2D joint detection results from file."""
+ joints_2d = np.load(det_file).astype(np.float32)
+
+ return joints_2d
+
+ @deprecated_api_warning(name_dict=dict(outputs='results'))
+ def evaluate(self, results, res_folder=None, metric='mpjpe', **kwargs):
+ metrics = metric if isinstance(metric, list) else [metric]
+ for _metric in metrics:
+ if _metric not in self.ALLOWED_METRICS:
+ raise ValueError(
+ f'Unsupported metric "{_metric}" for mpi-inf-3dhp dataset.'
+ f'Supported metrics are {self.ALLOWED_METRICS}')
+
+ if res_folder is not None:
+ tmp_folder = None
+ res_file = osp.join(res_folder, 'result_keypoints.json')
+ else:
+ tmp_folder = tempfile.TemporaryDirectory()
+ res_file = osp.join(tmp_folder.name, 'result_keypoints.json')
+
+ kpts = []
+ for result in results:
+ preds = result['preds']
+ image_paths = result['target_image_paths']
+ batch_size = len(image_paths)
+ for i in range(batch_size):
+ target_id = self.name2id[image_paths[i]]
+ kpts.append({
+ 'keypoints': preds[i],
+ 'target_id': target_id,
+ })
+
+ mmcv.dump(kpts, res_file)
+
+ name_value_tuples = []
+ for _metric in metrics:
+ if _metric == 'mpjpe':
+ _nv_tuples = self._report_mpjpe(kpts)
+ elif _metric == 'p-mpjpe':
+ _nv_tuples = self._report_mpjpe(kpts, mode='p-mpjpe')
+ elif _metric == '3dpck':
+ _nv_tuples = self._report_3d_pck(kpts)
+ elif _metric == 'p-3dpck':
+ _nv_tuples = self._report_3d_pck(kpts, mode='p-3dpck')
+ elif _metric == '3dauc':
+ _nv_tuples = self._report_3d_auc(kpts)
+ elif _metric == 'p-3dauc':
+ _nv_tuples = self._report_3d_auc(kpts, mode='p-3dauc')
+ else:
+ raise NotImplementedError
+ name_value_tuples.extend(_nv_tuples)
+
+ if tmp_folder is not None:
+ tmp_folder.cleanup()
+
+ return OrderedDict(name_value_tuples)
+
+ def _report_mpjpe(self, keypoint_results, mode='mpjpe'):
+ """Cauculate mean per joint position error (MPJPE) or its variants
+ P-MPJPE.
+
+ Args:
+ keypoint_results (list): Keypoint predictions. See
+ 'Body3DMpiInf3dhpDataset.evaluate' for details.
+ mode (str): Specify mpjpe variants. Supported options are:
+ - ``'mpjpe'``: Standard MPJPE.
+ - ``'p-mpjpe'``: MPJPE after aligning prediction to groundtruth
+ via a rigid transformation (scale, rotation and
+ translation).
+ """
+
+ preds = []
+ gts = []
+ for idx, result in enumerate(keypoint_results):
+ pred = result['keypoints']
+ target_id = result['target_id']
+ gt, gt_visible = np.split(
+ self.data_info['joints_3d'][target_id], [3], axis=-1)
+ preds.append(pred)
+ gts.append(gt)
+
+ preds = np.stack(preds)
+ gts = np.stack(gts)
+ masks = np.ones_like(gts[:, :, 0], dtype=bool)
+
+ err_name = mode.upper()
+ if mode == 'mpjpe':
+ alignment = 'none'
+ elif mode == 'p-mpjpe':
+ alignment = 'procrustes'
+ else:
+ raise ValueError(f'Invalid mode: {mode}')
+
+ error = keypoint_mpjpe(preds, gts, masks, alignment)
+ name_value_tuples = [(err_name, error)]
+
+ return name_value_tuples
+
+ def _report_3d_pck(self, keypoint_results, mode='3dpck'):
+ """Cauculate Percentage of Correct Keypoints (3DPCK) w. or w/o
+ Procrustes alignment.
+
+ Args:
+ keypoint_results (list): Keypoint predictions. See
+ 'Body3DMpiInf3dhpDataset.evaluate' for details.
+ mode (str): Specify mpjpe variants. Supported options are:
+ - ``'3dpck'``: Standard 3DPCK.
+ - ``'p-3dpck'``: 3DPCK after aligning prediction to groundtruth
+ via a rigid transformation (scale, rotation and
+ translation).
+ """
+
+ preds = []
+ gts = []
+ for idx, result in enumerate(keypoint_results):
+ pred = result['keypoints']
+ target_id = result['target_id']
+ gt, gt_visible = np.split(
+ self.data_info['joints_3d'][target_id], [3], axis=-1)
+ preds.append(pred)
+ gts.append(gt)
+
+ preds = np.stack(preds)
+ gts = np.stack(gts)
+ masks = np.ones_like(gts[:, :, 0], dtype=bool)
+
+ err_name = mode.upper()
+ if mode == '3dpck':
+ alignment = 'none'
+ elif mode == 'p-3dpck':
+ alignment = 'procrustes'
+ else:
+ raise ValueError(f'Invalid mode: {mode}')
+
+ error = keypoint_3d_pck(preds, gts, masks, alignment)
+ name_value_tuples = [(err_name, error)]
+
+ return name_value_tuples
+
+ def _report_3d_auc(self, keypoint_results, mode='3dauc'):
+ """Cauculate the Area Under the Curve (AUC) computed for a range of
+ 3DPCK thresholds.
+
+ Args:
+ keypoint_results (list): Keypoint predictions. See
+ 'Body3DMpiInf3dhpDataset.evaluate' for details.
+ mode (str): Specify mpjpe variants. Supported options are:
+
+ - ``'3dauc'``: Standard 3DAUC.
+ - ``'p-3dauc'``: 3DAUC after aligning prediction to
+ groundtruth via a rigid transformation (scale, rotation and
+ translation).
+ """
+
+ preds = []
+ gts = []
+ for idx, result in enumerate(keypoint_results):
+ pred = result['keypoints']
+ target_id = result['target_id']
+ gt, gt_visible = np.split(
+ self.data_info['joints_3d'][target_id], [3], axis=-1)
+ preds.append(pred)
+ gts.append(gt)
+
+ preds = np.stack(preds)
+ gts = np.stack(gts)
+ masks = np.ones_like(gts[:, :, 0], dtype=bool)
+
+ err_name = mode.upper()
+ if mode == '3dauc':
+ alignment = 'none'
+ elif mode == 'p-3dauc':
+ alignment = 'procrustes'
+ else:
+ raise ValueError(f'Invalid mode: {mode}')
+
+ error = keypoint_3d_auc(preds, gts, masks, alignment)
+ name_value_tuples = [(err_name, error)]
+
+ return name_value_tuples
+
+ def _load_camera_param(self, camear_param_file):
+ """Load camera parameters from file."""
+ return mmcv.load(camear_param_file)
+
+ def get_camera_param(self, imgname):
+ """Get camera parameters of a frame by its image name."""
+ assert hasattr(self, 'camera_param')
+ return self.camera_param[imgname[:-11]]
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/body3d/body3d_mview_direct_panoptic_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/body3d/body3d_mview_direct_panoptic_dataset.py
new file mode 100644
index 0000000..b5bf92d
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/body3d/body3d_mview_direct_panoptic_dataset.py
@@ -0,0 +1,493 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import copy
+import glob
+import json
+import os.path as osp
+import pickle
+import tempfile
+import warnings
+from collections import OrderedDict
+
+import mmcv
+import numpy as np
+from mmcv import Config, deprecated_api_warning
+
+from mmpose.core.camera import SimpleCamera
+from mmpose.datasets.builder import DATASETS
+from mmpose.datasets.datasets.base import Kpt3dMviewRgbImgDirectDataset
+
+
+@DATASETS.register_module()
+class Body3DMviewDirectPanopticDataset(Kpt3dMviewRgbImgDirectDataset):
+ """Panoptic dataset for direct multi-view human pose estimation.
+
+ `Panoptic Studio: A Massively Multiview System for Social Motion
+ Capture' ICCV'2015
+ More details can be found in the `paper
+ `__ .
+
+ The dataset loads both 2D and 3D annotations as well as camera parameters.
+
+ Panoptic keypoint indexes::
+
+ 'neck': 0,
+ 'nose': 1,
+ 'mid-hip': 2,
+ 'l-shoulder': 3,
+ 'l-elbow': 4,
+ 'l-wrist': 5,
+ 'l-hip': 6,
+ 'l-knee': 7,
+ 'l-ankle': 8,
+ 'r-shoulder': 9,
+ 'r-elbow': 10,
+ 'r-wrist': 11,
+ 'r-hip': 12,
+ 'r-knee': 13,
+ 'r-ankle': 14,
+ 'l-eye': 15,
+ 'l-ear': 16,
+ 'r-eye': 17,
+ 'r-ear': 18,
+
+ Args:
+ ann_file (str): Path to the annotation file.
+ img_prefix (str): Path to a directory where images are held.
+ Default: None.
+ data_cfg (dict): config
+ pipeline (list[dict | callable]): A sequence of data transforms.
+ dataset_info (DatasetInfo): A class containing all dataset info.
+ test_mode (bool): Store True when building test or
+ validation dataset. Default: False.
+ """
+ ALLOWED_METRICS = {'mpjpe', 'mAP'}
+
+ def __init__(self,
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=None,
+ test_mode=False):
+
+ if dataset_info is None:
+ warnings.warn(
+ 'dataset_info is missing. '
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 '
+ 'for details.', DeprecationWarning)
+ cfg = Config.fromfile('configs/_base_/datasets/panoptic_body3d.py')
+ dataset_info = cfg._cfg_dict['dataset_info']
+
+ super().__init__(
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=dataset_info,
+ test_mode=test_mode)
+
+ self.load_config(data_cfg)
+ self.ann_info['use_different_joint_weights'] = False
+
+ if ann_file is None:
+ self.db_file = osp.join(
+ img_prefix, f'group_{self.subset}_cam{self.num_cameras}.pkl')
+ else:
+ self.db_file = ann_file
+
+ if osp.exists(self.db_file):
+ with open(self.db_file, 'rb') as f:
+ info = pickle.load(f)
+ assert info['sequence_list'] == self.seq_list
+ assert info['interval'] == self.seq_frame_interval
+ assert info['cam_list'] == self.cam_list
+ self.db = info['db']
+ else:
+ self.db = self._get_db()
+ info = {
+ 'sequence_list': self.seq_list,
+ 'interval': self.seq_frame_interval,
+ 'cam_list': self.cam_list,
+ 'db': self.db
+ }
+ with open(self.db_file, 'wb') as f:
+ pickle.dump(info, f)
+
+ self.db_size = len(self.db)
+
+ print(f'=> load {len(self.db)} samples')
+
+ def load_config(self, data_cfg):
+ """Initialize dataset attributes according to the config.
+
+ Override this method to set dataset specific attributes.
+ """
+ self.num_joints = data_cfg['num_joints']
+ assert self.num_joints <= 19
+ self.seq_list = data_cfg['seq_list']
+ self.cam_list = data_cfg['cam_list']
+ self.num_cameras = data_cfg['num_cameras']
+ assert self.num_cameras == len(self.cam_list)
+ self.seq_frame_interval = data_cfg.get('seq_frame_interval', 1)
+ self.subset = data_cfg.get('subset', 'train')
+ self.need_camera_param = True
+ self.root_id = data_cfg.get('root_id', 0)
+ self.max_persons = data_cfg.get('max_num', 10)
+
+ def _get_scale(self, raw_image_size):
+ heatmap_size = self.ann_info['heatmap_size']
+ image_size = self.ann_info['image_size']
+ assert heatmap_size[0][0] / heatmap_size[0][1] \
+ == image_size[0] / image_size[1]
+ w, h = raw_image_size
+ w_resized, h_resized = image_size
+ if w / w_resized < h / h_resized:
+ w_pad = h / h_resized * w_resized
+ h_pad = h
+ else:
+ w_pad = w
+ h_pad = w / w_resized * h_resized
+
+ scale = np.array([w_pad, h_pad], dtype=np.float32)
+
+ return scale
+
+ def _get_cam(self, seq):
+ """Get camera parameters.
+
+ Args:
+ seq (str): Sequence name.
+
+ Returns: Camera parameters.
+ """
+ cam_file = osp.join(self.img_prefix, seq,
+ 'calibration_{:s}.json'.format(seq))
+ with open(cam_file) as cfile:
+ calib = json.load(cfile)
+
+ M = np.array([[1.0, 0.0, 0.0], [0.0, 0.0, -1.0], [0.0, 1.0, 0.0]])
+ cameras = {}
+ for cam in calib['cameras']:
+ if (cam['panel'], cam['node']) in self.cam_list:
+ sel_cam = {}
+ R_w2c = np.array(cam['R']).dot(M)
+ T_w2c = np.array(cam['t']).reshape((3, 1)) * 10.0 # cm to mm
+ R_c2w = R_w2c.T
+ T_c2w = -R_w2c.T @ T_w2c
+ sel_cam['R'] = R_c2w.tolist()
+ sel_cam['T'] = T_c2w.tolist()
+ sel_cam['K'] = cam['K'][:2]
+ distCoef = cam['distCoef']
+ sel_cam['k'] = [distCoef[0], distCoef[1], distCoef[4]]
+ sel_cam['p'] = [distCoef[2], distCoef[3]]
+ cameras[(cam['panel'], cam['node'])] = sel_cam
+
+ return cameras
+
+ def _get_db(self):
+ """Get dataset base.
+
+ Returns:
+ dict: the dataset base (2D and 3D information)
+ """
+ width = 1920
+ height = 1080
+ db = []
+ sample_id = 0
+ for seq in self.seq_list:
+ cameras = self._get_cam(seq)
+ curr_anno = osp.join(self.img_prefix, seq,
+ 'hdPose3d_stage1_coco19')
+ anno_files = sorted(glob.iglob('{:s}/*.json'.format(curr_anno)))
+ print(f'load sequence: {seq}', flush=True)
+ for i, file in enumerate(anno_files):
+ if i % self.seq_frame_interval == 0:
+ with open(file) as dfile:
+ bodies = json.load(dfile)['bodies']
+ if len(bodies) == 0:
+ continue
+
+ for k, cam_param in cameras.items():
+ single_view_camera = SimpleCamera(cam_param)
+ postfix = osp.basename(file).replace('body3DScene', '')
+ prefix = '{:02d}_{:02d}'.format(k[0], k[1])
+ image_file = osp.join(seq, 'hdImgs', prefix,
+ prefix + postfix)
+ image_file = image_file.replace('json', 'jpg')
+
+ all_poses_3d = np.zeros(
+ (self.max_persons, self.num_joints, 3),
+ dtype=np.float32)
+ all_poses_vis_3d = np.zeros(
+ (self.max_persons, self.num_joints, 3),
+ dtype=np.float32)
+ all_roots_3d = np.zeros((self.max_persons, 3),
+ dtype=np.float32)
+ all_poses = np.zeros(
+ (self.max_persons, self.num_joints, 3),
+ dtype=np.float32)
+
+ cnt = 0
+ person_ids = -np.ones(self.max_persons, dtype=np.int)
+ for body in bodies:
+ if cnt >= self.max_persons:
+ break
+ pose3d = np.array(body['joints19']).reshape(
+ (-1, 4))
+ pose3d = pose3d[:self.num_joints]
+
+ joints_vis = pose3d[:, -1] > 0.1
+
+ if not joints_vis[self.root_id]:
+ continue
+
+ # Coordinate transformation
+ M = np.array([[1.0, 0.0, 0.0], [0.0, 0.0, -1.0],
+ [0.0, 1.0, 0.0]])
+ pose3d[:, 0:3] = pose3d[:, 0:3].dot(M) * 10.0
+
+ all_poses_3d[cnt] = pose3d[:, :3]
+ all_roots_3d[cnt] = pose3d[self.root_id, :3]
+ all_poses_vis_3d[cnt] = np.repeat(
+ np.reshape(joints_vis, (-1, 1)), 3, axis=1)
+
+ pose2d = np.zeros((pose3d.shape[0], 3))
+ # get pose_2d from pose_3d
+ pose2d[:, :2] = single_view_camera.world_to_pixel(
+ pose3d[:, :3])
+ x_check = np.bitwise_and(pose2d[:, 0] >= 0,
+ pose2d[:, 0] <= width - 1)
+ y_check = np.bitwise_and(
+ pose2d[:, 1] >= 0, pose2d[:, 1] <= height - 1)
+ check = np.bitwise_and(x_check, y_check)
+ joints_vis[np.logical_not(check)] = 0
+ pose2d[:, -1] = joints_vis
+
+ all_poses[cnt] = pose2d
+ person_ids[cnt] = body['id']
+ cnt += 1
+
+ if cnt > 0:
+ db.append({
+ 'image_file':
+ osp.join(self.img_prefix, image_file),
+ 'joints_3d':
+ all_poses_3d,
+ 'person_ids':
+ person_ids,
+ 'joints_3d_visible':
+ all_poses_vis_3d,
+ 'joints': [all_poses],
+ 'roots_3d':
+ all_roots_3d,
+ 'camera':
+ cam_param,
+ 'num_persons':
+ cnt,
+ 'sample_id':
+ sample_id,
+ 'center':
+ np.array((width / 2, height / 2),
+ dtype=np.float32),
+ 'scale':
+ self._get_scale((width, height))
+ })
+ sample_id += 1
+ return db
+
+ @deprecated_api_warning(name_dict=dict(outputs='results'))
+ def evaluate(self, results, res_folder=None, metric='mpjpe', **kwargs):
+ """
+
+ Args:
+ results (list[dict]): Testing results containing the following
+ items:
+ - pose_3d (np.ndarray): predicted 3D human pose
+ - sample_id (np.ndarray): sample id of a frame.
+ res_folder (str, optional): The folder to save the testing
+ results. If not specified, a temp folder will be created.
+ Default: None.
+ metric (str | list[str]): Metric to be performed.
+ Defaults: 'mpjpe'.
+ **kwargs:
+
+ Returns:
+
+ """
+ pose_3ds = np.concatenate([result['pose_3d'] for result in results],
+ axis=0)
+ sample_ids = []
+ for result in results:
+ sample_ids.extend(result['sample_id'])
+
+ _results = [
+ dict(sample_id=sample_id, pose_3d=pose_3d)
+ for (sample_id, pose_3d) in zip(sample_ids, pose_3ds)
+ ]
+ _results = self._sort_and_unique_outputs(_results, key='sample_id')
+
+ metrics = metric if isinstance(metric, list) else [metric]
+ for _metric in metrics:
+ if _metric not in self.ALLOWED_METRICS:
+ raise ValueError(
+ f'Unsupported metric "{_metric}"'
+ f'Supported metrics are {self.ALLOWED_METRICS}')
+
+ if res_folder is not None:
+ tmp_folder = None
+ res_file = osp.join(res_folder, 'result_keypoints.json')
+ else:
+ tmp_folder = tempfile.TemporaryDirectory()
+ res_file = osp.join(tmp_folder.name, 'result_keypoints.json')
+
+ mmcv.dump(_results, res_file)
+
+ eval_list = []
+ gt_num = self.db_size // self.num_cameras
+ assert len(
+ _results) == gt_num, f'number mismatch: {len(_results)}, {gt_num}'
+
+ total_gt = 0
+ for i in range(gt_num):
+ index = self.num_cameras * i
+ db_rec = copy.deepcopy(self.db[index])
+ joints_3d = db_rec['joints_3d']
+ joints_3d_vis = db_rec['joints_3d_visible']
+
+ if joints_3d_vis.sum() < 1:
+ continue
+
+ pred = _results[i]['pose_3d'].copy()
+ pred = pred[pred[:, 0, 3] >= 0]
+ for pose in pred:
+ mpjpes = []
+ for (gt, gt_vis) in zip(joints_3d, joints_3d_vis):
+ vis = gt_vis[:, 0] > 0
+ if vis.sum() < 1:
+ break
+ mpjpe = np.mean(
+ np.sqrt(
+ np.sum((pose[vis, 0:3] - gt[vis])**2, axis=-1)))
+ mpjpes.append(mpjpe)
+ min_gt = np.argmin(mpjpes)
+ min_mpjpe = np.min(mpjpes)
+ score = pose[0, 4]
+ eval_list.append({
+ 'mpjpe': float(min_mpjpe),
+ 'score': float(score),
+ 'gt_id': int(total_gt + min_gt)
+ })
+
+ total_gt += (joints_3d_vis[:, :, 0].sum(-1) >= 1).sum()
+
+ mpjpe_threshold = np.arange(25, 155, 25)
+ aps = []
+ ars = []
+ for t in mpjpe_threshold:
+ ap, ar = self._eval_list_to_ap(eval_list, total_gt, t)
+ aps.append(ap)
+ ars.append(ar)
+
+ name_value_tuples = []
+ for _metric in metrics:
+ if _metric == 'mpjpe':
+ stats_names = ['RECALL 500mm', 'MPJPE 500mm']
+ info_str = list(
+ zip(stats_names, [
+ self._eval_list_to_recall(eval_list, total_gt),
+ self._eval_list_to_mpjpe(eval_list)
+ ]))
+ elif _metric == 'mAP':
+ stats_names = [
+ 'AP 25', 'AP 50', 'AP 75', 'AP 100', 'AP 125', 'AP 150',
+ 'mAP', 'AR 25', 'AR 50', 'AR 75', 'AR 100', 'AR 125',
+ 'AR 150', 'mAR'
+ ]
+ mAP = np.array(aps).mean()
+ mAR = np.array(ars).mean()
+ info_str = list(zip(stats_names, aps + [mAP] + ars + [mAR]))
+ else:
+ raise NotImplementedError
+ name_value_tuples.extend(info_str)
+
+ if tmp_folder is not None:
+ tmp_folder.cleanup()
+
+ return OrderedDict(name_value_tuples)
+
+ @staticmethod
+ def _eval_list_to_ap(eval_list, total_gt, threshold):
+ """Get Average Precision (AP) and Average Recall at a certain
+ threshold."""
+
+ eval_list.sort(key=lambda k: k['score'], reverse=True)
+ total_num = len(eval_list)
+
+ tp = np.zeros(total_num)
+ fp = np.zeros(total_num)
+ gt_det = []
+ for i, item in enumerate(eval_list):
+ if item['mpjpe'] < threshold and item['gt_id'] not in gt_det:
+ tp[i] = 1
+ gt_det.append(item['gt_id'])
+ else:
+ fp[i] = 1
+ tp = np.cumsum(tp)
+ fp = np.cumsum(fp)
+ recall = tp / (total_gt + 1e-5)
+ precise = tp / (tp + fp + 1e-5)
+ for n in range(total_num - 2, -1, -1):
+ precise[n] = max(precise[n], precise[n + 1])
+
+ precise = np.concatenate(([0], precise, [0]))
+ recall = np.concatenate(([0], recall, [1]))
+ index = np.where(recall[1:] != recall[:-1])[0]
+ ap = np.sum((recall[index + 1] - recall[index]) * precise[index + 1])
+
+ return ap, recall[-2]
+
+ @staticmethod
+ def _eval_list_to_mpjpe(eval_list, threshold=500):
+ """Get MPJPE within a certain threshold."""
+ eval_list.sort(key=lambda k: k['score'], reverse=True)
+ gt_det = []
+
+ mpjpes = []
+ for i, item in enumerate(eval_list):
+ if item['mpjpe'] < threshold and item['gt_id'] not in gt_det:
+ mpjpes.append(item['mpjpe'])
+ gt_det.append(item['gt_id'])
+
+ return np.mean(mpjpes) if len(mpjpes) > 0 else np.inf
+
+ @staticmethod
+ def _eval_list_to_recall(eval_list, total_gt, threshold=500):
+ """Get Recall at a certain threshold."""
+ gt_ids = [e['gt_id'] for e in eval_list if e['mpjpe'] < threshold]
+
+ return len(np.unique(gt_ids)) / total_gt
+
+ def __getitem__(self, idx):
+ """Get the sample given index."""
+ results = {}
+ for c in range(self.num_cameras):
+ result = copy.deepcopy(self.db[self.num_cameras * idx + c])
+ result['ann_info'] = self.ann_info
+ width = 1920
+ height = 1080
+ result['mask'] = [np.ones((height, width), dtype=np.float32)]
+ results[c] = result
+
+ return self.pipeline(results)
+
+ @staticmethod
+ def _sort_and_unique_outputs(outputs, key='sample_id'):
+ """sort outputs and remove the repeated ones."""
+ outputs = sorted(outputs, key=lambda x: x[key])
+ num_outputs = len(outputs)
+ for i in range(num_outputs - 1, 0, -1):
+ if outputs[i][key] == outputs[i - 1][key]:
+ del outputs[i]
+
+ return outputs
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/body3d/body3d_semi_supervision_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/body3d/body3d_semi_supervision_dataset.py
new file mode 100644
index 0000000..491d549
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/body3d/body3d_semi_supervision_dataset.py
@@ -0,0 +1,41 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import numpy as np
+from torch.utils.data import Dataset
+
+from mmpose.datasets.builder import DATASETS, build_dataset
+
+
+@DATASETS.register_module()
+class Body3DSemiSupervisionDataset(Dataset):
+ """Mix Dataset for semi-supervised training in 3D human pose estimation
+ task.
+
+ The dataset combines data from two datasets (a labeled one and an unlabeled
+ one) and return a dict containing data from two datasets.
+
+ Args:
+ labeled_dataset (Dataset): Dataset with 3D keypoint annotations.
+ unlabeled_dataset (Dataset): Dataset without 3D keypoint annotations.
+ """
+
+ def __init__(self, labeled_dataset, unlabeled_dataset):
+ super().__init__()
+ self.labeled_dataset = build_dataset(labeled_dataset)
+ self.unlabeled_dataset = build_dataset(unlabeled_dataset)
+ self.length = len(self.unlabeled_dataset)
+
+ def __len__(self):
+ """Get the size of the dataset."""
+ return self.length
+
+ def __getitem__(self, i):
+ """Given index, get the data from unlabeled dataset and randomly sample
+ an item from labeled dataset.
+
+ Return a dict containing data from labeled and unlabeled dataset.
+ """
+ data = self.unlabeled_dataset[i]
+ rand_ind = np.random.randint(0, len(self.labeled_dataset))
+ labeled_data = self.labeled_dataset[rand_ind]
+ data.update(labeled_data)
+ return data
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/bottom_up/__init__.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/bottom_up/__init__.py
new file mode 100644
index 0000000..2ac7937
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/bottom_up/__init__.py
@@ -0,0 +1,11 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from .bottom_up_aic import BottomUpAicDataset
+from .bottom_up_coco import BottomUpCocoDataset
+from .bottom_up_coco_wholebody import BottomUpCocoWholeBodyDataset
+from .bottom_up_crowdpose import BottomUpCrowdPoseDataset
+from .bottom_up_mhp import BottomUpMhpDataset
+
+__all__ = [
+ 'BottomUpCocoDataset', 'BottomUpCrowdPoseDataset', 'BottomUpMhpDataset',
+ 'BottomUpAicDataset', 'BottomUpCocoWholeBodyDataset'
+]
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/bottom_up/bottom_up_aic.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/bottom_up/bottom_up_aic.py
new file mode 100644
index 0000000..e56b725
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/bottom_up/bottom_up_aic.py
@@ -0,0 +1,105 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import warnings
+
+import json_tricks as json
+from mmcv import Config
+from xtcocotools.cocoeval import COCOeval
+
+from mmpose.datasets.builder import DATASETS
+from .bottom_up_coco import BottomUpCocoDataset
+
+
+@DATASETS.register_module()
+class BottomUpAicDataset(BottomUpCocoDataset):
+ """Aic dataset for bottom-up pose estimation.
+
+ "AI Challenger : A Large-scale Dataset for Going Deeper
+ in Image Understanding", arXiv'2017.
+ More details can be found in the `paper
+ `__
+
+ The dataset loads raw features and apply specified transforms
+ to return a dict containing the image tensors and other information.
+
+ AIC keypoint indexes::
+
+ 0: "right_shoulder",
+ 1: "right_elbow",
+ 2: "right_wrist",
+ 3: "left_shoulder",
+ 4: "left_elbow",
+ 5: "left_wrist",
+ 6: "right_hip",
+ 7: "right_knee",
+ 8: "right_ankle",
+ 9: "left_hip",
+ 10: "left_knee",
+ 11: "left_ankle",
+ 12: "head_top",
+ 13: "neck"
+
+ Args:
+ ann_file (str): Path to the annotation file.
+ img_prefix (str): Path to a directory where images are held.
+ Default: None.
+ data_cfg (dict): config
+ pipeline (list[dict | callable]): A sequence of data transforms.
+ dataset_info (DatasetInfo): A class containing all dataset info.
+ test_mode (bool): Store True when building test or
+ validation dataset. Default: False.
+ """
+
+ def __init__(self,
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=None,
+ test_mode=False):
+
+ if dataset_info is None:
+ warnings.warn(
+ 'dataset_info is missing. '
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 '
+ 'for details.', DeprecationWarning)
+ cfg = Config.fromfile('configs/_base_/datasets/aic.py')
+ dataset_info = cfg._cfg_dict['dataset_info']
+
+ super(BottomUpCocoDataset, self).__init__(
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=dataset_info,
+ test_mode=test_mode)
+
+ self.ann_info['use_different_joint_weights'] = False
+ print(f'=> num_images: {self.num_images}')
+
+ def _do_python_keypoint_eval(self, res_file):
+ """Keypoint evaluation using COCOAPI."""
+
+ stats_names = [
+ 'AP', 'AP .5', 'AP .75', 'AP (M)', 'AP (L)', 'AR', 'AR .5',
+ 'AR .75', 'AR (M)', 'AR (L)'
+ ]
+
+ with open(res_file, 'r') as file:
+ res_json = json.load(file)
+ if not res_json:
+ info_str = list(zip(stats_names, [
+ 0,
+ ] * len(stats_names)))
+ return info_str
+
+ coco_det = self.coco.loadRes(res_file)
+ coco_eval = COCOeval(
+ self.coco, coco_det, 'keypoints', self.sigmas, use_area=False)
+ coco_eval.params.useSegm = None
+ coco_eval.evaluate()
+ coco_eval.accumulate()
+ coco_eval.summarize()
+
+ info_str = list(zip(stats_names, coco_eval.stats))
+
+ return info_str
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/bottom_up/bottom_up_base_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/bottom_up/bottom_up_base_dataset.py
new file mode 100644
index 0000000..6a2fea5
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/bottom_up/bottom_up_base_dataset.py
@@ -0,0 +1,14 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from torch.utils.data import Dataset
+
+
+class BottomUpBaseDataset(Dataset):
+ """This class has been deprecated and replaced by
+ Kpt2dSviewRgbImgBottomUpDataset."""
+
+ def __init__(self, *args, **kwargs):
+ raise (ImportError(
+ 'BottomUpBaseDataset has been replaced by '
+ 'Kpt2dSviewRgbImgBottomUpDataset,'
+ 'check https://github.com/open-mmlab/mmpose/pull/663 for details.')
+ )
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/bottom_up/bottom_up_coco.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/bottom_up/bottom_up_coco.py
new file mode 100644
index 0000000..fa2967f
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/bottom_up/bottom_up_coco.py
@@ -0,0 +1,305 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os.path as osp
+import tempfile
+import warnings
+from collections import OrderedDict, defaultdict
+
+import json_tricks as json
+import numpy as np
+from mmcv import Config, deprecated_api_warning
+from xtcocotools.cocoeval import COCOeval
+
+from mmpose.core.post_processing import oks_nms, soft_oks_nms
+from mmpose.datasets.builder import DATASETS
+from mmpose.datasets.datasets.base import Kpt2dSviewRgbImgBottomUpDataset
+
+
+@DATASETS.register_module()
+class BottomUpCocoDataset(Kpt2dSviewRgbImgBottomUpDataset):
+ """COCO dataset for bottom-up pose estimation.
+
+ The dataset loads raw features and apply specified transforms
+ to return a dict containing the image tensors and other information.
+
+ COCO keypoint indexes::
+
+ 0: 'nose',
+ 1: 'left_eye',
+ 2: 'right_eye',
+ 3: 'left_ear',
+ 4: 'right_ear',
+ 5: 'left_shoulder',
+ 6: 'right_shoulder',
+ 7: 'left_elbow',
+ 8: 'right_elbow',
+ 9: 'left_wrist',
+ 10: 'right_wrist',
+ 11: 'left_hip',
+ 12: 'right_hip',
+ 13: 'left_knee',
+ 14: 'right_knee',
+ 15: 'left_ankle',
+ 16: 'right_ankle'
+
+ Args:
+ ann_file (str): Path to the annotation file.
+ img_prefix (str): Path to a directory where images are held.
+ Default: None.
+ data_cfg (dict): config
+ pipeline (list[dict | callable]): A sequence of data transforms.
+ dataset_info (DatasetInfo): A class containing all dataset info.
+ test_mode (bool): Store True when building test or
+ validation dataset. Default: False.
+ """
+
+ def __init__(self,
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=None,
+ test_mode=False):
+
+ if dataset_info is None:
+ warnings.warn(
+ 'dataset_info is missing. '
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 '
+ 'for details.', DeprecationWarning)
+ cfg = Config.fromfile('configs/_base_/datasets/coco.py')
+ dataset_info = cfg._cfg_dict['dataset_info']
+
+ super().__init__(
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=dataset_info,
+ test_mode=test_mode)
+
+ self.ann_info['use_different_joint_weights'] = False
+ print(f'=> num_images: {self.num_images}')
+
+ def _get_single(self, idx):
+ """Get anno for a single image.
+
+ Args:
+ idx (int): image idx
+
+ Returns:
+ dict: info for model training
+ """
+ coco = self.coco
+ img_id = self.img_ids[idx]
+ ann_ids = coco.getAnnIds(imgIds=img_id)
+ anno = coco.loadAnns(ann_ids)
+
+ mask = self._get_mask(anno, idx)
+ anno = [
+ obj.copy() for obj in anno
+ if obj['iscrowd'] == 0 or obj['num_keypoints'] > 0
+ ]
+
+ joints = self._get_joints(anno)
+ mask_list = [mask.copy() for _ in range(self.ann_info['num_scales'])]
+ joints_list = [
+ joints.copy() for _ in range(self.ann_info['num_scales'])
+ ]
+
+ db_rec = {}
+ db_rec['dataset'] = self.dataset_name
+ db_rec['image_file'] = osp.join(self.img_prefix, self.id2name[img_id])
+ db_rec['mask'] = mask_list
+ db_rec['joints'] = joints_list
+
+ return db_rec
+
+ def _get_joints(self, anno):
+ """Get joints for all people in an image."""
+ num_people = len(anno)
+
+ if self.ann_info['scale_aware_sigma']:
+ joints = np.zeros((num_people, self.ann_info['num_joints'], 4),
+ dtype=np.float32)
+ else:
+ joints = np.zeros((num_people, self.ann_info['num_joints'], 3),
+ dtype=np.float32)
+
+ for i, obj in enumerate(anno):
+ joints[i, :, :3] = \
+ np.array(obj['keypoints']).reshape([-1, 3])
+ if self.ann_info['scale_aware_sigma']:
+ # get person box
+ box = obj['bbox']
+ size = max(box[2], box[3])
+ sigma = size / self.base_size * self.base_sigma
+ if self.int_sigma:
+ sigma = int(np.ceil(sigma))
+ assert sigma > 0, sigma
+ joints[i, :, 3] = sigma
+
+ return joints
+
+ @deprecated_api_warning(name_dict=dict(outputs='results'))
+ def evaluate(self, results, res_folder=None, metric='mAP', **kwargs):
+ """Evaluate coco keypoint results. The pose prediction results will be
+ saved in ``${res_folder}/result_keypoints.json``.
+
+ Note:
+ - num_people: P
+ - num_keypoints: K
+
+ Args:
+ results (list[dict]): Testing results containing the following
+ items:
+
+ - preds (list[np.ndarray(P, K, 3+tag_num)]): \
+ Pose predictions for all people in images.
+ - scores (list[P]): List of person scores.
+ - image_path (list[str]): For example, ['coco/images/\
+ val2017/000000397133.jpg']
+ - heatmap (np.ndarray[N, K, H, W]): model outputs.
+
+ res_folder (str, optional): The folder to save the testing
+ results. If not specified, a temp folder will be created.
+ Default: None.
+ metric (str | list[str]): Metric to be performed. Defaults: 'mAP'.
+
+ Returns:
+ dict: Evaluation results for evaluation metric.
+ """
+ metrics = metric if isinstance(metric, list) else [metric]
+ allowed_metrics = ['mAP']
+ for metric in metrics:
+ if metric not in allowed_metrics:
+ raise KeyError(f'metric {metric} is not supported')
+
+ if res_folder is not None:
+ tmp_folder = None
+ res_file = osp.join(res_folder, 'result_keypoints.json')
+ else:
+ tmp_folder = tempfile.TemporaryDirectory()
+ res_file = osp.join(tmp_folder.name, 'result_keypoints.json')
+
+ preds = []
+ scores = []
+ image_paths = []
+
+ for result in results:
+ preds.append(result['preds'])
+ scores.append(result['scores'])
+ image_paths.append(result['image_paths'][0])
+
+ kpts = defaultdict(list)
+ # iterate over images
+ for idx, _preds in enumerate(preds):
+ str_image_path = image_paths[idx]
+ image_id = self.name2id[osp.basename(str_image_path)]
+ # iterate over people
+ for idx_person, kpt in enumerate(_preds):
+ # use bbox area
+ area = (np.max(kpt[:, 0]) - np.min(kpt[:, 0])) * (
+ np.max(kpt[:, 1]) - np.min(kpt[:, 1]))
+
+ kpts[image_id].append({
+ 'keypoints': kpt[:, 0:3],
+ 'score': scores[idx][idx_person],
+ 'tags': kpt[:, 3],
+ 'image_id': image_id,
+ 'area': area,
+ })
+
+ valid_kpts = []
+ for img in kpts.keys():
+ img_kpts = kpts[img]
+ if self.use_nms:
+ nms = soft_oks_nms if self.soft_nms else oks_nms
+ keep = nms(img_kpts, self.oks_thr, sigmas=self.sigmas)
+ valid_kpts.append([img_kpts[_keep] for _keep in keep])
+ else:
+ valid_kpts.append(img_kpts)
+
+ self._write_coco_keypoint_results(valid_kpts, res_file)
+
+ info_str = self._do_python_keypoint_eval(res_file)
+ name_value = OrderedDict(info_str)
+
+ if tmp_folder is not None:
+ tmp_folder.cleanup()
+
+ return name_value
+
+ def _write_coco_keypoint_results(self, keypoints, res_file):
+ """Write results into a json file."""
+ data_pack = [{
+ 'cat_id': self._class_to_coco_ind[cls],
+ 'cls_ind': cls_ind,
+ 'cls': cls,
+ 'ann_type': 'keypoints',
+ 'keypoints': keypoints
+ } for cls_ind, cls in enumerate(self.classes)
+ if not cls == '__background__']
+
+ results = self._coco_keypoint_results_one_category_kernel(data_pack[0])
+
+ with open(res_file, 'w') as f:
+ json.dump(results, f, sort_keys=True, indent=4)
+
+ def _coco_keypoint_results_one_category_kernel(self, data_pack):
+ """Get coco keypoint results."""
+ cat_id = data_pack['cat_id']
+ keypoints = data_pack['keypoints']
+ cat_results = []
+
+ for img_kpts in keypoints:
+ if len(img_kpts) == 0:
+ continue
+
+ _key_points = np.array(
+ [img_kpt['keypoints'] for img_kpt in img_kpts])
+ key_points = _key_points.reshape(-1,
+ self.ann_info['num_joints'] * 3)
+
+ for img_kpt, key_point in zip(img_kpts, key_points):
+ kpt = key_point.reshape((self.ann_info['num_joints'], 3))
+ left_top = np.amin(kpt, axis=0)
+ right_bottom = np.amax(kpt, axis=0)
+
+ w = right_bottom[0] - left_top[0]
+ h = right_bottom[1] - left_top[1]
+
+ cat_results.append({
+ 'image_id': img_kpt['image_id'],
+ 'category_id': cat_id,
+ 'keypoints': key_point.tolist(),
+ 'score': img_kpt['score'],
+ 'bbox': [left_top[0], left_top[1], w, h]
+ })
+
+ return cat_results
+
+ def _do_python_keypoint_eval(self, res_file):
+ """Keypoint evaluation using COCOAPI."""
+
+ stats_names = [
+ 'AP', 'AP .5', 'AP .75', 'AP (M)', 'AP (L)', 'AR', 'AR .5',
+ 'AR .75', 'AR (M)', 'AR (L)'
+ ]
+
+ with open(res_file, 'r') as file:
+ res_json = json.load(file)
+ if not res_json:
+ info_str = list(zip(stats_names, [
+ 0,
+ ] * len(stats_names)))
+ return info_str
+
+ coco_det = self.coco.loadRes(res_file)
+ coco_eval = COCOeval(self.coco, coco_det, 'keypoints', self.sigmas)
+ coco_eval.params.useSegm = None
+ coco_eval.evaluate()
+ coco_eval.accumulate()
+ coco_eval.summarize()
+
+ info_str = list(zip(stats_names, coco_eval.stats))
+
+ return info_str
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/bottom_up/bottom_up_coco_wholebody.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/bottom_up/bottom_up_coco_wholebody.py
new file mode 100644
index 0000000..363d2ef
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/bottom_up/bottom_up_coco_wholebody.py
@@ -0,0 +1,238 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import warnings
+
+import numpy as np
+from mmcv import Config
+from xtcocotools.cocoeval import COCOeval
+
+from mmpose.datasets.builder import DATASETS
+from .bottom_up_coco import BottomUpCocoDataset
+
+
+@DATASETS.register_module()
+class BottomUpCocoWholeBodyDataset(BottomUpCocoDataset):
+ """CocoWholeBodyDataset dataset for bottom-up pose estimation.
+
+ `Whole-Body Human Pose Estimation in the Wild', ECCV'2020.
+ More details can be found in the `paper
+ `__ .
+
+ The dataset loads raw features and apply specified transforms
+ to return a dict containing the image tensors and other information.
+
+ In total, we have 133 keypoints for wholebody pose estimation.
+
+ COCO-WholeBody keypoint indexes::
+
+ 0-16: 17 body keypoints,
+ 17-22: 6 foot keypoints,
+ 23-90: 68 face keypoints,
+ 91-132: 42 hand keypoints
+
+ Args:
+ ann_file (str): Path to the annotation file.
+ img_prefix (str): Path to a directory where images are held.
+ Default: None.
+ data_cfg (dict): config
+ pipeline (list[dict | callable]): A sequence of data transforms.
+ dataset_info (DatasetInfo): A class containing all dataset info.
+ test_mode (bool): Store True when building test or
+ validation dataset. Default: False.
+ """
+
+ def __init__(self,
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=None,
+ test_mode=False):
+
+ if dataset_info is None:
+ warnings.warn(
+ 'dataset_info is missing. '
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 '
+ 'for details.', DeprecationWarning)
+ cfg = Config.fromfile('configs/_base_/datasets/coco_wholebody.py')
+ dataset_info = cfg._cfg_dict['dataset_info']
+
+ super(BottomUpCocoDataset, self).__init__(
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=dataset_info,
+ test_mode=test_mode)
+
+ self.ann_info['use_different_joint_weights'] = False
+
+ self.body_num = 17
+ self.foot_num = 6
+ self.face_num = 68
+ self.left_hand_num = 21
+ self.right_hand_num = 21
+
+ print(f'=> num_images: {self.num_images}')
+
+ def _get_joints(self, anno):
+ """Get joints for all people in an image."""
+ num_people = len(anno)
+
+ if self.ann_info['scale_aware_sigma']:
+ joints = np.zeros((num_people, self.ann_info['num_joints'], 4),
+ dtype=np.float32)
+ else:
+ joints = np.zeros((num_people, self.ann_info['num_joints'], 3),
+ dtype=np.float32)
+
+ for i, obj in enumerate(anno):
+ keypoints = np.array(obj['keypoints'] + obj['foot_kpts'] +
+ obj['face_kpts'] + obj['lefthand_kpts'] +
+ obj['righthand_kpts']).reshape(-1, 3)
+
+ joints[i, :self.ann_info['num_joints'], :3] = keypoints
+ if self.ann_info['scale_aware_sigma']:
+ # get person box
+ box = obj['bbox']
+ size = max(box[2], box[3])
+ sigma = size / self.base_size * self.base_sigma
+ if self.int_sigma:
+ sigma = int(np.ceil(sigma))
+ assert sigma > 0, sigma
+ joints[i, :, 3] = sigma
+
+ return joints
+
+ def _coco_keypoint_results_one_category_kernel(self, data_pack):
+ """Get coco keypoint results."""
+ cat_id = data_pack['cat_id']
+ keypoints = data_pack['keypoints']
+ cat_results = []
+
+ for img_kpts in keypoints:
+ if len(img_kpts) == 0:
+ continue
+
+ _key_points = np.array(
+ [img_kpt['keypoints'] for img_kpt in img_kpts])
+ key_points = _key_points.reshape(-1,
+ self.ann_info['num_joints'] * 3)
+
+ cuts = np.cumsum([
+ 0, self.body_num, self.foot_num, self.face_num,
+ self.left_hand_num, self.right_hand_num
+ ]) * 3
+
+ for img_kpt, key_point in zip(img_kpts, key_points):
+ kpt = key_point.reshape((self.ann_info['num_joints'], 3))
+ left_top = np.amin(kpt, axis=0)
+ right_bottom = np.amax(kpt, axis=0)
+
+ w = right_bottom[0] - left_top[0]
+ h = right_bottom[1] - left_top[1]
+
+ cat_results.append({
+ 'image_id':
+ img_kpt['image_id'],
+ 'category_id':
+ cat_id,
+ 'keypoints':
+ key_point[cuts[0]:cuts[1]].tolist(),
+ 'foot_kpts':
+ key_point[cuts[1]:cuts[2]].tolist(),
+ 'face_kpts':
+ key_point[cuts[2]:cuts[3]].tolist(),
+ 'lefthand_kpts':
+ key_point[cuts[3]:cuts[4]].tolist(),
+ 'righthand_kpts':
+ key_point[cuts[4]:cuts[5]].tolist(),
+ 'score':
+ img_kpt['score'],
+ 'bbox': [left_top[0], left_top[1], w, h]
+ })
+
+ return cat_results
+
+ def _do_python_keypoint_eval(self, res_file):
+ """Keypoint evaluation using COCOAPI."""
+ coco_det = self.coco.loadRes(res_file)
+
+ cuts = np.cumsum([
+ 0, self.body_num, self.foot_num, self.face_num, self.left_hand_num,
+ self.right_hand_num
+ ])
+
+ coco_eval = COCOeval(
+ self.coco,
+ coco_det,
+ 'keypoints_body',
+ self.sigmas[cuts[0]:cuts[1]],
+ use_area=True)
+ coco_eval.params.useSegm = None
+ coco_eval.evaluate()
+ coco_eval.accumulate()
+ coco_eval.summarize()
+
+ coco_eval = COCOeval(
+ self.coco,
+ coco_det,
+ 'keypoints_foot',
+ self.sigmas[cuts[1]:cuts[2]],
+ use_area=True)
+ coco_eval.params.useSegm = None
+ coco_eval.evaluate()
+ coco_eval.accumulate()
+ coco_eval.summarize()
+
+ coco_eval = COCOeval(
+ self.coco,
+ coco_det,
+ 'keypoints_face',
+ self.sigmas[cuts[2]:cuts[3]],
+ use_area=True)
+ coco_eval.params.useSegm = None
+ coco_eval.evaluate()
+ coco_eval.accumulate()
+ coco_eval.summarize()
+
+ coco_eval = COCOeval(
+ self.coco,
+ coco_det,
+ 'keypoints_lefthand',
+ self.sigmas[cuts[3]:cuts[4]],
+ use_area=True)
+ coco_eval.params.useSegm = None
+ coco_eval.evaluate()
+ coco_eval.accumulate()
+ coco_eval.summarize()
+
+ coco_eval = COCOeval(
+ self.coco,
+ coco_det,
+ 'keypoints_righthand',
+ self.sigmas[cuts[4]:cuts[5]],
+ use_area=True)
+ coco_eval.params.useSegm = None
+ coco_eval.evaluate()
+ coco_eval.accumulate()
+ coco_eval.summarize()
+
+ coco_eval = COCOeval(
+ self.coco,
+ coco_det,
+ 'keypoints_wholebody',
+ self.sigmas,
+ use_area=True)
+ coco_eval.params.useSegm = None
+ coco_eval.evaluate()
+ coco_eval.accumulate()
+ coco_eval.summarize()
+
+ stats_names = [
+ 'AP', 'AP .5', 'AP .75', 'AP (M)', 'AP (L)', 'AR', 'AR .5',
+ 'AR .75', 'AR (M)', 'AR (L)'
+ ]
+
+ info_str = list(zip(stats_names, coco_eval.stats))
+
+ return info_str
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/bottom_up/bottom_up_crowdpose.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/bottom_up/bottom_up_crowdpose.py
new file mode 100644
index 0000000..ebabf3e
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/bottom_up/bottom_up_crowdpose.py
@@ -0,0 +1,109 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import warnings
+
+import json_tricks as json
+from mmcv import Config
+from xtcocotools.cocoeval import COCOeval
+
+from mmpose.datasets.builder import DATASETS
+from .bottom_up_coco import BottomUpCocoDataset
+
+
+@DATASETS.register_module()
+class BottomUpCrowdPoseDataset(BottomUpCocoDataset):
+ """CrowdPose dataset for bottom-up pose estimation.
+
+ "CrowdPose: Efficient Crowded Scenes Pose Estimation and
+ A New Benchmark", CVPR'2019.
+ More details can be found in the `paper
+ `__.
+
+ The dataset loads raw features and apply specified transforms
+ to return a dict containing the image tensors and other information.
+
+ CrowdPose keypoint indexes::
+
+ 0: 'left_shoulder',
+ 1: 'right_shoulder',
+ 2: 'left_elbow',
+ 3: 'right_elbow',
+ 4: 'left_wrist',
+ 5: 'right_wrist',
+ 6: 'left_hip',
+ 7: 'right_hip',
+ 8: 'left_knee',
+ 9: 'right_knee',
+ 10: 'left_ankle',
+ 11: 'right_ankle',
+ 12: 'top_head',
+ 13: 'neck'
+
+ Args:
+ ann_file (str): Path to the annotation file.
+ img_prefix (str): Path to a directory where images are held.
+ Default: None.
+ data_cfg (dict): config
+ pipeline (list[dict | callable]): A sequence of data transforms.
+ dataset_info (DatasetInfo): A class containing all dataset info.
+ test_mode (bool): Store True when building test or
+ validation dataset. Default: False.
+ """
+
+ def __init__(self,
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=None,
+ test_mode=False):
+
+ if dataset_info is None:
+ warnings.warn(
+ 'dataset_info is missing. '
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 '
+ 'for details.', DeprecationWarning)
+ cfg = Config.fromfile('configs/_base_/datasets/crowdpose.py')
+ dataset_info = cfg._cfg_dict['dataset_info']
+
+ super(BottomUpCocoDataset, self).__init__(
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=dataset_info,
+ test_mode=test_mode)
+
+ self.ann_info['use_different_joint_weights'] = False
+ print(f'=> num_images: {self.num_images}')
+
+ def _do_python_keypoint_eval(self, res_file):
+ """Keypoint evaluation using COCOAPI."""
+
+ stats_names = [
+ 'AP', 'AP .5', 'AP .75', 'AR', 'AR .5', 'AR .75', 'AP(E)', 'AP(M)',
+ 'AP(H)'
+ ]
+
+ with open(res_file, 'r') as file:
+ res_json = json.load(file)
+ if not res_json:
+ info_str = list(zip(stats_names, [
+ 0,
+ ] * len(stats_names)))
+ return info_str
+
+ coco_det = self.coco.loadRes(res_file)
+ coco_eval = COCOeval(
+ self.coco,
+ coco_det,
+ 'keypoints_crowd',
+ self.sigmas,
+ use_area=False)
+ coco_eval.params.useSegm = None
+ coco_eval.evaluate()
+ coco_eval.accumulate()
+ coco_eval.summarize()
+
+ info_str = list(zip(stats_names, coco_eval.stats))
+
+ return info_str
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/bottom_up/bottom_up_mhp.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/bottom_up/bottom_up_mhp.py
new file mode 100644
index 0000000..1438123
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/bottom_up/bottom_up_mhp.py
@@ -0,0 +1,108 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import warnings
+
+import json_tricks as json
+from mmcv import Config
+from xtcocotools.cocoeval import COCOeval
+
+from mmpose.datasets.builder import DATASETS
+from .bottom_up_coco import BottomUpCocoDataset
+
+
+@DATASETS.register_module()
+class BottomUpMhpDataset(BottomUpCocoDataset):
+ """MHPv2.0 dataset for top-down pose estimation.
+
+ "Understanding Humans in Crowded Scenes: Deep Nested Adversarial
+ Learning and A New Benchmark for Multi-Human Parsing", ACM MM'2018.
+ More details can be found in the `paper
+ `__
+
+ The dataset loads raw features and apply specified transforms
+ to return a dict containing the image tensors and other information.
+
+ MHP keypoint indexes::
+
+ 0: "right ankle",
+ 1: "right knee",
+ 2: "right hip",
+ 3: "left hip",
+ 4: "left knee",
+ 5: "left ankle",
+ 6: "pelvis",
+ 7: "thorax",
+ 8: "upper neck",
+ 9: "head top",
+ 10: "right wrist",
+ 11: "right elbow",
+ 12: "right shoulder",
+ 13: "left shoulder",
+ 14: "left elbow",
+ 15: "left wrist",
+
+ Args:
+ ann_file (str): Path to the annotation file.
+ img_prefix (str): Path to a directory where images are held.
+ Default: None.
+ data_cfg (dict): config
+ pipeline (list[dict | callable]): A sequence of data transforms.
+ dataset_info (DatasetInfo): A class containing all dataset info.
+ test_mode (bool): Store True when building test or
+ validation dataset. Default: False.
+ """
+
+ def __init__(self,
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=None,
+ test_mode=False):
+
+ if dataset_info is None:
+ warnings.warn(
+ 'dataset_info is missing. '
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 '
+ 'for details.', DeprecationWarning)
+ cfg = Config.fromfile('configs/_base_/datasets/mhp.py')
+ dataset_info = cfg._cfg_dict['dataset_info']
+
+ super(BottomUpCocoDataset, self).__init__(
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=dataset_info,
+ test_mode=test_mode)
+
+ self.ann_info['use_different_joint_weights'] = False
+ print(f'=> num_images: {self.num_images}')
+
+ def _do_python_keypoint_eval(self, res_file):
+ """Keypoint evaluation using COCOAPI."""
+
+ stats_names = [
+ 'AP', 'AP .5', 'AP .75', 'AP (M)', 'AP (L)', 'AR', 'AR .5',
+ 'AR .75', 'AR (M)', 'AR (L)'
+ ]
+
+ with open(res_file, 'r') as file:
+ res_json = json.load(file)
+ if not res_json:
+ info_str = list(zip(stats_names, [
+ 0,
+ ] * len(stats_names)))
+ return info_str
+
+ coco_det = self.coco.loadRes(res_file)
+
+ coco_eval = COCOeval(
+ self.coco, coco_det, 'keypoints', self.sigmas, use_area=False)
+ coco_eval.params.useSegm = None
+ coco_eval.evaluate()
+ coco_eval.accumulate()
+ coco_eval.summarize()
+
+ info_str = list(zip(stats_names, coco_eval.stats))
+
+ return info_str
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/face/__init__.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/face/__init__.py
new file mode 100644
index 0000000..1ba42d4
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/face/__init__.py
@@ -0,0 +1,11 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from .face_300w_dataset import Face300WDataset
+from .face_aflw_dataset import FaceAFLWDataset
+from .face_coco_wholebody_dataset import FaceCocoWholeBodyDataset
+from .face_cofw_dataset import FaceCOFWDataset
+from .face_wflw_dataset import FaceWFLWDataset
+
+__all__ = [
+ 'Face300WDataset', 'FaceAFLWDataset', 'FaceWFLWDataset', 'FaceCOFWDataset',
+ 'FaceCocoWholeBodyDataset'
+]
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/face/face_300w_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/face/face_300w_dataset.py
new file mode 100644
index 0000000..e5b602e
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/face/face_300w_dataset.py
@@ -0,0 +1,199 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os.path as osp
+import tempfile
+import warnings
+from collections import OrderedDict
+
+import numpy as np
+from mmcv import Config, deprecated_api_warning
+
+from mmpose.datasets.builder import DATASETS
+from ..base import Kpt2dSviewRgbImgTopDownDataset
+
+
+@DATASETS.register_module()
+class Face300WDataset(Kpt2dSviewRgbImgTopDownDataset):
+ """Face300W dataset for top-down face keypoint localization.
+
+ "300 faces In-the-wild challenge: Database and results",
+ Image and Vision Computing (IMAVIS) 2019.
+
+ The dataset loads raw images and apply specified transforms
+ to return a dict containing the image tensors and other information.
+
+ The landmark annotations follow the 68 points mark-up. The definition
+ can be found in `https://ibug.doc.ic.ac.uk/resources/300-W/`.
+
+ Args:
+ ann_file (str): Path to the annotation file.
+ img_prefix (str): Path to a directory where images are held.
+ Default: None.
+ data_cfg (dict): config
+ pipeline (list[dict | callable]): A sequence of data transforms.
+ dataset_info (DatasetInfo): A class containing all dataset info.
+ test_mode (bool): Store True when building test or
+ validation dataset. Default: False.
+ """
+
+ def __init__(self,
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=None,
+ test_mode=False):
+
+ if dataset_info is None:
+ warnings.warn(
+ 'dataset_info is missing. '
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 '
+ 'for details.', DeprecationWarning)
+ cfg = Config.fromfile('configs/_base_/datasets/300w.py')
+ dataset_info = cfg._cfg_dict['dataset_info']
+
+ super().__init__(
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=dataset_info,
+ test_mode=test_mode)
+
+ self.ann_info['use_different_joint_weights'] = False
+ self.db = self._get_db()
+ print(f'=> num_images: {self.num_images}')
+ print(f'=> load {len(self.db)} samples')
+
+ def _get_db(self):
+ """Load dataset."""
+ gt_db = []
+ bbox_id = 0
+ num_joints = self.ann_info['num_joints']
+ for img_id in self.img_ids:
+
+ ann_ids = self.coco.getAnnIds(imgIds=img_id, iscrowd=False)
+ objs = self.coco.loadAnns(ann_ids)
+
+ for obj in objs:
+ if max(obj['keypoints']) == 0:
+ continue
+ joints_3d = np.zeros((num_joints, 3), dtype=np.float32)
+ joints_3d_visible = np.zeros((num_joints, 3), dtype=np.float32)
+
+ keypoints = np.array(obj['keypoints']).reshape(-1, 3)
+ joints_3d[:, :2] = keypoints[:, :2]
+ joints_3d_visible[:, :2] = np.minimum(1, keypoints[:, 2:3])
+
+ if 'center' in obj and 'scale' in obj:
+ center = np.array(obj['center'])
+ scale = np.array([obj['scale'], obj['scale']]) * 1.25
+ else:
+ center, scale = self._xywh2cs(*obj['bbox'][:4], 1.25)
+
+ image_file = osp.join(self.img_prefix, self.id2name[img_id])
+ gt_db.append({
+ 'image_file': image_file,
+ 'center': center,
+ 'scale': scale,
+ 'rotation': 0,
+ 'joints_3d': joints_3d,
+ 'joints_3d_visible': joints_3d_visible,
+ 'dataset': self.dataset_name,
+ 'bbox': obj['bbox'],
+ 'bbox_score': 1,
+ 'bbox_id': bbox_id
+ })
+ bbox_id = bbox_id + 1
+ gt_db = sorted(gt_db, key=lambda x: x['bbox_id'])
+
+ return gt_db
+
+ def _get_normalize_factor(self, gts, *args, **kwargs):
+ """Get inter-ocular distance as the normalize factor, measured as the
+ Euclidean distance between the outer corners of the eyes.
+
+ Args:
+ gts (np.ndarray[N, K, 2]): Groundtruth keypoint location.
+
+ Returns:
+ np.ndarray[N, 2]: normalized factor
+ """
+
+ interocular = np.linalg.norm(
+ gts[:, 36, :] - gts[:, 45, :], axis=1, keepdims=True)
+ return np.tile(interocular, [1, 2])
+
+ @deprecated_api_warning(name_dict=dict(outputs='results'))
+ def evaluate(self, results, res_folder=None, metric='NME', **kwargs):
+ """Evaluate freihand keypoint results. The pose prediction results will
+ be saved in ``${res_folder}/result_keypoints.json``.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+ - heatmap height: H
+ - heatmap width: W
+
+ Args:
+ results (list[dict]): Testing results containing the following
+ items:
+
+ - preds (np.ndarray[1,K,3]): The first two dimensions are \
+ coordinates, score is the third dimension of the array.
+ - boxes (np.ndarray[1,6]): [center[0], center[1], scale[0], \
+ scale[1],area, score]
+ - image_path (list[str]): For example, ['300W/ibug/\
+ image_018.jpg']
+ - output_heatmap (np.ndarray[N, K, H, W]): model outputs.
+ res_folder (str, optional): The folder to save the testing
+ results. If not specified, a temp folder will be created.
+ Default: None.
+ metric (str | list[str]): Metric to be performed.
+ Options: 'NME'.
+
+ Returns:
+ dict: Evaluation results for evaluation metric.
+ """
+ metrics = metric if isinstance(metric, list) else [metric]
+ allowed_metrics = ['NME']
+ for metric in metrics:
+ if metric not in allowed_metrics:
+ raise KeyError(f'metric {metric} is not supported')
+
+ if res_folder is not None:
+ tmp_folder = None
+ res_file = osp.join(res_folder, 'result_keypoints.json')
+ else:
+ tmp_folder = tempfile.TemporaryDirectory()
+ res_file = osp.join(tmp_folder.name, 'result_keypoints.json')
+
+ kpts = []
+ for result in results:
+ preds = result['preds']
+ boxes = result['boxes']
+ image_paths = result['image_paths']
+ bbox_ids = result['bbox_ids']
+
+ batch_size = len(image_paths)
+ for i in range(batch_size):
+ image_id = self.name2id[image_paths[i][len(self.img_prefix):]]
+
+ kpts.append({
+ 'keypoints': preds[i].tolist(),
+ 'center': boxes[i][0:2].tolist(),
+ 'scale': boxes[i][2:4].tolist(),
+ 'area': float(boxes[i][4]),
+ 'score': float(boxes[i][5]),
+ 'image_id': image_id,
+ 'bbox_id': bbox_ids[i]
+ })
+ kpts = self._sort_and_unique_bboxes(kpts)
+
+ self._write_keypoint_results(kpts, res_file)
+ info_str = self._report_metric(res_file, metrics)
+ name_value = OrderedDict(info_str)
+
+ if tmp_folder is not None:
+ tmp_folder.cleanup()
+
+ return name_value
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/face/face_aflw_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/face/face_aflw_dataset.py
new file mode 100644
index 0000000..292d9ee
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/face/face_aflw_dataset.py
@@ -0,0 +1,205 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os.path as osp
+import tempfile
+import warnings
+from collections import OrderedDict
+
+import numpy as np
+from mmcv import Config, deprecated_api_warning
+
+from mmpose.datasets.builder import DATASETS
+from ..base import Kpt2dSviewRgbImgTopDownDataset
+
+
+@DATASETS.register_module()
+class FaceAFLWDataset(Kpt2dSviewRgbImgTopDownDataset):
+ """Face AFLW dataset for top-down face keypoint localization.
+
+ "Annotated Facial Landmarks in the Wild: A Large-scale,
+ Real-world Database for Facial Landmark Localization".
+ In Proc. First IEEE International Workshop on Benchmarking
+ Facial Image Analysis Technologies, 2011.
+
+ The dataset loads raw images and apply specified transforms
+ to return a dict containing the image tensors and other information.
+
+ The landmark annotations follow the 19 points mark-up. The definition
+ can be found in `https://www.tugraz.at/institute/icg/research`
+ `/team-bischof/lrs/downloads/aflw/`
+
+ Args:
+ ann_file (str): Path to the annotation file.
+ img_prefix (str): Path to a directory where images are held.
+ Default: None.
+ data_cfg (dict): config
+ pipeline (list[dict | callable]): A sequence of data transforms.
+ dataset_info (DatasetInfo): A class containing all dataset info.
+ test_mode (bool): Store True when building test or
+ validation dataset. Default: False.
+ """
+
+ def __init__(self,
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=None,
+ test_mode=False):
+
+ if dataset_info is None:
+ warnings.warn(
+ 'dataset_info is missing. '
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 '
+ 'for details.', DeprecationWarning)
+ cfg = Config.fromfile('configs/_base_/datasets/aflw.py')
+ dataset_info = cfg._cfg_dict['dataset_info']
+
+ super().__init__(
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=dataset_info,
+ test_mode=test_mode)
+
+ self.ann_info['use_different_joint_weights'] = False
+ self.db = self._get_db()
+
+ print(f'=> num_images: {self.num_images}')
+ print(f'=> load {len(self.db)} samples')
+
+ def _get_db(self):
+ """Load dataset."""
+ gt_db = []
+ bbox_id = 0
+ num_joints = self.ann_info['num_joints']
+ for img_id in self.img_ids:
+
+ ann_ids = self.coco.getAnnIds(imgIds=img_id, iscrowd=False)
+ objs = self.coco.loadAnns(ann_ids)
+
+ for obj in objs:
+ if self.test_mode:
+ # 'box_size' is used as normalization factor
+ assert 'box_size' in obj
+ if max(obj['keypoints']) == 0:
+ continue
+ joints_3d = np.zeros((num_joints, 3), dtype=np.float32)
+ joints_3d_visible = np.zeros((num_joints, 3), dtype=np.float32)
+
+ keypoints = np.array(obj['keypoints']).reshape(-1, 3)
+ joints_3d[:, :2] = keypoints[:, :2]
+ joints_3d_visible[:, :2] = np.minimum(1, keypoints[:, 2:3])
+
+ if 'center' in obj and 'scale' in obj:
+ center = np.array(obj['center'])
+ scale = np.array([obj['scale'], obj['scale']]) * 1.25
+ else:
+ center, scale = self._xywh2cs(*obj['bbox'][:4], 1.25)
+
+ image_file = osp.join(self.img_prefix, self.id2name[img_id])
+
+ gt_db.append({
+ 'image_file': image_file,
+ 'center': center,
+ 'scale': scale,
+ 'rotation': 0,
+ 'joints_3d': joints_3d,
+ 'joints_3d_visible': joints_3d_visible,
+ 'dataset': self.dataset_name,
+ 'bbox': obj['bbox'],
+ 'box_size': obj['box_size'],
+ 'bbox_score': 1,
+ 'bbox_id': bbox_id
+ })
+ bbox_id = bbox_id + 1
+ gt_db = sorted(gt_db, key=lambda x: x['bbox_id'])
+
+ return gt_db
+
+ def _get_normalize_factor(self, box_sizes, *args, **kwargs):
+ """Get normalize factor for evaluation.
+
+ Args:
+ box_sizes (np.ndarray[N, 1]): box size
+
+ Returns:
+ np.ndarray[N, 2]: normalized factor
+ """
+
+ return np.tile(box_sizes, [1, 2])
+
+ @deprecated_api_warning(name_dict=dict(outputs='results'))
+ def evaluate(self, results, res_folder=None, metric='NME', **kwargs):
+ """Evaluate freihand keypoint results. The pose prediction results will
+ be saved in ``${res_folder}/result_keypoints.json``.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+ - heatmap height: H
+ - heatmap width: W
+
+ Args:
+ results (list[dict]): Testing results containing the following
+ items:
+
+ - preds (np.ndarray[1,K,3]): The first two dimensions are \
+ coordinates, score is the third dimension of the array.
+ - boxes (np.ndarray[1,6]): [center[0], center[1], scale[0], \
+ scale[1],area, score]
+ - image_path (list[str]): For example, ['aflw/images/flickr/ \
+ 0/image00002.jpg']
+ - output_heatmap (np.ndarray[N, K, H, W]): model outputs.
+ res_folder (str, optional): The folder to save the testing
+ results. If not specified, a temp folder will be created.
+ Default: None.
+ metric (str | list[str]): Metric to be performed.
+ Options: 'NME'.
+
+ Returns:
+ dict: Evaluation results for evaluation metric.
+ """
+ metrics = metric if isinstance(metric, list) else [metric]
+ allowed_metrics = ['NME']
+ for metric in metrics:
+ if metric not in allowed_metrics:
+ raise KeyError(f'metric {metric} is not supported')
+
+ if res_folder is not None:
+ tmp_folder = None
+ res_file = osp.join(res_folder, 'result_keypoints.json')
+ else:
+ tmp_folder = tempfile.TemporaryDirectory()
+ res_file = osp.join(tmp_folder.name, 'result_keypoints.json')
+
+ kpts = []
+ for result in results:
+ preds = result['preds']
+ boxes = result['boxes']
+ image_paths = result['image_paths']
+ bbox_ids = result['bbox_ids']
+
+ batch_size = len(image_paths)
+ for i in range(batch_size):
+ image_id = self.name2id[image_paths[i][len(self.img_prefix):]]
+
+ kpts.append({
+ 'keypoints': preds[i].tolist(),
+ 'center': boxes[i][0:2].tolist(),
+ 'scale': boxes[i][2:4].tolist(),
+ 'area': float(boxes[i][4]),
+ 'score': float(boxes[i][5]),
+ 'image_id': image_id,
+ 'bbox_id': bbox_ids[i]
+ })
+ kpts = self._sort_and_unique_bboxes(kpts)
+
+ self._write_keypoint_results(kpts, res_file)
+ info_str = self._report_metric(res_file, metrics)
+ name_value = OrderedDict(info_str)
+
+ if tmp_folder is not None:
+ tmp_folder.cleanup()
+
+ return name_value
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/face/face_base_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/face/face_base_dataset.py
new file mode 100644
index 0000000..466fabb
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/face/face_base_dataset.py
@@ -0,0 +1,16 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from abc import ABCMeta
+
+from torch.utils.data import Dataset
+
+
+class FaceBaseDataset(Dataset, metaclass=ABCMeta):
+ """This class has been deprecated and replaced by
+ Kpt2dSviewRgbImgTopDownDataset."""
+
+ def __init__(self, *args, **kwargs):
+ raise (ImportError(
+ 'FaceBaseDataset has been replaced by '
+ 'Kpt2dSviewRgbImgTopDownDataset,'
+ 'check https://github.com/open-mmlab/mmpose/pull/663 for details.')
+ )
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/face/face_coco_wholebody_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/face/face_coco_wholebody_dataset.py
new file mode 100644
index 0000000..ef5117a
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/face/face_coco_wholebody_dataset.py
@@ -0,0 +1,198 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os.path as osp
+import tempfile
+import warnings
+from collections import OrderedDict
+
+import numpy as np
+from mmcv import Config, deprecated_api_warning
+
+from mmpose.datasets.builder import DATASETS
+from ..base import Kpt2dSviewRgbImgTopDownDataset
+
+
+@DATASETS.register_module()
+class FaceCocoWholeBodyDataset(Kpt2dSviewRgbImgTopDownDataset):
+ """CocoWholeBodyDataset for face keypoint localization.
+
+ `Whole-Body Human Pose Estimation in the Wild', ECCV'2020.
+ More details can be found in the `paper
+ `__ .
+
+ The dataset loads raw features and apply specified transforms
+ to return a dict containing the image tensors and other information.
+
+ The face landmark annotations follow the 68 points mark-up.
+
+ Args:
+ ann_file (str): Path to the annotation file.
+ img_prefix (str): Path to a directory where images are held.
+ Default: None.
+ data_cfg (dict): config
+ pipeline (list[dict | callable]): A sequence of data transforms.
+ dataset_info (DatasetInfo): A class containing all dataset info.
+ test_mode (bool): Store True when building test or
+ validation dataset. Default: False.
+ """
+
+ def __init__(self,
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=None,
+ test_mode=False):
+
+ if dataset_info is None:
+ warnings.warn(
+ 'dataset_info is missing. '
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 '
+ 'for details.', DeprecationWarning)
+ cfg = Config.fromfile('configs/_base_/datasets/'
+ 'coco_wholebody_face.py')
+ dataset_info = cfg._cfg_dict['dataset_info']
+
+ super().__init__(
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=dataset_info,
+ test_mode=test_mode)
+
+ self.ann_info['use_different_joint_weights'] = False
+ self.db = self._get_db()
+
+ print(f'=> num_images: {self.num_images}')
+ print(f'=> load {len(self.db)} samples')
+
+ def _get_db(self):
+ """Load dataset."""
+ gt_db = []
+ bbox_id = 0
+ num_joints = self.ann_info['num_joints']
+ for img_id in self.img_ids:
+
+ ann_ids = self.coco.getAnnIds(imgIds=img_id, iscrowd=False)
+ objs = self.coco.loadAnns(ann_ids)
+
+ for obj in objs:
+ if obj['face_valid'] and max(obj['face_kpts']) > 0:
+ joints_3d = np.zeros((num_joints, 3), dtype=np.float32)
+ joints_3d_visible = np.zeros((num_joints, 3),
+ dtype=np.float32)
+
+ keypoints = np.array(obj['face_kpts']).reshape(-1, 3)
+ joints_3d[:, :2] = keypoints[:, :2]
+ joints_3d_visible[:, :2] = np.minimum(1, keypoints[:, 2:3])
+
+ center, scale = self._xywh2cs(*obj['face_box'][:4], 1.25)
+
+ image_file = osp.join(self.img_prefix,
+ self.id2name[img_id])
+ gt_db.append({
+ 'image_file': image_file,
+ 'center': center,
+ 'scale': scale,
+ 'rotation': 0,
+ 'joints_3d': joints_3d,
+ 'joints_3d_visible': joints_3d_visible,
+ 'dataset': self.dataset_name,
+ 'bbox': obj['face_box'],
+ 'bbox_score': 1,
+ 'bbox_id': bbox_id
+ })
+ bbox_id = bbox_id + 1
+ gt_db = sorted(gt_db, key=lambda x: x['bbox_id'])
+
+ return gt_db
+
+ def _get_normalize_factor(self, gts, *args, **kwargs):
+ """Get inter-ocular distance as the normalize factor, measured as the
+ Euclidean distance between the outer corners of the eyes.
+
+ Args:
+ gts (np.ndarray[N, K, 2]): Groundtruth keypoint location.
+
+ Returns:
+ np.ndarray[N, 2]: normalized factor
+ """
+
+ interocular = np.linalg.norm(
+ gts[:, 36, :] - gts[:, 45, :], axis=1, keepdims=True)
+ return np.tile(interocular, [1, 2])
+
+ @deprecated_api_warning(name_dict=dict(outputs='results'))
+ def evaluate(self, results, res_folder=None, metric='NME', **kwargs):
+ """Evaluate COCO-WholeBody Face keypoint results. The pose prediction
+ results will be saved in ``${res_folder}/result_keypoints.json``.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+ - heatmap height: H
+ - heatmap width: W
+
+ Args:
+ results (list[dict]): Testing results containing the following
+ items:
+
+ - preds (np.ndarray[1,K,3]): The first two dimensions are \
+ coordinates, score is the third dimension of the array.
+ - boxes (np.ndarray[1,6]): [center[0], center[1], scale[0], \
+ scale[1],area, score]
+ - image_path (list[str]): For example, ['coco/train2017/\
+ 000000000009.jpg']
+ - output_heatmap (np.ndarray[N, K, H, W]): model outputs.
+ res_folder (str, optional): The folder to save the testing
+ results. If not specified, a temp folder will be created.
+ Default: None.
+ metric (str | list[str]): Metric to be performed.
+ Options: 'NME'.
+
+ Returns:
+ dict: Evaluation results for evaluation metric.
+ """
+ metrics = metric if isinstance(metric, list) else [metric]
+ allowed_metrics = ['NME']
+ for metric in metrics:
+ if metric not in allowed_metrics:
+ raise KeyError(f'metric {metric} is not supported')
+
+ if res_folder is not None:
+ tmp_folder = None
+ res_file = osp.join(res_folder, 'result_keypoints.json')
+ else:
+ tmp_folder = tempfile.TemporaryDirectory()
+ res_file = osp.join(tmp_folder.name, 'result_keypoints.json')
+
+ kpts = []
+ for result in results:
+ preds = result['preds']
+ boxes = result['boxes']
+ image_paths = result['image_paths']
+ bbox_ids = result['bbox_ids']
+
+ batch_size = len(image_paths)
+ for i in range(batch_size):
+ image_id = self.name2id[image_paths[i][len(self.img_prefix):]]
+
+ kpts.append({
+ 'keypoints': preds[i].tolist(),
+ 'center': boxes[i][0:2].tolist(),
+ 'scale': boxes[i][2:4].tolist(),
+ 'area': float(boxes[i][4]),
+ 'score': float(boxes[i][5]),
+ 'image_id': image_id,
+ 'bbox_id': bbox_ids[i]
+ })
+ kpts = self._sort_and_unique_bboxes(kpts)
+
+ self._write_keypoint_results(kpts, res_file)
+ info_str = self._report_metric(res_file, metrics)
+ name_value = OrderedDict(info_str)
+
+ if tmp_folder is not None:
+ tmp_folder.cleanup()
+
+ return name_value
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/face/face_cofw_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/face/face_cofw_dataset.py
new file mode 100644
index 0000000..456ea0e
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/face/face_cofw_dataset.py
@@ -0,0 +1,198 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os.path as osp
+import tempfile
+import warnings
+from collections import OrderedDict
+
+import numpy as np
+from mmcv import Config, deprecated_api_warning
+
+from mmpose.datasets.builder import DATASETS
+from ..base import Kpt2dSviewRgbImgTopDownDataset
+
+
+@DATASETS.register_module()
+class FaceCOFWDataset(Kpt2dSviewRgbImgTopDownDataset):
+ """Face COFW dataset for top-down face keypoint localization.
+
+ "Robust face landmark estimation under occlusion", ICCV'2013.
+
+ The dataset loads raw images and apply specified transforms
+ to return a dict containing the image tensors and other information.
+
+ The landmark annotations follow the 29 points mark-up. The definition
+ can be found in `http://www.vision.caltech.edu/xpburgos/ICCV13/`.
+
+ Args:
+ ann_file (str): Path to the annotation file.
+ img_prefix (str): Path to a directory where images are held.
+ Default: None.
+ data_cfg (dict): config
+ pipeline (list[dict | callable]): A sequence of data transforms.
+ dataset_info (DatasetInfo): A class containing all dataset info.
+ test_mode (bool): Store True when building test or
+ validation dataset. Default: False.
+ """
+
+ def __init__(self,
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=None,
+ test_mode=False):
+
+ if dataset_info is None:
+ warnings.warn(
+ 'dataset_info is missing. '
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 '
+ 'for details.', DeprecationWarning)
+ cfg = Config.fromfile('configs/_base_/datasets/cofw.py')
+ dataset_info = cfg._cfg_dict['dataset_info']
+
+ super().__init__(
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=dataset_info,
+ test_mode=test_mode)
+
+ self.ann_info['use_different_joint_weights'] = False
+ self.db = self._get_db()
+
+ print(f'=> num_images: {self.num_images}')
+ print(f'=> load {len(self.db)} samples')
+
+ def _get_db(self):
+ """Load dataset."""
+ gt_db = []
+ bbox_id = 0
+ num_joints = self.ann_info['num_joints']
+ for img_id in self.img_ids:
+
+ ann_ids = self.coco.getAnnIds(imgIds=img_id, iscrowd=False)
+ objs = self.coco.loadAnns(ann_ids)
+
+ for obj in objs:
+ if max(obj['keypoints']) == 0:
+ continue
+ joints_3d = np.zeros((num_joints, 3), dtype=np.float32)
+ joints_3d_visible = np.zeros((num_joints, 3), dtype=np.float32)
+
+ keypoints = np.array(obj['keypoints']).reshape(-1, 3)
+ joints_3d[:, :2] = keypoints[:, :2]
+ joints_3d_visible[:, :2] = np.minimum(1, keypoints[:, 2:3])
+
+ if 'center' in obj and 'scale' in obj:
+ center = np.array(obj['center'])
+ scale = np.array([obj['scale'], obj['scale']]) * 1.25
+ else:
+ center, scale = self._xywh2cs(*obj['bbox'][:4], 1.25)
+
+ image_file = osp.join(self.img_prefix, self.id2name[img_id])
+ gt_db.append({
+ 'image_file': image_file,
+ 'center': center,
+ 'scale': scale,
+ 'rotation': 0,
+ 'joints_3d': joints_3d,
+ 'joints_3d_visible': joints_3d_visible,
+ 'dataset': self.dataset_name,
+ 'bbox': obj['bbox'],
+ 'bbox_score': 1,
+ 'bbox_id': bbox_id
+ })
+ bbox_id = bbox_id + 1
+ gt_db = sorted(gt_db, key=lambda x: x['bbox_id'])
+
+ return gt_db
+
+ def _get_normalize_factor(self, gts, *args, **kwargs):
+ """Get normalize factor for evaluation.
+
+ Args:
+ gts (np.ndarray[N, K, 2]): Groundtruth keypoint location.
+
+ Returns:
+ np.ndarray[N, 2]: normalized factor
+ """
+
+ interocular = np.linalg.norm(
+ gts[:, 8, :] - gts[:, 9, :], axis=1, keepdims=True)
+ return np.tile(interocular, [1, 2])
+
+ @deprecated_api_warning(name_dict=dict(outputs='results'))
+ def evaluate(self, results, res_folder=None, metric='NME', **kwargs):
+ """Evaluate freihand keypoint results. The pose prediction results will
+ be saved in ``${res_folder}/result_keypoints.json``.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+ - heatmap height: H
+ - heatmap width: W
+
+ Args:
+ results (list[dict]): Testing results containing the following
+ items:
+
+ - preds (np.ndarray[1,K,3]): The first two dimensions are \
+ coordinates, score is the third dimension of the array.
+ - boxes (np.ndarray[1,6]): [center[0], center[1], scale[0], \
+ scale[1],area, score]
+ - image_path (list[str]): For example, ['cofw/images/\
+ 000001.jpg']
+ - output_heatmap (np.ndarray[N, K, H, W]): model outputs.
+ res_folder (str, optional): The folder to save the testing
+ results. If not specified, a temp folder will be created.
+ Default: None.
+ metric (str | list[str]): Metric to be performed.
+ Options: 'NME'.
+
+ Returns:
+ dict: Evaluation results for evaluation metric.
+ """
+ metrics = metric if isinstance(metric, list) else [metric]
+ allowed_metrics = ['NME']
+ for metric in metrics:
+ if metric not in allowed_metrics:
+ raise KeyError(f'metric {metric} is not supported')
+
+ if res_folder is not None:
+ tmp_folder = None
+ res_file = osp.join(res_folder, 'result_keypoints.json')
+ else:
+ tmp_folder = tempfile.TemporaryDirectory()
+ res_file = osp.join(tmp_folder.name, 'result_keypoints.json')
+
+ kpts = []
+ for result in results:
+ preds = result['preds']
+ boxes = result['boxes']
+ image_paths = result['image_paths']
+ bbox_ids = result['bbox_ids']
+
+ batch_size = len(image_paths)
+ for i in range(batch_size):
+ image_id = self.name2id[image_paths[i][len(self.img_prefix):]]
+
+ kpts.append({
+ 'keypoints': preds[i].tolist(),
+ 'center': boxes[i][0:2].tolist(),
+ 'scale': boxes[i][2:4].tolist(),
+ 'area': float(boxes[i][4]),
+ 'score': float(boxes[i][5]),
+ 'image_id': image_id,
+ 'bbox_id': bbox_ids[i]
+ })
+ kpts = self._sort_and_unique_bboxes(kpts)
+
+ self._write_keypoint_results(kpts, res_file)
+ info_str = self._report_metric(res_file, metrics)
+ name_value = OrderedDict(info_str)
+
+ if tmp_folder is not None:
+ tmp_folder.cleanup()
+
+ return name_value
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/face/face_wflw_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/face/face_wflw_dataset.py
new file mode 100644
index 0000000..e4611e1
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/face/face_wflw_dataset.py
@@ -0,0 +1,199 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os.path as osp
+import tempfile
+import warnings
+from collections import OrderedDict
+
+import numpy as np
+from mmcv import Config, deprecated_api_warning
+
+from mmpose.datasets.builder import DATASETS
+from ..base import Kpt2dSviewRgbImgTopDownDataset
+
+
+@DATASETS.register_module()
+class FaceWFLWDataset(Kpt2dSviewRgbImgTopDownDataset):
+ """Face WFLW dataset for top-down face keypoint localization.
+
+ "Look at Boundary: A Boundary-Aware Face Alignment Algorithm",
+ CVPR'2018.
+
+ The dataset loads raw images and apply specified transforms
+ to return a dict containing the image tensors and other information.
+
+ The landmark annotations follow the 98 points mark-up. The definition
+ can be found in `https://wywu.github.io/projects/LAB/WFLW.html`.
+
+ Args:
+ ann_file (str): Path to the annotation file.
+ img_prefix (str): Path to a directory where images are held.
+ Default: None.
+ data_cfg (dict): config
+ pipeline (list[dict | callable]): A sequence of data transforms.
+ dataset_info (DatasetInfo): A class containing all dataset info.
+ test_mode (bool): Store True when building test or
+ validation dataset. Default: False.
+ """
+
+ def __init__(self,
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=None,
+ test_mode=False):
+
+ if dataset_info is None:
+ warnings.warn(
+ 'dataset_info is missing. '
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 '
+ 'for details.', DeprecationWarning)
+ cfg = Config.fromfile('configs/_base_/datasets/wflw.py')
+ dataset_info = cfg._cfg_dict['dataset_info']
+
+ super().__init__(
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=dataset_info,
+ test_mode=test_mode)
+
+ self.ann_info['use_different_joint_weights'] = False
+ self.db = self._get_db()
+
+ print(f'=> num_images: {self.num_images}')
+ print(f'=> load {len(self.db)} samples')
+
+ def _get_db(self):
+ """Load dataset."""
+ gt_db = []
+ bbox_id = 0
+ num_joints = self.ann_info['num_joints']
+ for img_id in self.img_ids:
+
+ ann_ids = self.coco.getAnnIds(imgIds=img_id, iscrowd=False)
+ objs = self.coco.loadAnns(ann_ids)
+
+ for obj in objs:
+ if max(obj['keypoints']) == 0:
+ continue
+ joints_3d = np.zeros((num_joints, 3), dtype=np.float32)
+ joints_3d_visible = np.zeros((num_joints, 3), dtype=np.float32)
+
+ keypoints = np.array(obj['keypoints']).reshape(-1, 3)
+ joints_3d[:, :2] = keypoints[:, :2]
+ joints_3d_visible[:, :2] = np.minimum(1, keypoints[:, 2:3])
+
+ if 'center' in obj and 'scale' in obj:
+ center = np.array(obj['center'])
+ scale = np.array([obj['scale'], obj['scale']]) * 1.25
+ else:
+ center, scale = self._xywh2cs(*obj['bbox'][:4], 1.25)
+
+ image_file = osp.join(self.img_prefix, self.id2name[img_id])
+ gt_db.append({
+ 'image_file': image_file,
+ 'center': center,
+ 'scale': scale,
+ 'rotation': 0,
+ 'joints_3d': joints_3d,
+ 'joints_3d_visible': joints_3d_visible,
+ 'dataset': self.dataset_name,
+ 'bbox': obj['bbox'],
+ 'bbox_score': 1,
+ 'bbox_id': bbox_id
+ })
+ bbox_id = bbox_id + 1
+ gt_db = sorted(gt_db, key=lambda x: x['bbox_id'])
+
+ return gt_db
+
+ def _get_normalize_factor(self, gts, *args, **kwargs):
+ """Get normalize factor for evaluation.
+
+ Args:
+ gts (np.ndarray[N, K, 2]): Groundtruth keypoint location.
+
+ Returns:
+ np.ndarray[N, 2]: normalized factor
+ """
+
+ interocular = np.linalg.norm(
+ gts[:, 60, :] - gts[:, 72, :], axis=1, keepdims=True)
+ return np.tile(interocular, [1, 2])
+
+ @deprecated_api_warning(name_dict=dict(outputs='results'))
+ def evaluate(self, results, res_folder=None, metric='NME', **kwargs):
+ """Evaluate freihand keypoint results. The pose prediction results will
+ be saved in ``${res_folder}/result_keypoints.json``.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+ - heatmap height: H
+ - heatmap width: W
+
+ Args:
+ results (list[dict]): Testing results containing the following
+ items:
+
+ - preds (np.ndarray[1,K,3]): The first two dimensions are \
+ coordinates, score is the third dimension of the array.
+ - boxes (np.ndarray[1,6]): [center[0], center[1], scale[0], \
+ scale[1],area, score]
+ - image_path (list[str]): For example, ['wflw/images/\
+ 0--Parade/0_Parade_marchingband_1_1015.jpg']
+ - output_heatmap (np.ndarray[N, K, H, W]): model outputs.
+ res_folder (str, optional): The folder to save the testing
+ results. If not specified, a temp folder will be created.
+ Default: None.
+ metric (str | list[str]): Metric to be performed.
+ Options: 'NME'.
+
+ Returns:
+ dict: Evaluation results for evaluation metric.
+ """
+ metrics = metric if isinstance(metric, list) else [metric]
+ allowed_metrics = ['NME']
+ for metric in metrics:
+ if metric not in allowed_metrics:
+ raise KeyError(f'metric {metric} is not supported')
+
+ if res_folder is not None:
+ tmp_folder = None
+ res_file = osp.join(res_folder, 'result_keypoints.json')
+ else:
+ tmp_folder = tempfile.TemporaryDirectory()
+ res_file = osp.join(tmp_folder.name, 'result_keypoints.json')
+
+ kpts = []
+ for result in results:
+ preds = result['preds']
+ boxes = result['boxes']
+ image_paths = result['image_paths']
+ bbox_ids = result['bbox_ids']
+
+ batch_size = len(image_paths)
+ for i in range(batch_size):
+ image_id = self.name2id[image_paths[i][len(self.img_prefix):]]
+
+ kpts.append({
+ 'keypoints': preds[i].tolist(),
+ 'center': boxes[i][0:2].tolist(),
+ 'scale': boxes[i][2:4].tolist(),
+ 'area': float(boxes[i][4]),
+ 'score': float(boxes[i][5]),
+ 'image_id': image_id,
+ 'bbox_id': bbox_ids[i]
+ })
+ kpts = self._sort_and_unique_bboxes(kpts)
+
+ self._write_keypoint_results(kpts, res_file)
+ info_str = self._report_metric(res_file, metrics)
+ name_value = OrderedDict(info_str)
+
+ if tmp_folder is not None:
+ tmp_folder.cleanup()
+
+ return name_value
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/fashion/__init__.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/fashion/__init__.py
new file mode 100644
index 0000000..575d6ed
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/fashion/__init__.py
@@ -0,0 +1,4 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from .deepfashion_dataset import DeepFashionDataset
+
+__all__ = ['DeepFashionDataset']
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/fashion/deepfashion_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/fashion/deepfashion_dataset.py
new file mode 100644
index 0000000..0fef655
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/fashion/deepfashion_dataset.py
@@ -0,0 +1,225 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os.path as osp
+import tempfile
+import warnings
+from collections import OrderedDict
+
+import numpy as np
+from mmcv import Config, deprecated_api_warning
+
+from mmpose.datasets.builder import DATASETS
+from ..base import Kpt2dSviewRgbImgTopDownDataset
+
+
+@DATASETS.register_module()
+class DeepFashionDataset(Kpt2dSviewRgbImgTopDownDataset):
+ """DeepFashion dataset (full-body clothes) for fashion landmark detection.
+
+ "DeepFashion: Powering Robust Clothes Recognition
+ and Retrieval with Rich Annotations", CVPR'2016.
+ "Fashion Landmark Detection in the Wild", ECCV'2016.
+
+ The dataset loads raw features and apply specified transforms
+ to return a dict containing the image tensors and other information.
+
+ The dataset contains 3 categories for full-body, upper-body and lower-body.
+
+ Fashion landmark indexes for upper-body clothes::
+
+ 0: 'left collar',
+ 1: 'right collar',
+ 2: 'left sleeve',
+ 3: 'right sleeve',
+ 4: 'left hem',
+ 5: 'right hem'
+
+ Fashion landmark indexes for lower-body clothes::
+
+ 0: 'left waistline',
+ 1: 'right waistline',
+ 2: 'left hem',
+ 3: 'right hem'
+
+ Fashion landmark indexes for full-body clothes::
+
+ 0: 'left collar',
+ 1: 'right collar',
+ 2: 'left sleeve',
+ 3: 'right sleeve',
+ 4: 'left waistline',
+ 5: 'right waistline',
+ 6: 'left hem',
+ 7: 'right hem'
+
+ Args:
+ ann_file (str): Path to the annotation file.
+ img_prefix (str): Path to a directory where images are held.
+ Default: None.
+ data_cfg (dict): config
+ pipeline (list[dict | callable]): A sequence of data transforms.
+ dataset_info (DatasetInfo): A class containing all dataset info.
+ test_mode (bool): Store True when building test or
+ validation dataset. Default: False.
+ """
+
+ def __init__(self,
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ subset='',
+ dataset_info=None,
+ test_mode=False):
+
+ if dataset_info is None:
+ warnings.warn(
+ 'dataset_info is missing. '
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 '
+ 'for details.', DeprecationWarning)
+ if subset != '':
+ warnings.warn(
+ 'subset is deprecated.'
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 '
+ 'for details.', DeprecationWarning)
+ if subset == 'upper':
+ cfg = Config.fromfile(
+ 'configs/_base_/datasets/deepfashion_upper.py')
+ dataset_info = cfg._cfg_dict['dataset_info']
+ elif subset == 'lower':
+ cfg = Config.fromfile(
+ 'configs/_base_/datasets/deepfashion_lower.py')
+ dataset_info = cfg._cfg_dict['dataset_info']
+ elif subset == 'full':
+ cfg = Config.fromfile(
+ 'configs/_base_/datasets/deepfashion_full.py')
+ dataset_info = cfg._cfg_dict['dataset_info']
+
+ super().__init__(
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=dataset_info,
+ test_mode=test_mode)
+
+ self.ann_info['use_different_joint_weights'] = False
+
+ self.db = self._get_db()
+
+ print(f'=> num_images: {self.num_images}')
+ print(f'=> load {len(self.db)} samples')
+
+ def _get_db(self):
+ """Load dataset."""
+ gt_db = []
+ bbox_id = 0
+ num_joints = self.ann_info['num_joints']
+ for img_id in self.img_ids:
+
+ ann_ids = self.coco.getAnnIds(imgIds=img_id, iscrowd=False)
+ objs = self.coco.loadAnns(ann_ids)
+
+ for obj in objs:
+ if max(obj['keypoints']) == 0:
+ continue
+ joints_3d = np.zeros((num_joints, 3), dtype=np.float32)
+ joints_3d_visible = np.zeros((num_joints, 3), dtype=np.float32)
+
+ keypoints = np.array(obj['keypoints']).reshape(-1, 3)
+ joints_3d[:, :2] = keypoints[:, :2]
+ joints_3d_visible[:, :2] = np.minimum(1, keypoints[:, 2:3])
+
+ # use 1.25bbox as input
+ center, scale = self._xywh2cs(*obj['bbox'][:4], 1.25)
+
+ image_file = osp.join(self.img_prefix, self.id2name[img_id])
+ gt_db.append({
+ 'image_file': image_file,
+ 'center': center,
+ 'scale': scale,
+ 'rotation': 0,
+ 'joints_3d': joints_3d,
+ 'joints_3d_visible': joints_3d_visible,
+ 'dataset': self.dataset_name,
+ 'bbox': obj['bbox'],
+ 'bbox_score': 1,
+ 'bbox_id': bbox_id
+ })
+ bbox_id = bbox_id + 1
+ gt_db = sorted(gt_db, key=lambda x: x['bbox_id'])
+
+ return gt_db
+
+ @deprecated_api_warning(name_dict=dict(outputs='results'))
+ def evaluate(self, results, res_folder=None, metric='PCK', **kwargs):
+ """Evaluate freihand keypoint results. The pose prediction results will
+ be saved in ``${res_folder}/result_keypoints.json``.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+ - heatmap height: H
+ - heatmap width: W
+
+ Args:
+ results (list[dict]): Testing results containing the following
+ items:
+
+ - preds (np.ndarray[N,K,3]): The first two dimensions are \
+ coordinates, score is the third dimension of the array.
+ - boxes (np.ndarray[N,6]): [center[0], center[1], scale[0], \
+ scale[1],area, score]
+ - image_paths (list[str]): For example, ['img_00000001.jpg']
+ - output_heatmap (np.ndarray[N, K, H, W]): model outputs.
+ res_folder (str, optional): The folder to save the testing
+ results. If not specified, a temp folder will be created.
+ Default: None.
+ metric (str | list[str]): Metric to be performed.
+ Options: 'PCK', 'AUC', 'EPE'.
+
+ Returns:
+ dict: Evaluation results for evaluation metric.
+ """
+ metrics = metric if isinstance(metric, list) else [metric]
+ allowed_metrics = ['PCK', 'AUC', 'EPE']
+ for metric in metrics:
+ if metric not in allowed_metrics:
+ raise KeyError(f'metric {metric} is not supported')
+
+ if res_folder is not None:
+ tmp_folder = None
+ res_file = osp.join(res_folder, 'result_keypoints.json')
+ else:
+ tmp_folder = tempfile.TemporaryDirectory()
+ res_file = osp.join(tmp_folder.name, 'result_keypoints.json')
+
+ kpts = []
+ for result in results:
+ preds = result['preds']
+ boxes = result['boxes']
+ image_paths = result['image_paths']
+ bbox_ids = result['bbox_ids']
+
+ batch_size = len(image_paths)
+ for i in range(batch_size):
+ image_id = self.name2id[image_paths[i][len(self.img_prefix):]]
+
+ kpts.append({
+ 'keypoints': preds[i].tolist(),
+ 'center': boxes[i][0:2].tolist(),
+ 'scale': boxes[i][2:4].tolist(),
+ 'area': float(boxes[i][4]),
+ 'score': float(boxes[i][5]),
+ 'image_id': image_id,
+ 'bbox_id': bbox_ids[i]
+ })
+ kpts = self._sort_and_unique_bboxes(kpts)
+
+ self._write_keypoint_results(kpts, res_file)
+ info_str = self._report_metric(res_file, metrics)
+ name_value = OrderedDict(info_str)
+
+ if tmp_folder is not None:
+ tmp_folder.cleanup()
+
+ return name_value
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/fashion/fashion_base_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/fashion/fashion_base_dataset.py
new file mode 100644
index 0000000..d4e5860
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/fashion/fashion_base_dataset.py
@@ -0,0 +1,16 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from abc import ABCMeta
+
+from torch.utils.data import Dataset
+
+
+class FashionBaseDataset(Dataset, metaclass=ABCMeta):
+ """This class has been deprecated and replaced by
+ Kpt2dSviewRgbImgTopDownDataset."""
+
+ def __init__(self, *args, **kwargs):
+ raise (ImportError(
+ 'FashionBaseDataset has been replaced by '
+ 'Kpt2dSviewRgbImgTopDownDataset,'
+ 'check https://github.com/open-mmlab/mmpose/pull/663 for details.')
+ )
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/hand/__init__.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/hand/__init__.py
new file mode 100644
index 0000000..49159af
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/hand/__init__.py
@@ -0,0 +1,14 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from .freihand_dataset import FreiHandDataset
+from .hand_coco_wholebody_dataset import HandCocoWholeBodyDataset
+from .interhand2d_dataset import InterHand2DDataset
+from .interhand3d_dataset import InterHand3DDataset
+from .onehand10k_dataset import OneHand10KDataset
+from .panoptic_hand2d_dataset import PanopticDataset
+from .rhd2d_dataset import Rhd2DDataset
+
+__all__ = [
+ 'FreiHandDataset', 'InterHand2DDataset', 'InterHand3DDataset',
+ 'OneHand10KDataset', 'PanopticDataset', 'Rhd2DDataset',
+ 'HandCocoWholeBodyDataset'
+]
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/hand/freihand_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/hand/freihand_dataset.py
new file mode 100644
index 0000000..e9ceeff
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/hand/freihand_dataset.py
@@ -0,0 +1,205 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os.path as osp
+import tempfile
+import warnings
+from collections import OrderedDict
+
+import numpy as np
+from mmcv import Config, deprecated_api_warning
+
+from mmpose.datasets.builder import DATASETS
+from ..base import Kpt2dSviewRgbImgTopDownDataset
+
+
+@DATASETS.register_module()
+class FreiHandDataset(Kpt2dSviewRgbImgTopDownDataset):
+ """FreiHand dataset for top-down hand pose estimation.
+
+ "FreiHAND: A Dataset for Markerless Capture of Hand Pose
+ and Shape from Single RGB Images", ICCV'2019.
+ More details can be found in the `paper
+ `__ .
+
+ The dataset loads raw features and apply specified transforms
+ to return a dict containing the image tensors and other information.
+
+ FreiHand keypoint indexes::
+
+ 0: 'wrist',
+ 1: 'thumb1',
+ 2: 'thumb2',
+ 3: 'thumb3',
+ 4: 'thumb4',
+ 5: 'forefinger1',
+ 6: 'forefinger2',
+ 7: 'forefinger3',
+ 8: 'forefinger4',
+ 9: 'middle_finger1',
+ 10: 'middle_finger2',
+ 11: 'middle_finger3',
+ 12: 'middle_finger4',
+ 13: 'ring_finger1',
+ 14: 'ring_finger2',
+ 15: 'ring_finger3',
+ 16: 'ring_finger4',
+ 17: 'pinky_finger1',
+ 18: 'pinky_finger2',
+ 19: 'pinky_finger3',
+ 20: 'pinky_finger4'
+
+ Args:
+ ann_file (str): Path to the annotation file.
+ img_prefix (str): Path to a directory where images are held.
+ Default: None.
+ data_cfg (dict): config
+ pipeline (list[dict | callable]): A sequence of data transforms.
+ dataset_info (DatasetInfo): A class containing all dataset info.
+ test_mode (bool): Store True when building test or
+ validation dataset. Default: False.
+ """
+
+ def __init__(self,
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=None,
+ test_mode=False):
+
+ if dataset_info is None:
+ warnings.warn(
+ 'dataset_info is missing. '
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 '
+ 'for details.', DeprecationWarning)
+ cfg = Config.fromfile('configs/_base_/datasets/freihand2d.py')
+ dataset_info = cfg._cfg_dict['dataset_info']
+
+ super().__init__(
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=dataset_info,
+ test_mode=test_mode)
+
+ self.ann_info['use_different_joint_weights'] = False
+ self.db = self._get_db()
+
+ print(f'=> num_images: {self.num_images}')
+ print(f'=> load {len(self.db)} samples')
+
+ def _get_db(self):
+ """Load dataset."""
+ gt_db = []
+ bbox_id = 0
+ num_joints = self.ann_info['num_joints']
+ for img_id in self.img_ids:
+
+ ann_ids = self.coco.getAnnIds(imgIds=img_id, iscrowd=False)
+ objs = self.coco.loadAnns(ann_ids)
+
+ for obj in objs:
+ if max(obj['keypoints']) == 0:
+ continue
+ joints_3d = np.zeros((num_joints, 3), dtype=np.float32)
+ joints_3d_visible = np.zeros((num_joints, 3), dtype=np.float32)
+
+ keypoints = np.array(obj['keypoints']).reshape(-1, 3)
+ joints_3d[:, :2] = keypoints[:, :2]
+ joints_3d_visible[:, :2] = np.minimum(1, keypoints[:, 2:3])
+
+ # the ori image is 224x224
+ center, scale = self._xywh2cs(0, 0, 224, 224, 0.8)
+
+ image_file = osp.join(self.img_prefix, self.id2name[img_id])
+ gt_db.append({
+ 'image_file': image_file,
+ 'center': center,
+ 'scale': scale,
+ 'rotation': 0,
+ 'joints_3d': joints_3d,
+ 'joints_3d_visible': joints_3d_visible,
+ 'dataset': self.dataset_name,
+ 'bbox': obj['bbox'],
+ 'bbox_score': 1,
+ 'bbox_id': bbox_id
+ })
+ bbox_id = bbox_id + 1
+ gt_db = sorted(gt_db, key=lambda x: x['bbox_id'])
+
+ return gt_db
+
+ @deprecated_api_warning(name_dict=dict(outputs='results'))
+ def evaluate(self, results, res_folder=None, metric='PCK', **kwargs):
+ """Evaluate freihand keypoint results. The pose prediction results will
+ be saved in ``${res_folder}/result_keypoints.json``.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+ - heatmap height: H
+ - heatmap width: W
+
+ Args:
+ results (list[dict]): Testing results containing the following
+ items:
+
+ - preds (np.ndarray[N,K,3]): The first two dimensions are \
+ coordinates, score is the third dimension of the array.
+ - boxes (np.ndarray[N,6]): [center[0], center[1], scale[0], \
+ scale[1],area, score]
+ - image_paths (list[str]): For example, ['training/rgb/\
+ 00031426.jpg']
+ - output_heatmap (np.ndarray[N, K, H, W]): model outputs.
+ res_folder (str, optional): The folder to save the testing
+ results. If not specified, a temp folder will be created.
+ Default: None.
+ metric (str | list[str]): Metric to be performed.
+ Options: 'PCK', 'AUC', 'EPE'.
+
+ Returns:
+ dict: Evaluation results for evaluation metric.
+ """
+ metrics = metric if isinstance(metric, list) else [metric]
+ allowed_metrics = ['PCK', 'AUC', 'EPE']
+ for metric in metrics:
+ if metric not in allowed_metrics:
+ raise KeyError(f'metric {metric} is not supported')
+
+ if res_folder is not None:
+ tmp_folder = None
+ res_file = osp.join(res_folder, 'result_keypoints.json')
+ else:
+ tmp_folder = tempfile.TemporaryDirectory()
+ res_file = osp.join(tmp_folder.name, 'result_keypoints.json')
+
+ kpts = []
+ for result in results:
+ preds = result['preds']
+ boxes = result['boxes']
+ image_paths = result['image_paths']
+ bbox_ids = result['bbox_ids']
+
+ batch_size = len(image_paths)
+ for i in range(batch_size):
+ image_id = self.name2id[image_paths[i][len(self.img_prefix):]]
+
+ kpts.append({
+ 'keypoints': preds[i].tolist(),
+ 'center': boxes[i][0:2].tolist(),
+ 'scale': boxes[i][2:4].tolist(),
+ 'area': float(boxes[i][4]),
+ 'score': float(boxes[i][5]),
+ 'image_id': image_id,
+ 'bbox_id': bbox_ids[i]
+ })
+ kpts = self._sort_and_unique_bboxes(kpts)
+
+ self._write_keypoint_results(kpts, res_file)
+ info_str = self._report_metric(res_file, metrics)
+ name_value = OrderedDict(info_str)
+
+ if tmp_folder is not None:
+ tmp_folder.cleanup()
+
+ return name_value
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/hand/hand_base_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/hand/hand_base_dataset.py
new file mode 100644
index 0000000..fd20846
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/hand/hand_base_dataset.py
@@ -0,0 +1,16 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from abc import ABCMeta
+
+from torch.utils.data import Dataset
+
+
+class HandBaseDataset(Dataset, metaclass=ABCMeta):
+ """This class has been deprecated and replaced by
+ Kpt2dSviewRgbImgTopDownDataset."""
+
+ def __init__(self, *args, **kwargs):
+ raise (ImportError(
+ 'HandBaseDataset has been replaced by '
+ 'Kpt2dSviewRgbImgTopDownDataset,'
+ 'check https://github.com/open-mmlab/mmpose/pull/663 for details.')
+ )
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/hand/hand_coco_wholebody_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/hand/hand_coco_wholebody_dataset.py
new file mode 100644
index 0000000..7c95cc0
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/hand/hand_coco_wholebody_dataset.py
@@ -0,0 +1,211 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os.path as osp
+import tempfile
+import warnings
+from collections import OrderedDict
+
+import numpy as np
+from mmcv import Config, deprecated_api_warning
+
+from mmpose.datasets.builder import DATASETS
+from ..base import Kpt2dSviewRgbImgTopDownDataset
+
+
+@DATASETS.register_module()
+class HandCocoWholeBodyDataset(Kpt2dSviewRgbImgTopDownDataset):
+ """CocoWholeBodyDataset for top-down hand pose estimation.
+
+ "Whole-Body Human Pose Estimation in the Wild", ECCV'2020.
+ More details can be found in the `paper
+ `__ .
+
+ The dataset loads raw features and apply specified transforms
+ to return a dict containing the image tensors and other information.
+
+ COCO-WholeBody Hand keypoint indexes::
+
+ 0: 'wrist',
+ 1: 'thumb1',
+ 2: 'thumb2',
+ 3: 'thumb3',
+ 4: 'thumb4',
+ 5: 'forefinger1',
+ 6: 'forefinger2',
+ 7: 'forefinger3',
+ 8: 'forefinger4',
+ 9: 'middle_finger1',
+ 10: 'middle_finger2',
+ 11: 'middle_finger3',
+ 12: 'middle_finger4',
+ 13: 'ring_finger1',
+ 14: 'ring_finger2',
+ 15: 'ring_finger3',
+ 16: 'ring_finger4',
+ 17: 'pinky_finger1',
+ 18: 'pinky_finger2',
+ 19: 'pinky_finger3',
+ 20: 'pinky_finger4'
+
+ Args:
+ ann_file (str): Path to the annotation file.
+ img_prefix (str): Path to a directory where images are held.
+ Default: None.
+ data_cfg (dict): config
+ pipeline (list[dict | callable]): A sequence of data transforms.
+ dataset_info (DatasetInfo): A class containing all dataset info.
+ test_mode (bool): Store True when building test or
+ validation dataset. Default: False.
+ """
+
+ def __init__(self,
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=None,
+ test_mode=False):
+
+ if dataset_info is None:
+ warnings.warn(
+ 'dataset_info is missing. '
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 '
+ 'for details.', DeprecationWarning)
+ cfg = Config.fromfile(
+ 'configs/_base_/datasets/coco_wholebody_hand.py')
+ dataset_info = cfg._cfg_dict['dataset_info']
+
+ super().__init__(
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=dataset_info,
+ test_mode=test_mode)
+
+ self.ann_info['use_different_joint_weights'] = False
+ self.db = self._get_db()
+
+ print(f'=> num_images: {self.num_images}')
+ print(f'=> load {len(self.db)} samples')
+
+ def _get_db(self):
+ """Load dataset."""
+ gt_db = []
+ bbox_id = 0
+ num_joints = self.ann_info['num_joints']
+ for img_id in self.img_ids:
+
+ ann_ids = self.coco.getAnnIds(imgIds=img_id, iscrowd=False)
+ objs = self.coco.loadAnns(ann_ids)
+
+ for obj in objs:
+ for type in ['left', 'right']:
+ if obj[f'{type}hand_valid'] and max(
+ obj[f'{type}hand_kpts']) > 0:
+ joints_3d = np.zeros((num_joints, 3), dtype=np.float32)
+ joints_3d_visible = np.zeros((num_joints, 3),
+ dtype=np.float32)
+
+ keypoints = np.array(obj[f'{type}hand_kpts']).reshape(
+ -1, 3)
+ joints_3d[:, :2] = keypoints[:, :2]
+ joints_3d_visible[:, :2] = np.minimum(
+ 1, keypoints[:, 2:3])
+
+ # use 1.25 padded bbox as input
+ center, scale = self._xywh2cs(
+ *obj[f'{type}hand_box'][:4], 1.25)
+
+ image_file = osp.join(self.img_prefix,
+ self.id2name[img_id])
+
+ gt_db.append({
+ 'image_file': image_file,
+ 'center': center,
+ 'scale': scale,
+ 'rotation': 0,
+ 'joints_3d': joints_3d,
+ 'joints_3d_visible': joints_3d_visible,
+ 'dataset': self.dataset_name,
+ 'bbox': obj[f'{type}hand_box'],
+ 'bbox_score': 1,
+ 'bbox_id': bbox_id
+ })
+ bbox_id = bbox_id + 1
+ gt_db = sorted(gt_db, key=lambda x: x['bbox_id'])
+
+ return gt_db
+
+ @deprecated_api_warning(name_dict=dict(outputs='results'))
+ def evaluate(self, results, res_folder=None, metric='PCK', **kwargs):
+ """Evaluate COCO-WholeBody Hand keypoint results. The pose prediction
+ results will be saved in ``${res_folder}/result_keypoints.json``.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+ - heatmap height: H
+ - heatmap width: W
+
+ Args:
+ results (list[dict]): Testing results containing the following
+ items:
+
+ - preds (np.ndarray[N,K,3]): The first two dimensions are \
+ coordinates, score is the third dimension of the array.
+ - boxes (np.ndarray[N,6]): [center[0], center[1], scale[0], \
+ scale[1],area, score]
+ - image_paths (list[str]): For example, ['Test/source/0.jpg']
+ - output_heatmap (np.ndarray[N, K, H, W]): model outputs.
+ res_folder (str, optional): The folder to save the testing
+ results. If not specified, a temp folder will be created.
+ Default: None.
+ metric (str | list[str]): Metric to be performed.
+ Options: 'PCK', 'AUC', 'EPE'.
+
+ Returns:
+ dict: Evaluation results for evaluation metric.
+ """
+ metrics = metric if isinstance(metric, list) else [metric]
+ allowed_metrics = ['PCK', 'AUC', 'EPE']
+ for metric in metrics:
+ if metric not in allowed_metrics:
+ raise KeyError(f'metric {metric} is not supported')
+
+ if res_folder is not None:
+ tmp_folder = None
+ res_file = osp.join(res_folder, 'result_keypoints.json')
+ else:
+ tmp_folder = tempfile.TemporaryDirectory()
+ res_file = osp.join(tmp_folder.name, 'result_keypoints.json')
+
+ kpts = []
+ for result in results:
+ preds = result['preds']
+ boxes = result['boxes']
+ image_paths = result['image_paths']
+ bbox_ids = result['bbox_ids']
+
+ batch_size = len(image_paths)
+ for i in range(batch_size):
+ image_id = self.name2id[image_paths[i][len(self.img_prefix):]]
+
+ kpts.append({
+ 'keypoints': preds[i].tolist(),
+ 'center': boxes[i][0:2].tolist(),
+ 'scale': boxes[i][2:4].tolist(),
+ 'area': float(boxes[i][4]),
+ 'score': float(boxes[i][5]),
+ 'image_id': image_id,
+ 'bbox_id': bbox_ids[i]
+ })
+ kpts = self._sort_and_unique_bboxes(kpts)
+
+ self._write_keypoint_results(kpts, res_file)
+ info_str = self._report_metric(res_file, metrics)
+ name_value = OrderedDict(info_str)
+
+ if tmp_folder is not None:
+ tmp_folder.cleanup()
+
+ return name_value
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/hand/interhand2d_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/hand/interhand2d_dataset.py
new file mode 100644
index 0000000..fea17fa
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/hand/interhand2d_dataset.py
@@ -0,0 +1,306 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os.path as osp
+import tempfile
+import warnings
+from collections import OrderedDict
+
+import json_tricks as json
+import numpy as np
+from mmcv import Config, deprecated_api_warning
+
+from mmpose.datasets.builder import DATASETS
+from ..base import Kpt2dSviewRgbImgTopDownDataset
+
+
+@DATASETS.register_module()
+class InterHand2DDataset(Kpt2dSviewRgbImgTopDownDataset):
+ """InterHand2.6M 2D dataset for top-down hand pose estimation.
+
+ "InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose
+ Estimation from a Single RGB Image", ECCV'2020.
+ More details can be found in the `paper
+ `__ .
+
+ The dataset loads raw features and apply specified transforms
+ to return a dict containing the image tensors and other information.
+
+ InterHand2.6M keypoint indexes::
+
+ 0: 'thumb4',
+ 1: 'thumb3',
+ 2: 'thumb2',
+ 3: 'thumb1',
+ 4: 'forefinger4',
+ 5: 'forefinger3',
+ 6: 'forefinger2',
+ 7: 'forefinger1',
+ 8: 'middle_finger4',
+ 9: 'middle_finger3',
+ 10: 'middle_finger2',
+ 11: 'middle_finger1',
+ 12: 'ring_finger4',
+ 13: 'ring_finger3',
+ 14: 'ring_finger2',
+ 15: 'ring_finger1',
+ 16: 'pinky_finger4',
+ 17: 'pinky_finger3',
+ 18: 'pinky_finger2',
+ 19: 'pinky_finger1',
+ 20: 'wrist'
+
+ Args:
+ ann_file (str): Path to the annotation file.
+ camera_file (str): Path to the camera file.
+ joint_file (str): Path to the joint file.
+ img_prefix (str): Path to a directory where images are held.
+ Default: None.
+ data_cfg (dict): config
+ pipeline (list[dict | callable]): A sequence of data transforms.
+ dataset_info (DatasetInfo): A class containing all dataset info.
+ test_mode (str): Store True when building test or
+ validation dataset. Default: False.
+ """
+
+ def __init__(self,
+ ann_file,
+ camera_file,
+ joint_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=None,
+ test_mode=False):
+
+ if dataset_info is None:
+ warnings.warn(
+ 'dataset_info is missing. '
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 '
+ 'for details.', DeprecationWarning)
+ cfg = Config.fromfile('configs/_base_/datasets/interhand2d.py')
+ dataset_info = cfg._cfg_dict['dataset_info']
+
+ super().__init__(
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=dataset_info,
+ test_mode=test_mode)
+
+ self.ann_info['use_different_joint_weights'] = False
+ self.camera_file = camera_file
+ self.joint_file = joint_file
+ self.db = self._get_db()
+
+ print(f'=> num_images: {self.num_images}')
+ print(f'=> load {len(self.db)} samples')
+
+ @staticmethod
+ def _cam2pixel(cam_coord, f, c):
+ """Transform the joints from their camera coordinates to their pixel
+ coordinates.
+
+ Note:
+ - N: number of joints
+
+ Args:
+ cam_coord (ndarray[N, 3]): 3D joints coordinates
+ in the camera coordinate system
+ f (ndarray[2]): focal length of x and y axis
+ c (ndarray[2]): principal point of x and y axis
+
+ Returns:
+ img_coord (ndarray[N, 3]): the coordinates (x, y, 0)
+ in the image plane.
+ """
+ x = cam_coord[:, 0] / (cam_coord[:, 2] + 1e-8) * f[0] + c[0]
+ y = cam_coord[:, 1] / (cam_coord[:, 2] + 1e-8) * f[1] + c[1]
+ z = np.zeros_like(x)
+ img_coord = np.concatenate((x[:, None], y[:, None], z[:, None]), 1)
+ return img_coord
+
+ @staticmethod
+ def _world2cam(world_coord, R, T):
+ """Transform the joints from their world coordinates to their camera
+ coordinates.
+
+ Note:
+ - N: number of joints
+
+ Args:
+ world_coord (ndarray[3, N]): 3D joints coordinates
+ in the world coordinate system
+ R (ndarray[3, 3]): camera rotation matrix
+ T (ndarray[3]): camera position (x, y, z)
+
+ Returns:
+ cam_coord (ndarray[3, N]): 3D joints coordinates
+ in the camera coordinate system
+ """
+ cam_coord = np.dot(R, world_coord - T)
+ return cam_coord
+
+ def _get_db(self):
+ """Load dataset.
+
+ Adapted from 'https://github.com/facebookresearch/InterHand2.6M/'
+ 'blob/master/data/InterHand2.6M/dataset.py'
+ Copyright (c) FaceBook Research, under CC-BY-NC 4.0 license.
+ """
+ with open(self.camera_file, 'r') as f:
+ cameras = json.load(f)
+ with open(self.joint_file, 'r') as f:
+ joints = json.load(f)
+ gt_db = []
+ bbox_id = 0
+ for img_id in self.img_ids:
+ num_joints = self.ann_info['num_joints']
+
+ ann_id = self.coco.getAnnIds(imgIds=img_id, iscrowd=False)
+ ann = self.coco.loadAnns(ann_id)[0]
+ img = self.coco.loadImgs(img_id)[0]
+
+ capture_id = str(img['capture'])
+ camera_name = img['camera']
+ frame_idx = str(img['frame_idx'])
+ image_file = osp.join(self.img_prefix, self.id2name[img_id])
+
+ camera_pos, camera_rot = np.array(
+ cameras[capture_id]['campos'][camera_name],
+ dtype=np.float32), np.array(
+ cameras[capture_id]['camrot'][camera_name],
+ dtype=np.float32)
+ focal, principal_pt = np.array(
+ cameras[capture_id]['focal'][camera_name],
+ dtype=np.float32), np.array(
+ cameras[capture_id]['princpt'][camera_name],
+ dtype=np.float32)
+ joint_world = np.array(
+ joints[capture_id][frame_idx]['world_coord'], dtype=np.float32)
+ joint_cam = self._world2cam(
+ joint_world.transpose(1, 0), camera_rot,
+ camera_pos.reshape(3, 1)).transpose(1, 0)
+ joint_img = self._cam2pixel(joint_cam, focal, principal_pt)[:, :2]
+ joint_img = joint_img.reshape(2, -1, 2)
+
+ joint_valid = np.array(
+ ann['joint_valid'], dtype=np.float32).reshape(2, -1)
+ # if root is not valid -> root-relative 3D pose is also not valid.
+ # Therefore, mark all joints as invalid
+ for hand in range(2):
+ joint_valid[hand, :] *= joint_valid[hand][-1]
+
+ if np.sum(joint_valid[hand, :]) > 11:
+ joints_3d = np.zeros((num_joints, 3), dtype=np.float32)
+ joints_3d_visible = np.zeros((num_joints, 3),
+ dtype=np.float32)
+ joints_3d[:, :2] = joint_img[hand, :, :]
+ joints_3d_visible[:, :2] = np.minimum(
+ 1, joint_valid[hand, :].reshape(-1, 1))
+
+ # use the tightest bbox enclosing all keypoints as bbox
+ bbox = [img['width'], img['height'], 0, 0]
+ for i in range(num_joints):
+ if joints_3d_visible[i][0]:
+ bbox[0] = min(bbox[0], joints_3d[i][0])
+ bbox[1] = min(bbox[1], joints_3d[i][1])
+ bbox[2] = max(bbox[2], joints_3d[i][0])
+ bbox[3] = max(bbox[3], joints_3d[i][1])
+
+ bbox[2] -= bbox[0]
+ bbox[3] -= bbox[1]
+
+ # use 1.5bbox as input
+ center, scale = self._xywh2cs(*bbox, 1.5)
+
+ gt_db.append({
+ 'image_file': image_file,
+ 'center': center,
+ 'scale': scale,
+ 'rotation': 0,
+ 'joints_3d': joints_3d,
+ 'joints_3d_visible': joints_3d_visible,
+ 'dataset': self.dataset_name,
+ 'bbox': bbox,
+ 'bbox_score': 1,
+ 'bbox_id': bbox_id
+ })
+ bbox_id = bbox_id + 1
+ gt_db = sorted(gt_db, key=lambda x: x['bbox_id'])
+
+ return gt_db
+
+ @deprecated_api_warning(name_dict=dict(outputs='results'))
+ def evaluate(self, results, res_folder=None, metric='PCK', **kwargs):
+ """Evaluate interhand2d keypoint results. The pose prediction results
+ will be saved in ``${res_folder}/result_keypoints.json``.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+ - heatmap height: H
+ - heatmap width: W
+
+ Args:
+ results (list[dict]): Testing results containing the following
+ items:
+
+ - preds (np.ndarray[N,K,3]): The first two dimensions are \
+ coordinates, score is the third dimension of the array.
+ - boxes (np.ndarray[N,6]): [center[0], center[1], scale[0], \
+ scale[1],area, score]
+ - image_paths (list[str]): For example, ['Capture12/\
+ 0390_dh_touchROM/cam410209/image62434.jpg']
+ - output_heatmap (np.ndarray[N, K, H, W]): model outputs.
+ res_folder (str, optional): The folder to save the testing
+ results. If not specified, a temp folder will be created.
+ Default: None.
+ metric (str | list[str]): Metric to be performed.
+ Options: 'PCK', 'AUC', 'EPE'.
+
+ Returns:
+ dict: Evaluation results for evaluation metric.
+ """
+ metrics = metric if isinstance(metric, list) else [metric]
+ allowed_metrics = ['PCK', 'AUC', 'EPE']
+ for metric in metrics:
+ if metric not in allowed_metrics:
+ raise KeyError(f'metric {metric} is not supported')
+
+ if res_folder is not None:
+ tmp_folder = None
+ res_file = osp.join(res_folder, 'result_keypoints.json')
+ else:
+ tmp_folder = tempfile.TemporaryDirectory()
+ res_file = osp.join(tmp_folder.name, 'result_keypoints.json')
+
+ kpts = []
+ for result in results:
+ preds = result['preds']
+ boxes = result['boxes']
+ image_paths = result['image_paths']
+ bbox_ids = result['bbox_ids']
+
+ batch_size = len(image_paths)
+ for i in range(batch_size):
+ image_id = self.name2id[image_paths[i][len(self.img_prefix):]]
+
+ kpts.append({
+ 'keypoints': preds[i].tolist(),
+ 'center': boxes[i][0:2].tolist(),
+ 'scale': boxes[i][2:4].tolist(),
+ 'area': float(boxes[i][4]),
+ 'score': float(boxes[i][5]),
+ 'image_id': image_id,
+ 'bbox_id': bbox_ids[i]
+ })
+ kpts = self._sort_and_unique_bboxes(kpts)
+
+ self._write_keypoint_results(kpts, res_file)
+ info_str = self._report_metric(res_file, metrics)
+ name_value = OrderedDict(info_str)
+
+ if tmp_folder is not None:
+ tmp_folder.cleanup()
+
+ return name_value
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/hand/interhand3d_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/hand/interhand3d_dataset.py
new file mode 100644
index 0000000..318d73f
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/hand/interhand3d_dataset.py
@@ -0,0 +1,505 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os.path as osp
+import tempfile
+import warnings
+from collections import OrderedDict
+
+import json_tricks as json
+import numpy as np
+from mmcv import Config, deprecated_api_warning
+
+from mmpose.core.evaluation.top_down_eval import keypoint_epe
+from mmpose.datasets.builder import DATASETS
+from ..base import Kpt3dSviewRgbImgTopDownDataset
+
+
+@DATASETS.register_module()
+class InterHand3DDataset(Kpt3dSviewRgbImgTopDownDataset):
+ """InterHand2.6M 3D dataset for top-down hand pose estimation.
+
+ "InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose
+ Estimation from a Single RGB Image", ECCV'2020.
+ More details can be found in the `paper
+ `__ .
+
+ The dataset loads raw features and apply specified transforms
+ to return a dict containing the image tensors and other information.
+
+ InterHand2.6M keypoint indexes::
+
+ 0: 'r_thumb4',
+ 1: 'r_thumb3',
+ 2: 'r_thumb2',
+ 3: 'r_thumb1',
+ 4: 'r_index4',
+ 5: 'r_index3',
+ 6: 'r_index2',
+ 7: 'r_index1',
+ 8: 'r_middle4',
+ 9: 'r_middle3',
+ 10: 'r_middle2',
+ 11: 'r_middle1',
+ 12: 'r_ring4',
+ 13: 'r_ring3',
+ 14: 'r_ring2',
+ 15: 'r_ring1',
+ 16: 'r_pinky4',
+ 17: 'r_pinky3',
+ 18: 'r_pinky2',
+ 19: 'r_pinky1',
+ 20: 'r_wrist',
+ 21: 'l_thumb4',
+ 22: 'l_thumb3',
+ 23: 'l_thumb2',
+ 24: 'l_thumb1',
+ 25: 'l_index4',
+ 26: 'l_index3',
+ 27: 'l_index2',
+ 28: 'l_index1',
+ 29: 'l_middle4',
+ 30: 'l_middle3',
+ 31: 'l_middle2',
+ 32: 'l_middle1',
+ 33: 'l_ring4',
+ 34: 'l_ring3',
+ 35: 'l_ring2',
+ 36: 'l_ring1',
+ 37: 'l_pinky4',
+ 38: 'l_pinky3',
+ 39: 'l_pinky2',
+ 40: 'l_pinky1',
+ 41: 'l_wrist'
+
+ Args:
+ ann_file (str): Path to the annotation file.
+ camera_file (str): Path to the camera file.
+ joint_file (str): Path to the joint file.
+ img_prefix (str): Path to a directory where images are held.
+ Default: None.
+ data_cfg (dict): config
+ pipeline (list[dict | callable]): A sequence of data transforms.
+ use_gt_root_depth (bool): Using the ground truth depth of the wrist
+ or given depth from rootnet_result_file.
+ rootnet_result_file (str): Path to the wrist depth file.
+ dataset_info (DatasetInfo): A class containing all dataset info.
+ test_mode (str): Store True when building test or
+ validation dataset. Default: False.
+ """
+
+ def __init__(self,
+ ann_file,
+ camera_file,
+ joint_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ use_gt_root_depth=True,
+ rootnet_result_file=None,
+ dataset_info=None,
+ test_mode=False):
+
+ if dataset_info is None:
+ warnings.warn(
+ 'dataset_info is missing. '
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 '
+ 'for details.', DeprecationWarning)
+ cfg = Config.fromfile('configs/_base_/datasets/interhand3d.py')
+ dataset_info = cfg._cfg_dict['dataset_info']
+
+ super().__init__(
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=dataset_info,
+ test_mode=test_mode)
+
+ self.ann_info['heatmap3d_depth_bound'] = data_cfg[
+ 'heatmap3d_depth_bound']
+ self.ann_info['heatmap_size_root'] = data_cfg['heatmap_size_root']
+ self.ann_info['root_depth_bound'] = data_cfg['root_depth_bound']
+ self.ann_info['use_different_joint_weights'] = False
+
+ self.camera_file = camera_file
+ self.joint_file = joint_file
+
+ self.use_gt_root_depth = use_gt_root_depth
+ if not self.use_gt_root_depth:
+ assert rootnet_result_file is not None
+ self.rootnet_result_file = rootnet_result_file
+
+ self.db = self._get_db()
+
+ print(f'=> num_images: {self.num_images}')
+ print(f'=> load {len(self.db)} samples')
+
+ @staticmethod
+ def _encode_handtype(hand_type):
+ if hand_type == 'right':
+ return np.array([1, 0], dtype=np.float32)
+ elif hand_type == 'left':
+ return np.array([0, 1], dtype=np.float32)
+ elif hand_type == 'interacting':
+ return np.array([1, 1], dtype=np.float32)
+ else:
+ assert 0, f'Not support hand type: {hand_type}'
+
+ def _get_db(self):
+ """Load dataset.
+
+ Adapted from 'https://github.com/facebookresearch/InterHand2.6M/'
+ 'blob/master/data/InterHand2.6M/dataset.py'
+ Copyright (c) FaceBook Research, under CC-BY-NC 4.0 license.
+ """
+ with open(self.camera_file, 'r') as f:
+ cameras = json.load(f)
+ with open(self.joint_file, 'r') as f:
+ joints = json.load(f)
+
+ if not self.use_gt_root_depth:
+ rootnet_result = {}
+ with open(self.rootnet_result_file, 'r') as f:
+ rootnet_annot = json.load(f)
+ for i in range(len(rootnet_annot)):
+ rootnet_result[str(
+ rootnet_annot[i]['annot_id'])] = rootnet_annot[i]
+
+ gt_db = []
+ bbox_id = 0
+ for img_id in self.img_ids:
+ num_joints = self.ann_info['num_joints']
+
+ ann_id = self.coco.getAnnIds(imgIds=img_id, iscrowd=False)
+ ann = self.coco.loadAnns(ann_id)[0]
+ img = self.coco.loadImgs(img_id)[0]
+
+ capture_id = str(img['capture'])
+ camera_name = img['camera']
+ frame_idx = str(img['frame_idx'])
+ image_file = osp.join(self.img_prefix, self.id2name[img_id])
+
+ camera_pos = np.array(
+ cameras[capture_id]['campos'][camera_name], dtype=np.float32)
+ camera_rot = np.array(
+ cameras[capture_id]['camrot'][camera_name], dtype=np.float32)
+ focal = np.array(
+ cameras[capture_id]['focal'][camera_name], dtype=np.float32)
+ principal_pt = np.array(
+ cameras[capture_id]['princpt'][camera_name], dtype=np.float32)
+ joint_world = np.array(
+ joints[capture_id][frame_idx]['world_coord'], dtype=np.float32)
+ joint_cam = self._world2cam(
+ joint_world.transpose(1, 0), camera_rot,
+ camera_pos.reshape(3, 1)).transpose(1, 0)
+ joint_img = self._cam2pixel(joint_cam, focal, principal_pt)[:, :2]
+
+ joint_valid = np.array(
+ ann['joint_valid'], dtype=np.float32).flatten()
+ hand_type = self._encode_handtype(ann['hand_type'])
+ hand_type_valid = ann['hand_type_valid']
+
+ if self.use_gt_root_depth:
+ bbox = np.array(ann['bbox'], dtype=np.float32)
+ # extend the bbox to include some context
+ center, scale = self._xywh2cs(*bbox, 1.25)
+ abs_depth = [joint_cam[20, 2], joint_cam[41, 2]]
+ else:
+ rootnet_ann_data = rootnet_result[str(ann_id[0])]
+ bbox = np.array(rootnet_ann_data['bbox'], dtype=np.float32)
+ # the bboxes have been extended
+ center, scale = self._xywh2cs(*bbox, 1.0)
+ abs_depth = rootnet_ann_data['abs_depth']
+ # 41: 'l_wrist', left hand root
+ # 20: 'r_wrist', right hand root
+ rel_root_depth = joint_cam[41, 2] - joint_cam[20, 2]
+ # if root is not valid, root-relative 3D depth is also invalid.
+ rel_root_valid = joint_valid[20] * joint_valid[41]
+
+ # if root is not valid -> root-relative 3D pose is also not valid.
+ # Therefore, mark all joints as invalid
+ joint_valid[:20] *= joint_valid[20]
+ joint_valid[21:] *= joint_valid[41]
+
+ joints_3d = np.zeros((num_joints, 3), dtype=np.float32)
+ joints_3d_visible = np.zeros((num_joints, 3), dtype=np.float32)
+ joints_3d[:, :2] = joint_img
+ joints_3d[:21, 2] = joint_cam[:21, 2] - joint_cam[20, 2]
+ joints_3d[21:, 2] = joint_cam[21:, 2] - joint_cam[41, 2]
+ joints_3d_visible[...] = np.minimum(1, joint_valid.reshape(-1, 1))
+
+ gt_db.append({
+ 'image_file': image_file,
+ 'center': center,
+ 'scale': scale,
+ 'rotation': 0,
+ 'joints_3d': joints_3d,
+ 'joints_3d_visible': joints_3d_visible,
+ 'hand_type': hand_type,
+ 'hand_type_valid': hand_type_valid,
+ 'rel_root_depth': rel_root_depth,
+ 'rel_root_valid': rel_root_valid,
+ 'abs_depth': abs_depth,
+ 'joints_cam': joint_cam,
+ 'focal': focal,
+ 'princpt': principal_pt,
+ 'dataset': self.dataset_name,
+ 'bbox': bbox,
+ 'bbox_score': 1,
+ 'bbox_id': bbox_id
+ })
+ bbox_id = bbox_id + 1
+ gt_db = sorted(gt_db, key=lambda x: x['bbox_id'])
+
+ return gt_db
+
+ @deprecated_api_warning(name_dict=dict(outputs='results'))
+ def evaluate(self, results, res_folder=None, metric='MPJPE', **kwargs):
+ """Evaluate interhand2d keypoint results. The pose prediction results
+ will be saved in ``${res_folder}/result_keypoints.json``.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+ - heatmap height: H
+ - heatmap width: W
+
+ Args:
+ results (list[dict]): Testing results containing the following
+ items:
+
+ - preds (np.ndarray[N,K,3]): The first two dimensions are \
+ coordinates, score is the third dimension of the array.
+ - hand_type (np.ndarray[N, 4]): The first two dimensions are \
+ hand type, scores is the last two dimensions.
+ - rel_root_depth (np.ndarray[N]): The relative depth of left \
+ wrist and right wrist.
+ - boxes (np.ndarray[N,6]): [center[0], center[1], scale[0], \
+ scale[1],area, score]
+ - image_paths (list[str]): For example, ['Capture6/\
+ 0012_aokay_upright/cam410061/image4996.jpg']
+ - output_heatmap (np.ndarray[N, K, H, W]): model outputs.
+ res_folder (str, optional): The folder to save the testing
+ results. If not specified, a temp folder will be created.
+ Default: None.
+ metric (str | list[str]): Metric to be performed.
+ Options: 'MRRPE', 'MPJPE', 'Handedness_acc'.
+
+ Returns:
+ dict: Evaluation results for evaluation metric.
+ """
+ metrics = metric if isinstance(metric, list) else [metric]
+ allowed_metrics = ['MRRPE', 'MPJPE', 'Handedness_acc']
+ for metric in metrics:
+ if metric not in allowed_metrics:
+ raise KeyError(f'metric {metric} is not supported')
+
+ if res_folder is not None:
+ tmp_folder = None
+ res_file = osp.join(res_folder, 'result_keypoints.json')
+ else:
+ tmp_folder = tempfile.TemporaryDirectory()
+ res_file = osp.join(tmp_folder.name, 'result_keypoints.json')
+
+ kpts = []
+ for result in results:
+ preds = result.get('preds')
+ if preds is None and 'MPJPE' in metrics:
+ raise KeyError('metric MPJPE is not supported')
+
+ hand_type = result.get('hand_type')
+ if hand_type is None and 'Handedness_acc' in metrics:
+ raise KeyError('metric Handedness_acc is not supported')
+
+ rel_root_depth = result.get('rel_root_depth')
+ if rel_root_depth is None and 'MRRPE' in metrics:
+ raise KeyError('metric MRRPE is not supported')
+
+ boxes = result['boxes']
+ image_paths = result['image_paths']
+ bbox_ids = result['bbox_ids']
+
+ batch_size = len(image_paths)
+ for i in range(batch_size):
+ image_id = self.name2id[image_paths[i][len(self.img_prefix):]]
+
+ kpt = {
+ 'center': boxes[i][0:2].tolist(),
+ 'scale': boxes[i][2:4].tolist(),
+ 'area': float(boxes[i][4]),
+ 'score': float(boxes[i][5]),
+ 'image_id': image_id,
+ 'bbox_id': bbox_ids[i]
+ }
+
+ if preds is not None:
+ kpt['keypoints'] = preds[i, :, :3].tolist()
+ if hand_type is not None:
+ kpt['hand_type'] = hand_type[i][0:2].tolist()
+ kpt['hand_type_score'] = hand_type[i][2:4].tolist()
+ if rel_root_depth is not None:
+ kpt['rel_root_depth'] = float(rel_root_depth[i])
+
+ kpts.append(kpt)
+ kpts = self._sort_and_unique_bboxes(kpts)
+
+ self._write_keypoint_results(kpts, res_file)
+ info_str = self._report_metric(res_file, metrics)
+ name_value = OrderedDict(info_str)
+
+ if tmp_folder is not None:
+ tmp_folder.cleanup()
+
+ return name_value
+
+ @staticmethod
+ def _get_accuracy(outputs, gts, masks):
+ """Get accuracy of multi-label classification.
+
+ Note:
+ - batch_size: N
+ - label_num: C
+
+ Args:
+ outputs (np.array[N, C]): predicted multi-label.
+ gts (np.array[N, C]): Groundtruth muti-label.
+ masks (np.array[N, ]): masked outputs will be ignored for
+ accuracy calculation.
+
+ Returns:
+ float: mean accuracy
+ """
+ acc = (outputs == gts).all(axis=1)
+ return np.mean(acc[masks])
+
+ def _report_metric(self, res_file, metrics):
+ """Keypoint evaluation.
+
+ Args:
+ res_file (str): Json file stored prediction results.
+ metrics (str | list[str]): Metric to be performed.
+ Options: 'MRRPE', 'MPJPE', 'Handedness_acc'.
+
+ Returns:
+ list: Evaluation results for evaluation metric.
+ """
+ info_str = []
+
+ with open(res_file, 'r') as fin:
+ preds = json.load(fin)
+ assert len(preds) == len(self.db)
+
+ gts_rel_root = []
+ preds_rel_root = []
+ rel_root_masks = []
+ gts_joint_coord_cam = []
+ preds_joint_coord_cam = []
+ single_masks = []
+ interacting_masks = []
+ all_masks = []
+ gts_hand_type = []
+ preds_hand_type = []
+ hand_type_masks = []
+
+ for pred, item in zip(preds, self.db):
+ # mrrpe
+ if 'MRRPE' in metrics:
+ if item['hand_type'].all() and item['joints_3d_visible'][
+ 20, 0] and item['joints_3d_visible'][41, 0]:
+ rel_root_masks.append(True)
+
+ pred_left_root_img = np.array(
+ pred['keypoints'][41], dtype=np.float32)[None, :]
+ pred_left_root_img[:, 2] += item['abs_depth'][0] + pred[
+ 'rel_root_depth']
+ pred_left_root_cam = self._pixel2cam(
+ pred_left_root_img, item['focal'], item['princpt'])
+
+ pred_right_root_img = np.array(
+ pred['keypoints'][20], dtype=np.float32)[None, :]
+ pred_right_root_img[:, 2] += item['abs_depth'][0]
+ pred_right_root_cam = self._pixel2cam(
+ pred_right_root_img, item['focal'], item['princpt'])
+
+ preds_rel_root.append(pred_left_root_cam -
+ pred_right_root_cam)
+ gts_rel_root.append(
+ [item['joints_cam'][41] - item['joints_cam'][20]])
+ else:
+ rel_root_masks.append(False)
+ preds_rel_root.append([[0., 0., 0.]])
+ gts_rel_root.append([[0., 0., 0.]])
+
+ if 'MPJPE' in metrics:
+ pred_joint_coord_img = np.array(
+ pred['keypoints'], dtype=np.float32)
+ gt_joint_coord_cam = item['joints_cam'].copy()
+
+ pred_joint_coord_img[:21, 2] += item['abs_depth'][0]
+ pred_joint_coord_img[21:, 2] += item['abs_depth'][1]
+ pred_joint_coord_cam = self._pixel2cam(pred_joint_coord_img,
+ item['focal'],
+ item['princpt'])
+
+ pred_joint_coord_cam[:21] -= pred_joint_coord_cam[20]
+ pred_joint_coord_cam[21:] -= pred_joint_coord_cam[41]
+ gt_joint_coord_cam[:21] -= gt_joint_coord_cam[20]
+ gt_joint_coord_cam[21:] -= gt_joint_coord_cam[41]
+
+ preds_joint_coord_cam.append(pred_joint_coord_cam)
+ gts_joint_coord_cam.append(gt_joint_coord_cam)
+
+ mask = (np.array(item['joints_3d_visible'])[:, 0]) > 0
+
+ if item['hand_type'].all():
+ single_masks.append(
+ np.zeros(self.ann_info['num_joints'], dtype=bool))
+ interacting_masks.append(mask)
+ all_masks.append(mask)
+ else:
+ single_masks.append(mask)
+ interacting_masks.append(
+ np.zeros(self.ann_info['num_joints'], dtype=bool))
+ all_masks.append(mask)
+
+ if 'Handedness_acc' in metrics:
+ pred_hand_type = np.array(pred['hand_type'], dtype=int)
+ preds_hand_type.append(pred_hand_type)
+ gts_hand_type.append(item['hand_type'])
+ hand_type_masks.append(item['hand_type_valid'] > 0)
+
+ gts_rel_root = np.array(gts_rel_root, dtype=np.float32)
+ preds_rel_root = np.array(preds_rel_root, dtype=np.float32)
+ rel_root_masks = np.array(rel_root_masks, dtype=bool)[:, None]
+ gts_joint_coord_cam = np.array(gts_joint_coord_cam, dtype=np.float32)
+ preds_joint_coord_cam = np.array(
+ preds_joint_coord_cam, dtype=np.float32)
+ single_masks = np.array(single_masks, dtype=bool)
+ interacting_masks = np.array(interacting_masks, dtype=bool)
+ all_masks = np.array(all_masks, dtype=bool)
+ gts_hand_type = np.array(gts_hand_type, dtype=int)
+ preds_hand_type = np.array(preds_hand_type, dtype=int)
+ hand_type_masks = np.array(hand_type_masks, dtype=bool)
+
+ if 'MRRPE' in metrics:
+ info_str.append(('MRRPE',
+ keypoint_epe(preds_rel_root, gts_rel_root,
+ rel_root_masks)))
+
+ if 'MPJPE' in metrics:
+ info_str.append(('MPJPE_all',
+ keypoint_epe(preds_joint_coord_cam,
+ gts_joint_coord_cam, all_masks)))
+ info_str.append(('MPJPE_single',
+ keypoint_epe(preds_joint_coord_cam,
+ gts_joint_coord_cam, single_masks)))
+ info_str.append(
+ ('MPJPE_interacting',
+ keypoint_epe(preds_joint_coord_cam, gts_joint_coord_cam,
+ interacting_masks)))
+
+ if 'Handedness_acc' in metrics:
+ info_str.append(('Handedness_acc',
+ self._get_accuracy(preds_hand_type, gts_hand_type,
+ hand_type_masks)))
+
+ return info_str
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/hand/onehand10k_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/hand/onehand10k_dataset.py
new file mode 100644
index 0000000..9783cab
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/hand/onehand10k_dataset.py
@@ -0,0 +1,205 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os.path as osp
+import tempfile
+import warnings
+from collections import OrderedDict
+
+import numpy as np
+from mmcv import Config, deprecated_api_warning
+
+from mmpose.datasets.builder import DATASETS
+from ..base import Kpt2dSviewRgbImgTopDownDataset
+
+
+@DATASETS.register_module()
+class OneHand10KDataset(Kpt2dSviewRgbImgTopDownDataset):
+ """OneHand10K dataset for top-down hand pose estimation.
+
+ "Mask-pose Cascaded CNN for 2D Hand Pose Estimation from
+ Single Color Images", TCSVT'2019.
+ More details can be found in the `paper
+ `__ .
+
+ The dataset loads raw features and apply specified transforms
+ to return a dict containing the image tensors and other information.
+
+ OneHand10K keypoint indexes::
+
+ 0: 'wrist',
+ 1: 'thumb1',
+ 2: 'thumb2',
+ 3: 'thumb3',
+ 4: 'thumb4',
+ 5: 'forefinger1',
+ 6: 'forefinger2',
+ 7: 'forefinger3',
+ 8: 'forefinger4',
+ 9: 'middle_finger1',
+ 10: 'middle_finger2',
+ 11: 'middle_finger3',
+ 12: 'middle_finger4',
+ 13: 'ring_finger1',
+ 14: 'ring_finger2',
+ 15: 'ring_finger3',
+ 16: 'ring_finger4',
+ 17: 'pinky_finger1',
+ 18: 'pinky_finger2',
+ 19: 'pinky_finger3',
+ 20: 'pinky_finger4'
+
+ Args:
+ ann_file (str): Path to the annotation file.
+ img_prefix (str): Path to a directory where images are held.
+ Default: None.
+ data_cfg (dict): config
+ pipeline (list[dict | callable]): A sequence of data transforms.
+ dataset_info (DatasetInfo): A class containing all dataset info.
+ test_mode (bool): Store True when building test or
+ validation dataset. Default: False.
+ """
+
+ def __init__(self,
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=None,
+ test_mode=False):
+
+ if dataset_info is None:
+ warnings.warn(
+ 'dataset_info is missing. '
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 '
+ 'for details.', DeprecationWarning)
+ cfg = Config.fromfile('configs/_base_/datasets/onehand10k.py')
+ dataset_info = cfg._cfg_dict['dataset_info']
+
+ super().__init__(
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=dataset_info,
+ test_mode=test_mode)
+
+ self.ann_info['use_different_joint_weights'] = False
+ self.db = self._get_db()
+
+ print(f'=> num_images: {self.num_images}')
+ print(f'=> load {len(self.db)} samples')
+
+ def _get_db(self):
+ """Load dataset."""
+ gt_db = []
+ bbox_id = 0
+ num_joints = self.ann_info['num_joints']
+ for img_id in self.img_ids:
+
+ ann_ids = self.coco.getAnnIds(imgIds=img_id, iscrowd=False)
+ objs = self.coco.loadAnns(ann_ids)
+
+ for obj in objs:
+ if max(obj['keypoints']) == 0:
+ continue
+ joints_3d = np.zeros((num_joints, 3), dtype=np.float32)
+ joints_3d_visible = np.zeros((num_joints, 3), dtype=np.float32)
+
+ keypoints = np.array(obj['keypoints']).reshape(-1, 3)
+ joints_3d[:, :2] = keypoints[:, :2]
+ joints_3d_visible[:, :2] = np.minimum(1, keypoints[:, 2:3])
+
+ # use 1.25 padded bbox as input
+ center, scale = self._xywh2cs(*obj['bbox'][:4], 1.25)
+
+ image_file = osp.join(self.img_prefix, self.id2name[img_id])
+
+ gt_db.append({
+ 'image_file': image_file,
+ 'center': center,
+ 'scale': scale,
+ 'rotation': 0,
+ 'joints_3d': joints_3d,
+ 'joints_3d_visible': joints_3d_visible,
+ 'dataset': self.dataset_name,
+ 'bbox': obj['bbox'],
+ 'bbox_score': 1,
+ 'bbox_id': bbox_id
+ })
+ bbox_id = bbox_id + 1
+ gt_db = sorted(gt_db, key=lambda x: x['bbox_id'])
+
+ return gt_db
+
+ @deprecated_api_warning(name_dict=dict(outputs='results'))
+ def evaluate(self, results, res_folder=None, metric='PCK', **kwargs):
+ """Evaluate onehand10k keypoint results. The pose prediction results
+ will be saved in ``${res_folder}/result_keypoints.json``.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+ - heatmap height: H
+ - heatmap width: W
+
+ Args:
+ results (list[dict]): Testing results containing the following
+ items:
+
+ - preds (np.ndarray[N,K,3]): The first two dimensions are \
+ coordinates, score is the third dimension of the array.
+ - boxes (np.ndarray[N,6]): [center[0], center[1], scale[0], \
+ scale[1],area, score]
+ - image_paths (list[str]): For example, ['Test/source/0.jpg']
+ - output_heatmap (np.ndarray[N, K, H, W]): model outputs.
+ res_folder (str, optional): The folder to save the testing
+ results. If not specified, a temp folder will be created.
+ Default: None.
+ metric (str | list[str]): Metric to be performed.
+ Options: 'PCK', 'AUC', 'EPE'.
+
+ Returns:
+ dict: Evaluation results for evaluation metric.
+ """
+ metrics = metric if isinstance(metric, list) else [metric]
+ allowed_metrics = ['PCK', 'AUC', 'EPE']
+ for metric in metrics:
+ if metric not in allowed_metrics:
+ raise KeyError(f'metric {metric} is not supported')
+
+ if res_folder is not None:
+ tmp_folder = None
+ res_file = osp.join(res_folder, 'result_keypoints.json')
+ else:
+ tmp_folder = tempfile.TemporaryDirectory()
+ res_file = osp.join(tmp_folder.name, 'result_keypoints.json')
+
+ kpts = []
+ for result in results:
+ preds = result['preds']
+ boxes = result['boxes']
+ image_paths = result['image_paths']
+ bbox_ids = result['bbox_ids']
+
+ batch_size = len(image_paths)
+ for i in range(batch_size):
+ image_id = self.name2id[image_paths[i][len(self.img_prefix):]]
+
+ kpts.append({
+ 'keypoints': preds[i].tolist(),
+ 'center': boxes[i][0:2].tolist(),
+ 'scale': boxes[i][2:4].tolist(),
+ 'area': float(boxes[i][4]),
+ 'score': float(boxes[i][5]),
+ 'image_id': image_id,
+ 'bbox_id': bbox_ids[i]
+ })
+ kpts = self._sort_and_unique_bboxes(kpts)
+
+ self._write_keypoint_results(kpts, res_file)
+ info_str = self._report_metric(res_file, metrics)
+ name_value = OrderedDict(info_str)
+
+ if tmp_folder is not None:
+ tmp_folder.cleanup()
+
+ return name_value
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/hand/panoptic_hand2d_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/hand/panoptic_hand2d_dataset.py
new file mode 100644
index 0000000..c1d7fc6
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/hand/panoptic_hand2d_dataset.py
@@ -0,0 +1,208 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os.path as osp
+import tempfile
+import warnings
+from collections import OrderedDict
+
+import numpy as np
+from mmcv import Config, deprecated_api_warning
+
+from mmpose.datasets.builder import DATASETS
+from ..base import Kpt2dSviewRgbImgTopDownDataset
+
+
+@DATASETS.register_module()
+class PanopticDataset(Kpt2dSviewRgbImgTopDownDataset):
+ """Panoptic dataset for top-down hand pose estimation.
+
+ "Hand Keypoint Detection in Single Images using Multiview
+ Bootstrapping", CVPR'2017.
+ More details can be found in the `paper
+ `__ .
+
+ The dataset loads raw features and apply specified transforms
+ to return a dict containing the image tensors and other information.
+
+ Panoptic keypoint indexes::
+
+ 0: 'wrist',
+ 1: 'thumb1',
+ 2: 'thumb2',
+ 3: 'thumb3',
+ 4: 'thumb4',
+ 5: 'forefinger1',
+ 6: 'forefinger2',
+ 7: 'forefinger3',
+ 8: 'forefinger4',
+ 9: 'middle_finger1',
+ 10: 'middle_finger2',
+ 11: 'middle_finger3',
+ 12: 'middle_finger4',
+ 13: 'ring_finger1',
+ 14: 'ring_finger2',
+ 15: 'ring_finger3',
+ 16: 'ring_finger4',
+ 17: 'pinky_finger1',
+ 18: 'pinky_finger2',
+ 19: 'pinky_finger3',
+ 20: 'pinky_finger4'
+
+ Args:
+ ann_file (str): Path to the annotation file.
+ img_prefix (str): Path to a directory where images are held.
+ Default: None.
+ data_cfg (dict): config
+ pipeline (list[dict | callable]): A sequence of data transforms.
+ dataset_info (DatasetInfo): A class containing all dataset info.
+ test_mode (bool): Store True when building test or
+ validation dataset. Default: False.
+ """
+
+ def __init__(self,
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=None,
+ test_mode=False):
+
+ if dataset_info is None:
+ warnings.warn(
+ 'dataset_info is missing. '
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 '
+ 'for details.', DeprecationWarning)
+ cfg = Config.fromfile('configs/_base_/datasets/panoptic_hand2d.py')
+ dataset_info = cfg._cfg_dict['dataset_info']
+
+ super().__init__(
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=dataset_info,
+ test_mode=test_mode)
+
+ self.ann_info['use_different_joint_weights'] = False
+ self.db = self._get_db()
+
+ print(f'=> num_images: {self.num_images}')
+ print(f'=> load {len(self.db)} samples')
+
+ def _get_db(self):
+ """Load dataset."""
+ gt_db = []
+ bbox_id = 0
+ num_joints = self.ann_info['num_joints']
+ for img_id in self.img_ids:
+
+ ann_ids = self.coco.getAnnIds(imgIds=img_id, iscrowd=False)
+ objs = self.coco.loadAnns(ann_ids)
+
+ for obj in objs:
+ if max(obj['keypoints']) == 0:
+ continue
+ joints_3d = np.zeros((num_joints, 3), dtype=np.float32)
+ joints_3d_visible = np.zeros((num_joints, 3), dtype=np.float32)
+
+ keypoints = np.array(obj['keypoints']).reshape(-1, 3)
+ joints_3d[:, :2] = keypoints[:, :2]
+ joints_3d_visible[:, :2] = np.minimum(1, keypoints[:, 2:3])
+
+ # The bbox is the tightest bbox enclosing keypoints.
+ # The paper uses 2.2 bbox as the input, while
+ # we use 1.76 (2.2 * 0.8) bbox as the input.
+ center, scale = self._xywh2cs(*obj['bbox'][:4], 1.76)
+
+ image_file = osp.join(self.img_prefix, self.id2name[img_id])
+ gt_db.append({
+ 'image_file': image_file,
+ 'center': center,
+ 'scale': scale,
+ 'rotation': 0,
+ 'joints_3d': joints_3d,
+ 'joints_3d_visible': joints_3d_visible,
+ 'dataset': self.dataset_name,
+ 'bbox': obj['bbox'],
+ 'head_size': obj['head_size'],
+ 'bbox_score': 1,
+ 'bbox_id': bbox_id
+ })
+ bbox_id = bbox_id + 1
+ gt_db = sorted(gt_db, key=lambda x: x['bbox_id'])
+
+ return gt_db
+
+ @deprecated_api_warning(name_dict=dict(outputs='results'))
+ def evaluate(self, results, res_folder=None, metric='PCKh', **kwargs):
+ """Evaluate panoptic keypoint results. The pose prediction results will
+ be saved in ``${res_folder}/result_keypoints.json``.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+ - heatmap height: H
+ - heatmap width: W
+
+ Args:
+ results (list[dict]): Testing results containing the following
+ items:
+
+ - preds (np.ndarray[N,K,3]): The first two dimensions are \
+ coordinates, score is the third dimension of the array.
+ - boxes (np.ndarray[N,6]): [center[0], center[1], scale[0], \
+ scale[1],area, score]
+ - image_paths (list[str]): For example, ['hand_labels/\
+ manual_test/000648952_02_l.jpg']
+ - output_heatmap (np.ndarray[N, K, H, W]): model outputs.
+ res_folder (str, optional): The folder to save the testing
+ results. If not specified, a temp folder will be created.
+ Default: None.
+ metric (str | list[str]): Metric to be performed.
+ Options: 'PCKh', 'AUC', 'EPE'.
+
+ Returns:
+ dict: Evaluation results for evaluation metric.
+ """
+ metrics = metric if isinstance(metric, list) else [metric]
+ allowed_metrics = ['PCKh', 'AUC', 'EPE']
+ for metric in metrics:
+ if metric not in allowed_metrics:
+ raise KeyError(f'metric {metric} is not supported')
+
+ if res_folder is not None:
+ tmp_folder = None
+ res_file = osp.join(res_folder, 'result_keypoints.json')
+ else:
+ tmp_folder = tempfile.TemporaryDirectory()
+ res_file = osp.join(tmp_folder.name, 'result_keypoints.json')
+
+ kpts = []
+ for result in results:
+ preds = result['preds']
+ boxes = result['boxes']
+ image_paths = result['image_paths']
+ bbox_ids = result['bbox_ids']
+
+ batch_size = len(image_paths)
+ for i in range(batch_size):
+ image_id = self.name2id[image_paths[i][len(self.img_prefix):]]
+
+ kpts.append({
+ 'keypoints': preds[i].tolist(),
+ 'center': boxes[i][0:2].tolist(),
+ 'scale': boxes[i][2:4].tolist(),
+ 'area': float(boxes[i][4]),
+ 'score': float(boxes[i][5]),
+ 'image_id': image_id,
+ 'bbox_id': bbox_ids[i]
+ })
+ kpts = self._sort_and_unique_bboxes(kpts)
+
+ self._write_keypoint_results(kpts, res_file)
+ info_str = self._report_metric(res_file, metrics)
+ name_value = OrderedDict(info_str)
+
+ if tmp_folder is not None:
+ tmp_folder.cleanup()
+
+ return name_value
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/hand/rhd2d_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/hand/rhd2d_dataset.py
new file mode 100644
index 0000000..3667f5f
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/hand/rhd2d_dataset.py
@@ -0,0 +1,205 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os.path as osp
+import tempfile
+import warnings
+from collections import OrderedDict
+
+import numpy as np
+from mmcv import Config, deprecated_api_warning
+
+from mmpose.datasets.builder import DATASETS
+from ..base import Kpt2dSviewRgbImgTopDownDataset
+
+
+@DATASETS.register_module()
+class Rhd2DDataset(Kpt2dSviewRgbImgTopDownDataset):
+ """Rendered Handpose Dataset for top-down hand pose estimation.
+
+ "Learning to Estimate 3D Hand Pose from Single RGB Images",
+ ICCV'2017.
+ More details can be found in the `paper
+ `__ .
+
+ The dataset loads raw features and apply specified transforms
+ to return a dict containing the image tensors and other information.
+
+ Rhd keypoint indexes::
+
+ 0: 'wrist',
+ 1: 'thumb1',
+ 2: 'thumb2',
+ 3: 'thumb3',
+ 4: 'thumb4',
+ 5: 'forefinger1',
+ 6: 'forefinger2',
+ 7: 'forefinger3',
+ 8: 'forefinger4',
+ 9: 'middle_finger1',
+ 10: 'middle_finger2',
+ 11: 'middle_finger3',
+ 12: 'middle_finger4',
+ 13: 'ring_finger1',
+ 14: 'ring_finger2',
+ 15: 'ring_finger3',
+ 16: 'ring_finger4',
+ 17: 'pinky_finger1',
+ 18: 'pinky_finger2',
+ 19: 'pinky_finger3',
+ 20: 'pinky_finger4'
+
+ Args:
+ ann_file (str): Path to the annotation file.
+ img_prefix (str): Path to a directory where images are held.
+ Default: None.
+ data_cfg (dict): config
+ pipeline (list[dict | callable]): A sequence of data transforms.
+ dataset_info (DatasetInfo): A class containing all dataset info.
+ test_mode (bool): Store True when building test or
+ validation dataset. Default: False.
+ """
+
+ def __init__(self,
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=None,
+ test_mode=False):
+
+ if dataset_info is None:
+ warnings.warn(
+ 'dataset_info is missing. '
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 '
+ 'for details.', DeprecationWarning)
+ cfg = Config.fromfile('configs/_base_/datasets/rhd2d.py')
+ dataset_info = cfg._cfg_dict['dataset_info']
+
+ super().__init__(
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=dataset_info,
+ test_mode=test_mode)
+
+ self.ann_info['use_different_joint_weights'] = False
+ self.db = self._get_db()
+
+ print(f'=> num_images: {self.num_images}')
+ print(f'=> load {len(self.db)} samples')
+
+ def _get_db(self):
+ """Load dataset."""
+ gt_db = []
+ bbox_id = 0
+ num_joints = self.ann_info['num_joints']
+ for img_id in self.img_ids:
+
+ ann_ids = self.coco.getAnnIds(imgIds=img_id, iscrowd=False)
+ objs = self.coco.loadAnns(ann_ids)
+
+ for obj in objs:
+ if max(obj['keypoints']) == 0:
+ continue
+ joints_3d = np.zeros((num_joints, 3), dtype=np.float32)
+ joints_3d_visible = np.zeros((num_joints, 3), dtype=np.float32)
+
+ keypoints = np.array(obj['keypoints']).reshape(-1, 3)
+ joints_3d[:, :2] = keypoints[:, :2]
+ joints_3d_visible[:, :2] = np.minimum(1, keypoints[:, 2:3])
+
+ # the ori image is 224x224
+ center, scale = self._xywh2cs(*obj['bbox'][:4], padding=1.25)
+
+ image_file = osp.join(self.img_prefix, self.id2name[img_id])
+ gt_db.append({
+ 'image_file': image_file,
+ 'center': center,
+ 'scale': scale,
+ 'rotation': 0,
+ 'joints_3d': joints_3d,
+ 'joints_3d_visible': joints_3d_visible,
+ 'dataset': self.dataset_name,
+ 'bbox': obj['bbox'],
+ 'bbox_score': 1,
+ 'bbox_id': bbox_id
+ })
+ bbox_id = bbox_id + 1
+ gt_db = sorted(gt_db, key=lambda x: x['bbox_id'])
+
+ return gt_db
+
+ @deprecated_api_warning(name_dict=dict(outputs='results'))
+ def evaluate(self, results, res_folder=None, metric='PCK', **kwargs):
+ """Evaluate rhd keypoint results. The pose prediction results will be
+ saved in ``${res_folder}/result_keypoints.json``.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+ - heatmap height: H
+ - heatmap width: W
+
+ Args:
+ results (list[dict]): Testing results containing the following
+ items:
+
+ - preds (np.ndarray[N,K,3]): The first two dimensions are \
+ coordinates, score is the third dimension of the array.
+ - boxes (np.ndarray[N,6]): [center[0], center[1], scale[0], \
+ scale[1], area, score]
+ - image_paths (list[str]): For example,
+ ['training/rgb/00031426.jpg']
+ - output_heatmap (np.ndarray[N, K, H, W]): model outputs.
+ res_folder (str, optional): The folder to save the testing
+ results. If not specified, a temp folder will be created.
+ Default: None.
+ metric (str | list[str]): Metric to be performed.
+ Options: 'PCK', 'AUC', 'EPE'.
+
+ Returns:
+ dict: Evaluation results for evaluation metric.
+ """
+ metrics = metric if isinstance(metric, list) else [metric]
+ allowed_metrics = ['PCK', 'AUC', 'EPE']
+ for metric in metrics:
+ if metric not in allowed_metrics:
+ raise KeyError(f'metric {metric} is not supported')
+
+ if res_folder is not None:
+ tmp_folder = None
+ res_file = osp.join(res_folder, 'result_keypoints.json')
+ else:
+ tmp_folder = tempfile.TemporaryDirectory()
+ res_file = osp.join(tmp_folder.name, 'result_keypoints.json')
+
+ kpts = []
+ for result in results:
+ preds = result['preds']
+ boxes = result['boxes']
+ image_paths = result['image_paths']
+ bbox_ids = result['bbox_ids']
+
+ batch_size = len(image_paths)
+ for i in range(batch_size):
+ image_id = self.name2id[image_paths[i][len(self.img_prefix):]]
+
+ kpts.append({
+ 'keypoints': preds[i].tolist(),
+ 'center': boxes[i][0:2].tolist(),
+ 'scale': boxes[i][2:4].tolist(),
+ 'area': float(boxes[i][4]),
+ 'score': float(boxes[i][5]),
+ 'image_id': image_id,
+ 'bbox_id': bbox_ids[i]
+ })
+ kpts = self._sort_and_unique_bboxes(kpts)
+
+ self._write_keypoint_results(kpts, res_file)
+ info_str = self._report_metric(res_file, metrics)
+ name_value = OrderedDict(info_str)
+
+ if tmp_folder is not None:
+ tmp_folder.cleanup()
+
+ return name_value
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/mesh/__init__.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/mesh/__init__.py
new file mode 100644
index 0000000..14297c7
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/mesh/__init__.py
@@ -0,0 +1,10 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from .mesh_adv_dataset import MeshAdversarialDataset
+from .mesh_h36m_dataset import MeshH36MDataset
+from .mesh_mix_dataset import MeshMixDataset
+from .mosh_dataset import MoshDataset
+
+__all__ = [
+ 'MeshH36MDataset', 'MoshDataset', 'MeshMixDataset',
+ 'MeshAdversarialDataset'
+]
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/mesh/mesh_adv_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/mesh/mesh_adv_dataset.py
new file mode 100644
index 0000000..cd9ba39
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/mesh/mesh_adv_dataset.py
@@ -0,0 +1,43 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import numpy as np
+from torch.utils.data import Dataset
+
+from mmpose.datasets.builder import DATASETS, build_dataset
+
+
+@DATASETS.register_module()
+class MeshAdversarialDataset(Dataset):
+ """Mix Dataset for the adversarial training in 3D human mesh estimation
+ task.
+
+ The dataset combines data from two datasets and
+ return a dict containing data from two datasets.
+
+ Args:
+ train_dataset (Dataset): Dataset for 3D human mesh estimation.
+ adversarial_dataset (Dataset): Dataset for adversarial learning,
+ provides real SMPL parameters.
+ """
+
+ def __init__(self, train_dataset, adversarial_dataset):
+ super().__init__()
+ self.train_dataset = build_dataset(train_dataset)
+ self.adversarial_dataset = build_dataset(adversarial_dataset)
+ self.length = len(self.train_dataset)
+
+ def __len__(self):
+ """Get the size of the dataset."""
+ return self.length
+
+ def __getitem__(self, i):
+ """Given index, get the data from train dataset and randomly sample an
+ item from adversarial dataset.
+
+ Return a dict containing data from train and adversarial dataset.
+ """
+ data = self.train_dataset[i]
+ ind_adv = np.random.randint(
+ low=0, high=len(self.adversarial_dataset), dtype=int)
+ data.update(self.adversarial_dataset[ind_adv %
+ len(self.adversarial_dataset)])
+ return data
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/mesh/mesh_base_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/mesh/mesh_base_dataset.py
new file mode 100644
index 0000000..79c8a8a
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/mesh/mesh_base_dataset.py
@@ -0,0 +1,155 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import copy as cp
+import os
+from abc import ABCMeta
+
+import numpy as np
+from torch.utils.data import Dataset
+
+from mmpose.datasets.pipelines import Compose
+
+
+class MeshBaseDataset(Dataset, metaclass=ABCMeta):
+ """Base dataset for 3D human mesh estimation task. In 3D humamesh
+ estimation task, all datasets share this BaseDataset for training and have
+ their own evaluate function.
+
+ The dataset loads raw features and apply specified transforms
+ to return a dict containing the image tensors and other information.
+
+ This dataset can only be used for training.
+ For evaluation, subclass should write an extra evaluate function.
+
+ Args:
+ ann_file (str): Path to the annotation file.
+ img_prefix (str): Path to a directory where images are held.
+ Default: None.
+ data_cfg (dict): config
+ pipeline (list[dict | callable]): A sequence of data transforms.
+ """
+
+ def __init__(self,
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ test_mode=False):
+
+ self.image_info = {}
+ self.ann_info = {}
+
+ self.ann_file = ann_file
+ self.img_prefix = img_prefix
+ self.pipeline = pipeline
+ self.test_mode = test_mode
+
+ self.ann_info['image_size'] = np.array(data_cfg['image_size'])
+ self.ann_info['iuv_size'] = np.array(data_cfg['iuv_size'])
+ self.ann_info['num_joints'] = data_cfg['num_joints']
+ self.ann_info['flip_pairs'] = None
+ self.db = []
+ self.pipeline = Compose(self.pipeline)
+
+ # flip_pairs
+ # For all mesh dataset, we use 24 joints as CMR and SPIN.
+ self.ann_info['flip_pairs'] = [[0, 5], [1, 4], [2, 3], [6, 11],
+ [7, 10], [8, 9], [20, 21], [22, 23]]
+ self.ann_info['use_different_joint_weights'] = False
+ assert self.ann_info['num_joints'] == 24
+ self.ann_info['joint_weights'] = np.ones([24, 1], dtype=np.float32)
+
+ self.ann_info['uv_type'] = data_cfg['uv_type']
+ self.ann_info['use_IUV'] = data_cfg['use_IUV']
+ uv_type = self.ann_info['uv_type']
+ self.iuv_prefix = os.path.join(self.img_prefix, f'{uv_type}_IUV_gt')
+ self.db = self._get_db(ann_file)
+
+ def _get_db(self, ann_file):
+ """Load dataset."""
+ data = np.load(ann_file)
+ tmpl = dict(
+ image_file=None,
+ center=None,
+ scale=None,
+ rotation=0,
+ joints_2d=None,
+ joints_2d_visible=None,
+ joints_3d=None,
+ joints_3d_visible=None,
+ gender=None,
+ pose=None,
+ beta=None,
+ has_smpl=0,
+ iuv_file=None,
+ has_iuv=0)
+ gt_db = []
+
+ _imgnames = data['imgname']
+ _scales = data['scale'].astype(np.float32)
+ _centers = data['center'].astype(np.float32)
+ dataset_len = len(_imgnames)
+
+ # Get 2D keypoints
+ if 'part' in data.keys():
+ _keypoints = data['part'].astype(np.float32)
+ else:
+ _keypoints = np.zeros((dataset_len, 24, 3), dtype=np.float32)
+
+ # Get gt 3D joints, if available
+ if 'S' in data.keys():
+ _joints_3d = data['S'].astype(np.float32)
+ else:
+ _joints_3d = np.zeros((dataset_len, 24, 4), dtype=np.float32)
+
+ # Get gt SMPL parameters, if available
+ if 'pose' in data.keys() and 'shape' in data.keys():
+ _poses = data['pose'].astype(np.float32)
+ _betas = data['shape'].astype(np.float32)
+ has_smpl = 1
+ else:
+ _poses = np.zeros((dataset_len, 72), dtype=np.float32)
+ _betas = np.zeros((dataset_len, 10), dtype=np.float32)
+ has_smpl = 0
+
+ # Get gender data, if available
+ if 'gender' in data.keys():
+ _genders = data['gender']
+ _genders = np.array([str(g) != 'm' for g in _genders]).astype(int)
+ else:
+ _genders = -1 * np.ones(dataset_len).astype(int)
+
+ # Get IUV image, if available
+ if 'iuv_names' in data.keys():
+ _iuv_names = data['iuv_names']
+ has_iuv = has_smpl
+ else:
+ _iuv_names = [''] * dataset_len
+ has_iuv = 0
+
+ for i in range(len(_imgnames)):
+ newitem = cp.deepcopy(tmpl)
+ newitem['image_file'] = os.path.join(self.img_prefix, _imgnames[i])
+ newitem['scale'] = np.array([_scales[i], _scales[i]])
+ newitem['center'] = _centers[i]
+ newitem['joints_2d'] = _keypoints[i, :, :2]
+ newitem['joints_2d_visible'] = _keypoints[i, :, -1][:, None]
+ newitem['joints_3d'] = _joints_3d[i, :, :3]
+ newitem['joints_3d_visible'] = _joints_3d[i, :, -1][:, None]
+ newitem['pose'] = _poses[i]
+ newitem['beta'] = _betas[i]
+ newitem['has_smpl'] = has_smpl
+ newitem['gender'] = _genders[i]
+ newitem['iuv_file'] = os.path.join(self.iuv_prefix, _iuv_names[i])
+ newitem['has_iuv'] = has_iuv
+ gt_db.append(newitem)
+ return gt_db
+
+ def __len__(self, ):
+ """Get the size of the dataset."""
+ return len(self.db)
+
+ def __getitem__(self, idx):
+ """Get the sample given index."""
+ results = cp.deepcopy(self.db[idx])
+ results['ann_info'] = self.ann_info
+ return self.pipeline(results)
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/mesh/mesh_h36m_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/mesh/mesh_h36m_dataset.py
new file mode 100644
index 0000000..9ac9ead
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/mesh/mesh_h36m_dataset.py
@@ -0,0 +1,101 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os
+from collections import OrderedDict
+
+import json_tricks as json
+import numpy as np
+
+from mmpose.core.evaluation import keypoint_mpjpe
+from mmpose.datasets.builder import DATASETS
+from .mesh_base_dataset import MeshBaseDataset
+
+
+@DATASETS.register_module()
+class MeshH36MDataset(MeshBaseDataset):
+ """Human3.6M Dataset for 3D human mesh estimation. It inherits all function
+ from MeshBaseDataset and has its own evaluate function.
+
+ The dataset loads raw features and apply specified transforms
+ to return a dict containing the image tensors and other information.
+
+ Args:
+ ann_file (str): Path to the annotation file.
+ img_prefix (str): Path to a directory where images are held.
+ Default: None.
+ data_cfg (dict): config
+ pipeline (list[dict | callable]): A sequence of data transforms.
+ test_mode (bool): Store True when building test or
+ validation dataset. Default: False.
+ """
+
+ def evaluate(self, outputs, res_folder, metric='joint_error', logger=None):
+ """Evaluate 3D keypoint results."""
+ metrics = metric if isinstance(metric, list) else [metric]
+ allowed_metrics = ['joint_error']
+ for metric in metrics:
+ if metric not in allowed_metrics:
+ raise KeyError(f'metric {metric} is not supported')
+
+ res_file = os.path.join(res_folder, 'result_keypoints.json')
+ kpts = []
+ for out in outputs:
+ for (keypoints, image_path) in zip(out['keypoints_3d'],
+ out['image_path']):
+ kpts.append({
+ 'keypoints': keypoints.tolist(),
+ 'image': image_path,
+ })
+
+ self._write_keypoint_results(kpts, res_file)
+ info_str = self._report_metric(res_file)
+ name_value = OrderedDict(info_str)
+ return name_value
+
+ @staticmethod
+ def _write_keypoint_results(keypoints, res_file):
+ """Write results into a json file."""
+
+ with open(res_file, 'w') as f:
+ json.dump(keypoints, f, sort_keys=True, indent=4)
+
+ def _report_metric(self, res_file):
+ """Keypoint evaluation.
+
+ Report mean per joint position error (MPJPE) and mean per joint
+ position error after rigid alignment (MPJPE-PA)
+ """
+
+ with open(res_file, 'r') as fin:
+ preds = json.load(fin)
+ assert len(preds) == len(self.db)
+
+ pred_joints_3d = [pred['keypoints'] for pred in preds]
+ gt_joints_3d = [item['joints_3d'] for item in self.db]
+ gt_joints_visible = [item['joints_3d_visible'] for item in self.db]
+
+ pred_joints_3d = np.array(pred_joints_3d)
+ gt_joints_3d = np.array(gt_joints_3d)
+ gt_joints_visible = np.array(gt_joints_visible)
+
+ # we only evaluate on 14 lsp joints
+ joint_mapper = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 18]
+ pred_joints_3d = pred_joints_3d[:, joint_mapper, :]
+ pred_pelvis = (pred_joints_3d[:, 2] + pred_joints_3d[:, 3]) / 2
+ pred_joints_3d = pred_joints_3d - pred_pelvis[:, None, :]
+
+ gt_joints_3d = gt_joints_3d[:, joint_mapper, :]
+ gt_pelvis = (gt_joints_3d[:, 2] + gt_joints_3d[:, 3]) / 2
+ gt_joints_3d = gt_joints_3d - gt_pelvis[:, None, :]
+ gt_joints_visible = gt_joints_visible[:, joint_mapper, 0] > 0
+
+ mpjpe = keypoint_mpjpe(pred_joints_3d, gt_joints_3d, gt_joints_visible)
+ mpjpe_pa = keypoint_mpjpe(
+ pred_joints_3d,
+ gt_joints_3d,
+ gt_joints_visible,
+ alignment='procrustes')
+
+ info_str = []
+ info_str.append(('MPJPE', mpjpe * 1000))
+ info_str.append(('MPJPE-PA', mpjpe_pa * 1000))
+ return info_str
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/mesh/mesh_mix_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/mesh/mesh_mix_dataset.py
new file mode 100644
index 0000000..244a7c3
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/mesh/mesh_mix_dataset.py
@@ -0,0 +1,73 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from abc import ABCMeta
+
+import numpy as np
+from torch.utils.data import ConcatDataset, Dataset, WeightedRandomSampler
+
+from mmpose.datasets.builder import DATASETS
+from .mesh_base_dataset import MeshBaseDataset
+
+
+@DATASETS.register_module()
+class MeshMixDataset(Dataset, metaclass=ABCMeta):
+ """Mix Dataset for 3D human mesh estimation.
+
+ The dataset combines data from multiple datasets (MeshBaseDataset) and
+ sample the data from different datasets with the provided proportions.
+ The dataset loads raw features and apply specified transforms
+ to return a dict containing the image tensors and other information.
+
+ Args:
+ configs (list): List of configs for multiple datasets.
+ partition (list): Sample proportion of multiple datasets. The length
+ of partition should be same with that of configs. The elements
+ of it should be non-negative and is not necessary summing up to
+ one.
+
+ Example:
+ >>> from mmpose.datasets import MeshMixDataset
+ >>> data_cfg = dict(
+ >>> image_size=[256, 256],
+ >>> iuv_size=[64, 64],
+ >>> num_joints=24,
+ >>> use_IUV=True,
+ >>> uv_type='BF')
+ >>>
+ >>> mix_dataset = MeshMixDataset(
+ >>> configs=[
+ >>> dict(
+ >>> ann_file='tests/data/h36m/test_h36m.npz',
+ >>> img_prefix='tests/data/h36m',
+ >>> data_cfg=data_cfg,
+ >>> pipeline=[]),
+ >>> dict(
+ >>> ann_file='tests/data/h36m/test_h36m.npz',
+ >>> img_prefix='tests/data/h36m',
+ >>> data_cfg=data_cfg,
+ >>> pipeline=[]),
+ >>> ],
+ >>> partition=[0.6, 0.4])
+ """
+
+ def __init__(self, configs, partition):
+ """Load data from multiple datasets."""
+ assert min(partition) >= 0
+ datasets = [MeshBaseDataset(**cfg) for cfg in configs]
+ self.dataset = ConcatDataset(datasets)
+ self.length = max(len(ds) for ds in datasets)
+ weights = [
+ np.ones(len(ds)) * p / len(ds)
+ for (p, ds) in zip(partition, datasets)
+ ]
+ weights = np.concatenate(weights, axis=0)
+ self.sampler = WeightedRandomSampler(weights, 1)
+
+ def __len__(self):
+ """Get the size of the dataset."""
+ return self.length
+
+ def __getitem__(self, idx):
+ """Given index, sample the data from multiple datasets with the given
+ proportion."""
+ idx_new = list(self.sampler)[0]
+ return self.dataset[idx_new]
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/mesh/mosh_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/mesh/mosh_dataset.py
new file mode 100644
index 0000000..3185265
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/mesh/mosh_dataset.py
@@ -0,0 +1,68 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import copy as cp
+from abc import ABCMeta
+
+import numpy as np
+from torch.utils.data import Dataset
+
+from mmpose.datasets.builder import DATASETS
+from mmpose.datasets.pipelines import Compose
+
+
+@DATASETS.register_module()
+class MoshDataset(Dataset, metaclass=ABCMeta):
+ """Mosh Dataset for the adversarial training in 3D human mesh estimation
+ task.
+
+ The dataset return a dict containing real-world SMPL parameters.
+
+ Args:
+ ann_file (str): Path to the annotation file.
+ pipeline (list[dict | callable]): A sequence of data transforms.
+ test_mode (bool): Store True when building test or
+ validation dataset. Default: False.
+ """
+
+ def __init__(self, ann_file, pipeline, test_mode=False):
+
+ self.ann_file = ann_file
+ self.pipeline = pipeline
+ self.test_mode = test_mode
+
+ self.db = self._get_db(ann_file)
+ self.pipeline = Compose(self.pipeline)
+
+ @staticmethod
+ def _get_db(ann_file):
+ """Load dataset."""
+ data = np.load(ann_file)
+ _betas = data['shape'].astype(np.float32)
+ _poses = data['pose'].astype(np.float32)
+ tmpl = dict(
+ pose=None,
+ beta=None,
+ )
+ gt_db = []
+ dataset_len = len(_betas)
+
+ for i in range(dataset_len):
+ newitem = cp.deepcopy(tmpl)
+ newitem['pose'] = _poses[i]
+ newitem['beta'] = _betas[i]
+ gt_db.append(newitem)
+ return gt_db
+
+ def __len__(self, ):
+ """Get the size of the dataset."""
+ return len(self.db)
+
+ def __getitem__(self, idx):
+ """Get the sample given index."""
+ item = cp.deepcopy(self.db[idx])
+ trivial, pose, beta = \
+ np.zeros(3, dtype=np.float32), item['pose'], item['beta']
+ results = {
+ 'mosh_theta':
+ np.concatenate((trivial, pose, beta), axis=0).astype(np.float32)
+ }
+ return self.pipeline(results)
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/top_down/__init__.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/top_down/__init__.py
new file mode 100644
index 0000000..cc5b46a
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/top_down/__init__.py
@@ -0,0 +1,30 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from .topdown_aic_dataset import TopDownAicDataset
+from .topdown_coco_dataset import TopDownCocoDataset
+from .topdown_coco_wholebody_dataset import TopDownCocoWholeBodyDataset
+from .topdown_crowdpose_dataset import TopDownCrowdPoseDataset
+from .topdown_h36m_dataset import TopDownH36MDataset
+from .topdown_halpe_dataset import TopDownHalpeDataset
+from .topdown_jhmdb_dataset import TopDownJhmdbDataset
+from .topdown_mhp_dataset import TopDownMhpDataset
+from .topdown_mpii_dataset import TopDownMpiiDataset
+from .topdown_mpii_trb_dataset import TopDownMpiiTrbDataset
+from .topdown_ochuman_dataset import TopDownOCHumanDataset
+from .topdown_posetrack18_dataset import TopDownPoseTrack18Dataset
+from .topdown_posetrack18_video_dataset import TopDownPoseTrack18VideoDataset
+
+__all__ = [
+ 'TopDownAicDataset',
+ 'TopDownCocoDataset',
+ 'TopDownCocoWholeBodyDataset',
+ 'TopDownCrowdPoseDataset',
+ 'TopDownMpiiDataset',
+ 'TopDownMpiiTrbDataset',
+ 'TopDownOCHumanDataset',
+ 'TopDownPoseTrack18Dataset',
+ 'TopDownJhmdbDataset',
+ 'TopDownMhpDataset',
+ 'TopDownH36MDataset',
+ 'TopDownHalpeDataset',
+ 'TopDownPoseTrack18VideoDataset',
+]
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/top_down/topdown_aic_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/top_down/topdown_aic_dataset.py
new file mode 100644
index 0000000..13c41df
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/top_down/topdown_aic_dataset.py
@@ -0,0 +1,112 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import warnings
+
+from mmcv import Config
+from xtcocotools.cocoeval import COCOeval
+
+from ...builder import DATASETS
+from .topdown_coco_dataset import TopDownCocoDataset
+
+
+@DATASETS.register_module()
+class TopDownAicDataset(TopDownCocoDataset):
+ """AicDataset dataset for top-down pose estimation.
+
+ "AI Challenger : A Large-scale Dataset for Going Deeper
+ in Image Understanding", arXiv'2017.
+ More details can be found in the `paper
+ `__
+
+ The dataset loads raw features and apply specified transforms
+ to return a dict containing the image tensors and other information.
+
+ AIC keypoint indexes::
+
+ 0: "right_shoulder",
+ 1: "right_elbow",
+ 2: "right_wrist",
+ 3: "left_shoulder",
+ 4: "left_elbow",
+ 5: "left_wrist",
+ 6: "right_hip",
+ 7: "right_knee",
+ 8: "right_ankle",
+ 9: "left_hip",
+ 10: "left_knee",
+ 11: "left_ankle",
+ 12: "head_top",
+ 13: "neck"
+
+ Args:
+ ann_file (str): Path to the annotation file.
+ img_prefix (str): Path to a directory where images are held.
+ Default: None.
+ data_cfg (dict): config
+ pipeline (list[dict | callable]): A sequence of data transforms.
+ dataset_info (DatasetInfo): A class containing all dataset info.
+ test_mode (bool): Store True when building test or
+ validation dataset. Default: False.
+ """
+
+ def __init__(self,
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=None,
+ test_mode=False):
+
+ if dataset_info is None:
+ warnings.warn(
+ 'dataset_info is missing. '
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 '
+ 'for details.', DeprecationWarning)
+ cfg = Config.fromfile('configs/_base_/datasets/aic.py')
+ dataset_info = cfg._cfg_dict['dataset_info']
+
+ super(TopDownCocoDataset, self).__init__(
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=dataset_info,
+ test_mode=test_mode)
+
+ self.use_gt_bbox = data_cfg['use_gt_bbox']
+ self.bbox_file = data_cfg['bbox_file']
+ self.det_bbox_thr = data_cfg.get('det_bbox_thr', 0.0)
+ self.use_nms = data_cfg.get('use_nms', True)
+ self.soft_nms = data_cfg['soft_nms']
+ self.nms_thr = data_cfg['nms_thr']
+ self.oks_thr = data_cfg['oks_thr']
+ self.vis_thr = data_cfg['vis_thr']
+
+ self.db = self._get_db()
+
+ print(f'=> num_images: {self.num_images}')
+ print(f'=> load {len(self.db)} samples')
+
+ def _get_db(self):
+ """Load dataset."""
+ assert self.use_gt_bbox
+ gt_db = self._load_coco_keypoint_annotations()
+ return gt_db
+
+ def _do_python_keypoint_eval(self, res_file):
+ """Keypoint evaluation using COCOAPI."""
+ coco_det = self.coco.loadRes(res_file)
+ coco_eval = COCOeval(
+ self.coco, coco_det, 'keypoints', self.sigmas, use_area=False)
+ coco_eval.params.useSegm = None
+ coco_eval.evaluate()
+ coco_eval.accumulate()
+ coco_eval.summarize()
+
+ stats_names = [
+ 'AP', 'AP .5', 'AP .75', 'AP (M)', 'AP (L)', 'AR', 'AR .5',
+ 'AR .75', 'AR (M)', 'AR (L)'
+ ]
+
+ info_str = list(zip(stats_names, coco_eval.stats))
+
+ return info_str
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/top_down/topdown_base_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/top_down/topdown_base_dataset.py
new file mode 100644
index 0000000..dc99576
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/top_down/topdown_base_dataset.py
@@ -0,0 +1,16 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from abc import ABCMeta
+
+from torch.utils.data import Dataset
+
+
+class TopDownBaseDataset(Dataset, metaclass=ABCMeta):
+ """This class has been deprecated and replaced by
+ Kpt2dSviewRgbImgTopDownDataset."""
+
+ def __init__(self, *args, **kwargs):
+ raise (ImportError(
+ 'TopDownBaseDataset has been replaced by '
+ 'Kpt2dSviewRgbImgTopDownDataset,'
+ 'check https://github.com/open-mmlab/mmpose/pull/663 for details.')
+ )
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/top_down/topdown_coco_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/top_down/topdown_coco_dataset.py
new file mode 100644
index 0000000..664c881
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/top_down/topdown_coco_dataset.py
@@ -0,0 +1,405 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os.path as osp
+import tempfile
+import warnings
+from collections import OrderedDict, defaultdict
+
+import json_tricks as json
+import numpy as np
+from mmcv import Config, deprecated_api_warning
+from xtcocotools.cocoeval import COCOeval
+
+from ....core.post_processing import oks_nms, soft_oks_nms
+from ...builder import DATASETS
+from ..base import Kpt2dSviewRgbImgTopDownDataset
+
+
+@DATASETS.register_module()
+class TopDownCocoDataset(Kpt2dSviewRgbImgTopDownDataset):
+ """CocoDataset dataset for top-down pose estimation.
+
+ "Microsoft COCO: Common Objects in Context", ECCV'2014.
+ More details can be found in the `paper
+ `__ .
+
+ The dataset loads raw features and apply specified transforms
+ to return a dict containing the image tensors and other information.
+
+ COCO keypoint indexes::
+
+ 0: 'nose',
+ 1: 'left_eye',
+ 2: 'right_eye',
+ 3: 'left_ear',
+ 4: 'right_ear',
+ 5: 'left_shoulder',
+ 6: 'right_shoulder',
+ 7: 'left_elbow',
+ 8: 'right_elbow',
+ 9: 'left_wrist',
+ 10: 'right_wrist',
+ 11: 'left_hip',
+ 12: 'right_hip',
+ 13: 'left_knee',
+ 14: 'right_knee',
+ 15: 'left_ankle',
+ 16: 'right_ankle'
+
+ Args:
+ ann_file (str): Path to the annotation file.
+ img_prefix (str): Path to a directory where images are held.
+ Default: None.
+ data_cfg (dict): config
+ pipeline (list[dict | callable]): A sequence of data transforms.
+ dataset_info (DatasetInfo): A class containing all dataset info.
+ test_mode (bool): Store True when building test or
+ validation dataset. Default: False.
+ """
+
+ def __init__(self,
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=None,
+ test_mode=False):
+
+ if dataset_info is None:
+ warnings.warn(
+ 'dataset_info is missing. '
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 '
+ 'for details.', DeprecationWarning)
+ cfg = Config.fromfile('configs/_base_/datasets/coco.py')
+ dataset_info = cfg._cfg_dict['dataset_info']
+
+ super().__init__(
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=dataset_info,
+ test_mode=test_mode)
+
+ self.use_gt_bbox = data_cfg['use_gt_bbox']
+ self.bbox_file = data_cfg['bbox_file']
+ self.det_bbox_thr = data_cfg.get('det_bbox_thr', 0.0)
+ self.use_nms = data_cfg.get('use_nms', True)
+ self.soft_nms = data_cfg['soft_nms']
+ self.nms_thr = data_cfg['nms_thr']
+ self.oks_thr = data_cfg['oks_thr']
+ self.vis_thr = data_cfg['vis_thr']
+
+ self.db = self._get_db()
+
+ print(f'=> num_images: {self.num_images}')
+ print(f'=> load {len(self.db)} samples')
+
+ def _get_db(self):
+ """Load dataset."""
+ if (not self.test_mode) or self.use_gt_bbox:
+ # use ground truth bbox
+ gt_db = self._load_coco_keypoint_annotations()
+ else:
+ # use bbox from detection
+ gt_db = self._load_coco_person_detection_results()
+ return gt_db
+
+ def _load_coco_keypoint_annotations(self):
+ """Ground truth bbox and keypoints."""
+ gt_db = []
+ for img_id in self.img_ids:
+ gt_db.extend(self._load_coco_keypoint_annotation_kernel(img_id))
+ return gt_db
+
+ def _load_coco_keypoint_annotation_kernel(self, img_id):
+ """load annotation from COCOAPI.
+
+ Note:
+ bbox:[x1, y1, w, h]
+
+ Args:
+ img_id: coco image id
+
+ Returns:
+ dict: db entry
+ """
+ img_ann = self.coco.loadImgs(img_id)[0]
+ width = img_ann['width']
+ height = img_ann['height']
+ num_joints = self.ann_info['num_joints']
+
+ ann_ids = self.coco.getAnnIds(imgIds=img_id, iscrowd=False)
+ objs = self.coco.loadAnns(ann_ids)
+
+ # sanitize bboxes
+ valid_objs = []
+ for obj in objs:
+ if 'bbox' not in obj:
+ continue
+ x, y, w, h = obj['bbox']
+ x1 = max(0, x)
+ y1 = max(0, y)
+ x2 = min(width - 1, x1 + max(0, w - 1))
+ y2 = min(height - 1, y1 + max(0, h - 1))
+ if ('area' not in obj or obj['area'] > 0) and x2 > x1 and y2 > y1:
+ obj['clean_bbox'] = [x1, y1, x2 - x1, y2 - y1]
+ valid_objs.append(obj)
+ objs = valid_objs
+
+ bbox_id = 0
+ rec = []
+ for obj in objs:
+ if 'keypoints' not in obj:
+ continue
+ if max(obj['keypoints']) == 0:
+ continue
+ if 'num_keypoints' in obj and obj['num_keypoints'] == 0:
+ continue
+ joints_3d = np.zeros((num_joints, 3), dtype=np.float32)
+ joints_3d_visible = np.zeros((num_joints, 3), dtype=np.float32)
+
+ keypoints = np.array(obj['keypoints']).reshape(-1, 3)
+ joints_3d[:, :2] = keypoints[:, :2]
+ joints_3d_visible[:, :2] = np.minimum(1, keypoints[:, 2:3])
+
+ center, scale = self._xywh2cs(*obj['clean_bbox'][:4])
+
+ image_file = osp.join(self.img_prefix, self.id2name[img_id])
+ rec.append({
+ 'image_file': image_file,
+ 'center': center,
+ 'scale': scale,
+ 'bbox': obj['clean_bbox'][:4],
+ 'rotation': 0,
+ 'joints_3d': joints_3d,
+ 'joints_3d_visible': joints_3d_visible,
+ 'dataset': self.dataset_name,
+ 'bbox_score': 1,
+ 'bbox_id': bbox_id
+ })
+ bbox_id = bbox_id + 1
+
+ return rec
+
+ def _load_coco_person_detection_results(self):
+ """Load coco person detection results."""
+ num_joints = self.ann_info['num_joints']
+ all_boxes = None
+ with open(self.bbox_file, 'r') as f:
+ all_boxes = json.load(f)
+
+ if not all_boxes:
+ raise ValueError('=> Load %s fail!' % self.bbox_file)
+
+ print(f'=> Total boxes: {len(all_boxes)}')
+
+ kpt_db = []
+ bbox_id = 0
+ for det_res in all_boxes:
+ if det_res['category_id'] != 1:
+ continue
+
+ image_file = osp.join(self.img_prefix,
+ self.id2name[det_res['image_id']])
+ box = det_res['bbox']
+ score = det_res['score']
+
+ if score < self.det_bbox_thr:
+ continue
+
+ center, scale = self._xywh2cs(*box[:4])
+ joints_3d = np.zeros((num_joints, 3), dtype=np.float32)
+ joints_3d_visible = np.ones((num_joints, 3), dtype=np.float32)
+ kpt_db.append({
+ 'image_file': image_file,
+ 'center': center,
+ 'scale': scale,
+ 'rotation': 0,
+ 'bbox': box[:4],
+ 'bbox_score': score,
+ 'dataset': self.dataset_name,
+ 'joints_3d': joints_3d,
+ 'joints_3d_visible': joints_3d_visible,
+ 'bbox_id': bbox_id
+ })
+ bbox_id = bbox_id + 1
+ print(f'=> Total boxes after filter '
+ f'low score@{self.det_bbox_thr}: {bbox_id}')
+ return kpt_db
+
+ @deprecated_api_warning(name_dict=dict(outputs='results'))
+ def evaluate(self, results, res_folder=None, metric='mAP', **kwargs):
+ """Evaluate coco keypoint results. The pose prediction results will be
+ saved in ``${res_folder}/result_keypoints.json``.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+ - heatmap height: H
+ - heatmap width: W
+
+ Args:
+ results (list[dict]): Testing results containing the following
+ items:
+
+ - preds (np.ndarray[N,K,3]): The first two dimensions are \
+ coordinates, score is the third dimension of the array.
+ - boxes (np.ndarray[N,6]): [center[0], center[1], scale[0], \
+ scale[1],area, score]
+ - image_paths (list[str]): For example, ['data/coco/val2017\
+ /000000393226.jpg']
+ - heatmap (np.ndarray[N, K, H, W]): model output heatmap
+ - bbox_id (list(int)).
+ res_folder (str, optional): The folder to save the testing
+ results. If not specified, a temp folder will be created.
+ Default: None.
+ metric (str | list[str]): Metric to be performed. Defaults: 'mAP'.
+
+ Returns:
+ dict: Evaluation results for evaluation metric.
+ """
+ metrics = metric if isinstance(metric, list) else [metric]
+ allowed_metrics = ['mAP']
+ for metric in metrics:
+ if metric not in allowed_metrics:
+ raise KeyError(f'metric {metric} is not supported')
+
+ if res_folder is not None:
+ tmp_folder = None
+ res_file = osp.join(res_folder, 'result_keypoints.json')
+ else:
+ tmp_folder = tempfile.TemporaryDirectory()
+ res_file = osp.join(tmp_folder.name, 'result_keypoints.json')
+
+ kpts = defaultdict(list)
+
+ for result in results:
+ preds = result['preds']
+ boxes = result['boxes']
+ image_paths = result['image_paths']
+ bbox_ids = result['bbox_ids']
+
+ batch_size = len(image_paths)
+ for i in range(batch_size):
+ image_id = self.name2id[image_paths[i][len(self.img_prefix):]]
+ kpts[image_id].append({
+ 'keypoints': preds[i],
+ 'center': boxes[i][0:2],
+ 'scale': boxes[i][2:4],
+ 'area': boxes[i][4],
+ 'score': boxes[i][5],
+ 'image_id': image_id,
+ 'bbox_id': bbox_ids[i]
+ })
+ kpts = self._sort_and_unique_bboxes(kpts)
+
+ # rescoring and oks nms
+ num_joints = self.ann_info['num_joints']
+ vis_thr = self.vis_thr
+ oks_thr = self.oks_thr
+ valid_kpts = []
+ for image_id in kpts.keys():
+ img_kpts = kpts[image_id]
+ for n_p in img_kpts:
+ box_score = n_p['score']
+ kpt_score = 0
+ valid_num = 0
+ for n_jt in range(0, num_joints):
+ t_s = n_p['keypoints'][n_jt][2]
+ if t_s > vis_thr:
+ kpt_score = kpt_score + t_s
+ valid_num = valid_num + 1
+ if valid_num != 0:
+ kpt_score = kpt_score / valid_num
+ # rescoring
+ n_p['score'] = kpt_score * box_score
+
+ if self.use_nms:
+ nms = soft_oks_nms if self.soft_nms else oks_nms
+ keep = nms(img_kpts, oks_thr, sigmas=self.sigmas)
+ valid_kpts.append([img_kpts[_keep] for _keep in keep])
+ else:
+ valid_kpts.append(img_kpts)
+
+ self._write_coco_keypoint_results(valid_kpts, res_file)
+
+ info_str = self._do_python_keypoint_eval(res_file)
+ name_value = OrderedDict(info_str)
+
+ if tmp_folder is not None:
+ tmp_folder.cleanup()
+
+ return name_value
+
+ def _write_coco_keypoint_results(self, keypoints, res_file):
+ """Write results into a json file."""
+ data_pack = [{
+ 'cat_id': self._class_to_coco_ind[cls],
+ 'cls_ind': cls_ind,
+ 'cls': cls,
+ 'ann_type': 'keypoints',
+ 'keypoints': keypoints
+ } for cls_ind, cls in enumerate(self.classes)
+ if not cls == '__background__']
+
+ results = self._coco_keypoint_results_one_category_kernel(data_pack[0])
+
+ with open(res_file, 'w') as f:
+ json.dump(results, f, sort_keys=True, indent=4)
+
+ def _coco_keypoint_results_one_category_kernel(self, data_pack):
+ """Get coco keypoint results."""
+ cat_id = data_pack['cat_id']
+ keypoints = data_pack['keypoints']
+ cat_results = []
+
+ for img_kpts in keypoints:
+ if len(img_kpts) == 0:
+ continue
+
+ _key_points = np.array(
+ [img_kpt['keypoints'] for img_kpt in img_kpts])
+ key_points = _key_points.reshape(-1,
+ self.ann_info['num_joints'] * 3)
+
+ result = [{
+ 'image_id': img_kpt['image_id'],
+ 'category_id': cat_id,
+ 'keypoints': key_point.tolist(),
+ 'score': float(img_kpt['score']),
+ 'center': img_kpt['center'].tolist(),
+ 'scale': img_kpt['scale'].tolist()
+ } for img_kpt, key_point in zip(img_kpts, key_points)]
+
+ cat_results.extend(result)
+
+ return cat_results
+
+ def _do_python_keypoint_eval(self, res_file):
+ """Keypoint evaluation using COCOAPI."""
+ coco_det = self.coco.loadRes(res_file)
+ coco_eval = COCOeval(self.coco, coco_det, 'keypoints', self.sigmas)
+ coco_eval.params.useSegm = None
+ coco_eval.evaluate()
+ coco_eval.accumulate()
+ coco_eval.summarize()
+
+ stats_names = [
+ 'AP', 'AP .5', 'AP .75', 'AP (M)', 'AP (L)', 'AR', 'AR .5',
+ 'AR .75', 'AR (M)', 'AR (L)'
+ ]
+
+ info_str = list(zip(stats_names, coco_eval.stats))
+
+ return info_str
+
+ def _sort_and_unique_bboxes(self, kpts, key='bbox_id'):
+ """sort kpts and remove the repeated ones."""
+ for img_id, persons in kpts.items():
+ num = len(persons)
+ kpts[img_id] = sorted(kpts[img_id], key=lambda x: x[key])
+ for i in range(num - 1, 0, -1):
+ if kpts[img_id][i][key] == kpts[img_id][i - 1][key]:
+ del kpts[img_id][i]
+
+ return kpts
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/top_down/topdown_coco_wholebody_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/top_down/topdown_coco_wholebody_dataset.py
new file mode 100644
index 0000000..791a3c5
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/top_down/topdown_coco_wholebody_dataset.py
@@ -0,0 +1,274 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os
+import warnings
+
+import numpy as np
+from mmcv import Config
+from xtcocotools.cocoeval import COCOeval
+
+from ...builder import DATASETS
+from .topdown_coco_dataset import TopDownCocoDataset
+
+
+@DATASETS.register_module()
+class TopDownCocoWholeBodyDataset(TopDownCocoDataset):
+ """CocoWholeBodyDataset dataset for top-down pose estimation.
+
+ "Whole-Body Human Pose Estimation in the Wild", ECCV'2020.
+ More details can be found in the `paper
+ `__ .
+
+ The dataset loads raw features and apply specified transforms
+ to return a dict containing the image tensors and other information.
+
+ COCO-WholeBody keypoint indexes::
+
+ 0-16: 17 body keypoints,
+ 17-22: 6 foot keypoints,
+ 23-90: 68 face keypoints,
+ 91-132: 42 hand keypoints
+
+ In total, we have 133 keypoints for wholebody pose estimation.
+
+ Args:
+ ann_file (str): Path to the annotation file.
+ img_prefix (str): Path to a directory where images are held.
+ Default: None.
+ data_cfg (dict): config
+ pipeline (list[dict | callable]): A sequence of data transforms.
+ dataset_info (DatasetInfo): A class containing all dataset info.
+ test_mode (bool): Store True when building test or
+ validation dataset. Default: False.
+ """
+
+ def __init__(self,
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=None,
+ test_mode=False):
+
+ if dataset_info is None:
+ warnings.warn(
+ 'dataset_info is missing. '
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 '
+ 'for details.', DeprecationWarning)
+ cfg = Config.fromfile('configs/_base_/datasets/coco_wholebody.py')
+ dataset_info = cfg._cfg_dict['dataset_info']
+
+ super(TopDownCocoDataset, self).__init__(
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=dataset_info,
+ test_mode=test_mode)
+
+ self.use_gt_bbox = data_cfg['use_gt_bbox']
+ self.bbox_file = data_cfg['bbox_file']
+ self.det_bbox_thr = data_cfg.get('det_bbox_thr', 0.0)
+ self.use_nms = data_cfg.get('use_nms', True)
+ self.soft_nms = data_cfg['soft_nms']
+ self.nms_thr = data_cfg['nms_thr']
+ self.oks_thr = data_cfg['oks_thr']
+ self.vis_thr = data_cfg['vis_thr']
+
+ self.body_num = 17
+ self.foot_num = 6
+ self.face_num = 68
+ self.left_hand_num = 21
+ self.right_hand_num = 21
+
+ self.db = self._get_db()
+
+ print(f'=> num_images: {self.num_images}')
+ print(f'=> load {len(self.db)} samples')
+
+ def _load_coco_keypoint_annotation_kernel(self, img_id):
+ """load annotation from COCOAPI.
+
+ Note:
+ bbox:[x1, y1, w, h]
+ Args:
+ img_id: coco image id
+ Returns:
+ dict: db entry
+ """
+ img_ann = self.coco.loadImgs(img_id)[0]
+ width = img_ann['width']
+ height = img_ann['height']
+ num_joints = self.ann_info['num_joints']
+
+ ann_ids = self.coco.getAnnIds(imgIds=img_id, iscrowd=False)
+ objs = self.coco.loadAnns(ann_ids)
+
+ # sanitize bboxes
+ valid_objs = []
+ for obj in objs:
+ if 'bbox' not in obj:
+ continue
+ x, y, w, h = obj['bbox']
+ x1 = max(0, x)
+ y1 = max(0, y)
+ x2 = min(width - 1, x1 + max(0, w - 1))
+ y2 = min(height - 1, y1 + max(0, h - 1))
+ if ('area' not in obj or obj['area'] > 0) and x2 > x1 and y2 > y1:
+ obj['clean_bbox'] = [x1, y1, x2 - x1, y2 - y1]
+ valid_objs.append(obj)
+ objs = valid_objs
+
+ rec = []
+ bbox_id = 0
+ for obj in objs:
+ if 'keypoints' not in obj:
+ continue
+ if max(obj['keypoints']) == 0:
+ continue
+ joints_3d = np.zeros((num_joints, 3), dtype=np.float32)
+ joints_3d_visible = np.zeros((num_joints, 3), dtype=np.float32)
+
+ keypoints = np.array(obj['keypoints'] + obj['foot_kpts'] +
+ obj['face_kpts'] + obj['lefthand_kpts'] +
+ obj['righthand_kpts']).reshape(-1, 3)
+ joints_3d[:, :2] = keypoints[:, :2]
+ joints_3d_visible[:, :2] = np.minimum(1, keypoints[:, 2:3] > 0)
+
+ center, scale = self._xywh2cs(*obj['clean_bbox'][:4])
+
+ image_file = os.path.join(self.img_prefix, self.id2name[img_id])
+ rec.append({
+ 'image_file': image_file,
+ 'center': center,
+ 'scale': scale,
+ 'rotation': 0,
+ 'joints_3d': joints_3d,
+ 'joints_3d_visible': joints_3d_visible,
+ 'dataset': self.dataset_name,
+ 'bbox_score': 1,
+ 'bbox_id': bbox_id
+ })
+ bbox_id = bbox_id + 1
+
+ return rec
+
+ def _coco_keypoint_results_one_category_kernel(self, data_pack):
+ """Get coco keypoint results."""
+ cat_id = data_pack['cat_id']
+ keypoints = data_pack['keypoints']
+ cat_results = []
+
+ for img_kpts in keypoints:
+ if len(img_kpts) == 0:
+ continue
+
+ _key_points = np.array(
+ [img_kpt['keypoints'] for img_kpt in img_kpts])
+ key_points = _key_points.reshape(-1,
+ self.ann_info['num_joints'] * 3)
+
+ cuts = np.cumsum([
+ 0, self.body_num, self.foot_num, self.face_num,
+ self.left_hand_num, self.right_hand_num
+ ]) * 3
+
+ result = [{
+ 'image_id': img_kpt['image_id'],
+ 'category_id': cat_id,
+ 'keypoints': key_point[cuts[0]:cuts[1]].tolist(),
+ 'foot_kpts': key_point[cuts[1]:cuts[2]].tolist(),
+ 'face_kpts': key_point[cuts[2]:cuts[3]].tolist(),
+ 'lefthand_kpts': key_point[cuts[3]:cuts[4]].tolist(),
+ 'righthand_kpts': key_point[cuts[4]:cuts[5]].tolist(),
+ 'score': float(img_kpt['score']),
+ 'center': img_kpt['center'].tolist(),
+ 'scale': img_kpt['scale'].tolist()
+ } for img_kpt, key_point in zip(img_kpts, key_points)]
+
+ cat_results.extend(result)
+
+ return cat_results
+
+ def _do_python_keypoint_eval(self, res_file):
+ """Keypoint evaluation using COCOAPI."""
+ coco_det = self.coco.loadRes(res_file)
+
+ cuts = np.cumsum([
+ 0, self.body_num, self.foot_num, self.face_num, self.left_hand_num,
+ self.right_hand_num
+ ])
+
+ coco_eval = COCOeval(
+ self.coco,
+ coco_det,
+ 'keypoints_body',
+ self.sigmas[cuts[0]:cuts[1]],
+ use_area=True)
+ coco_eval.params.useSegm = None
+ coco_eval.evaluate()
+ coco_eval.accumulate()
+ coco_eval.summarize()
+
+ coco_eval = COCOeval(
+ self.coco,
+ coco_det,
+ 'keypoints_foot',
+ self.sigmas[cuts[1]:cuts[2]],
+ use_area=True)
+ coco_eval.params.useSegm = None
+ coco_eval.evaluate()
+ coco_eval.accumulate()
+ coco_eval.summarize()
+
+ coco_eval = COCOeval(
+ self.coco,
+ coco_det,
+ 'keypoints_face',
+ self.sigmas[cuts[2]:cuts[3]],
+ use_area=True)
+ coco_eval.params.useSegm = None
+ coco_eval.evaluate()
+ coco_eval.accumulate()
+ coco_eval.summarize()
+
+ coco_eval = COCOeval(
+ self.coco,
+ coco_det,
+ 'keypoints_lefthand',
+ self.sigmas[cuts[3]:cuts[4]],
+ use_area=True)
+ coco_eval.params.useSegm = None
+ coco_eval.evaluate()
+ coco_eval.accumulate()
+ coco_eval.summarize()
+
+ coco_eval = COCOeval(
+ self.coco,
+ coco_det,
+ 'keypoints_righthand',
+ self.sigmas[cuts[4]:cuts[5]],
+ use_area=True)
+ coco_eval.params.useSegm = None
+ coco_eval.evaluate()
+ coco_eval.accumulate()
+ coco_eval.summarize()
+
+ coco_eval = COCOeval(
+ self.coco,
+ coco_det,
+ 'keypoints_wholebody',
+ self.sigmas,
+ use_area=True)
+ coco_eval.params.useSegm = None
+ coco_eval.evaluate()
+ coco_eval.accumulate()
+ coco_eval.summarize()
+
+ stats_names = [
+ 'AP', 'AP .5', 'AP .75', 'AP (M)', 'AP (L)', 'AR', 'AR .5',
+ 'AR .75', 'AR (M)', 'AR (L)'
+ ]
+
+ info_str = list(zip(stats_names, coco_eval.stats))
+
+ return info_str
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/top_down/topdown_crowdpose_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/top_down/topdown_crowdpose_dataset.py
new file mode 100644
index 0000000..b9b196f
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/top_down/topdown_crowdpose_dataset.py
@@ -0,0 +1,110 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import warnings
+
+from mmcv import Config
+from xtcocotools.cocoeval import COCOeval
+
+from ...builder import DATASETS
+from .topdown_coco_dataset import TopDownCocoDataset
+
+
+@DATASETS.register_module()
+class TopDownCrowdPoseDataset(TopDownCocoDataset):
+ """CrowdPoseDataset dataset for top-down pose estimation.
+
+ "CrowdPose: Efficient Crowded Scenes Pose Estimation and
+ A New Benchmark", CVPR'2019.
+ More details can be found in the `paper
+ `__.
+
+ The dataset loads raw features and apply specified transforms
+ to return a dict containing the image tensors and other information.
+
+ CrowdPose keypoint indexes::
+
+ 0: 'left_shoulder',
+ 1: 'right_shoulder',
+ 2: 'left_elbow',
+ 3: 'right_elbow',
+ 4: 'left_wrist',
+ 5: 'right_wrist',
+ 6: 'left_hip',
+ 7: 'right_hip',
+ 8: 'left_knee',
+ 9: 'right_knee',
+ 10: 'left_ankle',
+ 11: 'right_ankle',
+ 12: 'top_head',
+ 13: 'neck'
+
+ Args:
+ ann_file (str): Path to the annotation file.
+ img_prefix (str): Path to a directory where images are held.
+ Default: None.
+ data_cfg (dict): config
+ pipeline (list[dict | callable]): A sequence of data transforms.
+ dataset_info (DatasetInfo): A class containing all dataset info.
+ test_mode (bool): Store True when building test or
+ validation dataset. Default: False.
+ """
+
+ def __init__(self,
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=None,
+ test_mode=False):
+
+ if dataset_info is None:
+ warnings.warn(
+ 'dataset_info is missing. '
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 '
+ 'for details.', DeprecationWarning)
+ cfg = Config.fromfile('configs/_base_/datasets/crowdpose.py')
+ dataset_info = cfg._cfg_dict['dataset_info']
+
+ super(TopDownCocoDataset, self).__init__(
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=dataset_info,
+ test_mode=test_mode)
+
+ self.use_gt_bbox = data_cfg['use_gt_bbox']
+ self.bbox_file = data_cfg['bbox_file']
+ self.det_bbox_thr = data_cfg.get('det_bbox_thr', 0.0)
+ self.use_nms = data_cfg.get('use_nms', True)
+ self.soft_nms = data_cfg['soft_nms']
+ self.nms_thr = data_cfg['nms_thr']
+ self.oks_thr = data_cfg['oks_thr']
+ self.vis_thr = data_cfg['vis_thr']
+
+ self.db = self._get_db()
+
+ print(f'=> num_images: {self.num_images}')
+ print(f'=> load {len(self.db)} samples')
+
+ def _do_python_keypoint_eval(self, res_file):
+ """Keypoint evaluation using COCOAPI."""
+ coco_det = self.coco.loadRes(res_file)
+ coco_eval = COCOeval(
+ self.coco,
+ coco_det,
+ 'keypoints_crowd',
+ self.sigmas,
+ use_area=False)
+ coco_eval.params.useSegm = None
+ coco_eval.evaluate()
+ coco_eval.accumulate()
+ coco_eval.summarize()
+
+ stats_names = [
+ 'AP', 'AP .5', 'AP .75', 'AR', 'AR .5', 'AR .75', 'AP(E)', 'AP(M)',
+ 'AP(H)'
+ ]
+
+ info_str = list(zip(stats_names, coco_eval.stats))
+
+ return info_str
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/top_down/topdown_h36m_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/top_down/topdown_h36m_dataset.py
new file mode 100644
index 0000000..6bc49e3
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/top_down/topdown_h36m_dataset.py
@@ -0,0 +1,206 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os.path as osp
+import tempfile
+import warnings
+from collections import OrderedDict
+
+import json_tricks as json
+import numpy as np
+from mmcv import Config, deprecated_api_warning
+
+from ...builder import DATASETS
+from ..base import Kpt2dSviewRgbImgTopDownDataset
+
+
+@DATASETS.register_module()
+class TopDownH36MDataset(Kpt2dSviewRgbImgTopDownDataset):
+ """Human3.6M dataset for top-down 2D pose estimation.
+
+ "Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human
+ Sensing in Natural Environments", TPAMI`2014.
+ More details can be found in the `paper
+ `__.
+
+ Human3.6M keypoint indexes::
+
+ 0: 'root (pelvis)',
+ 1: 'right_hip',
+ 2: 'right_knee',
+ 3: 'right_foot',
+ 4: 'left_hip',
+ 5: 'left_knee',
+ 6: 'left_foot',
+ 7: 'spine',
+ 8: 'thorax',
+ 9: 'neck_base',
+ 10: 'head',
+ 11: 'left_shoulder',
+ 12: 'left_elbow',
+ 13: 'left_wrist',
+ 14: 'right_shoulder',
+ 15: 'right_elbow',
+ 16: 'right_wrist'
+
+ Args:
+ ann_file (str): Path to the annotation file.
+ img_prefix (str): Path to a directory where images are held.
+ Default: None.
+ data_cfg (dict): config
+ pipeline (list[dict | callable]): A sequence of data transforms.
+ dataset_info (DatasetInfo): A class containing all dataset info.
+ test_mode (bool): Store True when building test or
+ validation dataset. Default: False.
+ """
+
+ def __init__(self,
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=None,
+ test_mode=False):
+
+ if dataset_info is None:
+ warnings.warn(
+ 'dataset_info is missing. '
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 '
+ 'for details.', DeprecationWarning)
+ cfg = Config.fromfile('configs/_base_/datasets/h36m.py')
+ dataset_info = cfg._cfg_dict['dataset_info']
+
+ super().__init__(
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=dataset_info,
+ test_mode=test_mode)
+
+ self.db = self._get_db()
+
+ print(f'=> num_images: {self.num_images}')
+ print(f'=> load {len(self.db)} samples')
+
+ def _get_db(self):
+ """Load dataset."""
+ gt_db = []
+ bbox_id = 0
+ num_joints = self.ann_info['num_joints']
+ for img_id in self.img_ids:
+
+ ann_ids = self.coco.getAnnIds(imgIds=img_id, iscrowd=False)
+ objs = self.coco.loadAnns(ann_ids)
+
+ for obj in objs:
+ if max(obj['keypoints']) == 0:
+ continue
+ joints_3d = np.zeros((num_joints, 3), dtype=np.float32)
+ joints_3d_visible = np.zeros((num_joints, 3), dtype=np.float32)
+
+ keypoints = np.array(obj['keypoints']).reshape(-1, 3)
+ joints_3d[:, :2] = keypoints[:, :2]
+ joints_3d_visible[:, :2] = np.minimum(1, keypoints[:, 2:3])
+
+ # use 1.25 padded bbox as input
+ center, scale = self._xywh2cs(*obj['bbox'][:4])
+
+ image_file = osp.join(self.img_prefix, self.id2name[img_id])
+
+ gt_db.append({
+ 'image_file': image_file,
+ 'center': center,
+ 'scale': scale,
+ 'rotation': 0,
+ 'joints_3d': joints_3d,
+ 'joints_3d_visible': joints_3d_visible,
+ 'dataset': self.dataset_name,
+ 'bbox': obj['bbox'],
+ 'bbox_score': 1,
+ 'bbox_id': bbox_id
+ })
+ bbox_id = bbox_id + 1
+ gt_db = sorted(gt_db, key=lambda x: x['bbox_id'])
+
+ return gt_db
+
+ @deprecated_api_warning(name_dict=dict(outputs='results'))
+ def evaluate(self, results, res_folder=None, metric='PCK', **kwargs):
+ """Evaluate human3.6m 2d keypoint results. The pose prediction results
+ will be saved in `${res_folder}/result_keypoints.json`.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+ - heatmap height: H
+ - heatmap width: W
+
+ Args:
+ results (list[dict]): Testing results containing the following
+ items:
+
+ - preds (np.ndarray[N,K,3]): The first two dimensions are
+ coordinates, score is the third dimension of the array.
+ - boxes (np.ndarray[N,6]): [center[0], center[1], scale[0],
+ scale[1],area, score]
+ - image_paths (list[str]): For example, ['data/coco/val2017
+ /000000393226.jpg']
+ - heatmap (np.ndarray[N, K, H, W]): model output heatmap
+ - bbox_id (list(int)).
+ res_folder (str, optional): The folder to save the testing
+ results. If not specified, a temp folder will be created.
+ Default: None.
+ metric (str | list[str]): Metric to be performed. Defaults: 'PCK'.
+
+ Returns:
+ dict: Evaluation results for evaluation metric.
+ """
+ metrics = metric if isinstance(metric, list) else [metric]
+ allowed_metrics = ['PCK', 'EPE']
+ for metric in metrics:
+ if metric not in allowed_metrics:
+ raise KeyError(f'metric {metric} is not supported')
+
+ if res_folder is not None:
+ tmp_folder = None
+ res_file = osp.join(res_folder, 'result_keypoints.json')
+ else:
+ tmp_folder = tempfile.TemporaryDirectory()
+ res_file = osp.join(tmp_folder.name, 'result_keypoints.json')
+
+ kpts = []
+ for result in results:
+ preds = result['preds']
+ boxes = result['boxes']
+ image_paths = result['image_paths']
+ bbox_ids = result['bbox_ids']
+
+ batch_size = len(image_paths)
+ for i in range(batch_size):
+ image_id = self.name2id[image_paths[i][len(self.img_prefix):]]
+
+ kpts.append({
+ 'keypoints': preds[i].tolist(),
+ 'center': boxes[i][0:2].tolist(),
+ 'scale': boxes[i][2:4].tolist(),
+ 'area': float(boxes[i][4]),
+ 'score': float(boxes[i][5]),
+ 'image_id': image_id,
+ 'bbox_id': bbox_ids[i]
+ })
+ kpts = self._sort_and_unique_bboxes(kpts)
+
+ self._write_keypoint_results(kpts, res_file)
+ info_str = self._report_metric(res_file, metrics)
+ name_value = OrderedDict(info_str)
+
+ if tmp_folder is not None:
+ tmp_folder.cleanup()
+
+ return name_value
+
+ @staticmethod
+ def _write_keypoint_results(keypoints, res_file):
+ """Write results into a json file."""
+
+ with open(res_file, 'w') as f:
+ json.dump(keypoints, f, sort_keys=True, indent=4)
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/top_down/topdown_halpe_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/top_down/topdown_halpe_dataset.py
new file mode 100644
index 0000000..7042daa
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/top_down/topdown_halpe_dataset.py
@@ -0,0 +1,77 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import warnings
+
+from mmcv import Config
+
+from ...builder import DATASETS
+from .topdown_coco_dataset import TopDownCocoDataset
+
+
+@DATASETS.register_module()
+class TopDownHalpeDataset(TopDownCocoDataset):
+ """HalpeDataset for top-down pose estimation.
+
+ 'https://github.com/Fang-Haoshu/Halpe-FullBody'
+
+ The dataset loads raw features and apply specified transforms
+ to return a dict containing the image tensors and other information.
+
+ Halpe keypoint indexes::
+
+ 0-19: 20 body keypoints,
+ 20-25: 6 foot keypoints,
+ 26-93: 68 face keypoints,
+ 94-135: 42 hand keypoints
+
+ In total, we have 136 keypoints for wholebody pose estimation.
+
+ Args:
+ ann_file (str): Path to the annotation file.
+ img_prefix (str): Path to a directory where images are held.
+ Default: None.
+ data_cfg (dict): config
+ pipeline (list[dict | callable]): A sequence of data transforms.
+ dataset_info (DatasetInfo): A class containing all dataset info.
+ test_mode (bool): Store True when building test or
+ validation dataset. Default: False.
+ """
+
+ def __init__(self,
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=None,
+ test_mode=False):
+
+ if dataset_info is None:
+ warnings.warn(
+ 'dataset_info is missing. '
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 '
+ 'for details.', DeprecationWarning)
+ cfg = Config.fromfile('configs/_base_/datasets/halpe.py')
+ dataset_info = cfg._cfg_dict['dataset_info']
+
+ super(TopDownCocoDataset, self).__init__(
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=dataset_info,
+ test_mode=test_mode)
+
+ self.use_gt_bbox = data_cfg['use_gt_bbox']
+ self.bbox_file = data_cfg['bbox_file']
+ self.det_bbox_thr = data_cfg.get('det_bbox_thr', 0.0)
+ self.use_nms = data_cfg.get('use_nms', True)
+ self.soft_nms = data_cfg['soft_nms']
+ self.nms_thr = data_cfg['nms_thr']
+ self.oks_thr = data_cfg['oks_thr']
+ self.vis_thr = data_cfg['vis_thr']
+
+ self.ann_info['use_different_joint_weights'] = False
+
+ self.db = self._get_db()
+
+ print(f'=> num_images: {self.num_images}')
+ print(f'=> load {len(self.db)} samples')
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/top_down/topdown_jhmdb_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/top_down/topdown_jhmdb_dataset.py
new file mode 100644
index 0000000..5204f04
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/top_down/topdown_jhmdb_dataset.py
@@ -0,0 +1,361 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os.path as osp
+import tempfile
+import warnings
+from collections import OrderedDict
+
+import json_tricks as json
+import numpy as np
+from mmcv import Config, deprecated_api_warning
+
+from mmpose.core.evaluation.top_down_eval import keypoint_pck_accuracy
+from ...builder import DATASETS
+from .topdown_coco_dataset import TopDownCocoDataset
+
+
+@DATASETS.register_module()
+class TopDownJhmdbDataset(TopDownCocoDataset):
+ """JhmdbDataset dataset for top-down pose estimation.
+
+ "Towards understanding action recognition", ICCV'2013.
+ More details can be found in the `paper
+ `__
+
+ The dataset loads raw features and apply specified transforms
+ to return a dict containing the image tensors and other information.
+
+ sub-JHMDB keypoint indexes::
+
+ 0: "neck",
+ 1: "belly",
+ 2: "head",
+ 3: "right_shoulder",
+ 4: "left_shoulder",
+ 5: "right_hip",
+ 6: "left_hip",
+ 7: "right_elbow",
+ 8: "left_elbow",
+ 9: "right_knee",
+ 10: "left_knee",
+ 11: "right_wrist",
+ 12: "left_wrist",
+ 13: "right_ankle",
+ 14: "left_ankle"
+
+ Args:
+ ann_file (str): Path to the annotation file.
+ img_prefix (str): Path to a directory where images are held.
+ Default: None.
+ data_cfg (dict): config
+ pipeline (list[dict | callable]): A sequence of data transforms.
+ dataset_info (DatasetInfo): A class containing all dataset info.
+ test_mode (bool): Store True when building test or
+ validation dataset. Default: False.
+ """
+
+ def __init__(self,
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=None,
+ test_mode=False):
+
+ if dataset_info is None:
+ warnings.warn(
+ 'dataset_info is missing. '
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 '
+ 'for details.', DeprecationWarning)
+ cfg = Config.fromfile('configs/_base_/datasets/jhmdb.py')
+ dataset_info = cfg._cfg_dict['dataset_info']
+
+ super(TopDownCocoDataset, self).__init__(
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=dataset_info,
+ test_mode=test_mode)
+
+ self.use_gt_bbox = data_cfg['use_gt_bbox']
+ self.bbox_file = data_cfg['bbox_file']
+ self.det_bbox_thr = data_cfg.get('det_bbox_thr', 0.0)
+ self.soft_nms = data_cfg['soft_nms']
+ self.nms_thr = data_cfg['nms_thr']
+ self.oks_thr = data_cfg['oks_thr']
+ self.vis_thr = data_cfg['vis_thr']
+
+ self.db = self._get_db()
+
+ print(f'=> num_images: {self.num_images}')
+ print(f'=> load {len(self.db)} samples')
+
+ def _get_db(self):
+ """Load dataset."""
+ assert self.use_gt_bbox
+ gt_db = self._load_coco_keypoint_annotations()
+ return gt_db
+
+ def _load_coco_keypoint_annotation_kernel(self, img_id):
+ """load annotation from COCOAPI.
+
+ Note:
+ bbox:[x1, y1, w, h]
+ Args:
+ img_id: coco image id
+ Returns:
+ dict: db entry
+ """
+ img_ann = self.coco.loadImgs(img_id)[0]
+ width = img_ann['width']
+ height = img_ann['height']
+ num_joints = self.ann_info['num_joints']
+
+ ann_ids = self.coco.getAnnIds(imgIds=img_id, iscrowd=False)
+ objs = self.coco.loadAnns(ann_ids)
+
+ # sanitize bboxes
+ valid_objs = []
+ for obj in objs:
+ if 'bbox' not in obj:
+ continue
+ x, y, w, h = obj['bbox']
+ # JHMDB uses matlab format, index is 1-based,
+ # we should first convert to 0-based index
+ x -= 1
+ y -= 1
+ x1 = max(0, x)
+ y1 = max(0, y)
+ x2 = min(width - 1, x1 + max(0, w - 1))
+ y2 = min(height - 1, y1 + max(0, h - 1))
+ if ('area' not in obj or obj['area'] > 0) and x2 > x1 and y2 > y1:
+ obj['clean_bbox'] = [x1, y1, x2 - x1, y2 - y1]
+ valid_objs.append(obj)
+ objs = valid_objs
+
+ rec = []
+ bbox_id = 0
+ for obj in objs:
+ if 'keypoints' not in obj:
+ continue
+ if max(obj['keypoints']) == 0:
+ continue
+ if 'num_keypoints' in obj and obj['num_keypoints'] == 0:
+ continue
+ joints_3d = np.zeros((num_joints, 3), dtype=np.float32)
+ joints_3d_visible = np.zeros((num_joints, 3), dtype=np.float32)
+
+ keypoints = np.array(obj['keypoints']).reshape(-1, 3)
+
+ # JHMDB uses matlab format, index is 1-based,
+ # we should first convert to 0-based index
+ joints_3d[:, :2] = keypoints[:, :2] - 1
+ joints_3d_visible[:, :2] = np.minimum(1, keypoints[:, 2:3])
+
+ center, scale = self._xywh2cs(*obj['clean_bbox'][:4])
+
+ image_file = osp.join(self.img_prefix, self.id2name[img_id])
+ rec.append({
+ 'image_file': image_file,
+ 'center': center,
+ 'scale': scale,
+ 'bbox': obj['clean_bbox'][:4],
+ 'rotation': 0,
+ 'joints_3d': joints_3d,
+ 'joints_3d_visible': joints_3d_visible,
+ 'dataset': self.dataset_name,
+ 'bbox_score': 1,
+ 'bbox_id': f'{img_id}_{bbox_id:03}'
+ })
+ bbox_id = bbox_id + 1
+
+ return rec
+
+ def _write_keypoint_results(self, keypoints, res_file):
+ """Write results into a json file."""
+
+ with open(res_file, 'w') as f:
+ json.dump(keypoints, f, sort_keys=True, indent=4)
+
+ def _report_metric(self, res_file, metrics, pck_thr=0.2):
+ """Keypoint evaluation.
+
+ Args:
+ res_file (str): Json file stored prediction results.
+ metrics (str | list[str]): Metric to be performed.
+ Options: 'PCK', 'PCKh', 'AUC', 'EPE'.
+ pck_thr (float): PCK threshold, default as 0.2.
+ pckh_thr (float): PCKh threshold, default as 0.7.
+ auc_nor (float): AUC normalization factor, default as 30 pixel.
+
+ Returns:
+ List: Evaluation results for evaluation metric.
+ """
+ info_str = []
+
+ with open(res_file, 'r') as fin:
+ preds = json.load(fin)
+ assert len(preds) == len(self.db)
+
+ outputs = []
+ gts = []
+ masks = []
+ threshold_bbox = []
+ threshold_torso = []
+
+ for pred, item in zip(preds, self.db):
+ outputs.append(np.array(pred['keypoints'])[:, :-1])
+ gts.append(np.array(item['joints_3d'])[:, :-1])
+ masks.append((np.array(item['joints_3d_visible'])[:, 0]) > 0)
+ if 'PCK' in metrics:
+ bbox = np.array(item['bbox'])
+ bbox_thr = np.max(bbox[2:])
+ threshold_bbox.append(np.array([bbox_thr, bbox_thr]))
+
+ if 'tPCK' in metrics:
+ torso_thr = np.linalg.norm(item['joints_3d'][4, :2] -
+ item['joints_3d'][5, :2])
+ if torso_thr < 1:
+ torso_thr = np.linalg.norm(
+ np.array(pred['keypoints'])[4, :2] -
+ np.array(pred['keypoints'])[5, :2])
+ warnings.warn('Torso Size < 1.')
+ threshold_torso.append(np.array([torso_thr, torso_thr]))
+
+ outputs = np.array(outputs)
+ gts = np.array(gts)
+ masks = np.array(masks)
+ threshold_bbox = np.array(threshold_bbox)
+ threshold_torso = np.array(threshold_torso)
+
+ if 'PCK' in metrics:
+ pck_p, pck, _ = keypoint_pck_accuracy(outputs, gts, masks, pck_thr,
+ threshold_bbox)
+
+ stats_names = [
+ 'Head PCK', 'Sho PCK', 'Elb PCK', 'Wri PCK', 'Hip PCK',
+ 'Knee PCK', 'Ank PCK', 'Mean PCK'
+ ]
+
+ stats = [
+ pck_p[2], 0.5 * pck_p[3] + 0.5 * pck_p[4],
+ 0.5 * pck_p[7] + 0.5 * pck_p[8],
+ 0.5 * pck_p[11] + 0.5 * pck_p[12],
+ 0.5 * pck_p[5] + 0.5 * pck_p[6],
+ 0.5 * pck_p[9] + 0.5 * pck_p[10],
+ 0.5 * pck_p[13] + 0.5 * pck_p[14], pck
+ ]
+
+ info_str.extend(list(zip(stats_names, stats)))
+
+ if 'tPCK' in metrics:
+ pck_p, pck, _ = keypoint_pck_accuracy(outputs, gts, masks, pck_thr,
+ threshold_torso)
+
+ stats_names = [
+ 'Head tPCK', 'Sho tPCK', 'Elb tPCK', 'Wri tPCK', 'Hip tPCK',
+ 'Knee tPCK', 'Ank tPCK', 'Mean tPCK'
+ ]
+
+ stats = [
+ pck_p[2], 0.5 * pck_p[3] + 0.5 * pck_p[4],
+ 0.5 * pck_p[7] + 0.5 * pck_p[8],
+ 0.5 * pck_p[11] + 0.5 * pck_p[12],
+ 0.5 * pck_p[5] + 0.5 * pck_p[6],
+ 0.5 * pck_p[9] + 0.5 * pck_p[10],
+ 0.5 * pck_p[13] + 0.5 * pck_p[14], pck
+ ]
+
+ info_str.extend(list(zip(stats_names, stats)))
+
+ return info_str
+
+ @deprecated_api_warning(name_dict=dict(outputs='results'))
+ def evaluate(self, results, res_folder=None, metric='PCK', **kwargs):
+ """Evaluate onehand10k keypoint results. The pose prediction results
+ will be saved in `${res_folder}/result_keypoints.json`.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+ - heatmap height: H
+ - heatmap width: W
+
+ Args:
+ results (list[dict]): Testing results containing the following
+ items:
+
+ - preds (np.ndarray[N,K,3]): The first two dimensions are \
+ coordinates, score is the third dimension of the array.
+ - boxes (np.ndarray[N,6]): [center[0], center[1], scale[0], \
+ scale[1],area, score]
+ - image_path (list[str])
+ - output_heatmap (np.ndarray[N, K, H, W]): model outputs.
+ res_folder (str, optional): The folder to save the testing
+ results. If not specified, a temp folder will be created.
+ Default: None.
+ metric (str | list[str]): Metric to be performed.
+ Options: 'PCK', 'tPCK'.
+ PCK means normalized by the bounding boxes, while tPCK
+ means normalized by the torso size.
+
+ Returns:
+ dict: Evaluation results for evaluation metric.
+ """
+ metrics = metric if isinstance(metric, list) else [metric]
+ allowed_metrics = ['PCK', 'tPCK']
+ for metric in metrics:
+ if metric not in allowed_metrics:
+ raise KeyError(f'metric {metric} is not supported')
+
+ if res_folder is not None:
+ tmp_folder = None
+ res_file = osp.join(res_folder, 'result_keypoints.json')
+ else:
+ tmp_folder = tempfile.TemporaryDirectory()
+ res_file = osp.join(tmp_folder.name, 'result_keypoints.json')
+
+ kpts = []
+
+ for result in results:
+ preds = result['preds']
+ boxes = result['boxes']
+ image_paths = result['image_paths']
+ bbox_ids = result['bbox_ids']
+
+ # convert 0-based index to 1-based index,
+ # and get the first two dimensions.
+ preds[..., :2] += 1.0
+ batch_size = len(image_paths)
+ for i in range(batch_size):
+ image_id = self.name2id[image_paths[i][len(self.img_prefix):]]
+ kpts.append({
+ 'keypoints': preds[i],
+ 'center': boxes[i][0:2],
+ 'scale': boxes[i][2:4],
+ 'area': boxes[i][4],
+ 'score': boxes[i][5],
+ 'image_id': image_id,
+ 'bbox_id': bbox_ids[i]
+ })
+ kpts = self._sort_and_unique_bboxes(kpts)
+
+ self._write_keypoint_results(kpts, res_file)
+ info_str = self._report_metric(res_file, metrics)
+ name_value = OrderedDict(info_str)
+
+ if tmp_folder is not None:
+ tmp_folder.cleanup()
+
+ return name_value
+
+ def _sort_and_unique_bboxes(self, kpts, key='bbox_id'):
+ """sort kpts and remove the repeated ones."""
+ kpts = sorted(kpts, key=lambda x: x[key])
+ num = len(kpts)
+ for i in range(num - 1, 0, -1):
+ if kpts[i][key] == kpts[i - 1][key]:
+ del kpts[i]
+
+ return kpts
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/top_down/topdown_mhp_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/top_down/topdown_mhp_dataset.py
new file mode 100644
index 0000000..050824a
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/top_down/topdown_mhp_dataset.py
@@ -0,0 +1,125 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import warnings
+
+from mmcv import Config
+from xtcocotools.cocoeval import COCOeval
+
+from ...builder import DATASETS
+from .topdown_coco_dataset import TopDownCocoDataset
+
+
+@DATASETS.register_module()
+class TopDownMhpDataset(TopDownCocoDataset):
+ """MHPv2.0 dataset for top-down pose estimation.
+
+ "Understanding Humans in Crowded Scenes: Deep Nested Adversarial
+ Learning and A New Benchmark for Multi-Human Parsing", ACM MM'2018.
+ More details can be found in the `paper
+ `__
+
+ Note that, the evaluation metric used here is mAP (adapted from COCO),
+ which may be different from the official evaluation codes.
+ 'https://github.com/ZhaoJ9014/Multi-Human-Parsing/tree/master/'
+ 'Evaluation/Multi-Human-Pose'
+ Please be cautious if you use the results in papers.
+
+ The dataset loads raw features and apply specified transforms
+ to return a dict containing the image tensors and other information.
+
+ MHP keypoint indexes::
+
+ 0: "right ankle",
+ 1: "right knee",
+ 2: "right hip",
+ 3: "left hip",
+ 4: "left knee",
+ 5: "left ankle",
+ 6: "pelvis",
+ 7: "thorax",
+ 8: "upper neck",
+ 9: "head top",
+ 10: "right wrist",
+ 11: "right elbow",
+ 12: "right shoulder",
+ 13: "left shoulder",
+ 14: "left elbow",
+ 15: "left wrist",
+
+ Args:
+ ann_file (str): Path to the annotation file.
+ img_prefix (str): Path to a directory where images are held.
+ Default: None.
+ data_cfg (dict): config
+ pipeline (list[dict | callable]): A sequence of data transforms.
+ dataset_info (DatasetInfo): A class containing all dataset info.
+ test_mode (bool): Store True when building test or
+ validation dataset. Default: False.
+ """
+
+ def __init__(self,
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=None,
+ test_mode=False):
+
+ if dataset_info is None:
+ warnings.warn(
+ 'dataset_info is missing. '
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 '
+ 'for details.', DeprecationWarning)
+ cfg = Config.fromfile('configs/_base_/datasets/mhp.py')
+ dataset_info = cfg._cfg_dict['dataset_info']
+
+ super(TopDownCocoDataset, self).__init__(
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=dataset_info,
+ test_mode=test_mode)
+
+ self.use_gt_bbox = data_cfg['use_gt_bbox']
+ self.bbox_file = data_cfg['bbox_file']
+ self.det_bbox_thr = data_cfg.get('det_bbox_thr', 0.0)
+ if 'image_thr' in data_cfg:
+ warnings.warn(
+ 'image_thr is deprecated, '
+ 'please use det_bbox_thr instead', DeprecationWarning)
+ self.det_bbox_thr = data_cfg['image_thr']
+ self.use_nms = data_cfg.get('use_nms', True)
+ self.soft_nms = data_cfg['soft_nms']
+ self.nms_thr = data_cfg['nms_thr']
+ self.oks_thr = data_cfg['oks_thr']
+ self.vis_thr = data_cfg['vis_thr']
+
+ self.db = self._get_db()
+
+ print(f'=> num_images: {self.num_images}')
+ print(f'=> load {len(self.db)} samples')
+
+ def _get_db(self):
+ """Load dataset."""
+ assert self.use_gt_bbox
+ gt_db = self._load_coco_keypoint_annotations()
+ return gt_db
+
+ def _do_python_keypoint_eval(self, res_file):
+ """Keypoint evaluation using COCOAPI."""
+ coco_det = self.coco.loadRes(res_file)
+ coco_eval = COCOeval(
+ self.coco, coco_det, 'keypoints', self.sigmas, use_area=False)
+ coco_eval.params.useSegm = None
+ coco_eval.evaluate()
+ coco_eval.accumulate()
+ coco_eval.summarize()
+
+ stats_names = [
+ 'AP', 'AP .5', 'AP .75', 'AP (M)', 'AP (L)', 'AR', 'AR .5',
+ 'AR .75', 'AR (M)', 'AR (L)'
+ ]
+
+ info_str = list(zip(stats_names, coco_eval.stats))
+
+ return info_str
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/top_down/topdown_mpii_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/top_down/topdown_mpii_dataset.py
new file mode 100644
index 0000000..751046a
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/top_down/topdown_mpii_dataset.py
@@ -0,0 +1,275 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import json
+import os.path as osp
+import warnings
+from collections import OrderedDict
+
+import numpy as np
+from mmcv import Config, deprecated_api_warning
+from scipy.io import loadmat, savemat
+
+from ...builder import DATASETS
+from ..base import Kpt2dSviewRgbImgTopDownDataset
+
+
+@DATASETS.register_module()
+class TopDownMpiiDataset(Kpt2dSviewRgbImgTopDownDataset):
+ """MPII Dataset for top-down pose estimation.
+
+ "2D Human Pose Estimation: New Benchmark and State of the Art Analysis"
+ ,CVPR'2014. More details can be found in the `paper
+ `__ .
+
+ The dataset loads raw features and apply specified transforms
+ to return a dict containing the image tensors and other information.
+
+ MPII keypoint indexes::
+
+ 0: 'right_ankle'
+ 1: 'right_knee',
+ 2: 'right_hip',
+ 3: 'left_hip',
+ 4: 'left_knee',
+ 5: 'left_ankle',
+ 6: 'pelvis',
+ 7: 'thorax',
+ 8: 'upper_neck',
+ 9: 'head_top',
+ 10: 'right_wrist',
+ 11: 'right_elbow',
+ 12: 'right_shoulder',
+ 13: 'left_shoulder',
+ 14: 'left_elbow',
+ 15: 'left_wrist'
+
+ Args:
+ ann_file (str): Path to the annotation file.
+ img_prefix (str): Path to a directory where images are held.
+ Default: None.
+ data_cfg (dict): config
+ pipeline (list[dict | callable]): A sequence of data transforms.
+ dataset_info (DatasetInfo): A class containing all dataset info.
+ test_mode (bool): Store True when building test or
+ validation dataset. Default: False.
+ """
+
+ def __init__(self,
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=None,
+ test_mode=False):
+
+ if dataset_info is None:
+ warnings.warn(
+ 'dataset_info is missing. '
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 '
+ 'for details.', DeprecationWarning)
+ cfg = Config.fromfile('configs/_base_/datasets/mpii.py')
+ dataset_info = cfg._cfg_dict['dataset_info']
+
+ super().__init__(
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=dataset_info,
+ coco_style=False,
+ test_mode=test_mode)
+
+ self.db = self._get_db()
+ self.image_set = set(x['image_file'] for x in self.db)
+ self.num_images = len(self.image_set)
+
+ print(f'=> num_images: {self.num_images}')
+ print(f'=> load {len(self.db)} samples')
+
+ def _get_db(self):
+ # create train/val split
+ with open(self.ann_file) as anno_file:
+ anno = json.load(anno_file)
+
+ gt_db = []
+ bbox_id = 0
+ for a in anno:
+ image_name = a['image']
+
+ center = np.array(a['center'], dtype=np.float32)
+ scale = np.array([a['scale'], a['scale']], dtype=np.float32)
+
+ # Adjust center/scale slightly to avoid cropping limbs
+ if center[0] != -1:
+ center[1] = center[1] + 15 * scale[1]
+ # padding to include proper amount of context
+ scale = scale * 1.25
+
+ # MPII uses matlab format, index is 1-based,
+ # we should first convert to 0-based index
+ center = center - 1
+
+ joints_3d = np.zeros((self.ann_info['num_joints'], 3),
+ dtype=np.float32)
+ joints_3d_visible = np.zeros((self.ann_info['num_joints'], 3),
+ dtype=np.float32)
+ if not self.test_mode:
+ joints = np.array(a['joints'])
+ joints_vis = np.array(a['joints_vis'])
+ assert len(joints) == self.ann_info['num_joints'], \
+ f'joint num diff: {len(joints)}' + \
+ f' vs {self.ann_info["num_joints"]}'
+
+ joints_3d[:, 0:2] = joints[:, 0:2] - 1
+ joints_3d_visible[:, :2] = joints_vis[:, None]
+ image_file = osp.join(self.img_prefix, image_name)
+ gt_db.append({
+ 'image_file': image_file,
+ 'bbox_id': bbox_id,
+ 'center': center,
+ 'scale': scale,
+ 'rotation': 0,
+ 'joints_3d': joints_3d,
+ 'joints_3d_visible': joints_3d_visible,
+ 'dataset': self.dataset_name,
+ 'bbox_score': 1
+ })
+ bbox_id = bbox_id + 1
+ gt_db = sorted(gt_db, key=lambda x: x['bbox_id'])
+
+ return gt_db
+
+ @deprecated_api_warning(name_dict=dict(outputs='results'))
+ def evaluate(self, results, res_folder=None, metric='PCKh', **kwargs):
+ """Evaluate PCKh for MPII dataset. Adapted from
+ https://github.com/leoxiaobin/deep-high-resolution-net.pytorch
+ Copyright (c) Microsoft, under the MIT License.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+ - heatmap height: H
+ - heatmap width: W
+
+ Args:
+ results (list[dict]): Testing results containing the following
+ items:
+
+ - preds (np.ndarray[N,K,3]): The first two dimensions are \
+ coordinates, score is the third dimension of the array.
+ - boxes (np.ndarray[N,6]): [center[0], center[1], scale[0], \
+ scale[1],area, score]
+ - image_paths (list[str]): For example, ['/val2017/000000\
+ 397133.jpg']
+ - heatmap (np.ndarray[N, K, H, W]): model output heatmap.
+ res_folder (str, optional): The folder to save the testing
+ results. Default: None.
+ metric (str | list[str]): Metrics to be performed.
+ Defaults: 'PCKh'.
+
+ Returns:
+ dict: PCKh for each joint
+ """
+
+ metrics = metric if isinstance(metric, list) else [metric]
+ allowed_metrics = ['PCKh']
+ for metric in metrics:
+ if metric not in allowed_metrics:
+ raise KeyError(f'metric {metric} is not supported')
+
+ kpts = []
+ for result in results:
+ preds = result['preds']
+ bbox_ids = result['bbox_ids']
+ batch_size = len(bbox_ids)
+ for i in range(batch_size):
+ kpts.append({'keypoints': preds[i], 'bbox_id': bbox_ids[i]})
+ kpts = self._sort_and_unique_bboxes(kpts)
+
+ preds = np.stack([kpt['keypoints'] for kpt in kpts])
+
+ # convert 0-based index to 1-based index,
+ # and get the first two dimensions.
+ preds = preds[..., :2] + 1.0
+
+ if res_folder:
+ pred_file = osp.join(res_folder, 'pred.mat')
+ savemat(pred_file, mdict={'preds': preds})
+
+ SC_BIAS = 0.6
+ threshold = 0.5
+
+ gt_file = osp.join(osp.dirname(self.ann_file), 'mpii_gt_val.mat')
+ gt_dict = loadmat(gt_file)
+ dataset_joints = gt_dict['dataset_joints']
+ jnt_missing = gt_dict['jnt_missing']
+ pos_gt_src = gt_dict['pos_gt_src']
+ headboxes_src = gt_dict['headboxes_src']
+
+ pos_pred_src = np.transpose(preds, [1, 2, 0])
+
+ head = np.where(dataset_joints == 'head')[1][0]
+ lsho = np.where(dataset_joints == 'lsho')[1][0]
+ lelb = np.where(dataset_joints == 'lelb')[1][0]
+ lwri = np.where(dataset_joints == 'lwri')[1][0]
+ lhip = np.where(dataset_joints == 'lhip')[1][0]
+ lkne = np.where(dataset_joints == 'lkne')[1][0]
+ lank = np.where(dataset_joints == 'lank')[1][0]
+
+ rsho = np.where(dataset_joints == 'rsho')[1][0]
+ relb = np.where(dataset_joints == 'relb')[1][0]
+ rwri = np.where(dataset_joints == 'rwri')[1][0]
+ rkne = np.where(dataset_joints == 'rkne')[1][0]
+ rank = np.where(dataset_joints == 'rank')[1][0]
+ rhip = np.where(dataset_joints == 'rhip')[1][0]
+
+ jnt_visible = 1 - jnt_missing
+ uv_error = pos_pred_src - pos_gt_src
+ uv_err = np.linalg.norm(uv_error, axis=1)
+ headsizes = headboxes_src[1, :, :] - headboxes_src[0, :, :]
+ headsizes = np.linalg.norm(headsizes, axis=0)
+ headsizes *= SC_BIAS
+ scale = headsizes * np.ones((len(uv_err), 1), dtype=np.float32)
+ scaled_uv_err = uv_err / scale
+ scaled_uv_err = scaled_uv_err * jnt_visible
+ jnt_count = np.sum(jnt_visible, axis=1)
+ less_than_threshold = (scaled_uv_err <= threshold) * jnt_visible
+ PCKh = 100. * np.sum(less_than_threshold, axis=1) / jnt_count
+
+ # save
+ rng = np.arange(0, 0.5 + 0.01, 0.01)
+ pckAll = np.zeros((len(rng), 16), dtype=np.float32)
+
+ for r, threshold in enumerate(rng):
+ less_than_threshold = (scaled_uv_err <= threshold) * jnt_visible
+ pckAll[r, :] = 100. * np.sum(
+ less_than_threshold, axis=1) / jnt_count
+
+ PCKh = np.ma.array(PCKh, mask=False)
+ PCKh.mask[6:8] = True
+
+ jnt_count = np.ma.array(jnt_count, mask=False)
+ jnt_count.mask[6:8] = True
+ jnt_ratio = jnt_count / np.sum(jnt_count).astype(np.float64)
+
+ name_value = [('Head', PCKh[head]),
+ ('Shoulder', 0.5 * (PCKh[lsho] + PCKh[rsho])),
+ ('Elbow', 0.5 * (PCKh[lelb] + PCKh[relb])),
+ ('Wrist', 0.5 * (PCKh[lwri] + PCKh[rwri])),
+ ('Hip', 0.5 * (PCKh[lhip] + PCKh[rhip])),
+ ('Knee', 0.5 * (PCKh[lkne] + PCKh[rkne])),
+ ('Ankle', 0.5 * (PCKh[lank] + PCKh[rank])),
+ ('PCKh', np.sum(PCKh * jnt_ratio)),
+ ('PCKh@0.1', np.sum(pckAll[10, :] * jnt_ratio))]
+ name_value = OrderedDict(name_value)
+
+ return name_value
+
+ def _sort_and_unique_bboxes(self, kpts, key='bbox_id'):
+ """sort kpts and remove the repeated ones."""
+ kpts = sorted(kpts, key=lambda x: x[key])
+ num = len(kpts)
+ for i in range(num - 1, 0, -1):
+ if kpts[i][key] == kpts[i - 1][key]:
+ del kpts[i]
+
+ return kpts
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/top_down/topdown_mpii_trb_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/top_down/topdown_mpii_trb_dataset.py
new file mode 100644
index 0000000..a0da65b
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/top_down/topdown_mpii_trb_dataset.py
@@ -0,0 +1,310 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import copy as cp
+import os.path as osp
+import tempfile
+import warnings
+from collections import OrderedDict
+
+import json_tricks as json
+import numpy as np
+from mmcv import Config, deprecated_api_warning
+
+from mmpose.datasets.builder import DATASETS
+from ..base import Kpt2dSviewRgbImgTopDownDataset
+
+
+@DATASETS.register_module()
+class TopDownMpiiTrbDataset(Kpt2dSviewRgbImgTopDownDataset):
+ """MPII-TRB Dataset dataset for top-down pose estimation.
+
+ "TRB: A Novel Triplet Representation for Understanding 2D Human Body",
+ ICCV'2019. More details can be found in the `paper
+ `__ .
+
+ The dataset loads raw features and apply specified transforms
+ to return a dict containing the image tensors and other information.
+
+ MPII-TRB keypoint indexes::
+
+ 0: 'left_shoulder'
+ 1: 'right_shoulder'
+ 2: 'left_elbow'
+ 3: 'right_elbow'
+ 4: 'left_wrist'
+ 5: 'right_wrist'
+ 6: 'left_hip'
+ 7: 'right_hip'
+ 8: 'left_knee'
+ 9: 'right_knee'
+ 10: 'left_ankle'
+ 11: 'right_ankle'
+ 12: 'head'
+ 13: 'neck'
+
+ 14: 'right_neck'
+ 15: 'left_neck'
+ 16: 'medial_right_shoulder'
+ 17: 'lateral_right_shoulder'
+ 18: 'medial_right_bow'
+ 19: 'lateral_right_bow'
+ 20: 'medial_right_wrist'
+ 21: 'lateral_right_wrist'
+ 22: 'medial_left_shoulder'
+ 23: 'lateral_left_shoulder'
+ 24: 'medial_left_bow'
+ 25: 'lateral_left_bow'
+ 26: 'medial_left_wrist'
+ 27: 'lateral_left_wrist'
+ 28: 'medial_right_hip'
+ 29: 'lateral_right_hip'
+ 30: 'medial_right_knee'
+ 31: 'lateral_right_knee'
+ 32: 'medial_right_ankle'
+ 33: 'lateral_right_ankle'
+ 34: 'medial_left_hip'
+ 35: 'lateral_left_hip'
+ 36: 'medial_left_knee'
+ 37: 'lateral_left_knee'
+ 38: 'medial_left_ankle'
+ 39: 'lateral_left_ankle'
+
+ Args:
+ ann_file (str): Path to the annotation file.
+ img_prefix (str): Path to a directory where images are held.
+ Default: None.
+ data_cfg (dict): config
+ pipeline (list[dict | callable]): A sequence of data transforms.
+ dataset_info (DatasetInfo): A class containing all dataset info.
+ test_mode (bool): Store True when building test or
+ validation dataset. Default: False.
+ """
+
+ def __init__(self,
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=None,
+ test_mode=False):
+
+ if dataset_info is None:
+ warnings.warn(
+ 'dataset_info is missing. '
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 '
+ 'for details.', DeprecationWarning)
+ cfg = Config.fromfile('configs/_base_/datasets/mpii_trb.py')
+ dataset_info = cfg._cfg_dict['dataset_info']
+
+ super().__init__(
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=dataset_info,
+ test_mode=test_mode)
+
+ self.db = self._get_db(ann_file)
+ self.image_set = set(x['image_file'] for x in self.db)
+ self.num_images = len(self.image_set)
+
+ print(f'=> num_images: {self.num_images}')
+ print(f'=> load {len(self.db)} samples')
+
+ def _get_db(self, ann_file):
+ """Load dataset."""
+ with open(ann_file, 'r') as f:
+ data = json.load(f)
+ tmpl = dict(
+ image_file=None,
+ bbox_id=None,
+ center=None,
+ scale=None,
+ rotation=0,
+ joints_3d=None,
+ joints_3d_visible=None,
+ dataset=self.dataset_name)
+
+ imid2info = {
+ int(osp.splitext(x['file_name'])[0]): x
+ for x in data['images']
+ }
+
+ num_joints = self.ann_info['num_joints']
+ gt_db = []
+
+ for anno in data['annotations']:
+ newitem = cp.deepcopy(tmpl)
+ image_id = anno['image_id']
+ newitem['bbox_id'] = anno['id']
+ newitem['image_file'] = osp.join(self.img_prefix,
+ imid2info[image_id]['file_name'])
+
+ if max(anno['keypoints']) == 0:
+ continue
+
+ joints_3d = np.zeros((num_joints, 3), dtype=np.float32)
+ joints_3d_visible = np.zeros((num_joints, 3), dtype=np.float32)
+
+ for ipt in range(num_joints):
+ joints_3d[ipt, 0] = anno['keypoints'][ipt * 3 + 0]
+ joints_3d[ipt, 1] = anno['keypoints'][ipt * 3 + 1]
+ joints_3d[ipt, 2] = 0
+ t_vis = min(anno['keypoints'][ipt * 3 + 2], 1)
+ joints_3d_visible[ipt, :] = (t_vis, t_vis, 0)
+
+ center = np.array(anno['center'], dtype=np.float32)
+ scale = self.ann_info['image_size'] / anno['scale'] / 200.0
+ newitem['center'] = center
+ newitem['scale'] = scale
+ newitem['joints_3d'] = joints_3d
+ newitem['joints_3d_visible'] = joints_3d_visible
+ if 'headbox' in anno:
+ newitem['headbox'] = anno['headbox']
+ gt_db.append(newitem)
+ gt_db = sorted(gt_db, key=lambda x: x['bbox_id'])
+
+ return gt_db
+
+ def _evaluate_kernel(self, pred, joints_3d, joints_3d_visible, headbox):
+ """Evaluate one example."""
+ num_joints = self.ann_info['num_joints']
+ headbox = np.array(headbox)
+ threshold = np.linalg.norm(headbox[:2] - headbox[2:]) * 0.3
+ hit = np.zeros(num_joints, dtype=np.float32)
+ exist = np.zeros(num_joints, dtype=np.float32)
+
+ for i in range(num_joints):
+ pred_pt = pred[i]
+ gt_pt = joints_3d[i]
+ vis = joints_3d_visible[i][0]
+ if vis:
+ exist[i] = 1
+ else:
+ continue
+ distance = np.linalg.norm(pred_pt[:2] - gt_pt[:2])
+ if distance < threshold:
+ hit[i] = 1
+ return hit, exist
+
+ @deprecated_api_warning(name_dict=dict(outputs='results'))
+ def evaluate(self, results, res_folder=None, metric='PCKh', **kwargs):
+ """Evaluate PCKh for MPII-TRB dataset.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+ - heatmap height: H
+ - heatmap width: W
+
+ Args:
+ results (list[dict]): Testing results containing the following
+ items:
+
+ - preds (np.ndarray[N,K,3]): The first two dimensions are \
+ coordinates, score is the third dimension of the array.
+ - boxes (np.ndarray[N,6]): [center[0], center[1], scale[0], \
+ scale[1],area, score]
+ - image_paths (list[str]): For example, ['/val2017/\
+ 000000397133.jpg']
+ - heatmap (np.ndarray[N, K, H, W]): model output heatmap.
+ - bbox_ids (list[str]): For example, ['27407'].
+ res_folder (str, optional): The folder to save the testing
+ results. If not specified, a temp folder will be created.
+ Default: None.
+ metric (str | list[str]): Metrics to be performed.
+ Defaults: 'PCKh'.
+
+ Returns:
+ dict: PCKh for each joint
+ """
+ metrics = metric if isinstance(metric, list) else [metric]
+ allowed_metrics = ['PCKh']
+ for metric in metrics:
+ if metric not in allowed_metrics:
+ raise KeyError(f'metric {metric} is not supported')
+
+ if res_folder is not None:
+ tmp_folder = None
+ res_file = osp.join(res_folder, 'result_keypoints.json')
+ else:
+ tmp_folder = tempfile.TemporaryDirectory()
+ res_file = osp.join(tmp_folder.name, 'result_keypoints.json')
+
+ kpts = []
+ for result in results:
+ preds = result['preds']
+ boxes = result['boxes']
+ image_paths = result['image_paths']
+ bbox_ids = result['bbox_ids']
+
+ batch_size = len(image_paths)
+ for i in range(batch_size):
+ str_image_path = image_paths[i]
+ image_id = int(osp.basename(osp.splitext(str_image_path)[0]))
+
+ kpts.append({
+ 'keypoints': preds[i].tolist(),
+ 'center': boxes[i][0:2].tolist(),
+ 'scale': boxes[i][2:4].tolist(),
+ 'area': float(boxes[i][4]),
+ 'score': float(boxes[i][5]),
+ 'image_id': image_id,
+ 'bbox_id': bbox_ids[i]
+ })
+ kpts = self._sort_and_unique_bboxes(kpts)
+
+ self._write_keypoint_results(kpts, res_file)
+ info_str = self._report_metric(res_file)
+ name_value = OrderedDict(info_str)
+
+ if tmp_folder is not None:
+ tmp_folder.cleanup()
+
+ return name_value
+
+ @staticmethod
+ def _write_keypoint_results(keypoints, res_file):
+ """Write results into a json file."""
+
+ with open(res_file, 'w') as f:
+ json.dump(keypoints, f, sort_keys=True, indent=4)
+
+ def _report_metric(self, res_file):
+ """Keypoint evaluation.
+
+ Report Mean Acc of skeleton, contour and all joints.
+ """
+ num_joints = self.ann_info['num_joints']
+ hit = np.zeros(num_joints, dtype=np.float32)
+ exist = np.zeros(num_joints, dtype=np.float32)
+
+ with open(res_file, 'r') as fin:
+ preds = json.load(fin)
+
+ assert len(preds) == len(
+ self.db), f'len(preds)={len(preds)}, len(self.db)={len(self.db)}'
+ for pred, item in zip(preds, self.db):
+ h, e = self._evaluate_kernel(pred['keypoints'], item['joints_3d'],
+ item['joints_3d_visible'],
+ item['headbox'])
+ hit += h
+ exist += e
+ skeleton = np.sum(hit[:14]) / np.sum(exist[:14])
+ contour = np.sum(hit[14:]) / np.sum(exist[14:])
+ mean = np.sum(hit) / np.sum(exist)
+
+ info_str = []
+ info_str.append(('Skeleton_acc', skeleton.item()))
+ info_str.append(('Contour_acc', contour.item()))
+ info_str.append(('PCKh', mean.item()))
+ return info_str
+
+ def _sort_and_unique_bboxes(self, kpts, key='bbox_id'):
+ """sort kpts and remove the repeated ones."""
+ kpts = sorted(kpts, key=lambda x: x[key])
+ num = len(kpts)
+ for i in range(num - 1, 0, -1):
+ if kpts[i][key] == kpts[i - 1][key]:
+ del kpts[i]
+
+ return kpts
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/top_down/topdown_ochuman_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/top_down/topdown_ochuman_dataset.py
new file mode 100644
index 0000000..0ad6b81
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/top_down/topdown_ochuman_dataset.py
@@ -0,0 +1,97 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import warnings
+
+from mmcv import Config
+
+from ...builder import DATASETS
+from .topdown_coco_dataset import TopDownCocoDataset
+
+
+@DATASETS.register_module()
+class TopDownOCHumanDataset(TopDownCocoDataset):
+ """OChuman dataset for top-down pose estimation.
+
+ "Pose2Seg: Detection Free Human Instance Segmentation", CVPR'2019.
+ More details can be found in the `paper
+ `__ .
+
+ "Occluded Human (OCHuman)" dataset contains 8110 heavily occluded
+ human instances within 4731 images. OCHuman dataset is designed for
+ validation and testing. To evaluate on OCHuman, the model should be
+ trained on COCO training set, and then test the robustness of the
+ model to occlusion using OCHuman.
+
+ OCHuman keypoint indexes (same as COCO)::
+
+ 0: 'nose',
+ 1: 'left_eye',
+ 2: 'right_eye',
+ 3: 'left_ear',
+ 4: 'right_ear',
+ 5: 'left_shoulder',
+ 6: 'right_shoulder',
+ 7: 'left_elbow',
+ 8: 'right_elbow',
+ 9: 'left_wrist',
+ 10: 'right_wrist',
+ 11: 'left_hip',
+ 12: 'right_hip',
+ 13: 'left_knee',
+ 14: 'right_knee',
+ 15: 'left_ankle',
+ 16: 'right_ankle'
+
+ Args:
+ ann_file (str): Path to the annotation file.
+ img_prefix (str): Path to a directory where images are held.
+ Default: None.
+ data_cfg (dict): config
+ pipeline (list[dict | callable]): A sequence of data transforms.
+ dataset_info (DatasetInfo): A class containing all dataset info.
+ test_mode (bool): Store True when building test or
+ validation dataset. Default: False.
+ """
+
+ def __init__(self,
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=None,
+ test_mode=False):
+
+ if dataset_info is None:
+ warnings.warn(
+ 'dataset_info is missing. '
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 '
+ 'for details.', DeprecationWarning)
+ cfg = Config.fromfile('configs/_base_/datasets/ochuman.py')
+ dataset_info = cfg._cfg_dict['dataset_info']
+
+ super(TopDownCocoDataset, self).__init__(
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=dataset_info,
+ test_mode=test_mode)
+
+ self.use_gt_bbox = data_cfg['use_gt_bbox']
+ self.bbox_file = data_cfg['bbox_file']
+ self.det_bbox_thr = data_cfg.get('det_bbox_thr', 0.0)
+ self.use_nms = data_cfg.get('use_nms', True)
+ self.soft_nms = data_cfg['soft_nms']
+ self.nms_thr = data_cfg['nms_thr']
+ self.oks_thr = data_cfg['oks_thr']
+ self.vis_thr = data_cfg['vis_thr']
+
+ self.db = self._get_db()
+
+ print(f'=> num_images: {self.num_images}')
+ print(f'=> load {len(self.db)} samples')
+
+ def _get_db(self):
+ """Load dataset."""
+ assert self.use_gt_bbox
+ gt_db = self._load_coco_keypoint_annotations()
+ return gt_db
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/top_down/topdown_posetrack18_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/top_down/topdown_posetrack18_dataset.py
new file mode 100644
index 0000000..c690860
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/top_down/topdown_posetrack18_dataset.py
@@ -0,0 +1,312 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os
+import os.path as osp
+import tempfile
+import warnings
+from collections import OrderedDict, defaultdict
+
+import json_tricks as json
+import numpy as np
+from mmcv import Config, deprecated_api_warning
+
+from ....core.post_processing import oks_nms, soft_oks_nms
+from ...builder import DATASETS
+from .topdown_coco_dataset import TopDownCocoDataset
+
+try:
+ from poseval import eval_helpers
+ from poseval.evaluateAP import evaluateAP
+ has_poseval = True
+except (ImportError, ModuleNotFoundError):
+ has_poseval = False
+
+
+@DATASETS.register_module()
+class TopDownPoseTrack18Dataset(TopDownCocoDataset):
+ """PoseTrack18 dataset for top-down pose estimation.
+
+ "Posetrack: A benchmark for human pose estimation and tracking", CVPR'2018.
+ More details can be found in the `paper
+ `__ .
+
+ The dataset loads raw features and apply specified transforms
+ to return a dict containing the image tensors and other information.
+
+ PoseTrack2018 keypoint indexes::
+
+ 0: 'nose',
+ 1: 'head_bottom',
+ 2: 'head_top',
+ 3: 'left_ear',
+ 4: 'right_ear',
+ 5: 'left_shoulder',
+ 6: 'right_shoulder',
+ 7: 'left_elbow',
+ 8: 'right_elbow',
+ 9: 'left_wrist',
+ 10: 'right_wrist',
+ 11: 'left_hip',
+ 12: 'right_hip',
+ 13: 'left_knee',
+ 14: 'right_knee',
+ 15: 'left_ankle',
+ 16: 'right_ankle'
+
+ Args:
+ ann_file (str): Path to the annotation file.
+ img_prefix (str): Path to a directory where images are held.
+ Default: None.
+ data_cfg (dict): config
+ pipeline (list[dict | callable]): A sequence of data transforms.
+ dataset_info (DatasetInfo): A class containing all dataset info.
+ test_mode (bool): Store True when building test or
+ validation dataset. Default: False.
+ """
+
+ def __init__(self,
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=None,
+ test_mode=False):
+
+ if dataset_info is None:
+ warnings.warn(
+ 'dataset_info is missing. '
+ 'Check https://github.com/open-mmlab/mmpose/pull/663 '
+ 'for details.', DeprecationWarning)
+ cfg = Config.fromfile('configs/_base_/datasets/posetrack18.py')
+ dataset_info = cfg._cfg_dict['dataset_info']
+
+ super(TopDownCocoDataset, self).__init__(
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=dataset_info,
+ test_mode=test_mode)
+
+ self.use_gt_bbox = data_cfg['use_gt_bbox']
+ self.bbox_file = data_cfg['bbox_file']
+ self.det_bbox_thr = data_cfg.get('det_bbox_thr', 0.0)
+ self.use_nms = data_cfg.get('use_nms', True)
+ self.soft_nms = data_cfg['soft_nms']
+ self.nms_thr = data_cfg['nms_thr']
+ self.oks_thr = data_cfg['oks_thr']
+ self.vis_thr = data_cfg['vis_thr']
+
+ self.db = self._get_db()
+
+ print(f'=> num_images: {self.num_images}')
+ print(f'=> load {len(self.db)} samples')
+
+ @deprecated_api_warning(name_dict=dict(outputs='results'))
+ def evaluate(self, results, res_folder=None, metric='mAP', **kwargs):
+ """Evaluate posetrack keypoint results. The pose prediction results
+ will be saved in ``${res_folder}/result_keypoints.json``.
+
+ Note:
+ - num_keypoints: K
+
+ Args:
+ results (list[dict]): Testing results containing the following
+ items:
+
+ - preds (np.ndarray[N,K,3]): The first two dimensions are \
+ coordinates, score is the third dimension of the array.
+ - boxes (np.ndarray[N,6]): [center[0], center[1], scale[0], \
+ scale[1],area, score]
+ - image_paths (list[str]): For example, ['val/010016_mpii_test\
+ /000024.jpg']
+ - heatmap (np.ndarray[N, K, H, W]): model output heatmap.
+ - bbox_id (list(int))
+ res_folder (str, optional): The folder to save the testing
+ results. If not specified, a temp folder will be created.
+ Default: None.
+ metric (str | list[str]): Metric to be performed. Defaults: 'mAP'.
+
+ Returns:
+ dict: Evaluation results for evaluation metric.
+ """
+ metrics = metric if isinstance(metric, list) else [metric]
+ allowed_metrics = ['mAP']
+ for metric in metrics:
+ if metric not in allowed_metrics:
+ raise KeyError(f'metric {metric} is not supported')
+
+ if res_folder is not None:
+ tmp_folder = None
+ else:
+ tmp_folder = tempfile.TemporaryDirectory()
+ res_folder = tmp_folder.name
+
+ gt_folder = osp.join(
+ osp.dirname(self.ann_file),
+ osp.splitext(self.ann_file.split('_')[-1])[0])
+
+ kpts = defaultdict(list)
+
+ for result in results:
+ preds = result['preds']
+ boxes = result['boxes']
+ image_paths = result['image_paths']
+ bbox_ids = result['bbox_ids']
+
+ batch_size = len(image_paths)
+ for i in range(batch_size):
+ image_id = self.name2id[image_paths[i][len(self.img_prefix):]]
+ kpts[image_id].append({
+ 'keypoints': preds[i],
+ 'center': boxes[i][0:2],
+ 'scale': boxes[i][2:4],
+ 'area': boxes[i][4],
+ 'score': boxes[i][5],
+ 'image_id': image_id,
+ 'bbox_id': bbox_ids[i]
+ })
+ kpts = self._sort_and_unique_bboxes(kpts)
+
+ # rescoring and oks nms
+ num_joints = self.ann_info['num_joints']
+ vis_thr = self.vis_thr
+ oks_thr = self.oks_thr
+ valid_kpts = defaultdict(list)
+ for image_id in kpts.keys():
+ img_kpts = kpts[image_id]
+ for n_p in img_kpts:
+ box_score = n_p['score']
+ kpt_score = 0
+ valid_num = 0
+ for n_jt in range(0, num_joints):
+ t_s = n_p['keypoints'][n_jt][2]
+ if t_s > vis_thr:
+ kpt_score = kpt_score + t_s
+ valid_num = valid_num + 1
+ if valid_num != 0:
+ kpt_score = kpt_score / valid_num
+ # rescoring
+ n_p['score'] = kpt_score * box_score
+
+ if self.use_nms:
+ nms = soft_oks_nms if self.soft_nms else oks_nms
+ keep = nms(img_kpts, oks_thr, sigmas=self.sigmas)
+ valid_kpts[image_id].append(
+ [img_kpts[_keep] for _keep in keep])
+ else:
+ valid_kpts[image_id].append(img_kpts)
+
+ self._write_posetrack18_keypoint_results(valid_kpts, gt_folder,
+ res_folder)
+
+ info_str = self._do_python_keypoint_eval(gt_folder, res_folder)
+ name_value = OrderedDict(info_str)
+
+ if tmp_folder is not None:
+ tmp_folder.cleanup()
+
+ return name_value
+
+ @staticmethod
+ def _write_posetrack18_keypoint_results(keypoint_results, gt_folder,
+ pred_folder):
+ """Write results into a json file.
+
+ Args:
+ keypoint_results (dict): keypoint results organized by image_id.
+ gt_folder (str): Path of directory for official gt files.
+ pred_folder (str): Path of directory to save the results.
+ """
+ categories = []
+
+ cat = {}
+ cat['supercategory'] = 'person'
+ cat['id'] = 1
+ cat['name'] = 'person'
+ cat['keypoints'] = [
+ 'nose', 'head_bottom', 'head_top', 'left_ear', 'right_ear',
+ 'left_shoulder', 'right_shoulder', 'left_elbow', 'right_elbow',
+ 'left_wrist', 'right_wrist', 'left_hip', 'right_hip', 'left_knee',
+ 'right_knee', 'left_ankle', 'right_ankle'
+ ]
+ cat['skeleton'] = [[16, 14], [14, 12], [17, 15], [15, 13], [12, 13],
+ [6, 12], [7, 13], [6, 7], [6, 8], [7, 9], [8, 10],
+ [9, 11], [2, 3], [1, 2], [1, 3], [2, 4], [3, 5],
+ [4, 6], [5, 7]]
+ categories.append(cat)
+
+ json_files = [
+ pos for pos in os.listdir(gt_folder) if pos.endswith('.json')
+ ]
+ for json_file in json_files:
+
+ with open(osp.join(gt_folder, json_file), 'r') as f:
+ gt = json.load(f)
+
+ annotations = []
+ images = []
+
+ for image in gt['images']:
+ im = {}
+ im['id'] = image['id']
+ im['file_name'] = image['file_name']
+ images.append(im)
+
+ img_kpts = keypoint_results[im['id']]
+
+ if len(img_kpts) == 0:
+ continue
+ for track_id, img_kpt in enumerate(img_kpts[0]):
+ ann = {}
+ ann['image_id'] = img_kpt['image_id']
+ ann['keypoints'] = np.array(
+ img_kpt['keypoints']).reshape(-1).tolist()
+ ann['scores'] = np.array(ann['keypoints']).reshape(
+ [-1, 3])[:, 2].tolist()
+ ann['score'] = float(img_kpt['score'])
+ ann['track_id'] = track_id
+ annotations.append(ann)
+
+ info = {}
+ info['images'] = images
+ info['categories'] = categories
+ info['annotations'] = annotations
+
+ with open(osp.join(pred_folder, json_file), 'w') as f:
+ json.dump(info, f, sort_keys=True, indent=4)
+
+ def _do_python_keypoint_eval(self, gt_folder, pred_folder):
+ """Keypoint evaluation using poseval."""
+
+ if not has_poseval:
+ raise ImportError('Please install poseval package for evaluation'
+ 'on PoseTrack dataset '
+ '(see requirements/optional.txt)')
+
+ argv = ['', gt_folder + '/', pred_folder + '/']
+
+ print('Loading data')
+ gtFramesAll, prFramesAll = eval_helpers.load_data_dir(argv)
+
+ print('# gt frames :', len(gtFramesAll))
+ print('# pred frames:', len(prFramesAll))
+
+ # evaluate per-frame multi-person pose estimation (AP)
+ # compute AP
+ print('Evaluation of per-frame multi-person pose estimation')
+ apAll, _, _ = evaluateAP(gtFramesAll, prFramesAll, None, False, False)
+
+ # print AP
+ print('Average Precision (AP) metric:')
+ eval_helpers.printTable(apAll)
+
+ stats = eval_helpers.getCum(apAll)
+
+ stats_names = [
+ 'Head AP', 'Shou AP', 'Elb AP', 'Wri AP', 'Hip AP', 'Knee AP',
+ 'Ankl AP', 'Total AP'
+ ]
+
+ info_str = list(zip(stats_names, stats))
+
+ return info_str
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/top_down/topdown_posetrack18_video_dataset.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/top_down/topdown_posetrack18_video_dataset.py
new file mode 100644
index 0000000..045148d
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/datasets/top_down/topdown_posetrack18_video_dataset.py
@@ -0,0 +1,549 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os
+import os.path as osp
+import tempfile
+import warnings
+from collections import OrderedDict, defaultdict
+
+import json_tricks as json
+import numpy as np
+from mmcv import deprecated_api_warning
+
+from ....core.post_processing import oks_nms, soft_oks_nms
+from ...builder import DATASETS
+from ..base import Kpt2dSviewRgbVidTopDownDataset
+
+try:
+ from poseval import eval_helpers
+ from poseval.evaluateAP import evaluateAP
+ has_poseval = True
+except (ImportError, ModuleNotFoundError):
+ has_poseval = False
+
+
+@DATASETS.register_module()
+class TopDownPoseTrack18VideoDataset(Kpt2dSviewRgbVidTopDownDataset):
+ """PoseTrack18 dataset for top-down pose estimation.
+
+ "Posetrack: A benchmark for human pose estimation and tracking", CVPR'2018.
+ More details can be found in the `paper
+ `__ .
+
+ The dataset loads raw features and apply specified transforms
+ to return a dict containing the image tensors and other information.
+
+ PoseTrack2018 keypoint indexes::
+
+ 0: 'nose',
+ 1: 'head_bottom',
+ 2: 'head_top',
+ 3: 'left_ear',
+ 4: 'right_ear',
+ 5: 'left_shoulder',
+ 6: 'right_shoulder',
+ 7: 'left_elbow',
+ 8: 'right_elbow',
+ 9: 'left_wrist',
+ 10: 'right_wrist',
+ 11: 'left_hip',
+ 12: 'right_hip',
+ 13: 'left_knee',
+ 14: 'right_knee',
+ 15: 'left_ankle',
+ 16: 'right_ankle'
+
+ Args:
+ ann_file (str): Path to the annotation file.
+ img_prefix (str): Path to a directory where videos/images are held.
+ Default: None.
+ data_cfg (dict): config
+ pipeline (list[dict | callable]): A sequence of data transforms.
+ dataset_info (DatasetInfo): A class containing all dataset info.
+ test_mode (bool): Store True when building test or
+ validation dataset. Default: False.
+ ph_fill_len (int): The length of the placeholder to fill in the
+ image filenames, default: 6 in PoseTrack18.
+ """
+
+ def __init__(self,
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=None,
+ test_mode=False,
+ ph_fill_len=6):
+ super().__init__(
+ ann_file,
+ img_prefix,
+ data_cfg,
+ pipeline,
+ dataset_info=dataset_info,
+ test_mode=test_mode)
+
+ self.use_gt_bbox = data_cfg['use_gt_bbox']
+ self.bbox_file = data_cfg['bbox_file']
+ self.det_bbox_thr = data_cfg.get('det_bbox_thr', 0.0)
+ self.use_nms = data_cfg.get('use_nms', True)
+ self.soft_nms = data_cfg['soft_nms']
+ self.nms_thr = data_cfg['nms_thr']
+ self.oks_thr = data_cfg['oks_thr']
+ self.vis_thr = data_cfg['vis_thr']
+ self.frame_weight_train = data_cfg['frame_weight_train']
+ self.frame_weight_test = data_cfg['frame_weight_test']
+ self.frame_weight = self.frame_weight_test \
+ if self.test_mode else self.frame_weight_train
+
+ self.ph_fill_len = ph_fill_len
+
+ # select the frame indices
+ self.frame_index_rand = data_cfg.get('frame_index_rand', True)
+ self.frame_index_range = data_cfg.get('frame_index_range', [-2, 2])
+ self.num_adj_frames = data_cfg.get('num_adj_frames', 1)
+ self.frame_indices_train = data_cfg.get('frame_indices_train', None)
+ self.frame_indices_test = data_cfg.get('frame_indices_test',
+ [-2, -1, 0, 1, 2])
+
+ if self.frame_indices_train is not None:
+ self.frame_indices_train.sort()
+ self.frame_indices_test.sort()
+
+ self.db = self._get_db()
+
+ print(f'=> num_images: {self.num_images}')
+ print(f'=> load {len(self.db)} samples')
+
+ def _get_db(self):
+ """Load dataset."""
+ if (not self.test_mode) or self.use_gt_bbox:
+ # use ground truth bbox
+ gt_db = self._load_coco_keypoint_annotations()
+ else:
+ # use bbox from detection
+ gt_db = self._load_posetrack_person_detection_results()
+ return gt_db
+
+ def _load_coco_keypoint_annotations(self):
+ """Ground truth bbox and keypoints."""
+ gt_db = []
+ for img_id in self.img_ids:
+ gt_db.extend(self._load_coco_keypoint_annotation_kernel(img_id))
+ return gt_db
+
+ def _load_coco_keypoint_annotation_kernel(self, img_id):
+ """load annotation from COCOAPI.
+
+ Note:
+ bbox:[x1, y1, w, h]
+ Args:
+ img_id: coco image id
+ Returns:
+ dict: db entry
+ """
+ img_ann = self.coco.loadImgs(img_id)[0]
+ width = img_ann['width']
+ height = img_ann['height']
+ num_joints = self.ann_info['num_joints']
+
+ file_name = img_ann['file_name']
+ nframes = int(img_ann['nframes'])
+ frame_id = int(img_ann['frame_id'])
+
+ ann_ids = self.coco.getAnnIds(imgIds=img_id, iscrowd=False)
+ objs = self.coco.loadAnns(ann_ids)
+
+ # sanitize bboxes
+ valid_objs = []
+ for obj in objs:
+ if 'bbox' not in obj:
+ continue
+ x, y, w, h = obj['bbox']
+ x1 = max(0, x)
+ y1 = max(0, y)
+ x2 = min(width - 1, x1 + max(0, w - 1))
+ y2 = min(height - 1, y1 + max(0, h - 1))
+ if ('area' not in obj or obj['area'] > 0) and x2 > x1 and y2 > y1:
+ obj['clean_bbox'] = [x1, y1, x2 - x1, y2 - y1]
+ valid_objs.append(obj)
+ objs = valid_objs
+
+ bbox_id = 0
+ rec = []
+ for obj in objs:
+ if 'keypoints' not in obj:
+ continue
+ if max(obj['keypoints']) == 0:
+ continue
+ if 'num_keypoints' in obj and obj['num_keypoints'] == 0:
+ continue
+ joints_3d = np.zeros((num_joints, 3), dtype=np.float32)
+ joints_3d_visible = np.zeros((num_joints, 3), dtype=np.float32)
+
+ keypoints = np.array(obj['keypoints']).reshape(-1, 3)
+ joints_3d[:, :2] = keypoints[:, :2]
+ joints_3d_visible[:, :2] = np.minimum(1, keypoints[:, 2:3])
+
+ center, scale = self._xywh2cs(*obj['clean_bbox'][:4])
+
+ image_files = []
+ cur_image_file = osp.join(self.img_prefix, self.id2name[img_id])
+ image_files.append(cur_image_file)
+
+ # "images/val/012834_mpii_test/000000.jpg" -->> "000000.jpg"
+ cur_image_name = file_name.split('/')[-1]
+ ref_idx = int(cur_image_name.replace('.jpg', ''))
+
+ # select the frame indices
+ if not self.test_mode and self.frame_indices_train is not None:
+ indices = self.frame_indices_train
+ elif not self.test_mode and self.frame_index_rand:
+ low, high = self.frame_index_range
+ indices = np.random.randint(low, high + 1, self.num_adj_frames)
+ else:
+ indices = self.frame_indices_test
+
+ for index in indices:
+ if self.test_mode and index == 0:
+ continue
+ # the supporting frame index
+ support_idx = ref_idx + index
+ support_idx = np.clip(support_idx, 0, nframes - 1)
+ sup_image_file = cur_image_file.replace(
+ cur_image_name,
+ str(support_idx).zfill(self.ph_fill_len) + '.jpg')
+
+ if osp.exists(sup_image_file):
+ image_files.append(sup_image_file)
+ else:
+ warnings.warn(
+ f'{sup_image_file} does not exist, '
+ f'use {cur_image_file} instead.', UserWarning)
+ image_files.append(cur_image_file)
+ rec.append({
+ 'image_file': image_files,
+ 'center': center,
+ 'scale': scale,
+ 'bbox': obj['clean_bbox'][:4],
+ 'rotation': 0,
+ 'joints_3d': joints_3d,
+ 'joints_3d_visible': joints_3d_visible,
+ 'dataset': self.dataset_name,
+ 'bbox_score': 1,
+ 'bbox_id': bbox_id,
+ 'nframes': nframes,
+ 'frame_id': frame_id,
+ 'frame_weight': self.frame_weight
+ })
+ bbox_id = bbox_id + 1
+
+ return rec
+
+ def _load_posetrack_person_detection_results(self):
+ """Load Posetrack person detection results.
+
+ Only in test mode.
+ """
+ num_joints = self.ann_info['num_joints']
+ all_boxes = None
+ with open(self.bbox_file, 'r') as f:
+ all_boxes = json.load(f)
+
+ if not all_boxes:
+ raise ValueError('=> Load %s fail!' % self.bbox_file)
+
+ print(f'=> Total boxes: {len(all_boxes)}')
+
+ kpt_db = []
+ bbox_id = 0
+ for det_res in all_boxes:
+ if det_res['category_id'] != 1:
+ continue
+
+ score = det_res['score']
+ if score < self.det_bbox_thr:
+ continue
+
+ box = det_res['bbox']
+
+ # deal with different bbox file formats
+ if 'nframes' in det_res and 'frame_id' in det_res:
+ nframes = int(det_res['nframes'])
+ frame_id = int(det_res['frame_id'])
+ elif 'image_name' in det_res:
+ img_id = self.name2id[det_res['image_name']]
+ img_ann = self.coco.loadImgs(img_id)[0]
+ nframes = int(img_ann['nframes'])
+ frame_id = int(img_ann['frame_id'])
+ else:
+ img_id = det_res['image_id']
+ img_ann = self.coco.loadImgs(img_id)[0]
+ nframes = int(img_ann['nframes'])
+ frame_id = int(img_ann['frame_id'])
+
+ image_files = []
+ if 'image_name' in det_res:
+ file_name = det_res['image_name']
+ else:
+ file_name = self.id2name[det_res['image_id']]
+
+ cur_image_file = osp.join(self.img_prefix, file_name)
+ image_files.append(cur_image_file)
+
+ # "images/val/012834_mpii_test/000000.jpg" -->> "000000.jpg"
+ cur_image_name = file_name.split('/')[-1]
+ ref_idx = int(cur_image_name.replace('.jpg', ''))
+
+ indices = self.frame_indices_test
+ for index in indices:
+ if self.test_mode and index == 0:
+ continue
+ # the supporting frame index
+ support_idx = ref_idx + index
+ support_idx = np.clip(support_idx, 0, nframes - 1)
+ sup_image_file = cur_image_file.replace(
+ cur_image_name,
+ str(support_idx).zfill(self.ph_fill_len) + '.jpg')
+
+ if osp.exists(sup_image_file):
+ image_files.append(sup_image_file)
+ else:
+ warnings.warn(f'{sup_image_file} does not exist, '
+ f'use {cur_image_file} instead.')
+ image_files.append(cur_image_file)
+
+ center, scale = self._xywh2cs(*box[:4])
+ joints_3d = np.zeros((num_joints, 3), dtype=np.float32)
+ joints_3d_visible = np.ones((num_joints, 3), dtype=np.float32)
+ kpt_db.append({
+ 'image_file': image_files,
+ 'center': center,
+ 'scale': scale,
+ 'rotation': 0,
+ 'bbox': box[:4],
+ 'bbox_score': score,
+ 'dataset': self.dataset_name,
+ 'joints_3d': joints_3d,
+ 'joints_3d_visible': joints_3d_visible,
+ 'bbox_id': bbox_id,
+ 'nframes': nframes,
+ 'frame_id': frame_id,
+ 'frame_weight': self.frame_weight
+ })
+ bbox_id = bbox_id + 1
+ print(f'=> Total boxes after filter '
+ f'low score@{self.det_bbox_thr}: {bbox_id}')
+ return kpt_db
+
+ @deprecated_api_warning(name_dict=dict(outputs='results'))
+ def evaluate(self, results, res_folder=None, metric='mAP', **kwargs):
+ """Evaluate posetrack keypoint results. The pose prediction results
+ will be saved in ``${res_folder}/result_keypoints.json``.
+
+ Note:
+ - num_keypoints: K
+
+ Args:
+ results (list[dict]): Testing results containing the following
+ items:
+
+ - preds (np.ndarray[N,K,3]): The first two dimensions are \
+ coordinates, score is the third dimension of the array.
+ - boxes (np.ndarray[N,6]): [center[0], center[1], scale[0], \
+ scale[1],area, score]
+ - image_paths (list[str]): For example, ['val/010016_mpii_test\
+ /000024.jpg']
+ - heatmap (np.ndarray[N, K, H, W]): model output heatmap.
+ - bbox_id (list(int))
+ res_folder (str, optional): The folder to save the testing
+ results. If not specified, a temp folder will be created.
+ Default: None.
+ metric (str | list[str]): Metric to be performed. Defaults: 'mAP'.
+
+ Returns:
+ dict: Evaluation results for evaluation metric.
+ """
+ metrics = metric if isinstance(metric, list) else [metric]
+ allowed_metrics = ['mAP']
+ for metric in metrics:
+ if metric not in allowed_metrics:
+ raise KeyError(f'metric {metric} is not supported')
+
+ if res_folder is not None:
+ tmp_folder = None
+ else:
+ tmp_folder = tempfile.TemporaryDirectory()
+ res_folder = tmp_folder.name
+
+ gt_folder = osp.join(
+ osp.dirname(self.ann_file),
+ osp.splitext(self.ann_file.split('_')[-1])[0])
+
+ kpts = defaultdict(list)
+
+ for result in results:
+ preds = result['preds']
+ boxes = result['boxes']
+ image_paths = result['image_paths']
+ bbox_ids = result['bbox_ids']
+
+ batch_size = len(image_paths)
+ for i in range(batch_size):
+ if not isinstance(image_paths[i], list):
+ image_id = self.name2id[image_paths[i]
+ [len(self.img_prefix):]]
+ else:
+ image_id = self.name2id[image_paths[i][0]
+ [len(self.img_prefix):]]
+
+ kpts[image_id].append({
+ 'keypoints': preds[i],
+ 'center': boxes[i][0:2],
+ 'scale': boxes[i][2:4],
+ 'area': boxes[i][4],
+ 'score': boxes[i][5],
+ 'image_id': image_id,
+ 'bbox_id': bbox_ids[i]
+ })
+ kpts = self._sort_and_unique_bboxes(kpts)
+
+ # rescoring and oks nms
+ num_joints = self.ann_info['num_joints']
+ vis_thr = self.vis_thr
+ oks_thr = self.oks_thr
+ valid_kpts = defaultdict(list)
+ for image_id in kpts.keys():
+ img_kpts = kpts[image_id]
+ for n_p in img_kpts:
+ box_score = n_p['score']
+ kpt_score = 0
+ valid_num = 0
+ for n_jt in range(0, num_joints):
+ t_s = n_p['keypoints'][n_jt][2]
+ if t_s > vis_thr:
+ kpt_score = kpt_score + t_s
+ valid_num = valid_num + 1
+ if valid_num != 0:
+ kpt_score = kpt_score / valid_num
+ # rescoring
+ n_p['score'] = kpt_score * box_score
+
+ if self.use_nms:
+ nms = soft_oks_nms if self.soft_nms else oks_nms
+ keep = nms(img_kpts, oks_thr, sigmas=self.sigmas)
+ valid_kpts[image_id].append(
+ [img_kpts[_keep] for _keep in keep])
+ else:
+ valid_kpts[image_id].append(img_kpts)
+
+ self._write_keypoint_results(valid_kpts, gt_folder, res_folder)
+
+ info_str = self._do_keypoint_eval(gt_folder, res_folder)
+ name_value = OrderedDict(info_str)
+
+ if tmp_folder is not None:
+ tmp_folder.cleanup()
+
+ return name_value
+
+ @staticmethod
+ def _write_keypoint_results(keypoint_results, gt_folder, pred_folder):
+ """Write results into a json file.
+
+ Args:
+ keypoint_results (dict): keypoint results organized by image_id.
+ gt_folder (str): Path of directory for official gt files.
+ pred_folder (str): Path of directory to save the results.
+ """
+ categories = []
+
+ cat = {}
+ cat['supercategory'] = 'person'
+ cat['id'] = 1
+ cat['name'] = 'person'
+ cat['keypoints'] = [
+ 'nose', 'head_bottom', 'head_top', 'left_ear', 'right_ear',
+ 'left_shoulder', 'right_shoulder', 'left_elbow', 'right_elbow',
+ 'left_wrist', 'right_wrist', 'left_hip', 'right_hip', 'left_knee',
+ 'right_knee', 'left_ankle', 'right_ankle'
+ ]
+ cat['skeleton'] = [[16, 14], [14, 12], [17, 15], [15, 13], [12, 13],
+ [6, 12], [7, 13], [6, 7], [6, 8], [7, 9], [8, 10],
+ [9, 11], [2, 3], [1, 2], [1, 3], [2, 4], [3, 5],
+ [4, 6], [5, 7]]
+ categories.append(cat)
+
+ json_files = [
+ pos for pos in os.listdir(gt_folder) if pos.endswith('.json')
+ ]
+ for json_file in json_files:
+
+ with open(osp.join(gt_folder, json_file), 'r') as f:
+ gt = json.load(f)
+
+ annotations = []
+ images = []
+
+ for image in gt['images']:
+ im = {}
+ im['id'] = image['id']
+ im['file_name'] = image['file_name']
+ images.append(im)
+
+ img_kpts = keypoint_results[im['id']]
+
+ if len(img_kpts) == 0:
+ continue
+ for track_id, img_kpt in enumerate(img_kpts[0]):
+ ann = {}
+ ann['image_id'] = img_kpt['image_id']
+ ann['keypoints'] = np.array(
+ img_kpt['keypoints']).reshape(-1).tolist()
+ ann['scores'] = np.array(ann['keypoints']).reshape(
+ [-1, 3])[:, 2].tolist()
+ ann['score'] = float(img_kpt['score'])
+ ann['track_id'] = track_id
+ annotations.append(ann)
+
+ info = {}
+ info['images'] = images
+ info['categories'] = categories
+ info['annotations'] = annotations
+
+ with open(osp.join(pred_folder, json_file), 'w') as f:
+ json.dump(info, f, sort_keys=True, indent=4)
+
+ def _do_keypoint_eval(self, gt_folder, pred_folder):
+ """Keypoint evaluation using poseval."""
+
+ if not has_poseval:
+ raise ImportError('Please install poseval package for evaluation'
+ 'on PoseTrack dataset '
+ '(see requirements/optional.txt)')
+
+ argv = ['', gt_folder + '/', pred_folder + '/']
+
+ print('Loading data')
+ gtFramesAll, prFramesAll = eval_helpers.load_data_dir(argv)
+
+ print('# gt frames :', len(gtFramesAll))
+ print('# pred frames:', len(prFramesAll))
+
+ # evaluate per-frame multi-person pose estimation (AP)
+ # compute AP
+ print('Evaluation of per-frame multi-person pose estimation')
+ apAll, _, _ = evaluateAP(gtFramesAll, prFramesAll, None, False, False)
+
+ # print AP
+ print('Average Precision (AP) metric:')
+ eval_helpers.printTable(apAll)
+
+ stats = eval_helpers.getCum(apAll)
+
+ stats_names = [
+ 'Head AP', 'Shou AP', 'Elb AP', 'Wri AP', 'Hip AP', 'Knee AP',
+ 'Ankl AP', 'Total AP'
+ ]
+
+ info_str = list(zip(stats_names, stats))
+
+ return info_str
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/pipelines/__init__.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/pipelines/__init__.py
new file mode 100644
index 0000000..cf06db1
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/pipelines/__init__.py
@@ -0,0 +1,8 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from .bottom_up_transform import * # noqa
+from .hand_transform import * # noqa
+from .loading import LoadImageFromFile # noqa
+from .mesh_transform import * # noqa
+from .pose3d_transform import * # noqa
+from .shared_transform import * # noqa
+from .top_down_transform import * # noqa
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/pipelines/bottom_up_transform.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/pipelines/bottom_up_transform.py
new file mode 100644
index 0000000..032ce45
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/pipelines/bottom_up_transform.py
@@ -0,0 +1,816 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import cv2
+import numpy as np
+
+from mmpose.core.post_processing import (get_affine_transform, get_warp_matrix,
+ warp_affine_joints)
+from mmpose.datasets.builder import PIPELINES
+from .shared_transform import Compose
+
+
+def _ceil_to_multiples_of(x, base=64):
+ """Transform x to the integral multiple of the base."""
+ return int(np.ceil(x / base)) * base
+
+
+def _get_multi_scale_size(image,
+ input_size,
+ current_scale,
+ min_scale,
+ use_udp=False):
+ """Get the size for multi-scale training.
+
+ Args:
+ image: Input image.
+ input_size (np.ndarray[2]): Size (w, h) of the image input.
+ current_scale (float): Scale factor.
+ min_scale (float): Minimal scale.
+ use_udp (bool): To use unbiased data processing.
+ Paper ref: Huang et al. The Devil is in the Details: Delving into
+ Unbiased Data Processing for Human Pose Estimation (CVPR 2020).
+
+ Returns:
+ tuple: A tuple containing multi-scale sizes.
+
+ - (w_resized, h_resized) (tuple(int)): resized width/height
+ - center (np.ndarray)image center
+ - scale (np.ndarray): scales wrt width/height
+ """
+ assert len(input_size) == 2
+ h, w, _ = image.shape
+
+ # calculate the size for min_scale
+ min_input_w = _ceil_to_multiples_of(min_scale * input_size[0], 64)
+ min_input_h = _ceil_to_multiples_of(min_scale * input_size[1], 64)
+ if w < h:
+ w_resized = int(min_input_w * current_scale / min_scale)
+ h_resized = int(
+ _ceil_to_multiples_of(min_input_w / w * h, 64) * current_scale /
+ min_scale)
+ if use_udp:
+ scale_w = w - 1.0
+ scale_h = (h_resized - 1.0) / (w_resized - 1.0) * (w - 1.0)
+ else:
+ scale_w = w / 200.0
+ scale_h = h_resized / w_resized * w / 200.0
+ else:
+ h_resized = int(min_input_h * current_scale / min_scale)
+ w_resized = int(
+ _ceil_to_multiples_of(min_input_h / h * w, 64) * current_scale /
+ min_scale)
+ if use_udp:
+ scale_h = h - 1.0
+ scale_w = (w_resized - 1.0) / (h_resized - 1.0) * (h - 1.0)
+ else:
+ scale_h = h / 200.0
+ scale_w = w_resized / h_resized * h / 200.0
+ if use_udp:
+ center = (scale_w / 2.0, scale_h / 2.0)
+ else:
+ center = np.array([round(w / 2.0), round(h / 2.0)])
+ return (w_resized, h_resized), center, np.array([scale_w, scale_h])
+
+
+def _resize_align_multi_scale(image, input_size, current_scale, min_scale):
+ """Resize the images for multi-scale training.
+
+ Args:
+ image: Input image
+ input_size (np.ndarray[2]): Size (w, h) of the image input
+ current_scale (float): Current scale
+ min_scale (float): Minimal scale
+
+ Returns:
+ tuple: A tuple containing image info.
+
+ - image_resized (np.ndarray): resized image
+ - center (np.ndarray): center of image
+ - scale (np.ndarray): scale
+ """
+ assert len(input_size) == 2
+ size_resized, center, scale = _get_multi_scale_size(
+ image, input_size, current_scale, min_scale)
+
+ trans = get_affine_transform(center, scale, 0, size_resized)
+ image_resized = cv2.warpAffine(image, trans, size_resized)
+
+ return image_resized, center, scale
+
+
+def _resize_align_multi_scale_udp(image, input_size, current_scale, min_scale):
+ """Resize the images for multi-scale training.
+
+ Args:
+ image: Input image
+ input_size (np.ndarray[2]): Size (w, h) of the image input
+ current_scale (float): Current scale
+ min_scale (float): Minimal scale
+
+ Returns:
+ tuple: A tuple containing image info.
+
+ - image_resized (np.ndarray): resized image
+ - center (np.ndarray): center of image
+ - scale (np.ndarray): scale
+ """
+ assert len(input_size) == 2
+ size_resized, _, _ = _get_multi_scale_size(image, input_size,
+ current_scale, min_scale, True)
+
+ _, center, scale = _get_multi_scale_size(image, input_size, min_scale,
+ min_scale, True)
+
+ trans = get_warp_matrix(
+ theta=0,
+ size_input=np.array(scale, dtype=np.float32),
+ size_dst=np.array(size_resized, dtype=np.float32) - 1.0,
+ size_target=np.array(scale, dtype=np.float32))
+ image_resized = cv2.warpAffine(
+ image.copy(), trans, size_resized, flags=cv2.INTER_LINEAR)
+
+ return image_resized, center, scale
+
+
+class HeatmapGenerator:
+ """Generate heatmaps for bottom-up models.
+
+ Args:
+ num_joints (int): Number of keypoints
+ output_size (np.ndarray): Size (w, h) of feature map
+ sigma (int): Sigma of the heatmaps.
+ use_udp (bool): To use unbiased data processing.
+ Paper ref: Huang et al. The Devil is in the Details: Delving into
+ Unbiased Data Processing for Human Pose Estimation (CVPR 2020).
+ """
+
+ def __init__(self, output_size, num_joints, sigma=-1, use_udp=False):
+ if not isinstance(output_size, np.ndarray):
+ output_size = np.array(output_size)
+ if output_size.size > 1:
+ assert len(output_size) == 2
+ self.output_size = output_size
+ else:
+ self.output_size = np.array([output_size, output_size],
+ dtype=np.int)
+ self.num_joints = num_joints
+ if sigma < 0:
+ sigma = self.output_size.prod()**0.5 / 64
+ self.sigma = sigma
+ size = 6 * sigma + 3
+ self.use_udp = use_udp
+ if use_udp:
+ self.x = np.arange(0, size, 1, np.float32)
+ self.y = self.x[:, None]
+ else:
+ x = np.arange(0, size, 1, np.float32)
+ y = x[:, None]
+ x0, y0 = 3 * sigma + 1, 3 * sigma + 1
+ self.g = np.exp(-((x - x0)**2 + (y - y0)**2) / (2 * sigma**2))
+
+ def __call__(self, joints):
+ """Generate heatmaps."""
+ hms = np.zeros(
+ (self.num_joints, self.output_size[1], self.output_size[0]),
+ dtype=np.float32)
+
+ sigma = self.sigma
+ for p in joints:
+ for idx, pt in enumerate(p):
+ if pt[2] > 0:
+ x, y = int(pt[0]), int(pt[1])
+ if x < 0 or y < 0 or \
+ x >= self.output_size[0] or y >= self.output_size[1]:
+ continue
+
+ if self.use_udp:
+ x0 = 3 * sigma + 1 + pt[0] - x
+ y0 = 3 * sigma + 1 + pt[1] - y
+ g = np.exp(-((self.x - x0)**2 + (self.y - y0)**2) /
+ (2 * sigma**2))
+ else:
+ g = self.g
+
+ ul = int(np.round(x - 3 * sigma -
+ 1)), int(np.round(y - 3 * sigma - 1))
+ br = int(np.round(x + 3 * sigma +
+ 2)), int(np.round(y + 3 * sigma + 2))
+
+ c, d = max(0,
+ -ul[0]), min(br[0], self.output_size[0]) - ul[0]
+ a, b = max(0,
+ -ul[1]), min(br[1], self.output_size[1]) - ul[1]
+
+ cc, dd = max(0, ul[0]), min(br[0], self.output_size[0])
+ aa, bb = max(0, ul[1]), min(br[1], self.output_size[1])
+ hms[idx, aa:bb,
+ cc:dd] = np.maximum(hms[idx, aa:bb, cc:dd], g[a:b,
+ c:d])
+ return hms
+
+
+class JointsEncoder:
+ """Encodes the visible joints into (coordinates, score); The coordinate of
+ one joint and its score are of `int` type.
+
+ (idx * output_size**2 + y * output_size + x, 1) or (0, 0).
+
+ Args:
+ max_num_people(int): Max number of people in an image
+ num_joints(int): Number of keypoints
+ output_size(np.ndarray): Size (w, h) of feature map
+ tag_per_joint(bool): Option to use one tag map per joint.
+ """
+
+ def __init__(self, max_num_people, num_joints, output_size, tag_per_joint):
+ self.max_num_people = max_num_people
+ self.num_joints = num_joints
+ if not isinstance(output_size, np.ndarray):
+ output_size = np.array(output_size)
+ if output_size.size > 1:
+ assert len(output_size) == 2
+ self.output_size = output_size
+ else:
+ self.output_size = np.array([output_size, output_size],
+ dtype=np.int)
+ self.tag_per_joint = tag_per_joint
+
+ def __call__(self, joints):
+ """
+ Note:
+ - number of people in image: N
+ - number of keypoints: K
+ - max number of people in an image: M
+
+ Args:
+ joints (np.ndarray[N,K,3])
+
+ Returns:
+ visible_kpts (np.ndarray[M,K,2]).
+ """
+ visible_kpts = np.zeros((self.max_num_people, self.num_joints, 2),
+ dtype=np.float32)
+ for i in range(len(joints)):
+ tot = 0
+ for idx, pt in enumerate(joints[i]):
+ x, y = int(pt[0]), int(pt[1])
+ if (pt[2] > 0 and 0 <= y < self.output_size[1]
+ and 0 <= x < self.output_size[0]):
+ if self.tag_per_joint:
+ visible_kpts[i][tot] = \
+ (idx * self.output_size.prod()
+ + y * self.output_size[0] + x, 1)
+ else:
+ visible_kpts[i][tot] = (y * self.output_size[0] + x, 1)
+ tot += 1
+ return visible_kpts
+
+
+class PAFGenerator:
+ """Generate part affinity fields.
+
+ Args:
+ output_size (np.ndarray): Size (w, h) of feature map.
+ limb_width (int): Limb width of part affinity fields.
+ skeleton (list[list]): connections of joints.
+ """
+
+ def __init__(self, output_size, limb_width, skeleton):
+ if not isinstance(output_size, np.ndarray):
+ output_size = np.array(output_size)
+ if output_size.size > 1:
+ assert len(output_size) == 2
+ self.output_size = output_size
+ else:
+ self.output_size = np.array([output_size, output_size],
+ dtype=np.int)
+ self.limb_width = limb_width
+ self.skeleton = skeleton
+
+ def _accumulate_paf_map_(self, pafs, src, dst, count):
+ """Accumulate part affinity fields between two given joints.
+
+ Args:
+ pafs (np.ndarray[2,H,W]): paf maps (2 dimensions:x axis and
+ y axis) for a certain limb connection. This argument will
+ be modified inplace.
+ src (np.ndarray[2,]): coordinates of the source joint.
+ dst (np.ndarray[2,]): coordinates of the destination joint.
+ count (np.ndarray[H,W]): count map that preserves the number
+ of non-zero vectors at each point. This argument will be
+ modified inplace.
+ """
+ limb_vec = dst - src
+ norm = np.linalg.norm(limb_vec)
+ if norm == 0:
+ unit_limb_vec = np.zeros(2)
+ else:
+ unit_limb_vec = limb_vec / norm
+
+ min_x = max(np.floor(min(src[0], dst[0]) - self.limb_width), 0)
+ max_x = min(
+ np.ceil(max(src[0], dst[0]) + self.limb_width),
+ self.output_size[0] - 1)
+ min_y = max(np.floor(min(src[1], dst[1]) - self.limb_width), 0)
+ max_y = min(
+ np.ceil(max(src[1], dst[1]) + self.limb_width),
+ self.output_size[1] - 1)
+
+ range_x = list(range(int(min_x), int(max_x + 1), 1))
+ range_y = list(range(int(min_y), int(max_y + 1), 1))
+
+ mask = np.zeros_like(count, dtype=bool)
+ if len(range_x) > 0 and len(range_y) > 0:
+ xx, yy = np.meshgrid(range_x, range_y)
+ delta_x = xx - src[0]
+ delta_y = yy - src[1]
+ dist = np.abs(delta_x * unit_limb_vec[1] -
+ delta_y * unit_limb_vec[0])
+ mask_local = (dist < self.limb_width)
+ mask[yy, xx] = mask_local
+
+ pafs[0, mask] += unit_limb_vec[0]
+ pafs[1, mask] += unit_limb_vec[1]
+ count += mask
+
+ return pafs, count
+
+ def __call__(self, joints):
+ """Generate the target part affinity fields."""
+ pafs = np.zeros(
+ (len(self.skeleton) * 2, self.output_size[1], self.output_size[0]),
+ dtype=np.float32)
+
+ for idx, sk in enumerate(self.skeleton):
+ count = np.zeros((self.output_size[1], self.output_size[0]),
+ dtype=np.float32)
+
+ for p in joints:
+ src = p[sk[0]]
+ dst = p[sk[1]]
+ if src[2] > 0 and dst[2] > 0:
+ self._accumulate_paf_map_(pafs[2 * idx:2 * idx + 2],
+ src[:2], dst[:2], count)
+
+ pafs[2 * idx:2 * idx + 2] /= np.maximum(count, 1)
+
+ return pafs
+
+
+@PIPELINES.register_module()
+class BottomUpRandomFlip:
+ """Data augmentation with random image flip for bottom-up.
+
+ Args:
+ flip_prob (float): Probability of flip.
+ """
+
+ def __init__(self, flip_prob=0.5):
+ self.flip_prob = flip_prob
+
+ def __call__(self, results):
+ """Perform data augmentation with random image flip."""
+ image, mask, joints = results['img'], results['mask'], results[
+ 'joints']
+ self.flip_index = results['ann_info']['flip_index']
+ self.output_size = results['ann_info']['heatmap_size']
+
+ assert isinstance(mask, list)
+ assert isinstance(joints, list)
+ assert len(mask) == len(joints)
+ assert len(mask) == len(self.output_size)
+
+ if np.random.random() < self.flip_prob:
+ image = image[:, ::-1].copy() - np.zeros_like(image)
+ for i, _output_size in enumerate(self.output_size):
+ if not isinstance(_output_size, np.ndarray):
+ _output_size = np.array(_output_size)
+ if _output_size.size > 1:
+ assert len(_output_size) == 2
+ else:
+ _output_size = np.array([_output_size, _output_size],
+ dtype=np.int)
+ mask[i] = mask[i][:, ::-1].copy()
+ joints[i] = joints[i][:, self.flip_index]
+ joints[i][:, :, 0] = _output_size[0] - joints[i][:, :, 0] - 1
+ results['img'], results['mask'], results[
+ 'joints'] = image, mask, joints
+ return results
+
+
+@PIPELINES.register_module()
+class BottomUpRandomAffine:
+ """Data augmentation with random scaling & rotating.
+
+ Args:
+ rot_factor (int): Rotating to [-rotation_factor, rotation_factor]
+ scale_factor (float): Scaling to [1-scale_factor, 1+scale_factor]
+ scale_type: wrt ``long`` or ``short`` length of the image.
+ trans_factor: Translation factor.
+ use_udp (bool): To use unbiased data processing.
+ Paper ref: Huang et al. The Devil is in the Details: Delving into
+ Unbiased Data Processing for Human Pose Estimation (CVPR 2020).
+ """
+
+ def __init__(self,
+ rot_factor,
+ scale_factor,
+ scale_type,
+ trans_factor,
+ use_udp=False):
+ self.max_rotation = rot_factor
+ self.min_scale = scale_factor[0]
+ self.max_scale = scale_factor[1]
+ self.scale_type = scale_type
+ self.trans_factor = trans_factor
+ self.use_udp = use_udp
+
+ def _get_scale(self, image_size, resized_size):
+ w, h = image_size
+ w_resized, h_resized = resized_size
+ if w / w_resized < h / h_resized:
+ if self.scale_type == 'long':
+ w_pad = h / h_resized * w_resized
+ h_pad = h
+ elif self.scale_type == 'short':
+ w_pad = w
+ h_pad = w / w_resized * h_resized
+ else:
+ raise ValueError(f'Unknown scale type: {self.scale_type}')
+ else:
+ if self.scale_type == 'long':
+ w_pad = w
+ h_pad = w / w_resized * h_resized
+ elif self.scale_type == 'short':
+ w_pad = h / h_resized * w_resized
+ h_pad = h
+ else:
+ raise ValueError(f'Unknown scale type: {self.scale_type}')
+
+ scale = np.array([w_pad, h_pad], dtype=np.float32)
+
+ return scale
+
+ def __call__(self, results):
+ """Perform data augmentation with random scaling & rotating."""
+ image, mask, joints = results['img'], results['mask'], results[
+ 'joints']
+
+ self.input_size = results['ann_info']['image_size']
+ if not isinstance(self.input_size, np.ndarray):
+ self.input_size = np.array(self.input_size)
+ if self.input_size.size > 1:
+ assert len(self.input_size) == 2
+ else:
+ self.input_size = [self.input_size, self.input_size]
+ self.output_size = results['ann_info']['heatmap_size']
+
+ assert isinstance(mask, list)
+ assert isinstance(joints, list)
+ assert len(mask) == len(joints)
+ assert len(mask) == len(self.output_size), (len(mask),
+ len(self.output_size),
+ self.output_size)
+
+ height, width = image.shape[:2]
+ if self.use_udp:
+ center = np.array(((width - 1.0) / 2, (height - 1.0) / 2))
+ else:
+ center = np.array((width / 2, height / 2))
+
+ img_scale = np.array([width, height], dtype=np.float32)
+ aug_scale = np.random.random() * (self.max_scale - self.min_scale) \
+ + self.min_scale
+ img_scale *= aug_scale
+ aug_rot = (np.random.random() * 2 - 1) * self.max_rotation
+
+ if self.trans_factor > 0:
+ dx = np.random.randint(-self.trans_factor * img_scale[0] / 200.0,
+ self.trans_factor * img_scale[0] / 200.0)
+ dy = np.random.randint(-self.trans_factor * img_scale[1] / 200.0,
+ self.trans_factor * img_scale[1] / 200.0)
+
+ center[0] += dx
+ center[1] += dy
+ if self.use_udp:
+ for i, _output_size in enumerate(self.output_size):
+ if not isinstance(_output_size, np.ndarray):
+ _output_size = np.array(_output_size)
+ if _output_size.size > 1:
+ assert len(_output_size) == 2
+ else:
+ _output_size = [_output_size, _output_size]
+
+ scale = self._get_scale(img_scale, _output_size)
+
+ trans = get_warp_matrix(
+ theta=aug_rot,
+ size_input=center * 2.0,
+ size_dst=np.array(
+ (_output_size[0], _output_size[1]), dtype=np.float32) -
+ 1.0,
+ size_target=scale)
+ mask[i] = cv2.warpAffine(
+ (mask[i] * 255).astype(np.uint8),
+ trans, (int(_output_size[0]), int(_output_size[1])),
+ flags=cv2.INTER_LINEAR) / 255
+ mask[i] = (mask[i] > 0.5).astype(np.float32)
+ joints[i][:, :, 0:2] = \
+ warp_affine_joints(joints[i][:, :, 0:2].copy(), trans)
+ if results['ann_info']['scale_aware_sigma']:
+ joints[i][:, :, 3] = joints[i][:, :, 3] / aug_scale
+ scale = self._get_scale(img_scale, self.input_size)
+ mat_input = get_warp_matrix(
+ theta=aug_rot,
+ size_input=center * 2.0,
+ size_dst=np.array((self.input_size[0], self.input_size[1]),
+ dtype=np.float32) - 1.0,
+ size_target=scale)
+ image = cv2.warpAffine(
+ image,
+ mat_input, (int(self.input_size[0]), int(self.input_size[1])),
+ flags=cv2.INTER_LINEAR)
+ else:
+ for i, _output_size in enumerate(self.output_size):
+ if not isinstance(_output_size, np.ndarray):
+ _output_size = np.array(_output_size)
+ if _output_size.size > 1:
+ assert len(_output_size) == 2
+ else:
+ _output_size = [_output_size, _output_size]
+ scale = self._get_scale(img_scale, _output_size)
+ mat_output = get_affine_transform(
+ center=center,
+ scale=scale / 200.0,
+ rot=aug_rot,
+ output_size=_output_size)
+ mask[i] = cv2.warpAffine(
+ (mask[i] * 255).astype(np.uint8), mat_output,
+ (int(_output_size[0]), int(_output_size[1]))) / 255
+ mask[i] = (mask[i] > 0.5).astype(np.float32)
+
+ joints[i][:, :, 0:2] = \
+ warp_affine_joints(joints[i][:, :, 0:2], mat_output)
+ if results['ann_info']['scale_aware_sigma']:
+ joints[i][:, :, 3] = joints[i][:, :, 3] / aug_scale
+
+ scale = self._get_scale(img_scale, self.input_size)
+ mat_input = get_affine_transform(
+ center=center,
+ scale=scale / 200.0,
+ rot=aug_rot,
+ output_size=self.input_size)
+ image = cv2.warpAffine(image, mat_input, (int(
+ self.input_size[0]), int(self.input_size[1])))
+
+ results['img'], results['mask'], results[
+ 'joints'] = image, mask, joints
+
+ return results
+
+
+@PIPELINES.register_module()
+class BottomUpGenerateHeatmapTarget:
+ """Generate multi-scale heatmap target for bottom-up.
+
+ Args:
+ sigma (int): Sigma of heatmap Gaussian
+ max_num_people (int): Maximum number of people in an image
+ use_udp (bool): To use unbiased data processing.
+ Paper ref: Huang et al. The Devil is in the Details: Delving into
+ Unbiased Data Processing for Human Pose Estimation (CVPR 2020).
+ """
+
+ def __init__(self, sigma, use_udp=False):
+ self.sigma = sigma
+ self.use_udp = use_udp
+
+ def _generate(self, num_joints, heatmap_size):
+ """Get heatmap generator."""
+ heatmap_generator = [
+ HeatmapGenerator(output_size, num_joints, self.sigma, self.use_udp)
+ for output_size in heatmap_size
+ ]
+ return heatmap_generator
+
+ def __call__(self, results):
+ """Generate multi-scale heatmap target for bottom-up."""
+ heatmap_generator = \
+ self._generate(results['ann_info']['num_joints'],
+ results['ann_info']['heatmap_size'])
+ target_list = list()
+ joints_list = results['joints']
+
+ for scale_id in range(results['ann_info']['num_scales']):
+ heatmaps = heatmap_generator[scale_id](joints_list[scale_id])
+ target_list.append(heatmaps.astype(np.float32))
+ results['target'] = target_list
+
+ return results
+
+
+@PIPELINES.register_module()
+class BottomUpGenerateTarget:
+ """Generate multi-scale heatmap target for associate embedding.
+
+ Args:
+ sigma (int): Sigma of heatmap Gaussian
+ max_num_people (int): Maximum number of people in an image
+ use_udp (bool): To use unbiased data processing.
+ Paper ref: Huang et al. The Devil is in the Details: Delving into
+ Unbiased Data Processing for Human Pose Estimation (CVPR 2020).
+ """
+
+ def __init__(self, sigma, max_num_people, use_udp=False):
+ self.sigma = sigma
+ self.max_num_people = max_num_people
+ self.use_udp = use_udp
+
+ def _generate(self, num_joints, heatmap_size):
+ """Get heatmap generator and joint encoder."""
+ heatmap_generator = [
+ HeatmapGenerator(output_size, num_joints, self.sigma, self.use_udp)
+ for output_size in heatmap_size
+ ]
+ joints_encoder = [
+ JointsEncoder(self.max_num_people, num_joints, output_size, True)
+ for output_size in heatmap_size
+ ]
+ return heatmap_generator, joints_encoder
+
+ def __call__(self, results):
+ """Generate multi-scale heatmap target for bottom-up."""
+ heatmap_generator, joints_encoder = \
+ self._generate(results['ann_info']['num_joints'],
+ results['ann_info']['heatmap_size'])
+ target_list = list()
+ mask_list, joints_list = results['mask'], results['joints']
+
+ for scale_id in range(results['ann_info']['num_scales']):
+ target_t = heatmap_generator[scale_id](joints_list[scale_id])
+ joints_t = joints_encoder[scale_id](joints_list[scale_id])
+
+ target_list.append(target_t.astype(np.float32))
+ mask_list[scale_id] = mask_list[scale_id].astype(np.float32)
+ joints_list[scale_id] = joints_t.astype(np.int32)
+
+ results['masks'], results['joints'] = mask_list, joints_list
+ results['targets'] = target_list
+
+ return results
+
+
+@PIPELINES.register_module()
+class BottomUpGeneratePAFTarget:
+ """Generate multi-scale heatmaps and part affinity fields (PAF) target for
+ bottom-up. Paper ref: Cao et al. Realtime Multi-Person 2D Human Pose
+ Estimation using Part Affinity Fields (CVPR 2017).
+
+ Args:
+ limb_width (int): Limb width of part affinity fields
+ """
+
+ def __init__(self, limb_width, skeleton=None):
+ self.limb_width = limb_width
+ self.skeleton = skeleton
+
+ def _generate(self, heatmap_size, skeleton):
+ """Get PAF generator."""
+ paf_generator = [
+ PAFGenerator(output_size, self.limb_width, skeleton)
+ for output_size in heatmap_size
+ ]
+ return paf_generator
+
+ def __call__(self, results):
+ """Generate multi-scale part affinity fields for bottom-up."""
+ if self.skeleton is None:
+ assert results['ann_info']['skeleton'] is not None
+ self.skeleton = results['ann_info']['skeleton']
+
+ paf_generator = \
+ self._generate(results['ann_info']['heatmap_size'],
+ self.skeleton)
+ target_list = list()
+ joints_list = results['joints']
+
+ for scale_id in range(results['ann_info']['num_scales']):
+ pafs = paf_generator[scale_id](joints_list[scale_id])
+ target_list.append(pafs.astype(np.float32))
+
+ results['target'] = target_list
+
+ return results
+
+
+@PIPELINES.register_module()
+class BottomUpGetImgSize:
+ """Get multi-scale image sizes for bottom-up, including base_size and
+ test_scale_factor. Keep the ratio and the image is resized to
+ `results['ann_info']['image_size']×current_scale`.
+
+ Args:
+ test_scale_factor (List[float]): Multi scale
+ current_scale (int): default 1
+ use_udp (bool): To use unbiased data processing.
+ Paper ref: Huang et al. The Devil is in the Details: Delving into
+ Unbiased Data Processing for Human Pose Estimation (CVPR 2020).
+ """
+
+ def __init__(self, test_scale_factor, current_scale=1, use_udp=False):
+ self.test_scale_factor = test_scale_factor
+ self.min_scale = min(test_scale_factor)
+ self.current_scale = current_scale
+ self.use_udp = use_udp
+
+ def __call__(self, results):
+ """Get multi-scale image sizes for bottom-up."""
+ input_size = results['ann_info']['image_size']
+ if not isinstance(input_size, np.ndarray):
+ input_size = np.array(input_size)
+ if input_size.size > 1:
+ assert len(input_size) == 2
+ else:
+ input_size = np.array([input_size, input_size], dtype=np.int)
+ img = results['img']
+
+ h, w, _ = img.shape
+
+ # calculate the size for min_scale
+ min_input_w = _ceil_to_multiples_of(self.min_scale * input_size[0], 64)
+ min_input_h = _ceil_to_multiples_of(self.min_scale * input_size[1], 64)
+ if w < h:
+ w_resized = int(min_input_w * self.current_scale / self.min_scale)
+ h_resized = int(
+ _ceil_to_multiples_of(min_input_w / w * h, 64) *
+ self.current_scale / self.min_scale)
+ if self.use_udp:
+ scale_w = w - 1.0
+ scale_h = (h_resized - 1.0) / (w_resized - 1.0) * (w - 1.0)
+ else:
+ scale_w = w / 200.0
+ scale_h = h_resized / w_resized * w / 200.0
+ else:
+ h_resized = int(min_input_h * self.current_scale / self.min_scale)
+ w_resized = int(
+ _ceil_to_multiples_of(min_input_h / h * w, 64) *
+ self.current_scale / self.min_scale)
+ if self.use_udp:
+ scale_h = h - 1.0
+ scale_w = (w_resized - 1.0) / (h_resized - 1.0) * (h - 1.0)
+ else:
+ scale_h = h / 200.0
+ scale_w = w_resized / h_resized * h / 200.0
+ if self.use_udp:
+ center = (scale_w / 2.0, scale_h / 2.0)
+ else:
+ center = np.array([round(w / 2.0), round(h / 2.0)])
+ results['ann_info']['test_scale_factor'] = self.test_scale_factor
+ results['ann_info']['base_size'] = (w_resized, h_resized)
+ results['ann_info']['center'] = center
+ results['ann_info']['scale'] = np.array([scale_w, scale_h])
+
+ return results
+
+
+@PIPELINES.register_module()
+class BottomUpResizeAlign:
+ """Resize multi-scale size and align transform for bottom-up.
+
+ Args:
+ transforms (List): ToTensor & Normalize
+ use_udp (bool): To use unbiased data processing.
+ Paper ref: Huang et al. The Devil is in the Details: Delving into
+ Unbiased Data Processing for Human Pose Estimation (CVPR 2020).
+ """
+
+ def __init__(self, transforms, use_udp=False):
+ self.transforms = Compose(transforms)
+ if use_udp:
+ self._resize_align_multi_scale = _resize_align_multi_scale_udp
+ else:
+ self._resize_align_multi_scale = _resize_align_multi_scale
+
+ def __call__(self, results):
+ """Resize multi-scale size and align transform for bottom-up."""
+ input_size = results['ann_info']['image_size']
+ if not isinstance(input_size, np.ndarray):
+ input_size = np.array(input_size)
+ if input_size.size > 1:
+ assert len(input_size) == 2
+ else:
+ input_size = np.array([input_size, input_size], dtype=np.int)
+ test_scale_factor = results['ann_info']['test_scale_factor']
+ aug_data = []
+
+ for _, s in enumerate(sorted(test_scale_factor, reverse=True)):
+ _results = results.copy()
+ image_resized, _, _ = self._resize_align_multi_scale(
+ _results['img'], input_size, s, min(test_scale_factor))
+ _results['img'] = image_resized
+ _results = self.transforms(_results)
+ transformed_img = _results['img'].unsqueeze(0)
+ aug_data.append(transformed_img)
+
+ results['ann_info']['aug_data'] = aug_data
+
+ return results
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/pipelines/hand_transform.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/pipelines/hand_transform.py
new file mode 100644
index 0000000..b83e399
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/pipelines/hand_transform.py
@@ -0,0 +1,63 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import numpy as np
+
+from mmpose.datasets.builder import PIPELINES
+from .top_down_transform import TopDownRandomFlip
+
+
+@PIPELINES.register_module()
+class HandRandomFlip(TopDownRandomFlip):
+ """Data augmentation with random image flip. A child class of
+ TopDownRandomFlip.
+
+ Required keys: 'img', 'joints_3d', 'joints_3d_visible', 'center',
+ 'hand_type', 'rel_root_depth' and 'ann_info'.
+
+ Modifies key: 'img', 'joints_3d', 'joints_3d_visible', 'center',
+ 'hand_type', 'rel_root_depth'.
+
+ Args:
+ flip_prob (float): Probability of flip.
+ """
+
+ def __call__(self, results):
+ """Perform data augmentation with random image flip."""
+ # base flip augmentation
+ super().__call__(results)
+
+ # flip hand type and root depth
+ hand_type = results['hand_type']
+ rel_root_depth = results['rel_root_depth']
+ flipped = results['flipped']
+ if flipped:
+ hand_type[0], hand_type[1] = hand_type[1], hand_type[0]
+ rel_root_depth = -rel_root_depth
+ results['hand_type'] = hand_type
+ results['rel_root_depth'] = rel_root_depth
+ return results
+
+
+@PIPELINES.register_module()
+class HandGenerateRelDepthTarget:
+ """Generate the target relative root depth.
+
+ Required keys: 'rel_root_depth', 'rel_root_valid', 'ann_info'.
+
+ Modified keys: 'target', 'target_weight'.
+ """
+
+ def __init__(self):
+ pass
+
+ def __call__(self, results):
+ """Generate the target heatmap."""
+ rel_root_depth = results['rel_root_depth']
+ rel_root_valid = results['rel_root_valid']
+ cfg = results['ann_info']
+ D = cfg['heatmap_size_root']
+ root_depth_bound = cfg['root_depth_bound']
+ target = (rel_root_depth / root_depth_bound + 0.5) * D
+ target_weight = rel_root_valid * (target >= 0) * (target <= D)
+ results['target'] = target * np.ones(1, dtype=np.float32)
+ results['target_weight'] = target_weight * np.ones(1, dtype=np.float32)
+ return results
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/pipelines/loading.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/pipelines/loading.py
new file mode 100644
index 0000000..6475005
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/pipelines/loading.py
@@ -0,0 +1,91 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import mmcv
+import numpy as np
+
+from ..builder import PIPELINES
+
+
+@PIPELINES.register_module()
+class LoadImageFromFile:
+ """Loading image(s) from file.
+
+ Required key: "image_file".
+
+ Added key: "img".
+
+ Args:
+ to_float32 (bool): Whether to convert the loaded image to a float32
+ numpy array. If set to False, the loaded image is an uint8 array.
+ Defaults to False.
+ color_type (str): Flags specifying the color type of a loaded image,
+ candidates are 'color', 'grayscale' and 'unchanged'.
+ channel_order (str): Order of channel, candidates are 'bgr' and 'rgb'.
+ file_client_args (dict): Arguments to instantiate a FileClient.
+ See :class:`mmcv.fileio.FileClient` for details.
+ Defaults to ``dict(backend='disk')``.
+ """
+
+ def __init__(self,
+ to_float32=False,
+ color_type='color',
+ channel_order='rgb',
+ file_client_args=dict(backend='disk')):
+ self.to_float32 = to_float32
+ self.color_type = color_type
+ self.channel_order = channel_order
+ self.file_client_args = file_client_args.copy()
+ self.file_client = None
+
+ def _read_image(self, path):
+ img_bytes = self.file_client.get(path)
+ img = mmcv.imfrombytes(
+ img_bytes, flag=self.color_type, channel_order=self.channel_order)
+ if img is None:
+ raise ValueError(f'Fail to read {path}')
+ if self.to_float32:
+ img = img.astype(np.float32)
+ return img
+
+ def __call__(self, results):
+ """Loading image(s) from file."""
+ if self.file_client is None:
+ self.file_client = mmcv.FileClient(**self.file_client_args)
+
+ image_file = results.get('image_file', None)
+
+ if isinstance(image_file, (list, tuple)):
+ # Load images from a list of paths
+ results['img'] = [self._read_image(path) for path in image_file]
+ elif image_file is not None:
+ # Load single image from path
+ results['img'] = self._read_image(image_file)
+ else:
+ if 'img' not in results:
+ # If `image_file`` is not in results, check the `img` exists
+ # and format the image. This for compatibility when the image
+ # is manually set outside the pipeline.
+ raise KeyError('Either `image_file` or `img` should exist in '
+ 'results.')
+ assert isinstance(results['img'], np.ndarray)
+ if self.color_type == 'color' and self.channel_order == 'rgb':
+ # The original results['img'] is assumed to be image(s) in BGR
+ # order, so we convert the color according to the arguments.
+ if results['img'].ndim == 3:
+ results['img'] = mmcv.bgr2rgb(results['img'])
+ elif results['img'].ndim == 4:
+ results['img'] = np.concatenate(
+ [mmcv.bgr2rgb(img) for img in results['img']], axis=0)
+ else:
+ raise ValueError('results["img"] has invalid shape '
+ f'{results["img"].shape}')
+
+ results['image_file'] = None
+
+ return results
+
+ def __repr__(self):
+ repr_str = (f'{self.__class__.__name__}('
+ f'to_float32={self.to_float32}, '
+ f"color_type='{self.color_type}', "
+ f'file_client_args={self.file_client_args})')
+ return repr_str
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/pipelines/mesh_transform.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/pipelines/mesh_transform.py
new file mode 100644
index 0000000..e3f32fe
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/pipelines/mesh_transform.py
@@ -0,0 +1,399 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import cv2
+import mmcv
+import numpy as np
+import torch
+
+from mmpose.core.post_processing import (affine_transform, fliplr_joints,
+ get_affine_transform)
+from mmpose.datasets.builder import PIPELINES
+
+
+def _flip_smpl_pose(pose):
+ """Flip SMPL pose parameters horizontally.
+
+ Args:
+ pose (np.ndarray([72])): SMPL pose parameters
+
+ Returns:
+ pose_flipped
+ """
+
+ flippedParts = [
+ 0, 1, 2, 6, 7, 8, 3, 4, 5, 9, 10, 11, 15, 16, 17, 12, 13, 14, 18, 19,
+ 20, 24, 25, 26, 21, 22, 23, 27, 28, 29, 33, 34, 35, 30, 31, 32, 36, 37,
+ 38, 42, 43, 44, 39, 40, 41, 45, 46, 47, 51, 52, 53, 48, 49, 50, 57, 58,
+ 59, 54, 55, 56, 63, 64, 65, 60, 61, 62, 69, 70, 71, 66, 67, 68
+ ]
+ pose_flipped = pose[flippedParts]
+ # Negate the second and the third dimension of the axis-angle
+ pose_flipped[1::3] = -pose_flipped[1::3]
+ pose_flipped[2::3] = -pose_flipped[2::3]
+ return pose_flipped
+
+
+def _flip_iuv(iuv, uv_type='BF'):
+ """Flip IUV image horizontally.
+
+ Note:
+ IUV image height: H
+ IUV image width: W
+
+ Args:
+ iuv np.ndarray([H, W, 3]): IUV image
+ uv_type (str): The type of the UV map.
+ Candidate values:
+ 'DP': The UV map used in DensePose project.
+ 'SMPL': The default UV map of SMPL model.
+ 'BF': The UV map used in DecoMR project.
+ Default: 'BF'
+
+ Returns:
+ iuv_flipped np.ndarray([H, W, 3]): Flipped IUV image
+ """
+ assert uv_type in ['DP', 'SMPL', 'BF']
+ if uv_type == 'BF':
+ iuv_flipped = iuv[:, ::-1, :]
+ iuv_flipped[:, :, 1] = 255 - iuv_flipped[:, :, 1]
+ else:
+ # The flip of other UV map is complex, not finished yet.
+ raise NotImplementedError(
+ f'The flip of {uv_type} UV map is not implemented yet.')
+
+ return iuv_flipped
+
+
+def _construct_rotation_matrix(rot, size=3):
+ """Construct the in-plane rotation matrix.
+
+ Args:
+ rot (float): Rotation angle (degree).
+ size (int): The size of the rotation matrix.
+ Candidate Values: 2, 3. Defaults to 3.
+
+ Returns:
+ rot_mat (np.ndarray([size, size]): Rotation matrix.
+ """
+ rot_mat = np.eye(size, dtype=np.float32)
+ if rot != 0:
+ rot_rad = np.deg2rad(rot)
+ sn, cs = np.sin(rot_rad), np.cos(rot_rad)
+ rot_mat[0, :2] = [cs, -sn]
+ rot_mat[1, :2] = [sn, cs]
+
+ return rot_mat
+
+
+def _rotate_joints_3d(joints_3d, rot):
+ """Rotate the 3D joints in the local coordinates.
+
+ Note:
+ Joints number: K
+
+ Args:
+ joints_3d (np.ndarray([K, 3])): Coordinates of keypoints.
+ rot (float): Rotation angle (degree).
+
+ Returns:
+ joints_3d_rotated
+ """
+ # in-plane rotation
+ # 3D joints are rotated counterclockwise,
+ # so the rot angle is inversed.
+ rot_mat = _construct_rotation_matrix(-rot, 3)
+
+ joints_3d_rotated = np.einsum('ij,kj->ki', rot_mat, joints_3d)
+ joints_3d_rotated = joints_3d_rotated.astype('float32')
+ return joints_3d_rotated
+
+
+def _rotate_smpl_pose(pose, rot):
+ """Rotate SMPL pose parameters. SMPL (https://smpl.is.tue.mpg.de/) is a 3D
+ human model.
+
+ Args:
+ pose (np.ndarray([72])): SMPL pose parameters
+ rot (float): Rotation angle (degree).
+
+ Returns:
+ pose_rotated
+ """
+ pose_rotated = pose.copy()
+ if rot != 0:
+ rot_mat = _construct_rotation_matrix(-rot)
+ orient = pose[:3]
+ # find the rotation of the body in camera frame
+ per_rdg, _ = cv2.Rodrigues(orient)
+ # apply the global rotation to the global orientation
+ res_rot, _ = cv2.Rodrigues(np.dot(rot_mat, per_rdg))
+ pose_rotated[:3] = (res_rot.T)[0]
+
+ return pose_rotated
+
+
+def _flip_joints_3d(joints_3d, joints_3d_visible, flip_pairs):
+ """Flip human joints in 3D space horizontally.
+
+ Note:
+ num_keypoints: K
+
+ Args:
+ joints_3d (np.ndarray([K, 3])): Coordinates of keypoints.
+ joints_3d_visible (np.ndarray([K, 1])): Visibility of keypoints.
+ flip_pairs (list[tuple()]): Pairs of keypoints which are mirrored
+ (for example, left ear -- right ear).
+
+ Returns:
+ joints_3d_flipped, joints_3d_visible_flipped
+ """
+
+ assert len(joints_3d) == len(joints_3d_visible)
+
+ joints_3d_flipped = joints_3d.copy()
+ joints_3d_visible_flipped = joints_3d_visible.copy()
+
+ # Swap left-right parts
+ for left, right in flip_pairs:
+ joints_3d_flipped[left, :] = joints_3d[right, :]
+ joints_3d_flipped[right, :] = joints_3d[left, :]
+
+ joints_3d_visible_flipped[left, :] = joints_3d_visible[right, :]
+ joints_3d_visible_flipped[right, :] = joints_3d_visible[left, :]
+
+ # Flip horizontally
+ joints_3d_flipped[:, 0] = -joints_3d_flipped[:, 0]
+ joints_3d_flipped = joints_3d_flipped * joints_3d_visible_flipped
+
+ return joints_3d_flipped, joints_3d_visible_flipped
+
+
+@PIPELINES.register_module()
+class LoadIUVFromFile:
+ """Loading IUV image from file."""
+
+ def __init__(self, to_float32=False):
+ self.to_float32 = to_float32
+ self.color_type = 'color'
+ # channel relations: iuv->bgr
+ self.channel_order = 'bgr'
+
+ def __call__(self, results):
+ """Loading image from file."""
+ has_iuv = results['has_iuv']
+ use_iuv = results['ann_info']['use_IUV']
+ if has_iuv and use_iuv:
+ iuv_file = results['iuv_file']
+ iuv = mmcv.imread(iuv_file, self.color_type, self.channel_order)
+ if iuv is None:
+ raise ValueError(f'Fail to read {iuv_file}')
+ else:
+ has_iuv = 0
+ iuv = None
+
+ results['has_iuv'] = has_iuv
+ results['iuv'] = iuv
+ return results
+
+
+@PIPELINES.register_module()
+class IUVToTensor:
+ """Transform IUV image to part index mask and uv coordinates image. The 3
+ channels of IUV image means: part index, u coordinates, v coordinates.
+
+ Required key: 'iuv', 'ann_info'.
+ Modifies key: 'part_index', 'uv_coordinates'.
+
+ Args:
+ results (dict): contain all information about training.
+ """
+
+ def __call__(self, results):
+ iuv = results['iuv']
+ if iuv is None:
+ H, W = results['ann_info']['iuv_size']
+ part_index = torch.zeros([1, H, W], dtype=torch.long)
+ uv_coordinates = torch.zeros([2, H, W], dtype=torch.float32)
+ else:
+ part_index = torch.LongTensor(iuv[:, :, 0])[None, :, :]
+ uv_coordinates = torch.FloatTensor(iuv[:, :, 1:]) / 255
+ uv_coordinates = uv_coordinates.permute(2, 0, 1)
+ results['part_index'] = part_index
+ results['uv_coordinates'] = uv_coordinates
+ return results
+
+
+@PIPELINES.register_module()
+class MeshRandomChannelNoise:
+ """Data augmentation with random channel noise.
+
+ Required keys: 'img'
+ Modifies key: 'img'
+
+ Args:
+ noise_factor (float): Multiply each channel with
+ a factor between``[1-scale_factor, 1+scale_factor]``
+ """
+
+ def __init__(self, noise_factor=0.4):
+ self.noise_factor = noise_factor
+
+ def __call__(self, results):
+ """Perform data augmentation with random channel noise."""
+ img = results['img']
+
+ # Each channel is multiplied with a number
+ # in the area [1-self.noise_factor, 1+self.noise_factor]
+ pn = np.random.uniform(1 - self.noise_factor, 1 + self.noise_factor,
+ (1, 3))
+ img = cv2.multiply(img, pn)
+
+ results['img'] = img
+ return results
+
+
+@PIPELINES.register_module()
+class MeshRandomFlip:
+ """Data augmentation with random image flip.
+
+ Required keys: 'img', 'joints_2d','joints_2d_visible', 'joints_3d',
+ 'joints_3d_visible', 'center', 'pose', 'iuv' and 'ann_info'.
+ Modifies key: 'img', 'joints_2d','joints_2d_visible', 'joints_3d',
+ 'joints_3d_visible', 'center', 'pose', 'iuv'.
+
+ Args:
+ flip_prob (float): Probability of flip.
+ """
+
+ def __init__(self, flip_prob=0.5):
+ self.flip_prob = flip_prob
+
+ def __call__(self, results):
+ """Perform data augmentation with random image flip."""
+ if np.random.rand() > self.flip_prob:
+ return results
+
+ img = results['img']
+ joints_2d = results['joints_2d']
+ joints_2d_visible = results['joints_2d_visible']
+ joints_3d = results['joints_3d']
+ joints_3d_visible = results['joints_3d_visible']
+ pose = results['pose']
+ center = results['center']
+
+ img = img[:, ::-1, :]
+ pose = _flip_smpl_pose(pose)
+
+ joints_2d, joints_2d_visible = fliplr_joints(
+ joints_2d, joints_2d_visible, img.shape[1],
+ results['ann_info']['flip_pairs'])
+
+ joints_3d, joints_3d_visible = _flip_joints_3d(
+ joints_3d, joints_3d_visible, results['ann_info']['flip_pairs'])
+ center[0] = img.shape[1] - center[0] - 1
+
+ if 'iuv' in results.keys():
+ iuv = results['iuv']
+ if iuv is not None:
+ iuv = _flip_iuv(iuv, results['ann_info']['uv_type'])
+ results['iuv'] = iuv
+
+ results['img'] = img
+ results['joints_2d'] = joints_2d
+ results['joints_2d_visible'] = joints_2d_visible
+ results['joints_3d'] = joints_3d
+ results['joints_3d_visible'] = joints_3d_visible
+ results['pose'] = pose
+ results['center'] = center
+ return results
+
+
+@PIPELINES.register_module()
+class MeshGetRandomScaleRotation:
+ """Data augmentation with random scaling & rotating.
+
+ Required key: 'scale'. Modifies key: 'scale' and 'rotation'.
+
+ Args:
+ rot_factor (int): Rotating to ``[-2*rot_factor, 2*rot_factor]``.
+ scale_factor (float): Scaling to ``[1-scale_factor, 1+scale_factor]``.
+ rot_prob (float): Probability of random rotation.
+ """
+
+ def __init__(self, rot_factor=30, scale_factor=0.25, rot_prob=0.6):
+ self.rot_factor = rot_factor
+ self.scale_factor = scale_factor
+ self.rot_prob = rot_prob
+
+ def __call__(self, results):
+ """Perform data augmentation with random scaling & rotating."""
+ s = results['scale']
+
+ sf = self.scale_factor
+ rf = self.rot_factor
+
+ s_factor = np.clip(np.random.randn() * sf + 1, 1 - sf, 1 + sf)
+ s = s * s_factor
+
+ r_factor = np.clip(np.random.randn() * rf, -rf * 2, rf * 2)
+ r = r_factor if np.random.rand() <= self.rot_prob else 0
+
+ results['scale'] = s
+ results['rotation'] = r
+
+ return results
+
+
+@PIPELINES.register_module()
+class MeshAffine:
+ """Affine transform the image to get input image. Affine transform the 2D
+ keypoints, 3D kepoints and IUV image too.
+
+ Required keys: 'img', 'joints_2d','joints_2d_visible', 'joints_3d',
+ 'joints_3d_visible', 'pose', 'iuv', 'ann_info','scale', 'rotation' and
+ 'center'. Modifies key: 'img', 'joints_2d','joints_2d_visible',
+ 'joints_3d', 'pose', 'iuv'.
+ """
+
+ def __call__(self, results):
+ image_size = results['ann_info']['image_size']
+
+ img = results['img']
+ joints_2d = results['joints_2d']
+ joints_2d_visible = results['joints_2d_visible']
+ joints_3d = results['joints_3d']
+ pose = results['pose']
+
+ c = results['center']
+ s = results['scale']
+ r = results['rotation']
+ trans = get_affine_transform(c, s, r, image_size)
+
+ img = cv2.warpAffine(
+ img,
+ trans, (int(image_size[0]), int(image_size[1])),
+ flags=cv2.INTER_LINEAR)
+
+ for i in range(results['ann_info']['num_joints']):
+ if joints_2d_visible[i, 0] > 0.0:
+ joints_2d[i] = affine_transform(joints_2d[i], trans)
+
+ joints_3d = _rotate_joints_3d(joints_3d, r)
+ pose = _rotate_smpl_pose(pose, r)
+
+ results['img'] = img
+ results['joints_2d'] = joints_2d
+ results['joints_2d_visible'] = joints_2d_visible
+ results['joints_3d'] = joints_3d
+ results['pose'] = pose
+
+ if 'iuv' in results.keys():
+ iuv = results['iuv']
+ if iuv is not None:
+ iuv_size = results['ann_info']['iuv_size']
+ iuv = cv2.warpAffine(
+ iuv,
+ trans, (int(iuv_size[0]), int(iuv_size[1])),
+ flags=cv2.INTER_NEAREST)
+ results['iuv'] = iuv
+
+ return results
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/pipelines/pose3d_transform.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/pipelines/pose3d_transform.py
new file mode 100644
index 0000000..1249378
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/pipelines/pose3d_transform.py
@@ -0,0 +1,643 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import copy
+
+import mmcv
+import numpy as np
+import torch
+from mmcv.utils import build_from_cfg
+
+from mmpose.core.camera import CAMERAS
+from mmpose.core.post_processing import fliplr_regression
+from mmpose.datasets.builder import PIPELINES
+
+
+@PIPELINES.register_module()
+class GetRootCenteredPose:
+ """Zero-center the pose around a given root joint. Optionally, the root
+ joint can be removed from the original pose and stored as a separate item.
+
+ Note that the root-centered joints may no longer align with some annotation
+ information (e.g. flip_pairs, num_joints, inference_channel, etc.) due to
+ the removal of the root joint.
+
+ Args:
+ item (str): The name of the pose to apply root-centering.
+ root_index (int): Root joint index in the pose.
+ visible_item (str): The name of the visibility item.
+ remove_root (bool): If true, remove the root joint from the pose
+ root_name (str): Optional. If not none, it will be used as the key to
+ store the root position separated from the original pose.
+
+ Required keys:
+ item
+
+ Modified keys:
+ item, visible_item, root_name
+ """
+
+ def __init__(self,
+ item,
+ root_index,
+ visible_item=None,
+ remove_root=False,
+ root_name=None):
+ self.item = item
+ self.root_index = root_index
+ self.remove_root = remove_root
+ self.root_name = root_name
+ self.visible_item = visible_item
+
+ def __call__(self, results):
+ assert self.item in results
+ joints = results[self.item]
+ root_idx = self.root_index
+
+ assert joints.ndim >= 2 and joints.shape[-2] > root_idx,\
+ f'Got invalid joint shape {joints.shape}'
+
+ root = joints[..., root_idx:root_idx + 1, :]
+ joints = joints - root
+
+ results[self.item] = joints
+ if self.root_name is not None:
+ results[self.root_name] = root
+
+ if self.remove_root:
+ results[self.item] = np.delete(
+ results[self.item], root_idx, axis=-2)
+ if self.visible_item is not None:
+ assert self.visible_item in results
+ results[self.visible_item] = np.delete(
+ results[self.visible_item], root_idx, axis=-2)
+ # Add a flag to avoid latter transforms that rely on the root
+ # joint or the original joint index
+ results[f'{self.item}_root_removed'] = True
+
+ # Save the root index which is necessary to restore the global pose
+ if self.root_name is not None:
+ results[f'{self.root_name}_index'] = self.root_index
+
+ return results
+
+
+@PIPELINES.register_module()
+class NormalizeJointCoordinate:
+ """Normalize the joint coordinate with given mean and std.
+
+ Args:
+ item (str): The name of the pose to normalize.
+ mean (array): Mean values of joint coordinates in shape [K, C].
+ std (array): Std values of joint coordinates in shape [K, C].
+ norm_param_file (str): Optionally load a dict containing `mean` and
+ `std` from a file using `mmcv.load`.
+
+ Required keys:
+ item
+
+ Modified keys:
+ item
+ """
+
+ def __init__(self, item, mean=None, std=None, norm_param_file=None):
+ self.item = item
+ self.norm_param_file = norm_param_file
+ if norm_param_file is not None:
+ norm_param = mmcv.load(norm_param_file)
+ assert 'mean' in norm_param and 'std' in norm_param
+ mean = norm_param['mean']
+ std = norm_param['std']
+ else:
+ assert mean is not None
+ assert std is not None
+
+ self.mean = np.array(mean, dtype=np.float32)
+ self.std = np.array(std, dtype=np.float32)
+
+ def __call__(self, results):
+ assert self.item in results
+ results[self.item] = (results[self.item] - self.mean) / self.std
+ results[f'{self.item}_mean'] = self.mean.copy()
+ results[f'{self.item}_std'] = self.std.copy()
+ return results
+
+
+@PIPELINES.register_module()
+class ImageCoordinateNormalization:
+ """Normalize the 2D joint coordinate with image width and height. Range [0,
+ w] is mapped to [-1, 1], while preserving the aspect ratio.
+
+ Args:
+ item (str|list[str]): The name of the pose to normalize.
+ norm_camera (bool): Whether to normalize camera intrinsics.
+ Default: False.
+ camera_param (dict|None): The camera parameter dict. See the camera
+ class definition for more details. If None is given, the camera
+ parameter will be obtained during processing of each data sample
+ with the key "camera_param".
+
+ Required keys:
+ item
+
+ Modified keys:
+ item (, camera_param)
+ """
+
+ def __init__(self, item, norm_camera=False, camera_param=None):
+ self.item = item
+ if isinstance(self.item, str):
+ self.item = [self.item]
+
+ self.norm_camera = norm_camera
+
+ if camera_param is None:
+ self.static_camera = False
+ else:
+ self.static_camera = True
+ self.camera_param = camera_param
+
+ def __call__(self, results):
+ center = np.array(
+ [0.5 * results['image_width'], 0.5 * results['image_height']],
+ dtype=np.float32)
+ scale = np.array(0.5 * results['image_width'], dtype=np.float32)
+
+ for item in self.item:
+ results[item] = (results[item] - center) / scale
+
+ if self.norm_camera:
+ if self.static_camera:
+ camera_param = copy.deepcopy(self.camera_param)
+ else:
+ assert 'camera_param' in results, \
+ 'Camera parameters are missing.'
+ camera_param = results['camera_param']
+ assert 'f' in camera_param and 'c' in camera_param
+ camera_param['f'] = camera_param['f'] / scale
+ camera_param['c'] = (camera_param['c'] - center[:, None]) / scale
+ if 'camera_param' not in results:
+ results['camera_param'] = dict()
+ results['camera_param'].update(camera_param)
+
+ return results
+
+
+@PIPELINES.register_module()
+class CollectCameraIntrinsics:
+ """Store camera intrinsics in a 1-dim array, including f, c, k, p.
+
+ Args:
+ camera_param (dict|None): The camera parameter dict. See the camera
+ class definition for more details. If None is given, the camera
+ parameter will be obtained during processing of each data sample
+ with the key "camera_param".
+ need_distortion (bool): Whether need distortion parameters k and p.
+ Default: True.
+
+ Required keys:
+ camera_param (if camera parameters are not given in initialization)
+
+ Modified keys:
+ intrinsics
+ """
+
+ def __init__(self, camera_param=None, need_distortion=True):
+ if camera_param is None:
+ self.static_camera = False
+ else:
+ self.static_camera = True
+ self.camera_param = camera_param
+ self.need_distortion = need_distortion
+
+ def __call__(self, results):
+ if self.static_camera:
+ camera_param = copy.deepcopy(self.camera_param)
+ else:
+ assert 'camera_param' in results, 'Camera parameters are missing.'
+ camera_param = results['camera_param']
+ assert 'f' in camera_param and 'c' in camera_param
+ intrinsics = np.concatenate(
+ [camera_param['f'].reshape(2), camera_param['c'].reshape(2)])
+ if self.need_distortion:
+ assert 'k' in camera_param and 'p' in camera_param
+ intrinsics = np.concatenate([
+ intrinsics, camera_param['k'].reshape(3),
+ camera_param['p'].reshape(2)
+ ])
+ results['intrinsics'] = intrinsics
+
+ return results
+
+
+@PIPELINES.register_module()
+class CameraProjection:
+ """Apply camera projection to joint coordinates.
+
+ Args:
+ item (str): The name of the pose to apply camera projection.
+ mode (str): The type of camera projection, supported options are
+
+ - world_to_camera
+ - world_to_pixel
+ - camera_to_world
+ - camera_to_pixel
+ output_name (str|None): The name of the projected pose. If None
+ (default) is given, the projected pose will be stored in place.
+ camera_type (str): The camera class name (should be registered in
+ CAMERA).
+ camera_param (dict|None): The camera parameter dict. See the camera
+ class definition for more details. If None is given, the camera
+ parameter will be obtained during processing of each data sample
+ with the key "camera_param".
+
+ Required keys:
+
+ - item
+ - camera_param (if camera parameters are not given in initialization)
+
+ Modified keys:
+ output_name
+ """
+
+ def __init__(self,
+ item,
+ mode,
+ output_name=None,
+ camera_type='SimpleCamera',
+ camera_param=None):
+ self.item = item
+ self.mode = mode
+ self.output_name = output_name
+ self.camera_type = camera_type
+ allowed_mode = {
+ 'world_to_camera',
+ 'world_to_pixel',
+ 'camera_to_world',
+ 'camera_to_pixel',
+ }
+ if mode not in allowed_mode:
+ raise ValueError(
+ f'Got invalid mode: {mode}, allowed modes are {allowed_mode}')
+
+ if camera_param is None:
+ self.static_camera = False
+ else:
+ self.static_camera = True
+ self.camera = self._build_camera(camera_param)
+
+ def _build_camera(self, param):
+ cfgs = dict(type=self.camera_type, param=param)
+ return build_from_cfg(cfgs, CAMERAS)
+
+ def __call__(self, results):
+ assert self.item in results
+ joints = results[self.item]
+
+ if self.static_camera:
+ camera = self.camera
+ else:
+ assert 'camera_param' in results, 'Camera parameters are missing.'
+ camera = self._build_camera(results['camera_param'])
+
+ if self.mode == 'world_to_camera':
+ output = camera.world_to_camera(joints)
+ elif self.mode == 'world_to_pixel':
+ output = camera.world_to_pixel(joints)
+ elif self.mode == 'camera_to_world':
+ output = camera.camera_to_world(joints)
+ elif self.mode == 'camera_to_pixel':
+ output = camera.camera_to_pixel(joints)
+ else:
+ raise NotImplementedError
+
+ output_name = self.output_name
+ if output_name is None:
+ output_name = self.item
+
+ results[output_name] = output
+ return results
+
+
+@PIPELINES.register_module()
+class RelativeJointRandomFlip:
+ """Data augmentation with random horizontal joint flip around a root joint.
+
+ Args:
+ item (str|list[str]): The name of the pose to flip.
+ flip_cfg (dict|list[dict]): Configurations of the fliplr_regression
+ function. It should contain the following arguments:
+
+ - ``center_mode``: The mode to set the center location on the \
+ x-axis to flip around.
+ - ``center_x`` or ``center_index``: Set the x-axis location or \
+ the root joint's index to define the flip center.
+
+ Please refer to the docstring of the fliplr_regression function for
+ more details.
+ visible_item (str|list[str]): The name of the visibility item which
+ will be flipped accordingly along with the pose.
+ flip_prob (float): Probability of flip.
+ flip_camera (bool): Whether to flip horizontal distortion coefficients.
+ camera_param (dict|None): The camera parameter dict. See the camera
+ class definition for more details. If None is given, the camera
+ parameter will be obtained during processing of each data sample
+ with the key "camera_param".
+
+ Required keys:
+ item
+
+ Modified keys:
+ item (, camera_param)
+ """
+
+ def __init__(self,
+ item,
+ flip_cfg,
+ visible_item=None,
+ flip_prob=0.5,
+ flip_camera=False,
+ camera_param=None):
+ self.item = item
+ self.flip_cfg = flip_cfg
+ self.vis_item = visible_item
+ self.flip_prob = flip_prob
+ self.flip_camera = flip_camera
+ if camera_param is None:
+ self.static_camera = False
+ else:
+ self.static_camera = True
+ self.camera_param = camera_param
+
+ if isinstance(self.item, str):
+ self.item = [self.item]
+ if isinstance(self.flip_cfg, dict):
+ self.flip_cfg = [self.flip_cfg] * len(self.item)
+ assert len(self.item) == len(self.flip_cfg)
+ if isinstance(self.vis_item, str):
+ self.vis_item = [self.vis_item]
+
+ def __call__(self, results):
+
+ if results.get(f'{self.item}_root_removed', False):
+ raise RuntimeError('The transform RelativeJointRandomFlip should '
+ f'not be applied to {self.item} whose root '
+ 'joint has been removed and joint indices have '
+ 'been changed')
+
+ if np.random.rand() <= self.flip_prob:
+
+ flip_pairs = results['ann_info']['flip_pairs']
+
+ # flip joint coordinates
+ for i, item in enumerate(self.item):
+ assert item in results
+ joints = results[item]
+
+ joints_flipped = fliplr_regression(joints, flip_pairs,
+ **self.flip_cfg[i])
+
+ results[item] = joints_flipped
+
+ # flip joint visibility
+ for vis_item in self.vis_item:
+ assert vis_item in results
+ visible = results[vis_item]
+ visible_flipped = visible.copy()
+ for left, right in flip_pairs:
+ visible_flipped[..., left, :] = visible[..., right, :]
+ visible_flipped[..., right, :] = visible[..., left, :]
+ results[vis_item] = visible_flipped
+
+ # flip horizontal distortion coefficients
+ if self.flip_camera:
+ if self.static_camera:
+ camera_param = copy.deepcopy(self.camera_param)
+ else:
+ assert 'camera_param' in results, \
+ 'Camera parameters are missing.'
+ camera_param = results['camera_param']
+ assert 'c' in camera_param
+ camera_param['c'][0] *= -1
+
+ if 'p' in camera_param:
+ camera_param['p'][0] *= -1
+
+ if 'camera_param' not in results:
+ results['camera_param'] = dict()
+ results['camera_param'].update(camera_param)
+
+ return results
+
+
+@PIPELINES.register_module()
+class PoseSequenceToTensor:
+ """Convert pose sequence from numpy array to Tensor.
+
+ The original pose sequence should have a shape of [T,K,C] or [K,C], where
+ T is the sequence length, K and C are keypoint number and dimension. The
+ converted pose sequence will have a shape of [KxC, T].
+
+ Args:
+ item (str): The name of the pose sequence
+
+ Required keys:
+ item
+
+ Modified keys:
+ item
+ """
+
+ def __init__(self, item):
+ self.item = item
+
+ def __call__(self, results):
+ assert self.item in results
+ seq = results[self.item]
+
+ assert isinstance(seq, np.ndarray)
+ assert seq.ndim in {2, 3}
+
+ if seq.ndim == 2:
+ seq = seq[None, ...]
+
+ T = seq.shape[0]
+ seq = seq.transpose(1, 2, 0).reshape(-1, T)
+ results[self.item] = torch.from_numpy(seq)
+
+ return results
+
+
+@PIPELINES.register_module()
+class Generate3DHeatmapTarget:
+ """Generate the target 3d heatmap.
+
+ Required keys: 'joints_3d', 'joints_3d_visible', 'ann_info'.
+ Modified keys: 'target', and 'target_weight'.
+
+ Args:
+ sigma: Sigma of heatmap gaussian.
+ joint_indices (list): Indices of joints used for heatmap generation.
+ If None (default) is given, all joints will be used.
+ max_bound (float): The maximal value of heatmap.
+ """
+
+ def __init__(self, sigma=2, joint_indices=None, max_bound=1.0):
+ self.sigma = sigma
+ self.joint_indices = joint_indices
+ self.max_bound = max_bound
+
+ def __call__(self, results):
+ """Generate the target heatmap."""
+ joints_3d = results['joints_3d']
+ joints_3d_visible = results['joints_3d_visible']
+ cfg = results['ann_info']
+ image_size = cfg['image_size']
+ W, H, D = cfg['heatmap_size']
+ heatmap3d_depth_bound = cfg['heatmap3d_depth_bound']
+ joint_weights = cfg['joint_weights']
+ use_different_joint_weights = cfg['use_different_joint_weights']
+
+ # select the joints used for target generation
+ if self.joint_indices is not None:
+ joints_3d = joints_3d[self.joint_indices, ...]
+ joints_3d_visible = joints_3d_visible[self.joint_indices, ...]
+ joint_weights = joint_weights[self.joint_indices, ...]
+ num_joints = joints_3d.shape[0]
+
+ # get the joint location in heatmap coordinates
+ mu_x = joints_3d[:, 0] * W / image_size[0]
+ mu_y = joints_3d[:, 1] * H / image_size[1]
+ mu_z = (joints_3d[:, 2] / heatmap3d_depth_bound + 0.5) * D
+
+ target = np.zeros([num_joints, D, H, W], dtype=np.float32)
+
+ target_weight = joints_3d_visible[:, 0].astype(np.float32)
+ target_weight = target_weight * (mu_z >= 0) * (mu_z < D)
+ if use_different_joint_weights:
+ target_weight = target_weight * joint_weights
+ target_weight = target_weight[:, None]
+
+ # only compute the voxel value near the joints location
+ tmp_size = 3 * self.sigma
+
+ # get neighboring voxels coordinates
+ x = y = z = np.arange(2 * tmp_size + 1, dtype=np.float32) - tmp_size
+ zz, yy, xx = np.meshgrid(z, y, x)
+ xx = xx[None, ...].astype(np.float32)
+ yy = yy[None, ...].astype(np.float32)
+ zz = zz[None, ...].astype(np.float32)
+ mu_x = mu_x[..., None, None, None]
+ mu_y = mu_y[..., None, None, None]
+ mu_z = mu_z[..., None, None, None]
+ xx, yy, zz = xx + mu_x, yy + mu_y, zz + mu_z
+
+ # round the coordinates
+ xx = xx.round().clip(0, W - 1)
+ yy = yy.round().clip(0, H - 1)
+ zz = zz.round().clip(0, D - 1)
+
+ # compute the target value near joints
+ local_target = \
+ np.exp(-((xx - mu_x)**2 + (yy - mu_y)**2 + (zz - mu_z)**2) /
+ (2 * self.sigma**2))
+
+ # put the local target value to the full target heatmap
+ local_size = xx.shape[1]
+ idx_joints = np.tile(
+ np.arange(num_joints)[:, None, None, None],
+ [1, local_size, local_size, local_size])
+ idx = np.stack([idx_joints, zz, yy, xx],
+ axis=-1).astype(int).reshape(-1, 4)
+ target[idx[:, 0], idx[:, 1], idx[:, 2],
+ idx[:, 3]] = local_target.reshape(-1)
+ target = target * self.max_bound
+ results['target'] = target
+ results['target_weight'] = target_weight
+ return results
+
+
+@PIPELINES.register_module()
+class GenerateVoxel3DHeatmapTarget:
+ """Generate the target 3d heatmap.
+
+ Required keys: 'joints_3d', 'joints_3d_visible', 'ann_info_3d'.
+ Modified keys: 'target', and 'target_weight'.
+
+ Args:
+ sigma: Sigma of heatmap gaussian (mm).
+ joint_indices (list): Indices of joints used for heatmap generation.
+ If None (default) is given, all joints will be used.
+ """
+
+ def __init__(self, sigma=200.0, joint_indices=None):
+ self.sigma = sigma # mm
+ self.joint_indices = joint_indices
+
+ def __call__(self, results):
+ """Generate the target heatmap."""
+ joints_3d = results['joints_3d']
+ joints_3d_visible = results['joints_3d_visible']
+ cfg = results['ann_info']
+
+ num_people = len(joints_3d)
+ num_joints = joints_3d[0].shape[0]
+
+ if self.joint_indices is not None:
+ num_joints = len(self.joint_indices)
+ joint_indices = self.joint_indices
+ else:
+ joint_indices = list(range(num_joints))
+
+ space_size = cfg['space_size']
+ space_center = cfg['space_center']
+ cube_size = cfg['cube_size']
+ grids_x = np.linspace(-space_size[0] / 2, space_size[0] / 2,
+ cube_size[0]) + space_center[0]
+ grids_y = np.linspace(-space_size[1] / 2, space_size[1] / 2,
+ cube_size[1]) + space_center[1]
+ grids_z = np.linspace(-space_size[2] / 2, space_size[2] / 2,
+ cube_size[2]) + space_center[2]
+
+ target = np.zeros(
+ (num_joints, cube_size[0], cube_size[1], cube_size[2]),
+ dtype=np.float32)
+
+ for n in range(num_people):
+ for idx, joint_id in enumerate(joint_indices):
+ mu_x = joints_3d[n][joint_id][0]
+ mu_y = joints_3d[n][joint_id][1]
+ mu_z = joints_3d[n][joint_id][2]
+ vis = joints_3d_visible[n][joint_id][0]
+ if vis < 1:
+ continue
+ i_x = [
+ np.searchsorted(grids_x, mu_x - 3 * self.sigma),
+ np.searchsorted(grids_x, mu_x + 3 * self.sigma, 'right')
+ ]
+ i_y = [
+ np.searchsorted(grids_y, mu_y - 3 * self.sigma),
+ np.searchsorted(grids_y, mu_y + 3 * self.sigma, 'right')
+ ]
+ i_z = [
+ np.searchsorted(grids_z, mu_z - 3 * self.sigma),
+ np.searchsorted(grids_z, mu_z + 3 * self.sigma, 'right')
+ ]
+ if i_x[0] >= i_x[1] or i_y[0] >= i_y[1] or i_z[0] >= i_z[1]:
+ continue
+ kernel_xs, kernel_ys, kernel_zs = np.meshgrid(
+ grids_x[i_x[0]:i_x[1]],
+ grids_y[i_y[0]:i_y[1]],
+ grids_z[i_z[0]:i_z[1]],
+ indexing='ij')
+ g = np.exp(-((kernel_xs - mu_x)**2 + (kernel_ys - mu_y)**2 +
+ (kernel_zs - mu_z)**2) / (2 * self.sigma**2))
+ target[idx, i_x[0]:i_x[1], i_y[0]:i_y[1], i_z[0]:i_z[1]] \
+ = np.maximum(target[idx, i_x[0]:i_x[1],
+ i_y[0]:i_y[1], i_z[0]:i_z[1]], g)
+
+ target = np.clip(target, 0, 1)
+ if target.shape[0] == 1:
+ target = target[0]
+
+ results['targets_3d'] = target
+
+ return results
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/pipelines/shared_transform.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/pipelines/shared_transform.py
new file mode 100644
index 0000000..e4fea80
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/pipelines/shared_transform.py
@@ -0,0 +1,527 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import warnings
+from collections.abc import Sequence
+
+import mmcv
+import numpy as np
+from mmcv.parallel import DataContainer as DC
+from mmcv.utils import build_from_cfg
+from numpy import random
+from torchvision.transforms import functional as F
+
+from ..builder import PIPELINES
+
+try:
+ import albumentations
+except ImportError:
+ albumentations = None
+
+
+@PIPELINES.register_module()
+class ToTensor:
+ """Transform image to Tensor.
+
+ Required key: 'img'. Modifies key: 'img'.
+
+ Args:
+ results (dict): contain all information about training.
+ """
+
+ def __call__(self, results):
+ if isinstance(results['img'], (list, tuple)):
+ results['img'] = [F.to_tensor(img) for img in results['img']]
+ else:
+ results['img'] = F.to_tensor(results['img'])
+
+ return results
+
+
+@PIPELINES.register_module()
+class NormalizeTensor:
+ """Normalize the Tensor image (CxHxW), with mean and std.
+
+ Required key: 'img'. Modifies key: 'img'.
+
+ Args:
+ mean (list[float]): Mean values of 3 channels.
+ std (list[float]): Std values of 3 channels.
+ """
+
+ def __init__(self, mean, std):
+ self.mean = mean
+ self.std = std
+
+ def __call__(self, results):
+ if isinstance(results['img'], (list, tuple)):
+ results['img'] = [
+ F.normalize(img, mean=self.mean, std=self.std)
+ for img in results['img']
+ ]
+ else:
+ results['img'] = F.normalize(
+ results['img'], mean=self.mean, std=self.std)
+
+ return results
+
+
+@PIPELINES.register_module()
+class Compose:
+ """Compose a data pipeline with a sequence of transforms.
+
+ Args:
+ transforms (list[dict | callable]): Either config
+ dicts of transforms or transform objects.
+ """
+
+ def __init__(self, transforms):
+ assert isinstance(transforms, Sequence)
+ self.transforms = []
+ for transform in transforms:
+ if isinstance(transform, dict):
+ transform = build_from_cfg(transform, PIPELINES)
+ self.transforms.append(transform)
+ elif callable(transform):
+ self.transforms.append(transform)
+ else:
+ raise TypeError('transform must be callable or a dict, but got'
+ f' {type(transform)}')
+
+ def __call__(self, data):
+ """Call function to apply transforms sequentially.
+
+ Args:
+ data (dict): A result dict contains the data to transform.
+
+ Returns:
+ dict: Transformed data.
+ """
+ for t in self.transforms:
+ data = t(data)
+ if data is None:
+ return None
+ return data
+
+ def __repr__(self):
+ """Compute the string representation."""
+ format_string = self.__class__.__name__ + '('
+ for t in self.transforms:
+ format_string += f'\n {t}'
+ format_string += '\n)'
+ return format_string
+
+
+@PIPELINES.register_module()
+class Collect:
+ """Collect data from the loader relevant to the specific task.
+
+ This keeps the items in `keys` as it is, and collect items in `meta_keys`
+ into a meta item called `meta_name`.This is usually the last stage of the
+ data loader pipeline.
+ For example, when keys='imgs', meta_keys=('filename', 'label',
+ 'original_shape'), meta_name='img_metas', the results will be a dict with
+ keys 'imgs' and 'img_metas', where 'img_metas' is a DataContainer of
+ another dict with keys 'filename', 'label', 'original_shape'.
+
+ Args:
+ keys (Sequence[str|tuple]): Required keys to be collected. If a tuple
+ (key, key_new) is given as an element, the item retrieved by key will
+ be renamed as key_new in collected data.
+ meta_name (str): The name of the key that contains meta information.
+ This key is always populated. Default: "img_metas".
+ meta_keys (Sequence[str|tuple]): Keys that are collected under
+ meta_name. The contents of the `meta_name` dictionary depends
+ on `meta_keys`.
+ """
+
+ def __init__(self, keys, meta_keys, meta_name='img_metas'):
+ self.keys = keys
+ self.meta_keys = meta_keys
+ self.meta_name = meta_name
+
+ def __call__(self, results):
+ """Performs the Collect formatting.
+
+ Args:
+ results (dict): The resulting dict to be modified and passed
+ to the next transform in pipeline.
+ """
+ if 'ann_info' in results:
+ results.update(results['ann_info'])
+
+ data = {}
+ for key in self.keys:
+ if isinstance(key, tuple):
+ assert len(key) == 2
+ key_src, key_tgt = key[:2]
+ else:
+ key_src = key_tgt = key
+ data[key_tgt] = results[key_src]
+
+ meta = {}
+ if len(self.meta_keys) != 0:
+ for key in self.meta_keys:
+ if isinstance(key, tuple):
+ assert len(key) == 2
+ key_src, key_tgt = key[:2]
+ else:
+ key_src = key_tgt = key
+ meta[key_tgt] = results[key_src]
+ if 'bbox_id' in results:
+ meta['bbox_id'] = results['bbox_id']
+ data[self.meta_name] = DC(meta, cpu_only=True)
+
+ return data
+
+ def __repr__(self):
+ """Compute the string representation."""
+ return (f'{self.__class__.__name__}('
+ f'keys={self.keys}, meta_keys={self.meta_keys})')
+
+
+@PIPELINES.register_module()
+class Albumentation:
+ """Albumentation augmentation (pixel-level transforms only). Adds custom
+ pixel-level transformations from Albumentations library. Please visit
+ `https://albumentations.readthedocs.io` to get more information.
+
+ Note: we only support pixel-level transforms.
+ Please visit `https://github.com/albumentations-team/`
+ `albumentations#pixel-level-transforms`
+ to get more information about pixel-level transforms.
+
+ An example of ``transforms`` is as followed:
+
+ .. code-block:: python
+
+ [
+ dict(
+ type='RandomBrightnessContrast',
+ brightness_limit=[0.1, 0.3],
+ contrast_limit=[0.1, 0.3],
+ p=0.2),
+ dict(type='ChannelShuffle', p=0.1),
+ dict(
+ type='OneOf',
+ transforms=[
+ dict(type='Blur', blur_limit=3, p=1.0),
+ dict(type='MedianBlur', blur_limit=3, p=1.0)
+ ],
+ p=0.1),
+ ]
+
+ Args:
+ transforms (list[dict]): A list of Albumentation transformations
+ keymap (dict): Contains {'input key':'albumentation-style key'},
+ e.g., {'img': 'image'}.
+ """
+
+ def __init__(self, transforms, keymap=None):
+ if albumentations is None:
+ raise RuntimeError('albumentations is not installed')
+
+ self.transforms = transforms
+ self.filter_lost_elements = False
+
+ self.aug = albumentations.Compose(
+ [self.albu_builder(t) for t in self.transforms])
+
+ if not keymap:
+ self.keymap_to_albu = {
+ 'img': 'image',
+ }
+ else:
+ self.keymap_to_albu = keymap
+ self.keymap_back = {v: k for k, v in self.keymap_to_albu.items()}
+
+ def albu_builder(self, cfg):
+ """Import a module from albumentations.
+
+ It resembles some of :func:`build_from_cfg` logic.
+
+ Args:
+ cfg (dict): Config dict. It should at least contain the key "type".
+
+ Returns:
+ obj: The constructed object.
+ """
+
+ assert isinstance(cfg, dict) and 'type' in cfg
+ args = cfg.copy()
+
+ obj_type = args.pop('type')
+ if mmcv.is_str(obj_type):
+ if albumentations is None:
+ raise RuntimeError('albumentations is not installed')
+ if not hasattr(albumentations.augmentations.transforms, obj_type):
+ warnings.warn('{obj_type} is not pixel-level transformations. '
+ 'Please use with caution.')
+ obj_cls = getattr(albumentations, obj_type)
+ else:
+ raise TypeError(f'type must be a str, but got {type(obj_type)}')
+
+ if 'transforms' in args:
+ args['transforms'] = [
+ self.albu_builder(transform)
+ for transform in args['transforms']
+ ]
+
+ return obj_cls(**args)
+
+ @staticmethod
+ def mapper(d, keymap):
+ """Dictionary mapper.
+
+ Renames keys according to keymap provided.
+
+ Args:
+ d (dict): old dict
+ keymap (dict): {'old_key':'new_key'}
+
+ Returns:
+ dict: new dict.
+ """
+
+ updated_dict = {keymap.get(k, k): v for k, v in d.items()}
+ return updated_dict
+
+ def __call__(self, results):
+ # dict to albumentations format
+ results = self.mapper(results, self.keymap_to_albu)
+
+ results = self.aug(**results)
+ # back to the original format
+ results = self.mapper(results, self.keymap_back)
+
+ return results
+
+ def __repr__(self):
+ repr_str = self.__class__.__name__ + f'(transforms={self.transforms})'
+ return repr_str
+
+
+@PIPELINES.register_module()
+class PhotometricDistortion:
+ """Apply photometric distortion to image sequentially, every transformation
+ is applied with a probability of 0.5. The position of random contrast is in
+ second or second to last.
+
+ 1. random brightness
+ 2. random contrast (mode 0)
+ 3. convert color from BGR to HSV
+ 4. random saturation
+ 5. random hue
+ 6. convert color from HSV to BGR
+ 7. random contrast (mode 1)
+ 8. randomly swap channels
+
+ Args:
+ brightness_delta (int): delta of brightness.
+ contrast_range (tuple): range of contrast.
+ saturation_range (tuple): range of saturation.
+ hue_delta (int): delta of hue.
+ """
+
+ def __init__(self,
+ brightness_delta=32,
+ contrast_range=(0.5, 1.5),
+ saturation_range=(0.5, 1.5),
+ hue_delta=18):
+ self.brightness_delta = brightness_delta
+ self.contrast_lower, self.contrast_upper = contrast_range
+ self.saturation_lower, self.saturation_upper = saturation_range
+ self.hue_delta = hue_delta
+
+ def convert(self, img, alpha=1, beta=0):
+ """Multiple with alpha and add beta with clip."""
+ img = img.astype(np.float32) * alpha + beta
+ img = np.clip(img, 0, 255)
+ return img.astype(np.uint8)
+
+ def brightness(self, img):
+ """Brightness distortion."""
+ if random.randint(2):
+ return self.convert(
+ img,
+ beta=random.uniform(-self.brightness_delta,
+ self.brightness_delta))
+ return img
+
+ def contrast(self, img):
+ """Contrast distortion."""
+ if random.randint(2):
+ return self.convert(
+ img,
+ alpha=random.uniform(self.contrast_lower, self.contrast_upper))
+ return img
+
+ def saturation(self, img):
+ # Apply saturation distortion to hsv-formatted img
+ img[:, :, 1] = self.convert(
+ img[:, :, 1],
+ alpha=random.uniform(self.saturation_lower, self.saturation_upper))
+ return img
+
+ def hue(self, img):
+ # Apply hue distortion to hsv-formatted img
+ img[:, :, 0] = (img[:, :, 0].astype(int) +
+ random.randint(-self.hue_delta, self.hue_delta)) % 180
+ return img
+
+ def swap_channels(self, img):
+ # Apply channel swap
+ if random.randint(2):
+ img = img[..., random.permutation(3)]
+ return img
+
+ def __call__(self, results):
+ """Call function to perform photometric distortion on images.
+
+ Args:
+ results (dict): Result dict from loading pipeline.
+
+ Returns:
+ dict: Result dict with images distorted.
+ """
+
+ img = results['img']
+ # random brightness
+ img = self.brightness(img)
+
+ # mode == 0 --> do random contrast first
+ # mode == 1 --> do random contrast last
+ mode = random.randint(2)
+ if mode == 1:
+ img = self.contrast(img)
+
+ hsv_mode = random.randint(4)
+ if hsv_mode:
+ # random saturation/hue distortion
+ img = mmcv.bgr2hsv(img)
+ if hsv_mode == 1 or hsv_mode == 3:
+ img = self.saturation(img)
+ if hsv_mode == 2 or hsv_mode == 3:
+ img = self.hue(img)
+ img = mmcv.hsv2bgr(img)
+
+ # random contrast
+ if mode == 0:
+ img = self.contrast(img)
+
+ # randomly swap channels
+ self.swap_channels(img)
+
+ results['img'] = img
+ return results
+
+ def __repr__(self):
+ repr_str = self.__class__.__name__
+ repr_str += (f'(brightness_delta={self.brightness_delta}, '
+ f'contrast_range=({self.contrast_lower}, '
+ f'{self.contrast_upper}), '
+ f'saturation_range=({self.saturation_lower}, '
+ f'{self.saturation_upper}), '
+ f'hue_delta={self.hue_delta})')
+ return repr_str
+
+
+@PIPELINES.register_module()
+class MultiItemProcess:
+ """Process each item and merge multi-item results to lists.
+
+ Args:
+ pipeline (dict): Dictionary to construct pipeline for a single item.
+ """
+
+ def __init__(self, pipeline):
+ self.pipeline = Compose(pipeline)
+
+ def __call__(self, results):
+ results_ = {}
+ for idx, result in results.items():
+ single_result = self.pipeline(result)
+ for k, v in single_result.items():
+ if k in results_:
+ results_[k].append(v)
+ else:
+ results_[k] = [v]
+
+ return results_
+
+
+@PIPELINES.register_module()
+class DiscardDuplicatedItems:
+
+ def __init__(self, keys_list):
+ """Discard duplicated single-item results.
+
+ Args:
+ keys_list (list): List of keys that need to be deduplicate.
+ """
+ self.keys_list = keys_list
+
+ def __call__(self, results):
+ for k, v in results.items():
+ if k in self.keys_list:
+ assert isinstance(v, Sequence)
+ results[k] = v[0]
+
+ return results
+
+
+@PIPELINES.register_module()
+class MultitaskGatherTarget:
+ """Gather the targets for multitask heads.
+
+ Args:
+ pipeline_list (list[list]): List of pipelines for all heads.
+ pipeline_indices (list[int]): Pipeline index of each head.
+ """
+
+ def __init__(self,
+ pipeline_list,
+ pipeline_indices=None,
+ keys=('target', 'target_weight')):
+ self.keys = keys
+ self.pipelines = []
+ for pipeline in pipeline_list:
+ self.pipelines.append(Compose(pipeline))
+ if pipeline_indices is None:
+ self.pipeline_indices = list(range(len(pipeline_list)))
+ else:
+ self.pipeline_indices = pipeline_indices
+
+ def __call__(self, results):
+ # generate target and target weights using all pipelines
+ pipeline_outputs = []
+ for pipeline in self.pipelines:
+ pipeline_output = pipeline(results)
+ pipeline_outputs.append(pipeline_output.copy())
+
+ for key in self.keys:
+ result_key = []
+ for ind in self.pipeline_indices:
+ result_key.append(pipeline_outputs[ind].get(key, None))
+ results[key] = result_key
+ return results
+
+
+@PIPELINES.register_module()
+class RenameKeys:
+ """Rename the keys.
+
+ Args:
+ key_pairs (Sequence[tuple]): Required keys to be renamed.
+ If a tuple (key_src, key_tgt) is given as an element,
+ the item retrieved by key_src will be renamed as key_tgt.
+ """
+
+ def __init__(self, key_pairs):
+ self.key_pairs = key_pairs
+
+ def __call__(self, results):
+ """Rename keys."""
+ for key_pair in self.key_pairs:
+ assert len(key_pair) == 2
+ key_src, key_tgt = key_pair
+ results[key_tgt] = results.pop(key_src)
+ return results
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/pipelines/top_down_transform.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/pipelines/top_down_transform.py
new file mode 100644
index 0000000..1af1ea9
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/pipelines/top_down_transform.py
@@ -0,0 +1,736 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import cv2
+import numpy as np
+
+from mmpose.core.post_processing import (affine_transform, fliplr_joints,
+ get_affine_transform, get_warp_matrix,
+ warp_affine_joints)
+from mmpose.datasets.builder import PIPELINES
+
+
+@PIPELINES.register_module()
+class TopDownRandomFlip:
+ """Data augmentation with random image flip.
+
+ Required keys: 'img', 'joints_3d', 'joints_3d_visible', 'center' and
+ 'ann_info'.
+
+ Modifies key: 'img', 'joints_3d', 'joints_3d_visible', 'center' and
+ 'flipped'.
+
+ Args:
+ flip (bool): Option to perform random flip.
+ flip_prob (float): Probability of flip.
+ """
+
+ def __init__(self, flip_prob=0.5):
+ self.flip_prob = flip_prob
+
+ def __call__(self, results):
+ """Perform data augmentation with random image flip."""
+ img = results['img']
+ joints_3d = results['joints_3d']
+ joints_3d_visible = results['joints_3d_visible']
+ center = results['center']
+
+ # A flag indicating whether the image is flipped,
+ # which can be used by child class.
+ flipped = False
+ if np.random.rand() <= self.flip_prob:
+ flipped = True
+ if not isinstance(img, list):
+ img = img[:, ::-1, :]
+ else:
+ img = [i[:, ::-1, :] for i in img]
+ if not isinstance(img, list):
+ joints_3d, joints_3d_visible = fliplr_joints(
+ joints_3d, joints_3d_visible, img.shape[1],
+ results['ann_info']['flip_pairs'])
+ center[0] = img.shape[1] - center[0] - 1
+ else:
+ joints_3d, joints_3d_visible = fliplr_joints(
+ joints_3d, joints_3d_visible, img[0].shape[1],
+ results['ann_info']['flip_pairs'])
+ center[0] = img[0].shape[1] - center[0] - 1
+
+ results['img'] = img
+ results['joints_3d'] = joints_3d
+ results['joints_3d_visible'] = joints_3d_visible
+ results['center'] = center
+ results['flipped'] = flipped
+
+ return results
+
+
+@PIPELINES.register_module()
+class TopDownHalfBodyTransform:
+ """Data augmentation with half-body transform. Keep only the upper body or
+ the lower body at random.
+
+ Required keys: 'joints_3d', 'joints_3d_visible', and 'ann_info'.
+
+ Modifies key: 'scale' and 'center'.
+
+ Args:
+ num_joints_half_body (int): Threshold of performing
+ half-body transform. If the body has fewer number
+ of joints (< num_joints_half_body), ignore this step.
+ prob_half_body (float): Probability of half-body transform.
+ """
+
+ def __init__(self, num_joints_half_body=8, prob_half_body=0.3):
+ self.num_joints_half_body = num_joints_half_body
+ self.prob_half_body = prob_half_body
+
+ @staticmethod
+ def half_body_transform(cfg, joints_3d, joints_3d_visible):
+ """Get center&scale for half-body transform."""
+ upper_joints = []
+ lower_joints = []
+ for joint_id in range(cfg['num_joints']):
+ if joints_3d_visible[joint_id][0] > 0:
+ if joint_id in cfg['upper_body_ids']:
+ upper_joints.append(joints_3d[joint_id])
+ else:
+ lower_joints.append(joints_3d[joint_id])
+
+ if np.random.randn() < 0.5 and len(upper_joints) > 2:
+ selected_joints = upper_joints
+ elif len(lower_joints) > 2:
+ selected_joints = lower_joints
+ else:
+ selected_joints = upper_joints
+
+ if len(selected_joints) < 2:
+ return None, None
+
+ selected_joints = np.array(selected_joints, dtype=np.float32)
+ center = selected_joints.mean(axis=0)[:2]
+
+ left_top = np.amin(selected_joints, axis=0)
+
+ right_bottom = np.amax(selected_joints, axis=0)
+
+ w = right_bottom[0] - left_top[0]
+ h = right_bottom[1] - left_top[1]
+
+ aspect_ratio = cfg['image_size'][0] / cfg['image_size'][1]
+
+ if w > aspect_ratio * h:
+ h = w * 1.0 / aspect_ratio
+ elif w < aspect_ratio * h:
+ w = h * aspect_ratio
+
+ scale = np.array([w / 200.0, h / 200.0], dtype=np.float32)
+ scale = scale * 1.5
+ return center, scale
+
+ def __call__(self, results):
+ """Perform data augmentation with half-body transform."""
+ joints_3d = results['joints_3d']
+ joints_3d_visible = results['joints_3d_visible']
+
+ if (np.sum(joints_3d_visible[:, 0]) > self.num_joints_half_body
+ and np.random.rand() < self.prob_half_body):
+
+ c_half_body, s_half_body = self.half_body_transform(
+ results['ann_info'], joints_3d, joints_3d_visible)
+
+ if c_half_body is not None and s_half_body is not None:
+ results['center'] = c_half_body
+ results['scale'] = s_half_body
+
+ return results
+
+
+@PIPELINES.register_module()
+class TopDownGetRandomScaleRotation:
+ """Data augmentation with random scaling & rotating.
+
+ Required key: 'scale'.
+
+ Modifies key: 'scale' and 'rotation'.
+
+ Args:
+ rot_factor (int): Rotating to ``[-2*rot_factor, 2*rot_factor]``.
+ scale_factor (float): Scaling to ``[1-scale_factor, 1+scale_factor]``.
+ rot_prob (float): Probability of random rotation.
+ """
+
+ def __init__(self, rot_factor=40, scale_factor=0.5, rot_prob=0.6):
+ self.rot_factor = rot_factor
+ self.scale_factor = scale_factor
+ self.rot_prob = rot_prob
+
+ def __call__(self, results):
+ """Perform data augmentation with random scaling & rotating."""
+ s = results['scale']
+
+ sf = self.scale_factor
+ rf = self.rot_factor
+
+ s_factor = np.clip(np.random.randn() * sf + 1, 1 - sf, 1 + sf)
+ s = s * s_factor
+
+ r_factor = np.clip(np.random.randn() * rf, -rf * 2, rf * 2)
+ r = r_factor if np.random.rand() <= self.rot_prob else 0
+
+ results['scale'] = s
+ results['rotation'] = r
+
+ return results
+
+
+@PIPELINES.register_module()
+class TopDownAffine:
+ """Affine transform the image to make input.
+
+ Required keys:'img', 'joints_3d', 'joints_3d_visible', 'ann_info','scale',
+ 'rotation' and 'center'.
+
+ Modified keys:'img', 'joints_3d', and 'joints_3d_visible'.
+
+ Args:
+ use_udp (bool): To use unbiased data processing.
+ Paper ref: Huang et al. The Devil is in the Details: Delving into
+ Unbiased Data Processing for Human Pose Estimation (CVPR 2020).
+ """
+
+ def __init__(self, use_udp=False):
+ self.use_udp = use_udp
+
+ def __call__(self, results):
+ image_size = results['ann_info']['image_size']
+
+ img = results['img']
+ joints_3d = results['joints_3d']
+ joints_3d_visible = results['joints_3d_visible']
+ c = results['center']
+ s = results['scale']
+ r = results['rotation']
+
+ if self.use_udp:
+ trans = get_warp_matrix(r, c * 2.0, image_size - 1.0, s * 200.0)
+ if not isinstance(img, list):
+ img = cv2.warpAffine(
+ img,
+ trans, (int(image_size[0]), int(image_size[1])),
+ flags=cv2.INTER_LINEAR)
+ else:
+ img = [
+ cv2.warpAffine(
+ i,
+ trans, (int(image_size[0]), int(image_size[1])),
+ flags=cv2.INTER_LINEAR) for i in img
+ ]
+
+ joints_3d[:, 0:2] = \
+ warp_affine_joints(joints_3d[:, 0:2].copy(), trans)
+
+ else:
+ trans = get_affine_transform(c, s, r, image_size)
+ if not isinstance(img, list):
+ img = cv2.warpAffine(
+ img,
+ trans, (int(image_size[0]), int(image_size[1])),
+ flags=cv2.INTER_LINEAR)
+ else:
+ img = [
+ cv2.warpAffine(
+ i,
+ trans, (int(image_size[0]), int(image_size[1])),
+ flags=cv2.INTER_LINEAR) for i in img
+ ]
+ for i in range(results['ann_info']['num_joints']):
+ if joints_3d_visible[i, 0] > 0.0:
+ joints_3d[i,
+ 0:2] = affine_transform(joints_3d[i, 0:2], trans)
+
+ results['img'] = img
+ results['joints_3d'] = joints_3d
+ results['joints_3d_visible'] = joints_3d_visible
+
+ return results
+
+
+@PIPELINES.register_module()
+class TopDownGenerateTarget:
+ """Generate the target heatmap.
+
+ Required keys: 'joints_3d', 'joints_3d_visible', 'ann_info'.
+
+ Modified keys: 'target', and 'target_weight'.
+
+ Args:
+ sigma: Sigma of heatmap gaussian for 'MSRA' approach.
+ kernel: Kernel of heatmap gaussian for 'Megvii' approach.
+ encoding (str): Approach to generate target heatmaps.
+ Currently supported approaches: 'MSRA', 'Megvii', 'UDP'.
+ Default:'MSRA'
+ unbiased_encoding (bool): Option to use unbiased
+ encoding methods.
+ Paper ref: Zhang et al. Distribution-Aware Coordinate
+ Representation for Human Pose Estimation (CVPR 2020).
+ keypoint_pose_distance: Keypoint pose distance for UDP.
+ Paper ref: Huang et al. The Devil is in the Details: Delving into
+ Unbiased Data Processing for Human Pose Estimation (CVPR 2020).
+ target_type (str): supported targets: 'GaussianHeatmap',
+ 'CombinedTarget'. Default:'GaussianHeatmap'
+ CombinedTarget: The combination of classification target
+ (response map) and regression target (offset map).
+ Paper ref: Huang et al. The Devil is in the Details: Delving into
+ Unbiased Data Processing for Human Pose Estimation (CVPR 2020).
+ """
+
+ def __init__(self,
+ sigma=2,
+ kernel=(11, 11),
+ valid_radius_factor=0.0546875,
+ target_type='GaussianHeatmap',
+ encoding='MSRA',
+ unbiased_encoding=False):
+ self.sigma = sigma
+ self.unbiased_encoding = unbiased_encoding
+ self.kernel = kernel
+ self.valid_radius_factor = valid_radius_factor
+ self.target_type = target_type
+ self.encoding = encoding
+
+ def _msra_generate_target(self, cfg, joints_3d, joints_3d_visible, sigma):
+ """Generate the target heatmap via "MSRA" approach.
+
+ Args:
+ cfg (dict): data config
+ joints_3d: np.ndarray ([num_joints, 3])
+ joints_3d_visible: np.ndarray ([num_joints, 3])
+ sigma: Sigma of heatmap gaussian
+ Returns:
+ tuple: A tuple containing targets.
+
+ - target: Target heatmaps.
+ - target_weight: (1: visible, 0: invisible)
+ """
+ num_joints = cfg['num_joints']
+ image_size = cfg['image_size']
+ W, H = cfg['heatmap_size']
+ joint_weights = cfg['joint_weights']
+ use_different_joint_weights = cfg['use_different_joint_weights']
+
+ target_weight = np.zeros((num_joints, 1), dtype=np.float32)
+ target = np.zeros((num_joints, H, W), dtype=np.float32)
+
+ # 3-sigma rule
+ tmp_size = sigma * 3
+
+ if self.unbiased_encoding:
+ for joint_id in range(num_joints):
+ target_weight[joint_id] = joints_3d_visible[joint_id, 0]
+
+ feat_stride = image_size / [W, H]
+ mu_x = joints_3d[joint_id][0] / feat_stride[0]
+ mu_y = joints_3d[joint_id][1] / feat_stride[1]
+ # Check that any part of the gaussian is in-bounds
+ ul = [mu_x - tmp_size, mu_y - tmp_size]
+ br = [mu_x + tmp_size + 1, mu_y + tmp_size + 1]
+ if ul[0] >= W or ul[1] >= H or br[0] < 0 or br[1] < 0:
+ target_weight[joint_id] = 0
+
+ if target_weight[joint_id] == 0:
+ continue
+
+ x = np.arange(0, W, 1, np.float32)
+ y = np.arange(0, H, 1, np.float32)
+ y = y[:, None]
+
+ if target_weight[joint_id] > 0.5:
+ target[joint_id] = np.exp(-((x - mu_x)**2 +
+ (y - mu_y)**2) /
+ (2 * sigma**2))
+ else:
+ for joint_id in range(num_joints):
+ target_weight[joint_id] = joints_3d_visible[joint_id, 0]
+
+ feat_stride = image_size / [W, H]
+ mu_x = int(joints_3d[joint_id][0] / feat_stride[0] + 0.5)
+ mu_y = int(joints_3d[joint_id][1] / feat_stride[1] + 0.5)
+ # Check that any part of the gaussian is in-bounds
+ ul = [int(mu_x - tmp_size), int(mu_y - tmp_size)]
+ br = [int(mu_x + tmp_size + 1), int(mu_y + tmp_size + 1)]
+ if ul[0] >= W or ul[1] >= H or br[0] < 0 or br[1] < 0:
+ target_weight[joint_id] = 0
+
+ if target_weight[joint_id] > 0.5:
+ size = 2 * tmp_size + 1
+ x = np.arange(0, size, 1, np.float32)
+ y = x[:, None]
+ x0 = y0 = size // 2
+ # The gaussian is not normalized,
+ # we want the center value to equal 1
+ g = np.exp(-((x - x0)**2 + (y - y0)**2) / (2 * sigma**2))
+
+ # Usable gaussian range
+ g_x = max(0, -ul[0]), min(br[0], W) - ul[0]
+ g_y = max(0, -ul[1]), min(br[1], H) - ul[1]
+ # Image range
+ img_x = max(0, ul[0]), min(br[0], W)
+ img_y = max(0, ul[1]), min(br[1], H)
+
+ target[joint_id][img_y[0]:img_y[1], img_x[0]:img_x[1]] = \
+ g[g_y[0]:g_y[1], g_x[0]:g_x[1]]
+
+ if use_different_joint_weights:
+ target_weight = np.multiply(target_weight, joint_weights)
+
+ return target, target_weight
+
+ def _megvii_generate_target(self, cfg, joints_3d, joints_3d_visible,
+ kernel):
+ """Generate the target heatmap via "Megvii" approach.
+
+ Args:
+ cfg (dict): data config
+ joints_3d: np.ndarray ([num_joints, 3])
+ joints_3d_visible: np.ndarray ([num_joints, 3])
+ kernel: Kernel of heatmap gaussian
+
+ Returns:
+ tuple: A tuple containing targets.
+
+ - target: Target heatmaps.
+ - target_weight: (1: visible, 0: invisible)
+ """
+
+ num_joints = cfg['num_joints']
+ image_size = cfg['image_size']
+ W, H = cfg['heatmap_size']
+ heatmaps = np.zeros((num_joints, H, W), dtype='float32')
+ target_weight = np.zeros((num_joints, 1), dtype=np.float32)
+
+ for i in range(num_joints):
+ target_weight[i] = joints_3d_visible[i, 0]
+
+ if target_weight[i] < 1:
+ continue
+
+ target_y = int(joints_3d[i, 1] * H / image_size[1])
+ target_x = int(joints_3d[i, 0] * W / image_size[0])
+
+ if (target_x >= W or target_x < 0) \
+ or (target_y >= H or target_y < 0):
+ target_weight[i] = 0
+ continue
+
+ heatmaps[i, target_y, target_x] = 1
+ heatmaps[i] = cv2.GaussianBlur(heatmaps[i], kernel, 0)
+ maxi = heatmaps[i, target_y, target_x]
+
+ heatmaps[i] /= maxi / 255
+
+ return heatmaps, target_weight
+
+ def _udp_generate_target(self, cfg, joints_3d, joints_3d_visible, factor,
+ target_type):
+ """Generate the target heatmap via 'UDP' approach. Paper ref: Huang et
+ al. The Devil is in the Details: Delving into Unbiased Data Processing
+ for Human Pose Estimation (CVPR 2020).
+
+ Note:
+ - num keypoints: K
+ - heatmap height: H
+ - heatmap width: W
+ - num target channels: C
+ - C = K if target_type=='GaussianHeatmap'
+ - C = 3*K if target_type=='CombinedTarget'
+
+ Args:
+ cfg (dict): data config
+ joints_3d (np.ndarray[K, 3]): Annotated keypoints.
+ joints_3d_visible (np.ndarray[K, 3]): Visibility of keypoints.
+ factor (float): kernel factor for GaussianHeatmap target or
+ valid radius factor for CombinedTarget.
+ target_type (str): 'GaussianHeatmap' or 'CombinedTarget'.
+ GaussianHeatmap: Heatmap target with gaussian distribution.
+ CombinedTarget: The combination of classification target
+ (response map) and regression target (offset map).
+
+ Returns:
+ tuple: A tuple containing targets.
+
+ - target (np.ndarray[C, H, W]): Target heatmaps.
+ - target_weight (np.ndarray[K, 1]): (1: visible, 0: invisible)
+ """
+ num_joints = cfg['num_joints']
+ image_size = cfg['image_size']
+ heatmap_size = cfg['heatmap_size']
+ joint_weights = cfg['joint_weights']
+ use_different_joint_weights = cfg['use_different_joint_weights']
+
+ target_weight = np.ones((num_joints, 1), dtype=np.float32)
+ target_weight[:, 0] = joints_3d_visible[:, 0]
+
+ if target_type.lower() == 'GaussianHeatmap'.lower():
+ target = np.zeros((num_joints, heatmap_size[1], heatmap_size[0]),
+ dtype=np.float32)
+
+ tmp_size = factor * 3
+
+ # prepare for gaussian
+ size = 2 * tmp_size + 1
+ x = np.arange(0, size, 1, np.float32)
+ y = x[:, None]
+
+ for joint_id in range(num_joints):
+ feat_stride = (image_size - 1.0) / (heatmap_size - 1.0)
+ mu_x = int(joints_3d[joint_id][0] / feat_stride[0] + 0.5)
+ mu_y = int(joints_3d[joint_id][1] / feat_stride[1] + 0.5)
+ # Check that any part of the gaussian is in-bounds
+ ul = [int(mu_x - tmp_size), int(mu_y - tmp_size)]
+ br = [int(mu_x + tmp_size + 1), int(mu_y + tmp_size + 1)]
+ if ul[0] >= heatmap_size[0] or ul[1] >= heatmap_size[1] \
+ or br[0] < 0 or br[1] < 0:
+ # If not, just return the image as is
+ target_weight[joint_id] = 0
+ continue
+
+ # # Generate gaussian
+ mu_x_ac = joints_3d[joint_id][0] / feat_stride[0]
+ mu_y_ac = joints_3d[joint_id][1] / feat_stride[1]
+ x0 = y0 = size // 2
+ x0 += mu_x_ac - mu_x
+ y0 += mu_y_ac - mu_y
+ g = np.exp(-((x - x0)**2 + (y - y0)**2) / (2 * factor**2))
+
+ # Usable gaussian range
+ g_x = max(0, -ul[0]), min(br[0], heatmap_size[0]) - ul[0]
+ g_y = max(0, -ul[1]), min(br[1], heatmap_size[1]) - ul[1]
+ # Image range
+ img_x = max(0, ul[0]), min(br[0], heatmap_size[0])
+ img_y = max(0, ul[1]), min(br[1], heatmap_size[1])
+
+ v = target_weight[joint_id]
+ if v > 0.5:
+ target[joint_id][img_y[0]:img_y[1], img_x[0]:img_x[1]] = \
+ g[g_y[0]:g_y[1], g_x[0]:g_x[1]]
+
+ elif target_type.lower() == 'CombinedTarget'.lower():
+ target = np.zeros(
+ (num_joints, 3, heatmap_size[1] * heatmap_size[0]),
+ dtype=np.float32)
+ feat_width = heatmap_size[0]
+ feat_height = heatmap_size[1]
+ feat_x_int = np.arange(0, feat_width)
+ feat_y_int = np.arange(0, feat_height)
+ feat_x_int, feat_y_int = np.meshgrid(feat_x_int, feat_y_int)
+ feat_x_int = feat_x_int.flatten()
+ feat_y_int = feat_y_int.flatten()
+ # Calculate the radius of the positive area in classification
+ # heatmap.
+ valid_radius = factor * heatmap_size[1]
+ feat_stride = (image_size - 1.0) / (heatmap_size - 1.0)
+ for joint_id in range(num_joints):
+ mu_x = joints_3d[joint_id][0] / feat_stride[0]
+ mu_y = joints_3d[joint_id][1] / feat_stride[1]
+ x_offset = (mu_x - feat_x_int) / valid_radius
+ y_offset = (mu_y - feat_y_int) / valid_radius
+ dis = x_offset**2 + y_offset**2
+ keep_pos = np.where(dis <= 1)[0]
+ v = target_weight[joint_id]
+ if v > 0.5:
+ target[joint_id, 0, keep_pos] = 1
+ target[joint_id, 1, keep_pos] = x_offset[keep_pos]
+ target[joint_id, 2, keep_pos] = y_offset[keep_pos]
+ target = target.reshape(num_joints * 3, heatmap_size[1],
+ heatmap_size[0])
+ else:
+ raise ValueError('target_type should be either '
+ "'GaussianHeatmap' or 'CombinedTarget'")
+
+ if use_different_joint_weights:
+ target_weight = np.multiply(target_weight, joint_weights)
+
+ return target, target_weight
+
+ def __call__(self, results):
+ """Generate the target heatmap."""
+ joints_3d = results['joints_3d']
+ joints_3d_visible = results['joints_3d_visible']
+
+ assert self.encoding in ['MSRA', 'Megvii', 'UDP']
+
+ if self.encoding == 'MSRA':
+ if isinstance(self.sigma, list):
+ num_sigmas = len(self.sigma)
+ cfg = results['ann_info']
+ num_joints = cfg['num_joints']
+ heatmap_size = cfg['heatmap_size']
+
+ target = np.empty(
+ (0, num_joints, heatmap_size[1], heatmap_size[0]),
+ dtype=np.float32)
+ target_weight = np.empty((0, num_joints, 1), dtype=np.float32)
+ for i in range(num_sigmas):
+ target_i, target_weight_i = self._msra_generate_target(
+ cfg, joints_3d, joints_3d_visible, self.sigma[i])
+ target = np.concatenate([target, target_i[None]], axis=0)
+ target_weight = np.concatenate(
+ [target_weight, target_weight_i[None]], axis=0)
+ else:
+ target, target_weight = self._msra_generate_target(
+ results['ann_info'], joints_3d, joints_3d_visible,
+ self.sigma)
+
+ elif self.encoding == 'Megvii':
+ if isinstance(self.kernel, list):
+ num_kernels = len(self.kernel)
+ cfg = results['ann_info']
+ num_joints = cfg['num_joints']
+ W, H = cfg['heatmap_size']
+
+ target = np.empty((0, num_joints, H, W), dtype=np.float32)
+ target_weight = np.empty((0, num_joints, 1), dtype=np.float32)
+ for i in range(num_kernels):
+ target_i, target_weight_i = self._megvii_generate_target(
+ cfg, joints_3d, joints_3d_visible, self.kernel[i])
+ target = np.concatenate([target, target_i[None]], axis=0)
+ target_weight = np.concatenate(
+ [target_weight, target_weight_i[None]], axis=0)
+ else:
+ target, target_weight = self._megvii_generate_target(
+ results['ann_info'], joints_3d, joints_3d_visible,
+ self.kernel)
+
+ elif self.encoding == 'UDP':
+ if self.target_type.lower() == 'CombinedTarget'.lower():
+ factors = self.valid_radius_factor
+ channel_factor = 3
+ elif self.target_type.lower() == 'GaussianHeatmap'.lower():
+ factors = self.sigma
+ channel_factor = 1
+ else:
+ raise ValueError('target_type should be either '
+ "'GaussianHeatmap' or 'CombinedTarget'")
+ if isinstance(factors, list):
+ num_factors = len(factors)
+ cfg = results['ann_info']
+ num_joints = cfg['num_joints']
+ W, H = cfg['heatmap_size']
+
+ target = np.empty((0, channel_factor * num_joints, H, W),
+ dtype=np.float32)
+ target_weight = np.empty((0, num_joints, 1), dtype=np.float32)
+ for i in range(num_factors):
+ target_i, target_weight_i = self._udp_generate_target(
+ cfg, joints_3d, joints_3d_visible, factors[i],
+ self.target_type)
+ target = np.concatenate([target, target_i[None]], axis=0)
+ target_weight = np.concatenate(
+ [target_weight, target_weight_i[None]], axis=0)
+ else:
+ target, target_weight = self._udp_generate_target(
+ results['ann_info'], joints_3d, joints_3d_visible, factors,
+ self.target_type)
+ else:
+ raise ValueError(
+ f'Encoding approach {self.encoding} is not supported!')
+
+ if results['ann_info'].get('max_num_joints', None) is not None:
+ W, H = results['ann_info']['heatmap_size']
+ padded_length = int(results['ann_info'].get('max_num_joints') - results['ann_info'].get('num_joints'))
+ target_weight = np.concatenate([target_weight, np.zeros((padded_length, 1), dtype=np.float32)], 0)
+ target = np.concatenate([target, np.zeros((padded_length, H, W), dtype=np.float32)], 0)
+
+ results['target'] = target
+ results['target_weight'] = target_weight
+
+ results['dataset_idx'] = results['ann_info'].get('dataset_idx', 0)
+
+ return results
+
+
+@PIPELINES.register_module()
+class TopDownGenerateTargetRegression:
+ """Generate the target regression vector (coordinates).
+
+ Required keys: 'joints_3d', 'joints_3d_visible', 'ann_info'. Modified keys:
+ 'target', and 'target_weight'.
+ """
+
+ def __init__(self):
+ pass
+
+ def _generate_target(self, cfg, joints_3d, joints_3d_visible):
+ """Generate the target regression vector.
+
+ Args:
+ cfg (dict): data config
+ joints_3d: np.ndarray([num_joints, 3])
+ joints_3d_visible: np.ndarray([num_joints, 3])
+
+ Returns:
+ target, target_weight(1: visible, 0: invisible)
+ """
+ image_size = cfg['image_size']
+ joint_weights = cfg['joint_weights']
+ use_different_joint_weights = cfg['use_different_joint_weights']
+
+ mask = (joints_3d[:, 0] >= 0) * (
+ joints_3d[:, 0] <= image_size[0] - 1) * (joints_3d[:, 1] >= 0) * (
+ joints_3d[:, 1] <= image_size[1] - 1)
+
+ target = joints_3d[:, :2] / image_size
+
+ target = target.astype(np.float32)
+ target_weight = joints_3d_visible[:, :2] * mask[:, None]
+
+ if use_different_joint_weights:
+ target_weight = np.multiply(target_weight, joint_weights)
+
+ return target, target_weight
+
+ def __call__(self, results):
+ """Generate the target heatmap."""
+ joints_3d = results['joints_3d']
+ joints_3d_visible = results['joints_3d_visible']
+
+ target, target_weight = self._generate_target(results['ann_info'],
+ joints_3d,
+ joints_3d_visible)
+
+ results['target'] = target
+ results['target_weight'] = target_weight
+
+ return results
+
+
+@PIPELINES.register_module()
+class TopDownRandomTranslation:
+ """Data augmentation with random translation.
+
+ Required key: 'scale' and 'center'.
+
+ Modifies key: 'center'.
+
+ Note:
+ - bbox height: H
+ - bbox width: W
+
+ Args:
+ trans_factor (float): Translating center to
+ ``[-trans_factor, trans_factor] * [W, H] + center``.
+ trans_prob (float): Probability of random translation.
+ """
+
+ def __init__(self, trans_factor=0.15, trans_prob=1.0):
+ self.trans_factor = trans_factor
+ self.trans_prob = trans_prob
+
+ def __call__(self, results):
+ """Perform data augmentation with random translation."""
+ center = results['center']
+ scale = results['scale']
+ if np.random.rand() <= self.trans_prob:
+ # reference bbox size is [200, 200] pixels
+ center += self.trans_factor * np.random.uniform(
+ -1, 1, size=2) * scale * 200
+ results['center'] = center
+ return results
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/registry.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/registry.py
new file mode 100644
index 0000000..ba3cc49
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/registry.py
@@ -0,0 +1,13 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import warnings
+
+from .builder import DATASETS, PIPELINES
+
+__all__ = ['DATASETS', 'PIPELINES']
+
+warnings.simplefilter('once', DeprecationWarning)
+warnings.warn(
+ 'Registries (DATASETS, PIPELINES) have been moved to '
+ 'mmpose.datasets.builder. Importing from '
+ 'mmpose.models.registry will be deprecated in the future.',
+ DeprecationWarning)
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/samplers/__init__.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/samplers/__init__.py
new file mode 100644
index 0000000..da09eff
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/samplers/__init__.py
@@ -0,0 +1,4 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from .distributed_sampler import DistributedSampler
+
+__all__ = ['DistributedSampler']
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/samplers/distributed_sampler.py b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/samplers/distributed_sampler.py
new file mode 100644
index 0000000..bcb5f52
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/datasets/samplers/distributed_sampler.py
@@ -0,0 +1,41 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import torch
+from torch.utils.data import DistributedSampler as _DistributedSampler
+
+
+class DistributedSampler(_DistributedSampler):
+ """DistributedSampler inheriting from
+ `torch.utils.data.DistributedSampler`.
+
+ In pytorch of lower versions, there is no `shuffle` argument. This child
+ class will port one to DistributedSampler.
+ """
+
+ def __init__(self,
+ dataset,
+ num_replicas=None,
+ rank=None,
+ shuffle=True,
+ seed=0):
+ super().__init__(
+ dataset, num_replicas=num_replicas, rank=rank, shuffle=shuffle)
+ # for the compatibility from PyTorch 1.3+
+ self.seed = seed if seed is not None else 0
+
+ def __iter__(self):
+ """Deterministically shuffle based on epoch."""
+ if self.shuffle:
+ g = torch.Generator()
+ g.manual_seed(self.epoch + self.seed)
+ indices = torch.randperm(len(self.dataset), generator=g).tolist()
+ else:
+ indices = torch.arange(len(self.dataset)).tolist()
+
+ # add extra samples to make it evenly divisible
+ indices += indices[:(self.total_size - len(indices))]
+ assert len(indices) == self.total_size
+
+ # subsample
+ indices = indices[self.rank:self.total_size:self.num_replicas]
+ assert len(indices) == self.num_samples
+ return iter(indices)
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/deprecated.py b/engine/pose_estimation/third-party/ViTPose/mmpose/deprecated.py
new file mode 100644
index 0000000..b930901
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/deprecated.py
@@ -0,0 +1,199 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import warnings
+
+from .datasets.builder import DATASETS
+from .datasets.datasets.base import Kpt2dSviewRgbImgTopDownDataset
+from .models.builder import HEADS, POSENETS
+from .models.detectors import AssociativeEmbedding
+from .models.heads import (AEHigherResolutionHead, AESimpleHead,
+ DeepposeRegressionHead, HMRMeshHead,
+ TopdownHeatmapMSMUHead,
+ TopdownHeatmapMultiStageHead,
+ TopdownHeatmapSimpleHead)
+
+
+@DATASETS.register_module()
+class TopDownFreiHandDataset(Kpt2dSviewRgbImgTopDownDataset):
+ """Deprecated TopDownFreiHandDataset."""
+
+ def __init__(self, *args, **kwargs):
+ raise (ImportError(
+ 'TopDownFreiHandDataset has been renamed into FreiHandDataset,'
+ 'check https://github.com/open-mmlab/mmpose/pull/202 for details.')
+ )
+
+ def _get_db(self):
+ return []
+
+ def evaluate(self, cfg, preds, output_dir, *args, **kwargs):
+ return None
+
+
+@DATASETS.register_module()
+class TopDownOneHand10KDataset(Kpt2dSviewRgbImgTopDownDataset):
+ """Deprecated TopDownOneHand10KDataset."""
+
+ def __init__(self, *args, **kwargs):
+ raise (ImportError(
+ 'TopDownOneHand10KDataset has been renamed into OneHand10KDataset,'
+ 'check https://github.com/open-mmlab/mmpose/pull/202 for details.')
+ )
+
+ def _get_db(self):
+ return []
+
+ def evaluate(self, cfg, preds, output_dir, *args, **kwargs):
+ return None
+
+
+@DATASETS.register_module()
+class TopDownPanopticDataset(Kpt2dSviewRgbImgTopDownDataset):
+ """Deprecated TopDownPanopticDataset."""
+
+ def __init__(self, *args, **kwargs):
+ raise (ImportError(
+ 'TopDownPanopticDataset has been renamed into PanopticDataset,'
+ 'check https://github.com/open-mmlab/mmpose/pull/202 for details.')
+ )
+
+ def _get_db(self):
+ return []
+
+ def evaluate(self, cfg, preds, output_dir, *args, **kwargs):
+ return None
+
+
+@HEADS.register_module()
+class BottomUpHigherResolutionHead(AEHigherResolutionHead):
+ """Bottom-up head for Higher Resolution.
+
+ BottomUpHigherResolutionHead has been renamed into AEHigherResolutionHead,
+ check https://github.com/open- mmlab/mmpose/pull/656 for details.
+ """
+
+ def __init__(self, *args, **kwargs):
+ super().__init__(*args, **kwargs)
+ warnings.warn(
+ 'BottomUpHigherResolutionHead has been renamed into '
+ 'AEHigherResolutionHead, check '
+ 'https://github.com/open-mmlab/mmpose/pull/656 for details.',
+ DeprecationWarning)
+
+
+@HEADS.register_module()
+class BottomUpSimpleHead(AESimpleHead):
+ """Bottom-up simple head.
+
+ BottomUpSimpleHead has been renamed into AESimpleHead, check
+ https://github.com/open-mmlab/mmpose/pull/656 for details.
+ """
+
+ def __init__(self, *args, **kwargs):
+ super().__init__(*args, **kwargs)
+ warnings.warn(
+ 'BottomUpHigherResolutionHead has been renamed into '
+ 'AEHigherResolutionHead, check '
+ 'https://github.com/open-mmlab/mmpose/pull/656 for details',
+ DeprecationWarning)
+
+
+@HEADS.register_module()
+class TopDownSimpleHead(TopdownHeatmapSimpleHead):
+ """Top-down heatmap simple head.
+
+ TopDownSimpleHead has been renamed into TopdownHeatmapSimpleHead, check
+ https://github.com/open-mmlab/mmpose/pull/656 for details.
+ """
+
+ def __init__(self, *args, **kwargs):
+ super().__init__(*args, **kwargs)
+ warnings.warn(
+ 'TopDownSimpleHead has been renamed into '
+ 'TopdownHeatmapSimpleHead, check '
+ 'https://github.com/open-mmlab/mmpose/pull/656 for details.',
+ DeprecationWarning)
+
+
+@HEADS.register_module()
+class TopDownMultiStageHead(TopdownHeatmapMultiStageHead):
+ """Top-down heatmap multi-stage head.
+
+ TopDownMultiStageHead has been renamed into TopdownHeatmapMultiStageHead,
+ check https://github.com/open-mmlab/mmpose/pull/656 for details.
+ """
+
+ def __init__(self, *args, **kwargs):
+ super().__init__(*args, **kwargs)
+ warnings.warn(
+ 'TopDownMultiStageHead has been renamed into '
+ 'TopdownHeatmapMultiStageHead, check '
+ 'https://github.com/open-mmlab/mmpose/pull/656 for details.',
+ DeprecationWarning)
+
+
+@HEADS.register_module()
+class TopDownMSMUHead(TopdownHeatmapMSMUHead):
+ """Heads for multi-stage multi-unit heads.
+
+ TopDownMSMUHead has been renamed into TopdownHeatmapMSMUHead, check
+ https://github.com/open-mmlab/mmpose/pull/656 for details.
+ """
+
+ def __init__(self, *args, **kwargs):
+ super().__init__(*args, **kwargs)
+ warnings.warn(
+ 'TopDownMSMUHead has been renamed into '
+ 'TopdownHeatmapMSMUHead, check '
+ 'https://github.com/open-mmlab/mmpose/pull/656 for details.',
+ DeprecationWarning)
+
+
+@HEADS.register_module()
+class MeshHMRHead(HMRMeshHead):
+ """SMPL parameters regressor head.
+
+ MeshHMRHead has been renamed into HMRMeshHead, check
+ https://github.com/open-mmlab/mmpose/pull/656 for details.
+ """
+
+ def __init__(self, *args, **kwargs):
+ super().__init__(*args, **kwargs)
+ warnings.warn(
+ 'MeshHMRHead has been renamed into '
+ 'HMRMeshHead, check '
+ 'https://github.com/open-mmlab/mmpose/pull/656 for details.',
+ DeprecationWarning)
+
+
+@HEADS.register_module()
+class FcHead(DeepposeRegressionHead):
+ """FcHead (deprecated).
+
+ FcHead has been renamed into DeepposeRegressionHead, check
+ https://github.com/open-mmlab/mmpose/pull/656 for details.
+ """
+
+ def __init__(self, *args, **kwargs):
+ super().__init__(*args, **kwargs)
+ warnings.warn(
+ 'FcHead has been renamed into '
+ 'DeepposeRegressionHead, check '
+ 'https://github.com/open-mmlab/mmpose/pull/656 for details.',
+ DeprecationWarning)
+
+
+@POSENETS.register_module()
+class BottomUp(AssociativeEmbedding):
+ """Associative Embedding.
+
+ BottomUp has been renamed into AssociativeEmbedding, check
+ https://github.com/open-mmlab/mmpose/pull/656 for details.
+ """
+
+ def __init__(self, *args, **kwargs):
+ super().__init__(*args, **kwargs)
+ warnings.warn(
+ 'BottomUp has been renamed into '
+ 'AssociativeEmbedding, check '
+ 'https://github.com/open-mmlab/mmpose/pull/656 for details.',
+ DeprecationWarning)
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/__init__.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/__init__.py
new file mode 100644
index 0000000..dbec55e
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/__init__.py
@@ -0,0 +1,16 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from .backbones import * # noqa
+from .builder import (BACKBONES, HEADS, LOSSES, MESH_MODELS, NECKS, POSENETS,
+ build_backbone, build_head, build_loss, build_mesh_model,
+ build_neck, build_posenet)
+from .detectors import * # noqa
+from .heads import * # noqa
+from .losses import * # noqa
+from .necks import * # noqa
+from .utils import * # noqa
+
+__all__ = [
+ 'BACKBONES', 'HEADS', 'NECKS', 'LOSSES', 'POSENETS', 'MESH_MODELS',
+ 'build_backbone', 'build_head', 'build_loss', 'build_posenet',
+ 'build_neck', 'build_mesh_model'
+]
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/__init__.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/__init__.py
new file mode 100644
index 0000000..2b8efcf
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/__init__.py
@@ -0,0 +1,36 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from .alexnet import AlexNet
+from .cpm import CPM
+from .hourglass import HourglassNet
+from .hourglass_ae import HourglassAENet
+from .hrformer import HRFormer
+from .hrnet import HRNet
+from .litehrnet import LiteHRNet
+from .mobilenet_v2 import MobileNetV2
+from .mobilenet_v3 import MobileNetV3
+from .mspn import MSPN
+from .regnet import RegNet
+from .resnest import ResNeSt
+from .resnet import ResNet, ResNetV1d
+from .resnext import ResNeXt
+from .rsn import RSN
+from .scnet import SCNet
+from .seresnet import SEResNet
+from .seresnext import SEResNeXt
+from .shufflenet_v1 import ShuffleNetV1
+from .shufflenet_v2 import ShuffleNetV2
+from .tcn import TCN
+from .v2v_net import V2VNet
+from .vgg import VGG
+from .vipnas_mbv3 import ViPNAS_MobileNetV3
+from .vipnas_resnet import ViPNAS_ResNet
+from .vit import ViT
+from .vit_moe import ViTMoE
+
+__all__ = [
+ 'AlexNet', 'HourglassNet', 'HourglassAENet', 'HRNet', 'MobileNetV2',
+ 'MobileNetV3', 'RegNet', 'ResNet', 'ResNetV1d', 'ResNeXt', 'SCNet',
+ 'SEResNet', 'SEResNeXt', 'ShuffleNetV1', 'ShuffleNetV2', 'CPM', 'RSN',
+ 'MSPN', 'ResNeSt', 'VGG', 'TCN', 'ViPNAS_ResNet', 'ViPNAS_MobileNetV3',
+ 'LiteHRNet', 'V2VNet', 'HRFormer', 'ViT', 'ViTMoE'
+]
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/alexnet.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/alexnet.py
new file mode 100644
index 0000000..a8efd74
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/alexnet.py
@@ -0,0 +1,56 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import torch.nn as nn
+
+from ..builder import BACKBONES
+from .base_backbone import BaseBackbone
+
+
+@BACKBONES.register_module()
+class AlexNet(BaseBackbone):
+ """`AlexNet `__ backbone.
+
+ The input for AlexNet is a 224x224 RGB image.
+
+ Args:
+ num_classes (int): number of classes for classification.
+ The default value is -1, which uses the backbone as
+ a feature extractor without the top classifier.
+ """
+
+ def __init__(self, num_classes=-1):
+ super().__init__()
+ self.num_classes = num_classes
+ self.features = nn.Sequential(
+ nn.Conv2d(3, 64, kernel_size=11, stride=4, padding=2),
+ nn.ReLU(inplace=True),
+ nn.MaxPool2d(kernel_size=3, stride=2),
+ nn.Conv2d(64, 192, kernel_size=5, padding=2),
+ nn.ReLU(inplace=True),
+ nn.MaxPool2d(kernel_size=3, stride=2),
+ nn.Conv2d(192, 384, kernel_size=3, padding=1),
+ nn.ReLU(inplace=True),
+ nn.Conv2d(384, 256, kernel_size=3, padding=1),
+ nn.ReLU(inplace=True),
+ nn.Conv2d(256, 256, kernel_size=3, padding=1),
+ nn.ReLU(inplace=True),
+ nn.MaxPool2d(kernel_size=3, stride=2),
+ )
+ if self.num_classes > 0:
+ self.classifier = nn.Sequential(
+ nn.Dropout(),
+ nn.Linear(256 * 6 * 6, 4096),
+ nn.ReLU(inplace=True),
+ nn.Dropout(),
+ nn.Linear(4096, 4096),
+ nn.ReLU(inplace=True),
+ nn.Linear(4096, num_classes),
+ )
+
+ def forward(self, x):
+
+ x = self.features(x)
+ if self.num_classes > 0:
+ x = x.view(x.size(0), 256 * 6 * 6)
+ x = self.classifier(x)
+
+ return x
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/base_backbone.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/base_backbone.py
new file mode 100644
index 0000000..d64dca1
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/base_backbone.py
@@ -0,0 +1,43 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import logging
+from abc import ABCMeta, abstractmethod
+
+import torch.nn as nn
+
+# from .utils import load_checkpoint
+from mmcv_custom.checkpoint import load_checkpoint
+
+class BaseBackbone(nn.Module, metaclass=ABCMeta):
+ """Base backbone.
+
+ This class defines the basic functions of a backbone. Any backbone that
+ inherits this class should at least define its own `forward` function.
+ """
+
+ def init_weights(self, pretrained=None, patch_padding='pad', part_features=None):
+ """Init backbone weights.
+
+ Args:
+ pretrained (str | None): If pretrained is a string, then it
+ initializes backbone weights by loading the pretrained
+ checkpoint. If pretrained is None, then it follows default
+ initializer or customized initializer in subclasses.
+ """
+ if isinstance(pretrained, str):
+ logger = logging.getLogger()
+ load_checkpoint(self, pretrained, strict=False, logger=logger, patch_padding=patch_padding, part_features=part_features)
+ elif pretrained is None:
+ # use default initializer or customized initializer in subclasses
+ pass
+ else:
+ raise TypeError('pretrained must be a str or None.'
+ f' But received {type(pretrained)}.')
+
+ @abstractmethod
+ def forward(self, x):
+ """Forward function.
+
+ Args:
+ x (Tensor | tuple[Tensor]): x could be a torch.Tensor or a tuple of
+ torch.Tensor, containing input data for forward computation.
+ """
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/cpm.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/cpm.py
new file mode 100644
index 0000000..458245d
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/cpm.py
@@ -0,0 +1,186 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import copy
+
+import torch
+import torch.nn as nn
+from mmcv.cnn import ConvModule, constant_init, normal_init
+from torch.nn.modules.batchnorm import _BatchNorm
+
+from mmpose.utils import get_root_logger
+from ..builder import BACKBONES
+from .base_backbone import BaseBackbone
+from .utils import load_checkpoint
+
+
+class CpmBlock(nn.Module):
+ """CpmBlock for Convolutional Pose Machine.
+
+ Args:
+ in_channels (int): Input channels of this block.
+ channels (list): Output channels of each conv module.
+ kernels (list): Kernel sizes of each conv module.
+ """
+
+ def __init__(self,
+ in_channels,
+ channels=(128, 128, 128),
+ kernels=(11, 11, 11),
+ norm_cfg=None):
+ super().__init__()
+
+ assert len(channels) == len(kernels)
+ layers = []
+ for i in range(len(channels)):
+ if i == 0:
+ input_channels = in_channels
+ else:
+ input_channels = channels[i - 1]
+ layers.append(
+ ConvModule(
+ input_channels,
+ channels[i],
+ kernels[i],
+ padding=(kernels[i] - 1) // 2,
+ norm_cfg=norm_cfg))
+ self.model = nn.Sequential(*layers)
+
+ def forward(self, x):
+ """Model forward function."""
+ out = self.model(x)
+ return out
+
+
+@BACKBONES.register_module()
+class CPM(BaseBackbone):
+ """CPM backbone.
+
+ Convolutional Pose Machines.
+ More details can be found in the `paper
+ `__ .
+
+ Args:
+ in_channels (int): The input channels of the CPM.
+ out_channels (int): The output channels of the CPM.
+ feat_channels (int): Feature channel of each CPM stage.
+ middle_channels (int): Feature channel of conv after the middle stage.
+ num_stages (int): Number of stages.
+ norm_cfg (dict): Dictionary to construct and config norm layer.
+
+ Example:
+ >>> from mmpose.models import CPM
+ >>> import torch
+ >>> self = CPM(3, 17)
+ >>> self.eval()
+ >>> inputs = torch.rand(1, 3, 368, 368)
+ >>> level_outputs = self.forward(inputs)
+ >>> for level_output in level_outputs:
+ ... print(tuple(level_output.shape))
+ (1, 17, 46, 46)
+ (1, 17, 46, 46)
+ (1, 17, 46, 46)
+ (1, 17, 46, 46)
+ (1, 17, 46, 46)
+ (1, 17, 46, 46)
+ """
+
+ def __init__(self,
+ in_channels,
+ out_channels,
+ feat_channels=128,
+ middle_channels=32,
+ num_stages=6,
+ norm_cfg=dict(type='BN', requires_grad=True)):
+ # Protect mutable default arguments
+ norm_cfg = copy.deepcopy(norm_cfg)
+ super().__init__()
+
+ assert in_channels == 3
+
+ self.num_stages = num_stages
+ assert self.num_stages >= 1
+
+ self.stem = nn.Sequential(
+ ConvModule(in_channels, 128, 9, padding=4, norm_cfg=norm_cfg),
+ nn.MaxPool2d(kernel_size=3, stride=2, padding=1),
+ ConvModule(128, 128, 9, padding=4, norm_cfg=norm_cfg),
+ nn.MaxPool2d(kernel_size=3, stride=2, padding=1),
+ ConvModule(128, 128, 9, padding=4, norm_cfg=norm_cfg),
+ nn.MaxPool2d(kernel_size=3, stride=2, padding=1),
+ ConvModule(128, 32, 5, padding=2, norm_cfg=norm_cfg),
+ ConvModule(32, 512, 9, padding=4, norm_cfg=norm_cfg),
+ ConvModule(512, 512, 1, padding=0, norm_cfg=norm_cfg),
+ ConvModule(512, out_channels, 1, padding=0, act_cfg=None))
+
+ self.middle = nn.Sequential(
+ ConvModule(in_channels, 128, 9, padding=4, norm_cfg=norm_cfg),
+ nn.MaxPool2d(kernel_size=3, stride=2, padding=1),
+ ConvModule(128, 128, 9, padding=4, norm_cfg=norm_cfg),
+ nn.MaxPool2d(kernel_size=3, stride=2, padding=1),
+ ConvModule(128, 128, 9, padding=4, norm_cfg=norm_cfg),
+ nn.MaxPool2d(kernel_size=3, stride=2, padding=1))
+
+ self.cpm_stages = nn.ModuleList([
+ CpmBlock(
+ middle_channels + out_channels,
+ channels=[feat_channels, feat_channels, feat_channels],
+ kernels=[11, 11, 11],
+ norm_cfg=norm_cfg) for _ in range(num_stages - 1)
+ ])
+
+ self.middle_conv = nn.ModuleList([
+ nn.Sequential(
+ ConvModule(
+ 128, middle_channels, 5, padding=2, norm_cfg=norm_cfg))
+ for _ in range(num_stages - 1)
+ ])
+
+ self.out_convs = nn.ModuleList([
+ nn.Sequential(
+ ConvModule(
+ feat_channels,
+ feat_channels,
+ 1,
+ padding=0,
+ norm_cfg=norm_cfg),
+ ConvModule(feat_channels, out_channels, 1, act_cfg=None))
+ for _ in range(num_stages - 1)
+ ])
+
+ def init_weights(self, pretrained=None):
+ """Initialize the weights in backbone.
+
+ Args:
+ pretrained (str, optional): Path to pre-trained weights.
+ Defaults to None.
+ """
+ if isinstance(pretrained, str):
+ logger = get_root_logger()
+ load_checkpoint(self, pretrained, strict=False, logger=logger)
+ elif pretrained is None:
+ for m in self.modules():
+ if isinstance(m, nn.Conv2d):
+ normal_init(m, std=0.001)
+ elif isinstance(m, (_BatchNorm, nn.GroupNorm)):
+ constant_init(m, 1)
+ else:
+ raise TypeError('pretrained must be a str or None')
+
+ def forward(self, x):
+ """Model forward function."""
+ stage1_out = self.stem(x)
+ middle_out = self.middle(x)
+ out_feats = []
+
+ out_feats.append(stage1_out)
+
+ for ind in range(self.num_stages - 1):
+ single_stage = self.cpm_stages[ind]
+ out_conv = self.out_convs[ind]
+
+ inp_feat = torch.cat(
+ [out_feats[-1], self.middle_conv[ind](middle_out)], 1)
+ cpm_feat = single_stage(inp_feat)
+ out_feat = out_conv(cpm_feat)
+ out_feats.append(out_feat)
+
+ return out_feats
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/hourglass.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/hourglass.py
new file mode 100644
index 0000000..bf75fad
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/hourglass.py
@@ -0,0 +1,212 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import copy
+
+import torch.nn as nn
+from mmcv.cnn import ConvModule, constant_init, normal_init
+from torch.nn.modules.batchnorm import _BatchNorm
+
+from mmpose.utils import get_root_logger
+from ..builder import BACKBONES
+from .base_backbone import BaseBackbone
+from .resnet import BasicBlock, ResLayer
+from .utils import load_checkpoint
+
+
+class HourglassModule(nn.Module):
+ """Hourglass Module for HourglassNet backbone.
+
+ Generate module recursively and use BasicBlock as the base unit.
+
+ Args:
+ depth (int): Depth of current HourglassModule.
+ stage_channels (list[int]): Feature channels of sub-modules in current
+ and follow-up HourglassModule.
+ stage_blocks (list[int]): Number of sub-modules stacked in current and
+ follow-up HourglassModule.
+ norm_cfg (dict): Dictionary to construct and config norm layer.
+ """
+
+ def __init__(self,
+ depth,
+ stage_channels,
+ stage_blocks,
+ norm_cfg=dict(type='BN', requires_grad=True)):
+ # Protect mutable default arguments
+ norm_cfg = copy.deepcopy(norm_cfg)
+ super().__init__()
+
+ self.depth = depth
+
+ cur_block = stage_blocks[0]
+ next_block = stage_blocks[1]
+
+ cur_channel = stage_channels[0]
+ next_channel = stage_channels[1]
+
+ self.up1 = ResLayer(
+ BasicBlock, cur_block, cur_channel, cur_channel, norm_cfg=norm_cfg)
+
+ self.low1 = ResLayer(
+ BasicBlock,
+ cur_block,
+ cur_channel,
+ next_channel,
+ stride=2,
+ norm_cfg=norm_cfg)
+
+ if self.depth > 1:
+ self.low2 = HourglassModule(depth - 1, stage_channels[1:],
+ stage_blocks[1:])
+ else:
+ self.low2 = ResLayer(
+ BasicBlock,
+ next_block,
+ next_channel,
+ next_channel,
+ norm_cfg=norm_cfg)
+
+ self.low3 = ResLayer(
+ BasicBlock,
+ cur_block,
+ next_channel,
+ cur_channel,
+ norm_cfg=norm_cfg,
+ downsample_first=False)
+
+ self.up2 = nn.Upsample(scale_factor=2)
+
+ def forward(self, x):
+ """Model forward function."""
+ up1 = self.up1(x)
+ low1 = self.low1(x)
+ low2 = self.low2(low1)
+ low3 = self.low3(low2)
+ up2 = self.up2(low3)
+ return up1 + up2
+
+
+@BACKBONES.register_module()
+class HourglassNet(BaseBackbone):
+ """HourglassNet backbone.
+
+ Stacked Hourglass Networks for Human Pose Estimation.
+ More details can be found in the `paper
+ `__ .
+
+ Args:
+ downsample_times (int): Downsample times in a HourglassModule.
+ num_stacks (int): Number of HourglassModule modules stacked,
+ 1 for Hourglass-52, 2 for Hourglass-104.
+ stage_channels (list[int]): Feature channel of each sub-module in a
+ HourglassModule.
+ stage_blocks (list[int]): Number of sub-modules stacked in a
+ HourglassModule.
+ feat_channel (int): Feature channel of conv after a HourglassModule.
+ norm_cfg (dict): Dictionary to construct and config norm layer.
+
+ Example:
+ >>> from mmpose.models import HourglassNet
+ >>> import torch
+ >>> self = HourglassNet()
+ >>> self.eval()
+ >>> inputs = torch.rand(1, 3, 511, 511)
+ >>> level_outputs = self.forward(inputs)
+ >>> for level_output in level_outputs:
+ ... print(tuple(level_output.shape))
+ (1, 256, 128, 128)
+ (1, 256, 128, 128)
+ """
+
+ def __init__(self,
+ downsample_times=5,
+ num_stacks=2,
+ stage_channels=(256, 256, 384, 384, 384, 512),
+ stage_blocks=(2, 2, 2, 2, 2, 4),
+ feat_channel=256,
+ norm_cfg=dict(type='BN', requires_grad=True)):
+ # Protect mutable default arguments
+ norm_cfg = copy.deepcopy(norm_cfg)
+ super().__init__()
+
+ self.num_stacks = num_stacks
+ assert self.num_stacks >= 1
+ assert len(stage_channels) == len(stage_blocks)
+ assert len(stage_channels) > downsample_times
+
+ cur_channel = stage_channels[0]
+
+ self.stem = nn.Sequential(
+ ConvModule(3, 128, 7, padding=3, stride=2, norm_cfg=norm_cfg),
+ ResLayer(BasicBlock, 1, 128, 256, stride=2, norm_cfg=norm_cfg))
+
+ self.hourglass_modules = nn.ModuleList([
+ HourglassModule(downsample_times, stage_channels, stage_blocks)
+ for _ in range(num_stacks)
+ ])
+
+ self.inters = ResLayer(
+ BasicBlock,
+ num_stacks - 1,
+ cur_channel,
+ cur_channel,
+ norm_cfg=norm_cfg)
+
+ self.conv1x1s = nn.ModuleList([
+ ConvModule(
+ cur_channel, cur_channel, 1, norm_cfg=norm_cfg, act_cfg=None)
+ for _ in range(num_stacks - 1)
+ ])
+
+ self.out_convs = nn.ModuleList([
+ ConvModule(
+ cur_channel, feat_channel, 3, padding=1, norm_cfg=norm_cfg)
+ for _ in range(num_stacks)
+ ])
+
+ self.remap_convs = nn.ModuleList([
+ ConvModule(
+ feat_channel, cur_channel, 1, norm_cfg=norm_cfg, act_cfg=None)
+ for _ in range(num_stacks - 1)
+ ])
+
+ self.relu = nn.ReLU(inplace=True)
+
+ def init_weights(self, pretrained=None):
+ """Initialize the weights in backbone.
+
+ Args:
+ pretrained (str, optional): Path to pre-trained weights.
+ Defaults to None.
+ """
+ if isinstance(pretrained, str):
+ logger = get_root_logger()
+ load_checkpoint(self, pretrained, strict=False, logger=logger)
+ elif pretrained is None:
+ for m in self.modules():
+ if isinstance(m, nn.Conv2d):
+ normal_init(m, std=0.001)
+ elif isinstance(m, (_BatchNorm, nn.GroupNorm)):
+ constant_init(m, 1)
+ else:
+ raise TypeError('pretrained must be a str or None')
+
+ def forward(self, x):
+ """Model forward function."""
+ inter_feat = self.stem(x)
+ out_feats = []
+
+ for ind in range(self.num_stacks):
+ single_hourglass = self.hourglass_modules[ind]
+ out_conv = self.out_convs[ind]
+
+ hourglass_feat = single_hourglass(inter_feat)
+ out_feat = out_conv(hourglass_feat)
+ out_feats.append(out_feat)
+
+ if ind < self.num_stacks - 1:
+ inter_feat = self.conv1x1s[ind](
+ inter_feat) + self.remap_convs[ind](
+ out_feat)
+ inter_feat = self.inters[ind](self.relu(inter_feat))
+
+ return out_feats
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/hourglass_ae.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/hourglass_ae.py
new file mode 100644
index 0000000..5a700e5
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/hourglass_ae.py
@@ -0,0 +1,212 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import copy
+
+import torch.nn as nn
+from mmcv.cnn import ConvModule, MaxPool2d, constant_init, normal_init
+from torch.nn.modules.batchnorm import _BatchNorm
+
+from mmpose.utils import get_root_logger
+from ..builder import BACKBONES
+from .base_backbone import BaseBackbone
+from .utils import load_checkpoint
+
+
+class HourglassAEModule(nn.Module):
+ """Modified Hourglass Module for HourglassNet_AE backbone.
+
+ Generate module recursively and use BasicBlock as the base unit.
+
+ Args:
+ depth (int): Depth of current HourglassModule.
+ stage_channels (list[int]): Feature channels of sub-modules in current
+ and follow-up HourglassModule.
+ norm_cfg (dict): Dictionary to construct and config norm layer.
+ """
+
+ def __init__(self,
+ depth,
+ stage_channels,
+ norm_cfg=dict(type='BN', requires_grad=True)):
+ # Protect mutable default arguments
+ norm_cfg = copy.deepcopy(norm_cfg)
+ super().__init__()
+
+ self.depth = depth
+
+ cur_channel = stage_channels[0]
+ next_channel = stage_channels[1]
+
+ self.up1 = ConvModule(
+ cur_channel, cur_channel, 3, padding=1, norm_cfg=norm_cfg)
+
+ self.pool1 = MaxPool2d(2, 2)
+
+ self.low1 = ConvModule(
+ cur_channel, next_channel, 3, padding=1, norm_cfg=norm_cfg)
+
+ if self.depth > 1:
+ self.low2 = HourglassAEModule(depth - 1, stage_channels[1:])
+ else:
+ self.low2 = ConvModule(
+ next_channel, next_channel, 3, padding=1, norm_cfg=norm_cfg)
+
+ self.low3 = ConvModule(
+ next_channel, cur_channel, 3, padding=1, norm_cfg=norm_cfg)
+
+ self.up2 = nn.UpsamplingNearest2d(scale_factor=2)
+
+ def forward(self, x):
+ """Model forward function."""
+ up1 = self.up1(x)
+ pool1 = self.pool1(x)
+ low1 = self.low1(pool1)
+ low2 = self.low2(low1)
+ low3 = self.low3(low2)
+ up2 = self.up2(low3)
+ return up1 + up2
+
+
+@BACKBONES.register_module()
+class HourglassAENet(BaseBackbone):
+ """Hourglass-AE Network proposed by Newell et al.
+
+ Associative Embedding: End-to-End Learning for Joint
+ Detection and Grouping.
+
+ More details can be found in the `paper
+ `__ .
+
+ Args:
+ downsample_times (int): Downsample times in a HourglassModule.
+ num_stacks (int): Number of HourglassModule modules stacked,
+ 1 for Hourglass-52, 2 for Hourglass-104.
+ stage_channels (list[int]): Feature channel of each sub-module in a
+ HourglassModule.
+ stage_blocks (list[int]): Number of sub-modules stacked in a
+ HourglassModule.
+ feat_channels (int): Feature channel of conv after a HourglassModule.
+ norm_cfg (dict): Dictionary to construct and config norm layer.
+
+ Example:
+ >>> from mmpose.models import HourglassAENet
+ >>> import torch
+ >>> self = HourglassAENet()
+ >>> self.eval()
+ >>> inputs = torch.rand(1, 3, 512, 512)
+ >>> level_outputs = self.forward(inputs)
+ >>> for level_output in level_outputs:
+ ... print(tuple(level_output.shape))
+ (1, 34, 128, 128)
+ """
+
+ def __init__(self,
+ downsample_times=4,
+ num_stacks=1,
+ out_channels=34,
+ stage_channels=(256, 384, 512, 640, 768),
+ feat_channels=256,
+ norm_cfg=dict(type='BN', requires_grad=True)):
+ # Protect mutable default arguments
+ norm_cfg = copy.deepcopy(norm_cfg)
+ super().__init__()
+
+ self.num_stacks = num_stacks
+ assert self.num_stacks >= 1
+ assert len(stage_channels) > downsample_times
+
+ cur_channels = stage_channels[0]
+
+ self.stem = nn.Sequential(
+ ConvModule(3, 64, 7, padding=3, stride=2, norm_cfg=norm_cfg),
+ ConvModule(64, 128, 3, padding=1, norm_cfg=norm_cfg),
+ MaxPool2d(2, 2),
+ ConvModule(128, 128, 3, padding=1, norm_cfg=norm_cfg),
+ ConvModule(128, feat_channels, 3, padding=1, norm_cfg=norm_cfg),
+ )
+
+ self.hourglass_modules = nn.ModuleList([
+ nn.Sequential(
+ HourglassAEModule(
+ downsample_times, stage_channels, norm_cfg=norm_cfg),
+ ConvModule(
+ feat_channels,
+ feat_channels,
+ 3,
+ padding=1,
+ norm_cfg=norm_cfg),
+ ConvModule(
+ feat_channels,
+ feat_channels,
+ 3,
+ padding=1,
+ norm_cfg=norm_cfg)) for _ in range(num_stacks)
+ ])
+
+ self.out_convs = nn.ModuleList([
+ ConvModule(
+ cur_channels,
+ out_channels,
+ 1,
+ padding=0,
+ norm_cfg=None,
+ act_cfg=None) for _ in range(num_stacks)
+ ])
+
+ self.remap_out_convs = nn.ModuleList([
+ ConvModule(
+ out_channels,
+ feat_channels,
+ 1,
+ norm_cfg=norm_cfg,
+ act_cfg=None) for _ in range(num_stacks - 1)
+ ])
+
+ self.remap_feature_convs = nn.ModuleList([
+ ConvModule(
+ feat_channels,
+ feat_channels,
+ 1,
+ norm_cfg=norm_cfg,
+ act_cfg=None) for _ in range(num_stacks - 1)
+ ])
+
+ self.relu = nn.ReLU(inplace=True)
+
+ def init_weights(self, pretrained=None):
+ """Initialize the weights in backbone.
+
+ Args:
+ pretrained (str, optional): Path to pre-trained weights.
+ Defaults to None.
+ """
+ if isinstance(pretrained, str):
+ logger = get_root_logger()
+ load_checkpoint(self, pretrained, strict=False, logger=logger)
+ elif pretrained is None:
+ for m in self.modules():
+ if isinstance(m, nn.Conv2d):
+ normal_init(m, std=0.001)
+ elif isinstance(m, (_BatchNorm, nn.GroupNorm)):
+ constant_init(m, 1)
+ else:
+ raise TypeError('pretrained must be a str or None')
+
+ def forward(self, x):
+ """Model forward function."""
+ inter_feat = self.stem(x)
+ out_feats = []
+
+ for ind in range(self.num_stacks):
+ single_hourglass = self.hourglass_modules[ind]
+ out_conv = self.out_convs[ind]
+
+ hourglass_feat = single_hourglass(inter_feat)
+ out_feat = out_conv(hourglass_feat)
+ out_feats.append(out_feat)
+
+ if ind < self.num_stacks - 1:
+ inter_feat = inter_feat + self.remap_out_convs[ind](
+ out_feat) + self.remap_feature_convs[ind](
+ hourglass_feat)
+
+ return out_feats
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/hrformer.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/hrformer.py
new file mode 100644
index 0000000..b843300
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/hrformer.py
@@ -0,0 +1,746 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+
+import math
+
+import torch
+import torch.nn as nn
+# from timm.models.layers import to_2tuple, trunc_normal_
+from mmcv.cnn import (build_activation_layer, build_conv_layer,
+ build_norm_layer, trunc_normal_init)
+from mmcv.cnn.bricks.transformer import build_dropout
+from mmcv.runner import BaseModule
+from torch.nn.functional import pad
+
+from ..builder import BACKBONES
+from .hrnet import Bottleneck, HRModule, HRNet
+
+
+def nlc_to_nchw(x, hw_shape):
+ """Convert [N, L, C] shape tensor to [N, C, H, W] shape tensor.
+
+ Args:
+ x (Tensor): The input tensor of shape [N, L, C] before conversion.
+ hw_shape (Sequence[int]): The height and width of output feature map.
+
+ Returns:
+ Tensor: The output tensor of shape [N, C, H, W] after conversion.
+ """
+ H, W = hw_shape
+ assert len(x.shape) == 3
+ B, L, C = x.shape
+ assert L == H * W, 'The seq_len doesn\'t match H, W'
+ return x.transpose(1, 2).reshape(B, C, H, W)
+
+
+def nchw_to_nlc(x):
+ """Flatten [N, C, H, W] shape tensor to [N, L, C] shape tensor.
+
+ Args:
+ x (Tensor): The input tensor of shape [N, C, H, W] before conversion.
+
+ Returns:
+ Tensor: The output tensor of shape [N, L, C] after conversion.
+ """
+ assert len(x.shape) == 4
+ return x.flatten(2).transpose(1, 2).contiguous()
+
+
+def build_drop_path(drop_path_rate):
+ """Build drop path layer."""
+ return build_dropout(dict(type='DropPath', drop_prob=drop_path_rate))
+
+
+class WindowMSA(BaseModule):
+ """Window based multi-head self-attention (W-MSA) module with relative
+ position bias.
+
+ Args:
+ embed_dims (int): Number of input channels.
+ num_heads (int): Number of attention heads.
+ window_size (tuple[int]): The height and width of the window.
+ qkv_bias (bool, optional): If True, add a learnable bias to q, k, v.
+ Default: True.
+ qk_scale (float | None, optional): Override default qk scale of
+ head_dim ** -0.5 if set. Default: None.
+ attn_drop_rate (float, optional): Dropout ratio of attention weight.
+ Default: 0.0
+ proj_drop_rate (float, optional): Dropout ratio of output. Default: 0.
+ with_rpe (bool, optional): If True, use relative position bias.
+ Default: True.
+ init_cfg (dict | None, optional): The Config for initialization.
+ Default: None.
+ """
+
+ def __init__(self,
+ embed_dims,
+ num_heads,
+ window_size,
+ qkv_bias=True,
+ qk_scale=None,
+ attn_drop_rate=0.,
+ proj_drop_rate=0.,
+ with_rpe=True,
+ init_cfg=None):
+
+ super().__init__(init_cfg=init_cfg)
+ self.embed_dims = embed_dims
+ self.window_size = window_size # Wh, Ww
+ self.num_heads = num_heads
+ head_embed_dims = embed_dims // num_heads
+ self.scale = qk_scale or head_embed_dims**-0.5
+
+ self.with_rpe = with_rpe
+ if self.with_rpe:
+ # define a parameter table of relative position bias
+ self.relative_position_bias_table = nn.Parameter(
+ torch.zeros(
+ (2 * window_size[0] - 1) * (2 * window_size[1] - 1),
+ num_heads)) # 2*Wh-1 * 2*Ww-1, nH
+
+ Wh, Ww = self.window_size
+ rel_index_coords = self.double_step_seq(2 * Ww - 1, Wh, 1, Ww)
+ rel_position_index = rel_index_coords + rel_index_coords.T
+ rel_position_index = rel_position_index.flip(1).contiguous()
+ self.register_buffer('relative_position_index', rel_position_index)
+
+ self.qkv = nn.Linear(embed_dims, embed_dims * 3, bias=qkv_bias)
+ self.attn_drop = nn.Dropout(attn_drop_rate)
+ self.proj = nn.Linear(embed_dims, embed_dims)
+ self.proj_drop = nn.Dropout(proj_drop_rate)
+
+ self.softmax = nn.Softmax(dim=-1)
+
+ def init_weights(self):
+ trunc_normal_init(self.relative_position_bias_table, std=0.02)
+
+ def forward(self, x, mask=None):
+ """
+ Args:
+
+ x (tensor): input features with shape of (B*num_windows, N, C)
+ mask (tensor | None, Optional): mask with shape of (num_windows,
+ Wh*Ww, Wh*Ww), value should be between (-inf, 0].
+ """
+ B, N, C = x.shape
+ qkv = self.qkv(x).reshape(B, N, 3, self.num_heads,
+ C // self.num_heads).permute(2, 0, 3, 1, 4)
+ q, k, v = qkv[0], qkv[1], qkv[2]
+
+ q = q * self.scale
+ attn = (q @ k.transpose(-2, -1))
+
+ if self.with_rpe:
+ relative_position_bias = self.relative_position_bias_table[
+ self.relative_position_index.view(-1)].view(
+ self.window_size[0] * self.window_size[1],
+ self.window_size[0] * self.window_size[1],
+ -1) # Wh*Ww,Wh*Ww,nH
+ relative_position_bias = relative_position_bias.permute(
+ 2, 0, 1).contiguous() # nH, Wh*Ww, Wh*Ww
+ attn = attn + relative_position_bias.unsqueeze(0)
+
+ if mask is not None:
+ nW = mask.shape[0]
+ attn = attn.view(B // nW, nW, self.num_heads, N,
+ N) + mask.unsqueeze(1).unsqueeze(0)
+ attn = attn.view(-1, self.num_heads, N, N)
+ attn = self.softmax(attn)
+
+ attn = self.attn_drop(attn)
+
+ x = (attn @ v).transpose(1, 2).reshape(B, N, C)
+ x = self.proj(x)
+ x = self.proj_drop(x)
+ return x
+
+ @staticmethod
+ def double_step_seq(step1, len1, step2, len2):
+ seq1 = torch.arange(0, step1 * len1, step1)
+ seq2 = torch.arange(0, step2 * len2, step2)
+ return (seq1[:, None] + seq2[None, :]).reshape(1, -1)
+
+
+class LocalWindowSelfAttention(BaseModule):
+ r""" Local-window Self Attention (LSA) module with relative position bias.
+
+ This module is the short-range self-attention module in the
+ Interlaced Sparse Self-Attention `_.
+
+ Args:
+ embed_dims (int): Number of input channels.
+ num_heads (int): Number of attention heads.
+ window_size (tuple[int] | int): The height and width of the window.
+ qkv_bias (bool, optional): If True, add a learnable bias to q, k, v.
+ Default: True.
+ qk_scale (float | None, optional): Override default qk scale of
+ head_dim ** -0.5 if set. Default: None.
+ attn_drop_rate (float, optional): Dropout ratio of attention weight.
+ Default: 0.0
+ proj_drop_rate (float, optional): Dropout ratio of output. Default: 0.
+ with_rpe (bool, optional): If True, use relative position bias.
+ Default: True.
+ with_pad_mask (bool, optional): If True, mask out the padded tokens in
+ the attention process. Default: False.
+ init_cfg (dict | None, optional): The Config for initialization.
+ Default: None.
+ """
+
+ def __init__(self,
+ embed_dims,
+ num_heads,
+ window_size,
+ qkv_bias=True,
+ qk_scale=None,
+ attn_drop_rate=0.,
+ proj_drop_rate=0.,
+ with_rpe=True,
+ with_pad_mask=False,
+ init_cfg=None):
+ super().__init__(init_cfg=init_cfg)
+ if isinstance(window_size, int):
+ window_size = (window_size, window_size)
+ self.window_size = window_size
+ self.with_pad_mask = with_pad_mask
+ self.attn = WindowMSA(
+ embed_dims=embed_dims,
+ num_heads=num_heads,
+ window_size=window_size,
+ qkv_bias=qkv_bias,
+ qk_scale=qk_scale,
+ attn_drop_rate=attn_drop_rate,
+ proj_drop_rate=proj_drop_rate,
+ with_rpe=with_rpe,
+ init_cfg=init_cfg)
+
+ def forward(self, x, H, W, **kwargs):
+ """Forward function."""
+ B, N, C = x.shape
+ x = x.view(B, H, W, C)
+ Wh, Ww = self.window_size
+
+ # center-pad the feature on H and W axes
+ pad_h = math.ceil(H / Wh) * Wh - H
+ pad_w = math.ceil(W / Ww) * Ww - W
+ x = pad(x, (0, 0, pad_w // 2, pad_w - pad_w // 2, pad_h // 2,
+ pad_h - pad_h // 2))
+
+ # permute
+ x = x.view(B, math.ceil(H / Wh), Wh, math.ceil(W / Ww), Ww, C)
+ x = x.permute(0, 1, 3, 2, 4, 5)
+ x = x.reshape(-1, Wh * Ww, C) # (B*num_window, Wh*Ww, C)
+
+ # attention
+ if self.with_pad_mask and pad_h > 0 and pad_w > 0:
+ pad_mask = x.new_zeros(1, H, W, 1)
+ pad_mask = pad(
+ pad_mask, [
+ 0, 0, pad_w // 2, pad_w - pad_w // 2, pad_h // 2,
+ pad_h - pad_h // 2
+ ],
+ value=-float('inf'))
+ pad_mask = pad_mask.view(1, math.ceil(H / Wh), Wh,
+ math.ceil(W / Ww), Ww, 1)
+ pad_mask = pad_mask.permute(1, 3, 0, 2, 4, 5)
+ pad_mask = pad_mask.reshape(-1, Wh * Ww)
+ pad_mask = pad_mask[:, None, :].expand([-1, Wh * Ww, -1])
+ out = self.attn(x, pad_mask, **kwargs)
+ else:
+ out = self.attn(x, **kwargs)
+
+ # reverse permutation
+ out = out.reshape(B, math.ceil(H / Wh), math.ceil(W / Ww), Wh, Ww, C)
+ out = out.permute(0, 1, 3, 2, 4, 5)
+ out = out.reshape(B, H + pad_h, W + pad_w, C)
+
+ # de-pad
+ out = out[:, pad_h // 2:H + pad_h // 2, pad_w // 2:W + pad_w // 2]
+ return out.reshape(B, N, C)
+
+
+class CrossFFN(BaseModule):
+ r"""FFN with Depthwise Conv of HRFormer.
+
+ Args:
+ in_features (int): The feature dimension.
+ hidden_features (int, optional): The hidden dimension of FFNs.
+ Defaults: The same as in_features.
+ act_cfg (dict, optional): Config of activation layer.
+ Default: dict(type='GELU').
+ dw_act_cfg (dict, optional): Config of activation layer appended
+ right after DW Conv. Default: dict(type='GELU').
+ norm_cfg (dict, optional): Config of norm layer.
+ Default: dict(type='SyncBN').
+ init_cfg (dict | list | None, optional): The init config.
+ Default: None.
+ """
+
+ def __init__(self,
+ in_features,
+ hidden_features=None,
+ out_features=None,
+ act_cfg=dict(type='GELU'),
+ dw_act_cfg=dict(type='GELU'),
+ norm_cfg=dict(type='SyncBN'),
+ init_cfg=None):
+ super().__init__(init_cfg=init_cfg)
+ out_features = out_features or in_features
+ hidden_features = hidden_features or in_features
+ self.fc1 = nn.Conv2d(in_features, hidden_features, kernel_size=1)
+ self.act1 = build_activation_layer(act_cfg)
+ self.norm1 = build_norm_layer(norm_cfg, hidden_features)[1]
+ self.dw3x3 = nn.Conv2d(
+ hidden_features,
+ hidden_features,
+ kernel_size=3,
+ stride=1,
+ groups=hidden_features,
+ padding=1)
+ self.act2 = build_activation_layer(dw_act_cfg)
+ self.norm2 = build_norm_layer(norm_cfg, hidden_features)[1]
+ self.fc2 = nn.Conv2d(hidden_features, out_features, kernel_size=1)
+ self.act3 = build_activation_layer(act_cfg)
+ self.norm3 = build_norm_layer(norm_cfg, out_features)[1]
+
+ # put the modules togather
+ self.layers = [
+ self.fc1, self.norm1, self.act1, self.dw3x3, self.norm2, self.act2,
+ self.fc2, self.norm3, self.act3
+ ]
+
+ def forward(self, x, H, W):
+ """Forward function."""
+ x = nlc_to_nchw(x, (H, W))
+ for layer in self.layers:
+ x = layer(x)
+ x = nchw_to_nlc(x)
+ return x
+
+
+class HRFormerBlock(BaseModule):
+ """High-Resolution Block for HRFormer.
+
+ Args:
+ in_features (int): The input dimension.
+ out_features (int): The output dimension.
+ num_heads (int): The number of head within each LSA.
+ window_size (int, optional): The window size for the LSA.
+ Default: 7
+ mlp_ratio (int, optional): The expansion ration of FFN.
+ Default: 4
+ act_cfg (dict, optional): Config of activation layer.
+ Default: dict(type='GELU').
+ norm_cfg (dict, optional): Config of norm layer.
+ Default: dict(type='SyncBN').
+ transformer_norm_cfg (dict, optional): Config of transformer norm
+ layer. Default: dict(type='LN', eps=1e-6).
+ init_cfg (dict | list | None, optional): The init config.
+ Default: None.
+ """
+
+ expansion = 1
+
+ def __init__(self,
+ in_features,
+ out_features,
+ num_heads,
+ window_size=7,
+ mlp_ratio=4.0,
+ drop_path=0.0,
+ act_cfg=dict(type='GELU'),
+ norm_cfg=dict(type='SyncBN'),
+ transformer_norm_cfg=dict(type='LN', eps=1e-6),
+ init_cfg=None,
+ **kwargs):
+ super(HRFormerBlock, self).__init__(init_cfg=init_cfg)
+ self.num_heads = num_heads
+ self.window_size = window_size
+ self.mlp_ratio = mlp_ratio
+
+ self.norm1 = build_norm_layer(transformer_norm_cfg, in_features)[1]
+ self.attn = LocalWindowSelfAttention(
+ in_features,
+ num_heads=num_heads,
+ window_size=window_size,
+ init_cfg=None,
+ **kwargs)
+
+ self.norm2 = build_norm_layer(transformer_norm_cfg, out_features)[1]
+ self.ffn = CrossFFN(
+ in_features=in_features,
+ hidden_features=int(in_features * mlp_ratio),
+ out_features=out_features,
+ norm_cfg=norm_cfg,
+ act_cfg=act_cfg,
+ dw_act_cfg=act_cfg,
+ init_cfg=None)
+
+ self.drop_path = build_drop_path(
+ drop_path) if drop_path > 0.0 else nn.Identity()
+
+ def forward(self, x):
+ """Forward function."""
+ B, C, H, W = x.size()
+ # Attention
+ x = x.view(B, C, -1).permute(0, 2, 1)
+ x = x + self.drop_path(self.attn(self.norm1(x), H, W))
+ # FFN
+ x = x + self.drop_path(self.ffn(self.norm2(x), H, W))
+ x = x.permute(0, 2, 1).view(B, C, H, W)
+ return x
+
+ def extra_repr(self):
+ """(Optional) Set the extra information about this module."""
+ return 'num_heads={}, window_size={}, mlp_ratio={}'.format(
+ self.num_heads, self.window_size, self.mlp_ratio)
+
+
+class HRFomerModule(HRModule):
+ """High-Resolution Module for HRFormer.
+
+ Args:
+ num_branches (int): The number of branches in the HRFormerModule.
+ block (nn.Module): The building block of HRFormer.
+ The block should be the HRFormerBlock.
+ num_blocks (tuple): The number of blocks in each branch.
+ The length must be equal to num_branches.
+ num_inchannels (tuple): The number of input channels in each branch.
+ The length must be equal to num_branches.
+ num_channels (tuple): The number of channels in each branch.
+ The length must be equal to num_branches.
+ num_heads (tuple): The number of heads within the LSAs.
+ num_window_sizes (tuple): The window size for the LSAs.
+ num_mlp_ratios (tuple): The expansion ratio for the FFNs.
+ drop_path (int, optional): The drop path rate of HRFomer.
+ Default: 0.0
+ multiscale_output (bool, optional): Whether to output multi-level
+ features produced by multiple branches. If False, only the first
+ level feature will be output. Default: True.
+ conv_cfg (dict, optional): Config of the conv layers.
+ Default: None.
+ norm_cfg (dict, optional): Config of the norm layers appended
+ right after conv. Default: dict(type='SyncBN', requires_grad=True)
+ transformer_norm_cfg (dict, optional): Config of the norm layers.
+ Default: dict(type='LN', eps=1e-6)
+ with_cp (bool): Use checkpoint or not. Using checkpoint will save some
+ memory while slowing down the training speed. Default: False
+ upsample_cfg(dict, optional): The config of upsample layers in fuse
+ layers. Default: dict(mode='bilinear', align_corners=False)
+ """
+
+ def __init__(self,
+ num_branches,
+ block,
+ num_blocks,
+ num_inchannels,
+ num_channels,
+ num_heads,
+ num_window_sizes,
+ num_mlp_ratios,
+ multiscale_output=True,
+ drop_paths=0.0,
+ with_rpe=True,
+ with_pad_mask=False,
+ conv_cfg=None,
+ norm_cfg=dict(type='SyncBN', requires_grad=True),
+ transformer_norm_cfg=dict(type='LN', eps=1e-6),
+ with_cp=False,
+ upsample_cfg=dict(mode='bilinear', align_corners=False)):
+
+ self.transformer_norm_cfg = transformer_norm_cfg
+ self.drop_paths = drop_paths
+ self.num_heads = num_heads
+ self.num_window_sizes = num_window_sizes
+ self.num_mlp_ratios = num_mlp_ratios
+ self.with_rpe = with_rpe
+ self.with_pad_mask = with_pad_mask
+
+ super().__init__(num_branches, block, num_blocks, num_inchannels,
+ num_channels, multiscale_output, with_cp, conv_cfg,
+ norm_cfg, upsample_cfg)
+
+ def _make_one_branch(self,
+ branch_index,
+ block,
+ num_blocks,
+ num_channels,
+ stride=1):
+ """Build one branch."""
+ # HRFormerBlock does not support down sample layer yet.
+ assert stride == 1 and self.in_channels[branch_index] == num_channels[
+ branch_index]
+ layers = []
+ layers.append(
+ block(
+ self.in_channels[branch_index],
+ num_channels[branch_index],
+ num_heads=self.num_heads[branch_index],
+ window_size=self.num_window_sizes[branch_index],
+ mlp_ratio=self.num_mlp_ratios[branch_index],
+ drop_path=self.drop_paths[0],
+ norm_cfg=self.norm_cfg,
+ transformer_norm_cfg=self.transformer_norm_cfg,
+ init_cfg=None,
+ with_rpe=self.with_rpe,
+ with_pad_mask=self.with_pad_mask))
+
+ self.in_channels[
+ branch_index] = self.in_channels[branch_index] * block.expansion
+ for i in range(1, num_blocks[branch_index]):
+ layers.append(
+ block(
+ self.in_channels[branch_index],
+ num_channels[branch_index],
+ num_heads=self.num_heads[branch_index],
+ window_size=self.num_window_sizes[branch_index],
+ mlp_ratio=self.num_mlp_ratios[branch_index],
+ drop_path=self.drop_paths[i],
+ norm_cfg=self.norm_cfg,
+ transformer_norm_cfg=self.transformer_norm_cfg,
+ init_cfg=None,
+ with_rpe=self.with_rpe,
+ with_pad_mask=self.with_pad_mask))
+ return nn.Sequential(*layers)
+
+ def _make_fuse_layers(self):
+ """Build fuse layers."""
+ if self.num_branches == 1:
+ return None
+ num_branches = self.num_branches
+ num_inchannels = self.in_channels
+ fuse_layers = []
+ for i in range(num_branches if self.multiscale_output else 1):
+ fuse_layer = []
+ for j in range(num_branches):
+ if j > i:
+ fuse_layer.append(
+ nn.Sequential(
+ build_conv_layer(
+ self.conv_cfg,
+ num_inchannels[j],
+ num_inchannels[i],
+ kernel_size=1,
+ stride=1,
+ bias=False),
+ build_norm_layer(self.norm_cfg,
+ num_inchannels[i])[1],
+ nn.Upsample(
+ scale_factor=2**(j - i),
+ mode=self.upsample_cfg['mode'],
+ align_corners=self.
+ upsample_cfg['align_corners'])))
+ elif j == i:
+ fuse_layer.append(None)
+ else:
+ conv3x3s = []
+ for k in range(i - j):
+ if k == i - j - 1:
+ num_outchannels_conv3x3 = num_inchannels[i]
+ with_out_act = False
+ else:
+ num_outchannels_conv3x3 = num_inchannels[j]
+ with_out_act = True
+ sub_modules = [
+ build_conv_layer(
+ self.conv_cfg,
+ num_inchannels[j],
+ num_inchannels[j],
+ kernel_size=3,
+ stride=2,
+ padding=1,
+ groups=num_inchannels[j],
+ bias=False,
+ ),
+ build_norm_layer(self.norm_cfg,
+ num_inchannels[j])[1],
+ build_conv_layer(
+ self.conv_cfg,
+ num_inchannels[j],
+ num_outchannels_conv3x3,
+ kernel_size=1,
+ stride=1,
+ bias=False,
+ ),
+ build_norm_layer(self.norm_cfg,
+ num_outchannels_conv3x3)[1]
+ ]
+ if with_out_act:
+ sub_modules.append(nn.ReLU(False))
+ conv3x3s.append(nn.Sequential(*sub_modules))
+ fuse_layer.append(nn.Sequential(*conv3x3s))
+ fuse_layers.append(nn.ModuleList(fuse_layer))
+
+ return nn.ModuleList(fuse_layers)
+
+ def get_num_inchannels(self):
+ """Return the number of input channels."""
+ return self.in_channels
+
+
+@BACKBONES.register_module()
+class HRFormer(HRNet):
+ """HRFormer backbone.
+
+ This backbone is the implementation of `HRFormer: High-Resolution
+ Transformer for Dense Prediction `_.
+
+ Args:
+ extra (dict): Detailed configuration for each stage of HRNet.
+ There must be 4 stages, the configuration for each stage must have
+ 5 keys:
+
+ - num_modules (int): The number of HRModule in this stage.
+ - num_branches (int): The number of branches in the HRModule.
+ - block (str): The type of block.
+ - num_blocks (tuple): The number of blocks in each branch.
+ The length must be equal to num_branches.
+ - num_channels (tuple): The number of channels in each branch.
+ The length must be equal to num_branches.
+ in_channels (int): Number of input image channels. Normally 3.
+ conv_cfg (dict): Dictionary to construct and config conv layer.
+ Default: None.
+ norm_cfg (dict): Config of norm layer.
+ Use `SyncBN` by default.
+ transformer_norm_cfg (dict): Config of transformer norm layer.
+ Use `LN` by default.
+ norm_eval (bool): Whether to set norm layers to eval mode, namely,
+ freeze running stats (mean and var). Note: Effect on Batch Norm
+ and its variants only. Default: False.
+ zero_init_residual (bool): Whether to use zero init for last norm layer
+ in resblocks to let them behave as identity. Default: False.
+ frozen_stages (int): Stages to be frozen (stop grad and set eval mode).
+ -1 means not freezing any parameters. Default: -1.
+ Example:
+ >>> from mmpose.models import HRFormer
+ >>> import torch
+ >>> extra = dict(
+ >>> stage1=dict(
+ >>> num_modules=1,
+ >>> num_branches=1,
+ >>> block='BOTTLENECK',
+ >>> num_blocks=(2, ),
+ >>> num_channels=(64, )),
+ >>> stage2=dict(
+ >>> num_modules=1,
+ >>> num_branches=2,
+ >>> block='HRFORMER',
+ >>> window_sizes=(7, 7),
+ >>> num_heads=(1, 2),
+ >>> mlp_ratios=(4, 4),
+ >>> num_blocks=(2, 2),
+ >>> num_channels=(32, 64)),
+ >>> stage3=dict(
+ >>> num_modules=4,
+ >>> num_branches=3,
+ >>> block='HRFORMER',
+ >>> window_sizes=(7, 7, 7),
+ >>> num_heads=(1, 2, 4),
+ >>> mlp_ratios=(4, 4, 4),
+ >>> num_blocks=(2, 2, 2),
+ >>> num_channels=(32, 64, 128)),
+ >>> stage4=dict(
+ >>> num_modules=2,
+ >>> num_branches=4,
+ >>> block='HRFORMER',
+ >>> window_sizes=(7, 7, 7, 7),
+ >>> num_heads=(1, 2, 4, 8),
+ >>> mlp_ratios=(4, 4, 4, 4),
+ >>> num_blocks=(2, 2, 2, 2),
+ >>> num_channels=(32, 64, 128, 256)))
+ >>> self = HRFormer(extra, in_channels=1)
+ >>> self.eval()
+ >>> inputs = torch.rand(1, 1, 32, 32)
+ >>> level_outputs = self.forward(inputs)
+ >>> for level_out in level_outputs:
+ ... print(tuple(level_out.shape))
+ (1, 32, 8, 8)
+ (1, 64, 4, 4)
+ (1, 128, 2, 2)
+ (1, 256, 1, 1)
+ """
+
+ blocks_dict = {'BOTTLENECK': Bottleneck, 'HRFORMERBLOCK': HRFormerBlock}
+
+ def __init__(self,
+ extra,
+ in_channels=3,
+ conv_cfg=None,
+ norm_cfg=dict(type='BN', requires_grad=True),
+ transformer_norm_cfg=dict(type='LN', eps=1e-6),
+ norm_eval=False,
+ with_cp=False,
+ zero_init_residual=False,
+ frozen_stages=-1):
+
+ # stochastic depth
+ depths = [
+ extra[stage]['num_blocks'][0] * extra[stage]['num_modules']
+ for stage in ['stage2', 'stage3', 'stage4']
+ ]
+ depth_s2, depth_s3, _ = depths
+ drop_path_rate = extra['drop_path_rate']
+ dpr = [
+ x.item() for x in torch.linspace(0, drop_path_rate, sum(depths))
+ ]
+ extra['stage2']['drop_path_rates'] = dpr[0:depth_s2]
+ extra['stage3']['drop_path_rates'] = dpr[depth_s2:depth_s2 + depth_s3]
+ extra['stage4']['drop_path_rates'] = dpr[depth_s2 + depth_s3:]
+
+ # HRFormer use bilinear upsample as default
+ upsample_cfg = extra.get('upsample', {
+ 'mode': 'bilinear',
+ 'align_corners': False
+ })
+ extra['upsample'] = upsample_cfg
+ self.transformer_norm_cfg = transformer_norm_cfg
+ self.with_rpe = extra.get('with_rpe', True)
+ self.with_pad_mask = extra.get('with_pad_mask', False)
+
+ super().__init__(extra, in_channels, conv_cfg, norm_cfg, norm_eval,
+ with_cp, zero_init_residual, frozen_stages)
+
+ def _make_stage(self,
+ layer_config,
+ num_inchannels,
+ multiscale_output=True):
+ """Make each stage."""
+ num_modules = layer_config['num_modules']
+ num_branches = layer_config['num_branches']
+ num_blocks = layer_config['num_blocks']
+ num_channels = layer_config['num_channels']
+ block = self.blocks_dict[layer_config['block']]
+ num_heads = layer_config['num_heads']
+ num_window_sizes = layer_config['window_sizes']
+ num_mlp_ratios = layer_config['mlp_ratios']
+ drop_path_rates = layer_config['drop_path_rates']
+
+ modules = []
+ for i in range(num_modules):
+ # multiscale_output is only used at the last module
+ if not multiscale_output and i == num_modules - 1:
+ reset_multiscale_output = False
+ else:
+ reset_multiscale_output = True
+
+ modules.append(
+ HRFomerModule(
+ num_branches,
+ block,
+ num_blocks,
+ num_inchannels,
+ num_channels,
+ num_heads,
+ num_window_sizes,
+ num_mlp_ratios,
+ reset_multiscale_output,
+ drop_paths=drop_path_rates[num_blocks[0] *
+ i:num_blocks[0] * (i + 1)],
+ with_rpe=self.with_rpe,
+ with_pad_mask=self.with_pad_mask,
+ conv_cfg=self.conv_cfg,
+ norm_cfg=self.norm_cfg,
+ transformer_norm_cfg=self.transformer_norm_cfg,
+ with_cp=self.with_cp,
+ upsample_cfg=self.upsample_cfg))
+ num_inchannels = modules[-1].get_num_inchannels()
+
+ return nn.Sequential(*modules), num_inchannels
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/hrnet.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/hrnet.py
new file mode 100644
index 0000000..87dc8ce
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/hrnet.py
@@ -0,0 +1,604 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import copy
+
+import torch.nn as nn
+from mmcv.cnn import (build_conv_layer, build_norm_layer, constant_init,
+ normal_init)
+from torch.nn.modules.batchnorm import _BatchNorm
+
+from mmpose.utils import get_root_logger
+from ..builder import BACKBONES
+from .resnet import BasicBlock, Bottleneck, get_expansion
+from .utils import load_checkpoint
+
+
+class HRModule(nn.Module):
+ """High-Resolution Module for HRNet.
+
+ In this module, every branch has 4 BasicBlocks/Bottlenecks. Fusion/Exchange
+ is in this module.
+ """
+
+ def __init__(self,
+ num_branches,
+ blocks,
+ num_blocks,
+ in_channels,
+ num_channels,
+ multiscale_output=False,
+ with_cp=False,
+ conv_cfg=None,
+ norm_cfg=dict(type='BN'),
+ upsample_cfg=dict(mode='nearest', align_corners=None)):
+
+ # Protect mutable default arguments
+ norm_cfg = copy.deepcopy(norm_cfg)
+ super().__init__()
+ self._check_branches(num_branches, num_blocks, in_channels,
+ num_channels)
+
+ self.in_channels = in_channels
+ self.num_branches = num_branches
+
+ self.multiscale_output = multiscale_output
+ self.norm_cfg = norm_cfg
+ self.conv_cfg = conv_cfg
+ self.upsample_cfg = upsample_cfg
+ self.with_cp = with_cp
+ self.branches = self._make_branches(num_branches, blocks, num_blocks,
+ num_channels)
+ self.fuse_layers = self._make_fuse_layers()
+ self.relu = nn.ReLU(inplace=True)
+
+ @staticmethod
+ def _check_branches(num_branches, num_blocks, in_channels, num_channels):
+ """Check input to avoid ValueError."""
+ if num_branches != len(num_blocks):
+ error_msg = f'NUM_BRANCHES({num_branches}) ' \
+ f'!= NUM_BLOCKS({len(num_blocks)})'
+ raise ValueError(error_msg)
+
+ if num_branches != len(num_channels):
+ error_msg = f'NUM_BRANCHES({num_branches}) ' \
+ f'!= NUM_CHANNELS({len(num_channels)})'
+ raise ValueError(error_msg)
+
+ if num_branches != len(in_channels):
+ error_msg = f'NUM_BRANCHES({num_branches}) ' \
+ f'!= NUM_INCHANNELS({len(in_channels)})'
+ raise ValueError(error_msg)
+
+ def _make_one_branch(self,
+ branch_index,
+ block,
+ num_blocks,
+ num_channels,
+ stride=1):
+ """Make one branch."""
+ downsample = None
+ if stride != 1 or \
+ self.in_channels[branch_index] != \
+ num_channels[branch_index] * get_expansion(block):
+ downsample = nn.Sequential(
+ build_conv_layer(
+ self.conv_cfg,
+ self.in_channels[branch_index],
+ num_channels[branch_index] * get_expansion(block),
+ kernel_size=1,
+ stride=stride,
+ bias=False),
+ build_norm_layer(
+ self.norm_cfg,
+ num_channels[branch_index] * get_expansion(block))[1])
+
+ layers = []
+ layers.append(
+ block(
+ self.in_channels[branch_index],
+ num_channels[branch_index] * get_expansion(block),
+ stride=stride,
+ downsample=downsample,
+ with_cp=self.with_cp,
+ norm_cfg=self.norm_cfg,
+ conv_cfg=self.conv_cfg))
+ self.in_channels[branch_index] = \
+ num_channels[branch_index] * get_expansion(block)
+ for _ in range(1, num_blocks[branch_index]):
+ layers.append(
+ block(
+ self.in_channels[branch_index],
+ num_channels[branch_index] * get_expansion(block),
+ with_cp=self.with_cp,
+ norm_cfg=self.norm_cfg,
+ conv_cfg=self.conv_cfg))
+
+ return nn.Sequential(*layers)
+
+ def _make_branches(self, num_branches, block, num_blocks, num_channels):
+ """Make branches."""
+ branches = []
+
+ for i in range(num_branches):
+ branches.append(
+ self._make_one_branch(i, block, num_blocks, num_channels))
+
+ return nn.ModuleList(branches)
+
+ def _make_fuse_layers(self):
+ """Make fuse layer."""
+ if self.num_branches == 1:
+ return None
+
+ num_branches = self.num_branches
+ in_channels = self.in_channels
+ fuse_layers = []
+ num_out_branches = num_branches if self.multiscale_output else 1
+
+ for i in range(num_out_branches):
+ fuse_layer = []
+ for j in range(num_branches):
+ if j > i:
+ fuse_layer.append(
+ nn.Sequential(
+ build_conv_layer(
+ self.conv_cfg,
+ in_channels[j],
+ in_channels[i],
+ kernel_size=1,
+ stride=1,
+ padding=0,
+ bias=False),
+ build_norm_layer(self.norm_cfg, in_channels[i])[1],
+ nn.Upsample(
+ scale_factor=2**(j - i),
+ mode=self.upsample_cfg['mode'],
+ align_corners=self.
+ upsample_cfg['align_corners'])))
+ elif j == i:
+ fuse_layer.append(None)
+ else:
+ conv_downsamples = []
+ for k in range(i - j):
+ if k == i - j - 1:
+ conv_downsamples.append(
+ nn.Sequential(
+ build_conv_layer(
+ self.conv_cfg,
+ in_channels[j],
+ in_channels[i],
+ kernel_size=3,
+ stride=2,
+ padding=1,
+ bias=False),
+ build_norm_layer(self.norm_cfg,
+ in_channels[i])[1]))
+ else:
+ conv_downsamples.append(
+ nn.Sequential(
+ build_conv_layer(
+ self.conv_cfg,
+ in_channels[j],
+ in_channels[j],
+ kernel_size=3,
+ stride=2,
+ padding=1,
+ bias=False),
+ build_norm_layer(self.norm_cfg,
+ in_channels[j])[1],
+ nn.ReLU(inplace=True)))
+ fuse_layer.append(nn.Sequential(*conv_downsamples))
+ fuse_layers.append(nn.ModuleList(fuse_layer))
+
+ return nn.ModuleList(fuse_layers)
+
+ def forward(self, x):
+ """Forward function."""
+ if self.num_branches == 1:
+ return [self.branches[0](x[0])]
+
+ for i in range(self.num_branches):
+ x[i] = self.branches[i](x[i])
+
+ x_fuse = []
+ for i in range(len(self.fuse_layers)):
+ y = 0
+ for j in range(self.num_branches):
+ if i == j:
+ y += x[j]
+ else:
+ y += self.fuse_layers[i][j](x[j])
+ x_fuse.append(self.relu(y))
+ return x_fuse
+
+
+@BACKBONES.register_module()
+class HRNet(nn.Module):
+ """HRNet backbone.
+
+ `High-Resolution Representations for Labeling Pixels and Regions
+ `__
+
+ Args:
+ extra (dict): detailed configuration for each stage of HRNet.
+ in_channels (int): Number of input image channels. Default: 3.
+ conv_cfg (dict): dictionary to construct and config conv layer.
+ norm_cfg (dict): dictionary to construct and config norm layer.
+ norm_eval (bool): Whether to set norm layers to eval mode, namely,
+ freeze running stats (mean and var). Note: Effect on Batch Norm
+ and its variants only. Default: False
+ with_cp (bool): Use checkpoint or not. Using checkpoint will save some
+ memory while slowing down the training speed.
+ zero_init_residual (bool): whether to use zero init for last norm layer
+ in resblocks to let them behave as identity.
+ frozen_stages (int): Stages to be frozen (stop grad and set eval mode).
+ -1 means not freezing any parameters. Default: -1.
+
+ Example:
+ >>> from mmpose.models import HRNet
+ >>> import torch
+ >>> extra = dict(
+ >>> stage1=dict(
+ >>> num_modules=1,
+ >>> num_branches=1,
+ >>> block='BOTTLENECK',
+ >>> num_blocks=(4, ),
+ >>> num_channels=(64, )),
+ >>> stage2=dict(
+ >>> num_modules=1,
+ >>> num_branches=2,
+ >>> block='BASIC',
+ >>> num_blocks=(4, 4),
+ >>> num_channels=(32, 64)),
+ >>> stage3=dict(
+ >>> num_modules=4,
+ >>> num_branches=3,
+ >>> block='BASIC',
+ >>> num_blocks=(4, 4, 4),
+ >>> num_channels=(32, 64, 128)),
+ >>> stage4=dict(
+ >>> num_modules=3,
+ >>> num_branches=4,
+ >>> block='BASIC',
+ >>> num_blocks=(4, 4, 4, 4),
+ >>> num_channels=(32, 64, 128, 256)))
+ >>> self = HRNet(extra, in_channels=1)
+ >>> self.eval()
+ >>> inputs = torch.rand(1, 1, 32, 32)
+ >>> level_outputs = self.forward(inputs)
+ >>> for level_out in level_outputs:
+ ... print(tuple(level_out.shape))
+ (1, 32, 8, 8)
+ """
+
+ blocks_dict = {'BASIC': BasicBlock, 'BOTTLENECK': Bottleneck}
+
+ def __init__(self,
+ extra,
+ in_channels=3,
+ conv_cfg=None,
+ norm_cfg=dict(type='BN'),
+ norm_eval=False,
+ with_cp=False,
+ zero_init_residual=False,
+ frozen_stages=-1):
+ # Protect mutable default arguments
+ norm_cfg = copy.deepcopy(norm_cfg)
+ super().__init__()
+ self.extra = extra
+ self.conv_cfg = conv_cfg
+ self.norm_cfg = norm_cfg
+ self.norm_eval = norm_eval
+ self.with_cp = with_cp
+ self.zero_init_residual = zero_init_residual
+ self.frozen_stages = frozen_stages
+
+ # stem net
+ self.norm1_name, norm1 = build_norm_layer(self.norm_cfg, 64, postfix=1)
+ self.norm2_name, norm2 = build_norm_layer(self.norm_cfg, 64, postfix=2)
+
+ self.conv1 = build_conv_layer(
+ self.conv_cfg,
+ in_channels,
+ 64,
+ kernel_size=3,
+ stride=2,
+ padding=1,
+ bias=False)
+
+ self.add_module(self.norm1_name, norm1)
+ self.conv2 = build_conv_layer(
+ self.conv_cfg,
+ 64,
+ 64,
+ kernel_size=3,
+ stride=2,
+ padding=1,
+ bias=False)
+
+ self.add_module(self.norm2_name, norm2)
+ self.relu = nn.ReLU(inplace=True)
+
+ self.upsample_cfg = self.extra.get('upsample', {
+ 'mode': 'nearest',
+ 'align_corners': None
+ })
+
+ # stage 1
+ self.stage1_cfg = self.extra['stage1']
+ num_channels = self.stage1_cfg['num_channels'][0]
+ block_type = self.stage1_cfg['block']
+ num_blocks = self.stage1_cfg['num_blocks'][0]
+
+ block = self.blocks_dict[block_type]
+ stage1_out_channels = num_channels * get_expansion(block)
+ self.layer1 = self._make_layer(block, 64, stage1_out_channels,
+ num_blocks)
+
+ # stage 2
+ self.stage2_cfg = self.extra['stage2']
+ num_channels = self.stage2_cfg['num_channels']
+ block_type = self.stage2_cfg['block']
+
+ block = self.blocks_dict[block_type]
+ num_channels = [
+ channel * get_expansion(block) for channel in num_channels
+ ]
+ self.transition1 = self._make_transition_layer([stage1_out_channels],
+ num_channels)
+ self.stage2, pre_stage_channels = self._make_stage(
+ self.stage2_cfg, num_channels)
+
+ # stage 3
+ self.stage3_cfg = self.extra['stage3']
+ num_channels = self.stage3_cfg['num_channels']
+ block_type = self.stage3_cfg['block']
+
+ block = self.blocks_dict[block_type]
+ num_channels = [
+ channel * get_expansion(block) for channel in num_channels
+ ]
+ self.transition2 = self._make_transition_layer(pre_stage_channels,
+ num_channels)
+ self.stage3, pre_stage_channels = self._make_stage(
+ self.stage3_cfg, num_channels)
+
+ # stage 4
+ self.stage4_cfg = self.extra['stage4']
+ num_channels = self.stage4_cfg['num_channels']
+ block_type = self.stage4_cfg['block']
+
+ block = self.blocks_dict[block_type]
+ num_channels = [
+ channel * get_expansion(block) for channel in num_channels
+ ]
+ self.transition3 = self._make_transition_layer(pre_stage_channels,
+ num_channels)
+
+ self.stage4, pre_stage_channels = self._make_stage(
+ self.stage4_cfg,
+ num_channels,
+ multiscale_output=self.stage4_cfg.get('multiscale_output', False))
+
+ self._freeze_stages()
+
+ @property
+ def norm1(self):
+ """nn.Module: the normalization layer named "norm1" """
+ return getattr(self, self.norm1_name)
+
+ @property
+ def norm2(self):
+ """nn.Module: the normalization layer named "norm2" """
+ return getattr(self, self.norm2_name)
+
+ def _make_transition_layer(self, num_channels_pre_layer,
+ num_channels_cur_layer):
+ """Make transition layer."""
+ num_branches_cur = len(num_channels_cur_layer)
+ num_branches_pre = len(num_channels_pre_layer)
+
+ transition_layers = []
+ for i in range(num_branches_cur):
+ if i < num_branches_pre:
+ if num_channels_cur_layer[i] != num_channels_pre_layer[i]:
+ transition_layers.append(
+ nn.Sequential(
+ build_conv_layer(
+ self.conv_cfg,
+ num_channels_pre_layer[i],
+ num_channels_cur_layer[i],
+ kernel_size=3,
+ stride=1,
+ padding=1,
+ bias=False),
+ build_norm_layer(self.norm_cfg,
+ num_channels_cur_layer[i])[1],
+ nn.ReLU(inplace=True)))
+ else:
+ transition_layers.append(None)
+ else:
+ conv_downsamples = []
+ for j in range(i + 1 - num_branches_pre):
+ in_channels = num_channels_pre_layer[-1]
+ out_channels = num_channels_cur_layer[i] \
+ if j == i - num_branches_pre else in_channels
+ conv_downsamples.append(
+ nn.Sequential(
+ build_conv_layer(
+ self.conv_cfg,
+ in_channels,
+ out_channels,
+ kernel_size=3,
+ stride=2,
+ padding=1,
+ bias=False),
+ build_norm_layer(self.norm_cfg, out_channels)[1],
+ nn.ReLU(inplace=True)))
+ transition_layers.append(nn.Sequential(*conv_downsamples))
+
+ return nn.ModuleList(transition_layers)
+
+ def _make_layer(self, block, in_channels, out_channels, blocks, stride=1):
+ """Make layer."""
+ downsample = None
+ if stride != 1 or in_channels != out_channels:
+ downsample = nn.Sequential(
+ build_conv_layer(
+ self.conv_cfg,
+ in_channels,
+ out_channels,
+ kernel_size=1,
+ stride=stride,
+ bias=False),
+ build_norm_layer(self.norm_cfg, out_channels)[1])
+
+ layers = []
+ layers.append(
+ block(
+ in_channels,
+ out_channels,
+ stride=stride,
+ downsample=downsample,
+ with_cp=self.with_cp,
+ norm_cfg=self.norm_cfg,
+ conv_cfg=self.conv_cfg))
+ for _ in range(1, blocks):
+ layers.append(
+ block(
+ out_channels,
+ out_channels,
+ with_cp=self.with_cp,
+ norm_cfg=self.norm_cfg,
+ conv_cfg=self.conv_cfg))
+
+ return nn.Sequential(*layers)
+
+ def _make_stage(self, layer_config, in_channels, multiscale_output=True):
+ """Make stage."""
+ num_modules = layer_config['num_modules']
+ num_branches = layer_config['num_branches']
+ num_blocks = layer_config['num_blocks']
+ num_channels = layer_config['num_channels']
+ block = self.blocks_dict[layer_config['block']]
+
+ hr_modules = []
+ for i in range(num_modules):
+ # multi_scale_output is only used for the last module
+ if not multiscale_output and i == num_modules - 1:
+ reset_multiscale_output = False
+ else:
+ reset_multiscale_output = True
+
+ hr_modules.append(
+ HRModule(
+ num_branches,
+ block,
+ num_blocks,
+ in_channels,
+ num_channels,
+ reset_multiscale_output,
+ with_cp=self.with_cp,
+ norm_cfg=self.norm_cfg,
+ conv_cfg=self.conv_cfg,
+ upsample_cfg=self.upsample_cfg))
+
+ in_channels = hr_modules[-1].in_channels
+
+ return nn.Sequential(*hr_modules), in_channels
+
+ def _freeze_stages(self):
+ """Freeze parameters."""
+ if self.frozen_stages >= 0:
+ self.norm1.eval()
+ self.norm2.eval()
+
+ for m in [self.conv1, self.norm1, self.conv2, self.norm2]:
+ for param in m.parameters():
+ param.requires_grad = False
+
+ for i in range(1, self.frozen_stages + 1):
+ if i == 1:
+ m = getattr(self, 'layer1')
+ else:
+ m = getattr(self, f'stage{i}')
+
+ m.eval()
+ for param in m.parameters():
+ param.requires_grad = False
+
+ if i < 4:
+ m = getattr(self, f'transition{i}')
+ m.eval()
+ for param in m.parameters():
+ param.requires_grad = False
+
+ def init_weights(self, pretrained=None):
+ """Initialize the weights in backbone.
+
+ Args:
+ pretrained (str, optional): Path to pre-trained weights.
+ Defaults to None.
+ """
+ if isinstance(pretrained, str):
+ logger = get_root_logger()
+ load_checkpoint(self, pretrained, strict=False, logger=logger)
+ elif pretrained is None:
+ for m in self.modules():
+ if isinstance(m, nn.Conv2d):
+ normal_init(m, std=0.001)
+ elif isinstance(m, (_BatchNorm, nn.GroupNorm)):
+ constant_init(m, 1)
+
+ if self.zero_init_residual:
+ for m in self.modules():
+ if isinstance(m, Bottleneck):
+ constant_init(m.norm3, 0)
+ elif isinstance(m, BasicBlock):
+ constant_init(m.norm2, 0)
+ else:
+ raise TypeError('pretrained must be a str or None')
+
+ def forward(self, x):
+ """Forward function."""
+ x = self.conv1(x)
+ x = self.norm1(x)
+ x = self.relu(x)
+ x = self.conv2(x)
+ x = self.norm2(x)
+ x = self.relu(x)
+ x = self.layer1(x)
+
+ x_list = []
+ for i in range(self.stage2_cfg['num_branches']):
+ if self.transition1[i] is not None:
+ x_list.append(self.transition1[i](x))
+ else:
+ x_list.append(x)
+ y_list = self.stage2(x_list)
+
+ x_list = []
+ for i in range(self.stage3_cfg['num_branches']):
+ if self.transition2[i] is not None:
+ x_list.append(self.transition2[i](y_list[-1]))
+ else:
+ x_list.append(y_list[i])
+ y_list = self.stage3(x_list)
+
+ x_list = []
+ for i in range(self.stage4_cfg['num_branches']):
+ if self.transition3[i] is not None:
+ x_list.append(self.transition3[i](y_list[-1]))
+ else:
+ x_list.append(y_list[i])
+ y_list = self.stage4(x_list)
+
+ return y_list
+
+ def train(self, mode=True):
+ """Convert the model into training mode."""
+ super().train(mode)
+ self._freeze_stages()
+ if mode and self.norm_eval:
+ for m in self.modules():
+ if isinstance(m, _BatchNorm):
+ m.eval()
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/litehrnet.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/litehrnet.py
new file mode 100644
index 0000000..9543688
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/litehrnet.py
@@ -0,0 +1,984 @@
+# ------------------------------------------------------------------------------
+# Adapted from https://github.com/HRNet/Lite-HRNet
+# Original licence: Apache License 2.0.
+# ------------------------------------------------------------------------------
+
+import mmcv
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+import torch.utils.checkpoint as cp
+from mmcv.cnn import (ConvModule, DepthwiseSeparableConvModule,
+ build_conv_layer, build_norm_layer, constant_init,
+ normal_init)
+from torch.nn.modules.batchnorm import _BatchNorm
+
+from mmpose.utils import get_root_logger
+from ..builder import BACKBONES
+from .utils import channel_shuffle, load_checkpoint
+
+
+class SpatialWeighting(nn.Module):
+ """Spatial weighting module.
+
+ Args:
+ channels (int): The channels of the module.
+ ratio (int): channel reduction ratio.
+ conv_cfg (dict): Config dict for convolution layer.
+ Default: None, which means using conv2d.
+ norm_cfg (dict): Config dict for normalization layer.
+ Default: None.
+ act_cfg (dict): Config dict for activation layer.
+ Default: (dict(type='ReLU'), dict(type='Sigmoid')).
+ The last ConvModule uses Sigmoid by default.
+ """
+
+ def __init__(self,
+ channels,
+ ratio=16,
+ conv_cfg=None,
+ norm_cfg=None,
+ act_cfg=(dict(type='ReLU'), dict(type='Sigmoid'))):
+ super().__init__()
+ if isinstance(act_cfg, dict):
+ act_cfg = (act_cfg, act_cfg)
+ assert len(act_cfg) == 2
+ assert mmcv.is_tuple_of(act_cfg, dict)
+ self.global_avgpool = nn.AdaptiveAvgPool2d(1)
+ self.conv1 = ConvModule(
+ in_channels=channels,
+ out_channels=int(channels / ratio),
+ kernel_size=1,
+ stride=1,
+ conv_cfg=conv_cfg,
+ norm_cfg=norm_cfg,
+ act_cfg=act_cfg[0])
+ self.conv2 = ConvModule(
+ in_channels=int(channels / ratio),
+ out_channels=channels,
+ kernel_size=1,
+ stride=1,
+ conv_cfg=conv_cfg,
+ norm_cfg=norm_cfg,
+ act_cfg=act_cfg[1])
+
+ def forward(self, x):
+ out = self.global_avgpool(x)
+ out = self.conv1(out)
+ out = self.conv2(out)
+ return x * out
+
+
+class CrossResolutionWeighting(nn.Module):
+ """Cross-resolution channel weighting module.
+
+ Args:
+ channels (int): The channels of the module.
+ ratio (int): channel reduction ratio.
+ conv_cfg (dict): Config dict for convolution layer.
+ Default: None, which means using conv2d.
+ norm_cfg (dict): Config dict for normalization layer.
+ Default: None.
+ act_cfg (dict): Config dict for activation layer.
+ Default: (dict(type='ReLU'), dict(type='Sigmoid')).
+ The last ConvModule uses Sigmoid by default.
+ """
+
+ def __init__(self,
+ channels,
+ ratio=16,
+ conv_cfg=None,
+ norm_cfg=None,
+ act_cfg=(dict(type='ReLU'), dict(type='Sigmoid'))):
+ super().__init__()
+ if isinstance(act_cfg, dict):
+ act_cfg = (act_cfg, act_cfg)
+ assert len(act_cfg) == 2
+ assert mmcv.is_tuple_of(act_cfg, dict)
+ self.channels = channels
+ total_channel = sum(channels)
+ self.conv1 = ConvModule(
+ in_channels=total_channel,
+ out_channels=int(total_channel / ratio),
+ kernel_size=1,
+ stride=1,
+ conv_cfg=conv_cfg,
+ norm_cfg=norm_cfg,
+ act_cfg=act_cfg[0])
+ self.conv2 = ConvModule(
+ in_channels=int(total_channel / ratio),
+ out_channels=total_channel,
+ kernel_size=1,
+ stride=1,
+ conv_cfg=conv_cfg,
+ norm_cfg=norm_cfg,
+ act_cfg=act_cfg[1])
+
+ def forward(self, x):
+ mini_size = x[-1].size()[-2:]
+ out = [F.adaptive_avg_pool2d(s, mini_size) for s in x[:-1]] + [x[-1]]
+ out = torch.cat(out, dim=1)
+ out = self.conv1(out)
+ out = self.conv2(out)
+ out = torch.split(out, self.channels, dim=1)
+ out = [
+ s * F.interpolate(a, size=s.size()[-2:], mode='nearest')
+ for s, a in zip(x, out)
+ ]
+ return out
+
+
+class ConditionalChannelWeighting(nn.Module):
+ """Conditional channel weighting block.
+
+ Args:
+ in_channels (int): The input channels of the block.
+ stride (int): Stride of the 3x3 convolution layer.
+ reduce_ratio (int): channel reduction ratio.
+ conv_cfg (dict): Config dict for convolution layer.
+ Default: None, which means using conv2d.
+ norm_cfg (dict): Config dict for normalization layer.
+ Default: dict(type='BN').
+ with_cp (bool): Use checkpoint or not. Using checkpoint will save some
+ memory while slowing down the training speed. Default: False.
+ """
+
+ def __init__(self,
+ in_channels,
+ stride,
+ reduce_ratio,
+ conv_cfg=None,
+ norm_cfg=dict(type='BN'),
+ with_cp=False):
+ super().__init__()
+ self.with_cp = with_cp
+ self.stride = stride
+ assert stride in [1, 2]
+
+ branch_channels = [channel // 2 for channel in in_channels]
+
+ self.cross_resolution_weighting = CrossResolutionWeighting(
+ branch_channels,
+ ratio=reduce_ratio,
+ conv_cfg=conv_cfg,
+ norm_cfg=norm_cfg)
+
+ self.depthwise_convs = nn.ModuleList([
+ ConvModule(
+ channel,
+ channel,
+ kernel_size=3,
+ stride=self.stride,
+ padding=1,
+ groups=channel,
+ conv_cfg=conv_cfg,
+ norm_cfg=norm_cfg,
+ act_cfg=None) for channel in branch_channels
+ ])
+
+ self.spatial_weighting = nn.ModuleList([
+ SpatialWeighting(channels=channel, ratio=4)
+ for channel in branch_channels
+ ])
+
+ def forward(self, x):
+
+ def _inner_forward(x):
+ x = [s.chunk(2, dim=1) for s in x]
+ x1 = [s[0] for s in x]
+ x2 = [s[1] for s in x]
+
+ x2 = self.cross_resolution_weighting(x2)
+ x2 = [dw(s) for s, dw in zip(x2, self.depthwise_convs)]
+ x2 = [sw(s) for s, sw in zip(x2, self.spatial_weighting)]
+
+ out = [torch.cat([s1, s2], dim=1) for s1, s2 in zip(x1, x2)]
+ out = [channel_shuffle(s, 2) for s in out]
+
+ return out
+
+ if self.with_cp and x.requires_grad:
+ out = cp.checkpoint(_inner_forward, x)
+ else:
+ out = _inner_forward(x)
+
+ return out
+
+
+class Stem(nn.Module):
+ """Stem network block.
+
+ Args:
+ in_channels (int): The input channels of the block.
+ stem_channels (int): Output channels of the stem layer.
+ out_channels (int): The output channels of the block.
+ expand_ratio (int): adjusts number of channels of the hidden layer
+ in InvertedResidual by this amount.
+ conv_cfg (dict): Config dict for convolution layer.
+ Default: None, which means using conv2d.
+ norm_cfg (dict): Config dict for normalization layer.
+ Default: dict(type='BN').
+ with_cp (bool): Use checkpoint or not. Using checkpoint will save some
+ memory while slowing down the training speed. Default: False.
+ """
+
+ def __init__(self,
+ in_channels,
+ stem_channels,
+ out_channels,
+ expand_ratio,
+ conv_cfg=None,
+ norm_cfg=dict(type='BN'),
+ with_cp=False):
+ super().__init__()
+ self.in_channels = in_channels
+ self.out_channels = out_channels
+ self.conv_cfg = conv_cfg
+ self.norm_cfg = norm_cfg
+ self.with_cp = with_cp
+
+ self.conv1 = ConvModule(
+ in_channels=in_channels,
+ out_channels=stem_channels,
+ kernel_size=3,
+ stride=2,
+ padding=1,
+ conv_cfg=self.conv_cfg,
+ norm_cfg=self.norm_cfg,
+ act_cfg=dict(type='ReLU'))
+
+ mid_channels = int(round(stem_channels * expand_ratio))
+ branch_channels = stem_channels // 2
+ if stem_channels == self.out_channels:
+ inc_channels = self.out_channels - branch_channels
+ else:
+ inc_channels = self.out_channels - stem_channels
+
+ self.branch1 = nn.Sequential(
+ ConvModule(
+ branch_channels,
+ branch_channels,
+ kernel_size=3,
+ stride=2,
+ padding=1,
+ groups=branch_channels,
+ conv_cfg=conv_cfg,
+ norm_cfg=norm_cfg,
+ act_cfg=None),
+ ConvModule(
+ branch_channels,
+ inc_channels,
+ kernel_size=1,
+ stride=1,
+ padding=0,
+ conv_cfg=conv_cfg,
+ norm_cfg=norm_cfg,
+ act_cfg=dict(type='ReLU')),
+ )
+
+ self.expand_conv = ConvModule(
+ branch_channels,
+ mid_channels,
+ kernel_size=1,
+ stride=1,
+ padding=0,
+ conv_cfg=conv_cfg,
+ norm_cfg=norm_cfg,
+ act_cfg=dict(type='ReLU'))
+ self.depthwise_conv = ConvModule(
+ mid_channels,
+ mid_channels,
+ kernel_size=3,
+ stride=2,
+ padding=1,
+ groups=mid_channels,
+ conv_cfg=conv_cfg,
+ norm_cfg=norm_cfg,
+ act_cfg=None)
+ self.linear_conv = ConvModule(
+ mid_channels,
+ branch_channels
+ if stem_channels == self.out_channels else stem_channels,
+ kernel_size=1,
+ stride=1,
+ padding=0,
+ conv_cfg=conv_cfg,
+ norm_cfg=norm_cfg,
+ act_cfg=dict(type='ReLU'))
+
+ def forward(self, x):
+
+ def _inner_forward(x):
+ x = self.conv1(x)
+ x1, x2 = x.chunk(2, dim=1)
+
+ x2 = self.expand_conv(x2)
+ x2 = self.depthwise_conv(x2)
+ x2 = self.linear_conv(x2)
+
+ out = torch.cat((self.branch1(x1), x2), dim=1)
+
+ out = channel_shuffle(out, 2)
+
+ return out
+
+ if self.with_cp and x.requires_grad:
+ out = cp.checkpoint(_inner_forward, x)
+ else:
+ out = _inner_forward(x)
+
+ return out
+
+
+class IterativeHead(nn.Module):
+ """Extra iterative head for feature learning.
+
+ Args:
+ in_channels (int): The input channels of the block.
+ norm_cfg (dict): Config dict for normalization layer.
+ Default: dict(type='BN').
+ """
+
+ def __init__(self, in_channels, norm_cfg=dict(type='BN')):
+ super().__init__()
+ projects = []
+ num_branchs = len(in_channels)
+ self.in_channels = in_channels[::-1]
+
+ for i in range(num_branchs):
+ if i != num_branchs - 1:
+ projects.append(
+ DepthwiseSeparableConvModule(
+ in_channels=self.in_channels[i],
+ out_channels=self.in_channels[i + 1],
+ kernel_size=3,
+ stride=1,
+ padding=1,
+ norm_cfg=norm_cfg,
+ act_cfg=dict(type='ReLU'),
+ dw_act_cfg=None,
+ pw_act_cfg=dict(type='ReLU')))
+ else:
+ projects.append(
+ DepthwiseSeparableConvModule(
+ in_channels=self.in_channels[i],
+ out_channels=self.in_channels[i],
+ kernel_size=3,
+ stride=1,
+ padding=1,
+ norm_cfg=norm_cfg,
+ act_cfg=dict(type='ReLU'),
+ dw_act_cfg=None,
+ pw_act_cfg=dict(type='ReLU')))
+ self.projects = nn.ModuleList(projects)
+
+ def forward(self, x):
+ x = x[::-1]
+
+ y = []
+ last_x = None
+ for i, s in enumerate(x):
+ if last_x is not None:
+ last_x = F.interpolate(
+ last_x,
+ size=s.size()[-2:],
+ mode='bilinear',
+ align_corners=True)
+ s = s + last_x
+ s = self.projects[i](s)
+ y.append(s)
+ last_x = s
+
+ return y[::-1]
+
+
+class ShuffleUnit(nn.Module):
+ """InvertedResidual block for ShuffleNetV2 backbone.
+
+ Args:
+ in_channels (int): The input channels of the block.
+ out_channels (int): The output channels of the block.
+ stride (int): Stride of the 3x3 convolution layer. Default: 1
+ conv_cfg (dict): Config dict for convolution layer.
+ Default: None, which means using conv2d.
+ norm_cfg (dict): Config dict for normalization layer.
+ Default: dict(type='BN').
+ act_cfg (dict): Config dict for activation layer.
+ Default: dict(type='ReLU').
+ with_cp (bool): Use checkpoint or not. Using checkpoint will save some
+ memory while slowing down the training speed. Default: False.
+ """
+
+ def __init__(self,
+ in_channels,
+ out_channels,
+ stride=1,
+ conv_cfg=None,
+ norm_cfg=dict(type='BN'),
+ act_cfg=dict(type='ReLU'),
+ with_cp=False):
+ super().__init__()
+ self.stride = stride
+ self.with_cp = with_cp
+
+ branch_features = out_channels // 2
+ if self.stride == 1:
+ assert in_channels == branch_features * 2, (
+ f'in_channels ({in_channels}) should equal to '
+ f'branch_features * 2 ({branch_features * 2}) '
+ 'when stride is 1')
+
+ if in_channels != branch_features * 2:
+ assert self.stride != 1, (
+ f'stride ({self.stride}) should not equal 1 when '
+ f'in_channels != branch_features * 2')
+
+ if self.stride > 1:
+ self.branch1 = nn.Sequential(
+ ConvModule(
+ in_channels,
+ in_channels,
+ kernel_size=3,
+ stride=self.stride,
+ padding=1,
+ groups=in_channels,
+ conv_cfg=conv_cfg,
+ norm_cfg=norm_cfg,
+ act_cfg=None),
+ ConvModule(
+ in_channels,
+ branch_features,
+ kernel_size=1,
+ stride=1,
+ padding=0,
+ conv_cfg=conv_cfg,
+ norm_cfg=norm_cfg,
+ act_cfg=act_cfg),
+ )
+
+ self.branch2 = nn.Sequential(
+ ConvModule(
+ in_channels if (self.stride > 1) else branch_features,
+ branch_features,
+ kernel_size=1,
+ stride=1,
+ padding=0,
+ conv_cfg=conv_cfg,
+ norm_cfg=norm_cfg,
+ act_cfg=act_cfg),
+ ConvModule(
+ branch_features,
+ branch_features,
+ kernel_size=3,
+ stride=self.stride,
+ padding=1,
+ groups=branch_features,
+ conv_cfg=conv_cfg,
+ norm_cfg=norm_cfg,
+ act_cfg=None),
+ ConvModule(
+ branch_features,
+ branch_features,
+ kernel_size=1,
+ stride=1,
+ padding=0,
+ conv_cfg=conv_cfg,
+ norm_cfg=norm_cfg,
+ act_cfg=act_cfg))
+
+ def forward(self, x):
+
+ def _inner_forward(x):
+ if self.stride > 1:
+ out = torch.cat((self.branch1(x), self.branch2(x)), dim=1)
+ else:
+ x1, x2 = x.chunk(2, dim=1)
+ out = torch.cat((x1, self.branch2(x2)), dim=1)
+
+ out = channel_shuffle(out, 2)
+
+ return out
+
+ if self.with_cp and x.requires_grad:
+ out = cp.checkpoint(_inner_forward, x)
+ else:
+ out = _inner_forward(x)
+
+ return out
+
+
+class LiteHRModule(nn.Module):
+ """High-Resolution Module for LiteHRNet.
+
+ It contains conditional channel weighting blocks and
+ shuffle blocks.
+
+
+ Args:
+ num_branches (int): Number of branches in the module.
+ num_blocks (int): Number of blocks in the module.
+ in_channels (list(int)): Number of input image channels.
+ reduce_ratio (int): Channel reduction ratio.
+ module_type (str): 'LITE' or 'NAIVE'
+ multiscale_output (bool): Whether to output multi-scale features.
+ with_fuse (bool): Whether to use fuse layers.
+ conv_cfg (dict): dictionary to construct and config conv layer.
+ norm_cfg (dict): dictionary to construct and config norm layer.
+ with_cp (bool): Use checkpoint or not. Using checkpoint will save some
+ memory while slowing down the training speed.
+ """
+
+ def __init__(
+ self,
+ num_branches,
+ num_blocks,
+ in_channels,
+ reduce_ratio,
+ module_type,
+ multiscale_output=False,
+ with_fuse=True,
+ conv_cfg=None,
+ norm_cfg=dict(type='BN'),
+ with_cp=False,
+ ):
+ super().__init__()
+ self._check_branches(num_branches, in_channels)
+
+ self.in_channels = in_channels
+ self.num_branches = num_branches
+
+ self.module_type = module_type
+ self.multiscale_output = multiscale_output
+ self.with_fuse = with_fuse
+ self.norm_cfg = norm_cfg
+ self.conv_cfg = conv_cfg
+ self.with_cp = with_cp
+
+ if self.module_type.upper() == 'LITE':
+ self.layers = self._make_weighting_blocks(num_blocks, reduce_ratio)
+ elif self.module_type.upper() == 'NAIVE':
+ self.layers = self._make_naive_branches(num_branches, num_blocks)
+ else:
+ raise ValueError("module_type should be either 'LITE' or 'NAIVE'.")
+ if self.with_fuse:
+ self.fuse_layers = self._make_fuse_layers()
+ self.relu = nn.ReLU()
+
+ def _check_branches(self, num_branches, in_channels):
+ """Check input to avoid ValueError."""
+ if num_branches != len(in_channels):
+ error_msg = f'NUM_BRANCHES({num_branches}) ' \
+ f'!= NUM_INCHANNELS({len(in_channels)})'
+ raise ValueError(error_msg)
+
+ def _make_weighting_blocks(self, num_blocks, reduce_ratio, stride=1):
+ """Make channel weighting blocks."""
+ layers = []
+ for i in range(num_blocks):
+ layers.append(
+ ConditionalChannelWeighting(
+ self.in_channels,
+ stride=stride,
+ reduce_ratio=reduce_ratio,
+ conv_cfg=self.conv_cfg,
+ norm_cfg=self.norm_cfg,
+ with_cp=self.with_cp))
+
+ return nn.Sequential(*layers)
+
+ def _make_one_branch(self, branch_index, num_blocks, stride=1):
+ """Make one branch."""
+ layers = []
+ layers.append(
+ ShuffleUnit(
+ self.in_channels[branch_index],
+ self.in_channels[branch_index],
+ stride=stride,
+ conv_cfg=self.conv_cfg,
+ norm_cfg=self.norm_cfg,
+ act_cfg=dict(type='ReLU'),
+ with_cp=self.with_cp))
+ for i in range(1, num_blocks):
+ layers.append(
+ ShuffleUnit(
+ self.in_channels[branch_index],
+ self.in_channels[branch_index],
+ stride=1,
+ conv_cfg=self.conv_cfg,
+ norm_cfg=self.norm_cfg,
+ act_cfg=dict(type='ReLU'),
+ with_cp=self.with_cp))
+
+ return nn.Sequential(*layers)
+
+ def _make_naive_branches(self, num_branches, num_blocks):
+ """Make branches."""
+ branches = []
+
+ for i in range(num_branches):
+ branches.append(self._make_one_branch(i, num_blocks))
+
+ return nn.ModuleList(branches)
+
+ def _make_fuse_layers(self):
+ """Make fuse layer."""
+ if self.num_branches == 1:
+ return None
+
+ num_branches = self.num_branches
+ in_channels = self.in_channels
+ fuse_layers = []
+ num_out_branches = num_branches if self.multiscale_output else 1
+ for i in range(num_out_branches):
+ fuse_layer = []
+ for j in range(num_branches):
+ if j > i:
+ fuse_layer.append(
+ nn.Sequential(
+ build_conv_layer(
+ self.conv_cfg,
+ in_channels[j],
+ in_channels[i],
+ kernel_size=1,
+ stride=1,
+ padding=0,
+ bias=False),
+ build_norm_layer(self.norm_cfg, in_channels[i])[1],
+ nn.Upsample(
+ scale_factor=2**(j - i), mode='nearest')))
+ elif j == i:
+ fuse_layer.append(None)
+ else:
+ conv_downsamples = []
+ for k in range(i - j):
+ if k == i - j - 1:
+ conv_downsamples.append(
+ nn.Sequential(
+ build_conv_layer(
+ self.conv_cfg,
+ in_channels[j],
+ in_channels[j],
+ kernel_size=3,
+ stride=2,
+ padding=1,
+ groups=in_channels[j],
+ bias=False),
+ build_norm_layer(self.norm_cfg,
+ in_channels[j])[1],
+ build_conv_layer(
+ self.conv_cfg,
+ in_channels[j],
+ in_channels[i],
+ kernel_size=1,
+ stride=1,
+ padding=0,
+ bias=False),
+ build_norm_layer(self.norm_cfg,
+ in_channels[i])[1]))
+ else:
+ conv_downsamples.append(
+ nn.Sequential(
+ build_conv_layer(
+ self.conv_cfg,
+ in_channels[j],
+ in_channels[j],
+ kernel_size=3,
+ stride=2,
+ padding=1,
+ groups=in_channels[j],
+ bias=False),
+ build_norm_layer(self.norm_cfg,
+ in_channels[j])[1],
+ build_conv_layer(
+ self.conv_cfg,
+ in_channels[j],
+ in_channels[j],
+ kernel_size=1,
+ stride=1,
+ padding=0,
+ bias=False),
+ build_norm_layer(self.norm_cfg,
+ in_channels[j])[1],
+ nn.ReLU(inplace=True)))
+ fuse_layer.append(nn.Sequential(*conv_downsamples))
+ fuse_layers.append(nn.ModuleList(fuse_layer))
+
+ return nn.ModuleList(fuse_layers)
+
+ def forward(self, x):
+ """Forward function."""
+ if self.num_branches == 1:
+ return [self.layers[0](x[0])]
+
+ if self.module_type.upper() == 'LITE':
+ out = self.layers(x)
+ elif self.module_type.upper() == 'NAIVE':
+ for i in range(self.num_branches):
+ x[i] = self.layers[i](x[i])
+ out = x
+
+ if self.with_fuse:
+ out_fuse = []
+ for i in range(len(self.fuse_layers)):
+ # `y = 0` will lead to decreased accuracy (0.5~1 mAP)
+ y = out[0] if i == 0 else self.fuse_layers[i][0](out[0])
+ for j in range(self.num_branches):
+ if i == j:
+ y += out[j]
+ else:
+ y += self.fuse_layers[i][j](out[j])
+ out_fuse.append(self.relu(y))
+ out = out_fuse
+ if not self.multiscale_output:
+ out = [out[0]]
+ return out
+
+
+@BACKBONES.register_module()
+class LiteHRNet(nn.Module):
+ """Lite-HRNet backbone.
+
+ `Lite-HRNet: A Lightweight High-Resolution Network
+ `_.
+
+ Code adapted from 'https://github.com/HRNet/Lite-HRNet'.
+
+ Args:
+ extra (dict): detailed configuration for each stage of HRNet.
+ in_channels (int): Number of input image channels. Default: 3.
+ conv_cfg (dict): dictionary to construct and config conv layer.
+ norm_cfg (dict): dictionary to construct and config norm layer.
+ norm_eval (bool): Whether to set norm layers to eval mode, namely,
+ freeze running stats (mean and var). Note: Effect on Batch Norm
+ and its variants only. Default: False
+ with_cp (bool): Use checkpoint or not. Using checkpoint will save some
+ memory while slowing down the training speed.
+
+ Example:
+ >>> from mmpose.models import LiteHRNet
+ >>> import torch
+ >>> extra=dict(
+ >>> stem=dict(stem_channels=32, out_channels=32, expand_ratio=1),
+ >>> num_stages=3,
+ >>> stages_spec=dict(
+ >>> num_modules=(2, 4, 2),
+ >>> num_branches=(2, 3, 4),
+ >>> num_blocks=(2, 2, 2),
+ >>> module_type=('LITE', 'LITE', 'LITE'),
+ >>> with_fuse=(True, True, True),
+ >>> reduce_ratios=(8, 8, 8),
+ >>> num_channels=(
+ >>> (40, 80),
+ >>> (40, 80, 160),
+ >>> (40, 80, 160, 320),
+ >>> )),
+ >>> with_head=False)
+ >>> self = LiteHRNet(extra, in_channels=1)
+ >>> self.eval()
+ >>> inputs = torch.rand(1, 1, 32, 32)
+ >>> level_outputs = self.forward(inputs)
+ >>> for level_out in level_outputs:
+ ... print(tuple(level_out.shape))
+ (1, 40, 8, 8)
+ """
+
+ def __init__(self,
+ extra,
+ in_channels=3,
+ conv_cfg=None,
+ norm_cfg=dict(type='BN'),
+ norm_eval=False,
+ with_cp=False):
+ super().__init__()
+ self.extra = extra
+ self.conv_cfg = conv_cfg
+ self.norm_cfg = norm_cfg
+ self.norm_eval = norm_eval
+ self.with_cp = with_cp
+
+ self.stem = Stem(
+ in_channels,
+ stem_channels=self.extra['stem']['stem_channels'],
+ out_channels=self.extra['stem']['out_channels'],
+ expand_ratio=self.extra['stem']['expand_ratio'],
+ conv_cfg=self.conv_cfg,
+ norm_cfg=self.norm_cfg)
+
+ self.num_stages = self.extra['num_stages']
+ self.stages_spec = self.extra['stages_spec']
+
+ num_channels_last = [
+ self.stem.out_channels,
+ ]
+ for i in range(self.num_stages):
+ num_channels = self.stages_spec['num_channels'][i]
+ num_channels = [num_channels[i] for i in range(len(num_channels))]
+ setattr(
+ self, f'transition{i}',
+ self._make_transition_layer(num_channels_last, num_channels))
+
+ stage, num_channels_last = self._make_stage(
+ self.stages_spec, i, num_channels, multiscale_output=True)
+ setattr(self, f'stage{i}', stage)
+
+ self.with_head = self.extra['with_head']
+ if self.with_head:
+ self.head_layer = IterativeHead(
+ in_channels=num_channels_last,
+ norm_cfg=self.norm_cfg,
+ )
+
+ def _make_transition_layer(self, num_channels_pre_layer,
+ num_channels_cur_layer):
+ """Make transition layer."""
+ num_branches_cur = len(num_channels_cur_layer)
+ num_branches_pre = len(num_channels_pre_layer)
+
+ transition_layers = []
+ for i in range(num_branches_cur):
+ if i < num_branches_pre:
+ if num_channels_cur_layer[i] != num_channels_pre_layer[i]:
+ transition_layers.append(
+ nn.Sequential(
+ build_conv_layer(
+ self.conv_cfg,
+ num_channels_pre_layer[i],
+ num_channels_pre_layer[i],
+ kernel_size=3,
+ stride=1,
+ padding=1,
+ groups=num_channels_pre_layer[i],
+ bias=False),
+ build_norm_layer(self.norm_cfg,
+ num_channels_pre_layer[i])[1],
+ build_conv_layer(
+ self.conv_cfg,
+ num_channels_pre_layer[i],
+ num_channels_cur_layer[i],
+ kernel_size=1,
+ stride=1,
+ padding=0,
+ bias=False),
+ build_norm_layer(self.norm_cfg,
+ num_channels_cur_layer[i])[1],
+ nn.ReLU()))
+ else:
+ transition_layers.append(None)
+ else:
+ conv_downsamples = []
+ for j in range(i + 1 - num_branches_pre):
+ in_channels = num_channels_pre_layer[-1]
+ out_channels = num_channels_cur_layer[i] \
+ if j == i - num_branches_pre else in_channels
+ conv_downsamples.append(
+ nn.Sequential(
+ build_conv_layer(
+ self.conv_cfg,
+ in_channels,
+ in_channels,
+ kernel_size=3,
+ stride=2,
+ padding=1,
+ groups=in_channels,
+ bias=False),
+ build_norm_layer(self.norm_cfg, in_channels)[1],
+ build_conv_layer(
+ self.conv_cfg,
+ in_channels,
+ out_channels,
+ kernel_size=1,
+ stride=1,
+ padding=0,
+ bias=False),
+ build_norm_layer(self.norm_cfg, out_channels)[1],
+ nn.ReLU()))
+ transition_layers.append(nn.Sequential(*conv_downsamples))
+
+ return nn.ModuleList(transition_layers)
+
+ def _make_stage(self,
+ stages_spec,
+ stage_index,
+ in_channels,
+ multiscale_output=True):
+ num_modules = stages_spec['num_modules'][stage_index]
+ num_branches = stages_spec['num_branches'][stage_index]
+ num_blocks = stages_spec['num_blocks'][stage_index]
+ reduce_ratio = stages_spec['reduce_ratios'][stage_index]
+ with_fuse = stages_spec['with_fuse'][stage_index]
+ module_type = stages_spec['module_type'][stage_index]
+
+ modules = []
+ for i in range(num_modules):
+ # multi_scale_output is only used last module
+ if not multiscale_output and i == num_modules - 1:
+ reset_multiscale_output = False
+ else:
+ reset_multiscale_output = True
+
+ modules.append(
+ LiteHRModule(
+ num_branches,
+ num_blocks,
+ in_channels,
+ reduce_ratio,
+ module_type,
+ multiscale_output=reset_multiscale_output,
+ with_fuse=with_fuse,
+ conv_cfg=self.conv_cfg,
+ norm_cfg=self.norm_cfg,
+ with_cp=self.with_cp))
+ in_channels = modules[-1].in_channels
+
+ return nn.Sequential(*modules), in_channels
+
+ def init_weights(self, pretrained=None):
+ """Initialize the weights in backbone.
+
+ Args:
+ pretrained (str, optional): Path to pre-trained weights.
+ Defaults to None.
+ """
+ if isinstance(pretrained, str):
+ logger = get_root_logger()
+ load_checkpoint(self, pretrained, strict=False, logger=logger)
+ elif pretrained is None:
+ for m in self.modules():
+ if isinstance(m, nn.Conv2d):
+ normal_init(m, std=0.001)
+ elif isinstance(m, (_BatchNorm, nn.GroupNorm)):
+ constant_init(m, 1)
+ else:
+ raise TypeError('pretrained must be a str or None')
+
+ def forward(self, x):
+ """Forward function."""
+ x = self.stem(x)
+
+ y_list = [x]
+ for i in range(self.num_stages):
+ x_list = []
+ transition = getattr(self, f'transition{i}')
+ for j in range(self.stages_spec['num_branches'][i]):
+ if transition[j]:
+ if j >= len(y_list):
+ x_list.append(transition[j](y_list[-1]))
+ else:
+ x_list.append(transition[j](y_list[j]))
+ else:
+ x_list.append(y_list[j])
+ y_list = getattr(self, f'stage{i}')(x_list)
+
+ x = y_list
+ if self.with_head:
+ x = self.head_layer(x)
+
+ return [x[0]]
+
+ def train(self, mode=True):
+ """Convert the model into training mode."""
+ super().train(mode)
+ if mode and self.norm_eval:
+ for m in self.modules():
+ if isinstance(m, _BatchNorm):
+ m.eval()
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/mobilenet_v2.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/mobilenet_v2.py
new file mode 100644
index 0000000..5dc0cd1
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/mobilenet_v2.py
@@ -0,0 +1,275 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import copy
+import logging
+
+import torch.nn as nn
+import torch.utils.checkpoint as cp
+from mmcv.cnn import ConvModule, constant_init, kaiming_init
+from torch.nn.modules.batchnorm import _BatchNorm
+
+from ..builder import BACKBONES
+from .base_backbone import BaseBackbone
+from .utils import load_checkpoint, make_divisible
+
+
+class InvertedResidual(nn.Module):
+ """InvertedResidual block for MobileNetV2.
+
+ Args:
+ in_channels (int): The input channels of the InvertedResidual block.
+ out_channels (int): The output channels of the InvertedResidual block.
+ stride (int): Stride of the middle (first) 3x3 convolution.
+ expand_ratio (int): adjusts number of channels of the hidden layer
+ in InvertedResidual by this amount.
+ conv_cfg (dict): Config dict for convolution layer.
+ Default: None, which means using conv2d.
+ norm_cfg (dict): Config dict for normalization layer.
+ Default: dict(type='BN').
+ act_cfg (dict): Config dict for activation layer.
+ Default: dict(type='ReLU6').
+ with_cp (bool): Use checkpoint or not. Using checkpoint will save some
+ memory while slowing down the training speed. Default: False.
+ """
+
+ def __init__(self,
+ in_channels,
+ out_channels,
+ stride,
+ expand_ratio,
+ conv_cfg=None,
+ norm_cfg=dict(type='BN'),
+ act_cfg=dict(type='ReLU6'),
+ with_cp=False):
+ # Protect mutable default arguments
+ norm_cfg = copy.deepcopy(norm_cfg)
+ act_cfg = copy.deepcopy(act_cfg)
+ super().__init__()
+ self.stride = stride
+ assert stride in [1, 2], f'stride must in [1, 2]. ' \
+ f'But received {stride}.'
+ self.with_cp = with_cp
+ self.use_res_connect = self.stride == 1 and in_channels == out_channels
+ hidden_dim = int(round(in_channels * expand_ratio))
+
+ layers = []
+ if expand_ratio != 1:
+ layers.append(
+ ConvModule(
+ in_channels=in_channels,
+ out_channels=hidden_dim,
+ kernel_size=1,
+ conv_cfg=conv_cfg,
+ norm_cfg=norm_cfg,
+ act_cfg=act_cfg))
+ layers.extend([
+ ConvModule(
+ in_channels=hidden_dim,
+ out_channels=hidden_dim,
+ kernel_size=3,
+ stride=stride,
+ padding=1,
+ groups=hidden_dim,
+ conv_cfg=conv_cfg,
+ norm_cfg=norm_cfg,
+ act_cfg=act_cfg),
+ ConvModule(
+ in_channels=hidden_dim,
+ out_channels=out_channels,
+ kernel_size=1,
+ conv_cfg=conv_cfg,
+ norm_cfg=norm_cfg,
+ act_cfg=None)
+ ])
+ self.conv = nn.Sequential(*layers)
+
+ def forward(self, x):
+
+ def _inner_forward(x):
+ if self.use_res_connect:
+ return x + self.conv(x)
+ return self.conv(x)
+
+ if self.with_cp and x.requires_grad:
+ out = cp.checkpoint(_inner_forward, x)
+ else:
+ out = _inner_forward(x)
+
+ return out
+
+
+@BACKBONES.register_module()
+class MobileNetV2(BaseBackbone):
+ """MobileNetV2 backbone.
+
+ Args:
+ widen_factor (float): Width multiplier, multiply number of
+ channels in each layer by this amount. Default: 1.0.
+ out_indices (None or Sequence[int]): Output from which stages.
+ Default: (7, ).
+ frozen_stages (int): Stages to be frozen (all param fixed).
+ Default: -1, which means not freezing any parameters.
+ conv_cfg (dict): Config dict for convolution layer.
+ Default: None, which means using conv2d.
+ norm_cfg (dict): Config dict for normalization layer.
+ Default: dict(type='BN').
+ act_cfg (dict): Config dict for activation layer.
+ Default: dict(type='ReLU6').
+ norm_eval (bool): Whether to set norm layers to eval mode, namely,
+ freeze running stats (mean and var). Note: Effect on Batch Norm
+ and its variants only. Default: False.
+ with_cp (bool): Use checkpoint or not. Using checkpoint will save some
+ memory while slowing down the training speed. Default: False.
+ """
+
+ # Parameters to build layers. 4 parameters are needed to construct a
+ # layer, from left to right: expand_ratio, channel, num_blocks, stride.
+ arch_settings = [[1, 16, 1, 1], [6, 24, 2, 2], [6, 32, 3, 2],
+ [6, 64, 4, 2], [6, 96, 3, 1], [6, 160, 3, 2],
+ [6, 320, 1, 1]]
+
+ def __init__(self,
+ widen_factor=1.,
+ out_indices=(7, ),
+ frozen_stages=-1,
+ conv_cfg=None,
+ norm_cfg=dict(type='BN'),
+ act_cfg=dict(type='ReLU6'),
+ norm_eval=False,
+ with_cp=False):
+ # Protect mutable default arguments
+ norm_cfg = copy.deepcopy(norm_cfg)
+ act_cfg = copy.deepcopy(act_cfg)
+ super().__init__()
+ self.widen_factor = widen_factor
+ self.out_indices = out_indices
+ for index in out_indices:
+ if index not in range(0, 8):
+ raise ValueError('the item in out_indices must in '
+ f'range(0, 8). But received {index}')
+
+ if frozen_stages not in range(-1, 8):
+ raise ValueError('frozen_stages must be in range(-1, 8). '
+ f'But received {frozen_stages}')
+ self.out_indices = out_indices
+ self.frozen_stages = frozen_stages
+ self.conv_cfg = conv_cfg
+ self.norm_cfg = norm_cfg
+ self.act_cfg = act_cfg
+ self.norm_eval = norm_eval
+ self.with_cp = with_cp
+
+ self.in_channels = make_divisible(32 * widen_factor, 8)
+
+ self.conv1 = ConvModule(
+ in_channels=3,
+ out_channels=self.in_channels,
+ kernel_size=3,
+ stride=2,
+ padding=1,
+ conv_cfg=self.conv_cfg,
+ norm_cfg=self.norm_cfg,
+ act_cfg=self.act_cfg)
+
+ self.layers = []
+
+ for i, layer_cfg in enumerate(self.arch_settings):
+ expand_ratio, channel, num_blocks, stride = layer_cfg
+ out_channels = make_divisible(channel * widen_factor, 8)
+ inverted_res_layer = self.make_layer(
+ out_channels=out_channels,
+ num_blocks=num_blocks,
+ stride=stride,
+ expand_ratio=expand_ratio)
+ layer_name = f'layer{i + 1}'
+ self.add_module(layer_name, inverted_res_layer)
+ self.layers.append(layer_name)
+
+ if widen_factor > 1.0:
+ self.out_channel = int(1280 * widen_factor)
+ else:
+ self.out_channel = 1280
+
+ layer = ConvModule(
+ in_channels=self.in_channels,
+ out_channels=self.out_channel,
+ kernel_size=1,
+ stride=1,
+ padding=0,
+ conv_cfg=self.conv_cfg,
+ norm_cfg=self.norm_cfg,
+ act_cfg=self.act_cfg)
+ self.add_module('conv2', layer)
+ self.layers.append('conv2')
+
+ def make_layer(self, out_channels, num_blocks, stride, expand_ratio):
+ """Stack InvertedResidual blocks to build a layer for MobileNetV2.
+
+ Args:
+ out_channels (int): out_channels of block.
+ num_blocks (int): number of blocks.
+ stride (int): stride of the first block. Default: 1
+ expand_ratio (int): Expand the number of channels of the
+ hidden layer in InvertedResidual by this ratio. Default: 6.
+ """
+ layers = []
+ for i in range(num_blocks):
+ if i >= 1:
+ stride = 1
+ layers.append(
+ InvertedResidual(
+ self.in_channels,
+ out_channels,
+ stride,
+ expand_ratio=expand_ratio,
+ conv_cfg=self.conv_cfg,
+ norm_cfg=self.norm_cfg,
+ act_cfg=self.act_cfg,
+ with_cp=self.with_cp))
+ self.in_channels = out_channels
+
+ return nn.Sequential(*layers)
+
+ def init_weights(self, pretrained=None):
+ if isinstance(pretrained, str):
+ logger = logging.getLogger()
+ load_checkpoint(self, pretrained, strict=False, logger=logger)
+ elif pretrained is None:
+ for m in self.modules():
+ if isinstance(m, nn.Conv2d):
+ kaiming_init(m)
+ elif isinstance(m, (_BatchNorm, nn.GroupNorm)):
+ constant_init(m, 1)
+ else:
+ raise TypeError('pretrained must be a str or None')
+
+ def forward(self, x):
+ x = self.conv1(x)
+
+ outs = []
+ for i, layer_name in enumerate(self.layers):
+ layer = getattr(self, layer_name)
+ x = layer(x)
+ if i in self.out_indices:
+ outs.append(x)
+
+ if len(outs) == 1:
+ return outs[0]
+ return tuple(outs)
+
+ def _freeze_stages(self):
+ if self.frozen_stages >= 0:
+ for param in self.conv1.parameters():
+ param.requires_grad = False
+ for i in range(1, self.frozen_stages + 1):
+ layer = getattr(self, f'layer{i}')
+ layer.eval()
+ for param in layer.parameters():
+ param.requires_grad = False
+
+ def train(self, mode=True):
+ super().train(mode)
+ self._freeze_stages()
+ if mode and self.norm_eval:
+ for m in self.modules():
+ if isinstance(m, _BatchNorm):
+ m.eval()
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/mobilenet_v3.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/mobilenet_v3.py
new file mode 100644
index 0000000..d640abe
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/mobilenet_v3.py
@@ -0,0 +1,188 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import copy
+import logging
+
+import torch.nn as nn
+from mmcv.cnn import ConvModule, constant_init, kaiming_init
+from torch.nn.modules.batchnorm import _BatchNorm
+
+from ..builder import BACKBONES
+from .base_backbone import BaseBackbone
+from .utils import InvertedResidual, load_checkpoint
+
+
+@BACKBONES.register_module()
+class MobileNetV3(BaseBackbone):
+ """MobileNetV3 backbone.
+
+ Args:
+ arch (str): Architecture of mobilnetv3, from {small, big}.
+ Default: small.
+ conv_cfg (dict): Config dict for convolution layer.
+ Default: None, which means using conv2d.
+ norm_cfg (dict): Config dict for normalization layer.
+ Default: dict(type='BN').
+ out_indices (None or Sequence[int]): Output from which stages.
+ Default: (-1, ), which means output tensors from final stage.
+ frozen_stages (int): Stages to be frozen (all param fixed).
+ Default: -1, which means not freezing any parameters.
+ norm_eval (bool): Whether to set norm layers to eval mode, namely,
+ freeze running stats (mean and var). Note: Effect on Batch Norm
+ and its variants only. Default: False.
+ with_cp (bool): Use checkpoint or not. Using checkpoint will save
+ some memory while slowing down the training speed.
+ Default: False.
+ """
+ # Parameters to build each block:
+ # [kernel size, mid channels, out channels, with_se, act type, stride]
+ arch_settings = {
+ 'small': [[3, 16, 16, True, 'ReLU', 2],
+ [3, 72, 24, False, 'ReLU', 2],
+ [3, 88, 24, False, 'ReLU', 1],
+ [5, 96, 40, True, 'HSwish', 2],
+ [5, 240, 40, True, 'HSwish', 1],
+ [5, 240, 40, True, 'HSwish', 1],
+ [5, 120, 48, True, 'HSwish', 1],
+ [5, 144, 48, True, 'HSwish', 1],
+ [5, 288, 96, True, 'HSwish', 2],
+ [5, 576, 96, True, 'HSwish', 1],
+ [5, 576, 96, True, 'HSwish', 1]],
+ 'big': [[3, 16, 16, False, 'ReLU', 1],
+ [3, 64, 24, False, 'ReLU', 2],
+ [3, 72, 24, False, 'ReLU', 1],
+ [5, 72, 40, True, 'ReLU', 2],
+ [5, 120, 40, True, 'ReLU', 1],
+ [5, 120, 40, True, 'ReLU', 1],
+ [3, 240, 80, False, 'HSwish', 2],
+ [3, 200, 80, False, 'HSwish', 1],
+ [3, 184, 80, False, 'HSwish', 1],
+ [3, 184, 80, False, 'HSwish', 1],
+ [3, 480, 112, True, 'HSwish', 1],
+ [3, 672, 112, True, 'HSwish', 1],
+ [5, 672, 160, True, 'HSwish', 1],
+ [5, 672, 160, True, 'HSwish', 2],
+ [5, 960, 160, True, 'HSwish', 1]]
+ } # yapf: disable
+
+ def __init__(self,
+ arch='small',
+ conv_cfg=None,
+ norm_cfg=dict(type='BN'),
+ out_indices=(-1, ),
+ frozen_stages=-1,
+ norm_eval=False,
+ with_cp=False):
+ # Protect mutable default arguments
+ norm_cfg = copy.deepcopy(norm_cfg)
+ super().__init__()
+ assert arch in self.arch_settings
+ for index in out_indices:
+ if index not in range(-len(self.arch_settings[arch]),
+ len(self.arch_settings[arch])):
+ raise ValueError('the item in out_indices must in '
+ f'range(0, {len(self.arch_settings[arch])}). '
+ f'But received {index}')
+
+ if frozen_stages not in range(-1, len(self.arch_settings[arch])):
+ raise ValueError('frozen_stages must be in range(-1, '
+ f'{len(self.arch_settings[arch])}). '
+ f'But received {frozen_stages}')
+ self.arch = arch
+ self.conv_cfg = conv_cfg
+ self.norm_cfg = norm_cfg
+ self.out_indices = out_indices
+ self.frozen_stages = frozen_stages
+ self.norm_eval = norm_eval
+ self.with_cp = with_cp
+
+ self.in_channels = 16
+ self.conv1 = ConvModule(
+ in_channels=3,
+ out_channels=self.in_channels,
+ kernel_size=3,
+ stride=2,
+ padding=1,
+ conv_cfg=conv_cfg,
+ norm_cfg=norm_cfg,
+ act_cfg=dict(type='HSwish'))
+
+ self.layers = self._make_layer()
+ self.feat_dim = self.arch_settings[arch][-1][2]
+
+ def _make_layer(self):
+ layers = []
+ layer_setting = self.arch_settings[self.arch]
+ for i, params in enumerate(layer_setting):
+ (kernel_size, mid_channels, out_channels, with_se, act,
+ stride) = params
+ if with_se:
+ se_cfg = dict(
+ channels=mid_channels,
+ ratio=4,
+ act_cfg=(dict(type='ReLU'), dict(type='HSigmoid')))
+ else:
+ se_cfg = None
+
+ layer = InvertedResidual(
+ in_channels=self.in_channels,
+ out_channels=out_channels,
+ mid_channels=mid_channels,
+ kernel_size=kernel_size,
+ stride=stride,
+ se_cfg=se_cfg,
+ with_expand_conv=True,
+ conv_cfg=self.conv_cfg,
+ norm_cfg=self.norm_cfg,
+ act_cfg=dict(type=act),
+ with_cp=self.with_cp)
+ self.in_channels = out_channels
+ layer_name = f'layer{i + 1}'
+ self.add_module(layer_name, layer)
+ layers.append(layer_name)
+ return layers
+
+ def init_weights(self, pretrained=None):
+ if isinstance(pretrained, str):
+ logger = logging.getLogger()
+ load_checkpoint(self, pretrained, strict=False, logger=logger)
+ elif pretrained is None:
+ for m in self.modules():
+ if isinstance(m, nn.Conv2d):
+ kaiming_init(m)
+ elif isinstance(m, nn.BatchNorm2d):
+ constant_init(m, 1)
+ else:
+ raise TypeError('pretrained must be a str or None')
+
+ def forward(self, x):
+ x = self.conv1(x)
+
+ outs = []
+ for i, layer_name in enumerate(self.layers):
+ layer = getattr(self, layer_name)
+ x = layer(x)
+ if i in self.out_indices or \
+ i - len(self.layers) in self.out_indices:
+ outs.append(x)
+
+ if len(outs) == 1:
+ return outs[0]
+ return tuple(outs)
+
+ def _freeze_stages(self):
+ if self.frozen_stages >= 0:
+ for param in self.conv1.parameters():
+ param.requires_grad = False
+ for i in range(1, self.frozen_stages + 1):
+ layer = getattr(self, f'layer{i}')
+ layer.eval()
+ for param in layer.parameters():
+ param.requires_grad = False
+
+ def train(self, mode=True):
+ super().train(mode)
+ self._freeze_stages()
+ if mode and self.norm_eval:
+ for m in self.modules():
+ if isinstance(m, _BatchNorm):
+ m.eval()
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/mspn.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/mspn.py
new file mode 100644
index 0000000..71cee34
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/mspn.py
@@ -0,0 +1,513 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import copy as cp
+from collections import OrderedDict
+
+import torch.nn as nn
+import torch.nn.functional as F
+from mmcv.cnn import (ConvModule, MaxPool2d, constant_init, kaiming_init,
+ normal_init)
+from mmcv.runner.checkpoint import load_state_dict
+
+from mmpose.utils import get_root_logger
+from ..builder import BACKBONES
+from .base_backbone import BaseBackbone
+from .resnet import Bottleneck as _Bottleneck
+from .utils.utils import get_state_dict
+
+
+class Bottleneck(_Bottleneck):
+ expansion = 4
+ """Bottleneck block for MSPN.
+
+ Args:
+ in_channels (int): Input channels of this block.
+ out_channels (int): Output channels of this block.
+ stride (int): stride of the block. Default: 1
+ downsample (nn.Module): downsample operation on identity branch.
+ Default: None
+ norm_cfg (dict): dictionary to construct and config norm layer.
+ Default: dict(type='BN')
+ """
+
+ def __init__(self, in_channels, out_channels, **kwargs):
+ super().__init__(in_channels, out_channels * 4, **kwargs)
+
+
+class DownsampleModule(nn.Module):
+ """Downsample module for MSPN.
+
+ Args:
+ block (nn.Module): Downsample block.
+ num_blocks (list): Number of blocks in each downsample unit.
+ num_units (int): Numbers of downsample units. Default: 4
+ has_skip (bool): Have skip connections from prior upsample
+ module or not. Default:False
+ norm_cfg (dict): dictionary to construct and config norm layer.
+ Default: dict(type='BN')
+ in_channels (int): Number of channels of the input feature to
+ downsample module. Default: 64
+ """
+
+ def __init__(self,
+ block,
+ num_blocks,
+ num_units=4,
+ has_skip=False,
+ norm_cfg=dict(type='BN'),
+ in_channels=64):
+ # Protect mutable default arguments
+ norm_cfg = cp.deepcopy(norm_cfg)
+ super().__init__()
+ self.has_skip = has_skip
+ self.in_channels = in_channels
+ assert len(num_blocks) == num_units
+ self.num_blocks = num_blocks
+ self.num_units = num_units
+ self.norm_cfg = norm_cfg
+ self.layer1 = self._make_layer(block, in_channels, num_blocks[0])
+ for i in range(1, num_units):
+ module_name = f'layer{i + 1}'
+ self.add_module(
+ module_name,
+ self._make_layer(
+ block, in_channels * pow(2, i), num_blocks[i], stride=2))
+
+ def _make_layer(self, block, out_channels, blocks, stride=1):
+ downsample = None
+ if stride != 1 or self.in_channels != out_channels * block.expansion:
+ downsample = ConvModule(
+ self.in_channels,
+ out_channels * block.expansion,
+ kernel_size=1,
+ stride=stride,
+ padding=0,
+ norm_cfg=self.norm_cfg,
+ act_cfg=None,
+ inplace=True)
+
+ units = list()
+ units.append(
+ block(
+ self.in_channels,
+ out_channels,
+ stride=stride,
+ downsample=downsample,
+ norm_cfg=self.norm_cfg))
+ self.in_channels = out_channels * block.expansion
+ for _ in range(1, blocks):
+ units.append(block(self.in_channels, out_channels))
+
+ return nn.Sequential(*units)
+
+ def forward(self, x, skip1, skip2):
+ out = list()
+ for i in range(self.num_units):
+ module_name = f'layer{i + 1}'
+ module_i = getattr(self, module_name)
+ x = module_i(x)
+ if self.has_skip:
+ x = x + skip1[i] + skip2[i]
+ out.append(x)
+ out.reverse()
+
+ return tuple(out)
+
+
+class UpsampleUnit(nn.Module):
+ """Upsample unit for upsample module.
+
+ Args:
+ ind (int): Indicates whether to interpolate (>0) and whether to
+ generate feature map for the next hourglass-like module.
+ num_units (int): Number of units that form a upsample module. Along
+ with ind and gen_cross_conv, nm_units is used to decide whether
+ to generate feature map for the next hourglass-like module.
+ in_channels (int): Channel number of the skip-in feature maps from
+ the corresponding downsample unit.
+ unit_channels (int): Channel number in this unit. Default:256.
+ gen_skip: (bool): Whether or not to generate skips for the posterior
+ downsample module. Default:False
+ gen_cross_conv (bool): Whether to generate feature map for the next
+ hourglass-like module. Default:False
+ norm_cfg (dict): dictionary to construct and config norm layer.
+ Default: dict(type='BN')
+ out_channels (int): Number of channels of feature output by upsample
+ module. Must equal to in_channels of downsample module. Default:64
+ """
+
+ def __init__(self,
+ ind,
+ num_units,
+ in_channels,
+ unit_channels=256,
+ gen_skip=False,
+ gen_cross_conv=False,
+ norm_cfg=dict(type='BN'),
+ out_channels=64):
+ # Protect mutable default arguments
+ norm_cfg = cp.deepcopy(norm_cfg)
+ super().__init__()
+ self.num_units = num_units
+ self.norm_cfg = norm_cfg
+ self.in_skip = ConvModule(
+ in_channels,
+ unit_channels,
+ kernel_size=1,
+ stride=1,
+ padding=0,
+ norm_cfg=self.norm_cfg,
+ act_cfg=None,
+ inplace=True)
+ self.relu = nn.ReLU(inplace=True)
+
+ self.ind = ind
+ if self.ind > 0:
+ self.up_conv = ConvModule(
+ unit_channels,
+ unit_channels,
+ kernel_size=1,
+ stride=1,
+ padding=0,
+ norm_cfg=self.norm_cfg,
+ act_cfg=None,
+ inplace=True)
+
+ self.gen_skip = gen_skip
+ if self.gen_skip:
+ self.out_skip1 = ConvModule(
+ in_channels,
+ in_channels,
+ kernel_size=1,
+ stride=1,
+ padding=0,
+ norm_cfg=self.norm_cfg,
+ inplace=True)
+
+ self.out_skip2 = ConvModule(
+ unit_channels,
+ in_channels,
+ kernel_size=1,
+ stride=1,
+ padding=0,
+ norm_cfg=self.norm_cfg,
+ inplace=True)
+
+ self.gen_cross_conv = gen_cross_conv
+ if self.ind == num_units - 1 and self.gen_cross_conv:
+ self.cross_conv = ConvModule(
+ unit_channels,
+ out_channels,
+ kernel_size=1,
+ stride=1,
+ padding=0,
+ norm_cfg=self.norm_cfg,
+ inplace=True)
+
+ def forward(self, x, up_x):
+ out = self.in_skip(x)
+
+ if self.ind > 0:
+ up_x = F.interpolate(
+ up_x,
+ size=(x.size(2), x.size(3)),
+ mode='bilinear',
+ align_corners=True)
+ up_x = self.up_conv(up_x)
+ out = out + up_x
+ out = self.relu(out)
+
+ skip1 = None
+ skip2 = None
+ if self.gen_skip:
+ skip1 = self.out_skip1(x)
+ skip2 = self.out_skip2(out)
+
+ cross_conv = None
+ if self.ind == self.num_units - 1 and self.gen_cross_conv:
+ cross_conv = self.cross_conv(out)
+
+ return out, skip1, skip2, cross_conv
+
+
+class UpsampleModule(nn.Module):
+ """Upsample module for MSPN.
+
+ Args:
+ unit_channels (int): Channel number in the upsample units.
+ Default:256.
+ num_units (int): Numbers of upsample units. Default: 4
+ gen_skip (bool): Whether to generate skip for posterior downsample
+ module or not. Default:False
+ gen_cross_conv (bool): Whether to generate feature map for the next
+ hourglass-like module. Default:False
+ norm_cfg (dict): dictionary to construct and config norm layer.
+ Default: dict(type='BN')
+ out_channels (int): Number of channels of feature output by upsample
+ module. Must equal to in_channels of downsample module. Default:64
+ """
+
+ def __init__(self,
+ unit_channels=256,
+ num_units=4,
+ gen_skip=False,
+ gen_cross_conv=False,
+ norm_cfg=dict(type='BN'),
+ out_channels=64):
+ # Protect mutable default arguments
+ norm_cfg = cp.deepcopy(norm_cfg)
+ super().__init__()
+ self.in_channels = list()
+ for i in range(num_units):
+ self.in_channels.append(Bottleneck.expansion * out_channels *
+ pow(2, i))
+ self.in_channels.reverse()
+ self.num_units = num_units
+ self.gen_skip = gen_skip
+ self.gen_cross_conv = gen_cross_conv
+ self.norm_cfg = norm_cfg
+ for i in range(num_units):
+ module_name = f'up{i + 1}'
+ self.add_module(
+ module_name,
+ UpsampleUnit(
+ i,
+ self.num_units,
+ self.in_channels[i],
+ unit_channels,
+ self.gen_skip,
+ self.gen_cross_conv,
+ norm_cfg=self.norm_cfg,
+ out_channels=64))
+
+ def forward(self, x):
+ out = list()
+ skip1 = list()
+ skip2 = list()
+ cross_conv = None
+ for i in range(self.num_units):
+ module_i = getattr(self, f'up{i + 1}')
+ if i == 0:
+ outi, skip1_i, skip2_i, _ = module_i(x[i], None)
+ elif i == self.num_units - 1:
+ outi, skip1_i, skip2_i, cross_conv = module_i(x[i], out[i - 1])
+ else:
+ outi, skip1_i, skip2_i, _ = module_i(x[i], out[i - 1])
+ out.append(outi)
+ skip1.append(skip1_i)
+ skip2.append(skip2_i)
+ skip1.reverse()
+ skip2.reverse()
+
+ return out, skip1, skip2, cross_conv
+
+
+class SingleStageNetwork(nn.Module):
+ """Single_stage Network.
+
+ Args:
+ unit_channels (int): Channel number in the upsample units. Default:256.
+ num_units (int): Numbers of downsample/upsample units. Default: 4
+ gen_skip (bool): Whether to generate skip for posterior downsample
+ module or not. Default:False
+ gen_cross_conv (bool): Whether to generate feature map for the next
+ hourglass-like module. Default:False
+ has_skip (bool): Have skip connections from prior upsample
+ module or not. Default:False
+ num_blocks (list): Number of blocks in each downsample unit.
+ Default: [2, 2, 2, 2] Note: Make sure num_units==len(num_blocks)
+ norm_cfg (dict): dictionary to construct and config norm layer.
+ Default: dict(type='BN')
+ in_channels (int): Number of channels of the feature from ResNetTop.
+ Default: 64.
+ """
+
+ def __init__(self,
+ has_skip=False,
+ gen_skip=False,
+ gen_cross_conv=False,
+ unit_channels=256,
+ num_units=4,
+ num_blocks=[2, 2, 2, 2],
+ norm_cfg=dict(type='BN'),
+ in_channels=64):
+ # Protect mutable default arguments
+ norm_cfg = cp.deepcopy(norm_cfg)
+ num_blocks = cp.deepcopy(num_blocks)
+ super().__init__()
+ assert len(num_blocks) == num_units
+ self.has_skip = has_skip
+ self.gen_skip = gen_skip
+ self.gen_cross_conv = gen_cross_conv
+ self.num_units = num_units
+ self.unit_channels = unit_channels
+ self.num_blocks = num_blocks
+ self.norm_cfg = norm_cfg
+
+ self.downsample = DownsampleModule(Bottleneck, num_blocks, num_units,
+ has_skip, norm_cfg, in_channels)
+ self.upsample = UpsampleModule(unit_channels, num_units, gen_skip,
+ gen_cross_conv, norm_cfg, in_channels)
+
+ def forward(self, x, skip1, skip2):
+ mid = self.downsample(x, skip1, skip2)
+ out, skip1, skip2, cross_conv = self.upsample(mid)
+
+ return out, skip1, skip2, cross_conv
+
+
+class ResNetTop(nn.Module):
+ """ResNet top for MSPN.
+
+ Args:
+ norm_cfg (dict): dictionary to construct and config norm layer.
+ Default: dict(type='BN')
+ channels (int): Number of channels of the feature output by ResNetTop.
+ """
+
+ def __init__(self, norm_cfg=dict(type='BN'), channels=64):
+ # Protect mutable default arguments
+ norm_cfg = cp.deepcopy(norm_cfg)
+ super().__init__()
+ self.top = nn.Sequential(
+ ConvModule(
+ 3,
+ channels,
+ kernel_size=7,
+ stride=2,
+ padding=3,
+ norm_cfg=norm_cfg,
+ inplace=True), MaxPool2d(kernel_size=3, stride=2, padding=1))
+
+ def forward(self, img):
+ return self.top(img)
+
+
+@BACKBONES.register_module()
+class MSPN(BaseBackbone):
+ """MSPN backbone. Paper ref: Li et al. "Rethinking on Multi-Stage Networks
+ for Human Pose Estimation" (CVPR 2020).
+
+ Args:
+ unit_channels (int): Number of Channels in an upsample unit.
+ Default: 256
+ num_stages (int): Number of stages in a multi-stage MSPN. Default: 4
+ num_units (int): Number of downsample/upsample units in a single-stage
+ network. Default: 4
+ Note: Make sure num_units == len(self.num_blocks)
+ num_blocks (list): Number of bottlenecks in each
+ downsample unit. Default: [2, 2, 2, 2]
+ norm_cfg (dict): dictionary to construct and config norm layer.
+ Default: dict(type='BN')
+ res_top_channels (int): Number of channels of feature from ResNetTop.
+ Default: 64.
+
+ Example:
+ >>> from mmpose.models import MSPN
+ >>> import torch
+ >>> self = MSPN(num_stages=2,num_units=2,num_blocks=[2,2])
+ >>> self.eval()
+ >>> inputs = torch.rand(1, 3, 511, 511)
+ >>> level_outputs = self.forward(inputs)
+ >>> for level_output in level_outputs:
+ ... for feature in level_output:
+ ... print(tuple(feature.shape))
+ ...
+ (1, 256, 64, 64)
+ (1, 256, 128, 128)
+ (1, 256, 64, 64)
+ (1, 256, 128, 128)
+ """
+
+ def __init__(self,
+ unit_channels=256,
+ num_stages=4,
+ num_units=4,
+ num_blocks=[2, 2, 2, 2],
+ norm_cfg=dict(type='BN'),
+ res_top_channels=64):
+ # Protect mutable default arguments
+ norm_cfg = cp.deepcopy(norm_cfg)
+ num_blocks = cp.deepcopy(num_blocks)
+ super().__init__()
+ self.unit_channels = unit_channels
+ self.num_stages = num_stages
+ self.num_units = num_units
+ self.num_blocks = num_blocks
+ self.norm_cfg = norm_cfg
+
+ assert self.num_stages > 0
+ assert self.num_units > 1
+ assert self.num_units == len(self.num_blocks)
+ self.top = ResNetTop(norm_cfg=norm_cfg)
+ self.multi_stage_mspn = nn.ModuleList([])
+ for i in range(self.num_stages):
+ if i == 0:
+ has_skip = False
+ else:
+ has_skip = True
+ if i != self.num_stages - 1:
+ gen_skip = True
+ gen_cross_conv = True
+ else:
+ gen_skip = False
+ gen_cross_conv = False
+ self.multi_stage_mspn.append(
+ SingleStageNetwork(has_skip, gen_skip, gen_cross_conv,
+ unit_channels, num_units, num_blocks,
+ norm_cfg, res_top_channels))
+
+ def forward(self, x):
+ """Model forward function."""
+ out_feats = []
+ skip1 = None
+ skip2 = None
+ x = self.top(x)
+ for i in range(self.num_stages):
+ out, skip1, skip2, x = self.multi_stage_mspn[i](x, skip1, skip2)
+ out_feats.append(out)
+
+ return out_feats
+
+ def init_weights(self, pretrained=None):
+ """Initialize model weights."""
+ if isinstance(pretrained, str):
+ logger = get_root_logger()
+ state_dict_tmp = get_state_dict(pretrained)
+ state_dict = OrderedDict()
+ state_dict['top'] = OrderedDict()
+ state_dict['bottlenecks'] = OrderedDict()
+ for k, v in state_dict_tmp.items():
+ if k.startswith('layer'):
+ if 'downsample.0' in k:
+ state_dict['bottlenecks'][k.replace(
+ 'downsample.0', 'downsample.conv')] = v
+ elif 'downsample.1' in k:
+ state_dict['bottlenecks'][k.replace(
+ 'downsample.1', 'downsample.bn')] = v
+ else:
+ state_dict['bottlenecks'][k] = v
+ elif k.startswith('conv1'):
+ state_dict['top'][k.replace('conv1', 'top.0.conv')] = v
+ elif k.startswith('bn1'):
+ state_dict['top'][k.replace('bn1', 'top.0.bn')] = v
+
+ load_state_dict(
+ self.top, state_dict['top'], strict=False, logger=logger)
+ for i in range(self.num_stages):
+ load_state_dict(
+ self.multi_stage_mspn[i].downsample,
+ state_dict['bottlenecks'],
+ strict=False,
+ logger=logger)
+ else:
+ for m in self.multi_stage_mspn.modules():
+ if isinstance(m, nn.Conv2d):
+ kaiming_init(m)
+ elif isinstance(m, nn.BatchNorm2d):
+ constant_init(m, 1)
+ elif isinstance(m, nn.Linear):
+ normal_init(m, std=0.01)
+
+ for m in self.top.modules():
+ if isinstance(m, nn.Conv2d):
+ kaiming_init(m)
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/regnet.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/regnet.py
new file mode 100644
index 0000000..693417c
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/regnet.py
@@ -0,0 +1,317 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import copy
+
+import numpy as np
+import torch.nn as nn
+from mmcv.cnn import build_conv_layer, build_norm_layer
+
+from ..builder import BACKBONES
+from .resnet import ResNet
+from .resnext import Bottleneck
+
+
+@BACKBONES.register_module()
+class RegNet(ResNet):
+ """RegNet backbone.
+
+ More details can be found in `paper `__ .
+
+ Args:
+ arch (dict): The parameter of RegNets.
+ - w0 (int): initial width
+ - wa (float): slope of width
+ - wm (float): quantization parameter to quantize the width
+ - depth (int): depth of the backbone
+ - group_w (int): width of group
+ - bot_mul (float): bottleneck ratio, i.e. expansion of bottleneck.
+ strides (Sequence[int]): Strides of the first block of each stage.
+ base_channels (int): Base channels after stem layer.
+ in_channels (int): Number of input image channels. Default: 3.
+ dilations (Sequence[int]): Dilation of each stage.
+ out_indices (Sequence[int]): Output from which stages.
+ style (str): `pytorch` or `caffe`. If set to "pytorch", the stride-two
+ layer is the 3x3 conv layer, otherwise the stride-two layer is
+ the first 1x1 conv layer. Default: "pytorch".
+ frozen_stages (int): Stages to be frozen (all param fixed). -1 means
+ not freezing any parameters. Default: -1.
+ norm_cfg (dict): dictionary to construct and config norm layer.
+ Default: dict(type='BN', requires_grad=True).
+ norm_eval (bool): Whether to set norm layers to eval mode, namely,
+ freeze running stats (mean and var). Note: Effect on Batch Norm
+ and its variants only. Default: False.
+ with_cp (bool): Use checkpoint or not. Using checkpoint will save some
+ memory while slowing down the training speed. Default: False.
+ zero_init_residual (bool): whether to use zero init for last norm layer
+ in resblocks to let them behave as identity. Default: True.
+
+ Example:
+ >>> from mmpose.models import RegNet
+ >>> import torch
+ >>> self = RegNet(
+ arch=dict(
+ w0=88,
+ wa=26.31,
+ wm=2.25,
+ group_w=48,
+ depth=25,
+ bot_mul=1.0),
+ out_indices=(0, 1, 2, 3))
+ >>> self.eval()
+ >>> inputs = torch.rand(1, 3, 32, 32)
+ >>> level_outputs = self.forward(inputs)
+ >>> for level_out in level_outputs:
+ ... print(tuple(level_out.shape))
+ (1, 96, 8, 8)
+ (1, 192, 4, 4)
+ (1, 432, 2, 2)
+ (1, 1008, 1, 1)
+ """
+ arch_settings = {
+ 'regnetx_400mf':
+ dict(w0=24, wa=24.48, wm=2.54, group_w=16, depth=22, bot_mul=1.0),
+ 'regnetx_800mf':
+ dict(w0=56, wa=35.73, wm=2.28, group_w=16, depth=16, bot_mul=1.0),
+ 'regnetx_1.6gf':
+ dict(w0=80, wa=34.01, wm=2.25, group_w=24, depth=18, bot_mul=1.0),
+ 'regnetx_3.2gf':
+ dict(w0=88, wa=26.31, wm=2.25, group_w=48, depth=25, bot_mul=1.0),
+ 'regnetx_4.0gf':
+ dict(w0=96, wa=38.65, wm=2.43, group_w=40, depth=23, bot_mul=1.0),
+ 'regnetx_6.4gf':
+ dict(w0=184, wa=60.83, wm=2.07, group_w=56, depth=17, bot_mul=1.0),
+ 'regnetx_8.0gf':
+ dict(w0=80, wa=49.56, wm=2.88, group_w=120, depth=23, bot_mul=1.0),
+ 'regnetx_12gf':
+ dict(w0=168, wa=73.36, wm=2.37, group_w=112, depth=19, bot_mul=1.0),
+ }
+
+ def __init__(self,
+ arch,
+ in_channels=3,
+ stem_channels=32,
+ base_channels=32,
+ strides=(2, 2, 2, 2),
+ dilations=(1, 1, 1, 1),
+ out_indices=(3, ),
+ style='pytorch',
+ deep_stem=False,
+ avg_down=False,
+ frozen_stages=-1,
+ conv_cfg=None,
+ norm_cfg=dict(type='BN', requires_grad=True),
+ norm_eval=False,
+ with_cp=False,
+ zero_init_residual=True):
+ # Protect mutable default arguments
+ norm_cfg = copy.deepcopy(norm_cfg)
+ super(ResNet, self).__init__()
+
+ # Generate RegNet parameters first
+ if isinstance(arch, str):
+ assert arch in self.arch_settings, \
+ f'"arch": "{arch}" is not one of the' \
+ ' arch_settings'
+ arch = self.arch_settings[arch]
+ elif not isinstance(arch, dict):
+ raise TypeError('Expect "arch" to be either a string '
+ f'or a dict, got {type(arch)}')
+
+ widths, num_stages = self.generate_regnet(
+ arch['w0'],
+ arch['wa'],
+ arch['wm'],
+ arch['depth'],
+ )
+ # Convert to per stage format
+ stage_widths, stage_blocks = self.get_stages_from_blocks(widths)
+ # Generate group widths and bot muls
+ group_widths = [arch['group_w'] for _ in range(num_stages)]
+ self.bottleneck_ratio = [arch['bot_mul'] for _ in range(num_stages)]
+ # Adjust the compatibility of stage_widths and group_widths
+ stage_widths, group_widths = self.adjust_width_group(
+ stage_widths, self.bottleneck_ratio, group_widths)
+
+ # Group params by stage
+ self.stage_widths = stage_widths
+ self.group_widths = group_widths
+ self.depth = sum(stage_blocks)
+ self.stem_channels = stem_channels
+ self.base_channels = base_channels
+ self.num_stages = num_stages
+ assert 1 <= num_stages <= 4
+ self.strides = strides
+ self.dilations = dilations
+ assert len(strides) == len(dilations) == num_stages
+ self.out_indices = out_indices
+ assert max(out_indices) < num_stages
+ self.style = style
+ self.deep_stem = deep_stem
+ if self.deep_stem:
+ raise NotImplementedError(
+ 'deep_stem has not been implemented for RegNet')
+ self.avg_down = avg_down
+ self.frozen_stages = frozen_stages
+ self.conv_cfg = conv_cfg
+ self.norm_cfg = norm_cfg
+ self.with_cp = with_cp
+ self.norm_eval = norm_eval
+ self.zero_init_residual = zero_init_residual
+ self.stage_blocks = stage_blocks[:num_stages]
+
+ self._make_stem_layer(in_channels, stem_channels)
+
+ _in_channels = stem_channels
+ self.res_layers = []
+ for i, num_blocks in enumerate(self.stage_blocks):
+ stride = self.strides[i]
+ dilation = self.dilations[i]
+ group_width = self.group_widths[i]
+ width = int(round(self.stage_widths[i] * self.bottleneck_ratio[i]))
+ stage_groups = width // group_width
+
+ res_layer = self.make_res_layer(
+ block=Bottleneck,
+ num_blocks=num_blocks,
+ in_channels=_in_channels,
+ out_channels=self.stage_widths[i],
+ expansion=1,
+ stride=stride,
+ dilation=dilation,
+ style=self.style,
+ avg_down=self.avg_down,
+ with_cp=self.with_cp,
+ conv_cfg=self.conv_cfg,
+ norm_cfg=self.norm_cfg,
+ base_channels=self.stage_widths[i],
+ groups=stage_groups,
+ width_per_group=group_width)
+ _in_channels = self.stage_widths[i]
+ layer_name = f'layer{i + 1}'
+ self.add_module(layer_name, res_layer)
+ self.res_layers.append(layer_name)
+
+ self._freeze_stages()
+
+ self.feat_dim = stage_widths[-1]
+
+ def _make_stem_layer(self, in_channels, base_channels):
+ self.conv1 = build_conv_layer(
+ self.conv_cfg,
+ in_channels,
+ base_channels,
+ kernel_size=3,
+ stride=2,
+ padding=1,
+ bias=False)
+ self.norm1_name, norm1 = build_norm_layer(
+ self.norm_cfg, base_channels, postfix=1)
+ self.add_module(self.norm1_name, norm1)
+ self.relu = nn.ReLU(inplace=True)
+
+ @staticmethod
+ def generate_regnet(initial_width,
+ width_slope,
+ width_parameter,
+ depth,
+ divisor=8):
+ """Generates per block width from RegNet parameters.
+
+ Args:
+ initial_width ([int]): Initial width of the backbone
+ width_slope ([float]): Slope of the quantized linear function
+ width_parameter ([int]): Parameter used to quantize the width.
+ depth ([int]): Depth of the backbone.
+ divisor (int, optional): The divisor of channels. Defaults to 8.
+
+ Returns:
+ list, int: return a list of widths of each stage and the number of
+ stages
+ """
+ assert width_slope >= 0
+ assert initial_width > 0
+ assert width_parameter > 1
+ assert initial_width % divisor == 0
+ widths_cont = np.arange(depth) * width_slope + initial_width
+ ks = np.round(
+ np.log(widths_cont / initial_width) / np.log(width_parameter))
+ widths = initial_width * np.power(width_parameter, ks)
+ widths = np.round(np.divide(widths, divisor)) * divisor
+ num_stages = len(np.unique(widths))
+ widths, widths_cont = widths.astype(int).tolist(), widths_cont.tolist()
+ return widths, num_stages
+
+ @staticmethod
+ def quantize_float(number, divisor):
+ """Converts a float to closest non-zero int divisible by divior.
+
+ Args:
+ number (int): Original number to be quantized.
+ divisor (int): Divisor used to quantize the number.
+
+ Returns:
+ int: quantized number that is divisible by devisor.
+ """
+ return int(round(number / divisor) * divisor)
+
+ def adjust_width_group(self, widths, bottleneck_ratio, groups):
+ """Adjusts the compatibility of widths and groups.
+
+ Args:
+ widths (list[int]): Width of each stage.
+ bottleneck_ratio (float): Bottleneck ratio.
+ groups (int): number of groups in each stage
+
+ Returns:
+ tuple(list): The adjusted widths and groups of each stage.
+ """
+ bottleneck_width = [
+ int(w * b) for w, b in zip(widths, bottleneck_ratio)
+ ]
+ groups = [min(g, w_bot) for g, w_bot in zip(groups, bottleneck_width)]
+ bottleneck_width = [
+ self.quantize_float(w_bot, g)
+ for w_bot, g in zip(bottleneck_width, groups)
+ ]
+ widths = [
+ int(w_bot / b)
+ for w_bot, b in zip(bottleneck_width, bottleneck_ratio)
+ ]
+ return widths, groups
+
+ def get_stages_from_blocks(self, widths):
+ """Gets widths/stage_blocks of network at each stage.
+
+ Args:
+ widths (list[int]): Width in each stage.
+
+ Returns:
+ tuple(list): width and depth of each stage
+ """
+ width_diff = [
+ width != width_prev
+ for width, width_prev in zip(widths + [0], [0] + widths)
+ ]
+ stage_widths = [
+ width for width, diff in zip(widths, width_diff[:-1]) if diff
+ ]
+ stage_blocks = np.diff([
+ depth for depth, diff in zip(range(len(width_diff)), width_diff)
+ if diff
+ ]).tolist()
+ return stage_widths, stage_blocks
+
+ def forward(self, x):
+ x = self.conv1(x)
+ x = self.norm1(x)
+ x = self.relu(x)
+
+ outs = []
+ for i, layer_name in enumerate(self.res_layers):
+ res_layer = getattr(self, layer_name)
+ x = res_layer(x)
+ if i in self.out_indices:
+ outs.append(x)
+
+ if len(outs) == 1:
+ return outs[0]
+ return tuple(outs)
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/resnest.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/resnest.py
new file mode 100644
index 0000000..0a2d408
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/resnest.py
@@ -0,0 +1,338 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+import torch.utils.checkpoint as cp
+from mmcv.cnn import build_conv_layer, build_norm_layer
+
+from ..builder import BACKBONES
+from .resnet import Bottleneck as _Bottleneck
+from .resnet import ResLayer, ResNetV1d
+
+
+class RSoftmax(nn.Module):
+ """Radix Softmax module in ``SplitAttentionConv2d``.
+
+ Args:
+ radix (int): Radix of input.
+ groups (int): Groups of input.
+ """
+
+ def __init__(self, radix, groups):
+ super().__init__()
+ self.radix = radix
+ self.groups = groups
+
+ def forward(self, x):
+ batch = x.size(0)
+ if self.radix > 1:
+ x = x.view(batch, self.groups, self.radix, -1).transpose(1, 2)
+ x = F.softmax(x, dim=1)
+ x = x.reshape(batch, -1)
+ else:
+ x = torch.sigmoid(x)
+ return x
+
+
+class SplitAttentionConv2d(nn.Module):
+ """Split-Attention Conv2d.
+
+ Args:
+ in_channels (int): Same as nn.Conv2d.
+ out_channels (int): Same as nn.Conv2d.
+ kernel_size (int | tuple[int]): Same as nn.Conv2d.
+ stride (int | tuple[int]): Same as nn.Conv2d.
+ padding (int | tuple[int]): Same as nn.Conv2d.
+ dilation (int | tuple[int]): Same as nn.Conv2d.
+ groups (int): Same as nn.Conv2d.
+ radix (int): Radix of SpltAtConv2d. Default: 2
+ reduction_factor (int): Reduction factor of SplitAttentionConv2d.
+ Default: 4.
+ conv_cfg (dict): Config dict for convolution layer. Default: None,
+ which means using conv2d.
+ norm_cfg (dict): Config dict for normalization layer. Default: None.
+ """
+
+ def __init__(self,
+ in_channels,
+ channels,
+ kernel_size,
+ stride=1,
+ padding=0,
+ dilation=1,
+ groups=1,
+ radix=2,
+ reduction_factor=4,
+ conv_cfg=None,
+ norm_cfg=dict(type='BN')):
+ super().__init__()
+ inter_channels = max(in_channels * radix // reduction_factor, 32)
+ self.radix = radix
+ self.groups = groups
+ self.channels = channels
+ self.conv = build_conv_layer(
+ conv_cfg,
+ in_channels,
+ channels * radix,
+ kernel_size,
+ stride=stride,
+ padding=padding,
+ dilation=dilation,
+ groups=groups * radix,
+ bias=False)
+ self.norm0_name, norm0 = build_norm_layer(
+ norm_cfg, channels * radix, postfix=0)
+ self.add_module(self.norm0_name, norm0)
+ self.relu = nn.ReLU(inplace=True)
+ self.fc1 = build_conv_layer(
+ None, channels, inter_channels, 1, groups=self.groups)
+ self.norm1_name, norm1 = build_norm_layer(
+ norm_cfg, inter_channels, postfix=1)
+ self.add_module(self.norm1_name, norm1)
+ self.fc2 = build_conv_layer(
+ None, inter_channels, channels * radix, 1, groups=self.groups)
+ self.rsoftmax = RSoftmax(radix, groups)
+
+ @property
+ def norm0(self):
+ return getattr(self, self.norm0_name)
+
+ @property
+ def norm1(self):
+ return getattr(self, self.norm1_name)
+
+ def forward(self, x):
+ x = self.conv(x)
+ x = self.norm0(x)
+ x = self.relu(x)
+
+ batch, rchannel = x.shape[:2]
+ if self.radix > 1:
+ splits = x.view(batch, self.radix, -1, *x.shape[2:])
+ gap = splits.sum(dim=1)
+ else:
+ gap = x
+ gap = F.adaptive_avg_pool2d(gap, 1)
+ gap = self.fc1(gap)
+
+ gap = self.norm1(gap)
+ gap = self.relu(gap)
+
+ atten = self.fc2(gap)
+ atten = self.rsoftmax(atten).view(batch, -1, 1, 1)
+
+ if self.radix > 1:
+ attens = atten.view(batch, self.radix, -1, *atten.shape[2:])
+ out = torch.sum(attens * splits, dim=1)
+ else:
+ out = atten * x
+ return out.contiguous()
+
+
+class Bottleneck(_Bottleneck):
+ """Bottleneck block for ResNeSt.
+
+ Args:
+ in_channels (int): Input channels of this block.
+ out_channels (int): Output channels of this block.
+ groups (int): Groups of conv2.
+ width_per_group (int): Width per group of conv2. 64x4d indicates
+ ``groups=64, width_per_group=4`` and 32x8d indicates
+ ``groups=32, width_per_group=8``.
+ radix (int): Radix of SpltAtConv2d. Default: 2
+ reduction_factor (int): Reduction factor of SplitAttentionConv2d.
+ Default: 4.
+ avg_down_stride (bool): Whether to use average pool for stride in
+ Bottleneck. Default: True.
+ stride (int): stride of the block. Default: 1
+ dilation (int): dilation of convolution. Default: 1
+ downsample (nn.Module): downsample operation on identity branch.
+ Default: None
+ style (str): `pytorch` or `caffe`. If set to "pytorch", the stride-two
+ layer is the 3x3 conv layer, otherwise the stride-two layer is
+ the first 1x1 conv layer.
+ conv_cfg (dict): dictionary to construct and config conv layer.
+ Default: None
+ norm_cfg (dict): dictionary to construct and config norm layer.
+ Default: dict(type='BN')
+ with_cp (bool): Use checkpoint or not. Using checkpoint will save some
+ memory while slowing down the training speed.
+ """
+
+ def __init__(self,
+ in_channels,
+ out_channels,
+ groups=1,
+ width_per_group=4,
+ base_channels=64,
+ radix=2,
+ reduction_factor=4,
+ avg_down_stride=True,
+ **kwargs):
+ super().__init__(in_channels, out_channels, **kwargs)
+
+ self.groups = groups
+ self.width_per_group = width_per_group
+
+ # For ResNet bottleneck, middle channels are determined by expansion
+ # and out_channels, but for ResNeXt bottleneck, it is determined by
+ # groups and width_per_group and the stage it is located in.
+ if groups != 1:
+ assert self.mid_channels % base_channels == 0
+ self.mid_channels = (
+ groups * width_per_group * self.mid_channels // base_channels)
+
+ self.avg_down_stride = avg_down_stride and self.conv2_stride > 1
+
+ self.norm1_name, norm1 = build_norm_layer(
+ self.norm_cfg, self.mid_channels, postfix=1)
+ self.norm3_name, norm3 = build_norm_layer(
+ self.norm_cfg, self.out_channels, postfix=3)
+
+ self.conv1 = build_conv_layer(
+ self.conv_cfg,
+ self.in_channels,
+ self.mid_channels,
+ kernel_size=1,
+ stride=self.conv1_stride,
+ bias=False)
+ self.add_module(self.norm1_name, norm1)
+ self.conv2 = SplitAttentionConv2d(
+ self.mid_channels,
+ self.mid_channels,
+ kernel_size=3,
+ stride=1 if self.avg_down_stride else self.conv2_stride,
+ padding=self.dilation,
+ dilation=self.dilation,
+ groups=groups,
+ radix=radix,
+ reduction_factor=reduction_factor,
+ conv_cfg=self.conv_cfg,
+ norm_cfg=self.norm_cfg)
+ delattr(self, self.norm2_name)
+
+ if self.avg_down_stride:
+ self.avd_layer = nn.AvgPool2d(3, self.conv2_stride, padding=1)
+
+ self.conv3 = build_conv_layer(
+ self.conv_cfg,
+ self.mid_channels,
+ self.out_channels,
+ kernel_size=1,
+ bias=False)
+ self.add_module(self.norm3_name, norm3)
+
+ def forward(self, x):
+
+ def _inner_forward(x):
+ identity = x
+
+ out = self.conv1(x)
+ out = self.norm1(out)
+ out = self.relu(out)
+
+ out = self.conv2(out)
+
+ if self.avg_down_stride:
+ out = self.avd_layer(out)
+
+ out = self.conv3(out)
+ out = self.norm3(out)
+
+ if self.downsample is not None:
+ identity = self.downsample(x)
+
+ out += identity
+
+ return out
+
+ if self.with_cp and x.requires_grad:
+ out = cp.checkpoint(_inner_forward, x)
+ else:
+ out = _inner_forward(x)
+
+ out = self.relu(out)
+
+ return out
+
+
+@BACKBONES.register_module()
+class ResNeSt(ResNetV1d):
+ """ResNeSt backbone.
+
+ Please refer to the `paper `__
+ for details.
+
+ Args:
+ depth (int): Network depth, from {50, 101, 152, 200}.
+ groups (int): Groups of conv2 in Bottleneck. Default: 32.
+ width_per_group (int): Width per group of conv2 in Bottleneck.
+ Default: 4.
+ radix (int): Radix of SpltAtConv2d. Default: 2
+ reduction_factor (int): Reduction factor of SplitAttentionConv2d.
+ Default: 4.
+ avg_down_stride (bool): Whether to use average pool for stride in
+ Bottleneck. Default: True.
+ in_channels (int): Number of input image channels. Default: 3.
+ stem_channels (int): Output channels of the stem layer. Default: 64.
+ num_stages (int): Stages of the network. Default: 4.
+ strides (Sequence[int]): Strides of the first block of each stage.
+ Default: ``(1, 2, 2, 2)``.
+ dilations (Sequence[int]): Dilation of each stage.
+ Default: ``(1, 1, 1, 1)``.
+ out_indices (Sequence[int]): Output from which stages. If only one
+ stage is specified, a single tensor (feature map) is returned,
+ otherwise multiple stages are specified, a tuple of tensors will
+ be returned. Default: ``(3, )``.
+ style (str): `pytorch` or `caffe`. If set to "pytorch", the stride-two
+ layer is the 3x3 conv layer, otherwise the stride-two layer is
+ the first 1x1 conv layer.
+ deep_stem (bool): Replace 7x7 conv in input stem with 3 3x3 conv.
+ Default: False.
+ avg_down (bool): Use AvgPool instead of stride conv when
+ downsampling in the bottleneck. Default: False.
+ frozen_stages (int): Stages to be frozen (stop grad and set eval mode).
+ -1 means not freezing any parameters. Default: -1.
+ conv_cfg (dict | None): The config dict for conv layers. Default: None.
+ norm_cfg (dict): The config dict for norm layers.
+ norm_eval (bool): Whether to set norm layers to eval mode, namely,
+ freeze running stats (mean and var). Note: Effect on Batch Norm
+ and its variants only. Default: False.
+ with_cp (bool): Use checkpoint or not. Using checkpoint will save some
+ memory while slowing down the training speed. Default: False.
+ zero_init_residual (bool): Whether to use zero init for last norm layer
+ in resblocks to let them behave as identity. Default: True.
+ """
+
+ arch_settings = {
+ 50: (Bottleneck, (3, 4, 6, 3)),
+ 101: (Bottleneck, (3, 4, 23, 3)),
+ 152: (Bottleneck, (3, 8, 36, 3)),
+ 200: (Bottleneck, (3, 24, 36, 3)),
+ 269: (Bottleneck, (3, 30, 48, 8))
+ }
+
+ def __init__(self,
+ depth,
+ groups=1,
+ width_per_group=4,
+ radix=2,
+ reduction_factor=4,
+ avg_down_stride=True,
+ **kwargs):
+ self.groups = groups
+ self.width_per_group = width_per_group
+ self.radix = radix
+ self.reduction_factor = reduction_factor
+ self.avg_down_stride = avg_down_stride
+ super().__init__(depth=depth, **kwargs)
+
+ def make_res_layer(self, **kwargs):
+ return ResLayer(
+ groups=self.groups,
+ width_per_group=self.width_per_group,
+ base_channels=self.base_channels,
+ radix=self.radix,
+ reduction_factor=self.reduction_factor,
+ avg_down_stride=self.avg_down_stride,
+ **kwargs)
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/resnet.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/resnet.py
new file mode 100644
index 0000000..649496a
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/resnet.py
@@ -0,0 +1,701 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import copy
+
+import torch.nn as nn
+import torch.utils.checkpoint as cp
+from mmcv.cnn import (ConvModule, build_conv_layer, build_norm_layer,
+ constant_init, kaiming_init)
+from mmcv.utils.parrots_wrapper import _BatchNorm
+
+from ..builder import BACKBONES
+from .base_backbone import BaseBackbone
+
+
+class BasicBlock(nn.Module):
+ """BasicBlock for ResNet.
+
+ Args:
+ in_channels (int): Input channels of this block.
+ out_channels (int): Output channels of this block.
+ expansion (int): The ratio of ``out_channels/mid_channels`` where
+ ``mid_channels`` is the output channels of conv1. This is a
+ reserved argument in BasicBlock and should always be 1. Default: 1.
+ stride (int): stride of the block. Default: 1
+ dilation (int): dilation of convolution. Default: 1
+ downsample (nn.Module): downsample operation on identity branch.
+ Default: None.
+ style (str): `pytorch` or `caffe`. It is unused and reserved for
+ unified API with Bottleneck.
+ with_cp (bool): Use checkpoint or not. Using checkpoint will save some
+ memory while slowing down the training speed.
+ conv_cfg (dict): dictionary to construct and config conv layer.
+ Default: None
+ norm_cfg (dict): dictionary to construct and config norm layer.
+ Default: dict(type='BN')
+ """
+
+ def __init__(self,
+ in_channels,
+ out_channels,
+ expansion=1,
+ stride=1,
+ dilation=1,
+ downsample=None,
+ style='pytorch',
+ with_cp=False,
+ conv_cfg=None,
+ norm_cfg=dict(type='BN')):
+ # Protect mutable default arguments
+ norm_cfg = copy.deepcopy(norm_cfg)
+ super().__init__()
+ self.in_channels = in_channels
+ self.out_channels = out_channels
+ self.expansion = expansion
+ assert self.expansion == 1
+ assert out_channels % expansion == 0
+ self.mid_channels = out_channels // expansion
+ self.stride = stride
+ self.dilation = dilation
+ self.style = style
+ self.with_cp = with_cp
+ self.conv_cfg = conv_cfg
+ self.norm_cfg = norm_cfg
+
+ self.norm1_name, norm1 = build_norm_layer(
+ norm_cfg, self.mid_channels, postfix=1)
+ self.norm2_name, norm2 = build_norm_layer(
+ norm_cfg, out_channels, postfix=2)
+
+ self.conv1 = build_conv_layer(
+ conv_cfg,
+ in_channels,
+ self.mid_channels,
+ 3,
+ stride=stride,
+ padding=dilation,
+ dilation=dilation,
+ bias=False)
+ self.add_module(self.norm1_name, norm1)
+ self.conv2 = build_conv_layer(
+ conv_cfg,
+ self.mid_channels,
+ out_channels,
+ 3,
+ padding=1,
+ bias=False)
+ self.add_module(self.norm2_name, norm2)
+
+ self.relu = nn.ReLU(inplace=True)
+ self.downsample = downsample
+
+ @property
+ def norm1(self):
+ """nn.Module: the normalization layer named "norm1" """
+ return getattr(self, self.norm1_name)
+
+ @property
+ def norm2(self):
+ """nn.Module: the normalization layer named "norm2" """
+ return getattr(self, self.norm2_name)
+
+ def forward(self, x):
+ """Forward function."""
+
+ def _inner_forward(x):
+ identity = x
+
+ out = self.conv1(x)
+ out = self.norm1(out)
+ out = self.relu(out)
+
+ out = self.conv2(out)
+ out = self.norm2(out)
+
+ if self.downsample is not None:
+ identity = self.downsample(x)
+
+ out += identity
+
+ return out
+
+ if self.with_cp and x.requires_grad:
+ out = cp.checkpoint(_inner_forward, x)
+ else:
+ out = _inner_forward(x)
+
+ out = self.relu(out)
+
+ return out
+
+
+class Bottleneck(nn.Module):
+ """Bottleneck block for ResNet.
+
+ Args:
+ in_channels (int): Input channels of this block.
+ out_channels (int): Output channels of this block.
+ expansion (int): The ratio of ``out_channels/mid_channels`` where
+ ``mid_channels`` is the input/output channels of conv2. Default: 4.
+ stride (int): stride of the block. Default: 1
+ dilation (int): dilation of convolution. Default: 1
+ downsample (nn.Module): downsample operation on identity branch.
+ Default: None.
+ style (str): ``"pytorch"`` or ``"caffe"``. If set to "pytorch", the
+ stride-two layer is the 3x3 conv layer, otherwise the stride-two
+ layer is the first 1x1 conv layer. Default: "pytorch".
+ with_cp (bool): Use checkpoint or not. Using checkpoint will save some
+ memory while slowing down the training speed.
+ conv_cfg (dict): dictionary to construct and config conv layer.
+ Default: None
+ norm_cfg (dict): dictionary to construct and config norm layer.
+ Default: dict(type='BN')
+ """
+
+ def __init__(self,
+ in_channels,
+ out_channels,
+ expansion=4,
+ stride=1,
+ dilation=1,
+ downsample=None,
+ style='pytorch',
+ with_cp=False,
+ conv_cfg=None,
+ norm_cfg=dict(type='BN')):
+ # Protect mutable default arguments
+ norm_cfg = copy.deepcopy(norm_cfg)
+ super().__init__()
+ assert style in ['pytorch', 'caffe']
+
+ self.in_channels = in_channels
+ self.out_channels = out_channels
+ self.expansion = expansion
+ assert out_channels % expansion == 0
+ self.mid_channels = out_channels // expansion
+ self.stride = stride
+ self.dilation = dilation
+ self.style = style
+ self.with_cp = with_cp
+ self.conv_cfg = conv_cfg
+ self.norm_cfg = norm_cfg
+
+ if self.style == 'pytorch':
+ self.conv1_stride = 1
+ self.conv2_stride = stride
+ else:
+ self.conv1_stride = stride
+ self.conv2_stride = 1
+
+ self.norm1_name, norm1 = build_norm_layer(
+ norm_cfg, self.mid_channels, postfix=1)
+ self.norm2_name, norm2 = build_norm_layer(
+ norm_cfg, self.mid_channels, postfix=2)
+ self.norm3_name, norm3 = build_norm_layer(
+ norm_cfg, out_channels, postfix=3)
+
+ self.conv1 = build_conv_layer(
+ conv_cfg,
+ in_channels,
+ self.mid_channels,
+ kernel_size=1,
+ stride=self.conv1_stride,
+ bias=False)
+ self.add_module(self.norm1_name, norm1)
+ self.conv2 = build_conv_layer(
+ conv_cfg,
+ self.mid_channels,
+ self.mid_channels,
+ kernel_size=3,
+ stride=self.conv2_stride,
+ padding=dilation,
+ dilation=dilation,
+ bias=False)
+
+ self.add_module(self.norm2_name, norm2)
+ self.conv3 = build_conv_layer(
+ conv_cfg,
+ self.mid_channels,
+ out_channels,
+ kernel_size=1,
+ bias=False)
+ self.add_module(self.norm3_name, norm3)
+
+ self.relu = nn.ReLU(inplace=True)
+ self.downsample = downsample
+
+ @property
+ def norm1(self):
+ """nn.Module: the normalization layer named "norm1" """
+ return getattr(self, self.norm1_name)
+
+ @property
+ def norm2(self):
+ """nn.Module: the normalization layer named "norm2" """
+ return getattr(self, self.norm2_name)
+
+ @property
+ def norm3(self):
+ """nn.Module: the normalization layer named "norm3" """
+ return getattr(self, self.norm3_name)
+
+ def forward(self, x):
+ """Forward function."""
+
+ def _inner_forward(x):
+ identity = x
+
+ out = self.conv1(x)
+ out = self.norm1(out)
+ out = self.relu(out)
+
+ out = self.conv2(out)
+ out = self.norm2(out)
+ out = self.relu(out)
+
+ out = self.conv3(out)
+ out = self.norm3(out)
+
+ if self.downsample is not None:
+ identity = self.downsample(x)
+
+ out += identity
+
+ return out
+
+ if self.with_cp and x.requires_grad:
+ out = cp.checkpoint(_inner_forward, x)
+ else:
+ out = _inner_forward(x)
+
+ out = self.relu(out)
+
+ return out
+
+
+def get_expansion(block, expansion=None):
+ """Get the expansion of a residual block.
+
+ The block expansion will be obtained by the following order:
+
+ 1. If ``expansion`` is given, just return it.
+ 2. If ``block`` has the attribute ``expansion``, then return
+ ``block.expansion``.
+ 3. Return the default value according the the block type:
+ 1 for ``BasicBlock`` and 4 for ``Bottleneck``.
+
+ Args:
+ block (class): The block class.
+ expansion (int | None): The given expansion ratio.
+
+ Returns:
+ int: The expansion of the block.
+ """
+ if isinstance(expansion, int):
+ assert expansion > 0
+ elif expansion is None:
+ if hasattr(block, 'expansion'):
+ expansion = block.expansion
+ elif issubclass(block, BasicBlock):
+ expansion = 1
+ elif issubclass(block, Bottleneck):
+ expansion = 4
+ else:
+ raise TypeError(f'expansion is not specified for {block.__name__}')
+ else:
+ raise TypeError('expansion must be an integer or None')
+
+ return expansion
+
+
+class ResLayer(nn.Sequential):
+ """ResLayer to build ResNet style backbone.
+
+ Args:
+ block (nn.Module): Residual block used to build ResLayer.
+ num_blocks (int): Number of blocks.
+ in_channels (int): Input channels of this block.
+ out_channels (int): Output channels of this block.
+ expansion (int, optional): The expansion for BasicBlock/Bottleneck.
+ If not specified, it will firstly be obtained via
+ ``block.expansion``. If the block has no attribute "expansion",
+ the following default values will be used: 1 for BasicBlock and
+ 4 for Bottleneck. Default: None.
+ stride (int): stride of the first block. Default: 1.
+ avg_down (bool): Use AvgPool instead of stride conv when
+ downsampling in the bottleneck. Default: False
+ conv_cfg (dict): dictionary to construct and config conv layer.
+ Default: None
+ norm_cfg (dict): dictionary to construct and config norm layer.
+ Default: dict(type='BN')
+ downsample_first (bool): Downsample at the first block or last block.
+ False for Hourglass, True for ResNet. Default: True
+ """
+
+ def __init__(self,
+ block,
+ num_blocks,
+ in_channels,
+ out_channels,
+ expansion=None,
+ stride=1,
+ avg_down=False,
+ conv_cfg=None,
+ norm_cfg=dict(type='BN'),
+ downsample_first=True,
+ **kwargs):
+ # Protect mutable default arguments
+ norm_cfg = copy.deepcopy(norm_cfg)
+ self.block = block
+ self.expansion = get_expansion(block, expansion)
+
+ downsample = None
+ if stride != 1 or in_channels != out_channels:
+ downsample = []
+ conv_stride = stride
+ if avg_down and stride != 1:
+ conv_stride = 1
+ downsample.append(
+ nn.AvgPool2d(
+ kernel_size=stride,
+ stride=stride,
+ ceil_mode=True,
+ count_include_pad=False))
+ downsample.extend([
+ build_conv_layer(
+ conv_cfg,
+ in_channels,
+ out_channels,
+ kernel_size=1,
+ stride=conv_stride,
+ bias=False),
+ build_norm_layer(norm_cfg, out_channels)[1]
+ ])
+ downsample = nn.Sequential(*downsample)
+
+ layers = []
+ if downsample_first:
+ layers.append(
+ block(
+ in_channels=in_channels,
+ out_channels=out_channels,
+ expansion=self.expansion,
+ stride=stride,
+ downsample=downsample,
+ conv_cfg=conv_cfg,
+ norm_cfg=norm_cfg,
+ **kwargs))
+ in_channels = out_channels
+ for _ in range(1, num_blocks):
+ layers.append(
+ block(
+ in_channels=in_channels,
+ out_channels=out_channels,
+ expansion=self.expansion,
+ stride=1,
+ conv_cfg=conv_cfg,
+ norm_cfg=norm_cfg,
+ **kwargs))
+ else: # downsample_first=False is for HourglassModule
+ for i in range(0, num_blocks - 1):
+ layers.append(
+ block(
+ in_channels=in_channels,
+ out_channels=in_channels,
+ expansion=self.expansion,
+ stride=1,
+ conv_cfg=conv_cfg,
+ norm_cfg=norm_cfg,
+ **kwargs))
+ layers.append(
+ block(
+ in_channels=in_channels,
+ out_channels=out_channels,
+ expansion=self.expansion,
+ stride=stride,
+ downsample=downsample,
+ conv_cfg=conv_cfg,
+ norm_cfg=norm_cfg,
+ **kwargs))
+
+ super().__init__(*layers)
+
+
+@BACKBONES.register_module()
+class ResNet(BaseBackbone):
+ """ResNet backbone.
+
+ Please refer to the `paper `__ for
+ details.
+
+ Args:
+ depth (int): Network depth, from {18, 34, 50, 101, 152}.
+ in_channels (int): Number of input image channels. Default: 3.
+ stem_channels (int): Output channels of the stem layer. Default: 64.
+ base_channels (int): Middle channels of the first stage. Default: 64.
+ num_stages (int): Stages of the network. Default: 4.
+ strides (Sequence[int]): Strides of the first block of each stage.
+ Default: ``(1, 2, 2, 2)``.
+ dilations (Sequence[int]): Dilation of each stage.
+ Default: ``(1, 1, 1, 1)``.
+ out_indices (Sequence[int]): Output from which stages. If only one
+ stage is specified, a single tensor (feature map) is returned,
+ otherwise multiple stages are specified, a tuple of tensors will
+ be returned. Default: ``(3, )``.
+ style (str): `pytorch` or `caffe`. If set to "pytorch", the stride-two
+ layer is the 3x3 conv layer, otherwise the stride-two layer is
+ the first 1x1 conv layer.
+ deep_stem (bool): Replace 7x7 conv in input stem with 3 3x3 conv.
+ Default: False.
+ avg_down (bool): Use AvgPool instead of stride conv when
+ downsampling in the bottleneck. Default: False.
+ frozen_stages (int): Stages to be frozen (stop grad and set eval mode).
+ -1 means not freezing any parameters. Default: -1.
+ conv_cfg (dict | None): The config dict for conv layers. Default: None.
+ norm_cfg (dict): The config dict for norm layers.
+ norm_eval (bool): Whether to set norm layers to eval mode, namely,
+ freeze running stats (mean and var). Note: Effect on Batch Norm
+ and its variants only. Default: False.
+ with_cp (bool): Use checkpoint or not. Using checkpoint will save some
+ memory while slowing down the training speed. Default: False.
+ zero_init_residual (bool): Whether to use zero init for last norm layer
+ in resblocks to let them behave as identity. Default: True.
+
+ Example:
+ >>> from mmpose.models import ResNet
+ >>> import torch
+ >>> self = ResNet(depth=18, out_indices=(0, 1, 2, 3))
+ >>> self.eval()
+ >>> inputs = torch.rand(1, 3, 32, 32)
+ >>> level_outputs = self.forward(inputs)
+ >>> for level_out in level_outputs:
+ ... print(tuple(level_out.shape))
+ (1, 64, 8, 8)
+ (1, 128, 4, 4)
+ (1, 256, 2, 2)
+ (1, 512, 1, 1)
+ """
+
+ arch_settings = {
+ 18: (BasicBlock, (2, 2, 2, 2)),
+ 34: (BasicBlock, (3, 4, 6, 3)),
+ 50: (Bottleneck, (3, 4, 6, 3)),
+ 101: (Bottleneck, (3, 4, 23, 3)),
+ 152: (Bottleneck, (3, 8, 36, 3))
+ }
+
+ def __init__(self,
+ depth,
+ in_channels=3,
+ stem_channels=64,
+ base_channels=64,
+ expansion=None,
+ num_stages=4,
+ strides=(1, 2, 2, 2),
+ dilations=(1, 1, 1, 1),
+ out_indices=(3, ),
+ style='pytorch',
+ deep_stem=False,
+ avg_down=False,
+ frozen_stages=-1,
+ conv_cfg=None,
+ norm_cfg=dict(type='BN', requires_grad=True),
+ norm_eval=False,
+ with_cp=False,
+ zero_init_residual=True):
+ # Protect mutable default arguments
+ norm_cfg = copy.deepcopy(norm_cfg)
+ super().__init__()
+ if depth not in self.arch_settings:
+ raise KeyError(f'invalid depth {depth} for resnet')
+ self.depth = depth
+ self.stem_channels = stem_channels
+ self.base_channels = base_channels
+ self.num_stages = num_stages
+ assert 1 <= num_stages <= 4
+ self.strides = strides
+ self.dilations = dilations
+ assert len(strides) == len(dilations) == num_stages
+ self.out_indices = out_indices
+ assert max(out_indices) < num_stages
+ self.style = style
+ self.deep_stem = deep_stem
+ self.avg_down = avg_down
+ self.frozen_stages = frozen_stages
+ self.conv_cfg = conv_cfg
+ self.norm_cfg = norm_cfg
+ self.with_cp = with_cp
+ self.norm_eval = norm_eval
+ self.zero_init_residual = zero_init_residual
+ self.block, stage_blocks = self.arch_settings[depth]
+ self.stage_blocks = stage_blocks[:num_stages]
+ self.expansion = get_expansion(self.block, expansion)
+
+ self._make_stem_layer(in_channels, stem_channels)
+
+ self.res_layers = []
+ _in_channels = stem_channels
+ _out_channels = base_channels * self.expansion
+ for i, num_blocks in enumerate(self.stage_blocks):
+ stride = strides[i]
+ dilation = dilations[i]
+ res_layer = self.make_res_layer(
+ block=self.block,
+ num_blocks=num_blocks,
+ in_channels=_in_channels,
+ out_channels=_out_channels,
+ expansion=self.expansion,
+ stride=stride,
+ dilation=dilation,
+ style=self.style,
+ avg_down=self.avg_down,
+ with_cp=with_cp,
+ conv_cfg=conv_cfg,
+ norm_cfg=norm_cfg)
+ _in_channels = _out_channels
+ _out_channels *= 2
+ layer_name = f'layer{i + 1}'
+ self.add_module(layer_name, res_layer)
+ self.res_layers.append(layer_name)
+
+ self._freeze_stages()
+
+ self.feat_dim = res_layer[-1].out_channels
+
+ def make_res_layer(self, **kwargs):
+ """Make a ResLayer."""
+ return ResLayer(**kwargs)
+
+ @property
+ def norm1(self):
+ """nn.Module: the normalization layer named "norm1" """
+ return getattr(self, self.norm1_name)
+
+ def _make_stem_layer(self, in_channels, stem_channels):
+ """Make stem layer."""
+ if self.deep_stem:
+ self.stem = nn.Sequential(
+ ConvModule(
+ in_channels,
+ stem_channels // 2,
+ kernel_size=3,
+ stride=2,
+ padding=1,
+ conv_cfg=self.conv_cfg,
+ norm_cfg=self.norm_cfg,
+ inplace=True),
+ ConvModule(
+ stem_channels // 2,
+ stem_channels // 2,
+ kernel_size=3,
+ stride=1,
+ padding=1,
+ conv_cfg=self.conv_cfg,
+ norm_cfg=self.norm_cfg,
+ inplace=True),
+ ConvModule(
+ stem_channels // 2,
+ stem_channels,
+ kernel_size=3,
+ stride=1,
+ padding=1,
+ conv_cfg=self.conv_cfg,
+ norm_cfg=self.norm_cfg,
+ inplace=True))
+ else:
+ self.conv1 = build_conv_layer(
+ self.conv_cfg,
+ in_channels,
+ stem_channels,
+ kernel_size=7,
+ stride=2,
+ padding=3,
+ bias=False)
+ self.norm1_name, norm1 = build_norm_layer(
+ self.norm_cfg, stem_channels, postfix=1)
+ self.add_module(self.norm1_name, norm1)
+ self.relu = nn.ReLU(inplace=True)
+ self.maxpool = nn.MaxPool2d(kernel_size=3, stride=2, padding=1)
+
+ def _freeze_stages(self):
+ """Freeze parameters."""
+ if self.frozen_stages >= 0:
+ if self.deep_stem:
+ self.stem.eval()
+ for param in self.stem.parameters():
+ param.requires_grad = False
+ else:
+ self.norm1.eval()
+ for m in [self.conv1, self.norm1]:
+ for param in m.parameters():
+ param.requires_grad = False
+
+ for i in range(1, self.frozen_stages + 1):
+ m = getattr(self, f'layer{i}')
+ m.eval()
+ for param in m.parameters():
+ param.requires_grad = False
+
+ def init_weights(self, pretrained=None):
+ """Initialize the weights in backbone.
+
+ Args:
+ pretrained (str, optional): Path to pre-trained weights.
+ Defaults to None.
+ """
+ super().init_weights(pretrained)
+ if pretrained is None:
+ for m in self.modules():
+ if isinstance(m, nn.Conv2d):
+ kaiming_init(m)
+ elif isinstance(m, (_BatchNorm, nn.GroupNorm)):
+ constant_init(m, 1)
+
+ if self.zero_init_residual:
+ for m in self.modules():
+ if isinstance(m, Bottleneck):
+ constant_init(m.norm3, 0)
+ elif isinstance(m, BasicBlock):
+ constant_init(m.norm2, 0)
+
+ def forward(self, x):
+ """Forward function."""
+ if self.deep_stem:
+ x = self.stem(x)
+ else:
+ x = self.conv1(x)
+ x = self.norm1(x)
+ x = self.relu(x)
+ x = self.maxpool(x)
+ outs = []
+ for i, layer_name in enumerate(self.res_layers):
+ res_layer = getattr(self, layer_name)
+ x = res_layer(x)
+ if i in self.out_indices:
+ outs.append(x)
+ if len(outs) == 1:
+ return outs[0]
+ return tuple(outs)
+
+ def train(self, mode=True):
+ """Convert the model into training mode."""
+ super().train(mode)
+ self._freeze_stages()
+ if mode and self.norm_eval:
+ for m in self.modules():
+ # trick: eval have effect on BatchNorm only
+ if isinstance(m, _BatchNorm):
+ m.eval()
+
+
+@BACKBONES.register_module()
+class ResNetV1d(ResNet):
+ r"""ResNetV1d variant described in `Bag of Tricks
+ `__.
+
+ Compared with default ResNet(ResNetV1b), ResNetV1d replaces the 7x7 conv in
+ the input stem with three 3x3 convs. And in the downsampling block, a 2x2
+ avg_pool with stride 2 is added before conv, whose stride is changed to 1.
+ """
+
+ def __init__(self, **kwargs):
+ super().__init__(deep_stem=True, avg_down=True, **kwargs)
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/resnext.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/resnext.py
new file mode 100644
index 0000000..c10dc33
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/resnext.py
@@ -0,0 +1,162 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from mmcv.cnn import build_conv_layer, build_norm_layer
+
+from ..builder import BACKBONES
+from .resnet import Bottleneck as _Bottleneck
+from .resnet import ResLayer, ResNet
+
+
+class Bottleneck(_Bottleneck):
+ """Bottleneck block for ResNeXt.
+
+ Args:
+ in_channels (int): Input channels of this block.
+ out_channels (int): Output channels of this block.
+ groups (int): Groups of conv2.
+ width_per_group (int): Width per group of conv2. 64x4d indicates
+ ``groups=64, width_per_group=4`` and 32x8d indicates
+ ``groups=32, width_per_group=8``.
+ stride (int): stride of the block. Default: 1
+ dilation (int): dilation of convolution. Default: 1
+ downsample (nn.Module): downsample operation on identity branch.
+ Default: None
+ style (str): `pytorch` or `caffe`. If set to "pytorch", the stride-two
+ layer is the 3x3 conv layer, otherwise the stride-two layer is
+ the first 1x1 conv layer.
+ conv_cfg (dict): dictionary to construct and config conv layer.
+ Default: None
+ norm_cfg (dict): dictionary to construct and config norm layer.
+ Default: dict(type='BN')
+ with_cp (bool): Use checkpoint or not. Using checkpoint will save some
+ memory while slowing down the training speed.
+ """
+
+ def __init__(self,
+ in_channels,
+ out_channels,
+ base_channels=64,
+ groups=32,
+ width_per_group=4,
+ **kwargs):
+ super().__init__(in_channels, out_channels, **kwargs)
+ self.groups = groups
+ self.width_per_group = width_per_group
+
+ # For ResNet bottleneck, middle channels are determined by expansion
+ # and out_channels, but for ResNeXt bottleneck, it is determined by
+ # groups and width_per_group and the stage it is located in.
+ if groups != 1:
+ assert self.mid_channels % base_channels == 0
+ self.mid_channels = (
+ groups * width_per_group * self.mid_channels // base_channels)
+
+ self.norm1_name, norm1 = build_norm_layer(
+ self.norm_cfg, self.mid_channels, postfix=1)
+ self.norm2_name, norm2 = build_norm_layer(
+ self.norm_cfg, self.mid_channels, postfix=2)
+ self.norm3_name, norm3 = build_norm_layer(
+ self.norm_cfg, self.out_channels, postfix=3)
+
+ self.conv1 = build_conv_layer(
+ self.conv_cfg,
+ self.in_channels,
+ self.mid_channels,
+ kernel_size=1,
+ stride=self.conv1_stride,
+ bias=False)
+ self.add_module(self.norm1_name, norm1)
+ self.conv2 = build_conv_layer(
+ self.conv_cfg,
+ self.mid_channels,
+ self.mid_channels,
+ kernel_size=3,
+ stride=self.conv2_stride,
+ padding=self.dilation,
+ dilation=self.dilation,
+ groups=groups,
+ bias=False)
+
+ self.add_module(self.norm2_name, norm2)
+ self.conv3 = build_conv_layer(
+ self.conv_cfg,
+ self.mid_channels,
+ self.out_channels,
+ kernel_size=1,
+ bias=False)
+ self.add_module(self.norm3_name, norm3)
+
+
+@BACKBONES.register_module()
+class ResNeXt(ResNet):
+ """ResNeXt backbone.
+
+ Please refer to the `paper `__ for
+ details.
+
+ Args:
+ depth (int): Network depth, from {50, 101, 152}.
+ groups (int): Groups of conv2 in Bottleneck. Default: 32.
+ width_per_group (int): Width per group of conv2 in Bottleneck.
+ Default: 4.
+ in_channels (int): Number of input image channels. Default: 3.
+ stem_channels (int): Output channels of the stem layer. Default: 64.
+ num_stages (int): Stages of the network. Default: 4.
+ strides (Sequence[int]): Strides of the first block of each stage.
+ Default: ``(1, 2, 2, 2)``.
+ dilations (Sequence[int]): Dilation of each stage.
+ Default: ``(1, 1, 1, 1)``.
+ out_indices (Sequence[int]): Output from which stages. If only one
+ stage is specified, a single tensor (feature map) is returned,
+ otherwise multiple stages are specified, a tuple of tensors will
+ be returned. Default: ``(3, )``.
+ style (str): `pytorch` or `caffe`. If set to "pytorch", the stride-two
+ layer is the 3x3 conv layer, otherwise the stride-two layer is
+ the first 1x1 conv layer.
+ deep_stem (bool): Replace 7x7 conv in input stem with 3 3x3 conv.
+ Default: False.
+ avg_down (bool): Use AvgPool instead of stride conv when
+ downsampling in the bottleneck. Default: False.
+ frozen_stages (int): Stages to be frozen (stop grad and set eval mode).
+ -1 means not freezing any parameters. Default: -1.
+ conv_cfg (dict | None): The config dict for conv layers. Default: None.
+ norm_cfg (dict): The config dict for norm layers.
+ norm_eval (bool): Whether to set norm layers to eval mode, namely,
+ freeze running stats (mean and var). Note: Effect on Batch Norm
+ and its variants only. Default: False.
+ with_cp (bool): Use checkpoint or not. Using checkpoint will save some
+ memory while slowing down the training speed. Default: False.
+ zero_init_residual (bool): Whether to use zero init for last norm layer
+ in resblocks to let them behave as identity. Default: True.
+
+ Example:
+ >>> from mmpose.models import ResNeXt
+ >>> import torch
+ >>> self = ResNeXt(depth=50, out_indices=(0, 1, 2, 3))
+ >>> self.eval()
+ >>> inputs = torch.rand(1, 3, 32, 32)
+ >>> level_outputs = self.forward(inputs)
+ >>> for level_out in level_outputs:
+ ... print(tuple(level_out.shape))
+ (1, 256, 8, 8)
+ (1, 512, 4, 4)
+ (1, 1024, 2, 2)
+ (1, 2048, 1, 1)
+ """
+
+ arch_settings = {
+ 50: (Bottleneck, (3, 4, 6, 3)),
+ 101: (Bottleneck, (3, 4, 23, 3)),
+ 152: (Bottleneck, (3, 8, 36, 3))
+ }
+
+ def __init__(self, depth, groups=32, width_per_group=4, **kwargs):
+ self.groups = groups
+ self.width_per_group = width_per_group
+ super().__init__(depth, **kwargs)
+
+ def make_res_layer(self, **kwargs):
+ return ResLayer(
+ groups=self.groups,
+ width_per_group=self.width_per_group,
+ base_channels=self.base_channels,
+ **kwargs)
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/rsn.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/rsn.py
new file mode 100644
index 0000000..29038af
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/rsn.py
@@ -0,0 +1,616 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import copy as cp
+
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+from mmcv.cnn import (ConvModule, MaxPool2d, constant_init, kaiming_init,
+ normal_init)
+
+from ..builder import BACKBONES
+from .base_backbone import BaseBackbone
+
+
+class RSB(nn.Module):
+ """Residual Steps block for RSN. Paper ref: Cai et al. "Learning Delicate
+ Local Representations for Multi-Person Pose Estimation" (ECCV 2020).
+
+ Args:
+ in_channels (int): Input channels of this block.
+ out_channels (int): Output channels of this block.
+ num_steps (int): Numbers of steps in RSB
+ stride (int): stride of the block. Default: 1
+ downsample (nn.Module): downsample operation on identity branch.
+ Default: None.
+ norm_cfg (dict): dictionary to construct and config norm layer.
+ Default: dict(type='BN')
+ expand_times (int): Times by which the in_channels are expanded.
+ Default:26.
+ res_top_channels (int): Number of channels of feature output by
+ ResNet_top. Default:64.
+ """
+
+ expansion = 1
+
+ def __init__(self,
+ in_channels,
+ out_channels,
+ num_steps=4,
+ stride=1,
+ downsample=None,
+ with_cp=False,
+ norm_cfg=dict(type='BN'),
+ expand_times=26,
+ res_top_channels=64):
+ # Protect mutable default arguments
+ norm_cfg = cp.deepcopy(norm_cfg)
+ super().__init__()
+ assert num_steps > 1
+ self.in_channels = in_channels
+ self.branch_channels = self.in_channels * expand_times
+ self.branch_channels //= res_top_channels
+ self.out_channels = out_channels
+ self.stride = stride
+ self.downsample = downsample
+ self.with_cp = with_cp
+ self.norm_cfg = norm_cfg
+ self.num_steps = num_steps
+ self.conv_bn_relu1 = ConvModule(
+ self.in_channels,
+ self.num_steps * self.branch_channels,
+ kernel_size=1,
+ stride=self.stride,
+ padding=0,
+ norm_cfg=self.norm_cfg,
+ inplace=False)
+ for i in range(self.num_steps):
+ for j in range(i + 1):
+ module_name = f'conv_bn_relu2_{i + 1}_{j + 1}'
+ self.add_module(
+ module_name,
+ ConvModule(
+ self.branch_channels,
+ self.branch_channels,
+ kernel_size=3,
+ stride=1,
+ padding=1,
+ norm_cfg=self.norm_cfg,
+ inplace=False))
+ self.conv_bn3 = ConvModule(
+ self.num_steps * self.branch_channels,
+ self.out_channels * self.expansion,
+ kernel_size=1,
+ stride=1,
+ padding=0,
+ act_cfg=None,
+ norm_cfg=self.norm_cfg,
+ inplace=False)
+ self.relu = nn.ReLU(inplace=False)
+
+ def forward(self, x):
+ """Forward function."""
+
+ identity = x
+ x = self.conv_bn_relu1(x)
+ spx = torch.split(x, self.branch_channels, 1)
+ outputs = list()
+ outs = list()
+ for i in range(self.num_steps):
+ outputs_i = list()
+ outputs.append(outputs_i)
+ for j in range(i + 1):
+ if j == 0:
+ inputs = spx[i]
+ else:
+ inputs = outputs[i][j - 1]
+ if i > j:
+ inputs = inputs + outputs[i - 1][j]
+ module_name = f'conv_bn_relu2_{i + 1}_{j + 1}'
+ module_i_j = getattr(self, module_name)
+ outputs[i].append(module_i_j(inputs))
+
+ outs.append(outputs[i][i])
+ out = torch.cat(tuple(outs), 1)
+ out = self.conv_bn3(out)
+
+ if self.downsample is not None:
+ identity = self.downsample(identity)
+ out = out + identity
+
+ out = self.relu(out)
+
+ return out
+
+
+class Downsample_module(nn.Module):
+ """Downsample module for RSN.
+
+ Args:
+ block (nn.Module): Downsample block.
+ num_blocks (list): Number of blocks in each downsample unit.
+ num_units (int): Numbers of downsample units. Default: 4
+ has_skip (bool): Have skip connections from prior upsample
+ module or not. Default:False
+ num_steps (int): Number of steps in a block. Default:4
+ norm_cfg (dict): dictionary to construct and config norm layer.
+ Default: dict(type='BN')
+ in_channels (int): Number of channels of the input feature to
+ downsample module. Default: 64
+ expand_times (int): Times by which the in_channels are expanded.
+ Default:26.
+ """
+
+ def __init__(self,
+ block,
+ num_blocks,
+ num_steps=4,
+ num_units=4,
+ has_skip=False,
+ norm_cfg=dict(type='BN'),
+ in_channels=64,
+ expand_times=26):
+ # Protect mutable default arguments
+ norm_cfg = cp.deepcopy(norm_cfg)
+ super().__init__()
+ self.has_skip = has_skip
+ self.in_channels = in_channels
+ assert len(num_blocks) == num_units
+ self.num_blocks = num_blocks
+ self.num_units = num_units
+ self.num_steps = num_steps
+ self.norm_cfg = norm_cfg
+ self.layer1 = self._make_layer(
+ block,
+ in_channels,
+ num_blocks[0],
+ expand_times=expand_times,
+ res_top_channels=in_channels)
+ for i in range(1, num_units):
+ module_name = f'layer{i + 1}'
+ self.add_module(
+ module_name,
+ self._make_layer(
+ block,
+ in_channels * pow(2, i),
+ num_blocks[i],
+ stride=2,
+ expand_times=expand_times,
+ res_top_channels=in_channels))
+
+ def _make_layer(self,
+ block,
+ out_channels,
+ blocks,
+ stride=1,
+ expand_times=26,
+ res_top_channels=64):
+ downsample = None
+ if stride != 1 or self.in_channels != out_channels * block.expansion:
+ downsample = ConvModule(
+ self.in_channels,
+ out_channels * block.expansion,
+ kernel_size=1,
+ stride=stride,
+ padding=0,
+ norm_cfg=self.norm_cfg,
+ act_cfg=None,
+ inplace=True)
+
+ units = list()
+ units.append(
+ block(
+ self.in_channels,
+ out_channels,
+ num_steps=self.num_steps,
+ stride=stride,
+ downsample=downsample,
+ norm_cfg=self.norm_cfg,
+ expand_times=expand_times,
+ res_top_channels=res_top_channels))
+ self.in_channels = out_channels * block.expansion
+ for _ in range(1, blocks):
+ units.append(
+ block(
+ self.in_channels,
+ out_channels,
+ num_steps=self.num_steps,
+ expand_times=expand_times,
+ res_top_channels=res_top_channels))
+
+ return nn.Sequential(*units)
+
+ def forward(self, x, skip1, skip2):
+ out = list()
+ for i in range(self.num_units):
+ module_name = f'layer{i + 1}'
+ module_i = getattr(self, module_name)
+ x = module_i(x)
+ if self.has_skip:
+ x = x + skip1[i] + skip2[i]
+ out.append(x)
+ out.reverse()
+
+ return tuple(out)
+
+
+class Upsample_unit(nn.Module):
+ """Upsample unit for upsample module.
+
+ Args:
+ ind (int): Indicates whether to interpolate (>0) and whether to
+ generate feature map for the next hourglass-like module.
+ num_units (int): Number of units that form a upsample module. Along
+ with ind and gen_cross_conv, nm_units is used to decide whether
+ to generate feature map for the next hourglass-like module.
+ in_channels (int): Channel number of the skip-in feature maps from
+ the corresponding downsample unit.
+ unit_channels (int): Channel number in this unit. Default:256.
+ gen_skip: (bool): Whether or not to generate skips for the posterior
+ downsample module. Default:False
+ gen_cross_conv (bool): Whether to generate feature map for the next
+ hourglass-like module. Default:False
+ norm_cfg (dict): dictionary to construct and config norm layer.
+ Default: dict(type='BN')
+ out_channels (in): Number of channels of feature output by upsample
+ module. Must equal to in_channels of downsample module. Default:64
+ """
+
+ def __init__(self,
+ ind,
+ num_units,
+ in_channels,
+ unit_channels=256,
+ gen_skip=False,
+ gen_cross_conv=False,
+ norm_cfg=dict(type='BN'),
+ out_channels=64):
+ # Protect mutable default arguments
+ norm_cfg = cp.deepcopy(norm_cfg)
+ super().__init__()
+ self.num_units = num_units
+ self.norm_cfg = norm_cfg
+ self.in_skip = ConvModule(
+ in_channels,
+ unit_channels,
+ kernel_size=1,
+ stride=1,
+ padding=0,
+ norm_cfg=self.norm_cfg,
+ act_cfg=None,
+ inplace=True)
+ self.relu = nn.ReLU(inplace=True)
+
+ self.ind = ind
+ if self.ind > 0:
+ self.up_conv = ConvModule(
+ unit_channels,
+ unit_channels,
+ kernel_size=1,
+ stride=1,
+ padding=0,
+ norm_cfg=self.norm_cfg,
+ act_cfg=None,
+ inplace=True)
+
+ self.gen_skip = gen_skip
+ if self.gen_skip:
+ self.out_skip1 = ConvModule(
+ in_channels,
+ in_channels,
+ kernel_size=1,
+ stride=1,
+ padding=0,
+ norm_cfg=self.norm_cfg,
+ inplace=True)
+
+ self.out_skip2 = ConvModule(
+ unit_channels,
+ in_channels,
+ kernel_size=1,
+ stride=1,
+ padding=0,
+ norm_cfg=self.norm_cfg,
+ inplace=True)
+
+ self.gen_cross_conv = gen_cross_conv
+ if self.ind == num_units - 1 and self.gen_cross_conv:
+ self.cross_conv = ConvModule(
+ unit_channels,
+ out_channels,
+ kernel_size=1,
+ stride=1,
+ padding=0,
+ norm_cfg=self.norm_cfg,
+ inplace=True)
+
+ def forward(self, x, up_x):
+ out = self.in_skip(x)
+
+ if self.ind > 0:
+ up_x = F.interpolate(
+ up_x,
+ size=(x.size(2), x.size(3)),
+ mode='bilinear',
+ align_corners=True)
+ up_x = self.up_conv(up_x)
+ out = out + up_x
+ out = self.relu(out)
+
+ skip1 = None
+ skip2 = None
+ if self.gen_skip:
+ skip1 = self.out_skip1(x)
+ skip2 = self.out_skip2(out)
+
+ cross_conv = None
+ if self.ind == self.num_units - 1 and self.gen_cross_conv:
+ cross_conv = self.cross_conv(out)
+
+ return out, skip1, skip2, cross_conv
+
+
+class Upsample_module(nn.Module):
+ """Upsample module for RSN.
+
+ Args:
+ unit_channels (int): Channel number in the upsample units.
+ Default:256.
+ num_units (int): Numbers of upsample units. Default: 4
+ gen_skip (bool): Whether to generate skip for posterior downsample
+ module or not. Default:False
+ gen_cross_conv (bool): Whether to generate feature map for the next
+ hourglass-like module. Default:False
+ norm_cfg (dict): dictionary to construct and config norm layer.
+ Default: dict(type='BN')
+ out_channels (int): Number of channels of feature output by upsample
+ module. Must equal to in_channels of downsample module. Default:64
+ """
+
+ def __init__(self,
+ unit_channels=256,
+ num_units=4,
+ gen_skip=False,
+ gen_cross_conv=False,
+ norm_cfg=dict(type='BN'),
+ out_channels=64):
+ # Protect mutable default arguments
+ norm_cfg = cp.deepcopy(norm_cfg)
+ super().__init__()
+ self.in_channels = list()
+ for i in range(num_units):
+ self.in_channels.append(RSB.expansion * out_channels * pow(2, i))
+ self.in_channels.reverse()
+ self.num_units = num_units
+ self.gen_skip = gen_skip
+ self.gen_cross_conv = gen_cross_conv
+ self.norm_cfg = norm_cfg
+ for i in range(num_units):
+ module_name = f'up{i + 1}'
+ self.add_module(
+ module_name,
+ Upsample_unit(
+ i,
+ self.num_units,
+ self.in_channels[i],
+ unit_channels,
+ self.gen_skip,
+ self.gen_cross_conv,
+ norm_cfg=self.norm_cfg,
+ out_channels=64))
+
+ def forward(self, x):
+ out = list()
+ skip1 = list()
+ skip2 = list()
+ cross_conv = None
+ for i in range(self.num_units):
+ module_i = getattr(self, f'up{i + 1}')
+ if i == 0:
+ outi, skip1_i, skip2_i, _ = module_i(x[i], None)
+ elif i == self.num_units - 1:
+ outi, skip1_i, skip2_i, cross_conv = module_i(x[i], out[i - 1])
+ else:
+ outi, skip1_i, skip2_i, _ = module_i(x[i], out[i - 1])
+ out.append(outi)
+ skip1.append(skip1_i)
+ skip2.append(skip2_i)
+ skip1.reverse()
+ skip2.reverse()
+
+ return out, skip1, skip2, cross_conv
+
+
+class Single_stage_RSN(nn.Module):
+ """Single_stage Residual Steps Network.
+
+ Args:
+ unit_channels (int): Channel number in the upsample units. Default:256.
+ num_units (int): Numbers of downsample/upsample units. Default: 4
+ gen_skip (bool): Whether to generate skip for posterior downsample
+ module or not. Default:False
+ gen_cross_conv (bool): Whether to generate feature map for the next
+ hourglass-like module. Default:False
+ has_skip (bool): Have skip connections from prior upsample
+ module or not. Default:False
+ num_steps (int): Number of steps in RSB. Default: 4
+ num_blocks (list): Number of blocks in each downsample unit.
+ Default: [2, 2, 2, 2] Note: Make sure num_units==len(num_blocks)
+ norm_cfg (dict): dictionary to construct and config norm layer.
+ Default: dict(type='BN')
+ in_channels (int): Number of channels of the feature from ResNet_Top.
+ Default: 64.
+ expand_times (int): Times by which the in_channels are expanded in RSB.
+ Default:26.
+ """
+
+ def __init__(self,
+ has_skip=False,
+ gen_skip=False,
+ gen_cross_conv=False,
+ unit_channels=256,
+ num_units=4,
+ num_steps=4,
+ num_blocks=[2, 2, 2, 2],
+ norm_cfg=dict(type='BN'),
+ in_channels=64,
+ expand_times=26):
+ # Protect mutable default arguments
+ norm_cfg = cp.deepcopy(norm_cfg)
+ num_blocks = cp.deepcopy(num_blocks)
+ super().__init__()
+ assert len(num_blocks) == num_units
+ self.has_skip = has_skip
+ self.gen_skip = gen_skip
+ self.gen_cross_conv = gen_cross_conv
+ self.num_units = num_units
+ self.num_steps = num_steps
+ self.unit_channels = unit_channels
+ self.num_blocks = num_blocks
+ self.norm_cfg = norm_cfg
+
+ self.downsample = Downsample_module(RSB, num_blocks, num_steps,
+ num_units, has_skip, norm_cfg,
+ in_channels, expand_times)
+ self.upsample = Upsample_module(unit_channels, num_units, gen_skip,
+ gen_cross_conv, norm_cfg, in_channels)
+
+ def forward(self, x, skip1, skip2):
+ mid = self.downsample(x, skip1, skip2)
+ out, skip1, skip2, cross_conv = self.upsample(mid)
+
+ return out, skip1, skip2, cross_conv
+
+
+class ResNet_top(nn.Module):
+ """ResNet top for RSN.
+
+ Args:
+ norm_cfg (dict): dictionary to construct and config norm layer.
+ Default: dict(type='BN')
+ channels (int): Number of channels of the feature output by ResNet_top.
+ """
+
+ def __init__(self, norm_cfg=dict(type='BN'), channels=64):
+ # Protect mutable default arguments
+ norm_cfg = cp.deepcopy(norm_cfg)
+ super().__init__()
+ self.top = nn.Sequential(
+ ConvModule(
+ 3,
+ channels,
+ kernel_size=7,
+ stride=2,
+ padding=3,
+ norm_cfg=norm_cfg,
+ inplace=True), MaxPool2d(kernel_size=3, stride=2, padding=1))
+
+ def forward(self, img):
+ return self.top(img)
+
+
+@BACKBONES.register_module()
+class RSN(BaseBackbone):
+ """Residual Steps Network backbone. Paper ref: Cai et al. "Learning
+ Delicate Local Representations for Multi-Person Pose Estimation" (ECCV
+ 2020).
+
+ Args:
+ unit_channels (int): Number of Channels in an upsample unit.
+ Default: 256
+ num_stages (int): Number of stages in a multi-stage RSN. Default: 4
+ num_units (int): NUmber of downsample/upsample units in a single-stage
+ RSN. Default: 4 Note: Make sure num_units == len(self.num_blocks)
+ num_blocks (list): Number of RSBs (Residual Steps Block) in each
+ downsample unit. Default: [2, 2, 2, 2]
+ num_steps (int): Number of steps in a RSB. Default:4
+ norm_cfg (dict): dictionary to construct and config norm layer.
+ Default: dict(type='BN')
+ res_top_channels (int): Number of channels of feature from ResNet_top.
+ Default: 64.
+ expand_times (int): Times by which the in_channels are expanded in RSB.
+ Default:26.
+ Example:
+ >>> from mmpose.models import RSN
+ >>> import torch
+ >>> self = RSN(num_stages=2,num_units=2,num_blocks=[2,2])
+ >>> self.eval()
+ >>> inputs = torch.rand(1, 3, 511, 511)
+ >>> level_outputs = self.forward(inputs)
+ >>> for level_output in level_outputs:
+ ... for feature in level_output:
+ ... print(tuple(feature.shape))
+ ...
+ (1, 256, 64, 64)
+ (1, 256, 128, 128)
+ (1, 256, 64, 64)
+ (1, 256, 128, 128)
+ """
+
+ def __init__(self,
+ unit_channels=256,
+ num_stages=4,
+ num_units=4,
+ num_blocks=[2, 2, 2, 2],
+ num_steps=4,
+ norm_cfg=dict(type='BN'),
+ res_top_channels=64,
+ expand_times=26):
+ # Protect mutable default arguments
+ norm_cfg = cp.deepcopy(norm_cfg)
+ num_blocks = cp.deepcopy(num_blocks)
+ super().__init__()
+ self.unit_channels = unit_channels
+ self.num_stages = num_stages
+ self.num_units = num_units
+ self.num_blocks = num_blocks
+ self.num_steps = num_steps
+ self.norm_cfg = norm_cfg
+
+ assert self.num_stages > 0
+ assert self.num_steps > 1
+ assert self.num_units > 1
+ assert self.num_units == len(self.num_blocks)
+ self.top = ResNet_top(norm_cfg=norm_cfg)
+ self.multi_stage_rsn = nn.ModuleList([])
+ for i in range(self.num_stages):
+ if i == 0:
+ has_skip = False
+ else:
+ has_skip = True
+ if i != self.num_stages - 1:
+ gen_skip = True
+ gen_cross_conv = True
+ else:
+ gen_skip = False
+ gen_cross_conv = False
+ self.multi_stage_rsn.append(
+ Single_stage_RSN(has_skip, gen_skip, gen_cross_conv,
+ unit_channels, num_units, num_steps,
+ num_blocks, norm_cfg, res_top_channels,
+ expand_times))
+
+ def forward(self, x):
+ """Model forward function."""
+ out_feats = []
+ skip1 = None
+ skip2 = None
+ x = self.top(x)
+ for i in range(self.num_stages):
+ out, skip1, skip2, x = self.multi_stage_rsn[i](x, skip1, skip2)
+ out_feats.append(out)
+
+ return out_feats
+
+ def init_weights(self, pretrained=None):
+ """Initialize model weights."""
+ for m in self.multi_stage_rsn.modules():
+ if isinstance(m, nn.Conv2d):
+ kaiming_init(m)
+ elif isinstance(m, nn.BatchNorm2d):
+ constant_init(m, 1)
+ elif isinstance(m, nn.Linear):
+ normal_init(m, std=0.01)
+
+ for m in self.top.modules():
+ if isinstance(m, nn.Conv2d):
+ kaiming_init(m)
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/scnet.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/scnet.py
new file mode 100644
index 0000000..3786c57
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/scnet.py
@@ -0,0 +1,248 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import copy
+
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+import torch.utils.checkpoint as cp
+from mmcv.cnn import build_conv_layer, build_norm_layer
+
+from ..builder import BACKBONES
+from .resnet import Bottleneck, ResNet
+
+
+class SCConv(nn.Module):
+ """SCConv (Self-calibrated Convolution)
+
+ Args:
+ in_channels (int): The input channels of the SCConv.
+ out_channels (int): The output channel of the SCConv.
+ stride (int): stride of SCConv.
+ pooling_r (int): size of pooling for scconv.
+ conv_cfg (dict): dictionary to construct and config conv layer.
+ Default: None
+ norm_cfg (dict): dictionary to construct and config norm layer.
+ Default: dict(type='BN')
+ """
+
+ def __init__(self,
+ in_channels,
+ out_channels,
+ stride,
+ pooling_r,
+ conv_cfg=None,
+ norm_cfg=dict(type='BN', momentum=0.1)):
+ # Protect mutable default arguments
+ norm_cfg = copy.deepcopy(norm_cfg)
+ super().__init__()
+
+ assert in_channels == out_channels
+
+ self.k2 = nn.Sequential(
+ nn.AvgPool2d(kernel_size=pooling_r, stride=pooling_r),
+ build_conv_layer(
+ conv_cfg,
+ in_channels,
+ in_channels,
+ kernel_size=3,
+ stride=1,
+ padding=1,
+ bias=False),
+ build_norm_layer(norm_cfg, in_channels)[1],
+ )
+ self.k3 = nn.Sequential(
+ build_conv_layer(
+ conv_cfg,
+ in_channels,
+ in_channels,
+ kernel_size=3,
+ stride=1,
+ padding=1,
+ bias=False),
+ build_norm_layer(norm_cfg, in_channels)[1],
+ )
+ self.k4 = nn.Sequential(
+ build_conv_layer(
+ conv_cfg,
+ in_channels,
+ in_channels,
+ kernel_size=3,
+ stride=stride,
+ padding=1,
+ bias=False),
+ build_norm_layer(norm_cfg, out_channels)[1],
+ nn.ReLU(inplace=True),
+ )
+
+ def forward(self, x):
+ """Forward function."""
+ identity = x
+
+ out = torch.sigmoid(
+ torch.add(identity, F.interpolate(self.k2(x),
+ identity.size()[2:])))
+ out = torch.mul(self.k3(x), out)
+ out = self.k4(out)
+
+ return out
+
+
+class SCBottleneck(Bottleneck):
+ """SC(Self-calibrated) Bottleneck.
+
+ Args:
+ in_channels (int): The input channels of the SCBottleneck block.
+ out_channels (int): The output channel of the SCBottleneck block.
+ """
+
+ pooling_r = 4
+
+ def __init__(self, in_channels, out_channels, **kwargs):
+ super().__init__(in_channels, out_channels, **kwargs)
+ self.mid_channels = out_channels // self.expansion // 2
+
+ self.norm1_name, norm1 = build_norm_layer(
+ self.norm_cfg, self.mid_channels, postfix=1)
+ self.norm2_name, norm2 = build_norm_layer(
+ self.norm_cfg, self.mid_channels, postfix=2)
+ self.norm3_name, norm3 = build_norm_layer(
+ self.norm_cfg, out_channels, postfix=3)
+
+ self.conv1 = build_conv_layer(
+ self.conv_cfg,
+ in_channels,
+ self.mid_channels,
+ kernel_size=1,
+ stride=1,
+ bias=False)
+ self.add_module(self.norm1_name, norm1)
+
+ self.k1 = nn.Sequential(
+ build_conv_layer(
+ self.conv_cfg,
+ self.mid_channels,
+ self.mid_channels,
+ kernel_size=3,
+ stride=self.stride,
+ padding=1,
+ bias=False),
+ build_norm_layer(self.norm_cfg, self.mid_channels)[1],
+ nn.ReLU(inplace=True))
+
+ self.conv2 = build_conv_layer(
+ self.conv_cfg,
+ in_channels,
+ self.mid_channels,
+ kernel_size=1,
+ stride=1,
+ bias=False)
+ self.add_module(self.norm2_name, norm2)
+
+ self.scconv = SCConv(self.mid_channels, self.mid_channels, self.stride,
+ self.pooling_r, self.conv_cfg, self.norm_cfg)
+
+ self.conv3 = build_conv_layer(
+ self.conv_cfg,
+ self.mid_channels * 2,
+ out_channels,
+ kernel_size=1,
+ stride=1,
+ bias=False)
+ self.add_module(self.norm3_name, norm3)
+
+ def forward(self, x):
+ """Forward function."""
+
+ def _inner_forward(x):
+ identity = x
+
+ out_a = self.conv1(x)
+ out_a = self.norm1(out_a)
+ out_a = self.relu(out_a)
+
+ out_a = self.k1(out_a)
+
+ out_b = self.conv2(x)
+ out_b = self.norm2(out_b)
+ out_b = self.relu(out_b)
+
+ out_b = self.scconv(out_b)
+
+ out = self.conv3(torch.cat([out_a, out_b], dim=1))
+ out = self.norm3(out)
+
+ if self.downsample is not None:
+ identity = self.downsample(x)
+
+ out += identity
+
+ return out
+
+ if self.with_cp and x.requires_grad:
+ out = cp.checkpoint(_inner_forward, x)
+ else:
+ out = _inner_forward(x)
+
+ out = self.relu(out)
+
+ return out
+
+
+@BACKBONES.register_module()
+class SCNet(ResNet):
+ """SCNet backbone.
+
+ Improving Convolutional Networks with Self-Calibrated Convolutions,
+ Jiang-Jiang Liu, Qibin Hou, Ming-Ming Cheng, Changhu Wang, Jiashi Feng,
+ IEEE CVPR, 2020.
+ http://mftp.mmcheng.net/Papers/20cvprSCNet.pdf
+
+ Args:
+ depth (int): Depth of scnet, from {50, 101}.
+ in_channels (int): Number of input image channels. Normally 3.
+ base_channels (int): Number of base channels of hidden layer.
+ num_stages (int): SCNet stages, normally 4.
+ strides (Sequence[int]): Strides of the first block of each stage.
+ dilations (Sequence[int]): Dilation of each stage.
+ out_indices (Sequence[int]): Output from which stages.
+ style (str): `pytorch` or `caffe`. If set to "pytorch", the stride-two
+ layer is the 3x3 conv layer, otherwise the stride-two layer is
+ the first 1x1 conv layer.
+ deep_stem (bool): Replace 7x7 conv in input stem with 3 3x3 conv
+ avg_down (bool): Use AvgPool instead of stride conv when
+ downsampling in the bottleneck.
+ frozen_stages (int): Stages to be frozen (stop grad and set eval mode).
+ -1 means not freezing any parameters.
+ norm_cfg (dict): Dictionary to construct and config norm layer.
+ norm_eval (bool): Whether to set norm layers to eval mode, namely,
+ freeze running stats (mean and var). Note: Effect on Batch Norm
+ and its variants only.
+ with_cp (bool): Use checkpoint or not. Using checkpoint will save some
+ memory while slowing down the training speed.
+ zero_init_residual (bool): Whether to use zero init for last norm layer
+ in resblocks to let them behave as identity.
+
+ Example:
+ >>> from mmpose.models import SCNet
+ >>> import torch
+ >>> self = SCNet(depth=50, out_indices=(0, 1, 2, 3))
+ >>> self.eval()
+ >>> inputs = torch.rand(1, 3, 224, 224)
+ >>> level_outputs = self.forward(inputs)
+ >>> for level_out in level_outputs:
+ ... print(tuple(level_out.shape))
+ (1, 256, 56, 56)
+ (1, 512, 28, 28)
+ (1, 1024, 14, 14)
+ (1, 2048, 7, 7)
+ """
+
+ arch_settings = {
+ 50: (SCBottleneck, [3, 4, 6, 3]),
+ 101: (SCBottleneck, [3, 4, 23, 3])
+ }
+
+ def __init__(self, depth, **kwargs):
+ if depth not in self.arch_settings:
+ raise KeyError(f'invalid depth {depth} for SCNet')
+ super().__init__(depth, **kwargs)
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/seresnet.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/seresnet.py
new file mode 100644
index 0000000..ac2d53b
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/seresnet.py
@@ -0,0 +1,125 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import torch.utils.checkpoint as cp
+
+from ..builder import BACKBONES
+from .resnet import Bottleneck, ResLayer, ResNet
+from .utils.se_layer import SELayer
+
+
+class SEBottleneck(Bottleneck):
+ """SEBottleneck block for SEResNet.
+
+ Args:
+ in_channels (int): The input channels of the SEBottleneck block.
+ out_channels (int): The output channel of the SEBottleneck block.
+ se_ratio (int): Squeeze ratio in SELayer. Default: 16
+ """
+
+ def __init__(self, in_channels, out_channels, se_ratio=16, **kwargs):
+ super().__init__(in_channels, out_channels, **kwargs)
+ self.se_layer = SELayer(out_channels, ratio=se_ratio)
+
+ def forward(self, x):
+
+ def _inner_forward(x):
+ identity = x
+
+ out = self.conv1(x)
+ out = self.norm1(out)
+ out = self.relu(out)
+
+ out = self.conv2(out)
+ out = self.norm2(out)
+ out = self.relu(out)
+
+ out = self.conv3(out)
+ out = self.norm3(out)
+
+ out = self.se_layer(out)
+
+ if self.downsample is not None:
+ identity = self.downsample(x)
+
+ out += identity
+
+ return out
+
+ if self.with_cp and x.requires_grad:
+ out = cp.checkpoint(_inner_forward, x)
+ else:
+ out = _inner_forward(x)
+
+ out = self.relu(out)
+
+ return out
+
+
+@BACKBONES.register_module()
+class SEResNet(ResNet):
+ """SEResNet backbone.
+
+ Please refer to the `paper `__ for
+ details.
+
+ Args:
+ depth (int): Network depth, from {50, 101, 152}.
+ se_ratio (int): Squeeze ratio in SELayer. Default: 16.
+ in_channels (int): Number of input image channels. Default: 3.
+ stem_channels (int): Output channels of the stem layer. Default: 64.
+ num_stages (int): Stages of the network. Default: 4.
+ strides (Sequence[int]): Strides of the first block of each stage.
+ Default: ``(1, 2, 2, 2)``.
+ dilations (Sequence[int]): Dilation of each stage.
+ Default: ``(1, 1, 1, 1)``.
+ out_indices (Sequence[int]): Output from which stages. If only one
+ stage is specified, a single tensor (feature map) is returned,
+ otherwise multiple stages are specified, a tuple of tensors will
+ be returned. Default: ``(3, )``.
+ style (str): `pytorch` or `caffe`. If set to "pytorch", the stride-two
+ layer is the 3x3 conv layer, otherwise the stride-two layer is
+ the first 1x1 conv layer.
+ deep_stem (bool): Replace 7x7 conv in input stem with 3 3x3 conv.
+ Default: False.
+ avg_down (bool): Use AvgPool instead of stride conv when
+ downsampling in the bottleneck. Default: False.
+ frozen_stages (int): Stages to be frozen (stop grad and set eval mode).
+ -1 means not freezing any parameters. Default: -1.
+ conv_cfg (dict | None): The config dict for conv layers. Default: None.
+ norm_cfg (dict): The config dict for norm layers.
+ norm_eval (bool): Whether to set norm layers to eval mode, namely,
+ freeze running stats (mean and var). Note: Effect on Batch Norm
+ and its variants only. Default: False.
+ with_cp (bool): Use checkpoint or not. Using checkpoint will save some
+ memory while slowing down the training speed. Default: False.
+ zero_init_residual (bool): Whether to use zero init for last norm layer
+ in resblocks to let them behave as identity. Default: True.
+
+ Example:
+ >>> from mmpose.models import SEResNet
+ >>> import torch
+ >>> self = SEResNet(depth=50, out_indices=(0, 1, 2, 3))
+ >>> self.eval()
+ >>> inputs = torch.rand(1, 3, 224, 224)
+ >>> level_outputs = self.forward(inputs)
+ >>> for level_out in level_outputs:
+ ... print(tuple(level_out.shape))
+ (1, 256, 56, 56)
+ (1, 512, 28, 28)
+ (1, 1024, 14, 14)
+ (1, 2048, 7, 7)
+ """
+
+ arch_settings = {
+ 50: (SEBottleneck, (3, 4, 6, 3)),
+ 101: (SEBottleneck, (3, 4, 23, 3)),
+ 152: (SEBottleneck, (3, 8, 36, 3))
+ }
+
+ def __init__(self, depth, se_ratio=16, **kwargs):
+ if depth not in self.arch_settings:
+ raise KeyError(f'invalid depth {depth} for SEResNet')
+ self.se_ratio = se_ratio
+ super().__init__(depth, **kwargs)
+
+ def make_res_layer(self, **kwargs):
+ return ResLayer(se_ratio=self.se_ratio, **kwargs)
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/seresnext.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/seresnext.py
new file mode 100644
index 0000000..c5c4e4c
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/seresnext.py
@@ -0,0 +1,168 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from mmcv.cnn import build_conv_layer, build_norm_layer
+
+from ..builder import BACKBONES
+from .resnet import ResLayer
+from .seresnet import SEBottleneck as _SEBottleneck
+from .seresnet import SEResNet
+
+
+class SEBottleneck(_SEBottleneck):
+ """SEBottleneck block for SEResNeXt.
+
+ Args:
+ in_channels (int): Input channels of this block.
+ out_channels (int): Output channels of this block.
+ base_channels (int): Middle channels of the first stage. Default: 64.
+ groups (int): Groups of conv2.
+ width_per_group (int): Width per group of conv2. 64x4d indicates
+ ``groups=64, width_per_group=4`` and 32x8d indicates
+ ``groups=32, width_per_group=8``.
+ stride (int): stride of the block. Default: 1
+ dilation (int): dilation of convolution. Default: 1
+ downsample (nn.Module): downsample operation on identity branch.
+ Default: None
+ se_ratio (int): Squeeze ratio in SELayer. Default: 16
+ style (str): `pytorch` or `caffe`. If set to "pytorch", the stride-two
+ layer is the 3x3 conv layer, otherwise the stride-two layer is
+ the first 1x1 conv layer.
+ conv_cfg (dict): dictionary to construct and config conv layer.
+ Default: None
+ norm_cfg (dict): dictionary to construct and config norm layer.
+ Default: dict(type='BN')
+ with_cp (bool): Use checkpoint or not. Using checkpoint will save some
+ memory while slowing down the training speed.
+ """
+
+ def __init__(self,
+ in_channels,
+ out_channels,
+ base_channels=64,
+ groups=32,
+ width_per_group=4,
+ se_ratio=16,
+ **kwargs):
+ super().__init__(in_channels, out_channels, se_ratio, **kwargs)
+ self.groups = groups
+ self.width_per_group = width_per_group
+
+ # We follow the same rational of ResNext to compute mid_channels.
+ # For SEResNet bottleneck, middle channels are determined by expansion
+ # and out_channels, but for SEResNeXt bottleneck, it is determined by
+ # groups and width_per_group and the stage it is located in.
+ if groups != 1:
+ assert self.mid_channels % base_channels == 0
+ self.mid_channels = (
+ groups * width_per_group * self.mid_channels // base_channels)
+
+ self.norm1_name, norm1 = build_norm_layer(
+ self.norm_cfg, self.mid_channels, postfix=1)
+ self.norm2_name, norm2 = build_norm_layer(
+ self.norm_cfg, self.mid_channels, postfix=2)
+ self.norm3_name, norm3 = build_norm_layer(
+ self.norm_cfg, self.out_channels, postfix=3)
+
+ self.conv1 = build_conv_layer(
+ self.conv_cfg,
+ self.in_channels,
+ self.mid_channels,
+ kernel_size=1,
+ stride=self.conv1_stride,
+ bias=False)
+ self.add_module(self.norm1_name, norm1)
+ self.conv2 = build_conv_layer(
+ self.conv_cfg,
+ self.mid_channels,
+ self.mid_channels,
+ kernel_size=3,
+ stride=self.conv2_stride,
+ padding=self.dilation,
+ dilation=self.dilation,
+ groups=groups,
+ bias=False)
+
+ self.add_module(self.norm2_name, norm2)
+ self.conv3 = build_conv_layer(
+ self.conv_cfg,
+ self.mid_channels,
+ self.out_channels,
+ kernel_size=1,
+ bias=False)
+ self.add_module(self.norm3_name, norm3)
+
+
+@BACKBONES.register_module()
+class SEResNeXt(SEResNet):
+ """SEResNeXt backbone.
+
+ Please refer to the `paper `__ for
+ details.
+
+ Args:
+ depth (int): Network depth, from {50, 101, 152}.
+ groups (int): Groups of conv2 in Bottleneck. Default: 32.
+ width_per_group (int): Width per group of conv2 in Bottleneck.
+ Default: 4.
+ se_ratio (int): Squeeze ratio in SELayer. Default: 16.
+ in_channels (int): Number of input image channels. Default: 3.
+ stem_channels (int): Output channels of the stem layer. Default: 64.
+ num_stages (int): Stages of the network. Default: 4.
+ strides (Sequence[int]): Strides of the first block of each stage.
+ Default: ``(1, 2, 2, 2)``.
+ dilations (Sequence[int]): Dilation of each stage.
+ Default: ``(1, 1, 1, 1)``.
+ out_indices (Sequence[int]): Output from which stages. If only one
+ stage is specified, a single tensor (feature map) is returned,
+ otherwise multiple stages are specified, a tuple of tensors will
+ be returned. Default: ``(3, )``.
+ style (str): `pytorch` or `caffe`. If set to "pytorch", the stride-two
+ layer is the 3x3 conv layer, otherwise the stride-two layer is
+ the first 1x1 conv layer.
+ deep_stem (bool): Replace 7x7 conv in input stem with 3 3x3 conv.
+ Default: False.
+ avg_down (bool): Use AvgPool instead of stride conv when
+ downsampling in the bottleneck. Default: False.
+ frozen_stages (int): Stages to be frozen (stop grad and set eval mode).
+ -1 means not freezing any parameters. Default: -1.
+ conv_cfg (dict | None): The config dict for conv layers. Default: None.
+ norm_cfg (dict): The config dict for norm layers.
+ norm_eval (bool): Whether to set norm layers to eval mode, namely,
+ freeze running stats (mean and var). Note: Effect on Batch Norm
+ and its variants only. Default: False.
+ with_cp (bool): Use checkpoint or not. Using checkpoint will save some
+ memory while slowing down the training speed. Default: False.
+ zero_init_residual (bool): Whether to use zero init for last norm layer
+ in resblocks to let them behave as identity. Default: True.
+
+ Example:
+ >>> from mmpose.models import SEResNeXt
+ >>> import torch
+ >>> self = SEResNet(depth=50, out_indices=(0, 1, 2, 3))
+ >>> self.eval()
+ >>> inputs = torch.rand(1, 3, 224, 224)
+ >>> level_outputs = self.forward(inputs)
+ >>> for level_out in level_outputs:
+ ... print(tuple(level_out.shape))
+ (1, 256, 56, 56)
+ (1, 512, 28, 28)
+ (1, 1024, 14, 14)
+ (1, 2048, 7, 7)
+ """
+
+ arch_settings = {
+ 50: (SEBottleneck, (3, 4, 6, 3)),
+ 101: (SEBottleneck, (3, 4, 23, 3)),
+ 152: (SEBottleneck, (3, 8, 36, 3))
+ }
+
+ def __init__(self, depth, groups=32, width_per_group=4, **kwargs):
+ self.groups = groups
+ self.width_per_group = width_per_group
+ super().__init__(depth, **kwargs)
+
+ def make_res_layer(self, **kwargs):
+ return ResLayer(
+ groups=self.groups,
+ width_per_group=self.width_per_group,
+ base_channels=self.base_channels,
+ **kwargs)
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/shufflenet_v1.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/shufflenet_v1.py
new file mode 100644
index 0000000..9f98cbd
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/shufflenet_v1.py
@@ -0,0 +1,329 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import copy
+import logging
+
+import torch
+import torch.nn as nn
+import torch.utils.checkpoint as cp
+from mmcv.cnn import (ConvModule, build_activation_layer, constant_init,
+ normal_init)
+from torch.nn.modules.batchnorm import _BatchNorm
+
+from ..builder import BACKBONES
+from .base_backbone import BaseBackbone
+from .utils import channel_shuffle, load_checkpoint, make_divisible
+
+
+class ShuffleUnit(nn.Module):
+ """ShuffleUnit block.
+
+ ShuffleNet unit with pointwise group convolution (GConv) and channel
+ shuffle.
+
+ Args:
+ in_channels (int): The input channels of the ShuffleUnit.
+ out_channels (int): The output channels of the ShuffleUnit.
+ groups (int, optional): The number of groups to be used in grouped 1x1
+ convolutions in each ShuffleUnit. Default: 3
+ first_block (bool, optional): Whether it is the first ShuffleUnit of a
+ sequential ShuffleUnits. Default: True, which means not using the
+ grouped 1x1 convolution.
+ combine (str, optional): The ways to combine the input and output
+ branches. Default: 'add'.
+ conv_cfg (dict): Config dict for convolution layer. Default: None,
+ which means using conv2d.
+ norm_cfg (dict): Config dict for normalization layer.
+ Default: dict(type='BN').
+ act_cfg (dict): Config dict for activation layer.
+ Default: dict(type='ReLU').
+ with_cp (bool, optional): Use checkpoint or not. Using checkpoint
+ will save some memory while slowing down the training speed.
+ Default: False.
+
+ Returns:
+ Tensor: The output tensor.
+ """
+
+ def __init__(self,
+ in_channels,
+ out_channels,
+ groups=3,
+ first_block=True,
+ combine='add',
+ conv_cfg=None,
+ norm_cfg=dict(type='BN'),
+ act_cfg=dict(type='ReLU'),
+ with_cp=False):
+ # Protect mutable default arguments
+ norm_cfg = copy.deepcopy(norm_cfg)
+ act_cfg = copy.deepcopy(act_cfg)
+ super().__init__()
+ self.in_channels = in_channels
+ self.out_channels = out_channels
+ self.first_block = first_block
+ self.combine = combine
+ self.groups = groups
+ self.bottleneck_channels = self.out_channels // 4
+ self.with_cp = with_cp
+
+ if self.combine == 'add':
+ self.depthwise_stride = 1
+ self._combine_func = self._add
+ assert in_channels == out_channels, (
+ 'in_channels must be equal to out_channels when combine '
+ 'is add')
+ elif self.combine == 'concat':
+ self.depthwise_stride = 2
+ self._combine_func = self._concat
+ self.out_channels -= self.in_channels
+ self.avgpool = nn.AvgPool2d(kernel_size=3, stride=2, padding=1)
+ else:
+ raise ValueError(f'Cannot combine tensors with {self.combine}. '
+ 'Only "add" and "concat" are supported')
+
+ self.first_1x1_groups = 1 if first_block else self.groups
+ self.g_conv_1x1_compress = ConvModule(
+ in_channels=self.in_channels,
+ out_channels=self.bottleneck_channels,
+ kernel_size=1,
+ groups=self.first_1x1_groups,
+ conv_cfg=conv_cfg,
+ norm_cfg=norm_cfg,
+ act_cfg=act_cfg)
+
+ self.depthwise_conv3x3_bn = ConvModule(
+ in_channels=self.bottleneck_channels,
+ out_channels=self.bottleneck_channels,
+ kernel_size=3,
+ stride=self.depthwise_stride,
+ padding=1,
+ groups=self.bottleneck_channels,
+ conv_cfg=conv_cfg,
+ norm_cfg=norm_cfg,
+ act_cfg=None)
+
+ self.g_conv_1x1_expand = ConvModule(
+ in_channels=self.bottleneck_channels,
+ out_channels=self.out_channels,
+ kernel_size=1,
+ groups=self.groups,
+ conv_cfg=conv_cfg,
+ norm_cfg=norm_cfg,
+ act_cfg=None)
+
+ self.act = build_activation_layer(act_cfg)
+
+ @staticmethod
+ def _add(x, out):
+ # residual connection
+ return x + out
+
+ @staticmethod
+ def _concat(x, out):
+ # concatenate along channel axis
+ return torch.cat((x, out), 1)
+
+ def forward(self, x):
+
+ def _inner_forward(x):
+ residual = x
+
+ out = self.g_conv_1x1_compress(x)
+ out = self.depthwise_conv3x3_bn(out)
+
+ if self.groups > 1:
+ out = channel_shuffle(out, self.groups)
+
+ out = self.g_conv_1x1_expand(out)
+
+ if self.combine == 'concat':
+ residual = self.avgpool(residual)
+ out = self.act(out)
+ out = self._combine_func(residual, out)
+ else:
+ out = self._combine_func(residual, out)
+ out = self.act(out)
+ return out
+
+ if self.with_cp and x.requires_grad:
+ out = cp.checkpoint(_inner_forward, x)
+ else:
+ out = _inner_forward(x)
+
+ return out
+
+
+@BACKBONES.register_module()
+class ShuffleNetV1(BaseBackbone):
+ """ShuffleNetV1 backbone.
+
+ Args:
+ groups (int, optional): The number of groups to be used in grouped 1x1
+ convolutions in each ShuffleUnit. Default: 3.
+ widen_factor (float, optional): Width multiplier - adjusts the number
+ of channels in each layer by this amount. Default: 1.0.
+ out_indices (Sequence[int]): Output from which stages.
+ Default: (2, )
+ frozen_stages (int): Stages to be frozen (all param fixed).
+ Default: -1, which means not freezing any parameters.
+ conv_cfg (dict): Config dict for convolution layer. Default: None,
+ which means using conv2d.
+ norm_cfg (dict): Config dict for normalization layer.
+ Default: dict(type='BN').
+ act_cfg (dict): Config dict for activation layer.
+ Default: dict(type='ReLU').
+ norm_eval (bool): Whether to set norm layers to eval mode, namely,
+ freeze running stats (mean and var). Note: Effect on Batch Norm
+ and its variants only. Default: False.
+ with_cp (bool): Use checkpoint or not. Using checkpoint will save some
+ memory while slowing down the training speed. Default: False.
+ """
+
+ def __init__(self,
+ groups=3,
+ widen_factor=1.0,
+ out_indices=(2, ),
+ frozen_stages=-1,
+ conv_cfg=None,
+ norm_cfg=dict(type='BN'),
+ act_cfg=dict(type='ReLU'),
+ norm_eval=False,
+ with_cp=False):
+ # Protect mutable default arguments
+ norm_cfg = copy.deepcopy(norm_cfg)
+ act_cfg = copy.deepcopy(act_cfg)
+ super().__init__()
+ self.stage_blocks = [4, 8, 4]
+ self.groups = groups
+
+ for index in out_indices:
+ if index not in range(0, 3):
+ raise ValueError('the item in out_indices must in '
+ f'range(0, 3). But received {index}')
+
+ if frozen_stages not in range(-1, 3):
+ raise ValueError('frozen_stages must be in range(-1, 3). '
+ f'But received {frozen_stages}')
+ self.out_indices = out_indices
+ self.frozen_stages = frozen_stages
+ self.conv_cfg = conv_cfg
+ self.norm_cfg = norm_cfg
+ self.act_cfg = act_cfg
+ self.norm_eval = norm_eval
+ self.with_cp = with_cp
+
+ if groups == 1:
+ channels = (144, 288, 576)
+ elif groups == 2:
+ channels = (200, 400, 800)
+ elif groups == 3:
+ channels = (240, 480, 960)
+ elif groups == 4:
+ channels = (272, 544, 1088)
+ elif groups == 8:
+ channels = (384, 768, 1536)
+ else:
+ raise ValueError(f'{groups} groups is not supported for 1x1 '
+ 'Grouped Convolutions')
+
+ channels = [make_divisible(ch * widen_factor, 8) for ch in channels]
+
+ self.in_channels = int(24 * widen_factor)
+
+ self.conv1 = ConvModule(
+ in_channels=3,
+ out_channels=self.in_channels,
+ kernel_size=3,
+ stride=2,
+ padding=1,
+ conv_cfg=conv_cfg,
+ norm_cfg=norm_cfg,
+ act_cfg=act_cfg)
+ self.maxpool = nn.MaxPool2d(kernel_size=3, stride=2, padding=1)
+
+ self.layers = nn.ModuleList()
+ for i, num_blocks in enumerate(self.stage_blocks):
+ first_block = (i == 0)
+ layer = self.make_layer(channels[i], num_blocks, first_block)
+ self.layers.append(layer)
+
+ def _freeze_stages(self):
+ if self.frozen_stages >= 0:
+ for param in self.conv1.parameters():
+ param.requires_grad = False
+ for i in range(self.frozen_stages):
+ layer = self.layers[i]
+ layer.eval()
+ for param in layer.parameters():
+ param.requires_grad = False
+
+ def init_weights(self, pretrained=None):
+ if isinstance(pretrained, str):
+ logger = logging.getLogger()
+ load_checkpoint(self, pretrained, strict=False, logger=logger)
+ elif pretrained is None:
+ for name, m in self.named_modules():
+ if isinstance(m, nn.Conv2d):
+ if 'conv1' in name:
+ normal_init(m, mean=0, std=0.01)
+ else:
+ normal_init(m, mean=0, std=1.0 / m.weight.shape[1])
+ elif isinstance(m, (_BatchNorm, nn.GroupNorm)):
+ constant_init(m, val=1, bias=0.0001)
+ if isinstance(m, _BatchNorm):
+ if m.running_mean is not None:
+ nn.init.constant_(m.running_mean, 0)
+ else:
+ raise TypeError('pretrained must be a str or None. But received '
+ f'{type(pretrained)}')
+
+ def make_layer(self, out_channels, num_blocks, first_block=False):
+ """Stack ShuffleUnit blocks to make a layer.
+
+ Args:
+ out_channels (int): out_channels of the block.
+ num_blocks (int): Number of blocks.
+ first_block (bool, optional): Whether is the first ShuffleUnit of a
+ sequential ShuffleUnits. Default: False, which means using
+ the grouped 1x1 convolution.
+ """
+ layers = []
+ for i in range(num_blocks):
+ first_block = first_block if i == 0 else False
+ combine_mode = 'concat' if i == 0 else 'add'
+ layers.append(
+ ShuffleUnit(
+ self.in_channels,
+ out_channels,
+ groups=self.groups,
+ first_block=first_block,
+ combine=combine_mode,
+ conv_cfg=self.conv_cfg,
+ norm_cfg=self.norm_cfg,
+ act_cfg=self.act_cfg,
+ with_cp=self.with_cp))
+ self.in_channels = out_channels
+
+ return nn.Sequential(*layers)
+
+ def forward(self, x):
+ x = self.conv1(x)
+ x = self.maxpool(x)
+
+ outs = []
+ for i, layer in enumerate(self.layers):
+ x = layer(x)
+ if i in self.out_indices:
+ outs.append(x)
+
+ if len(outs) == 1:
+ return outs[0]
+ return tuple(outs)
+
+ def train(self, mode=True):
+ super().train(mode)
+ self._freeze_stages()
+ if mode and self.norm_eval:
+ for m in self.modules():
+ if isinstance(m, _BatchNorm):
+ m.eval()
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/shufflenet_v2.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/shufflenet_v2.py
new file mode 100644
index 0000000..e935333
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/shufflenet_v2.py
@@ -0,0 +1,302 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import copy
+import logging
+
+import torch
+import torch.nn as nn
+import torch.utils.checkpoint as cp
+from mmcv.cnn import ConvModule, constant_init, normal_init
+from torch.nn.modules.batchnorm import _BatchNorm
+
+from ..builder import BACKBONES
+from .base_backbone import BaseBackbone
+from .utils import channel_shuffle, load_checkpoint
+
+
+class InvertedResidual(nn.Module):
+ """InvertedResidual block for ShuffleNetV2 backbone.
+
+ Args:
+ in_channels (int): The input channels of the block.
+ out_channels (int): The output channels of the block.
+ stride (int): Stride of the 3x3 convolution layer. Default: 1
+ conv_cfg (dict): Config dict for convolution layer.
+ Default: None, which means using conv2d.
+ norm_cfg (dict): Config dict for normalization layer.
+ Default: dict(type='BN').
+ act_cfg (dict): Config dict for activation layer.
+ Default: dict(type='ReLU').
+ with_cp (bool): Use checkpoint or not. Using checkpoint will save some
+ memory while slowing down the training speed. Default: False.
+ """
+
+ def __init__(self,
+ in_channels,
+ out_channels,
+ stride=1,
+ conv_cfg=None,
+ norm_cfg=dict(type='BN'),
+ act_cfg=dict(type='ReLU'),
+ with_cp=False):
+ # Protect mutable default arguments
+ norm_cfg = copy.deepcopy(norm_cfg)
+ act_cfg = copy.deepcopy(act_cfg)
+ super().__init__()
+ self.stride = stride
+ self.with_cp = with_cp
+
+ branch_features = out_channels // 2
+ if self.stride == 1:
+ assert in_channels == branch_features * 2, (
+ f'in_channels ({in_channels}) should equal to '
+ f'branch_features * 2 ({branch_features * 2}) '
+ 'when stride is 1')
+
+ if in_channels != branch_features * 2:
+ assert self.stride != 1, (
+ f'stride ({self.stride}) should not equal 1 when '
+ f'in_channels != branch_features * 2')
+
+ if self.stride > 1:
+ self.branch1 = nn.Sequential(
+ ConvModule(
+ in_channels,
+ in_channels,
+ kernel_size=3,
+ stride=self.stride,
+ padding=1,
+ groups=in_channels,
+ conv_cfg=conv_cfg,
+ norm_cfg=norm_cfg,
+ act_cfg=None),
+ ConvModule(
+ in_channels,
+ branch_features,
+ kernel_size=1,
+ stride=1,
+ padding=0,
+ conv_cfg=conv_cfg,
+ norm_cfg=norm_cfg,
+ act_cfg=act_cfg),
+ )
+
+ self.branch2 = nn.Sequential(
+ ConvModule(
+ in_channels if (self.stride > 1) else branch_features,
+ branch_features,
+ kernel_size=1,
+ stride=1,
+ padding=0,
+ conv_cfg=conv_cfg,
+ norm_cfg=norm_cfg,
+ act_cfg=act_cfg),
+ ConvModule(
+ branch_features,
+ branch_features,
+ kernel_size=3,
+ stride=self.stride,
+ padding=1,
+ groups=branch_features,
+ conv_cfg=conv_cfg,
+ norm_cfg=norm_cfg,
+ act_cfg=None),
+ ConvModule(
+ branch_features,
+ branch_features,
+ kernel_size=1,
+ stride=1,
+ padding=0,
+ conv_cfg=conv_cfg,
+ norm_cfg=norm_cfg,
+ act_cfg=act_cfg))
+
+ def forward(self, x):
+
+ def _inner_forward(x):
+ if self.stride > 1:
+ out = torch.cat((self.branch1(x), self.branch2(x)), dim=1)
+ else:
+ x1, x2 = x.chunk(2, dim=1)
+ out = torch.cat((x1, self.branch2(x2)), dim=1)
+
+ out = channel_shuffle(out, 2)
+
+ return out
+
+ if self.with_cp and x.requires_grad:
+ out = cp.checkpoint(_inner_forward, x)
+ else:
+ out = _inner_forward(x)
+
+ return out
+
+
+@BACKBONES.register_module()
+class ShuffleNetV2(BaseBackbone):
+ """ShuffleNetV2 backbone.
+
+ Args:
+ widen_factor (float): Width multiplier - adjusts the number of
+ channels in each layer by this amount. Default: 1.0.
+ out_indices (Sequence[int]): Output from which stages.
+ Default: (0, 1, 2, 3).
+ frozen_stages (int): Stages to be frozen (all param fixed).
+ Default: -1, which means not freezing any parameters.
+ conv_cfg (dict): Config dict for convolution layer.
+ Default: None, which means using conv2d.
+ norm_cfg (dict): Config dict for normalization layer.
+ Default: dict(type='BN').
+ act_cfg (dict): Config dict for activation layer.
+ Default: dict(type='ReLU').
+ norm_eval (bool): Whether to set norm layers to eval mode, namely,
+ freeze running stats (mean and var). Note: Effect on Batch Norm
+ and its variants only. Default: False.
+ with_cp (bool): Use checkpoint or not. Using checkpoint will save some
+ memory while slowing down the training speed. Default: False.
+ """
+
+ def __init__(self,
+ widen_factor=1.0,
+ out_indices=(3, ),
+ frozen_stages=-1,
+ conv_cfg=None,
+ norm_cfg=dict(type='BN'),
+ act_cfg=dict(type='ReLU'),
+ norm_eval=False,
+ with_cp=False):
+ # Protect mutable default arguments
+ norm_cfg = copy.deepcopy(norm_cfg)
+ act_cfg = copy.deepcopy(act_cfg)
+ super().__init__()
+ self.stage_blocks = [4, 8, 4]
+ for index in out_indices:
+ if index not in range(0, 4):
+ raise ValueError('the item in out_indices must in '
+ f'range(0, 4). But received {index}')
+
+ if frozen_stages not in range(-1, 4):
+ raise ValueError('frozen_stages must be in range(-1, 4). '
+ f'But received {frozen_stages}')
+ self.out_indices = out_indices
+ self.frozen_stages = frozen_stages
+ self.conv_cfg = conv_cfg
+ self.norm_cfg = norm_cfg
+ self.act_cfg = act_cfg
+ self.norm_eval = norm_eval
+ self.with_cp = with_cp
+
+ if widen_factor == 0.5:
+ channels = [48, 96, 192, 1024]
+ elif widen_factor == 1.0:
+ channels = [116, 232, 464, 1024]
+ elif widen_factor == 1.5:
+ channels = [176, 352, 704, 1024]
+ elif widen_factor == 2.0:
+ channels = [244, 488, 976, 2048]
+ else:
+ raise ValueError('widen_factor must be in [0.5, 1.0, 1.5, 2.0]. '
+ f'But received {widen_factor}')
+
+ self.in_channels = 24
+ self.conv1 = ConvModule(
+ in_channels=3,
+ out_channels=self.in_channels,
+ kernel_size=3,
+ stride=2,
+ padding=1,
+ conv_cfg=conv_cfg,
+ norm_cfg=norm_cfg,
+ act_cfg=act_cfg)
+
+ self.maxpool = nn.MaxPool2d(kernel_size=3, stride=2, padding=1)
+
+ self.layers = nn.ModuleList()
+ for i, num_blocks in enumerate(self.stage_blocks):
+ layer = self._make_layer(channels[i], num_blocks)
+ self.layers.append(layer)
+
+ output_channels = channels[-1]
+ self.layers.append(
+ ConvModule(
+ in_channels=self.in_channels,
+ out_channels=output_channels,
+ kernel_size=1,
+ conv_cfg=conv_cfg,
+ norm_cfg=norm_cfg,
+ act_cfg=act_cfg))
+
+ def _make_layer(self, out_channels, num_blocks):
+ """Stack blocks to make a layer.
+
+ Args:
+ out_channels (int): out_channels of the block.
+ num_blocks (int): number of blocks.
+ """
+ layers = []
+ for i in range(num_blocks):
+ stride = 2 if i == 0 else 1
+ layers.append(
+ InvertedResidual(
+ in_channels=self.in_channels,
+ out_channels=out_channels,
+ stride=stride,
+ conv_cfg=self.conv_cfg,
+ norm_cfg=self.norm_cfg,
+ act_cfg=self.act_cfg,
+ with_cp=self.with_cp))
+ self.in_channels = out_channels
+
+ return nn.Sequential(*layers)
+
+ def _freeze_stages(self):
+ if self.frozen_stages >= 0:
+ for param in self.conv1.parameters():
+ param.requires_grad = False
+
+ for i in range(self.frozen_stages):
+ m = self.layers[i]
+ m.eval()
+ for param in m.parameters():
+ param.requires_grad = False
+
+ def init_weights(self, pretrained=None):
+ if isinstance(pretrained, str):
+ logger = logging.getLogger()
+ load_checkpoint(self, pretrained, strict=False, logger=logger)
+ elif pretrained is None:
+ for name, m in self.named_modules():
+ if isinstance(m, nn.Conv2d):
+ if 'conv1' in name:
+ normal_init(m, mean=0, std=0.01)
+ else:
+ normal_init(m, mean=0, std=1.0 / m.weight.shape[1])
+ elif isinstance(m, (_BatchNorm, nn.GroupNorm)):
+ constant_init(m.weight, val=1, bias=0.0001)
+ if isinstance(m, _BatchNorm):
+ if m.running_mean is not None:
+ nn.init.constant_(m.running_mean, 0)
+ else:
+ raise TypeError('pretrained must be a str or None. But received '
+ f'{type(pretrained)}')
+
+ def forward(self, x):
+ x = self.conv1(x)
+ x = self.maxpool(x)
+
+ outs = []
+ for i, layer in enumerate(self.layers):
+ x = layer(x)
+ if i in self.out_indices:
+ outs.append(x)
+
+ if len(outs) == 1:
+ return outs[0]
+ return tuple(outs)
+
+ def train(self, mode=True):
+ super().train(mode)
+ self._freeze_stages()
+ if mode and self.norm_eval:
+ for m in self.modules():
+ if isinstance(m, nn.BatchNorm2d):
+ m.eval()
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/tcn.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/tcn.py
new file mode 100644
index 0000000..deca229
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/tcn.py
@@ -0,0 +1,267 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import copy
+
+import torch.nn as nn
+from mmcv.cnn import ConvModule, build_conv_layer, constant_init, kaiming_init
+from mmcv.utils.parrots_wrapper import _BatchNorm
+
+from mmpose.core import WeightNormClipHook
+from ..builder import BACKBONES
+from .base_backbone import BaseBackbone
+
+
+class BasicTemporalBlock(nn.Module):
+ """Basic block for VideoPose3D.
+
+ Args:
+ in_channels (int): Input channels of this block.
+ out_channels (int): Output channels of this block.
+ mid_channels (int): The output channels of conv1. Default: 1024.
+ kernel_size (int): Size of the convolving kernel. Default: 3.
+ dilation (int): Spacing between kernel elements. Default: 3.
+ dropout (float): Dropout rate. Default: 0.25.
+ causal (bool): Use causal convolutions instead of symmetric
+ convolutions (for real-time applications). Default: False.
+ residual (bool): Use residual connection. Default: True.
+ use_stride_conv (bool): Use optimized TCN that designed
+ specifically for single-frame batching, i.e. where batches have
+ input length = receptive field, and output length = 1. This
+ implementation replaces dilated convolutions with strided
+ convolutions to avoid generating unused intermediate results.
+ Default: False.
+ conv_cfg (dict): dictionary to construct and config conv layer.
+ Default: dict(type='Conv1d').
+ norm_cfg (dict): dictionary to construct and config norm layer.
+ Default: dict(type='BN1d').
+ """
+
+ def __init__(self,
+ in_channels,
+ out_channels,
+ mid_channels=1024,
+ kernel_size=3,
+ dilation=3,
+ dropout=0.25,
+ causal=False,
+ residual=True,
+ use_stride_conv=False,
+ conv_cfg=dict(type='Conv1d'),
+ norm_cfg=dict(type='BN1d')):
+ # Protect mutable default arguments
+ conv_cfg = copy.deepcopy(conv_cfg)
+ norm_cfg = copy.deepcopy(norm_cfg)
+ super().__init__()
+ self.in_channels = in_channels
+ self.out_channels = out_channels
+ self.mid_channels = mid_channels
+ self.kernel_size = kernel_size
+ self.dilation = dilation
+ self.dropout = dropout
+ self.causal = causal
+ self.residual = residual
+ self.use_stride_conv = use_stride_conv
+
+ self.pad = (kernel_size - 1) * dilation // 2
+ if use_stride_conv:
+ self.stride = kernel_size
+ self.causal_shift = kernel_size // 2 if causal else 0
+ self.dilation = 1
+ else:
+ self.stride = 1
+ self.causal_shift = kernel_size // 2 * dilation if causal else 0
+
+ self.conv1 = nn.Sequential(
+ ConvModule(
+ in_channels,
+ mid_channels,
+ kernel_size=kernel_size,
+ stride=self.stride,
+ dilation=self.dilation,
+ bias='auto',
+ conv_cfg=conv_cfg,
+ norm_cfg=norm_cfg))
+ self.conv2 = nn.Sequential(
+ ConvModule(
+ mid_channels,
+ out_channels,
+ kernel_size=1,
+ bias='auto',
+ conv_cfg=conv_cfg,
+ norm_cfg=norm_cfg))
+
+ if residual and in_channels != out_channels:
+ self.short_cut = build_conv_layer(conv_cfg, in_channels,
+ out_channels, 1)
+ else:
+ self.short_cut = None
+
+ self.dropout = nn.Dropout(dropout) if dropout > 0 else None
+
+ def forward(self, x):
+ """Forward function."""
+ if self.use_stride_conv:
+ assert self.causal_shift + self.kernel_size // 2 < x.shape[2]
+ else:
+ assert 0 <= self.pad + self.causal_shift < x.shape[2] - \
+ self.pad + self.causal_shift <= x.shape[2]
+
+ out = self.conv1(x)
+ if self.dropout is not None:
+ out = self.dropout(out)
+
+ out = self.conv2(out)
+ if self.dropout is not None:
+ out = self.dropout(out)
+
+ if self.residual:
+ if self.use_stride_conv:
+ res = x[:, :, self.causal_shift +
+ self.kernel_size // 2::self.kernel_size]
+ else:
+ res = x[:, :,
+ (self.pad + self.causal_shift):(x.shape[2] - self.pad +
+ self.causal_shift)]
+
+ if self.short_cut is not None:
+ res = self.short_cut(res)
+ out = out + res
+
+ return out
+
+
+@BACKBONES.register_module()
+class TCN(BaseBackbone):
+ """TCN backbone.
+
+ Temporal Convolutional Networks.
+ More details can be found in the
+ `paper `__ .
+
+ Args:
+ in_channels (int): Number of input channels, which equals to
+ num_keypoints * num_features.
+ stem_channels (int): Number of feature channels. Default: 1024.
+ num_blocks (int): NUmber of basic temporal convolutional blocks.
+ Default: 2.
+ kernel_sizes (Sequence[int]): Sizes of the convolving kernel of
+ each basic block. Default: ``(3, 3, 3)``.
+ dropout (float): Dropout rate. Default: 0.25.
+ causal (bool): Use causal convolutions instead of symmetric
+ convolutions (for real-time applications).
+ Default: False.
+ residual (bool): Use residual connection. Default: True.
+ use_stride_conv (bool): Use TCN backbone optimized for
+ single-frame batching, i.e. where batches have input length =
+ receptive field, and output length = 1. This implementation
+ replaces dilated convolutions with strided convolutions to avoid
+ generating unused intermediate results. The weights are
+ interchangeable with the reference implementation. Default: False
+ conv_cfg (dict): dictionary to construct and config conv layer.
+ Default: dict(type='Conv1d').
+ norm_cfg (dict): dictionary to construct and config norm layer.
+ Default: dict(type='BN1d').
+ max_norm (float|None): if not None, the weight of convolution layers
+ will be clipped to have a maximum norm of max_norm.
+
+ Example:
+ >>> from mmpose.models import TCN
+ >>> import torch
+ >>> self = TCN(in_channels=34)
+ >>> self.eval()
+ >>> inputs = torch.rand(1, 34, 243)
+ >>> level_outputs = self.forward(inputs)
+ >>> for level_out in level_outputs:
+ ... print(tuple(level_out.shape))
+ (1, 1024, 235)
+ (1, 1024, 217)
+ """
+
+ def __init__(self,
+ in_channels,
+ stem_channels=1024,
+ num_blocks=2,
+ kernel_sizes=(3, 3, 3),
+ dropout=0.25,
+ causal=False,
+ residual=True,
+ use_stride_conv=False,
+ conv_cfg=dict(type='Conv1d'),
+ norm_cfg=dict(type='BN1d'),
+ max_norm=None):
+ # Protect mutable default arguments
+ conv_cfg = copy.deepcopy(conv_cfg)
+ norm_cfg = copy.deepcopy(norm_cfg)
+ super().__init__()
+ self.in_channels = in_channels
+ self.stem_channels = stem_channels
+ self.num_blocks = num_blocks
+ self.kernel_sizes = kernel_sizes
+ self.dropout = dropout
+ self.causal = causal
+ self.residual = residual
+ self.use_stride_conv = use_stride_conv
+ self.max_norm = max_norm
+
+ assert num_blocks == len(kernel_sizes) - 1
+ for ks in kernel_sizes:
+ assert ks % 2 == 1, 'Only odd filter widths are supported.'
+
+ self.expand_conv = ConvModule(
+ in_channels,
+ stem_channels,
+ kernel_size=kernel_sizes[0],
+ stride=kernel_sizes[0] if use_stride_conv else 1,
+ bias='auto',
+ conv_cfg=conv_cfg,
+ norm_cfg=norm_cfg)
+
+ dilation = kernel_sizes[0]
+ self.tcn_blocks = nn.ModuleList()
+ for i in range(1, num_blocks + 1):
+ self.tcn_blocks.append(
+ BasicTemporalBlock(
+ in_channels=stem_channels,
+ out_channels=stem_channels,
+ mid_channels=stem_channels,
+ kernel_size=kernel_sizes[i],
+ dilation=dilation,
+ dropout=dropout,
+ causal=causal,
+ residual=residual,
+ use_stride_conv=use_stride_conv,
+ conv_cfg=conv_cfg,
+ norm_cfg=norm_cfg))
+ dilation *= kernel_sizes[i]
+
+ if self.max_norm is not None:
+ # Apply weight norm clip to conv layers
+ weight_clip = WeightNormClipHook(self.max_norm)
+ for module in self.modules():
+ if isinstance(module, nn.modules.conv._ConvNd):
+ weight_clip.register(module)
+
+ self.dropout = nn.Dropout(dropout) if dropout > 0 else None
+
+ def forward(self, x):
+ """Forward function."""
+ x = self.expand_conv(x)
+
+ if self.dropout is not None:
+ x = self.dropout(x)
+
+ outs = []
+ for i in range(self.num_blocks):
+ x = self.tcn_blocks[i](x)
+ outs.append(x)
+
+ return tuple(outs)
+
+ def init_weights(self, pretrained=None):
+ """Initialize the weights."""
+ super().init_weights(pretrained)
+ if pretrained is None:
+ for m in self.modules():
+ if isinstance(m, nn.modules.conv._ConvNd):
+ kaiming_init(m, mode='fan_in', nonlinearity='relu')
+ elif isinstance(m, _BatchNorm):
+ constant_init(m, 1)
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/utils/__init__.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/utils/__init__.py
new file mode 100644
index 0000000..52a30ca
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/utils/__init__.py
@@ -0,0 +1,11 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from .channel_shuffle import channel_shuffle
+from .inverted_residual import InvertedResidual
+from .make_divisible import make_divisible
+from .se_layer import SELayer
+from .utils import load_checkpoint
+
+__all__ = [
+ 'channel_shuffle', 'make_divisible', 'InvertedResidual', 'SELayer',
+ 'load_checkpoint'
+]
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/utils/channel_shuffle.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/utils/channel_shuffle.py
new file mode 100644
index 0000000..27006a8
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/utils/channel_shuffle.py
@@ -0,0 +1,29 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import torch
+
+
+def channel_shuffle(x, groups):
+ """Channel Shuffle operation.
+
+ This function enables cross-group information flow for multiple groups
+ convolution layers.
+
+ Args:
+ x (Tensor): The input tensor.
+ groups (int): The number of groups to divide the input tensor
+ in the channel dimension.
+
+ Returns:
+ Tensor: The output tensor after channel shuffle operation.
+ """
+
+ batch_size, num_channels, height, width = x.size()
+ assert (num_channels % groups == 0), ('num_channels should be '
+ 'divisible by groups')
+ channels_per_group = num_channels // groups
+
+ x = x.view(batch_size, groups, channels_per_group, height, width)
+ x = torch.transpose(x, 1, 2).contiguous()
+ x = x.view(batch_size, -1, height, width)
+
+ return x
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/utils/inverted_residual.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/utils/inverted_residual.py
new file mode 100644
index 0000000..dff762c
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/utils/inverted_residual.py
@@ -0,0 +1,128 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import copy
+
+import torch.nn as nn
+import torch.utils.checkpoint as cp
+from mmcv.cnn import ConvModule
+
+from .se_layer import SELayer
+
+
+class InvertedResidual(nn.Module):
+ """Inverted Residual Block.
+
+ Args:
+ in_channels (int): The input channels of this Module.
+ out_channels (int): The output channels of this Module.
+ mid_channels (int): The input channels of the depthwise convolution.
+ kernel_size (int): The kernel size of the depthwise convolution.
+ Default: 3.
+ groups (None or int): The group number of the depthwise convolution.
+ Default: None, which means group number = mid_channels.
+ stride (int): The stride of the depthwise convolution. Default: 1.
+ se_cfg (dict): Config dict for se layer. Default: None, which means no
+ se layer.
+ with_expand_conv (bool): Use expand conv or not. If set False,
+ mid_channels must be the same with in_channels.
+ Default: True.
+ conv_cfg (dict): Config dict for convolution layer. Default: None,
+ which means using conv2d.
+ norm_cfg (dict): Config dict for normalization layer.
+ Default: dict(type='BN').
+ act_cfg (dict): Config dict for activation layer.
+ Default: dict(type='ReLU').
+ with_cp (bool): Use checkpoint or not. Using checkpoint will save some
+ memory while slowing down the training speed. Default: False.
+
+ Returns:
+ Tensor: The output tensor.
+ """
+
+ def __init__(self,
+ in_channels,
+ out_channels,
+ mid_channels,
+ kernel_size=3,
+ groups=None,
+ stride=1,
+ se_cfg=None,
+ with_expand_conv=True,
+ conv_cfg=None,
+ norm_cfg=dict(type='BN'),
+ act_cfg=dict(type='ReLU'),
+ with_cp=False):
+ # Protect mutable default arguments
+ norm_cfg = copy.deepcopy(norm_cfg)
+ act_cfg = copy.deepcopy(act_cfg)
+ super().__init__()
+ self.with_res_shortcut = (stride == 1 and in_channels == out_channels)
+ assert stride in [1, 2]
+ self.with_cp = with_cp
+ self.with_se = se_cfg is not None
+ self.with_expand_conv = with_expand_conv
+
+ if groups is None:
+ groups = mid_channels
+
+ if self.with_se:
+ assert isinstance(se_cfg, dict)
+ if not self.with_expand_conv:
+ assert mid_channels == in_channels
+
+ if self.with_expand_conv:
+ self.expand_conv = ConvModule(
+ in_channels=in_channels,
+ out_channels=mid_channels,
+ kernel_size=1,
+ stride=1,
+ padding=0,
+ conv_cfg=conv_cfg,
+ norm_cfg=norm_cfg,
+ act_cfg=act_cfg)
+ self.depthwise_conv = ConvModule(
+ in_channels=mid_channels,
+ out_channels=mid_channels,
+ kernel_size=kernel_size,
+ stride=stride,
+ padding=kernel_size // 2,
+ groups=groups,
+ conv_cfg=conv_cfg,
+ norm_cfg=norm_cfg,
+ act_cfg=act_cfg)
+ if self.with_se:
+ self.se = SELayer(**se_cfg)
+ self.linear_conv = ConvModule(
+ in_channels=mid_channels,
+ out_channels=out_channels,
+ kernel_size=1,
+ stride=1,
+ padding=0,
+ conv_cfg=conv_cfg,
+ norm_cfg=norm_cfg,
+ act_cfg=None)
+
+ def forward(self, x):
+
+ def _inner_forward(x):
+ out = x
+
+ if self.with_expand_conv:
+ out = self.expand_conv(out)
+
+ out = self.depthwise_conv(out)
+
+ if self.with_se:
+ out = self.se(out)
+
+ out = self.linear_conv(out)
+
+ if self.with_res_shortcut:
+ return x + out
+ return out
+
+ if self.with_cp and x.requires_grad:
+ out = cp.checkpoint(_inner_forward, x)
+ else:
+ out = _inner_forward(x)
+
+ return out
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/utils/make_divisible.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/utils/make_divisible.py
new file mode 100644
index 0000000..b7666be
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/utils/make_divisible.py
@@ -0,0 +1,25 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+def make_divisible(value, divisor, min_value=None, min_ratio=0.9):
+ """Make divisible function.
+
+ This function rounds the channel number down to the nearest value that can
+ be divisible by the divisor.
+
+ Args:
+ value (int): The original channel number.
+ divisor (int): The divisor to fully divide the channel number.
+ min_value (int, optional): The minimum value of the output channel.
+ Default: None, means that the minimum value equal to the divisor.
+ min_ratio (float, optional): The minimum ratio of the rounded channel
+ number to the original channel number. Default: 0.9.
+ Returns:
+ int: The modified output channel number
+ """
+
+ if min_value is None:
+ min_value = divisor
+ new_value = max(min_value, int(value + divisor / 2) // divisor * divisor)
+ # Make sure that round down does not go down by more than (1-min_ratio).
+ if new_value < min_ratio * value:
+ new_value += divisor
+ return new_value
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/utils/se_layer.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/utils/se_layer.py
new file mode 100644
index 0000000..07f7080
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/utils/se_layer.py
@@ -0,0 +1,54 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import mmcv
+import torch.nn as nn
+from mmcv.cnn import ConvModule
+
+
+class SELayer(nn.Module):
+ """Squeeze-and-Excitation Module.
+
+ Args:
+ channels (int): The input (and output) channels of the SE layer.
+ ratio (int): Squeeze ratio in SELayer, the intermediate channel will be
+ ``int(channels/ratio)``. Default: 16.
+ conv_cfg (None or dict): Config dict for convolution layer.
+ Default: None, which means using conv2d.
+ act_cfg (dict or Sequence[dict]): Config dict for activation layer.
+ If act_cfg is a dict, two activation layers will be configurated
+ by this dict. If act_cfg is a sequence of dicts, the first
+ activation layer will be configurated by the first dict and the
+ second activation layer will be configurated by the second dict.
+ Default: (dict(type='ReLU'), dict(type='Sigmoid'))
+ """
+
+ def __init__(self,
+ channels,
+ ratio=16,
+ conv_cfg=None,
+ act_cfg=(dict(type='ReLU'), dict(type='Sigmoid'))):
+ super().__init__()
+ if isinstance(act_cfg, dict):
+ act_cfg = (act_cfg, act_cfg)
+ assert len(act_cfg) == 2
+ assert mmcv.is_tuple_of(act_cfg, dict)
+ self.global_avgpool = nn.AdaptiveAvgPool2d(1)
+ self.conv1 = ConvModule(
+ in_channels=channels,
+ out_channels=int(channels / ratio),
+ kernel_size=1,
+ stride=1,
+ conv_cfg=conv_cfg,
+ act_cfg=act_cfg[0])
+ self.conv2 = ConvModule(
+ in_channels=int(channels / ratio),
+ out_channels=channels,
+ kernel_size=1,
+ stride=1,
+ conv_cfg=conv_cfg,
+ act_cfg=act_cfg[1])
+
+ def forward(self, x):
+ out = self.global_avgpool(x)
+ out = self.conv1(out)
+ out = self.conv2(out)
+ return x * out
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/utils/utils.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/utils/utils.py
new file mode 100644
index 0000000..a9ac948
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/utils/utils.py
@@ -0,0 +1,87 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from collections import OrderedDict
+
+from mmcv.runner.checkpoint import _load_checkpoint, load_state_dict
+
+
+def load_checkpoint(model,
+ filename,
+ map_location='cpu',
+ strict=False,
+ logger=None):
+ """Load checkpoint from a file or URI.
+
+ Args:
+ model (Module): Module to load checkpoint.
+ filename (str): Accept local filepath, URL, ``torchvision://xxx``,
+ ``open-mmlab://xxx``.
+ map_location (str): Same as :func:`torch.load`.
+ strict (bool): Whether to allow different params for the model and
+ checkpoint.
+ logger (:mod:`logging.Logger` or None): The logger for error message.
+
+ Returns:
+ dict or OrderedDict: The loaded checkpoint.
+ """
+ checkpoint = _load_checkpoint(filename, map_location)
+ # OrderedDict is a subclass of dict
+ if not isinstance(checkpoint, dict):
+ raise RuntimeError(
+ f'No state_dict found in checkpoint file {filename}')
+ # get state_dict from checkpoint
+ if 'state_dict' in checkpoint:
+ state_dict_tmp = checkpoint['state_dict']
+ else:
+ state_dict_tmp = checkpoint
+
+ state_dict = OrderedDict()
+ # strip prefix of state_dict
+ for k, v in state_dict_tmp.items():
+ if k.startswith('module.backbone.'):
+ state_dict[k[16:]] = v
+ elif k.startswith('module.'):
+ state_dict[k[7:]] = v
+ elif k.startswith('backbone.'):
+ state_dict[k[9:]] = v
+ else:
+ state_dict[k] = v
+ # load state_dict
+ load_state_dict(model, state_dict, strict, logger)
+ return checkpoint
+
+
+def get_state_dict(filename, map_location='cpu'):
+ """Get state_dict from a file or URI.
+
+ Args:
+ filename (str): Accept local filepath, URL, ``torchvision://xxx``,
+ ``open-mmlab://xxx``.
+ map_location (str): Same as :func:`torch.load`.
+
+ Returns:
+ OrderedDict: The state_dict.
+ """
+ checkpoint = _load_checkpoint(filename, map_location)
+ # OrderedDict is a subclass of dict
+ if not isinstance(checkpoint, dict):
+ raise RuntimeError(
+ f'No state_dict found in checkpoint file {filename}')
+ # get state_dict from checkpoint
+ if 'state_dict' in checkpoint:
+ state_dict_tmp = checkpoint['state_dict']
+ else:
+ state_dict_tmp = checkpoint
+
+ state_dict = OrderedDict()
+ # strip prefix of state_dict
+ for k, v in state_dict_tmp.items():
+ if k.startswith('module.backbone.'):
+ state_dict[k[16:]] = v
+ elif k.startswith('module.'):
+ state_dict[k[7:]] = v
+ elif k.startswith('backbone.'):
+ state_dict[k[9:]] = v
+ else:
+ state_dict[k] = v
+
+ return state_dict
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/v2v_net.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/v2v_net.py
new file mode 100644
index 0000000..99462af
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/v2v_net.py
@@ -0,0 +1,257 @@
+# ------------------------------------------------------------------------------
+# Copyright and License Information
+# Adapted from
+# https://github.com/microsoft/voxelpose-pytorch/blob/main/lib/models/v2v_net.py
+# Original Licence: MIT License
+# ------------------------------------------------------------------------------
+
+import torch.nn as nn
+import torch.nn.functional as F
+from mmcv.cnn import ConvModule
+
+from ..builder import BACKBONES
+from .base_backbone import BaseBackbone
+
+
+class Basic3DBlock(nn.Module):
+ """A basic 3D convolutional block.
+
+ Args:
+ in_channels (int): Input channels of this block.
+ out_channels (int): Output channels of this block.
+ kernel_size (int): Kernel size of the convolution operation
+ conv_cfg (dict): Dictionary to construct and config conv layer.
+ Default: dict(type='Conv3d')
+ norm_cfg (dict): Dictionary to construct and config norm layer.
+ Default: dict(type='BN3d')
+ """
+
+ def __init__(self,
+ in_channels,
+ out_channels,
+ kernel_size,
+ conv_cfg=dict(type='Conv3d'),
+ norm_cfg=dict(type='BN3d')):
+ super(Basic3DBlock, self).__init__()
+ self.block = ConvModule(
+ in_channels,
+ out_channels,
+ kernel_size,
+ stride=1,
+ padding=((kernel_size - 1) // 2),
+ conv_cfg=conv_cfg,
+ norm_cfg=norm_cfg,
+ bias=True)
+
+ def forward(self, x):
+ """Forward function."""
+ return self.block(x)
+
+
+class Res3DBlock(nn.Module):
+ """A residual 3D convolutional block.
+
+ Args:
+ in_channels (int): Input channels of this block.
+ out_channels (int): Output channels of this block.
+ kernel_size (int): Kernel size of the convolution operation
+ Default: 3
+ conv_cfg (dict): Dictionary to construct and config conv layer.
+ Default: dict(type='Conv3d')
+ norm_cfg (dict): Dictionary to construct and config norm layer.
+ Default: dict(type='BN3d')
+ """
+
+ def __init__(self,
+ in_channels,
+ out_channels,
+ kernel_size=3,
+ conv_cfg=dict(type='Conv3d'),
+ norm_cfg=dict(type='BN3d')):
+ super(Res3DBlock, self).__init__()
+ self.res_branch = nn.Sequential(
+ ConvModule(
+ in_channels,
+ out_channels,
+ kernel_size,
+ stride=1,
+ padding=((kernel_size - 1) // 2),
+ conv_cfg=conv_cfg,
+ norm_cfg=norm_cfg,
+ bias=True),
+ ConvModule(
+ out_channels,
+ out_channels,
+ kernel_size,
+ stride=1,
+ padding=((kernel_size - 1) // 2),
+ conv_cfg=conv_cfg,
+ norm_cfg=norm_cfg,
+ act_cfg=None,
+ bias=True))
+
+ if in_channels == out_channels:
+ self.skip_con = nn.Sequential()
+ else:
+ self.skip_con = ConvModule(
+ in_channels,
+ out_channels,
+ 1,
+ stride=1,
+ padding=0,
+ conv_cfg=conv_cfg,
+ norm_cfg=norm_cfg,
+ act_cfg=None,
+ bias=True)
+
+ def forward(self, x):
+ """Forward function."""
+ res = self.res_branch(x)
+ skip = self.skip_con(x)
+ return F.relu(res + skip, True)
+
+
+class Pool3DBlock(nn.Module):
+ """A 3D max-pool block.
+
+ Args:
+ pool_size (int): Pool size of the 3D max-pool layer
+ """
+
+ def __init__(self, pool_size):
+ super(Pool3DBlock, self).__init__()
+ self.pool_size = pool_size
+
+ def forward(self, x):
+ """Forward function."""
+ return F.max_pool3d(
+ x, kernel_size=self.pool_size, stride=self.pool_size)
+
+
+class Upsample3DBlock(nn.Module):
+ """A 3D upsample block.
+
+ Args:
+ in_channels (int): Input channels of this block.
+ out_channels (int): Output channels of this block.
+ kernel_size (int): Kernel size of the transposed convolution operation.
+ Default: 2
+ stride (int): Kernel size of the transposed convolution operation.
+ Default: 2
+ """
+
+ def __init__(self, in_channels, out_channels, kernel_size=2, stride=2):
+ super(Upsample3DBlock, self).__init__()
+ assert kernel_size == 2
+ assert stride == 2
+ self.block = nn.Sequential(
+ nn.ConvTranspose3d(
+ in_channels,
+ out_channels,
+ kernel_size=kernel_size,
+ stride=stride,
+ padding=0,
+ output_padding=0), nn.BatchNorm3d(out_channels), nn.ReLU(True))
+
+ def forward(self, x):
+ """Forward function."""
+ return self.block(x)
+
+
+class EncoderDecorder(nn.Module):
+ """An encoder-decoder block.
+
+ Args:
+ in_channels (int): Input channels of this block
+ """
+
+ def __init__(self, in_channels=32):
+ super(EncoderDecorder, self).__init__()
+
+ self.encoder_pool1 = Pool3DBlock(2)
+ self.encoder_res1 = Res3DBlock(in_channels, in_channels * 2)
+ self.encoder_pool2 = Pool3DBlock(2)
+ self.encoder_res2 = Res3DBlock(in_channels * 2, in_channels * 4)
+
+ self.mid_res = Res3DBlock(in_channels * 4, in_channels * 4)
+
+ self.decoder_res2 = Res3DBlock(in_channels * 4, in_channels * 4)
+ self.decoder_upsample2 = Upsample3DBlock(in_channels * 4,
+ in_channels * 2, 2, 2)
+ self.decoder_res1 = Res3DBlock(in_channels * 2, in_channels * 2)
+ self.decoder_upsample1 = Upsample3DBlock(in_channels * 2, in_channels,
+ 2, 2)
+
+ self.skip_res1 = Res3DBlock(in_channels, in_channels)
+ self.skip_res2 = Res3DBlock(in_channels * 2, in_channels * 2)
+
+ def forward(self, x):
+ """Forward function."""
+ skip_x1 = self.skip_res1(x)
+ x = self.encoder_pool1(x)
+ x = self.encoder_res1(x)
+
+ skip_x2 = self.skip_res2(x)
+ x = self.encoder_pool2(x)
+ x = self.encoder_res2(x)
+
+ x = self.mid_res(x)
+
+ x = self.decoder_res2(x)
+ x = self.decoder_upsample2(x)
+ x = x + skip_x2
+
+ x = self.decoder_res1(x)
+ x = self.decoder_upsample1(x)
+ x = x + skip_x1
+
+ return x
+
+
+@BACKBONES.register_module()
+class V2VNet(BaseBackbone):
+ """V2VNet.
+
+ Please refer to the `paper `
+ for details.
+
+ Args:
+ input_channels (int):
+ Number of channels of the input feature volume.
+ output_channels (int):
+ Number of channels of the output volume.
+ mid_channels (int):
+ Input and output channels of the encoder-decoder block.
+ """
+
+ def __init__(self, input_channels, output_channels, mid_channels=32):
+ super(V2VNet, self).__init__()
+
+ self.front_layers = nn.Sequential(
+ Basic3DBlock(input_channels, mid_channels // 2, 7),
+ Res3DBlock(mid_channels // 2, mid_channels),
+ )
+
+ self.encoder_decoder = EncoderDecorder(in_channels=mid_channels)
+
+ self.output_layer = nn.Conv3d(
+ mid_channels, output_channels, kernel_size=1, stride=1, padding=0)
+
+ self._initialize_weights()
+
+ def forward(self, x):
+ """Forward function."""
+ x = self.front_layers(x)
+ x = self.encoder_decoder(x)
+ x = self.output_layer(x)
+
+ return x
+
+ def _initialize_weights(self):
+ for m in self.modules():
+ if isinstance(m, nn.Conv3d):
+ nn.init.normal_(m.weight, 0, 0.001)
+ nn.init.constant_(m.bias, 0)
+ elif isinstance(m, nn.ConvTranspose3d):
+ nn.init.normal_(m.weight, 0, 0.001)
+ nn.init.constant_(m.bias, 0)
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/vgg.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/vgg.py
new file mode 100644
index 0000000..f7d4670
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/vgg.py
@@ -0,0 +1,193 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import torch.nn as nn
+from mmcv.cnn import ConvModule, constant_init, kaiming_init, normal_init
+from mmcv.utils.parrots_wrapper import _BatchNorm
+
+from ..builder import BACKBONES
+from .base_backbone import BaseBackbone
+
+
+def make_vgg_layer(in_channels,
+ out_channels,
+ num_blocks,
+ conv_cfg=None,
+ norm_cfg=None,
+ act_cfg=dict(type='ReLU'),
+ dilation=1,
+ with_norm=False,
+ ceil_mode=False):
+ layers = []
+ for _ in range(num_blocks):
+ layer = ConvModule(
+ in_channels=in_channels,
+ out_channels=out_channels,
+ kernel_size=3,
+ dilation=dilation,
+ padding=dilation,
+ bias=True,
+ conv_cfg=conv_cfg,
+ norm_cfg=norm_cfg,
+ act_cfg=act_cfg)
+ layers.append(layer)
+ in_channels = out_channels
+ layers.append(nn.MaxPool2d(kernel_size=2, stride=2, ceil_mode=ceil_mode))
+
+ return layers
+
+
+@BACKBONES.register_module()
+class VGG(BaseBackbone):
+ """VGG backbone.
+
+ Args:
+ depth (int): Depth of vgg, from {11, 13, 16, 19}.
+ with_norm (bool): Use BatchNorm or not.
+ num_classes (int): number of classes for classification.
+ num_stages (int): VGG stages, normally 5.
+ dilations (Sequence[int]): Dilation of each stage.
+ out_indices (Sequence[int]): Output from which stages. If only one
+ stage is specified, a single tensor (feature map) is returned,
+ otherwise multiple stages are specified, a tuple of tensors will
+ be returned. When it is None, the default behavior depends on
+ whether num_classes is specified. If num_classes <= 0, the default
+ value is (4, ), outputting the last feature map before classifier.
+ If num_classes > 0, the default value is (5, ), outputting the
+ classification score. Default: None.
+ frozen_stages (int): Stages to be frozen (all param fixed). -1 means
+ not freezing any parameters.
+ norm_eval (bool): Whether to set norm layers to eval mode, namely,
+ freeze running stats (mean and var). Note: Effect on Batch Norm
+ and its variants only. Default: False.
+ ceil_mode (bool): Whether to use ceil_mode of MaxPool. Default: False.
+ with_last_pool (bool): Whether to keep the last pooling before
+ classifier. Default: True.
+ """
+
+ # Parameters to build layers. Each element specifies the number of conv in
+ # each stage. For example, VGG11 contains 11 layers with learnable
+ # parameters. 11 is computed as 11 = (1 + 1 + 2 + 2 + 2) + 3,
+ # where 3 indicates the last three fully-connected layers.
+ arch_settings = {
+ 11: (1, 1, 2, 2, 2),
+ 13: (2, 2, 2, 2, 2),
+ 16: (2, 2, 3, 3, 3),
+ 19: (2, 2, 4, 4, 4)
+ }
+
+ def __init__(self,
+ depth,
+ num_classes=-1,
+ num_stages=5,
+ dilations=(1, 1, 1, 1, 1),
+ out_indices=None,
+ frozen_stages=-1,
+ conv_cfg=None,
+ norm_cfg=None,
+ act_cfg=dict(type='ReLU'),
+ norm_eval=False,
+ ceil_mode=False,
+ with_last_pool=True):
+ super().__init__()
+ if depth not in self.arch_settings:
+ raise KeyError(f'invalid depth {depth} for vgg')
+ assert num_stages >= 1 and num_stages <= 5
+ stage_blocks = self.arch_settings[depth]
+ self.stage_blocks = stage_blocks[:num_stages]
+ assert len(dilations) == num_stages
+
+ self.num_classes = num_classes
+ self.frozen_stages = frozen_stages
+ self.norm_eval = norm_eval
+ with_norm = norm_cfg is not None
+
+ if out_indices is None:
+ out_indices = (5, ) if num_classes > 0 else (4, )
+ assert max(out_indices) <= num_stages
+ self.out_indices = out_indices
+
+ self.in_channels = 3
+ start_idx = 0
+ vgg_layers = []
+ self.range_sub_modules = []
+ for i, num_blocks in enumerate(self.stage_blocks):
+ num_modules = num_blocks + 1
+ end_idx = start_idx + num_modules
+ dilation = dilations[i]
+ out_channels = 64 * 2**i if i < 4 else 512
+ vgg_layer = make_vgg_layer(
+ self.in_channels,
+ out_channels,
+ num_blocks,
+ conv_cfg=conv_cfg,
+ norm_cfg=norm_cfg,
+ act_cfg=act_cfg,
+ dilation=dilation,
+ with_norm=with_norm,
+ ceil_mode=ceil_mode)
+ vgg_layers.extend(vgg_layer)
+ self.in_channels = out_channels
+ self.range_sub_modules.append([start_idx, end_idx])
+ start_idx = end_idx
+ if not with_last_pool:
+ vgg_layers.pop(-1)
+ self.range_sub_modules[-1][1] -= 1
+ self.module_name = 'features'
+ self.add_module(self.module_name, nn.Sequential(*vgg_layers))
+
+ if self.num_classes > 0:
+ self.classifier = nn.Sequential(
+ nn.Linear(512 * 7 * 7, 4096),
+ nn.ReLU(True),
+ nn.Dropout(),
+ nn.Linear(4096, 4096),
+ nn.ReLU(True),
+ nn.Dropout(),
+ nn.Linear(4096, num_classes),
+ )
+
+ def init_weights(self, pretrained=None):
+ super().init_weights(pretrained)
+ if pretrained is None:
+ for m in self.modules():
+ if isinstance(m, nn.Conv2d):
+ kaiming_init(m)
+ elif isinstance(m, _BatchNorm):
+ constant_init(m, 1)
+ elif isinstance(m, nn.Linear):
+ normal_init(m, std=0.01)
+
+ def forward(self, x):
+ outs = []
+ vgg_layers = getattr(self, self.module_name)
+ for i in range(len(self.stage_blocks)):
+ for j in range(*self.range_sub_modules[i]):
+ vgg_layer = vgg_layers[j]
+ x = vgg_layer(x)
+ if i in self.out_indices:
+ outs.append(x)
+ if self.num_classes > 0:
+ x = x.view(x.size(0), -1)
+ x = self.classifier(x)
+ outs.append(x)
+ if len(outs) == 1:
+ return outs[0]
+ else:
+ return tuple(outs)
+
+ def _freeze_stages(self):
+ vgg_layers = getattr(self, self.module_name)
+ for i in range(self.frozen_stages):
+ for j in range(*self.range_sub_modules[i]):
+ m = vgg_layers[j]
+ m.eval()
+ for param in m.parameters():
+ param.requires_grad = False
+
+ def train(self, mode=True):
+ super().train(mode)
+ self._freeze_stages()
+ if mode and self.norm_eval:
+ for m in self.modules():
+ # trick: eval have effect on BatchNorm only
+ if isinstance(m, _BatchNorm):
+ m.eval()
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/vipnas_mbv3.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/vipnas_mbv3.py
new file mode 100644
index 0000000..ed990e3
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/vipnas_mbv3.py
@@ -0,0 +1,179 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import copy
+import logging
+
+import torch.nn as nn
+from mmcv.cnn import ConvModule
+from torch.nn.modules.batchnorm import _BatchNorm
+
+from ..builder import BACKBONES
+from .base_backbone import BaseBackbone
+from .utils import InvertedResidual, load_checkpoint
+
+
+@BACKBONES.register_module()
+class ViPNAS_MobileNetV3(BaseBackbone):
+ """ViPNAS_MobileNetV3 backbone.
+
+ "ViPNAS: Efficient Video Pose Estimation via Neural Architecture Search"
+ More details can be found in the `paper
+ `__ .
+
+ Args:
+ wid (list(int)): Searched width config for each stage.
+ expan (list(int)): Searched expansion ratio config for each stage.
+ dep (list(int)): Searched depth config for each stage.
+ ks (list(int)): Searched kernel size config for each stage.
+ group (list(int)): Searched group number config for each stage.
+ att (list(bool)): Searched attention config for each stage.
+ stride (list(int)): Stride config for each stage.
+ act (list(dict)): Activation config for each stage.
+ conv_cfg (dict): Config dict for convolution layer.
+ Default: None, which means using conv2d.
+ norm_cfg (dict): Config dict for normalization layer.
+ Default: dict(type='BN').
+ frozen_stages (int): Stages to be frozen (all param fixed).
+ Default: -1, which means not freezing any parameters.
+ norm_eval (bool): Whether to set norm layers to eval mode, namely,
+ freeze running stats (mean and var). Note: Effect on Batch Norm
+ and its variants only. Default: False.
+ with_cp (bool): Use checkpoint or not. Using checkpoint will save
+ some memory while slowing down the training speed.
+ Default: False.
+ """
+
+ def __init__(self,
+ wid=[16, 16, 24, 40, 80, 112, 160],
+ expan=[None, 1, 5, 4, 5, 5, 6],
+ dep=[None, 1, 4, 4, 4, 4, 4],
+ ks=[3, 3, 7, 7, 5, 7, 5],
+ group=[None, 8, 120, 20, 100, 280, 240],
+ att=[None, True, True, False, True, True, True],
+ stride=[2, 1, 2, 2, 2, 1, 2],
+ act=[
+ 'HSwish', 'ReLU', 'ReLU', 'ReLU', 'HSwish', 'HSwish',
+ 'HSwish'
+ ],
+ conv_cfg=None,
+ norm_cfg=dict(type='BN'),
+ frozen_stages=-1,
+ norm_eval=False,
+ with_cp=False):
+ # Protect mutable default arguments
+ norm_cfg = copy.deepcopy(norm_cfg)
+ super().__init__()
+ self.wid = wid
+ self.expan = expan
+ self.dep = dep
+ self.ks = ks
+ self.group = group
+ self.att = att
+ self.stride = stride
+ self.act = act
+ self.conv_cfg = conv_cfg
+ self.norm_cfg = norm_cfg
+ self.frozen_stages = frozen_stages
+ self.norm_eval = norm_eval
+ self.with_cp = with_cp
+
+ self.conv1 = ConvModule(
+ in_channels=3,
+ out_channels=self.wid[0],
+ kernel_size=self.ks[0],
+ stride=self.stride[0],
+ padding=self.ks[0] // 2,
+ conv_cfg=conv_cfg,
+ norm_cfg=norm_cfg,
+ act_cfg=dict(type=self.act[0]))
+
+ self.layers = self._make_layer()
+
+ def _make_layer(self):
+ layers = []
+ layer_index = 0
+ for i, dep in enumerate(self.dep[1:]):
+ mid_channels = self.wid[i + 1] * self.expan[i + 1]
+
+ if self.att[i + 1]:
+ se_cfg = dict(
+ channels=mid_channels,
+ ratio=4,
+ act_cfg=(dict(type='ReLU'), dict(type='HSigmoid')))
+ else:
+ se_cfg = None
+
+ if self.expan[i + 1] == 1:
+ with_expand_conv = False
+ else:
+ with_expand_conv = True
+
+ for j in range(dep):
+ if j == 0:
+ stride = self.stride[i + 1]
+ in_channels = self.wid[i]
+ else:
+ stride = 1
+ in_channels = self.wid[i + 1]
+
+ layer = InvertedResidual(
+ in_channels=in_channels,
+ out_channels=self.wid[i + 1],
+ mid_channels=mid_channels,
+ kernel_size=self.ks[i + 1],
+ groups=self.group[i + 1],
+ stride=stride,
+ se_cfg=se_cfg,
+ with_expand_conv=with_expand_conv,
+ conv_cfg=self.conv_cfg,
+ norm_cfg=self.norm_cfg,
+ act_cfg=dict(type=self.act[i + 1]),
+ with_cp=self.with_cp)
+ layer_index += 1
+ layer_name = f'layer{layer_index}'
+ self.add_module(layer_name, layer)
+ layers.append(layer_name)
+ return layers
+
+ def init_weights(self, pretrained=None):
+ if isinstance(pretrained, str):
+ logger = logging.getLogger()
+ load_checkpoint(self, pretrained, strict=False, logger=logger)
+ elif pretrained is None:
+ for m in self.modules():
+ if isinstance(m, nn.Conv2d):
+ nn.init.normal_(m.weight, std=0.001)
+ for name, _ in m.named_parameters():
+ if name in ['bias']:
+ nn.init.constant_(m.bias, 0)
+ elif isinstance(m, nn.BatchNorm2d):
+ nn.init.constant_(m.weight, 1)
+ nn.init.constant_(m.bias, 0)
+ else:
+ raise TypeError('pretrained must be a str or None')
+
+ def forward(self, x):
+ x = self.conv1(x)
+
+ for i, layer_name in enumerate(self.layers):
+ layer = getattr(self, layer_name)
+ x = layer(x)
+
+ return x
+
+ def _freeze_stages(self):
+ if self.frozen_stages >= 0:
+ for param in self.conv1.parameters():
+ param.requires_grad = False
+ for i in range(1, self.frozen_stages + 1):
+ layer = getattr(self, f'layer{i}')
+ layer.eval()
+ for param in layer.parameters():
+ param.requires_grad = False
+
+ def train(self, mode=True):
+ super().train(mode)
+ self._freeze_stages()
+ if mode and self.norm_eval:
+ for m in self.modules():
+ if isinstance(m, _BatchNorm):
+ m.eval()
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/vipnas_resnet.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/vipnas_resnet.py
new file mode 100644
index 0000000..81b028e
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/vipnas_resnet.py
@@ -0,0 +1,589 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import copy
+
+import torch.nn as nn
+import torch.utils.checkpoint as cp
+from mmcv.cnn import ConvModule, build_conv_layer, build_norm_layer
+from mmcv.cnn.bricks import ContextBlock
+from mmcv.utils.parrots_wrapper import _BatchNorm
+
+from ..builder import BACKBONES
+from .base_backbone import BaseBackbone
+
+
+class ViPNAS_Bottleneck(nn.Module):
+ """Bottleneck block for ViPNAS_ResNet.
+
+ Args:
+ in_channels (int): Input channels of this block.
+ out_channels (int): Output channels of this block.
+ expansion (int): The ratio of ``out_channels/mid_channels`` where
+ ``mid_channels`` is the input/output channels of conv2. Default: 4.
+ stride (int): stride of the block. Default: 1
+ dilation (int): dilation of convolution. Default: 1
+ downsample (nn.Module): downsample operation on identity branch.
+ Default: None.
+ style (str): ``"pytorch"`` or ``"caffe"``. If set to "pytorch", the
+ stride-two layer is the 3x3 conv layer, otherwise the stride-two
+ layer is the first 1x1 conv layer. Default: "pytorch".
+ with_cp (bool): Use checkpoint or not. Using checkpoint will save some
+ memory while slowing down the training speed.
+ conv_cfg (dict): dictionary to construct and config conv layer.
+ Default: None
+ norm_cfg (dict): dictionary to construct and config norm layer.
+ Default: dict(type='BN')
+ kernel_size (int): kernel size of conv2 searched in ViPANS.
+ groups (int): group number of conv2 searched in ViPNAS.
+ attention (bool): whether to use attention module in the end of
+ the block.
+ """
+
+ def __init__(self,
+ in_channels,
+ out_channels,
+ expansion=4,
+ stride=1,
+ dilation=1,
+ downsample=None,
+ style='pytorch',
+ with_cp=False,
+ conv_cfg=None,
+ norm_cfg=dict(type='BN'),
+ kernel_size=3,
+ groups=1,
+ attention=False):
+ # Protect mutable default arguments
+ norm_cfg = copy.deepcopy(norm_cfg)
+ super().__init__()
+ assert style in ['pytorch', 'caffe']
+
+ self.in_channels = in_channels
+ self.out_channels = out_channels
+ self.expansion = expansion
+ assert out_channels % expansion == 0
+ self.mid_channels = out_channels // expansion
+ self.stride = stride
+ self.dilation = dilation
+ self.style = style
+ self.with_cp = with_cp
+ self.conv_cfg = conv_cfg
+ self.norm_cfg = norm_cfg
+
+ if self.style == 'pytorch':
+ self.conv1_stride = 1
+ self.conv2_stride = stride
+ else:
+ self.conv1_stride = stride
+ self.conv2_stride = 1
+
+ self.norm1_name, norm1 = build_norm_layer(
+ norm_cfg, self.mid_channels, postfix=1)
+ self.norm2_name, norm2 = build_norm_layer(
+ norm_cfg, self.mid_channels, postfix=2)
+ self.norm3_name, norm3 = build_norm_layer(
+ norm_cfg, out_channels, postfix=3)
+
+ self.conv1 = build_conv_layer(
+ conv_cfg,
+ in_channels,
+ self.mid_channels,
+ kernel_size=1,
+ stride=self.conv1_stride,
+ bias=False)
+ self.add_module(self.norm1_name, norm1)
+ self.conv2 = build_conv_layer(
+ conv_cfg,
+ self.mid_channels,
+ self.mid_channels,
+ kernel_size=kernel_size,
+ stride=self.conv2_stride,
+ padding=kernel_size // 2,
+ groups=groups,
+ dilation=dilation,
+ bias=False)
+
+ self.add_module(self.norm2_name, norm2)
+ self.conv3 = build_conv_layer(
+ conv_cfg,
+ self.mid_channels,
+ out_channels,
+ kernel_size=1,
+ bias=False)
+ self.add_module(self.norm3_name, norm3)
+
+ if attention:
+ self.attention = ContextBlock(out_channels,
+ max(1.0 / 16, 16.0 / out_channels))
+ else:
+ self.attention = None
+
+ self.relu = nn.ReLU(inplace=True)
+ self.downsample = downsample
+
+ @property
+ def norm1(self):
+ """nn.Module: the normalization layer named "norm1" """
+ return getattr(self, self.norm1_name)
+
+ @property
+ def norm2(self):
+ """nn.Module: the normalization layer named "norm2" """
+ return getattr(self, self.norm2_name)
+
+ @property
+ def norm3(self):
+ """nn.Module: the normalization layer named "norm3" """
+ return getattr(self, self.norm3_name)
+
+ def forward(self, x):
+ """Forward function."""
+
+ def _inner_forward(x):
+ identity = x
+
+ out = self.conv1(x)
+ out = self.norm1(out)
+ out = self.relu(out)
+
+ out = self.conv2(out)
+ out = self.norm2(out)
+ out = self.relu(out)
+
+ out = self.conv3(out)
+ out = self.norm3(out)
+
+ if self.attention is not None:
+ out = self.attention(out)
+
+ if self.downsample is not None:
+ identity = self.downsample(x)
+
+ out += identity
+
+ return out
+
+ if self.with_cp and x.requires_grad:
+ out = cp.checkpoint(_inner_forward, x)
+ else:
+ out = _inner_forward(x)
+
+ out = self.relu(out)
+
+ return out
+
+
+def get_expansion(block, expansion=None):
+ """Get the expansion of a residual block.
+
+ The block expansion will be obtained by the following order:
+
+ 1. If ``expansion`` is given, just return it.
+ 2. If ``block`` has the attribute ``expansion``, then return
+ ``block.expansion``.
+ 3. Return the default value according the the block type:
+ 4 for ``ViPNAS_Bottleneck``.
+
+ Args:
+ block (class): The block class.
+ expansion (int | None): The given expansion ratio.
+
+ Returns:
+ int: The expansion of the block.
+ """
+ if isinstance(expansion, int):
+ assert expansion > 0
+ elif expansion is None:
+ if hasattr(block, 'expansion'):
+ expansion = block.expansion
+ elif issubclass(block, ViPNAS_Bottleneck):
+ expansion = 1
+ else:
+ raise TypeError(f'expansion is not specified for {block.__name__}')
+ else:
+ raise TypeError('expansion must be an integer or None')
+
+ return expansion
+
+
+class ViPNAS_ResLayer(nn.Sequential):
+ """ViPNAS_ResLayer to build ResNet style backbone.
+
+ Args:
+ block (nn.Module): Residual block used to build ViPNAS ResLayer.
+ num_blocks (int): Number of blocks.
+ in_channels (int): Input channels of this block.
+ out_channels (int): Output channels of this block.
+ expansion (int, optional): The expansion for BasicBlock/Bottleneck.
+ If not specified, it will firstly be obtained via
+ ``block.expansion``. If the block has no attribute "expansion",
+ the following default values will be used: 1 for BasicBlock and
+ 4 for Bottleneck. Default: None.
+ stride (int): stride of the first block. Default: 1.
+ avg_down (bool): Use AvgPool instead of stride conv when
+ downsampling in the bottleneck. Default: False
+ conv_cfg (dict): dictionary to construct and config conv layer.
+ Default: None
+ norm_cfg (dict): dictionary to construct and config norm layer.
+ Default: dict(type='BN')
+ downsample_first (bool): Downsample at the first block or last block.
+ False for Hourglass, True for ResNet. Default: True
+ kernel_size (int): Kernel Size of the corresponding convolution layer
+ searched in the block.
+ groups (int): Group number of the corresponding convolution layer
+ searched in the block.
+ attention (bool): Whether to use attention module in the end of the
+ block.
+ """
+
+ def __init__(self,
+ block,
+ num_blocks,
+ in_channels,
+ out_channels,
+ expansion=None,
+ stride=1,
+ avg_down=False,
+ conv_cfg=None,
+ norm_cfg=dict(type='BN'),
+ downsample_first=True,
+ kernel_size=3,
+ groups=1,
+ attention=False,
+ **kwargs):
+ # Protect mutable default arguments
+ norm_cfg = copy.deepcopy(norm_cfg)
+ self.block = block
+ self.expansion = get_expansion(block, expansion)
+
+ downsample = None
+ if stride != 1 or in_channels != out_channels:
+ downsample = []
+ conv_stride = stride
+ if avg_down and stride != 1:
+ conv_stride = 1
+ downsample.append(
+ nn.AvgPool2d(
+ kernel_size=stride,
+ stride=stride,
+ ceil_mode=True,
+ count_include_pad=False))
+ downsample.extend([
+ build_conv_layer(
+ conv_cfg,
+ in_channels,
+ out_channels,
+ kernel_size=1,
+ stride=conv_stride,
+ bias=False),
+ build_norm_layer(norm_cfg, out_channels)[1]
+ ])
+ downsample = nn.Sequential(*downsample)
+
+ layers = []
+ if downsample_first:
+ layers.append(
+ block(
+ in_channels=in_channels,
+ out_channels=out_channels,
+ expansion=self.expansion,
+ stride=stride,
+ downsample=downsample,
+ conv_cfg=conv_cfg,
+ norm_cfg=norm_cfg,
+ kernel_size=kernel_size,
+ groups=groups,
+ attention=attention,
+ **kwargs))
+ in_channels = out_channels
+ for _ in range(1, num_blocks):
+ layers.append(
+ block(
+ in_channels=in_channels,
+ out_channels=out_channels,
+ expansion=self.expansion,
+ stride=1,
+ conv_cfg=conv_cfg,
+ norm_cfg=norm_cfg,
+ kernel_size=kernel_size,
+ groups=groups,
+ attention=attention,
+ **kwargs))
+ else: # downsample_first=False is for HourglassModule
+ for i in range(0, num_blocks - 1):
+ layers.append(
+ block(
+ in_channels=in_channels,
+ out_channels=in_channels,
+ expansion=self.expansion,
+ stride=1,
+ conv_cfg=conv_cfg,
+ norm_cfg=norm_cfg,
+ kernel_size=kernel_size,
+ groups=groups,
+ attention=attention,
+ **kwargs))
+ layers.append(
+ block(
+ in_channels=in_channels,
+ out_channels=out_channels,
+ expansion=self.expansion,
+ stride=stride,
+ downsample=downsample,
+ conv_cfg=conv_cfg,
+ norm_cfg=norm_cfg,
+ kernel_size=kernel_size,
+ groups=groups,
+ attention=attention,
+ **kwargs))
+
+ super().__init__(*layers)
+
+
+@BACKBONES.register_module()
+class ViPNAS_ResNet(BaseBackbone):
+ """ViPNAS_ResNet backbone.
+
+ "ViPNAS: Efficient Video Pose Estimation via Neural Architecture Search"
+ More details can be found in the `paper
+ `__ .
+
+ Args:
+ depth (int): Network depth, from {18, 34, 50, 101, 152}.
+ in_channels (int): Number of input image channels. Default: 3.
+ num_stages (int): Stages of the network. Default: 4.
+ strides (Sequence[int]): Strides of the first block of each stage.
+ Default: ``(1, 2, 2, 2)``.
+ dilations (Sequence[int]): Dilation of each stage.
+ Default: ``(1, 1, 1, 1)``.
+ out_indices (Sequence[int]): Output from which stages. If only one
+ stage is specified, a single tensor (feature map) is returned,
+ otherwise multiple stages are specified, a tuple of tensors will
+ be returned. Default: ``(3, )``.
+ style (str): `pytorch` or `caffe`. If set to "pytorch", the stride-two
+ layer is the 3x3 conv layer, otherwise the stride-two layer is
+ the first 1x1 conv layer.
+ deep_stem (bool): Replace 7x7 conv in input stem with 3 3x3 conv.
+ Default: False.
+ avg_down (bool): Use AvgPool instead of stride conv when
+ downsampling in the bottleneck. Default: False.
+ frozen_stages (int): Stages to be frozen (stop grad and set eval mode).
+ -1 means not freezing any parameters. Default: -1.
+ conv_cfg (dict | None): The config dict for conv layers. Default: None.
+ norm_cfg (dict): The config dict for norm layers.
+ norm_eval (bool): Whether to set norm layers to eval mode, namely,
+ freeze running stats (mean and var). Note: Effect on Batch Norm
+ and its variants only. Default: False.
+ with_cp (bool): Use checkpoint or not. Using checkpoint will save some
+ memory while slowing down the training speed. Default: False.
+ zero_init_residual (bool): Whether to use zero init for last norm layer
+ in resblocks to let them behave as identity. Default: True.
+ wid (list(int)): Searched width config for each stage.
+ expan (list(int)): Searched expansion ratio config for each stage.
+ dep (list(int)): Searched depth config for each stage.
+ ks (list(int)): Searched kernel size config for each stage.
+ group (list(int)): Searched group number config for each stage.
+ att (list(bool)): Searched attention config for each stage.
+ """
+
+ arch_settings = {
+ 50: ViPNAS_Bottleneck,
+ }
+
+ def __init__(self,
+ depth,
+ in_channels=3,
+ num_stages=4,
+ strides=(1, 2, 2, 2),
+ dilations=(1, 1, 1, 1),
+ out_indices=(3, ),
+ style='pytorch',
+ deep_stem=False,
+ avg_down=False,
+ frozen_stages=-1,
+ conv_cfg=None,
+ norm_cfg=dict(type='BN', requires_grad=True),
+ norm_eval=False,
+ with_cp=False,
+ zero_init_residual=True,
+ wid=[48, 80, 160, 304, 608],
+ expan=[None, 1, 1, 1, 1],
+ dep=[None, 4, 6, 7, 3],
+ ks=[7, 3, 5, 5, 5],
+ group=[None, 16, 16, 16, 16],
+ att=[None, True, False, True, True]):
+ # Protect mutable default arguments
+ norm_cfg = copy.deepcopy(norm_cfg)
+ super().__init__()
+ if depth not in self.arch_settings:
+ raise KeyError(f'invalid depth {depth} for resnet')
+ self.depth = depth
+ self.stem_channels = dep[0]
+ self.num_stages = num_stages
+ assert 1 <= num_stages <= 4
+ self.strides = strides
+ self.dilations = dilations
+ assert len(strides) == len(dilations) == num_stages
+ self.out_indices = out_indices
+ assert max(out_indices) < num_stages
+ self.style = style
+ self.deep_stem = deep_stem
+ self.avg_down = avg_down
+ self.frozen_stages = frozen_stages
+ self.conv_cfg = conv_cfg
+ self.norm_cfg = norm_cfg
+ self.with_cp = with_cp
+ self.norm_eval = norm_eval
+ self.zero_init_residual = zero_init_residual
+ self.block = self.arch_settings[depth]
+ self.stage_blocks = dep[1:1 + num_stages]
+
+ self._make_stem_layer(in_channels, wid[0], ks[0])
+
+ self.res_layers = []
+ _in_channels = wid[0]
+ for i, num_blocks in enumerate(self.stage_blocks):
+ expansion = get_expansion(self.block, expan[i + 1])
+ _out_channels = wid[i + 1] * expansion
+ stride = strides[i]
+ dilation = dilations[i]
+ res_layer = self.make_res_layer(
+ block=self.block,
+ num_blocks=num_blocks,
+ in_channels=_in_channels,
+ out_channels=_out_channels,
+ expansion=expansion,
+ stride=stride,
+ dilation=dilation,
+ style=self.style,
+ avg_down=self.avg_down,
+ with_cp=with_cp,
+ conv_cfg=conv_cfg,
+ norm_cfg=norm_cfg,
+ kernel_size=ks[i + 1],
+ groups=group[i + 1],
+ attention=att[i + 1])
+ _in_channels = _out_channels
+ layer_name = f'layer{i + 1}'
+ self.add_module(layer_name, res_layer)
+ self.res_layers.append(layer_name)
+
+ self._freeze_stages()
+
+ self.feat_dim = res_layer[-1].out_channels
+
+ def make_res_layer(self, **kwargs):
+ """Make a ViPNAS ResLayer."""
+ return ViPNAS_ResLayer(**kwargs)
+
+ @property
+ def norm1(self):
+ """nn.Module: the normalization layer named "norm1" """
+ return getattr(self, self.norm1_name)
+
+ def _make_stem_layer(self, in_channels, stem_channels, kernel_size):
+ """Make stem layer."""
+ if self.deep_stem:
+ self.stem = nn.Sequential(
+ ConvModule(
+ in_channels,
+ stem_channels // 2,
+ kernel_size=3,
+ stride=2,
+ padding=1,
+ conv_cfg=self.conv_cfg,
+ norm_cfg=self.norm_cfg,
+ inplace=True),
+ ConvModule(
+ stem_channels // 2,
+ stem_channels // 2,
+ kernel_size=3,
+ stride=1,
+ padding=1,
+ conv_cfg=self.conv_cfg,
+ norm_cfg=self.norm_cfg,
+ inplace=True),
+ ConvModule(
+ stem_channels // 2,
+ stem_channels,
+ kernel_size=3,
+ stride=1,
+ padding=1,
+ conv_cfg=self.conv_cfg,
+ norm_cfg=self.norm_cfg,
+ inplace=True))
+ else:
+ self.conv1 = build_conv_layer(
+ self.conv_cfg,
+ in_channels,
+ stem_channels,
+ kernel_size=kernel_size,
+ stride=2,
+ padding=kernel_size // 2,
+ bias=False)
+ self.norm1_name, norm1 = build_norm_layer(
+ self.norm_cfg, stem_channels, postfix=1)
+ self.add_module(self.norm1_name, norm1)
+ self.relu = nn.ReLU(inplace=True)
+ self.maxpool = nn.MaxPool2d(kernel_size=3, stride=2, padding=1)
+
+ def _freeze_stages(self):
+ """Freeze parameters."""
+ if self.frozen_stages >= 0:
+ if self.deep_stem:
+ self.stem.eval()
+ for param in self.stem.parameters():
+ param.requires_grad = False
+ else:
+ self.norm1.eval()
+ for m in [self.conv1, self.norm1]:
+ for param in m.parameters():
+ param.requires_grad = False
+
+ for i in range(1, self.frozen_stages + 1):
+ m = getattr(self, f'layer{i}')
+ m.eval()
+ for param in m.parameters():
+ param.requires_grad = False
+
+ def init_weights(self, pretrained=None):
+ """Initialize model weights."""
+ super().init_weights(pretrained)
+ if pretrained is None:
+ for m in self.modules():
+ if isinstance(m, nn.Conv2d):
+ nn.init.normal_(m.weight, std=0.001)
+ for name, _ in m.named_parameters():
+ if name in ['bias']:
+ nn.init.constant_(m.bias, 0)
+ elif isinstance(m, nn.BatchNorm2d):
+ nn.init.constant_(m.weight, 1)
+ nn.init.constant_(m.bias, 0)
+
+ def forward(self, x):
+ """Forward function."""
+ if self.deep_stem:
+ x = self.stem(x)
+ else:
+ x = self.conv1(x)
+ x = self.norm1(x)
+ x = self.relu(x)
+ x = self.maxpool(x)
+ outs = []
+ for i, layer_name in enumerate(self.res_layers):
+ res_layer = getattr(self, layer_name)
+ x = res_layer(x)
+ if i in self.out_indices:
+ outs.append(x)
+ if len(outs) == 1:
+ return outs[0]
+ return tuple(outs)
+
+ def train(self, mode=True):
+ """Convert the model into training mode."""
+ super().train(mode)
+ self._freeze_stages()
+ if mode and self.norm_eval:
+ for m in self.modules():
+ # trick: eval have effect on BatchNorm only
+ if isinstance(m, _BatchNorm):
+ m.eval()
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/vit.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/vit.py
new file mode 100644
index 0000000..2719d1a
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/vit.py
@@ -0,0 +1,341 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import math
+
+import torch
+from functools import partial
+import torch.nn as nn
+import torch.nn.functional as F
+import torch.utils.checkpoint as checkpoint
+
+from timm.models.layers import drop_path, to_2tuple, trunc_normal_
+
+from ..builder import BACKBONES
+from .base_backbone import BaseBackbone
+
+def get_abs_pos(abs_pos, h, w, ori_h, ori_w, has_cls_token=True):
+ """
+ Calculate absolute positional embeddings. If needed, resize embeddings and remove cls_token
+ dimension for the original embeddings.
+ Args:
+ abs_pos (Tensor): absolute positional embeddings with (1, num_position, C).
+ has_cls_token (bool): If true, has 1 embedding in abs_pos for cls token.
+ hw (Tuple): size of input image tokens.
+
+ Returns:
+ Absolute positional embeddings after processing with shape (1, H, W, C)
+ """
+ cls_token = None
+ B, L, C = abs_pos.shape
+ if has_cls_token:
+ cls_token = abs_pos[:, 0:1]
+ abs_pos = abs_pos[:, 1:]
+
+ if ori_h != h or ori_w != w:
+ new_abs_pos = F.interpolate(
+ abs_pos.reshape(1, ori_h, ori_w, -1).permute(0, 3, 1, 2),
+ size=(h, w),
+ mode="bicubic",
+ align_corners=False,
+ ).permute(0, 2, 3, 1).reshape(B, -1, C)
+
+ else:
+ new_abs_pos = abs_pos
+
+ if cls_token is not None:
+ new_abs_pos = torch.cat([cls_token, new_abs_pos], dim=1)
+ return new_abs_pos
+
+class DropPath(nn.Module):
+ """Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks).
+ """
+ def __init__(self, drop_prob=None):
+ super(DropPath, self).__init__()
+ self.drop_prob = drop_prob
+
+ def forward(self, x):
+ return drop_path(x, self.drop_prob, self.training)
+
+ def extra_repr(self):
+ return 'p={}'.format(self.drop_prob)
+
+class Mlp(nn.Module):
+ def __init__(self, in_features, hidden_features=None, out_features=None, act_layer=nn.GELU, drop=0.):
+ super().__init__()
+ out_features = out_features or in_features
+ hidden_features = hidden_features or in_features
+ self.fc1 = nn.Linear(in_features, hidden_features)
+ self.act = act_layer()
+ self.fc2 = nn.Linear(hidden_features, out_features)
+ self.drop = nn.Dropout(drop)
+
+ def forward(self, x):
+ x = self.fc1(x)
+ x = self.act(x)
+ x = self.fc2(x)
+ x = self.drop(x)
+ return x
+
+class Attention(nn.Module):
+ def __init__(
+ self, dim, num_heads=8, qkv_bias=False, qk_scale=None, attn_drop=0.,
+ proj_drop=0., attn_head_dim=None,):
+ super().__init__()
+ self.num_heads = num_heads
+ head_dim = dim // num_heads
+ self.dim = dim
+
+ if attn_head_dim is not None:
+ head_dim = attn_head_dim
+ all_head_dim = head_dim * self.num_heads
+
+ self.scale = qk_scale or head_dim ** -0.5
+
+ self.qkv = nn.Linear(dim, all_head_dim * 3, bias=qkv_bias)
+
+ self.attn_drop = nn.Dropout(attn_drop)
+ self.proj = nn.Linear(all_head_dim, dim)
+ self.proj_drop = nn.Dropout(proj_drop)
+
+ def forward(self, x):
+ B, N, C = x.shape
+ qkv = self.qkv(x)
+ qkv = qkv.reshape(B, N, 3, self.num_heads, -1).permute(2, 0, 3, 1, 4)
+ q, k, v = qkv[0], qkv[1], qkv[2] # make torchscript happy (cannot use tensor as tuple)
+
+ q = q * self.scale
+ attn = (q @ k.transpose(-2, -1))
+
+ attn = attn.softmax(dim=-1)
+ attn = self.attn_drop(attn)
+
+ x = (attn @ v).transpose(1, 2).reshape(B, N, -1)
+ x = self.proj(x)
+ x = self.proj_drop(x)
+
+ return x
+
+class Block(nn.Module):
+
+ def __init__(self, dim, num_heads, mlp_ratio=4., qkv_bias=False, qk_scale=None,
+ drop=0., attn_drop=0., drop_path=0., act_layer=nn.GELU,
+ norm_layer=nn.LayerNorm, attn_head_dim=None
+ ):
+ super().__init__()
+
+ self.norm1 = norm_layer(dim)
+ self.attn = Attention(
+ dim, num_heads=num_heads, qkv_bias=qkv_bias, qk_scale=qk_scale,
+ attn_drop=attn_drop, proj_drop=drop, attn_head_dim=attn_head_dim
+ )
+
+ # NOTE: drop path for stochastic depth, we shall see if this is better than dropout here
+ self.drop_path = DropPath(drop_path) if drop_path > 0. else nn.Identity()
+ self.norm2 = norm_layer(dim)
+ mlp_hidden_dim = int(dim * mlp_ratio)
+ self.mlp = Mlp(in_features=dim, hidden_features=mlp_hidden_dim, act_layer=act_layer, drop=drop)
+
+ def forward(self, x):
+ x = x + self.drop_path(self.attn(self.norm1(x)))
+ x = x + self.drop_path(self.mlp(self.norm2(x)))
+ return x
+
+
+class PatchEmbed(nn.Module):
+ """ Image to Patch Embedding
+ """
+ def __init__(self, img_size=224, patch_size=16, in_chans=3, embed_dim=768, ratio=1):
+ super().__init__()
+ img_size = to_2tuple(img_size)
+ patch_size = to_2tuple(patch_size)
+ num_patches = (img_size[1] // patch_size[1]) * (img_size[0] // patch_size[0]) * (ratio ** 2)
+ self.patch_shape = (int(img_size[0] // patch_size[0] * ratio), int(img_size[1] // patch_size[1] * ratio))
+ self.origin_patch_shape = (int(img_size[0] // patch_size[0]), int(img_size[1] // patch_size[1]))
+ self.img_size = img_size
+ self.patch_size = patch_size
+ self.num_patches = num_patches
+
+ self.proj = nn.Conv2d(in_chans, embed_dim, kernel_size=patch_size, stride=(patch_size[0] // ratio), padding=4 + 2 * (ratio//2-1))
+
+ def forward(self, x, **kwargs):
+ B, C, H, W = x.shape
+ x = self.proj(x)
+ Hp, Wp = x.shape[2], x.shape[3]
+
+ x = x.flatten(2).transpose(1, 2)
+ return x, (Hp, Wp)
+
+
+class HybridEmbed(nn.Module):
+ """ CNN Feature Map Embedding
+ Extract feature map from CNN, flatten, project to embedding dim.
+ """
+ def __init__(self, backbone, img_size=224, feature_size=None, in_chans=3, embed_dim=768):
+ super().__init__()
+ assert isinstance(backbone, nn.Module)
+ img_size = to_2tuple(img_size)
+ self.img_size = img_size
+ self.backbone = backbone
+ if feature_size is None:
+ with torch.no_grad():
+ training = backbone.training
+ if training:
+ backbone.eval()
+ o = self.backbone(torch.zeros(1, in_chans, img_size[0], img_size[1]))[-1]
+ feature_size = o.shape[-2:]
+ feature_dim = o.shape[1]
+ backbone.train(training)
+ else:
+ feature_size = to_2tuple(feature_size)
+ feature_dim = self.backbone.feature_info.channels()[-1]
+ self.num_patches = feature_size[0] * feature_size[1]
+ self.proj = nn.Linear(feature_dim, embed_dim)
+
+ def forward(self, x):
+ x = self.backbone(x)[-1]
+ x = x.flatten(2).transpose(1, 2)
+ x = self.proj(x)
+ return x
+
+
+@BACKBONES.register_module()
+class ViT(BaseBackbone):
+
+ def __init__(self,
+ img_size=224, patch_size=16, in_chans=3, num_classes=80, embed_dim=768, depth=12,
+ num_heads=12, mlp_ratio=4., qkv_bias=False, qk_scale=None, drop_rate=0., attn_drop_rate=0.,
+ drop_path_rate=0., hybrid_backbone=None, norm_layer=None, use_checkpoint=False,
+ frozen_stages=-1, ratio=1, last_norm=True,
+ patch_padding='pad', freeze_attn=False, freeze_ffn=False,
+ ):
+ # Protect mutable default arguments
+ super(ViT, self).__init__()
+ norm_layer = norm_layer or partial(nn.LayerNorm, eps=1e-6)
+ self.num_classes = num_classes
+ self.num_features = self.embed_dim = embed_dim # num_features for consistency with other models
+ self.frozen_stages = frozen_stages
+ self.use_checkpoint = use_checkpoint
+ self.patch_padding = patch_padding
+ self.freeze_attn = freeze_attn
+ self.freeze_ffn = freeze_ffn
+ self.depth = depth
+
+ if hybrid_backbone is not None:
+ self.patch_embed = HybridEmbed(
+ hybrid_backbone, img_size=img_size, in_chans=in_chans, embed_dim=embed_dim)
+ else:
+ self.patch_embed = PatchEmbed(
+ img_size=img_size, patch_size=patch_size, in_chans=in_chans, embed_dim=embed_dim, ratio=ratio)
+ num_patches = self.patch_embed.num_patches
+
+ # since the pretraining model has class token
+ self.pos_embed = nn.Parameter(torch.zeros(1, num_patches + 1, embed_dim))
+
+ dpr = [x.item() for x in torch.linspace(0, drop_path_rate, depth)] # stochastic depth decay rule
+
+ self.blocks = nn.ModuleList([
+ Block(
+ dim=embed_dim, num_heads=num_heads, mlp_ratio=mlp_ratio, qkv_bias=qkv_bias, qk_scale=qk_scale,
+ drop=drop_rate, attn_drop=attn_drop_rate, drop_path=dpr[i], norm_layer=norm_layer,
+ )
+ for i in range(depth)])
+
+ self.last_norm = norm_layer(embed_dim) if last_norm else nn.Identity()
+
+ if self.pos_embed is not None:
+ trunc_normal_(self.pos_embed, std=.02)
+
+ self._freeze_stages()
+
+ def _freeze_stages(self):
+ """Freeze parameters."""
+ if self.frozen_stages >= 0:
+ self.patch_embed.eval()
+ for param in self.patch_embed.parameters():
+ param.requires_grad = False
+
+ for i in range(1, self.frozen_stages + 1):
+ m = self.blocks[i]
+ m.eval()
+ for param in m.parameters():
+ param.requires_grad = False
+
+ if self.freeze_attn:
+ for i in range(0, self.depth):
+ m = self.blocks[i]
+ m.attn.eval()
+ m.norm1.eval()
+ for param in m.attn.parameters():
+ param.requires_grad = False
+ for param in m.norm1.parameters():
+ param.requires_grad = False
+
+ if self.freeze_ffn:
+ self.pos_embed.requires_grad = False
+ self.patch_embed.eval()
+ for param in self.patch_embed.parameters():
+ param.requires_grad = False
+ for i in range(0, self.depth):
+ m = self.blocks[i]
+ m.mlp.eval()
+ m.norm2.eval()
+ for param in m.mlp.parameters():
+ param.requires_grad = False
+ for param in m.norm2.parameters():
+ param.requires_grad = False
+
+ def init_weights(self, pretrained=None):
+ """Initialize the weights in backbone.
+ Args:
+ pretrained (str, optional): Path to pre-trained weights.
+ Defaults to None.
+ """
+ super().init_weights(pretrained, patch_padding=self.patch_padding)
+
+ if pretrained is None:
+ def _init_weights(m):
+ if isinstance(m, nn.Linear):
+ trunc_normal_(m.weight, std=.02)
+ if isinstance(m, nn.Linear) and m.bias is not None:
+ nn.init.constant_(m.bias, 0)
+ elif isinstance(m, nn.LayerNorm):
+ nn.init.constant_(m.bias, 0)
+ nn.init.constant_(m.weight, 1.0)
+
+ self.apply(_init_weights)
+
+ def get_num_layers(self):
+ return len(self.blocks)
+
+ @torch.jit.ignore
+ def no_weight_decay(self):
+ return {'pos_embed', 'cls_token'}
+
+ def forward_features(self, x):
+ B, C, H, W = x.shape
+ x, (Hp, Wp) = self.patch_embed(x)
+
+ if self.pos_embed is not None:
+ # fit for multiple GPU training
+ # since the first element for pos embed (sin-cos manner) is zero, it will cause no difference
+ x = x + self.pos_embed[:, 1:] + self.pos_embed[:, :1]
+
+ for blk in self.blocks:
+ if self.use_checkpoint:
+ x = checkpoint.checkpoint(blk, x)
+ else:
+ x = blk(x)
+
+ x = self.last_norm(x)
+
+ xp = x.permute(0, 2, 1).reshape(B, -1, Hp, Wp).contiguous()
+
+ return xp
+
+ def forward(self, x):
+ x = self.forward_features(x)
+ return x
+
+ def train(self, mode=True):
+ """Convert the model into training mode."""
+ super().train(mode)
+ self._freeze_stages()
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/vit_moe.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/vit_moe.py
new file mode 100644
index 0000000..880a58f
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/backbones/vit_moe.py
@@ -0,0 +1,385 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import math
+
+import torch
+from functools import partial
+import torch.nn as nn
+import torch.nn.functional as F
+import torch.utils.checkpoint as checkpoint
+
+from timm.models.layers import drop_path, to_2tuple, trunc_normal_
+
+from ..builder import BACKBONES
+from .base_backbone import BaseBackbone
+
+def get_abs_pos(abs_pos, h, w, ori_h, ori_w, has_cls_token=True):
+ """
+ Calculate absolute positional embeddings. If needed, resize embeddings and remove cls_token
+ dimension for the original embeddings.
+ Args:
+ abs_pos (Tensor): absolute positional embeddings with (1, num_position, C).
+ has_cls_token (bool): If true, has 1 embedding in abs_pos for cls token.
+ hw (Tuple): size of input image tokens.
+
+ Returns:
+ Absolute positional embeddings after processing with shape (1, H, W, C)
+ """
+ cls_token = None
+ B, L, C = abs_pos.shape
+ if has_cls_token:
+ cls_token = abs_pos[:, 0:1]
+ abs_pos = abs_pos[:, 1:]
+
+ if ori_h != h or ori_w != w:
+ new_abs_pos = F.interpolate(
+ abs_pos.reshape(1, ori_h, ori_w, -1).permute(0, 3, 1, 2),
+ size=(h, w),
+ mode="bicubic",
+ align_corners=False,
+ ).permute(0, 2, 3, 1).reshape(B, -1, C)
+
+ else:
+ new_abs_pos = abs_pos
+
+ if cls_token is not None:
+ new_abs_pos = torch.cat([cls_token, new_abs_pos], dim=1)
+ return new_abs_pos
+
+class DropPath(nn.Module):
+ """Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks).
+ """
+ def __init__(self, drop_prob=None):
+ super(DropPath, self).__init__()
+ self.drop_prob = drop_prob
+
+ def forward(self, x):
+ return drop_path(x, self.drop_prob, self.training)
+
+ def extra_repr(self):
+ return 'p={}'.format(self.drop_prob)
+
+class Mlp(nn.Module):
+ def __init__(self, in_features, hidden_features=None, out_features=None, act_layer=nn.GELU, drop=0.):
+ super().__init__()
+ out_features = out_features or in_features
+ hidden_features = hidden_features or in_features
+ self.fc1 = nn.Linear(in_features, hidden_features)
+ self.act = act_layer()
+ self.fc2 = nn.Linear(hidden_features, out_features)
+ self.drop = nn.Dropout(drop)
+
+ def forward(self, x):
+ x = self.fc1(x)
+ x = self.act(x)
+ x = self.fc2(x)
+ x = self.drop(x)
+ return x
+
+class MoEMlp(nn.Module):
+ def __init__(self, num_expert=1, in_features=1024, hidden_features=None, out_features=None, act_layer=nn.GELU, drop=0., part_features=256):
+ super().__init__()
+ out_features = out_features or in_features
+ hidden_features = hidden_features or in_features
+ self.part_features = part_features
+ self.fc1 = nn.Linear(in_features, hidden_features)
+ self.act = act_layer()
+ self.fc2 = nn.Linear(hidden_features, out_features - part_features)
+ self.drop = nn.Dropout(drop)
+
+ self.num_expert = num_expert
+ experts = []
+
+ for i in range(num_expert):
+ experts.append(
+ nn.Linear(hidden_features, part_features)
+ )
+ self.experts = nn.ModuleList(experts)
+
+ def forward(self, x, indices):
+
+ expert_x = torch.zeros_like(x[:, :, -self.part_features:], device=x.device, dtype=x.dtype)
+
+ x = self.fc1(x)
+ x = self.act(x)
+ shared_x = self.fc2(x)
+ indices = indices.view(-1, 1, 1)
+
+ # to support ddp training
+ for i in range(self.num_expert):
+ selectedIndex = (indices == i)
+ current_x = self.experts[i](x) * selectedIndex
+ expert_x = expert_x + current_x
+
+ x = torch.cat([shared_x, expert_x], dim=-1)
+
+ return x
+
+class Attention(nn.Module):
+ def __init__(
+ self, dim, num_heads=8, qkv_bias=False, qk_scale=None, attn_drop=0.,
+ proj_drop=0., attn_head_dim=None,):
+ super().__init__()
+ self.num_heads = num_heads
+ head_dim = dim // num_heads
+ self.dim = dim
+
+ if attn_head_dim is not None:
+ head_dim = attn_head_dim
+ all_head_dim = head_dim * self.num_heads
+
+ self.scale = qk_scale or head_dim ** -0.5
+
+ self.qkv = nn.Linear(dim, all_head_dim * 3, bias=qkv_bias)
+
+ self.attn_drop = nn.Dropout(attn_drop)
+ self.proj = nn.Linear(all_head_dim, dim)
+ self.proj_drop = nn.Dropout(proj_drop)
+
+ def forward(self, x):
+ B, N, C = x.shape
+ qkv = self.qkv(x)
+ qkv = qkv.reshape(B, N, 3, self.num_heads, -1).permute(2, 0, 3, 1, 4)
+ q, k, v = qkv[0], qkv[1], qkv[2] # make torchscript happy (cannot use tensor as tuple)
+
+ q = q * self.scale
+ attn = (q @ k.transpose(-2, -1))
+
+ attn = attn.softmax(dim=-1)
+ attn = self.attn_drop(attn)
+
+ x = (attn @ v).transpose(1, 2).reshape(B, N, -1)
+ x = self.proj(x)
+ x = self.proj_drop(x)
+
+ return x
+
+class Block(nn.Module):
+
+ def __init__(self, dim, num_heads, mlp_ratio=4., qkv_bias=False, qk_scale=None,
+ drop=0., attn_drop=0., drop_path=0., act_layer=nn.GELU,
+ norm_layer=nn.LayerNorm, attn_head_dim=None, num_expert=1, part_features=None
+ ):
+ super().__init__()
+
+ self.norm1 = norm_layer(dim)
+ self.attn = Attention(
+ dim, num_heads=num_heads, qkv_bias=qkv_bias, qk_scale=qk_scale,
+ attn_drop=attn_drop, proj_drop=drop, attn_head_dim=attn_head_dim
+ )
+
+ # NOTE: drop path for stochastic depth, we shall see if this is better than dropout here
+ self.drop_path = DropPath(drop_path) if drop_path > 0. else nn.Identity()
+ self.norm2 = norm_layer(dim)
+ mlp_hidden_dim = int(dim * mlp_ratio)
+ self.mlp = MoEMlp(num_expert=num_expert, in_features=dim, hidden_features=mlp_hidden_dim,
+ act_layer=act_layer, drop=drop, part_features=part_features)
+
+ def forward(self, x, indices=None):
+
+ x = x + self.drop_path(self.attn(self.norm1(x)))
+ x = x + self.drop_path(self.mlp(self.norm2(x), indices))
+ return x
+
+
+class PatchEmbed(nn.Module):
+ """ Image to Patch Embedding
+ """
+ def __init__(self, img_size=224, patch_size=16, in_chans=3, embed_dim=768, ratio=1):
+ super().__init__()
+ img_size = to_2tuple(img_size)
+ patch_size = to_2tuple(patch_size)
+ num_patches = (img_size[1] // patch_size[1]) * (img_size[0] // patch_size[0]) * (ratio ** 2)
+ self.patch_shape = (int(img_size[0] // patch_size[0] * ratio), int(img_size[1] // patch_size[1] * ratio))
+ self.origin_patch_shape = (int(img_size[0] // patch_size[0]), int(img_size[1] // patch_size[1]))
+ self.img_size = img_size
+ self.patch_size = patch_size
+ self.num_patches = num_patches
+
+ self.proj = nn.Conv2d(in_chans, embed_dim, kernel_size=patch_size, stride=(patch_size[0] // ratio), padding=4 + 2 * (ratio//2-1))
+
+ def forward(self, x, **kwargs):
+ B, C, H, W = x.shape
+ x = self.proj(x)
+ Hp, Wp = x.shape[2], x.shape[3]
+
+ x = x.flatten(2).transpose(1, 2)
+ return x, (Hp, Wp)
+
+
+class HybridEmbed(nn.Module):
+ """ CNN Feature Map Embedding
+ Extract feature map from CNN, flatten, project to embedding dim.
+ """
+ def __init__(self, backbone, img_size=224, feature_size=None, in_chans=3, embed_dim=768):
+ super().__init__()
+ assert isinstance(backbone, nn.Module)
+ img_size = to_2tuple(img_size)
+ self.img_size = img_size
+ self.backbone = backbone
+ if feature_size is None:
+ with torch.no_grad():
+ training = backbone.training
+ if training:
+ backbone.eval()
+ o = self.backbone(torch.zeros(1, in_chans, img_size[0], img_size[1]))[-1]
+ feature_size = o.shape[-2:]
+ feature_dim = o.shape[1]
+ backbone.train(training)
+ else:
+ feature_size = to_2tuple(feature_size)
+ feature_dim = self.backbone.feature_info.channels()[-1]
+ self.num_patches = feature_size[0] * feature_size[1]
+ self.proj = nn.Linear(feature_dim, embed_dim)
+
+ def forward(self, x):
+ x = self.backbone(x)[-1]
+ x = x.flatten(2).transpose(1, 2)
+ x = self.proj(x)
+ return x
+
+
+@BACKBONES.register_module()
+class ViTMoE(BaseBackbone):
+
+ def __init__(self,
+ img_size=224, patch_size=16, in_chans=3, num_classes=80, embed_dim=768, depth=12,
+ num_heads=12, mlp_ratio=4., qkv_bias=False, qk_scale=None, drop_rate=0., attn_drop_rate=0.,
+ drop_path_rate=0., hybrid_backbone=None, norm_layer=None, use_checkpoint=False,
+ frozen_stages=-1, ratio=1, last_norm=True,
+ patch_padding='pad', freeze_attn=False, freeze_ffn=False,
+ num_expert=1, part_features=None
+ ):
+ # Protect mutable default arguments
+ super(ViTMoE, self).__init__()
+ norm_layer = norm_layer or partial(nn.LayerNorm, eps=1e-6)
+ self.num_classes = num_classes
+ self.num_features = self.embed_dim = embed_dim # num_features for consistency with other models
+ self.frozen_stages = frozen_stages
+ self.use_checkpoint = use_checkpoint
+ self.patch_padding = patch_padding
+ self.freeze_attn = freeze_attn
+ self.freeze_ffn = freeze_ffn
+ self.depth = depth
+
+ if hybrid_backbone is not None:
+ self.patch_embed = HybridEmbed(
+ hybrid_backbone, img_size=img_size, in_chans=in_chans, embed_dim=embed_dim)
+ else:
+ self.patch_embed = PatchEmbed(
+ img_size=img_size, patch_size=patch_size, in_chans=in_chans, embed_dim=embed_dim, ratio=ratio)
+ num_patches = self.patch_embed.num_patches
+
+ self.part_features = part_features
+
+ self.pos_embed = nn.Parameter(torch.zeros(1, num_patches + 1, embed_dim))
+
+ dpr = [x.item() for x in torch.linspace(0, drop_path_rate, depth)] # stochastic depth decay rule
+
+ self.blocks = nn.ModuleList([
+ Block(
+ dim=embed_dim, num_heads=num_heads, mlp_ratio=mlp_ratio, qkv_bias=qkv_bias, qk_scale=qk_scale,
+ drop=drop_rate, attn_drop=attn_drop_rate, drop_path=dpr[i], norm_layer=norm_layer,
+ num_expert=num_expert, part_features=part_features
+ )
+ for i in range(depth)])
+
+ self.last_norm = norm_layer(embed_dim) if last_norm else nn.Identity()
+
+ if self.pos_embed is not None:
+ trunc_normal_(self.pos_embed, std=.02)
+
+ self._freeze_stages()
+
+ def _freeze_stages(self):
+ """Freeze parameters."""
+ if self.frozen_stages >= 0:
+ self.patch_embed.eval()
+ for param in self.patch_embed.parameters():
+ param.requires_grad = False
+
+ for i in range(1, self.frozen_stages + 1):
+ m = self.blocks[i]
+ m.eval()
+ for param in m.parameters():
+ param.requires_grad = False
+
+ if self.freeze_attn:
+ for i in range(0, self.depth):
+ m = self.blocks[i]
+ m.attn.eval()
+ m.norm1.eval()
+ for param in m.attn.parameters():
+ param.requires_grad = False
+ for param in m.norm1.parameters():
+ param.requires_grad = False
+
+ if self.freeze_ffn:
+ self.pos_embed.requires_grad = False
+ self.patch_embed.eval()
+ for param in self.patch_embed.parameters():
+ param.requires_grad = False
+ for i in range(0, self.depth):
+ m = self.blocks[i]
+ m.mlp.eval()
+ m.norm2.eval()
+ for param in m.mlp.parameters():
+ param.requires_grad = False
+ for param in m.norm2.parameters():
+ param.requires_grad = False
+
+ def init_weights(self, pretrained=None):
+ """Initialize the weights in backbone.
+ Args:
+ pretrained (str, optional): Path to pre-trained weights.
+ Defaults to None.
+ """
+ super().init_weights(pretrained, patch_padding=self.patch_padding, part_features=self.part_features)
+
+ if pretrained is None:
+ def _init_weights(m):
+ if isinstance(m, nn.Linear):
+ trunc_normal_(m.weight, std=.02)
+ if isinstance(m, nn.Linear) and m.bias is not None:
+ nn.init.constant_(m.bias, 0)
+ elif isinstance(m, nn.LayerNorm):
+ nn.init.constant_(m.bias, 0)
+ nn.init.constant_(m.weight, 1.0)
+
+ self.apply(_init_weights)
+
+ def get_num_layers(self):
+ return len(self.blocks)
+
+ @torch.jit.ignore
+ def no_weight_decay(self):
+ return {'pos_embed', 'cls_token'}
+
+ def forward_features(self, x, dataset_source=None):
+ B, C, H, W = x.shape
+ x, (Hp, Wp) = self.patch_embed(x)
+
+ if self.pos_embed is not None:
+ # fit for multiple GPU training
+ # since the first element for pos embed (sin-cos manner) is zero, it will cause no difference
+ x = x + self.pos_embed[:, 1:] + self.pos_embed[:, :1]
+
+ for blk in self.blocks:
+ if self.use_checkpoint:
+ x = checkpoint.checkpoint(blk, x, dataset_source)
+ else:
+ x = blk(x, dataset_source)
+
+ x = self.last_norm(x)
+
+ xp = x.permute(0, 2, 1).reshape(B, -1, Hp, Wp).contiguous()
+
+ return xp
+
+ def forward(self, x, dataset_source=None):
+ x = self.forward_features(x, dataset_source)
+ return x
+
+ def train(self, mode=True):
+ """Convert the model into training mode."""
+ super().train(mode)
+ self._freeze_stages()
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/builder.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/builder.py
new file mode 100644
index 0000000..220839d
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/builder.py
@@ -0,0 +1,44 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from mmcv.cnn import MODELS as MMCV_MODELS
+from mmcv.cnn import build_model_from_cfg
+from mmcv.utils import Registry
+
+MODELS = Registry(
+ 'models', build_func=build_model_from_cfg, parent=MMCV_MODELS)
+
+BACKBONES = MODELS
+NECKS = MODELS
+HEADS = MODELS
+LOSSES = MODELS
+POSENETS = MODELS
+MESH_MODELS = MODELS
+
+
+def build_backbone(cfg):
+ """Build backbone."""
+ return BACKBONES.build(cfg)
+
+
+def build_neck(cfg):
+ """Build neck."""
+ return NECKS.build(cfg)
+
+
+def build_head(cfg):
+ """Build head."""
+ return HEADS.build(cfg)
+
+
+def build_loss(cfg):
+ """Build loss."""
+ return LOSSES.build(cfg)
+
+
+def build_posenet(cfg):
+ """Build posenet."""
+ return POSENETS.build(cfg)
+
+
+def build_mesh_model(cfg):
+ """Build mesh model."""
+ return MESH_MODELS.build(cfg)
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/detectors/__init__.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/detectors/__init__.py
new file mode 100644
index 0000000..8078cb9
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/detectors/__init__.py
@@ -0,0 +1,18 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from .associative_embedding import AssociativeEmbedding
+from .interhand_3d import Interhand3D
+from .mesh import ParametricMesh
+from .multi_task import MultiTask
+from .multiview_pose import (DetectAndRegress, VoxelCenterDetector,
+ VoxelSinglePose)
+from .pose_lifter import PoseLifter
+from .posewarper import PoseWarper
+from .top_down import TopDown
+from .top_down_moe import TopDownMoE
+from .top_down_coco_plus import TopDownCoCoPlus
+
+__all__ = [
+ 'TopDown', 'AssociativeEmbedding', 'ParametricMesh', 'MultiTask',
+ 'PoseLifter', 'Interhand3D', 'PoseWarper', 'DetectAndRegress',
+ 'VoxelCenterDetector', 'VoxelSinglePose', 'TopDownMoE', 'TopDownCoCoPlus'
+]
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/detectors/associative_embedding.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/detectors/associative_embedding.py
new file mode 100644
index 0000000..100c780
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/detectors/associative_embedding.py
@@ -0,0 +1,420 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import warnings
+
+import mmcv
+import torch
+from mmcv.image import imwrite
+from mmcv.utils.misc import deprecated_api_warning
+from mmcv.visualization.image import imshow
+
+from mmpose.core.evaluation import (aggregate_scale, aggregate_stage_flip,
+ flip_feature_maps, get_group_preds,
+ split_ae_outputs)
+from mmpose.core.post_processing.group import HeatmapParser
+from mmpose.core.visualization import imshow_keypoints
+from .. import builder
+from ..builder import POSENETS
+from .base import BasePose
+
+try:
+ from mmcv.runner import auto_fp16
+except ImportError:
+ warnings.warn('auto_fp16 from mmpose will be deprecated from v0.15.0'
+ 'Please install mmcv>=1.1.4')
+ from mmpose.core import auto_fp16
+
+
+@POSENETS.register_module()
+class AssociativeEmbedding(BasePose):
+ """Associative embedding pose detectors.
+
+ Args:
+ backbone (dict): Backbone modules to extract feature.
+ keypoint_head (dict): Keypoint head to process feature.
+ train_cfg (dict): Config for training. Default: None.
+ test_cfg (dict): Config for testing. Default: None.
+ pretrained (str): Path to the pretrained models.
+ loss_pose (None): Deprecated arguments. Please use
+ ``loss_keypoint`` for heads instead.
+ """
+
+ def __init__(self,
+ backbone,
+ keypoint_head=None,
+ train_cfg=None,
+ test_cfg=None,
+ pretrained=None,
+ loss_pose=None):
+ super().__init__()
+ self.fp16_enabled = False
+
+ self.backbone = builder.build_backbone(backbone)
+
+ if keypoint_head is not None:
+ if 'loss_keypoint' not in keypoint_head and loss_pose is not None:
+ warnings.warn(
+ '`loss_pose` for BottomUp is deprecated, '
+ 'use `loss_keypoint` for heads instead. See '
+ 'https://github.com/open-mmlab/mmpose/pull/382'
+ ' for more information.', DeprecationWarning)
+ keypoint_head['loss_keypoint'] = loss_pose
+
+ self.keypoint_head = builder.build_head(keypoint_head)
+
+ self.train_cfg = train_cfg
+ self.test_cfg = test_cfg
+ self.use_udp = test_cfg.get('use_udp', False)
+ self.parser = HeatmapParser(self.test_cfg)
+ self.init_weights(pretrained=pretrained)
+
+ @property
+ def with_keypoint(self):
+ """Check if has keypoint_head."""
+ return hasattr(self, 'keypoint_head')
+
+ def init_weights(self, pretrained=None):
+ """Weight initialization for model."""
+ self.backbone.init_weights(pretrained)
+ if self.with_keypoint:
+ self.keypoint_head.init_weights()
+
+ @auto_fp16(apply_to=('img', ))
+ def forward(self,
+ img=None,
+ targets=None,
+ masks=None,
+ joints=None,
+ img_metas=None,
+ return_loss=True,
+ return_heatmap=False,
+ **kwargs):
+ """Calls either forward_train or forward_test depending on whether
+ return_loss is True.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+ - num_img_channel: C
+ - img_width: imgW
+ - img_height: imgH
+ - heatmaps weight: W
+ - heatmaps height: H
+ - max_num_people: M
+
+ Args:
+ img (torch.Tensor[N,C,imgH,imgW]): Input image.
+ targets (list(torch.Tensor[N,K,H,W])): Multi-scale target heatmaps.
+ masks (list(torch.Tensor[N,H,W])): Masks of multi-scale target
+ heatmaps
+ joints (list(torch.Tensor[N,M,K,2])): Joints of multi-scale target
+ heatmaps for ae loss
+ img_metas (dict): Information about val & test.
+ By default it includes:
+
+ - "image_file": image path
+ - "aug_data": input
+ - "test_scale_factor": test scale factor
+ - "base_size": base size of input
+ - "center": center of image
+ - "scale": scale of image
+ - "flip_index": flip index of keypoints
+ return loss (bool): ``return_loss=True`` for training,
+ ``return_loss=False`` for validation & test.
+ return_heatmap (bool) : Option to return heatmap.
+
+ Returns:
+ dict|tuple: if 'return_loss' is true, then return losses. \
+ Otherwise, return predicted poses, scores, image \
+ paths and heatmaps.
+ """
+
+ if return_loss:
+ return self.forward_train(img, targets, masks, joints, img_metas,
+ **kwargs)
+ return self.forward_test(
+ img, img_metas, return_heatmap=return_heatmap, **kwargs)
+
+ def forward_train(self, img, targets, masks, joints, img_metas, **kwargs):
+ """Forward the bottom-up model and calculate the loss.
+
+ Note:
+ batch_size: N
+ num_keypoints: K
+ num_img_channel: C
+ img_width: imgW
+ img_height: imgH
+ heatmaps weight: W
+ heatmaps height: H
+ max_num_people: M
+
+ Args:
+ img (torch.Tensor[N,C,imgH,imgW]): Input image.
+ targets (List(torch.Tensor[N,K,H,W])): Multi-scale target heatmaps.
+ masks (List(torch.Tensor[N,H,W])): Masks of multi-scale target
+ heatmaps
+ joints (List(torch.Tensor[N,M,K,2])): Joints of multi-scale target
+ heatmaps for ae loss
+ img_metas (dict):Information about val&test
+ By default this includes:
+ - "image_file": image path
+ - "aug_data": input
+ - "test_scale_factor": test scale factor
+ - "base_size": base size of input
+ - "center": center of image
+ - "scale": scale of image
+ - "flip_index": flip index of keypoints
+
+ Returns:
+ dict: The total loss for bottom-up
+ """
+
+ output = self.backbone(img)
+
+ if self.with_keypoint:
+ output = self.keypoint_head(output)
+
+ # if return loss
+ losses = dict()
+ if self.with_keypoint:
+ keypoint_losses = self.keypoint_head.get_loss(
+ output, targets, masks, joints)
+ losses.update(keypoint_losses)
+
+ return losses
+
+ def forward_dummy(self, img):
+ """Used for computing network FLOPs.
+
+ See ``tools/get_flops.py``.
+
+ Args:
+ img (torch.Tensor): Input image.
+
+ Returns:
+ Tensor: Outputs.
+ """
+ output = self.backbone(img)
+ if self.with_keypoint:
+ output = self.keypoint_head(output)
+ return output
+
+ def forward_test(self, img, img_metas, return_heatmap=False, **kwargs):
+ """Inference the bottom-up model.
+
+ Note:
+ - Batchsize: N (currently support batchsize = 1)
+ - num_img_channel: C
+ - img_width: imgW
+ - img_height: imgH
+
+ Args:
+ flip_index (List(int)):
+ aug_data (List(Tensor[NxCximgHximgW])): Multi-scale image
+ test_scale_factor (List(float)): Multi-scale factor
+ base_size (Tuple(int)): Base size of image when scale is 1
+ center (np.ndarray): center of image
+ scale (np.ndarray): the scale of image
+ """
+ assert img.size(0) == 1
+ assert len(img_metas) == 1
+
+ img_metas = img_metas[0]
+
+ aug_data = img_metas['aug_data']
+
+ test_scale_factor = img_metas['test_scale_factor']
+ base_size = img_metas['base_size']
+ center = img_metas['center']
+ scale = img_metas['scale']
+
+ result = {}
+
+ scale_heatmaps_list = []
+ scale_tags_list = []
+
+ for idx, s in enumerate(sorted(test_scale_factor, reverse=True)):
+ image_resized = aug_data[idx].to(img.device)
+
+ features = self.backbone(image_resized)
+ if self.with_keypoint:
+ outputs = self.keypoint_head(features)
+
+ heatmaps, tags = split_ae_outputs(
+ outputs, self.test_cfg['num_joints'],
+ self.test_cfg['with_heatmaps'], self.test_cfg['with_ae'],
+ self.test_cfg.get('select_output_index', range(len(outputs))))
+
+ if self.test_cfg.get('flip_test', True):
+ # use flip test
+ features_flipped = self.backbone(
+ torch.flip(image_resized, [3]))
+ if self.with_keypoint:
+ outputs_flipped = self.keypoint_head(features_flipped)
+
+ heatmaps_flipped, tags_flipped = split_ae_outputs(
+ outputs_flipped, self.test_cfg['num_joints'],
+ self.test_cfg['with_heatmaps'], self.test_cfg['with_ae'],
+ self.test_cfg.get('select_output_index',
+ range(len(outputs))))
+
+ heatmaps_flipped = flip_feature_maps(
+ heatmaps_flipped, flip_index=img_metas['flip_index'])
+ if self.test_cfg['tag_per_joint']:
+ tags_flipped = flip_feature_maps(
+ tags_flipped, flip_index=img_metas['flip_index'])
+ else:
+ tags_flipped = flip_feature_maps(
+ tags_flipped, flip_index=None, flip_output=True)
+
+ else:
+ heatmaps_flipped = None
+ tags_flipped = None
+
+ aggregated_heatmaps = aggregate_stage_flip(
+ heatmaps,
+ heatmaps_flipped,
+ index=-1,
+ project2image=self.test_cfg['project2image'],
+ size_projected=base_size,
+ align_corners=self.test_cfg.get('align_corners', True),
+ aggregate_stage='average',
+ aggregate_flip='average')
+
+ aggregated_tags = aggregate_stage_flip(
+ tags,
+ tags_flipped,
+ index=-1,
+ project2image=self.test_cfg['project2image'],
+ size_projected=base_size,
+ align_corners=self.test_cfg.get('align_corners', True),
+ aggregate_stage='concat',
+ aggregate_flip='concat')
+
+ if s == 1 or len(test_scale_factor) == 1:
+ if isinstance(aggregated_tags, list):
+ scale_tags_list.extend(aggregated_tags)
+ else:
+ scale_tags_list.append(aggregated_tags)
+
+ if isinstance(aggregated_heatmaps, list):
+ scale_heatmaps_list.extend(aggregated_heatmaps)
+ else:
+ scale_heatmaps_list.append(aggregated_heatmaps)
+
+ aggregated_heatmaps = aggregate_scale(
+ scale_heatmaps_list,
+ align_corners=self.test_cfg.get('align_corners', True),
+ aggregate_scale='average')
+
+ aggregated_tags = aggregate_scale(
+ scale_tags_list,
+ align_corners=self.test_cfg.get('align_corners', True),
+ aggregate_scale='unsqueeze_concat')
+
+ heatmap_size = aggregated_heatmaps.shape[2:4]
+ tag_size = aggregated_tags.shape[2:4]
+ if heatmap_size != tag_size:
+ tmp = []
+ for idx in range(aggregated_tags.shape[-1]):
+ tmp.append(
+ torch.nn.functional.interpolate(
+ aggregated_tags[..., idx],
+ size=heatmap_size,
+ mode='bilinear',
+ align_corners=self.test_cfg.get('align_corners',
+ True)).unsqueeze(-1))
+ aggregated_tags = torch.cat(tmp, dim=-1)
+
+ # perform grouping
+ grouped, scores = self.parser.parse(aggregated_heatmaps,
+ aggregated_tags,
+ self.test_cfg['adjust'],
+ self.test_cfg['refine'])
+
+ preds = get_group_preds(
+ grouped,
+ center,
+ scale, [aggregated_heatmaps.size(3),
+ aggregated_heatmaps.size(2)],
+ use_udp=self.use_udp)
+
+ image_paths = []
+ image_paths.append(img_metas['image_file'])
+
+ if return_heatmap:
+ output_heatmap = aggregated_heatmaps.detach().cpu().numpy()
+ else:
+ output_heatmap = None
+
+ result['preds'] = preds
+ result['scores'] = scores
+ result['image_paths'] = image_paths
+ result['output_heatmap'] = output_heatmap
+
+ return result
+
+ @deprecated_api_warning({'pose_limb_color': 'pose_link_color'},
+ cls_name='AssociativeEmbedding')
+ def show_result(self,
+ img,
+ result,
+ skeleton=None,
+ kpt_score_thr=0.3,
+ bbox_color=None,
+ pose_kpt_color=None,
+ pose_link_color=None,
+ radius=4,
+ thickness=1,
+ font_scale=0.5,
+ win_name='',
+ show=False,
+ show_keypoint_weight=False,
+ wait_time=0,
+ out_file=None):
+ """Draw `result` over `img`.
+
+ Args:
+ img (str or Tensor): The image to be displayed.
+ result (list[dict]): The results to draw over `img`
+ (bbox_result, pose_result).
+ skeleton (list[list]): The connection of keypoints.
+ skeleton is 0-based indexing.
+ kpt_score_thr (float, optional): Minimum score of keypoints
+ to be shown. Default: 0.3.
+ pose_kpt_color (np.array[Nx3]`): Color of N keypoints.
+ If None, do not draw keypoints.
+ pose_link_color (np.array[Mx3]): Color of M links.
+ If None, do not draw links.
+ radius (int): Radius of circles.
+ thickness (int): Thickness of lines.
+ font_scale (float): Font scales of texts.
+ win_name (str): The window name.
+ show (bool): Whether to show the image. Default: False.
+ show_keypoint_weight (bool): Whether to change the transparency
+ using the predicted confidence scores of keypoints.
+ wait_time (int): Value of waitKey param.
+ Default: 0.
+ out_file (str or None): The filename to write the image.
+ Default: None.
+
+ Returns:
+ Tensor: Visualized image only if not `show` or `out_file`
+ """
+ img = mmcv.imread(img)
+ img = img.copy()
+ img_h, img_w, _ = img.shape
+
+ pose_result = []
+ for res in result:
+ pose_result.append(res['keypoints'])
+
+ imshow_keypoints(img, pose_result, skeleton, kpt_score_thr,
+ pose_kpt_color, pose_link_color, radius, thickness)
+
+ if show:
+ imshow(img, win_name, wait_time)
+
+ if out_file is not None:
+ imwrite(img, out_file)
+
+ return img
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/detectors/base.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/detectors/base.py
new file mode 100644
index 0000000..5d459b4
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/detectors/base.py
@@ -0,0 +1,131 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from abc import ABCMeta, abstractmethod
+from collections import OrderedDict
+
+import torch
+import torch.distributed as dist
+import torch.nn as nn
+
+
+class BasePose(nn.Module, metaclass=ABCMeta):
+ """Base class for pose detectors.
+
+ All recognizers should subclass it.
+ All subclass should overwrite:
+ Methods:`forward_train`, supporting to forward when training.
+ Methods:`forward_test`, supporting to forward when testing.
+
+ Args:
+ backbone (dict): Backbone modules to extract feature.
+ head (dict): Head modules to give output.
+ train_cfg (dict): Config for training. Default: None.
+ test_cfg (dict): Config for testing. Default: None.
+ """
+
+ @abstractmethod
+ def forward_train(self, img, img_metas, **kwargs):
+ """Defines the computation performed at training."""
+
+ @abstractmethod
+ def forward_test(self, img, img_metas, **kwargs):
+ """Defines the computation performed at testing."""
+
+ @abstractmethod
+ def forward(self, img, img_metas, return_loss=True, **kwargs):
+ """Forward function."""
+
+ @staticmethod
+ def _parse_losses(losses):
+ """Parse the raw outputs (losses) of the network.
+
+ Args:
+ losses (dict): Raw output of the network, which usually contain
+ losses and other necessary information.
+
+ Returns:
+ tuple[Tensor, dict]: (loss, log_vars), loss is the loss tensor \
+ which may be a weighted sum of all losses, log_vars \
+ contains all the variables to be sent to the logger.
+ """
+ log_vars = OrderedDict()
+ for loss_name, loss_value in losses.items():
+ if isinstance(loss_value, torch.Tensor):
+ log_vars[loss_name] = loss_value.mean()
+ elif isinstance(loss_value, float):
+ log_vars[loss_name] = loss_value
+ elif isinstance(loss_value, list):
+ log_vars[loss_name] = sum(_loss.mean() for _loss in loss_value)
+ else:
+ raise TypeError(
+ f'{loss_name} is not a tensor or list of tensors or float')
+
+ loss = sum(_value for _key, _value in log_vars.items()
+ if 'loss' in _key)
+
+ log_vars['loss'] = loss
+ for loss_name, loss_value in log_vars.items():
+ # reduce loss when distributed training
+ if not isinstance(loss_value, float):
+ if dist.is_available() and dist.is_initialized():
+ loss_value = loss_value.data.clone()
+ dist.all_reduce(loss_value.div_(dist.get_world_size()))
+ log_vars[loss_name] = loss_value.item()
+ else:
+ log_vars[loss_name] = loss_value
+
+ return loss, log_vars
+
+ def train_step(self, data_batch, optimizer, **kwargs):
+ """The iteration step during training.
+
+ This method defines an iteration step during training, except for the
+ back propagation and optimizer updating, which are done in an optimizer
+ hook. Note that in some complicated cases or models, the whole process
+ including back propagation and optimizer updating is also defined in
+ this method, such as GAN.
+
+ Args:
+ data_batch (dict): The output of dataloader.
+ optimizer (:obj:`torch.optim.Optimizer` | dict): The optimizer of
+ runner is passed to ``train_step()``. This argument is unused
+ and reserved.
+
+ Returns:
+ dict: It should contain at least 3 keys: ``loss``, ``log_vars``,
+ ``num_samples``.
+ ``loss`` is a tensor for back propagation, which can be a
+ weighted sum of multiple losses.
+ ``log_vars`` contains all the variables to be sent to the
+ logger.
+ ``num_samples`` indicates the batch size (when the model is
+ DDP, it means the batch size on each GPU), which is used for
+ averaging the logs.
+ """
+ losses = self.forward(**data_batch)
+
+ loss, log_vars = self._parse_losses(losses)
+
+ outputs = dict(
+ loss=loss,
+ log_vars=log_vars,
+ num_samples=len(next(iter(data_batch.values()))))
+
+ return outputs
+
+ def val_step(self, data_batch, optimizer, **kwargs):
+ """The iteration step during validation.
+
+ This method shares the same signature as :func:`train_step`, but used
+ during val epochs. Note that the evaluation after training epochs is
+ not implemented with this method, but an evaluation hook.
+ """
+ results = self.forward(return_loss=False, **data_batch)
+
+ outputs = dict(results=results)
+
+ return outputs
+
+ @abstractmethod
+ def show_result(self, **kwargs):
+ """Visualize the results."""
+ raise NotImplementedError
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/detectors/interhand_3d.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/detectors/interhand_3d.py
new file mode 100644
index 0000000..5a4d6bd
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/detectors/interhand_3d.py
@@ -0,0 +1,227 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import mmcv
+import numpy as np
+from mmcv.utils.misc import deprecated_api_warning
+
+from mmpose.core import imshow_keypoints, imshow_keypoints_3d
+from ..builder import POSENETS
+from .top_down import TopDown
+
+
+@POSENETS.register_module()
+class Interhand3D(TopDown):
+ """Top-down interhand 3D pose detector of paper ref: Gyeongsik Moon.
+
+ "InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose
+ Estimation from a Single RGB Image". A child class of TopDown detector.
+ """
+
+ def forward(self,
+ img,
+ target=None,
+ target_weight=None,
+ img_metas=None,
+ return_loss=True,
+ **kwargs):
+ """Calls either forward_train or forward_test depending on whether
+ return_loss=True. Note this setting will change the expected inputs.
+ When `return_loss=True`, img and img_meta are single-nested (i.e.
+ Tensor and List[dict]), and when `resturn_loss=False`, img and img_meta
+ should be double nested (i.e. list[Tensor], list[list[dict]]), with
+ the outer list indicating test time augmentations.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+ - num_img_channel: C (Default: 3)
+ - img height: imgH
+ - img width: imgW
+ - heatmaps height: H
+ - heatmaps weight: W
+
+ Args:
+ img (torch.Tensor[NxCximgHximgW]): Input images.
+ target (list[torch.Tensor]): Target heatmaps, relative hand
+ root depth and hand type.
+ target_weight (list[torch.Tensor]): Weights for target
+ heatmaps, relative hand root depth and hand type.
+ img_metas (list(dict)): Information about data augmentation
+ By default this includes:
+
+ - "image_file: path to the image file
+ - "center": center of the bbox
+ - "scale": scale of the bbox
+ - "rotation": rotation of the bbox
+ - "bbox_score": score of bbox
+ - "heatmap3d_depth_bound": depth bound of hand keypoint 3D
+ heatmap
+ - "root_depth_bound": depth bound of relative root depth 1D
+ heatmap
+ return_loss (bool): Option to `return loss`. `return loss=True`
+ for training, `return loss=False` for validation & test.
+
+ Returns:
+ dict|tuple: if `return loss` is true, then return losses. \
+ Otherwise, return predicted poses, boxes, image paths, \
+ heatmaps, relative hand root depth and hand type.
+ """
+ if return_loss:
+ return self.forward_train(img, target, target_weight, img_metas,
+ **kwargs)
+ return self.forward_test(img, img_metas, **kwargs)
+
+ def forward_test(self, img, img_metas, **kwargs):
+ """Defines the computation performed at every call when testing."""
+ assert img.size(0) == len(img_metas)
+ batch_size, _, img_height, img_width = img.shape
+ if batch_size > 1:
+ assert 'bbox_id' in img_metas[0]
+
+ features = self.backbone(img)
+ if self.with_neck:
+ features = self.neck(features)
+ if self.with_keypoint:
+ output = self.keypoint_head.inference_model(
+ features, flip_pairs=None)
+
+ if self.test_cfg.get('flip_test', True):
+ img_flipped = img.flip(3)
+ features_flipped = self.backbone(img_flipped)
+ if self.with_neck:
+ features_flipped = self.neck(features_flipped)
+ if self.with_keypoint:
+ output_flipped = self.keypoint_head.inference_model(
+ features_flipped, img_metas[0]['flip_pairs'])
+ output = [(out + out_flipped) * 0.5
+ for out, out_flipped in zip(output, output_flipped)]
+
+ if self.with_keypoint:
+ result = self.keypoint_head.decode(
+ img_metas, output, img_size=[img_width, img_height])
+ else:
+ result = {}
+ return result
+
+ @deprecated_api_warning({'pose_limb_color': 'pose_link_color'},
+ cls_name='Interhand3D')
+ def show_result(self,
+ result,
+ img=None,
+ skeleton=None,
+ kpt_score_thr=0.3,
+ radius=8,
+ bbox_color='green',
+ thickness=2,
+ pose_kpt_color=None,
+ pose_link_color=None,
+ vis_height=400,
+ num_instances=-1,
+ win_name='',
+ show=False,
+ wait_time=0,
+ out_file=None):
+ """Visualize 3D pose estimation results.
+
+ Args:
+ result (list[dict]): The pose estimation results containing:
+
+ - "keypoints_3d" ([K,4]): 3D keypoints
+ - "keypoints" ([K,3] or [T,K,3]): Optional for visualizing
+ 2D inputs. If a sequence is given, only the last frame
+ will be used for visualization
+ - "bbox" ([4,] or [T,4]): Optional for visualizing 2D inputs
+ - "title" (str): title for the subplot
+ img (str or Tensor): Optional. The image to visualize 2D inputs on.
+ skeleton (list of [idx_i,idx_j]): Skeleton described by a list of
+ links, each is a pair of joint indices.
+ kpt_score_thr (float, optional): Minimum score of keypoints
+ to be shown. Default: 0.3.
+ radius (int): Radius of circles.
+ bbox_color (str or tuple or :obj:`Color`): Color of bbox lines.
+ thickness (int): Thickness of lines.
+ pose_kpt_color (np.array[Nx3]`): Color of N keypoints.
+ If None, do not draw keypoints.
+ pose_link_color (np.array[Mx3]): Color of M limbs.
+ If None, do not draw limbs.
+ vis_height (int): The image height of the visualization. The width
+ will be N*vis_height depending on the number of visualized
+ items.
+ num_instances (int): Number of instances to be shown in 3D. If
+ smaller than 0, all the instances in the pose_result will be
+ shown. Otherwise, pad or truncate the pose_result to a length
+ of num_instances.
+ win_name (str): The window name.
+ show (bool): Whether to show the image. Default: False.
+ wait_time (int): Value of waitKey param.
+ Default: 0.
+ out_file (str or None): The filename to write the image.
+ Default: None.
+
+ Returns:
+ Tensor: Visualized img, only if not `show` or `out_file`.
+ """
+ if num_instances < 0:
+ assert len(result) > 0
+ result = sorted(result, key=lambda x: x.get('track_id', 0))
+
+ # draw image and 2d poses
+ if img is not None:
+ img = mmcv.imread(img)
+
+ bbox_result = []
+ pose_2d = []
+ for res in result:
+ if 'bbox' in res:
+ bbox = np.array(res['bbox'])
+ if bbox.ndim != 1:
+ assert bbox.ndim == 2
+ bbox = bbox[-1] # Get bbox from the last frame
+ bbox_result.append(bbox)
+ if 'keypoints' in res:
+ kpts = np.array(res['keypoints'])
+ if kpts.ndim != 2:
+ assert kpts.ndim == 3
+ kpts = kpts[-1] # Get 2D keypoints from the last frame
+ pose_2d.append(kpts)
+
+ if len(bbox_result) > 0:
+ bboxes = np.vstack(bbox_result)
+ mmcv.imshow_bboxes(
+ img,
+ bboxes,
+ colors=bbox_color,
+ top_k=-1,
+ thickness=2,
+ show=False)
+ if len(pose_2d) > 0:
+ imshow_keypoints(
+ img,
+ pose_2d,
+ skeleton,
+ kpt_score_thr=kpt_score_thr,
+ pose_kpt_color=pose_kpt_color,
+ pose_link_color=pose_link_color,
+ radius=radius,
+ thickness=thickness)
+ img = mmcv.imrescale(img, scale=vis_height / img.shape[0])
+
+ img_vis = imshow_keypoints_3d(
+ result,
+ img,
+ skeleton,
+ pose_kpt_color,
+ pose_link_color,
+ vis_height,
+ axis_limit=300,
+ axis_azimuth=-115,
+ axis_elev=15,
+ kpt_score_thr=kpt_score_thr,
+ num_instances=num_instances)
+
+ if show:
+ mmcv.visualization.imshow(img_vis, win_name, wait_time)
+
+ if out_file is not None:
+ mmcv.imwrite(img_vis, out_file)
+
+ return img_vis
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/detectors/mesh.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/detectors/mesh.py
new file mode 100644
index 0000000..0af18e3
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/detectors/mesh.py
@@ -0,0 +1,438 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import cv2
+import mmcv
+import numpy as np
+import torch
+
+from mmpose.core.visualization.image import imshow_mesh_3d
+from mmpose.models.misc.discriminator import SMPLDiscriminator
+from .. import builder
+from ..builder import POSENETS
+from .base import BasePose
+
+
+def set_requires_grad(nets, requires_grad=False):
+ """Set requies_grad for all the networks.
+
+ Args:
+ nets (nn.Module | list[nn.Module]): A list of networks or a single
+ network.
+ requires_grad (bool): Whether the networks require gradients or not
+ """
+ if not isinstance(nets, list):
+ nets = [nets]
+ for net in nets:
+ if net is not None:
+ for param in net.parameters():
+ param.requires_grad = requires_grad
+
+
+@POSENETS.register_module()
+class ParametricMesh(BasePose):
+ """Model-based 3D human mesh detector. Take a single color image as input
+ and output 3D joints, SMPL parameters and camera parameters.
+
+ Args:
+ backbone (dict): Backbone modules to extract feature.
+ mesh_head (dict): Mesh head to process feature.
+ smpl (dict): Config for SMPL model.
+ disc (dict): Discriminator for SMPL parameters. Default: None.
+ loss_gan (dict): Config for adversarial loss. Default: None.
+ loss_mesh (dict): Config for mesh loss. Default: None.
+ train_cfg (dict): Config for training. Default: None.
+ test_cfg (dict): Config for testing. Default: None.
+ pretrained (str): Path to the pretrained models.
+ """
+
+ def __init__(self,
+ backbone,
+ mesh_head,
+ smpl,
+ disc=None,
+ loss_gan=None,
+ loss_mesh=None,
+ train_cfg=None,
+ test_cfg=None,
+ pretrained=None):
+ super().__init__()
+
+ self.backbone = builder.build_backbone(backbone)
+ self.mesh_head = builder.build_head(mesh_head)
+ self.generator = torch.nn.Sequential(self.backbone, self.mesh_head)
+
+ self.smpl = builder.build_mesh_model(smpl)
+
+ self.with_gan = disc is not None and loss_gan is not None
+ if self.with_gan:
+ self.discriminator = SMPLDiscriminator(**disc)
+ self.loss_gan = builder.build_loss(loss_gan)
+ self.disc_step_count = 0
+
+ self.train_cfg = train_cfg
+ self.test_cfg = test_cfg
+
+ self.loss_mesh = builder.build_loss(loss_mesh)
+ self.init_weights(pretrained=pretrained)
+
+ def init_weights(self, pretrained=None):
+ """Weight initialization for model."""
+ self.backbone.init_weights(pretrained)
+ self.mesh_head.init_weights()
+ if self.with_gan:
+ self.discriminator.init_weights()
+
+ def train_step(self, data_batch, optimizer, **kwargs):
+ """Train step function.
+
+ In this function, the detector will finish the train step following
+ the pipeline:
+
+ 1. get fake and real SMPL parameters
+ 2. optimize discriminator (if have)
+ 3. optimize generator
+
+ If `self.train_cfg.disc_step > 1`, the train step will contain multiple
+ iterations for optimizing discriminator with different input data and
+ only one iteration for optimizing generator after `disc_step`
+ iterations for discriminator.
+
+ Args:
+ data_batch (torch.Tensor): Batch of data as input.
+ optimizer (dict[torch.optim.Optimizer]): Dict with optimizers for
+ generator and discriminator (if have).
+
+ Returns:
+ outputs (dict): Dict with loss, information for logger,
+ the number of samples.
+ """
+
+ img = data_batch['img']
+ pred_smpl = self.generator(img)
+ pred_pose, pred_beta, pred_camera = pred_smpl
+
+ # optimize discriminator (if have)
+ if self.train_cfg['disc_step'] > 0 and self.with_gan:
+ set_requires_grad(self.discriminator, True)
+ fake_data = (pred_camera.detach(), pred_pose.detach(),
+ pred_beta.detach())
+ mosh_theta = data_batch['mosh_theta']
+ real_data = (mosh_theta[:, :3], mosh_theta[:,
+ 3:75], mosh_theta[:,
+ 75:])
+ fake_score = self.discriminator(fake_data)
+ real_score = self.discriminator(real_data)
+
+ disc_losses = {}
+ disc_losses['real_loss'] = self.loss_gan(
+ real_score, target_is_real=True, is_disc=True)
+ disc_losses['fake_loss'] = self.loss_gan(
+ fake_score, target_is_real=False, is_disc=True)
+ loss_disc, log_vars_d = self._parse_losses(disc_losses)
+
+ optimizer['discriminator'].zero_grad()
+ loss_disc.backward()
+ optimizer['discriminator'].step()
+ self.disc_step_count = \
+ (self.disc_step_count + 1) % self.train_cfg['disc_step']
+
+ if self.disc_step_count != 0:
+ outputs = dict(
+ loss=loss_disc,
+ log_vars=log_vars_d,
+ num_samples=len(next(iter(data_batch.values()))))
+ return outputs
+
+ # optimize generator
+ pred_out = self.smpl(
+ betas=pred_beta,
+ body_pose=pred_pose[:, 1:],
+ global_orient=pred_pose[:, :1])
+ pred_vertices, pred_joints_3d = pred_out['vertices'], pred_out[
+ 'joints']
+
+ gt_beta = data_batch['beta']
+ gt_pose = data_batch['pose']
+ gt_vertices = self.smpl(
+ betas=gt_beta,
+ body_pose=gt_pose[:, 3:],
+ global_orient=gt_pose[:, :3])['vertices']
+
+ pred = dict(
+ pose=pred_pose,
+ beta=pred_beta,
+ camera=pred_camera,
+ vertices=pred_vertices,
+ joints_3d=pred_joints_3d)
+
+ target = {
+ key: data_batch[key]
+ for key in [
+ 'pose', 'beta', 'has_smpl', 'joints_3d', 'joints_2d',
+ 'joints_3d_visible', 'joints_2d_visible'
+ ]
+ }
+ target['vertices'] = gt_vertices
+
+ losses = self.loss_mesh(pred, target)
+
+ if self.with_gan:
+ set_requires_grad(self.discriminator, False)
+ pred_theta = (pred_camera, pred_pose, pred_beta)
+ pred_score = self.discriminator(pred_theta)
+ loss_adv = self.loss_gan(
+ pred_score, target_is_real=True, is_disc=False)
+ losses['adv_loss'] = loss_adv
+
+ loss, log_vars = self._parse_losses(losses)
+ optimizer['generator'].zero_grad()
+ loss.backward()
+ optimizer['generator'].step()
+
+ outputs = dict(
+ loss=loss,
+ log_vars=log_vars,
+ num_samples=len(next(iter(data_batch.values()))))
+
+ return outputs
+
+ def forward_train(self, *args, **kwargs):
+ """Forward function for training.
+
+ For ParametricMesh, we do not use this interface.
+ """
+ raise NotImplementedError('This interface should not be used in '
+ 'current training schedule. Please use '
+ '`train_step` for training.')
+
+ def val_step(self, data_batch, **kwargs):
+ """Forward function for evaluation.
+
+ Args:
+ data_batch (dict): Contain data for forward.
+
+ Returns:
+ dict: Contain the results from model.
+ """
+ output = self.forward_test(**data_batch, **kwargs)
+ return output
+
+ def forward_dummy(self, img):
+ """Used for computing network FLOPs.
+
+ See ``tools/get_flops.py``.
+
+ Args:
+ img (torch.Tensor): Input image.
+
+ Returns:
+ Tensor: Outputs.
+ """
+ output = self.generator(img)
+ return output
+
+ def forward_test(self,
+ img,
+ img_metas,
+ return_vertices=False,
+ return_faces=False,
+ **kwargs):
+ """Defines the computation performed at every call when testing."""
+
+ pred_smpl = self.generator(img)
+ pred_pose, pred_beta, pred_camera = pred_smpl
+ pred_out = self.smpl(
+ betas=pred_beta,
+ body_pose=pred_pose[:, 1:],
+ global_orient=pred_pose[:, :1])
+ pred_vertices, pred_joints_3d = pred_out['vertices'], pred_out[
+ 'joints']
+
+ all_preds = {}
+ all_preds['keypoints_3d'] = pred_joints_3d.detach().cpu().numpy()
+ all_preds['smpl_pose'] = pred_pose.detach().cpu().numpy()
+ all_preds['smpl_beta'] = pred_beta.detach().cpu().numpy()
+ all_preds['camera'] = pred_camera.detach().cpu().numpy()
+
+ if return_vertices:
+ all_preds['vertices'] = pred_vertices.detach().cpu().numpy()
+ if return_faces:
+ all_preds['faces'] = self.smpl.get_faces()
+
+ all_boxes = []
+ image_path = []
+ for img_meta in img_metas:
+ box = np.zeros(6, dtype=np.float32)
+ c = img_meta['center']
+ s = img_meta['scale']
+ if 'bbox_score' in img_metas:
+ score = np.array(img_metas['bbox_score']).reshape(-1)
+ else:
+ score = 1.0
+ box[0:2] = c
+ box[2:4] = s
+ box[4] = np.prod(s * 200.0, axis=0)
+ box[5] = score
+ all_boxes.append(box)
+ image_path.append(img_meta['image_file'])
+
+ all_preds['bboxes'] = np.stack(all_boxes, axis=0)
+ all_preds['image_path'] = image_path
+ return all_preds
+
+ def get_3d_joints_from_mesh(self, vertices):
+ """Get 3D joints from 3D mesh using predefined joints regressor."""
+ return torch.matmul(
+ self.joints_regressor.to(vertices.device), vertices)
+
+ def forward(self, img, img_metas=None, return_loss=False, **kwargs):
+ """Forward function.
+
+ Calls either forward_train or forward_test depending on whether
+ return_loss=True.
+
+ Note:
+ - batch_size: N
+ - num_img_channel: C (Default: 3)
+ - img height: imgH
+ - img width: imgW
+
+ Args:
+ img (torch.Tensor[N x C x imgH x imgW]): Input images.
+ img_metas (list(dict)): Information about data augmentation
+ By default this includes:
+
+ - "image_file: path to the image file
+ - "center": center of the bbox
+ - "scale": scale of the bbox
+ - "rotation": rotation of the bbox
+ - "bbox_score": score of bbox
+ return_loss (bool): Option to `return loss`. `return loss=True`
+ for training, `return loss=False` for validation & test.
+
+ Returns:
+ Return predicted 3D joints, SMPL parameters, boxes and image paths.
+ """
+
+ if return_loss:
+ return self.forward_train(img, img_metas, **kwargs)
+ return self.forward_test(img, img_metas, **kwargs)
+
+ def show_result(self,
+ result,
+ img,
+ show=False,
+ out_file=None,
+ win_name='',
+ wait_time=0,
+ bbox_color='green',
+ mesh_color=(76, 76, 204),
+ **kwargs):
+ """Visualize 3D mesh estimation results.
+
+ Args:
+ result (list[dict]): The mesh estimation results containing:
+
+ - "bbox" (ndarray[4]): instance bounding bbox
+ - "center" (ndarray[2]): bbox center
+ - "scale" (ndarray[2]): bbox scale
+ - "keypoints_3d" (ndarray[K,3]): predicted 3D keypoints
+ - "camera" (ndarray[3]): camera parameters
+ - "vertices" (ndarray[V, 3]): predicted 3D vertices
+ - "faces" (ndarray[F, 3]): mesh faces
+ img (str or Tensor): Optional. The image to visualize 2D inputs on.
+ win_name (str): The window name.
+ show (bool): Whether to show the image. Default: False.
+ wait_time (int): Value of waitKey param. Default: 0.
+ out_file (str or None): The filename to write the image.
+ Default: None.
+ bbox_color (str or tuple or :obj:`Color`): Color of bbox lines.
+ mesh_color (str or tuple or :obj:`Color`): Color of mesh surface.
+
+ Returns:
+ ndarray: Visualized img, only if not `show` or `out_file`.
+ """
+
+ if img is not None:
+ img = mmcv.imread(img)
+
+ focal_length = self.loss_mesh.focal_length
+ H, W, C = img.shape
+ img_center = np.array([[0.5 * W], [0.5 * H]])
+
+ # show bounding boxes
+ bboxes = [res['bbox'] for res in result]
+ bboxes = np.vstack(bboxes)
+ mmcv.imshow_bboxes(
+ img, bboxes, colors=bbox_color, top_k=-1, thickness=2, show=False)
+
+ vertex_list = []
+ face_list = []
+ for res in result:
+ vertices = res['vertices']
+ faces = res['faces']
+ camera = res['camera']
+ camera_center = res['center']
+ scale = res['scale']
+
+ # predicted vertices are in root-relative space,
+ # we need to translate them to camera space.
+ translation = np.array([
+ camera[1], camera[2],
+ 2 * focal_length / (scale[0] * 200.0 * camera[0] + 1e-9)
+ ])
+ mean_depth = vertices[:, -1].mean() + translation[-1]
+ translation[:2] += (camera_center -
+ img_center[:, 0]) / focal_length * mean_depth
+ vertices += translation[None, :]
+
+ vertex_list.append(vertices)
+ face_list.append(faces)
+
+ # render from front view
+ img_vis = imshow_mesh_3d(
+ img,
+ vertex_list,
+ face_list,
+ img_center, [focal_length, focal_length],
+ colors=mesh_color)
+
+ # render from side view
+ # rotate mesh vertices
+ R = cv2.Rodrigues(np.array([0, np.radians(90.), 0]))[0]
+ rot_vertex_list = [np.dot(vert, R) for vert in vertex_list]
+
+ # get the 3D bbox containing all meshes
+ rot_vertices = np.concatenate(rot_vertex_list, axis=0)
+ min_corner = rot_vertices.min(0)
+ max_corner = rot_vertices.max(0)
+
+ center_3d = 0.5 * (min_corner + max_corner)
+ ratio = 0.8
+ bbox3d_size = max_corner - min_corner
+
+ # set appropriate translation to make all meshes appear in the image
+ z_x = bbox3d_size[0] * focal_length / (ratio * W) - min_corner[2]
+ z_y = bbox3d_size[1] * focal_length / (ratio * H) - min_corner[2]
+ z = max(z_x, z_y)
+ translation = -center_3d
+ translation[2] = z
+ translation = translation[None, :]
+ rot_vertex_list = [
+ rot_vert + translation for rot_vert in rot_vertex_list
+ ]
+
+ # render from side view
+ img_side = imshow_mesh_3d(
+ np.ones_like(img) * 255, rot_vertex_list, face_list, img_center,
+ [focal_length, focal_length])
+
+ # merger images from front view and side view
+ img_vis = np.concatenate([img_vis, img_side], axis=1)
+
+ if show:
+ mmcv.visualization.imshow(img_vis, win_name, wait_time)
+
+ if out_file is not None:
+ mmcv.imwrite(img_vis, out_file)
+
+ return img_vis
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/detectors/multi_task.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/detectors/multi_task.py
new file mode 100644
index 0000000..1b6f317
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/detectors/multi_task.py
@@ -0,0 +1,187 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import torch.nn as nn
+
+from .. import builder
+from ..builder import POSENETS
+
+
+@POSENETS.register_module()
+class MultiTask(nn.Module):
+ """Multi-task detectors.
+
+ Args:
+ backbone (dict): Backbone modules to extract feature.
+ heads (list[dict]): heads to output predictions.
+ necks (list[dict] | None): necks to process feature.
+ head2neck (dict{int:int}): head index to neck index.
+ pretrained (str): Path to the pretrained models.
+ """
+
+ def __init__(self,
+ backbone,
+ heads,
+ necks=None,
+ head2neck=None,
+ pretrained=None):
+ super().__init__()
+
+ self.backbone = builder.build_backbone(backbone)
+
+ if head2neck is None:
+ assert necks is None
+ head2neck = {}
+
+ self.head2neck = {}
+ for i in range(len(heads)):
+ self.head2neck[i] = head2neck[i] if i in head2neck else -1
+
+ self.necks = nn.ModuleList([])
+ if necks is not None:
+ for neck in necks:
+ self.necks.append(builder.build_neck(neck))
+ self.necks.append(nn.Identity())
+
+ self.heads = nn.ModuleList([])
+ assert heads is not None
+ for head in heads:
+ assert head is not None
+ self.heads.append(builder.build_head(head))
+
+ self.init_weights(pretrained=pretrained)
+
+ @property
+ def with_necks(self):
+ """Check if has keypoint_head."""
+ return hasattr(self, 'necks')
+
+ def init_weights(self, pretrained=None):
+ """Weight initialization for model."""
+ self.backbone.init_weights(pretrained)
+ if self.with_necks:
+ for neck in self.necks:
+ if hasattr(neck, 'init_weights'):
+ neck.init_weights()
+
+ for head in self.heads:
+ if hasattr(head, 'init_weights'):
+ head.init_weights()
+
+ def forward(self,
+ img,
+ target=None,
+ target_weight=None,
+ img_metas=None,
+ return_loss=True,
+ **kwargs):
+ """Calls either forward_train or forward_test depending on whether
+ return_loss=True. Note this setting will change the expected inputs.
+ When `return_loss=True`, img and img_meta are single-nested (i.e.
+ Tensor and List[dict]), and when `resturn_loss=False`, img and img_meta
+ should be double nested (i.e. List[Tensor], List[List[dict]]), with
+ the outer list indicating test time augmentations.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+ - num_img_channel: C (Default: 3)
+ - img height: imgH
+ - img weight: imgW
+ - heatmaps height: H
+ - heatmaps weight: W
+
+ Args:
+ img (torch.Tensor[N,C,imgH,imgW]): Input images.
+ target (list[torch.Tensor]): Targets.
+ target_weight (List[torch.Tensor]): Weights.
+ img_metas (list(dict)): Information about data augmentation
+ By default this includes:
+
+ - "image_file: path to the image file
+ - "center": center of the bbox
+ - "scale": scale of the bbox
+ - "rotation": rotation of the bbox
+ - "bbox_score": score of bbox
+ return_loss (bool): Option to `return loss`. `return loss=True`
+ for training, `return loss=False` for validation & test.
+
+ Returns:
+ dict|tuple: if `return loss` is true, then return losses. \
+ Otherwise, return predicted poses, boxes, image paths \
+ and heatmaps.
+ """
+ if return_loss:
+ return self.forward_train(img, target, target_weight, img_metas,
+ **kwargs)
+ return self.forward_test(img, img_metas, **kwargs)
+
+ def forward_train(self, img, target, target_weight, img_metas, **kwargs):
+ """Defines the computation performed at every call when training."""
+ features = self.backbone(img)
+ outputs = []
+
+ for head_id, head in enumerate(self.heads):
+ neck_id = self.head2neck[head_id]
+ outputs.append(head(self.necks[neck_id](features)))
+
+ # if return loss
+ losses = dict()
+
+ for head, output, gt, gt_weight in zip(self.heads, outputs, target,
+ target_weight):
+ loss = head.get_loss(output, gt, gt_weight)
+ assert len(set(losses.keys()).intersection(set(loss.keys()))) == 0
+ losses.update(loss)
+
+ if hasattr(head, 'get_accuracy'):
+ acc = head.get_accuracy(output, gt, gt_weight)
+ assert len(set(losses.keys()).intersection(set(
+ acc.keys()))) == 0
+ losses.update(acc)
+
+ return losses
+
+ def forward_test(self, img, img_metas, **kwargs):
+ """Defines the computation performed at every call when testing."""
+ assert img.size(0) == len(img_metas)
+ batch_size, _, img_height, img_width = img.shape
+ if batch_size > 1:
+ assert 'bbox_id' in img_metas[0]
+
+ results = {}
+
+ features = self.backbone(img)
+ outputs = []
+
+ for head_id, head in enumerate(self.heads):
+ neck_id = self.head2neck[head_id]
+ if hasattr(head, 'inference_model'):
+ head_output = head.inference_model(
+ self.necks[neck_id](features), flip_pairs=None)
+ else:
+ head_output = head(
+ self.necks[neck_id](features)).detach().cpu().numpy()
+ outputs.append(head_output)
+
+ for head, output in zip(self.heads, outputs):
+ result = head.decode(
+ img_metas, output, img_size=[img_width, img_height])
+ results.update(result)
+ return results
+
+ def forward_dummy(self, img):
+ """Used for computing network FLOPs.
+
+ See ``tools/get_flops.py``.
+
+ Args:
+ img (torch.Tensor): Input image.
+
+ Returns:
+ list[Tensor]: Outputs.
+ """
+ features = self.backbone(img)
+ outputs = []
+ for head_id, head in enumerate(self.heads):
+ neck_id = self.head2neck[head_id]
+ outputs.append(head(self.necks[neck_id](features)))
+ return outputs
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/detectors/multiview_pose.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/detectors/multiview_pose.py
new file mode 100644
index 0000000..c3d2221
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/detectors/multiview_pose.py
@@ -0,0 +1,889 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+from mmcv.runner import load_checkpoint
+
+from mmpose.core.camera import SimpleCameraTorch
+from mmpose.core.post_processing.post_transforms import (
+ affine_transform_torch, get_affine_transform)
+from .. import builder
+from ..builder import POSENETS
+from .base import BasePose
+
+
+class ProjectLayer(nn.Module):
+
+ def __init__(self, image_size, heatmap_size):
+ """Project layer to get voxel feature. Adapted from
+ https://github.com/microsoft/voxelpose-
+ pytorch/blob/main/lib/models/project_layer.py.
+
+ Args:
+ image_size (int or list): input size of the 2D model
+ heatmap_size (int or list): output size of the 2D model
+ """
+ super(ProjectLayer, self).__init__()
+ self.image_size = image_size
+ self.heatmap_size = heatmap_size
+ if isinstance(self.image_size, int):
+ self.image_size = [self.image_size, self.image_size]
+ if isinstance(self.heatmap_size, int):
+ self.heatmap_size = [self.heatmap_size, self.heatmap_size]
+
+ def compute_grid(self, box_size, box_center, num_bins, device=None):
+ if isinstance(box_size, int) or isinstance(box_size, float):
+ box_size = [box_size, box_size, box_size]
+ if isinstance(num_bins, int):
+ num_bins = [num_bins, num_bins, num_bins]
+
+ grid_1D_x = torch.linspace(
+ -box_size[0] / 2, box_size[0] / 2, num_bins[0], device=device)
+ grid_1D_y = torch.linspace(
+ -box_size[1] / 2, box_size[1] / 2, num_bins[1], device=device)
+ grid_1D_z = torch.linspace(
+ -box_size[2] / 2, box_size[2] / 2, num_bins[2], device=device)
+ grid_x, grid_y, grid_z = torch.meshgrid(
+ grid_1D_x + box_center[0],
+ grid_1D_y + box_center[1],
+ grid_1D_z + box_center[2],
+ )
+ grid_x = grid_x.contiguous().view(-1, 1)
+ grid_y = grid_y.contiguous().view(-1, 1)
+ grid_z = grid_z.contiguous().view(-1, 1)
+ grid = torch.cat([grid_x, grid_y, grid_z], dim=1)
+
+ return grid
+
+ def get_voxel(self, feature_maps, meta, grid_size, grid_center, cube_size):
+ device = feature_maps[0].device
+ batch_size = feature_maps[0].shape[0]
+ num_channels = feature_maps[0].shape[1]
+ num_bins = cube_size[0] * cube_size[1] * cube_size[2]
+ n = len(feature_maps)
+ cubes = torch.zeros(
+ batch_size, num_channels, 1, num_bins, n, device=device)
+ w, h = self.heatmap_size
+ grids = torch.zeros(batch_size, num_bins, 3, device=device)
+ bounding = torch.zeros(batch_size, 1, 1, num_bins, n, device=device)
+ for i in range(batch_size):
+ if len(grid_center[0]) == 3 or grid_center[i][3] >= 0:
+ if len(grid_center) == 1:
+ grid = self.compute_grid(
+ grid_size, grid_center[0], cube_size, device=device)
+ else:
+ grid = self.compute_grid(
+ grid_size, grid_center[i], cube_size, device=device)
+ grids[i:i + 1] = grid
+ for c in range(n):
+ center = meta[i]['center'][c]
+ scale = meta[i]['scale'][c]
+
+ width, height = center * 2
+ trans = torch.as_tensor(
+ get_affine_transform(center, scale / 200.0, 0,
+ self.image_size),
+ dtype=torch.float,
+ device=device)
+
+ cam_param = meta[i]['camera'][c].copy()
+
+ single_view_camera = SimpleCameraTorch(
+ param=cam_param, device=device)
+ xy = single_view_camera.world_to_pixel(grid)
+
+ bounding[i, 0, 0, :, c] = (xy[:, 0] >= 0) & (
+ xy[:, 1] >= 0) & (xy[:, 0] < width) & (
+ xy[:, 1] < height)
+ xy = torch.clamp(xy, -1.0, max(width, height))
+ xy = affine_transform_torch(xy, trans)
+ xy = xy * torch.tensor(
+ [w, h], dtype=torch.float,
+ device=device) / torch.tensor(
+ self.image_size, dtype=torch.float, device=device)
+ sample_grid = xy / torch.tensor([w - 1, h - 1],
+ dtype=torch.float,
+ device=device) * 2.0 - 1.0
+ sample_grid = torch.clamp(
+ sample_grid.view(1, 1, num_bins, 2), -1.1, 1.1)
+
+ cubes[i:i + 1, :, :, :, c] += F.grid_sample(
+ feature_maps[c][i:i + 1, :, :, :],
+ sample_grid,
+ align_corners=True)
+
+ cubes = torch.sum(
+ torch.mul(cubes, bounding), dim=-1) / (
+ torch.sum(bounding, dim=-1) + 1e-6)
+ cubes[cubes != cubes] = 0.0
+ cubes = cubes.clamp(0.0, 1.0)
+
+ cubes = cubes.view(batch_size, num_channels, cube_size[0],
+ cube_size[1], cube_size[2])
+ return cubes, grids
+
+ def forward(self, feature_maps, meta, grid_size, grid_center, cube_size):
+ cubes, grids = self.get_voxel(feature_maps, meta, grid_size,
+ grid_center, cube_size)
+ return cubes, grids
+
+
+@POSENETS.register_module()
+class DetectAndRegress(BasePose):
+ """DetectAndRegress approach for multiview human pose detection.
+
+ Args:
+ backbone (ConfigDict): Dictionary to construct the 2D pose detector
+ human_detector (ConfigDict): dictionary to construct human detector
+ pose_regressor (ConfigDict): dictionary to construct pose regressor
+ train_cfg (ConfigDict): Config for training. Default: None.
+ test_cfg (ConfigDict): Config for testing. Default: None.
+ pretrained (str): Path to the pretrained 2D model. Default: None.
+ freeze_2d (bool): Whether to freeze the 2D model in training.
+ Default: True.
+ """
+
+ def __init__(self,
+ backbone,
+ human_detector,
+ pose_regressor,
+ train_cfg=None,
+ test_cfg=None,
+ pretrained=None,
+ freeze_2d=True):
+ super(DetectAndRegress, self).__init__()
+ if backbone is not None:
+ self.backbone = builder.build_posenet(backbone)
+ if self.training and pretrained is not None:
+ load_checkpoint(self.backbone, pretrained)
+ else:
+ self.backbone = None
+
+ self.freeze_2d = freeze_2d
+ self.human_detector = builder.MODELS.build(human_detector)
+ self.pose_regressor = builder.MODELS.build(pose_regressor)
+
+ self.train_cfg = train_cfg
+ self.test_cfg = test_cfg
+
+ @staticmethod
+ def _freeze(model):
+ """Freeze parameters."""
+ model.eval()
+ for param in model.parameters():
+ param.requires_grad = False
+
+ def train(self, mode=True):
+ """Sets the module in training mode.
+ Args:
+ mode (bool): whether to set training mode (``True``)
+ or evaluation mode (``False``). Default: ``True``.
+
+ Returns:
+ Module: self
+ """
+ super().train(mode)
+ if mode and self.freeze_2d and self.backbone is not None:
+ self._freeze(self.backbone)
+
+ return self
+
+ def forward(self,
+ img=None,
+ img_metas=None,
+ return_loss=True,
+ targets=None,
+ masks=None,
+ targets_3d=None,
+ input_heatmaps=None,
+ **kwargs):
+ """
+ Note:
+ batch_size: N
+ num_keypoints: K
+ num_img_channel: C
+ img_width: imgW
+ img_height: imgH
+ feature_maps width: W
+ feature_maps height: H
+ volume_length: cubeL
+ volume_width: cubeW
+ volume_height: cubeH
+
+ Args:
+ img (list(torch.Tensor[NxCximgHximgW])):
+ Multi-camera input images to the 2D model.
+ img_metas (list(dict)):
+ Information about image, 3D groundtruth and camera parameters.
+ return_loss: Option to `return loss`. `return loss=True`
+ for training, `return loss=False` for validation & test.
+ targets (list(torch.Tensor[NxKxHxW])):
+ Multi-camera target feature_maps of the 2D model.
+ masks (list(torch.Tensor[NxHxW])):
+ Multi-camera masks of the input to the 2D model.
+ targets_3d (torch.Tensor[NxcubeLxcubeWxcubeH]):
+ Ground-truth 3D heatmap of human centers.
+ input_heatmaps (list(torch.Tensor[NxKxHxW])):
+ Multi-camera feature_maps when the 2D model is not available.
+ Default: None.
+ **kwargs:
+
+ Returns:
+ dict: if 'return_loss' is true, then return losses.
+ Otherwise, return predicted poses, human centers and sample_id
+
+ """
+ if return_loss:
+ return self.forward_train(img, img_metas, targets, masks,
+ targets_3d, input_heatmaps)
+ else:
+ return self.forward_test(img, img_metas, input_heatmaps)
+
+ def train_step(self, data_batch, optimizer, **kwargs):
+ """The iteration step during training.
+
+ This method defines an iteration step during training, except for the
+ back propagation and optimizer updating, which are done in an optimizer
+ hook. Note that in some complicated cases or models, the whole process
+ including back propagation and optimizer updating is also defined in
+ this method, such as GAN.
+
+ Args:
+ data_batch (dict): The output of dataloader.
+ optimizer (:obj:`torch.optim.Optimizer` | dict): The optimizer of
+ runner is passed to ``train_step()``. This argument is unused
+ and reserved.
+
+ Returns:
+ dict: It should contain at least 3 keys: ``loss``, ``log_vars``,
+ ``num_samples``.
+ ``loss`` is a tensor for back propagation, which can be a
+ weighted sum of multiple losses.
+ ``log_vars`` contains all the variables to be sent to the
+ logger.
+ ``num_samples`` indicates the batch size (when the model is
+ DDP, it means the batch size on each GPU), which is used for
+ averaging the logs.
+ """
+ losses = self.forward(**data_batch)
+
+ loss, log_vars = self._parse_losses(losses)
+ if 'img' in data_batch:
+ batch_size = data_batch['img'][0].shape[0]
+ else:
+ assert 'input_heatmaps' in data_batch
+ batch_size = data_batch['input_heatmaps'][0][0].shape[0]
+
+ outputs = dict(loss=loss, log_vars=log_vars, num_samples=batch_size)
+
+ return outputs
+
+ def forward_train(self,
+ img,
+ img_metas,
+ targets=None,
+ masks=None,
+ targets_3d=None,
+ input_heatmaps=None):
+ """
+ Note:
+ batch_size: N
+ num_keypoints: K
+ num_img_channel: C
+ img_width: imgW
+ img_height: imgH
+ feature_maps width: W
+ feature_maps height: H
+ volume_length: cubeL
+ volume_width: cubeW
+ volume_height: cubeH
+
+ Args:
+ img (list(torch.Tensor[NxCximgHximgW])):
+ Multi-camera input images to the 2D model.
+ img_metas (list(dict)):
+ Information about image, 3D groundtruth and camera parameters.
+ targets (list(torch.Tensor[NxKxHxW])):
+ Multi-camera target feature_maps of the 2D model.
+ masks (list(torch.Tensor[NxHxW])):
+ Multi-camera masks of the input to the 2D model.
+ targets_3d (torch.Tensor[NxcubeLxcubeWxcubeH]):
+ Ground-truth 3D heatmap of human centers.
+ input_heatmaps (list(torch.Tensor[NxKxHxW])):
+ Multi-camera feature_maps when the 2D model is not available.
+ Default: None.
+
+ Returns:
+ dict: losses.
+
+ """
+ if self.backbone is None:
+ assert input_heatmaps is not None
+ feature_maps = []
+ for input_heatmap in input_heatmaps:
+ feature_maps.append(input_heatmap[0])
+ else:
+ feature_maps = []
+ assert isinstance(img, list)
+ for img_ in img:
+ feature_maps.append(self.backbone.forward_dummy(img_)[0])
+
+ losses = dict()
+ human_candidates, human_loss = self.human_detector.forward_train(
+ None, img_metas, feature_maps, targets_3d, return_preds=True)
+ losses.update(human_loss)
+
+ pose_loss = self.pose_regressor(
+ None,
+ img_metas,
+ return_loss=True,
+ feature_maps=feature_maps,
+ human_candidates=human_candidates)
+ losses.update(pose_loss)
+
+ if not self.freeze_2d:
+ losses_2d = {}
+ heatmaps_tensor = torch.cat(feature_maps, dim=0)
+ targets_tensor = torch.cat(targets, dim=0)
+ masks_tensor = torch.cat(masks, dim=0)
+ losses_2d_ = self.backbone.get_loss(heatmaps_tensor,
+ targets_tensor, masks_tensor)
+ for k, v in losses_2d_.items():
+ losses_2d[k + '_2d'] = v
+ losses.update(losses_2d)
+
+ return losses
+
+ def forward_test(
+ self,
+ img,
+ img_metas,
+ input_heatmaps=None,
+ ):
+ """
+ Note:
+ batch_size: N
+ num_keypoints: K
+ num_img_channel: C
+ img_width: imgW
+ img_height: imgH
+ feature_maps width: W
+ feature_maps height: H
+ volume_length: cubeL
+ volume_width: cubeW
+ volume_height: cubeH
+
+ Args:
+ img (list(torch.Tensor[NxCximgHximgW])):
+ Multi-camera input images to the 2D model.
+ img_metas (list(dict)):
+ Information about image, 3D groundtruth and camera parameters.
+ input_heatmaps (list(torch.Tensor[NxKxHxW])):
+ Multi-camera feature_maps when the 2D model is not available.
+ Default: None.
+
+ Returns:
+ dict: predicted poses, human centers and sample_id
+
+ """
+ if self.backbone is None:
+ assert input_heatmaps is not None
+ feature_maps = []
+ for input_heatmap in input_heatmaps:
+ feature_maps.append(input_heatmap[0])
+ else:
+ feature_maps = []
+ assert isinstance(img, list)
+ for img_ in img:
+ feature_maps.append(self.backbone.forward_dummy(img_)[0])
+
+ human_candidates = self.human_detector.forward_test(
+ None, img_metas, feature_maps)
+
+ human_poses = self.pose_regressor(
+ None,
+ img_metas,
+ return_loss=False,
+ feature_maps=feature_maps,
+ human_candidates=human_candidates)
+
+ result = {}
+ result['pose_3d'] = human_poses.cpu().numpy()
+ result['human_detection_3d'] = human_candidates.cpu().numpy()
+ result['sample_id'] = [img_meta['sample_id'] for img_meta in img_metas]
+
+ return result
+
+ def show_result(self, **kwargs):
+ """Visualize the results."""
+ raise NotImplementedError
+
+ def forward_dummy(self, img, input_heatmaps=None, num_candidates=5):
+ """Used for computing network FLOPs."""
+ if self.backbone is None:
+ assert input_heatmaps is not None
+ feature_maps = []
+ for input_heatmap in input_heatmaps:
+ feature_maps.append(input_heatmap[0])
+ else:
+ feature_maps = []
+ assert isinstance(img, list)
+ for img_ in img:
+ feature_maps.append(self.backbone.forward_dummy(img_)[0])
+
+ _ = self.human_detector.forward_dummy(feature_maps)
+
+ _ = self.pose_regressor.forward_dummy(feature_maps, num_candidates)
+
+
+@POSENETS.register_module()
+class VoxelSinglePose(BasePose):
+ """VoxelPose Please refer to the `paper `
+ for details.
+
+ Args:
+ image_size (list): input size of the 2D model.
+ heatmap_size (list): output size of the 2D model.
+ sub_space_size (list): Size of the cuboid human proposal.
+ sub_cube_size (list): Size of the input volume to the pose net.
+ pose_net (ConfigDict): Dictionary to construct the pose net.
+ pose_head (ConfigDict): Dictionary to construct the pose head.
+ train_cfg (ConfigDict): Config for training. Default: None.
+ test_cfg (ConfigDict): Config for testing. Default: None.
+ """
+
+ def __init__(
+ self,
+ image_size,
+ heatmap_size,
+ sub_space_size,
+ sub_cube_size,
+ num_joints,
+ pose_net,
+ pose_head,
+ train_cfg=None,
+ test_cfg=None,
+ ):
+ super(VoxelSinglePose, self).__init__()
+ self.project_layer = ProjectLayer(image_size, heatmap_size)
+ self.pose_net = builder.build_backbone(pose_net)
+ self.pose_head = builder.build_head(pose_head)
+
+ self.sub_space_size = sub_space_size
+ self.sub_cube_size = sub_cube_size
+
+ self.num_joints = num_joints
+ self.train_cfg = train_cfg
+ self.test_cfg = test_cfg
+
+ def forward(self,
+ img,
+ img_metas,
+ return_loss=True,
+ feature_maps=None,
+ human_candidates=None,
+ **kwargs):
+ """
+ Note:
+ batch_size: N
+ num_keypoints: K
+ num_img_channel: C
+ img_width: imgW
+ img_height: imgH
+ feature_maps width: W
+ feature_maps height: H
+ volume_length: cubeL
+ volume_width: cubeW
+ volume_height: cubeH
+
+ Args:
+ img (list(torch.Tensor[NxCximgHximgW])):
+ Multi-camera input images to the 2D model.
+ feature_maps (list(torch.Tensor[NxCxHxW])):
+ Multi-camera input feature_maps.
+ img_metas (list(dict)):
+ Information about image, 3D groundtruth and camera parameters.
+ human_candidates (torch.Tensor[NxPx5]):
+ Human candidates.
+ return_loss: Option to `return loss`. `return loss=True`
+ for training, `return loss=False` for validation & test.
+
+ """
+ if return_loss:
+ return self.forward_train(img, img_metas, feature_maps,
+ human_candidates)
+ else:
+ return self.forward_test(img, img_metas, feature_maps,
+ human_candidates)
+
+ def forward_train(self,
+ img,
+ img_metas,
+ feature_maps=None,
+ human_candidates=None,
+ return_preds=False,
+ **kwargs):
+ """Defines the computation performed at training.
+ Note:
+ batch_size: N
+ num_keypoints: K
+ num_img_channel: C
+ img_width: imgW
+ img_height: imgH
+ feature_maps width: W
+ feature_maps height: H
+ volume_length: cubeL
+ volume_width: cubeW
+ volume_height: cubeH
+
+ Args:
+ img (list(torch.Tensor[NxCximgHximgW])):
+ Multi-camera input images to the 2D model.
+ feature_maps (list(torch.Tensor[NxCxHxW])):
+ Multi-camera input feature_maps.
+ img_metas (list(dict)):
+ Information about image, 3D groundtruth and camera parameters.
+ human_candidates (torch.Tensor[NxPx5]):
+ Human candidates.
+ return_preds (bool): Whether to return prediction results
+
+ Returns:
+ dict: losses.
+
+ """
+ batch_size, num_candidates, _ = human_candidates.shape
+ pred = human_candidates.new_zeros(batch_size, num_candidates,
+ self.num_joints, 5)
+ pred[:, :, :, 3:] = human_candidates[:, :, None, 3:]
+
+ device = feature_maps[0].device
+ gt_3d = torch.stack([
+ torch.tensor(img_meta['joints_3d'], device=device)
+ for img_meta in img_metas
+ ])
+ gt_3d_vis = torch.stack([
+ torch.tensor(img_meta['joints_3d_visible'], device=device)
+ for img_meta in img_metas
+ ])
+ valid_preds = []
+ valid_targets = []
+ valid_weights = []
+
+ for n in range(num_candidates):
+ index = pred[:, n, 0, 3] >= 0
+ num_valid = index.sum()
+ if num_valid > 0:
+ pose_input_cube, coordinates \
+ = self.project_layer(feature_maps,
+ img_metas,
+ self.sub_space_size,
+ human_candidates[:, n, :3],
+ self.sub_cube_size)
+ pose_heatmaps_3d = self.pose_net(pose_input_cube)
+ pose_3d = self.pose_head(pose_heatmaps_3d[index],
+ coordinates[index])
+
+ pred[index, n, :, 0:3] = pose_3d.detach()
+ valid_targets.append(gt_3d[index, pred[index, n, 0, 3].long()])
+ valid_weights.append(gt_3d_vis[index, pred[index, n, 0,
+ 3].long(), :,
+ 0:1].float())
+ valid_preds.append(pose_3d)
+
+ losses = dict()
+ if len(valid_preds) > 0:
+ valid_targets = torch.cat(valid_targets, dim=0)
+ valid_weights = torch.cat(valid_weights, dim=0)
+ valid_preds = torch.cat(valid_preds, dim=0)
+ losses.update(
+ self.pose_head.get_loss(valid_preds, valid_targets,
+ valid_weights))
+ else:
+ pose_input_cube = feature_maps[0].new_zeros(
+ batch_size, self.num_joints, *self.sub_cube_size)
+ coordinates = feature_maps[0].new_zeros(batch_size,
+ *self.sub_cube_size,
+ 3).view(batch_size, -1, 3)
+ pseudo_targets = feature_maps[0].new_zeros(batch_size,
+ self.num_joints, 3)
+ pseudo_weights = feature_maps[0].new_zeros(batch_size,
+ self.num_joints, 1)
+ pose_heatmaps_3d = self.pose_net(pose_input_cube)
+ pose_3d = self.pose_head(pose_heatmaps_3d, coordinates)
+ losses.update(
+ self.pose_head.get_loss(pose_3d, pseudo_targets,
+ pseudo_weights))
+ if return_preds:
+ return pred, losses
+ else:
+ return losses
+
+ def forward_test(self,
+ img,
+ img_metas,
+ feature_maps=None,
+ human_candidates=None,
+ **kwargs):
+ """Defines the computation performed at training.
+ Note:
+ batch_size: N
+ num_keypoints: K
+ num_img_channel: C
+ img_width: imgW
+ img_height: imgH
+ feature_maps width: W
+ feature_maps height: H
+ volume_length: cubeL
+ volume_width: cubeW
+ volume_height: cubeH
+
+ Args:
+ img (list(torch.Tensor[NxCximgHximgW])):
+ Multi-camera input images to the 2D model.
+ feature_maps (list(torch.Tensor[NxCxHxW])):
+ Multi-camera input feature_maps.
+ img_metas (list(dict)):
+ Information about image, 3D groundtruth and camera parameters.
+ human_candidates (torch.Tensor[NxPx5]):
+ Human candidates.
+
+ Returns:
+ dict: predicted poses, human centers and sample_id
+
+ """
+ batch_size, num_candidates, _ = human_candidates.shape
+ pred = human_candidates.new_zeros(batch_size, num_candidates,
+ self.num_joints, 5)
+ pred[:, :, :, 3:] = human_candidates[:, :, None, 3:]
+
+ for n in range(num_candidates):
+ index = pred[:, n, 0, 3] >= 0
+ num_valid = index.sum()
+ if num_valid > 0:
+ pose_input_cube, coordinates \
+ = self.project_layer(feature_maps,
+ img_metas,
+ self.sub_space_size,
+ human_candidates[:, n, :3],
+ self.sub_cube_size)
+ pose_heatmaps_3d = self.pose_net(pose_input_cube)
+ pose_3d = self.pose_head(pose_heatmaps_3d[index],
+ coordinates[index])
+
+ pred[index, n, :, 0:3] = pose_3d.detach()
+
+ return pred
+
+ def show_result(self, **kwargs):
+ """Visualize the results."""
+ raise NotImplementedError
+
+ def forward_dummy(self, feature_maps, num_candidates=5):
+ """Used for computing network FLOPs."""
+ batch_size, num_channels = feature_maps[0].shape
+ pose_input_cube = feature_maps[0].new_zeros(batch_size, num_channels,
+ *self.sub_cube_size)
+ for n in range(num_candidates):
+ _ = self.pose_net(pose_input_cube)
+
+
+@POSENETS.register_module()
+class VoxelCenterDetector(BasePose):
+ """Detect human center by 3D CNN on voxels.
+
+ Please refer to the
+ `paper ` for details.
+ Args:
+ image_size (list): input size of the 2D model.
+ heatmap_size (list): output size of the 2D model.
+ space_size (list): Size of the 3D space.
+ cube_size (list): Size of the input volume to the 3D CNN.
+ space_center (list): Coordinate of the center of the 3D space.
+ center_net (ConfigDict): Dictionary to construct the center net.
+ center_head (ConfigDict): Dictionary to construct the center head.
+ train_cfg (ConfigDict): Config for training. Default: None.
+ test_cfg (ConfigDict): Config for testing. Default: None.
+ """
+
+ def __init__(
+ self,
+ image_size,
+ heatmap_size,
+ space_size,
+ cube_size,
+ space_center,
+ center_net,
+ center_head,
+ train_cfg=None,
+ test_cfg=None,
+ ):
+ super(VoxelCenterDetector, self).__init__()
+ self.project_layer = ProjectLayer(image_size, heatmap_size)
+ self.center_net = builder.build_backbone(center_net)
+ self.center_head = builder.build_head(center_head)
+
+ self.space_size = space_size
+ self.cube_size = cube_size
+ self.space_center = space_center
+
+ self.train_cfg = train_cfg
+ self.test_cfg = test_cfg
+
+ def assign2gt(self, center_candidates, gt_centers, gt_num_persons):
+ """"Assign gt id to each valid human center candidate."""
+ det_centers = center_candidates[..., :3]
+ batch_size = center_candidates.shape[0]
+ cand_num = center_candidates.shape[1]
+ cand2gt = torch.zeros(batch_size, cand_num)
+
+ for i in range(batch_size):
+ cand = det_centers[i].view(cand_num, 1, -1)
+ gt = gt_centers[None, i, :gt_num_persons[i]]
+
+ dist = torch.sqrt(torch.sum((cand - gt)**2, dim=-1))
+ min_dist, min_gt = torch.min(dist, dim=-1)
+
+ cand2gt[i] = min_gt
+ cand2gt[i][min_dist > self.train_cfg['dist_threshold']] = -1.0
+
+ center_candidates[:, :, 3] = cand2gt
+
+ return center_candidates
+
+ def forward(self,
+ img,
+ img_metas,
+ return_loss=True,
+ feature_maps=None,
+ targets_3d=None):
+ """
+ Note:
+ batch_size: N
+ num_keypoints: K
+ num_img_channel: C
+ img_width: imgW
+ img_height: imgH
+ heatmaps width: W
+ heatmaps height: H
+ Args:
+ img (list(torch.Tensor[NxCximgHximgW])):
+ Multi-camera input images to the 2D model.
+ img_metas (list(dict)):
+ Information about image, 3D groundtruth and camera parameters.
+ return_loss: Option to `return loss`. `return loss=True`
+ for training, `return loss=False` for validation & test.
+ targets_3d (torch.Tensor[NxcubeLxcubeWxcubeH]):
+ Ground-truth 3D heatmap of human centers.
+ feature_maps (list(torch.Tensor[NxKxHxW])):
+ Multi-camera feature_maps.
+ Returns:
+ dict: if 'return_loss' is true, then return losses.
+ Otherwise, return predicted poses
+ """
+ if return_loss:
+ return self.forward_train(img, img_metas, feature_maps, targets_3d)
+ else:
+ return self.forward_test(img, img_metas, feature_maps)
+
+ def forward_train(self,
+ img,
+ img_metas,
+ feature_maps=None,
+ targets_3d=None,
+ return_preds=False):
+ """
+ Note:
+ batch_size: N
+ num_keypoints: K
+ num_img_channel: C
+ img_width: imgW
+ img_height: imgH
+ heatmaps width: W
+ heatmaps height: H
+ Args:
+ img (list(torch.Tensor[NxCximgHximgW])):
+ Multi-camera input images to the 2D model.
+ img_metas (list(dict)):
+ Information about image, 3D groundtruth and camera parameters.
+ targets_3d (torch.Tensor[NxcubeLxcubeWxcubeH]):
+ Ground-truth 3D heatmap of human centers.
+ feature_maps (list(torch.Tensor[NxKxHxW])):
+ Multi-camera feature_maps.
+ return_preds (bool): Whether to return prediction results
+ Returns:
+ dict: if 'return_pred' is true, then return losses
+ and human centers. Otherwise, return losses only
+ """
+ initial_cubes, _ = self.project_layer(feature_maps, img_metas,
+ self.space_size,
+ [self.space_center],
+ self.cube_size)
+ center_heatmaps_3d = self.center_net(initial_cubes)
+ center_heatmaps_3d = center_heatmaps_3d.squeeze(1)
+ center_candidates = self.center_head(center_heatmaps_3d)
+
+ device = center_candidates.device
+
+ gt_centers = torch.stack([
+ torch.tensor(img_meta['roots_3d'], device=device)
+ for img_meta in img_metas
+ ])
+ gt_num_persons = torch.stack([
+ torch.tensor(img_meta['num_persons'], device=device)
+ for img_meta in img_metas
+ ])
+ center_candidates = self.assign2gt(center_candidates, gt_centers,
+ gt_num_persons)
+
+ losses = dict()
+ losses.update(
+ self.center_head.get_loss(center_heatmaps_3d, targets_3d))
+
+ if return_preds:
+ return center_candidates, losses
+ else:
+ return losses
+
+ def forward_test(self, img, img_metas, feature_maps=None):
+ """
+ Note:
+ batch_size: N
+ num_keypoints: K
+ num_img_channel: C
+ img_width: imgW
+ img_height: imgH
+ heatmaps width: W
+ heatmaps height: H
+ Args:
+ img (list(torch.Tensor[NxCximgHximgW])):
+ Multi-camera input images to the 2D model.
+ img_metas (list(dict)):
+ Information about image, 3D groundtruth and camera parameters.
+ feature_maps (list(torch.Tensor[NxKxHxW])):
+ Multi-camera feature_maps.
+ Returns:
+ human centers
+ """
+ initial_cubes, _ = self.project_layer(feature_maps, img_metas,
+ self.space_size,
+ [self.space_center],
+ self.cube_size)
+ center_heatmaps_3d = self.center_net(initial_cubes)
+ center_heatmaps_3d = center_heatmaps_3d.squeeze(1)
+ center_candidates = self.center_head(center_heatmaps_3d)
+ center_candidates[..., 3] = \
+ (center_candidates[..., 4] >
+ self.test_cfg['center_threshold']).float() - 1.0
+
+ return center_candidates
+
+ def show_result(self, **kwargs):
+ """Visualize the results."""
+ raise NotImplementedError
+
+ def forward_dummy(self, feature_maps):
+ """Used for computing network FLOPs."""
+ batch_size, num_channels, _, _ = feature_maps[0].shape
+ initial_cubes = feature_maps[0].new_zeros(batch_size, num_channels,
+ *self.cube_size)
+ _ = self.center_net(initial_cubes)
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/detectors/pose_lifter.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/detectors/pose_lifter.py
new file mode 100644
index 0000000..ace6b9f
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/detectors/pose_lifter.py
@@ -0,0 +1,392 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import warnings
+
+import mmcv
+import numpy as np
+from mmcv.utils.misc import deprecated_api_warning
+
+from mmpose.core import imshow_bboxes, imshow_keypoints, imshow_keypoints_3d
+from .. import builder
+from ..builder import POSENETS
+from .base import BasePose
+
+try:
+ from mmcv.runner import auto_fp16
+except ImportError:
+ warnings.warn('auto_fp16 from mmpose will be deprecated from v0.15.0'
+ 'Please install mmcv>=1.1.4')
+ from mmpose.core import auto_fp16
+
+
+@POSENETS.register_module()
+class PoseLifter(BasePose):
+ """Pose lifter that lifts 2D pose to 3D pose.
+
+ The basic model is a pose model that predicts root-relative pose. If
+ traj_head is not None, a trajectory model that predicts absolute root joint
+ position is also built.
+
+ Args:
+ backbone (dict): Config for the backbone of pose model.
+ neck (dict|None): Config for the neck of pose model.
+ keypoint_head (dict|None): Config for the head of pose model.
+ traj_backbone (dict|None): Config for the backbone of trajectory model.
+ If traj_backbone is None and traj_head is not None, trajectory
+ model will share backbone with pose model.
+ traj_neck (dict|None): Config for the neck of trajectory model.
+ traj_head (dict|None): Config for the head of trajectory model.
+ loss_semi (dict|None): Config for semi-supervision loss.
+ train_cfg (dict|None): Config for keypoint head during training.
+ test_cfg (dict|None): Config for keypoint head during testing.
+ pretrained (str|None): Path to pretrained weights.
+ """
+
+ def __init__(self,
+ backbone,
+ neck=None,
+ keypoint_head=None,
+ traj_backbone=None,
+ traj_neck=None,
+ traj_head=None,
+ loss_semi=None,
+ train_cfg=None,
+ test_cfg=None,
+ pretrained=None):
+ super().__init__()
+ self.fp16_enabled = False
+
+ self.train_cfg = train_cfg
+ self.test_cfg = test_cfg
+
+ # pose model
+ self.backbone = builder.build_backbone(backbone)
+
+ if neck is not None:
+ self.neck = builder.build_neck(neck)
+
+ if keypoint_head is not None:
+ keypoint_head['train_cfg'] = train_cfg
+ keypoint_head['test_cfg'] = test_cfg
+ self.keypoint_head = builder.build_head(keypoint_head)
+
+ # trajectory model
+ if traj_head is not None:
+ self.traj_head = builder.build_head(traj_head)
+
+ if traj_backbone is not None:
+ self.traj_backbone = builder.build_backbone(traj_backbone)
+ else:
+ self.traj_backbone = self.backbone
+
+ if traj_neck is not None:
+ self.traj_neck = builder.build_neck(traj_neck)
+
+ # semi-supervised learning
+ self.semi = loss_semi is not None
+ if self.semi:
+ assert keypoint_head is not None and traj_head is not None
+ self.loss_semi = builder.build_loss(loss_semi)
+
+ self.init_weights(pretrained=pretrained)
+
+ @property
+ def with_neck(self):
+ """Check if has keypoint_neck."""
+ return hasattr(self, 'neck')
+
+ @property
+ def with_keypoint(self):
+ """Check if has keypoint_head."""
+ return hasattr(self, 'keypoint_head')
+
+ @property
+ def with_traj_backbone(self):
+ """Check if has trajectory_backbone."""
+ return hasattr(self, 'traj_backbone')
+
+ @property
+ def with_traj_neck(self):
+ """Check if has trajectory_neck."""
+ return hasattr(self, 'traj_neck')
+
+ @property
+ def with_traj(self):
+ """Check if has trajectory_head."""
+ return hasattr(self, 'traj_head')
+
+ @property
+ def causal(self):
+ if hasattr(self.backbone, 'causal'):
+ return self.backbone.causal
+ else:
+ raise AttributeError('A PoseLifter\'s backbone should have '
+ 'the bool attribute "causal" to indicate if'
+ 'it performs causal inference.')
+
+ def init_weights(self, pretrained=None):
+ """Weight initialization for model."""
+ self.backbone.init_weights(pretrained)
+ if self.with_neck:
+ self.neck.init_weights()
+ if self.with_keypoint:
+ self.keypoint_head.init_weights()
+ if self.with_traj_backbone:
+ self.traj_backbone.init_weights(pretrained)
+ if self.with_traj_neck:
+ self.traj_neck.init_weights()
+ if self.with_traj:
+ self.traj_head.init_weights()
+
+ @auto_fp16(apply_to=('input', ))
+ def forward(self,
+ input,
+ target=None,
+ target_weight=None,
+ metas=None,
+ return_loss=True,
+ **kwargs):
+ """Calls either forward_train or forward_test depending on whether
+ return_loss=True.
+
+ Note:
+ - batch_size: N
+ - num_input_keypoints: Ki
+ - input_keypoint_dim: Ci
+ - input_sequence_len: Ti
+ - num_output_keypoints: Ko
+ - output_keypoint_dim: Co
+ - input_sequence_len: To
+
+ Args:
+ input (torch.Tensor[NxKixCixTi]): Input keypoint coordinates.
+ target (torch.Tensor[NxKoxCoxTo]): Output keypoint coordinates.
+ Defaults to None.
+ target_weight (torch.Tensor[NxKox1]): Weights across different
+ joint types. Defaults to None.
+ metas (list(dict)): Information about data augmentation
+ return_loss (bool): Option to `return loss`. `return loss=True`
+ for training, `return loss=False` for validation & test.
+
+ Returns:
+ dict|Tensor: If `reutrn_loss` is true, return losses. \
+ Otherwise return predicted poses.
+ """
+ if return_loss:
+ return self.forward_train(input, target, target_weight, metas,
+ **kwargs)
+ else:
+ return self.forward_test(input, metas, **kwargs)
+
+ def forward_train(self, input, target, target_weight, metas, **kwargs):
+ """Defines the computation performed at every call when training."""
+ assert input.size(0) == len(metas)
+
+ # supervised learning
+ # pose model
+ features = self.backbone(input)
+ if self.with_neck:
+ features = self.neck(features)
+ if self.with_keypoint:
+ output = self.keypoint_head(features)
+
+ losses = dict()
+ if self.with_keypoint:
+ keypoint_losses = self.keypoint_head.get_loss(
+ output, target, target_weight)
+ keypoint_accuracy = self.keypoint_head.get_accuracy(
+ output, target, target_weight, metas)
+ losses.update(keypoint_losses)
+ losses.update(keypoint_accuracy)
+
+ # trajectory model
+ if self.with_traj:
+ traj_features = self.traj_backbone(input)
+ if self.with_traj_neck:
+ traj_features = self.traj_neck(traj_features)
+ traj_output = self.traj_head(traj_features)
+
+ traj_losses = self.traj_head.get_loss(traj_output,
+ kwargs['traj_target'], None)
+ losses.update(traj_losses)
+
+ # semi-supervised learning
+ if self.semi:
+ ul_input = kwargs['unlabeled_input']
+ ul_features = self.backbone(ul_input)
+ if self.with_neck:
+ ul_features = self.neck(ul_features)
+ ul_output = self.keypoint_head(ul_features)
+
+ ul_traj_features = self.traj_backbone(ul_input)
+ if self.with_traj_neck:
+ ul_traj_features = self.traj_neck(ul_traj_features)
+ ul_traj_output = self.traj_head(ul_traj_features)
+
+ output_semi = dict(
+ labeled_pose=output,
+ unlabeled_pose=ul_output,
+ unlabeled_traj=ul_traj_output)
+ target_semi = dict(
+ unlabeled_target_2d=kwargs['unlabeled_target_2d'],
+ intrinsics=kwargs['intrinsics'])
+
+ semi_losses = self.loss_semi(output_semi, target_semi)
+ losses.update(semi_losses)
+
+ return losses
+
+ def forward_test(self, input, metas, **kwargs):
+ """Defines the computation performed at every call when training."""
+ assert input.size(0) == len(metas)
+
+ results = {}
+
+ features = self.backbone(input)
+ if self.with_neck:
+ features = self.neck(features)
+ if self.with_keypoint:
+ output = self.keypoint_head.inference_model(features)
+ keypoint_result = self.keypoint_head.decode(metas, output)
+ results.update(keypoint_result)
+
+ if self.with_traj:
+ traj_features = self.traj_backbone(input)
+ if self.with_traj_neck:
+ traj_features = self.traj_neck(traj_features)
+ traj_output = self.traj_head.inference_model(traj_features)
+ results['traj_preds'] = traj_output
+
+ return results
+
+ def forward_dummy(self, input):
+ """Used for computing network FLOPs. See ``tools/get_flops.py``.
+
+ Args:
+ input (torch.Tensor): Input pose
+
+ Returns:
+ Tensor: Model output
+ """
+ output = self.backbone(input)
+ if self.with_neck:
+ output = self.neck(output)
+ if self.with_keypoint:
+ output = self.keypoint_head(output)
+
+ if self.with_traj:
+ traj_features = self.traj_backbone(input)
+ if self.with_neck:
+ traj_features = self.traj_neck(traj_features)
+ traj_output = self.traj_head(traj_features)
+ output = output + traj_output
+
+ return output
+
+ @deprecated_api_warning({'pose_limb_color': 'pose_link_color'},
+ cls_name='PoseLifter')
+ def show_result(self,
+ result,
+ img=None,
+ skeleton=None,
+ pose_kpt_color=None,
+ pose_link_color=None,
+ radius=8,
+ thickness=2,
+ vis_height=400,
+ num_instances=-1,
+ win_name='',
+ show=False,
+ wait_time=0,
+ out_file=None):
+ """Visualize 3D pose estimation results.
+
+ Args:
+ result (list[dict]): The pose estimation results containing:
+
+ - "keypoints_3d" ([K,4]): 3D keypoints
+ - "keypoints" ([K,3] or [T,K,3]): Optional for visualizing
+ 2D inputs. If a sequence is given, only the last frame
+ will be used for visualization
+ - "bbox" ([4,] or [T,4]): Optional for visualizing 2D inputs
+ - "title" (str): title for the subplot
+ img (str or Tensor): Optional. The image to visualize 2D inputs on.
+ skeleton (list of [idx_i,idx_j]): Skeleton described by a list of
+ links, each is a pair of joint indices.
+ pose_kpt_color (np.array[Nx3]`): Color of N keypoints.
+ If None, do not draw keypoints.
+ pose_link_color (np.array[Mx3]): Color of M links.
+ If None, do not draw links.
+ radius (int): Radius of circles.
+ thickness (int): Thickness of lines.
+ vis_height (int): The image height of the visualization. The width
+ will be N*vis_height depending on the number of visualized
+ items.
+ win_name (str): The window name.
+ wait_time (int): Value of waitKey param.
+ Default: 0.
+ out_file (str or None): The filename to write the image.
+ Default: None.
+
+ Returns:
+ Tensor: Visualized img, only if not `show` or `out_file`.
+ """
+ if num_instances < 0:
+ assert len(result) > 0
+ result = sorted(result, key=lambda x: x.get('track_id', 1e4))
+
+ # draw image and input 2d poses
+ if img is not None:
+ img = mmcv.imread(img)
+
+ bbox_result = []
+ pose_input_2d = []
+ for res in result:
+ if 'bbox' in res:
+ bbox = np.array(res['bbox'])
+ if bbox.ndim != 1:
+ assert bbox.ndim == 2
+ bbox = bbox[-1] # Get bbox from the last frame
+ bbox_result.append(bbox)
+ if 'keypoints' in res:
+ kpts = np.array(res['keypoints'])
+ if kpts.ndim != 2:
+ assert kpts.ndim == 3
+ kpts = kpts[-1] # Get 2D keypoints from the last frame
+ pose_input_2d.append(kpts)
+
+ if len(bbox_result) > 0:
+ bboxes = np.vstack(bbox_result)
+ imshow_bboxes(
+ img,
+ bboxes,
+ colors='green',
+ thickness=thickness,
+ show=False)
+ if len(pose_input_2d) > 0:
+ imshow_keypoints(
+ img,
+ pose_input_2d,
+ skeleton,
+ kpt_score_thr=0.3,
+ pose_kpt_color=pose_kpt_color,
+ pose_link_color=pose_link_color,
+ radius=radius,
+ thickness=thickness)
+ img = mmcv.imrescale(img, scale=vis_height / img.shape[0])
+
+ img_vis = imshow_keypoints_3d(
+ result,
+ img,
+ skeleton,
+ pose_kpt_color,
+ pose_link_color,
+ vis_height,
+ num_instances=num_instances)
+
+ if show:
+ mmcv.visualization.imshow(img_vis, win_name, wait_time)
+
+ if out_file is not None:
+ mmcv.imwrite(img_vis, out_file)
+
+ return img_vis
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/detectors/posewarper.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/detectors/posewarper.py
new file mode 100644
index 0000000..aa1d05f
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/detectors/posewarper.py
@@ -0,0 +1,244 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import warnings
+
+import numpy as np
+import torch
+
+from ..builder import POSENETS
+from .top_down import TopDown
+
+try:
+ from mmcv.runner import auto_fp16
+except ImportError:
+ warnings.warn('auto_fp16 from mmpose will be deprecated from v0.15.0'
+ 'Please install mmcv>=1.1.4')
+ from mmpose.core import auto_fp16
+
+
+@POSENETS.register_module()
+class PoseWarper(TopDown):
+ """Top-down pose detectors for multi-frame settings for video inputs.
+
+ `"Learning temporal pose estimation from sparsely-labeled videos"
+ `_.
+
+ A child class of TopDown detector. The main difference between PoseWarper
+ and TopDown lies in that the former takes a list of tensors as input image
+ while the latter takes a single tensor as input image in forward method.
+
+ Args:
+ backbone (dict): Backbone modules to extract features.
+ neck (dict): intermediate modules to transform features.
+ keypoint_head (dict): Keypoint head to process feature.
+ train_cfg (dict): Config for training. Default: None.
+ test_cfg (dict): Config for testing. Default: None.
+ pretrained (str): Path to the pretrained models.
+ loss_pose (None): Deprecated arguments. Please use
+ `loss_keypoint` for heads instead.
+ concat_tensors (bool): Whether to concat the tensors on the batch dim,
+ which can speed up, Default: True
+ """
+
+ def __init__(self,
+ backbone,
+ neck=None,
+ keypoint_head=None,
+ train_cfg=None,
+ test_cfg=None,
+ pretrained=None,
+ loss_pose=None,
+ concat_tensors=True):
+ super().__init__(
+ backbone=backbone,
+ neck=neck,
+ keypoint_head=keypoint_head,
+ train_cfg=train_cfg,
+ test_cfg=test_cfg,
+ pretrained=pretrained,
+ loss_pose=loss_pose)
+ self.concat_tensors = concat_tensors
+
+ @auto_fp16(apply_to=('img', ))
+ def forward(self,
+ img,
+ target=None,
+ target_weight=None,
+ img_metas=None,
+ return_loss=True,
+ return_heatmap=False,
+ **kwargs):
+ """Calls either forward_train or forward_test depending on whether
+ return_loss=True. Note this setting will change the expected inputs.
+ When `return_loss=True`, img and img_meta are single-nested (i.e.
+ Tensor and List[dict]), and when `resturn_loss=False`, img and img_meta
+ should be double nested (i.e. List[Tensor], List[List[dict]]), with
+ the outer list indicating test time augmentations.
+
+ Note:
+ - number of frames: F
+ - batch_size: N
+ - num_keypoints: K
+ - num_img_channel: C (Default: 3)
+ - img height: imgH
+ - img width: imgW
+ - heatmaps height: H
+ - heatmaps weight: W
+
+ Args:
+ imgs (list[F,torch.Tensor[N,C,imgH,imgW]]): multiple input frames
+ target (torch.Tensor[N,K,H,W]): Target heatmaps for one frame.
+ target_weight (torch.Tensor[N,K,1]): Weights across
+ different joint types.
+ img_metas (list(dict)): Information about data augmentation
+ By default this includes:
+
+ - "image_file: paths to multiple video frames
+ - "center": center of the bbox
+ - "scale": scale of the bbox
+ - "rotation": rotation of the bbox
+ - "bbox_score": score of bbox
+ return_loss (bool): Option to `return loss`. `return loss=True`
+ for training, `return loss=False` for validation & test.
+ return_heatmap (bool) : Option to return heatmap.
+
+ Returns:
+ dict|tuple: if `return loss` is true, then return losses. \
+ Otherwise, return predicted poses, boxes, image paths \
+ and heatmaps.
+ """
+ if return_loss:
+ return self.forward_train(img, target, target_weight, img_metas,
+ **kwargs)
+ return self.forward_test(
+ img, img_metas, return_heatmap=return_heatmap, **kwargs)
+
+ def forward_train(self, imgs, target, target_weight, img_metas, **kwargs):
+ """Defines the computation performed at every call when training."""
+ # imgs (list[Fxtorch.Tensor[NxCximgHximgW]]): multiple input frames
+ assert imgs[0].size(0) == len(img_metas)
+ num_frames = len(imgs)
+ frame_weight = img_metas[0]['frame_weight']
+
+ assert num_frames == len(frame_weight), f'The number of frames ' \
+ f'({num_frames}) and the length of weights for each frame ' \
+ f'({len(frame_weight)}) must match'
+
+ if self.concat_tensors:
+ features = [self.backbone(torch.cat(imgs, 0))]
+ else:
+ features = [self.backbone(img) for img in imgs]
+
+ if self.with_neck:
+ features = self.neck(features, frame_weight=frame_weight)
+
+ if self.with_keypoint:
+ output = self.keypoint_head(features)
+
+ # if return loss
+ losses = dict()
+ if self.with_keypoint:
+ keypoint_losses = self.keypoint_head.get_loss(
+ output, target, target_weight)
+ losses.update(keypoint_losses)
+ keypoint_accuracy = self.keypoint_head.get_accuracy(
+ output, target, target_weight)
+ losses.update(keypoint_accuracy)
+
+ return losses
+
+ def forward_test(self, imgs, img_metas, return_heatmap=False, **kwargs):
+ """Defines the computation performed at every call when testing."""
+ # imgs (list[Fxtorch.Tensor[NxCximgHximgW]]): multiple input frames
+ assert imgs[0].size(0) == len(img_metas)
+ num_frames = len(imgs)
+ frame_weight = img_metas[0]['frame_weight']
+
+ assert num_frames == len(frame_weight), f'The number of frames ' \
+ f'({num_frames}) and the length of weights for each frame ' \
+ f'({len(frame_weight)}) must match'
+
+ batch_size, _, img_height, img_width = imgs[0].shape
+
+ if batch_size > 1:
+ assert 'bbox_id' in img_metas[0]
+
+ result = {}
+
+ if self.concat_tensors:
+ features = [self.backbone(torch.cat(imgs, 0))]
+ else:
+ features = [self.backbone(img) for img in imgs]
+
+ if self.with_neck:
+ features = self.neck(features, frame_weight=frame_weight)
+
+ if self.with_keypoint:
+ output_heatmap = self.keypoint_head.inference_model(
+ features, flip_pairs=None)
+
+ if self.test_cfg.get('flip_test', True):
+ imgs_flipped = [img.flip(3) for img in imgs]
+
+ if self.concat_tensors:
+ features_flipped = [self.backbone(torch.cat(imgs_flipped, 0))]
+ else:
+ features_flipped = [
+ self.backbone(img_flipped) for img_flipped in imgs_flipped
+ ]
+
+ if self.with_neck:
+ features_flipped = self.neck(
+ features_flipped, frame_weight=frame_weight)
+
+ if self.with_keypoint:
+ output_flipped_heatmap = self.keypoint_head.inference_model(
+ features_flipped, img_metas[0]['flip_pairs'])
+ output_heatmap = (output_heatmap +
+ output_flipped_heatmap) * 0.5
+
+ if self.with_keypoint:
+ keypoint_result = self.keypoint_head.decode(
+ img_metas, output_heatmap, img_size=[img_width, img_height])
+ result.update(keypoint_result)
+
+ if not return_heatmap:
+ output_heatmap = None
+
+ result['output_heatmap'] = output_heatmap
+
+ return result
+
+ def forward_dummy(self, img):
+ """Used for computing network FLOPs.
+
+ See ``tools/get_flops.py``.
+
+ Args:
+ img (torch.Tensor[N,C,imgH,imgW], or list|tuple of tensors):
+ multiple input frames, N >= 2.
+
+ Returns:
+ Tensor: Output heatmaps.
+ """
+ # concat tensors if they are in a list
+ if isinstance(img, (list, tuple)):
+ img = torch.cat(img, 0)
+
+ batch_size = img.size(0)
+ assert batch_size > 1, 'Input batch size to PoseWarper ' \
+ 'should be larger than 1.'
+ if batch_size == 2:
+ warnings.warn('Current batch size: 2, for pytorch2onnx and '
+ 'getting flops both.')
+ else:
+ warnings.warn(
+ f'Current batch size: {batch_size}, for getting flops only.')
+
+ frame_weight = np.random.uniform(0, 1, batch_size)
+ output = [self.backbone(img)]
+
+ if self.with_neck:
+ output = self.neck(output, frame_weight=frame_weight)
+ if self.with_keypoint:
+ output = self.keypoint_head(output)
+ return output
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/detectors/top_down.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/detectors/top_down.py
new file mode 100644
index 0000000..af0ab51
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/detectors/top_down.py
@@ -0,0 +1,307 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import warnings
+
+import mmcv
+import numpy as np
+from mmcv.image import imwrite
+from mmcv.utils.misc import deprecated_api_warning
+from mmcv.visualization.image import imshow
+
+from mmpose.core import imshow_bboxes, imshow_keypoints
+from .. import builder
+from ..builder import POSENETS
+from .base import BasePose
+
+try:
+ from mmcv.runner import auto_fp16
+except ImportError:
+ warnings.warn('auto_fp16 from mmpose will be deprecated from v0.15.0'
+ 'Please install mmcv>=1.1.4')
+ from mmpose.core import auto_fp16
+
+
+@POSENETS.register_module()
+class TopDown(BasePose):
+ """Top-down pose detectors.
+
+ Args:
+ backbone (dict): Backbone modules to extract feature.
+ keypoint_head (dict): Keypoint head to process feature.
+ train_cfg (dict): Config for training. Default: None.
+ test_cfg (dict): Config for testing. Default: None.
+ pretrained (str): Path to the pretrained models.
+ loss_pose (None): Deprecated arguments. Please use
+ `loss_keypoint` for heads instead.
+ """
+
+ def __init__(self,
+ backbone,
+ neck=None,
+ keypoint_head=None,
+ train_cfg=None,
+ test_cfg=None,
+ pretrained=None,
+ loss_pose=None):
+ super().__init__()
+ self.fp16_enabled = False
+
+ self.backbone = builder.build_backbone(backbone)
+
+ self.train_cfg = train_cfg
+ self.test_cfg = test_cfg
+
+ if neck is not None:
+ self.neck = builder.build_neck(neck)
+
+ if keypoint_head is not None:
+ keypoint_head['train_cfg'] = train_cfg
+ keypoint_head['test_cfg'] = test_cfg
+
+ if 'loss_keypoint' not in keypoint_head and loss_pose is not None:
+ warnings.warn(
+ '`loss_pose` for TopDown is deprecated, '
+ 'use `loss_keypoint` for heads instead. See '
+ 'https://github.com/open-mmlab/mmpose/pull/382'
+ ' for more information.', DeprecationWarning)
+ keypoint_head['loss_keypoint'] = loss_pose
+
+ self.keypoint_head = builder.build_head(keypoint_head)
+
+ self.init_weights(pretrained=pretrained)
+
+ @property
+ def with_neck(self):
+ """Check if has neck."""
+ return hasattr(self, 'neck')
+
+ @property
+ def with_keypoint(self):
+ """Check if has keypoint_head."""
+ return hasattr(self, 'keypoint_head')
+
+ def init_weights(self, pretrained=None):
+ """Weight initialization for model."""
+ self.backbone.init_weights(pretrained)
+ if self.with_neck:
+ self.neck.init_weights()
+ if self.with_keypoint:
+ self.keypoint_head.init_weights()
+
+ @auto_fp16(apply_to=('img', ))
+ def forward(self,
+ img,
+ target=None,
+ target_weight=None,
+ img_metas=None,
+ return_loss=True,
+ return_heatmap=False,
+ **kwargs):
+ """Calls either forward_train or forward_test depending on whether
+ return_loss=True. Note this setting will change the expected inputs.
+ When `return_loss=True`, img and img_meta are single-nested (i.e.
+ Tensor and List[dict]), and when `resturn_loss=False`, img and img_meta
+ should be double nested (i.e. List[Tensor], List[List[dict]]), with
+ the outer list indicating test time augmentations.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+ - num_img_channel: C (Default: 3)
+ - img height: imgH
+ - img width: imgW
+ - heatmaps height: H
+ - heatmaps weight: W
+
+ Args:
+ img (torch.Tensor[NxCximgHximgW]): Input images.
+ target (torch.Tensor[NxKxHxW]): Target heatmaps.
+ target_weight (torch.Tensor[NxKx1]): Weights across
+ different joint types.
+ img_metas (list(dict)): Information about data augmentation
+ By default this includes:
+
+ - "image_file: path to the image file
+ - "center": center of the bbox
+ - "scale": scale of the bbox
+ - "rotation": rotation of the bbox
+ - "bbox_score": score of bbox
+ return_loss (bool): Option to `return loss`. `return loss=True`
+ for training, `return loss=False` for validation & test.
+ return_heatmap (bool) : Option to return heatmap.
+
+ Returns:
+ dict|tuple: if `return loss` is true, then return losses. \
+ Otherwise, return predicted poses, boxes, image paths \
+ and heatmaps.
+ """
+ if return_loss:
+ return self.forward_train(img, target, target_weight, img_metas,
+ **kwargs)
+ return self.forward_test(
+ img, img_metas, return_heatmap=return_heatmap, **kwargs)
+
+ def forward_train(self, img, target, target_weight, img_metas, **kwargs):
+ """Defines the computation performed at every call when training."""
+ output = self.backbone(img)
+ if self.with_neck:
+ output = self.neck(output)
+ if self.with_keypoint:
+ output = self.keypoint_head(output)
+
+ # if return loss
+ losses = dict()
+ if self.with_keypoint:
+ keypoint_losses = self.keypoint_head.get_loss(
+ output, target, target_weight)
+ losses.update(keypoint_losses)
+ keypoint_accuracy = self.keypoint_head.get_accuracy(
+ output, target, target_weight)
+ losses.update(keypoint_accuracy)
+
+ return losses
+
+ def forward_test(self, img, img_metas, return_heatmap=False, **kwargs):
+ """Defines the computation performed at every call when testing."""
+ assert img.size(0) == len(img_metas)
+ batch_size, _, img_height, img_width = img.shape
+ if batch_size > 1:
+ assert 'bbox_id' in img_metas[0]
+
+ result = {}
+
+ features = self.backbone(img)
+ if self.with_neck:
+ features = self.neck(features)
+ if self.with_keypoint:
+ output_heatmap = self.keypoint_head.inference_model(
+ features, flip_pairs=None)
+
+ if self.test_cfg.get('flip_test', True):
+ img_flipped = img.flip(3)
+ features_flipped = self.backbone(img_flipped)
+ if self.with_neck:
+ features_flipped = self.neck(features_flipped)
+ if self.with_keypoint:
+ output_flipped_heatmap = self.keypoint_head.inference_model(
+ features_flipped, img_metas[0]['flip_pairs'])
+ output_heatmap = (output_heatmap +
+ output_flipped_heatmap) * 0.5
+
+ if self.with_keypoint:
+ keypoint_result = self.keypoint_head.decode(
+ img_metas, output_heatmap, img_size=[img_width, img_height])
+ result.update(keypoint_result)
+
+ if not return_heatmap:
+ output_heatmap = None
+
+ result['output_heatmap'] = output_heatmap
+
+ return result
+
+ def forward_dummy(self, img):
+ """Used for computing network FLOPs.
+
+ See ``tools/get_flops.py``.
+
+ Args:
+ img (torch.Tensor): Input image.
+
+ Returns:
+ Tensor: Output heatmaps.
+ """
+ output = self.backbone(img)
+ if self.with_neck:
+ output = self.neck(output)
+ if self.with_keypoint:
+ output = self.keypoint_head(output)
+ return output
+
+ @deprecated_api_warning({'pose_limb_color': 'pose_link_color'},
+ cls_name='TopDown')
+ def show_result(self,
+ img,
+ result,
+ skeleton=None,
+ kpt_score_thr=0.3,
+ bbox_color='green',
+ pose_kpt_color=None,
+ pose_link_color=None,
+ text_color='white',
+ radius=4,
+ thickness=1,
+ font_scale=0.5,
+ bbox_thickness=1,
+ win_name='',
+ show=False,
+ show_keypoint_weight=False,
+ wait_time=0,
+ out_file=None):
+ """Draw `result` over `img`.
+
+ Args:
+ img (str or Tensor): The image to be displayed.
+ result (list[dict]): The results to draw over `img`
+ (bbox_result, pose_result).
+ skeleton (list[list]): The connection of keypoints.
+ skeleton is 0-based indexing.
+ kpt_score_thr (float, optional): Minimum score of keypoints
+ to be shown. Default: 0.3.
+ bbox_color (str or tuple or :obj:`Color`): Color of bbox lines.
+ pose_kpt_color (np.array[Nx3]`): Color of N keypoints.
+ If None, do not draw keypoints.
+ pose_link_color (np.array[Mx3]): Color of M links.
+ If None, do not draw links.
+ text_color (str or tuple or :obj:`Color`): Color of texts.
+ radius (int): Radius of circles.
+ thickness (int): Thickness of lines.
+ font_scale (float): Font scales of texts.
+ win_name (str): The window name.
+ show (bool): Whether to show the image. Default: False.
+ show_keypoint_weight (bool): Whether to change the transparency
+ using the predicted confidence scores of keypoints.
+ wait_time (int): Value of waitKey param.
+ Default: 0.
+ out_file (str or None): The filename to write the image.
+ Default: None.
+
+ Returns:
+ Tensor: Visualized img, only if not `show` or `out_file`.
+ """
+ img = mmcv.imread(img)
+ img = img.copy()
+
+ bbox_result = []
+ bbox_labels = []
+ pose_result = []
+ for res in result:
+ if 'bbox' in res:
+ bbox_result.append(res['bbox'])
+ bbox_labels.append(res.get('label', None))
+ pose_result.append(res['keypoints'])
+
+ if bbox_result:
+ bboxes = np.vstack(bbox_result)
+ # draw bounding boxes
+ imshow_bboxes(
+ img,
+ bboxes,
+ labels=bbox_labels,
+ colors=bbox_color,
+ text_color=text_color,
+ thickness=bbox_thickness,
+ font_scale=font_scale,
+ show=False)
+
+ if pose_result:
+ imshow_keypoints(img, pose_result, skeleton, kpt_score_thr,
+ pose_kpt_color, pose_link_color, radius,
+ thickness)
+
+ if show:
+ imshow(img, win_name, wait_time)
+
+ if out_file is not None:
+ imwrite(img, out_file)
+
+ return img
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/detectors/top_down_coco_plus.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/detectors/top_down_coco_plus.py
new file mode 100644
index 0000000..47b1077
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/detectors/top_down_coco_plus.py
@@ -0,0 +1,359 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import warnings
+
+import mmcv
+import numpy as np
+from mmcv.image import imwrite
+from mmcv.utils.misc import deprecated_api_warning
+from mmcv.visualization.image import imshow
+
+from mmpose.core import imshow_bboxes, imshow_keypoints
+from .. import builder
+from ..builder import POSENETS
+from .base import BasePose
+import torch
+try:
+ from mmcv.runner import auto_fp16
+except ImportError:
+ warnings.warn('auto_fp16 from mmpose will be deprecated from v0.15.0'
+ 'Please install mmcv>=1.1.4')
+ from mmpose.core import auto_fp16
+
+
+@POSENETS.register_module()
+class TopDownCoCoPlus(BasePose):
+ """Top-down pose detectors.
+
+ Args:
+ backbone (dict): Backbone modules to extract feature.
+ keypoint_head (dict): Keypoint head to process feature.
+ train_cfg (dict): Config for training. Default: None.
+ test_cfg (dict): Config for testing. Default: None.
+ pretrained (str): Path to the pretrained models.
+ loss_pose (None): Deprecated arguments. Please use
+ `loss_keypoint` for heads instead.
+ """
+
+ def __init__(self,
+ backbone,
+ neck=None,
+ keypoint_head=None,
+ extend_keypoint_head=None,
+ train_cfg=None,
+ test_cfg=None,
+ pretrained=None,
+ loss_pose=None):
+ super().__init__()
+ self.fp16_enabled = False
+
+ self.backbone = builder.build_backbone(backbone)
+
+ self.train_cfg = train_cfg
+ self.test_cfg = test_cfg
+
+ if neck is not None:
+ self.neck = builder.build_neck(neck)
+
+ if keypoint_head is not None:
+ keypoint_head['train_cfg'] = train_cfg
+ keypoint_head['test_cfg'] = test_cfg
+
+ if 'loss_keypoint' not in keypoint_head and loss_pose is not None:
+ warnings.warn(
+ '`loss_pose` for TopDown is deprecated, '
+ 'use `loss_keypoint` for heads instead. See '
+ 'https://github.com/open-mmlab/mmpose/pull/382'
+ ' for more information.', DeprecationWarning)
+ keypoint_head['loss_keypoint'] = loss_pose
+
+ self.keypoint_head = builder.build_head(keypoint_head)
+
+ if extend_keypoint_head is not None:
+ extend_keypoint_head['train_cfg'] = train_cfg
+ extend_keypoint_head['test_cfg'] = test_cfg
+
+ if 'loss_keypoint' not in extend_keypoint_head and loss_pose is not None:
+ warnings.warn(
+ '`loss_pose` for TopDown is deprecated, '
+ 'use `loss_keypoint` for heads instead. See '
+ 'https://github.com/open-mmlab/mmpose/pull/382'
+ ' for more information.', DeprecationWarning)
+ extend_keypoint_head['loss_keypoint'] = loss_pose
+
+ self.extend_keypoint_head = builder.build_head(extend_keypoint_head)
+
+ self.init_weights(pretrained=pretrained)
+
+ @property
+ def with_neck(self):
+ """Check if has neck."""
+ return hasattr(self, 'neck')
+
+ @property
+ def with_keypoint(self):
+ """Check if has keypoint_head."""
+ return hasattr(self, 'keypoint_head')
+ def with_extend_keypoint(self):
+ """Check if has keypoint_head."""
+ return hasattr(self, 'extend_keypoint_head')
+
+ def init_weights(self, pretrained=None):
+ """Weight initialization for model."""
+ self.backbone.init_weights(pretrained)
+ if self.with_neck:
+ self.neck.init_weights()
+ if self.with_keypoint:
+ self.keypoint_head.init_weights()
+ if self.with_extend_keypoint:
+ self.extend_keypoint_head.init_weights()
+
+ @auto_fp16(apply_to=('img', ))
+ def forward(self,
+ img,
+ target=None,
+ target_weight=None,
+ img_metas=None,
+ return_loss=True,
+ return_heatmap=False,
+ **kwargs):
+ """Calls either forward_train or forward_test depending on whether
+ return_loss=True. Note this setting will change the expected inputs.
+ When `return_loss=True`, img and img_meta are single-nested (i.e.
+ Tensor and List[dict]), and when `resturn_loss=False`, img and img_meta
+ should be double nested (i.e. List[Tensor], List[List[dict]]), with
+ the outer list indicating test time augmentations.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+ - num_img_channel: C (Default: 3)
+ - img height: imgH
+ - img width: imgW
+ - heatmaps height: H
+ - heatmaps weight: W
+
+ Args:
+ img (torch.Tensor[NxCximgHximgW]): Input images.
+ target (torch.Tensor[NxKxHxW]): Target heatmaps.
+ target_weight (torch.Tensor[NxKx1]): Weights across
+ different joint types.
+ img_metas (list(dict)): Information about data augmentation
+ By default this includes:
+
+ - "image_file: path to the image file
+ - "center": center of the bbox
+ - "scale": scale of the bbox
+ - "rotation": rotation of the bbox
+ - "bbox_score": score of bbox
+ return_loss (bool): Option to `return loss`. `return loss=True`
+ for training, `return loss=False` for validation & test.
+ return_heatmap (bool) : Option to return heatmap.
+
+ Returns:
+ dict|tuple: if `return loss` is true, then return losses. \
+ Otherwise, return predicted poses, boxes, image paths \
+ and heatmaps.
+ """
+ if return_loss:
+ return self.forward_train(img, target, target_weight, img_metas,
+ **kwargs)
+ return self.forward_test(
+ img, img_metas, return_heatmap=return_heatmap, **kwargs)
+
+ def forward_train(self, img, target, target_weight, img_metas, **kwargs):
+ """Defines the computation performed at every call when training."""
+ with torch.no_grad():
+ feat = self.backbone(img)
+ if self.with_neck:
+ feat = self.neck(feat)
+ if self.with_keypoint:
+ output = self.keypoint_head(feat)
+ if self.with_extend_keypoint:
+ extend_output = self.extend_keypoint_head(feat)
+
+ with torch.no_grad():
+ # if return loss
+ losses = dict()
+ if self.with_keypoint:
+ keypoint_losses = self.keypoint_head.get_loss(
+ output, target[:,:17], target_weight[:,:17])
+ losses.update(keypoint_losses)
+ keypoint_accuracy = self.keypoint_head.get_accuracy(
+ output, target[:,:17], target_weight[:,:17])
+ losses.update(keypoint_accuracy)
+
+ if self.with_extend_keypoint:
+ extend_keypoint_losses = self.extend_keypoint_head.get_loss(
+ extend_output, target[:,17:], target_weight[:,17:])
+ losses.update(extend_keypoint_losses)
+ extend_keypoint_accuracy = self.extend_keypoint_head.get_accuracy(
+ extend_output, target[:,17:], target_weight[:,17:])
+ losses.update(extend_keypoint_accuracy)
+
+ return losses
+
+ def forward_test(self, img, img_metas, return_heatmap=False, **kwargs):
+ """Defines the computation performed at every call when testing."""
+ assert img.size(0) == len(img_metas)
+ batch_size, _, img_height, img_width = img.shape
+ if batch_size > 1:
+ assert 'bbox_id' in img_metas[0]
+
+ result = {}
+
+ features = self.backbone(img)
+ if self.with_neck:
+ features = self.neck(features)
+ if self.with_keypoint:
+ output_heatmap = self.keypoint_head.inference_model(
+ features, flip_pairs=None)
+ if self.with_extend_keypoint:
+ extend_output_heatmap = self.extend_keypoint_head.inference_model(
+ features, flip_pairs=None)
+
+ if self.test_cfg.get('flip_test', True):
+ img_flipped = img.flip(3)
+ features_flipped = self.backbone(img_flipped)
+ if self.with_neck:
+ features_flipped = self.neck(features_flipped)
+ if self.with_keypoint:
+ output_flipped_heatmap = self.keypoint_head.inference_model(
+ features_flipped, img_metas[0]['flip_pairs'][:8])
+ output_heatmap = (output_heatmap +
+ output_flipped_heatmap) * 0.5
+ if self.with_extend_keypoint:
+ flip_pairs = img_metas[0]['flip_pairs'][8:]
+
+ new_flip_pairs = [[item - 17 for item in flip_pair] for flip_pair in flip_pairs]
+ extend_output_flipped_heatmap = self.extend_keypoint_head.inference_model(
+ features_flipped, new_flip_pairs)
+ extend_output_heatmap = (extend_output_heatmap +
+ extend_output_flipped_heatmap) * 0.5
+
+ if self.with_keypoint:
+ keypoint_result = self.keypoint_head.decode(
+ img_metas, output_heatmap, img_size=[img_width, img_height])
+
+ if self.with_extend_keypoint:
+ extend_keypoint_result = self.extend_keypoint_head.decode(
+ img_metas, extend_output_heatmap, img_size=[img_width, img_height])
+ keypoint_result['preds'] = np.concatenate((keypoint_result['preds'][:,:17,:],
+ extend_keypoint_result['preds']), axis = 1)
+
+ result.update(keypoint_result)
+
+ if not return_heatmap:
+ output_heatmap = None
+
+ result['output_heatmap'] = output_heatmap
+
+ return result
+
+ def forward_dummy(self, img):
+ """Used for computing network FLOPs.
+
+ See ``tools/get_flops.py``.
+
+ Args:
+ img (torch.Tensor): Input image.
+
+ Returns:
+ Tensor: Output heatmaps.
+ """
+ feat = self.backbone(img)
+ if self.with_neck:
+ feat = self.neck(feat)
+ if self.with_keypoint:
+ output = self.keypoint_head(feat)
+ if self.with_extend_keypoint:
+ output = self.extend_keypoint_head(feat)
+ return output
+
+ @deprecated_api_warning({'pose_limb_color': 'pose_link_color'},
+ cls_name='TopDown')
+ def show_result(self,
+ img,
+ result,
+ skeleton=None,
+ kpt_score_thr=0.3,
+ bbox_color='green',
+ pose_kpt_color=None,
+ pose_link_color=None,
+ text_color='white',
+ radius=4,
+ thickness=1,
+ font_scale=0.5,
+ bbox_thickness=1,
+ win_name='',
+ show=False,
+ show_keypoint_weight=False,
+ wait_time=0,
+ out_file=None):
+ """Draw `result` over `img`.
+
+ Args:
+ img (str or Tensor): The image to be displayed.
+ result (list[dict]): The results to draw over `img`
+ (bbox_result, pose_result).
+ skeleton (list[list]): The connection of keypoints.
+ skeleton is 0-based indexing.
+ kpt_score_thr (float, optional): Minimum score of keypoints
+ to be shown. Default: 0.3.
+ bbox_color (str or tuple or :obj:`Color`): Color of bbox lines.
+ pose_kpt_color (np.array[Nx3]`): Color of N keypoints.
+ If None, do not draw keypoints.
+ pose_link_color (np.array[Mx3]): Color of M links.
+ If None, do not draw links.
+ text_color (str or tuple or :obj:`Color`): Color of texts.
+ radius (int): Radius of circles.
+ thickness (int): Thickness of lines.
+ font_scale (float): Font scales of texts.
+ win_name (str): The window name.
+ show (bool): Whether to show the image. Default: False.
+ show_keypoint_weight (bool): Whether to change the transparency
+ using the predicted confidence scores of keypoints.
+ wait_time (int): Value of waitKey param.
+ Default: 0.
+ out_file (str or None): The filename to write the image.
+ Default: None.
+
+ Returns:
+ Tensor: Visualized img, only if not `show` or `out_file`.
+ """
+ img = mmcv.imread(img)
+ img = img.copy()
+
+ bbox_result = []
+ bbox_labels = []
+ pose_result = []
+ for res in result:
+ if 'bbox' in res:
+ bbox_result.append(res['bbox'])
+ bbox_labels.append(res.get('label', None))
+ pose_result.append(res['keypoints'])
+
+ if bbox_result:
+ bboxes = np.vstack(bbox_result)
+ # draw bounding boxes
+ imshow_bboxes(
+ img,
+ bboxes,
+ labels=bbox_labels,
+ colors=bbox_color,
+ text_color=text_color,
+ thickness=bbox_thickness,
+ font_scale=font_scale,
+ show=False)
+
+ if pose_result:
+ imshow_keypoints(img, pose_result, skeleton, kpt_score_thr,
+ pose_kpt_color, pose_link_color, radius,
+ thickness)
+
+ if show:
+ imshow(img, win_name, wait_time)
+
+ if out_file is not None:
+ imwrite(img, out_file)
+
+ return img
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/detectors/top_down_moe.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/detectors/top_down_moe.py
new file mode 100644
index 0000000..7d499b7
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/detectors/top_down_moe.py
@@ -0,0 +1,351 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import warnings
+
+import torch
+import torch.nn as nn
+
+import mmcv
+import numpy as np
+from mmcv.image import imwrite
+from mmcv.utils.misc import deprecated_api_warning
+from mmcv.visualization.image import imshow
+
+from mmpose.core import imshow_bboxes, imshow_keypoints
+from .. import builder
+from ..builder import POSENETS
+from .base import BasePose
+
+try:
+ from mmcv.runner import auto_fp16
+except ImportError:
+ warnings.warn('auto_fp16 from mmpose will be deprecated from v0.15.0'
+ 'Please install mmcv>=1.1.4')
+ from mmpose.core import auto_fp16
+
+
+@POSENETS.register_module()
+class TopDownMoE(BasePose):
+ """Top-down pose detectors.
+
+ Args:
+ backbone (dict): Backbone modules to extract feature.
+ keypoint_head (dict): Keypoint head to process feature.
+ train_cfg (dict): Config for training. Default: None.
+ test_cfg (dict): Config for testing. Default: None.
+ pretrained (str): Path to the pretrained models.
+ loss_pose (None): Deprecated arguments. Please use
+ `loss_keypoint` for heads instead.
+ """
+
+ def __init__(self,
+ backbone,
+ neck=None,
+ keypoint_head=None,
+ associate_keypoint_head=None,
+ train_cfg=None,
+ test_cfg=None,
+ pretrained=None,
+ loss_pose=None):
+ super().__init__()
+ self.fp16_enabled = False
+
+ self.backbone = builder.build_backbone(backbone)
+
+ self.train_cfg = train_cfg
+ self.test_cfg = test_cfg
+
+ if neck is not None:
+ self.neck = builder.build_neck(neck)
+
+ if keypoint_head is not None:
+ keypoint_head['train_cfg'] = train_cfg
+ keypoint_head['test_cfg'] = test_cfg
+
+ if 'loss_keypoint' not in keypoint_head and loss_pose is not None:
+ warnings.warn(
+ '`loss_pose` for TopDown is deprecated, '
+ 'use `loss_keypoint` for heads instead. See '
+ 'https://github.com/open-mmlab/mmpose/pull/382'
+ ' for more information.', DeprecationWarning)
+ keypoint_head['loss_keypoint'] = loss_pose
+
+ self.keypoint_head = builder.build_head(keypoint_head)
+
+
+ associate_keypoint_heads = []
+ keypoint_heads_cnt = 1
+
+ if associate_keypoint_head is not None:
+ if not isinstance(associate_keypoint_head, list):
+ associate_keypoint_head = [associate_keypoint_head]
+ for single_keypoint_head in associate_keypoint_head:
+ single_keypoint_head['train_cfg'] = train_cfg
+ single_keypoint_head['test_cfg'] = test_cfg
+ associate_keypoint_heads.append(builder.build_head(single_keypoint_head))
+ keypoint_heads_cnt += 1
+
+ self.associate_keypoint_heads = nn.ModuleList(associate_keypoint_heads)
+
+ self.keypoint_heads_cnt = keypoint_heads_cnt
+
+ self.init_weights(pretrained=pretrained)
+
+ @property
+ def with_neck(self):
+ """Check if has neck."""
+ return hasattr(self, 'neck')
+
+ @property
+ def with_keypoint(self):
+ """Check if has keypoint_head."""
+ return hasattr(self, 'keypoint_head')
+
+ def init_weights(self, pretrained=None):
+ """Weight initialization for model."""
+ self.backbone.init_weights(pretrained)
+ if self.with_neck:
+ self.neck.init_weights()
+ if self.with_keypoint:
+ self.keypoint_head.init_weights()
+ for item in self.associate_keypoint_heads:
+ item.init_weights()
+
+ @auto_fp16(apply_to=('img', ))
+ def forward(self,
+ img,
+ target=None,
+ target_weight=None,
+ img_metas=None,
+ return_loss=True,
+ return_heatmap=False,
+ **kwargs):
+ """Calls either forward_train or forward_test depending on whether
+ return_loss=True. Note this setting will change the expected inputs.
+ When `return_loss=True`, img and img_meta are single-nested (i.e.
+ Tensor and List[dict]), and when `resturn_loss=False`, img and img_meta
+ should be double nested (i.e. List[Tensor], List[List[dict]]), with
+ the outer list indicating test time augmentations.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+ - num_img_channel: C (Default: 3)
+ - img height: imgH
+ - img width: imgW
+ - heatmaps height: H
+ - heatmaps weight: W
+
+ Args:
+ img (torch.Tensor[NxCximgHximgW]): Input images.
+ target (torch.Tensor[NxKxHxW]): Target heatmaps.
+ target_weight (torch.Tensor[NxKx1]): Weights across
+ different joint types.
+ img_metas (list(dict)): Information about data augmentation
+ By default this includes:
+
+ - "image_file: path to the image file
+ - "center": center of the bbox
+ - "scale": scale of the bbox
+ - "rotation": rotation of the bbox
+ - "bbox_score": score of bbox
+ return_loss (bool): Option to `return loss`. `return loss=True`
+ for training, `return loss=False` for validation & test.
+ return_heatmap (bool) : Option to return heatmap.
+
+ Returns:
+ dict|tuple: if `return loss` is true, then return losses. \
+ Otherwise, return predicted poses, boxes, image paths \
+ and heatmaps.
+ """
+ if return_loss:
+ return self.forward_train(img, target, target_weight, img_metas,
+ **kwargs)
+ return self.forward_test(
+ img, img_metas, return_heatmap=return_heatmap, **kwargs)
+
+ def forward_train(self, img, target, target_weight, img_metas, **kwargs):
+ """Defines the computation performed at every call when training."""
+
+ img_sources = torch.from_numpy(np.array([ele['dataset_idx'] for ele in img_metas])).to(img.device)
+
+ output = self.backbone(img, img_sources)
+ if self.with_neck:
+ output = self.neck(output)
+ # if return loss
+ losses = dict()
+
+ main_stream_select = (img_sources == 0)
+ # if torch.sum(main_stream_select) > 0:
+ output_select = self.keypoint_head(output)
+
+ target_select = target * main_stream_select.view(-1, 1, 1, 1)
+ target_weight_select = target_weight * main_stream_select.view(-1, 1, 1)
+
+ keypoint_losses = self.keypoint_head.get_loss(
+ output_select, target_select, target_weight_select)
+ losses['main_stream_loss'] = keypoint_losses['heatmap_loss']
+ keypoint_accuracy = self.keypoint_head.get_accuracy(
+ output_select, target_select, target_weight_select)
+ losses['main_stream_acc'] = keypoint_accuracy['acc_pose']
+
+ for idx in range(1, self.keypoint_heads_cnt):
+ idx_select = (img_sources == idx)
+ target_select = target * idx_select.view(-1, 1, 1, 1)
+ target_weight_select = target_weight * idx_select.view(-1, 1, 1)
+ output_select = self.associate_keypoint_heads[idx - 1](output)
+ keypoint_losses = self.associate_keypoint_heads[idx - 1].get_loss(
+ output_select, target_select, target_weight_select)
+ losses[f'{idx}_loss'] = keypoint_losses['heatmap_loss']
+ keypoint_accuracy = self.associate_keypoint_heads[idx - 1].get_accuracy(
+ output_select, target_select, target_weight_select)
+ losses[f'{idx}_acc'] = keypoint_accuracy['acc_pose']
+
+ return losses
+
+ def forward_test(self, img, img_metas, return_heatmap=False, **kwargs):
+ """Defines the computation performed at every call when testing."""
+ assert img.size(0) == len(img_metas)
+ batch_size, _, img_height, img_width = img.shape
+ if batch_size > 1:
+ assert 'bbox_id' in img_metas[0]
+
+ result = {}
+ img_sources = torch.from_numpy(np.array([ele['dataset_idx'] for ele in img_metas])).to(img.device)
+
+ features = self.backbone(img, img_sources)
+
+ if self.with_neck:
+ features = self.neck(features)
+ if self.with_keypoint:
+ output_heatmap = self.keypoint_head.inference_model(
+ features, flip_pairs=None)
+
+ if self.test_cfg.get('flip_test', True):
+ img_flipped = img.flip(3)
+ features_flipped = self.backbone(img_flipped, img_sources)
+ if self.with_neck:
+ features_flipped = self.neck(features_flipped)
+ if self.with_keypoint:
+ output_flipped_heatmap = self.keypoint_head.inference_model(
+ features_flipped, img_metas[0]['flip_pairs'])
+ output_heatmap = (output_heatmap +
+ output_flipped_heatmap) * 0.5
+
+ if self.with_keypoint:
+ keypoint_result = self.keypoint_head.decode(
+ img_metas, output_heatmap, img_size=[img_width, img_height])
+ result.update(keypoint_result)
+
+ if not return_heatmap:
+ output_heatmap = None
+
+ result['output_heatmap'] = output_heatmap
+
+ return result
+
+ def forward_dummy(self, img):
+ """Used for computing network FLOPs.
+
+ See ``tools/get_flops.py``.
+
+ Args:
+ img (torch.Tensor): Input image.
+
+ Returns:
+ Tensor: Output heatmaps.
+ """
+ output = self.backbone(img)
+ if self.with_neck:
+ output = self.neck(output)
+ if self.with_keypoint:
+ output = self.keypoint_head(output)
+ return output
+
+ @deprecated_api_warning({'pose_limb_color': 'pose_link_color'},
+ cls_name='TopDown')
+ def show_result(self,
+ img,
+ result,
+ skeleton=None,
+ kpt_score_thr=0.3,
+ bbox_color='green',
+ pose_kpt_color=None,
+ pose_link_color=None,
+ text_color='white',
+ radius=4,
+ thickness=1,
+ font_scale=0.5,
+ bbox_thickness=1,
+ win_name='',
+ show=False,
+ show_keypoint_weight=False,
+ wait_time=0,
+ out_file=None):
+ """Draw `result` over `img`.
+
+ Args:
+ img (str or Tensor): The image to be displayed.
+ result (list[dict]): The results to draw over `img`
+ (bbox_result, pose_result).
+ skeleton (list[list]): The connection of keypoints.
+ skeleton is 0-based indexing.
+ kpt_score_thr (float, optional): Minimum score of keypoints
+ to be shown. Default: 0.3.
+ bbox_color (str or tuple or :obj:`Color`): Color of bbox lines.
+ pose_kpt_color (np.array[Nx3]`): Color of N keypoints.
+ If None, do not draw keypoints.
+ pose_link_color (np.array[Mx3]): Color of M links.
+ If None, do not draw links.
+ text_color (str or tuple or :obj:`Color`): Color of texts.
+ radius (int): Radius of circles.
+ thickness (int): Thickness of lines.
+ font_scale (float): Font scales of texts.
+ win_name (str): The window name.
+ show (bool): Whether to show the image. Default: False.
+ show_keypoint_weight (bool): Whether to change the transparency
+ using the predicted confidence scores of keypoints.
+ wait_time (int): Value of waitKey param.
+ Default: 0.
+ out_file (str or None): The filename to write the image.
+ Default: None.
+
+ Returns:
+ Tensor: Visualized img, only if not `show` or `out_file`.
+ """
+ img = mmcv.imread(img)
+ img = img.copy()
+
+ bbox_result = []
+ bbox_labels = []
+ pose_result = []
+ for res in result:
+ if 'bbox' in res:
+ bbox_result.append(res['bbox'])
+ bbox_labels.append(res.get('label', None))
+ pose_result.append(res['keypoints'])
+
+ if bbox_result:
+ bboxes = np.vstack(bbox_result)
+ # draw bounding boxes
+ imshow_bboxes(
+ img,
+ bboxes,
+ labels=bbox_labels,
+ colors=bbox_color,
+ text_color=text_color,
+ thickness=bbox_thickness,
+ font_scale=font_scale,
+ show=False)
+
+ if pose_result:
+ imshow_keypoints(img, pose_result, skeleton, kpt_score_thr,
+ pose_kpt_color, pose_link_color, radius,
+ thickness)
+
+ if show:
+ imshow(img, win_name, wait_time)
+
+ if out_file is not None:
+ imwrite(img, out_file)
+
+ return img
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/heads/__init__.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/heads/__init__.py
new file mode 100644
index 0000000..a98e911
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/heads/__init__.py
@@ -0,0 +1,24 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from .ae_higher_resolution_head import AEHigherResolutionHead
+from .ae_multi_stage_head import AEMultiStageHead
+from .ae_simple_head import AESimpleHead
+from .deconv_head import DeconvHead
+from .deeppose_regression_head import DeepposeRegressionHead
+from .hmr_head import HMRMeshHead
+from .interhand_3d_head import Interhand3DHead
+from .temporal_regression_head import TemporalRegressionHead
+from .topdown_heatmap_base_head import TopdownHeatmapBaseHead
+from .topdown_heatmap_multi_stage_head import (TopdownHeatmapMSMUHead,
+ TopdownHeatmapMultiStageHead)
+from .topdown_heatmap_simple_head import TopdownHeatmapSimpleHead
+from .vipnas_heatmap_simple_head import ViPNASHeatmapSimpleHead
+from .voxelpose_head import CuboidCenterHead, CuboidPoseHead
+
+__all__ = [
+ 'TopdownHeatmapSimpleHead', 'TopdownHeatmapMultiStageHead',
+ 'TopdownHeatmapMSMUHead', 'TopdownHeatmapBaseHead',
+ 'AEHigherResolutionHead', 'AESimpleHead', 'AEMultiStageHead',
+ 'DeepposeRegressionHead', 'TemporalRegressionHead', 'Interhand3DHead',
+ 'HMRMeshHead', 'DeconvHead', 'ViPNASHeatmapSimpleHead', 'CuboidCenterHead',
+ 'CuboidPoseHead'
+]
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/heads/ae_higher_resolution_head.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/heads/ae_higher_resolution_head.py
new file mode 100644
index 0000000..9bf3399
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/heads/ae_higher_resolution_head.py
@@ -0,0 +1,249 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import torch
+import torch.nn as nn
+from mmcv.cnn import (build_conv_layer, build_upsample_layer, constant_init,
+ normal_init)
+
+from mmpose.models.builder import build_loss
+from ..backbones.resnet import BasicBlock
+from ..builder import HEADS
+
+
+@HEADS.register_module()
+class AEHigherResolutionHead(nn.Module):
+ """Associative embedding with higher resolution head. paper ref: Bowen
+ Cheng et al. "HigherHRNet: Scale-Aware Representation Learning for Bottom-
+ Up Human Pose Estimation".
+
+ Args:
+ in_channels (int): Number of input channels.
+ num_joints (int): Number of joints
+ tag_per_joint (bool): If tag_per_joint is True,
+ the dimension of tags equals to num_joints,
+ else the dimension of tags is 1. Default: True
+ extra (dict): Configs for extra conv layers. Default: None
+ num_deconv_layers (int): Number of deconv layers.
+ num_deconv_layers should >= 0. Note that 0 means
+ no deconv layers.
+ num_deconv_filters (list|tuple): Number of filters.
+ If num_deconv_layers > 0, the length of
+ num_deconv_kernels (list|tuple): Kernel sizes.
+ cat_output (list[bool]): Option to concat outputs.
+ with_ae_loss (list[bool]): Option to use ae loss.
+ loss_keypoint (dict): Config for loss. Default: None.
+ """
+
+ def __init__(self,
+ in_channels,
+ num_joints,
+ tag_per_joint=True,
+ extra=None,
+ num_deconv_layers=1,
+ num_deconv_filters=(32, ),
+ num_deconv_kernels=(4, ),
+ num_basic_blocks=4,
+ cat_output=None,
+ with_ae_loss=None,
+ loss_keypoint=None):
+ super().__init__()
+
+ self.loss = build_loss(loss_keypoint)
+ dim_tag = num_joints if tag_per_joint else 1
+
+ self.num_deconvs = num_deconv_layers
+ self.cat_output = cat_output
+
+ final_layer_output_channels = []
+
+ if with_ae_loss[0]:
+ out_channels = num_joints + dim_tag
+ else:
+ out_channels = num_joints
+
+ final_layer_output_channels.append(out_channels)
+ for i in range(num_deconv_layers):
+ if with_ae_loss[i + 1]:
+ out_channels = num_joints + dim_tag
+ else:
+ out_channels = num_joints
+ final_layer_output_channels.append(out_channels)
+
+ deconv_layer_output_channels = []
+ for i in range(num_deconv_layers):
+ if with_ae_loss[i]:
+ out_channels = num_joints + dim_tag
+ else:
+ out_channels = num_joints
+ deconv_layer_output_channels.append(out_channels)
+
+ self.final_layers = self._make_final_layers(
+ in_channels, final_layer_output_channels, extra, num_deconv_layers,
+ num_deconv_filters)
+ self.deconv_layers = self._make_deconv_layers(
+ in_channels, deconv_layer_output_channels, num_deconv_layers,
+ num_deconv_filters, num_deconv_kernels, num_basic_blocks,
+ cat_output)
+
+ @staticmethod
+ def _make_final_layers(in_channels, final_layer_output_channels, extra,
+ num_deconv_layers, num_deconv_filters):
+ """Make final layers."""
+ if extra is not None and 'final_conv_kernel' in extra:
+ assert extra['final_conv_kernel'] in [1, 3]
+ if extra['final_conv_kernel'] == 3:
+ padding = 1
+ else:
+ padding = 0
+ kernel_size = extra['final_conv_kernel']
+ else:
+ kernel_size = 1
+ padding = 0
+
+ final_layers = []
+ final_layers.append(
+ build_conv_layer(
+ cfg=dict(type='Conv2d'),
+ in_channels=in_channels,
+ out_channels=final_layer_output_channels[0],
+ kernel_size=kernel_size,
+ stride=1,
+ padding=padding))
+
+ for i in range(num_deconv_layers):
+ in_channels = num_deconv_filters[i]
+ final_layers.append(
+ build_conv_layer(
+ cfg=dict(type='Conv2d'),
+ in_channels=in_channels,
+ out_channels=final_layer_output_channels[i + 1],
+ kernel_size=kernel_size,
+ stride=1,
+ padding=padding))
+
+ return nn.ModuleList(final_layers)
+
+ def _make_deconv_layers(self, in_channels, deconv_layer_output_channels,
+ num_deconv_layers, num_deconv_filters,
+ num_deconv_kernels, num_basic_blocks, cat_output):
+ """Make deconv layers."""
+ deconv_layers = []
+ for i in range(num_deconv_layers):
+ if cat_output[i]:
+ in_channels += deconv_layer_output_channels[i]
+
+ planes = num_deconv_filters[i]
+ deconv_kernel, padding, output_padding = \
+ self._get_deconv_cfg(num_deconv_kernels[i])
+
+ layers = []
+ layers.append(
+ nn.Sequential(
+ build_upsample_layer(
+ dict(type='deconv'),
+ in_channels=in_channels,
+ out_channels=planes,
+ kernel_size=deconv_kernel,
+ stride=2,
+ padding=padding,
+ output_padding=output_padding,
+ bias=False), nn.BatchNorm2d(planes, momentum=0.1),
+ nn.ReLU(inplace=True)))
+ for _ in range(num_basic_blocks):
+ layers.append(nn.Sequential(BasicBlock(planes, planes), ))
+ deconv_layers.append(nn.Sequential(*layers))
+ in_channels = planes
+
+ return nn.ModuleList(deconv_layers)
+
+ @staticmethod
+ def _get_deconv_cfg(deconv_kernel):
+ """Get configurations for deconv layers."""
+ if deconv_kernel == 4:
+ padding = 1
+ output_padding = 0
+ elif deconv_kernel == 3:
+ padding = 1
+ output_padding = 1
+ elif deconv_kernel == 2:
+ padding = 0
+ output_padding = 0
+ else:
+ raise ValueError(f'Not supported num_kernels ({deconv_kernel}).')
+
+ return deconv_kernel, padding, output_padding
+
+ def get_loss(self, outputs, targets, masks, joints):
+ """Calculate bottom-up keypoint loss.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+ - num_outputs: O
+ - heatmaps height: H
+ - heatmaps weight: W
+
+ Args:
+ outputs (list(torch.Tensor[N,K,H,W])): Multi-scale output heatmaps.
+ targets (List(torch.Tensor[N,K,H,W])): Multi-scale target heatmaps.
+ masks (List(torch.Tensor[N,H,W])): Masks of multi-scale target
+ heatmaps
+ joints (List(torch.Tensor[N,M,K,2])): Joints of multi-scale target
+ heatmaps for ae loss
+ """
+
+ losses = dict()
+
+ heatmaps_losses, push_losses, pull_losses = self.loss(
+ outputs, targets, masks, joints)
+
+ for idx in range(len(targets)):
+ if heatmaps_losses[idx] is not None:
+ heatmaps_loss = heatmaps_losses[idx].mean(dim=0)
+ if 'heatmap_loss' not in losses:
+ losses['heatmap_loss'] = heatmaps_loss
+ else:
+ losses['heatmap_loss'] += heatmaps_loss
+ if push_losses[idx] is not None:
+ push_loss = push_losses[idx].mean(dim=0)
+ if 'push_loss' not in losses:
+ losses['push_loss'] = push_loss
+ else:
+ losses['push_loss'] += push_loss
+ if pull_losses[idx] is not None:
+ pull_loss = pull_losses[idx].mean(dim=0)
+ if 'pull_loss' not in losses:
+ losses['pull_loss'] = pull_loss
+ else:
+ losses['pull_loss'] += pull_loss
+
+ return losses
+
+ def forward(self, x):
+ """Forward function."""
+ if isinstance(x, list):
+ x = x[0]
+
+ final_outputs = []
+ y = self.final_layers[0](x)
+ final_outputs.append(y)
+
+ for i in range(self.num_deconvs):
+ if self.cat_output[i]:
+ x = torch.cat((x, y), 1)
+
+ x = self.deconv_layers[i](x)
+ y = self.final_layers[i + 1](x)
+ final_outputs.append(y)
+
+ return final_outputs
+
+ def init_weights(self):
+ """Initialize model weights."""
+ for _, m in self.deconv_layers.named_modules():
+ if isinstance(m, nn.ConvTranspose2d):
+ normal_init(m, std=0.001)
+ elif isinstance(m, nn.BatchNorm2d):
+ constant_init(m, 1)
+ for _, m in self.final_layers.named_modules():
+ if isinstance(m, nn.Conv2d):
+ normal_init(m, std=0.001, bias=0)
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/heads/ae_multi_stage_head.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/heads/ae_multi_stage_head.py
new file mode 100644
index 0000000..195666b
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/heads/ae_multi_stage_head.py
@@ -0,0 +1,222 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import torch.nn as nn
+from mmcv.cnn import (build_conv_layer, build_upsample_layer, constant_init,
+ normal_init)
+
+from mmpose.models.builder import build_loss
+from ..builder import HEADS
+
+
+@HEADS.register_module()
+class AEMultiStageHead(nn.Module):
+ """Associative embedding multi-stage head.
+ paper ref: Alejandro Newell et al. "Associative
+ Embedding: End-to-end Learning for Joint Detection
+ and Grouping"
+
+ Args:
+ in_channels (int): Number of input channels.
+ out_channels (int): Number of output channels.
+ num_deconv_layers (int): Number of deconv layers.
+ num_deconv_layers should >= 0. Note that 0 means
+ no deconv layers.
+ num_deconv_filters (list|tuple): Number of filters.
+ If num_deconv_layers > 0, the length of
+ num_deconv_kernels (list|tuple): Kernel sizes.
+ loss_keypoint (dict): Config for loss. Default: None.
+ """
+
+ def __init__(self,
+ in_channels,
+ out_channels,
+ num_stages=1,
+ num_deconv_layers=3,
+ num_deconv_filters=(256, 256, 256),
+ num_deconv_kernels=(4, 4, 4),
+ extra=None,
+ loss_keypoint=None):
+ super().__init__()
+
+ self.loss = build_loss(loss_keypoint)
+
+ self.in_channels = in_channels
+ self.num_stages = num_stages
+
+ if extra is not None and not isinstance(extra, dict):
+ raise TypeError('extra should be dict or None.')
+
+ # build multi-stage deconv layers
+ self.multi_deconv_layers = nn.ModuleList([])
+ for _ in range(self.num_stages):
+ if num_deconv_layers > 0:
+ deconv_layers = self._make_deconv_layer(
+ num_deconv_layers,
+ num_deconv_filters,
+ num_deconv_kernels,
+ )
+ elif num_deconv_layers == 0:
+ deconv_layers = nn.Identity()
+ else:
+ raise ValueError(
+ f'num_deconv_layers ({num_deconv_layers}) should >= 0.')
+ self.multi_deconv_layers.append(deconv_layers)
+
+ identity_final_layer = False
+ if extra is not None and 'final_conv_kernel' in extra:
+ assert extra['final_conv_kernel'] in [0, 1, 3]
+ if extra['final_conv_kernel'] == 3:
+ padding = 1
+ elif extra['final_conv_kernel'] == 1:
+ padding = 0
+ else:
+ # 0 for Identity mapping.
+ identity_final_layer = True
+ kernel_size = extra['final_conv_kernel']
+ else:
+ kernel_size = 1
+ padding = 0
+
+ # build multi-stage final layers
+ self.multi_final_layers = nn.ModuleList([])
+ for i in range(self.num_stages):
+ if identity_final_layer:
+ final_layer = nn.Identity()
+ else:
+ final_layer = build_conv_layer(
+ cfg=dict(type='Conv2d'),
+ in_channels=num_deconv_filters[-1]
+ if num_deconv_layers > 0 else in_channels,
+ out_channels=out_channels,
+ kernel_size=kernel_size,
+ stride=1,
+ padding=padding)
+ self.multi_final_layers.append(final_layer)
+
+ def get_loss(self, output, targets, masks, joints):
+ """Calculate bottom-up keypoint loss.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+ - heatmaps height: H
+ - heatmaps weight: W
+
+ Args:
+ output (List(torch.Tensor[NxKxHxW])): Output heatmaps.
+ targets(List(List(torch.Tensor[NxKxHxW]))):
+ Multi-stage and multi-scale target heatmaps.
+ masks(List(List(torch.Tensor[NxHxW]))):
+ Masks of multi-stage and multi-scale target heatmaps
+ joints(List(List(torch.Tensor[NxMxKx2]))):
+ Joints of multi-stage multi-scale target heatmaps for ae loss
+ """
+
+ losses = dict()
+
+ # Flatten list:
+ # [stage_1_scale_1, stage_1_scale_2, ... , stage_1_scale_m,
+ # ...
+ # stage_n_scale_1, stage_n_scale_2, ... , stage_n_scale_m]
+ targets = [target for _targets in targets for target in _targets]
+ masks = [mask for _masks in masks for mask in _masks]
+ joints = [joint for _joints in joints for joint in _joints]
+
+ heatmaps_losses, push_losses, pull_losses = self.loss(
+ output, targets, masks, joints)
+
+ for idx in range(len(targets)):
+ if heatmaps_losses[idx] is not None:
+ heatmaps_loss = heatmaps_losses[idx].mean(dim=0)
+ if 'heatmap_loss' not in losses:
+ losses['heatmap_loss'] = heatmaps_loss
+ else:
+ losses['heatmap_loss'] += heatmaps_loss
+ if push_losses[idx] is not None:
+ push_loss = push_losses[idx].mean(dim=0)
+ if 'push_loss' not in losses:
+ losses['push_loss'] = push_loss
+ else:
+ losses['push_loss'] += push_loss
+ if pull_losses[idx] is not None:
+ pull_loss = pull_losses[idx].mean(dim=0)
+ if 'pull_loss' not in losses:
+ losses['pull_loss'] = pull_loss
+ else:
+ losses['pull_loss'] += pull_loss
+
+ return losses
+
+ def forward(self, x):
+ """Forward function.
+
+ Returns:
+ out (list[Tensor]): a list of heatmaps from multiple stages.
+ """
+ out = []
+ assert isinstance(x, list)
+ for i in range(self.num_stages):
+ y = self.multi_deconv_layers[i](x[i])
+ y = self.multi_final_layers[i](y)
+ out.append(y)
+ return out
+
+ def _make_deconv_layer(self, num_layers, num_filters, num_kernels):
+ """Make deconv layers."""
+ if num_layers != len(num_filters):
+ error_msg = f'num_layers({num_layers}) ' \
+ f'!= length of num_filters({len(num_filters)})'
+ raise ValueError(error_msg)
+ if num_layers != len(num_kernels):
+ error_msg = f'num_layers({num_layers}) ' \
+ f'!= length of num_kernels({len(num_kernels)})'
+ raise ValueError(error_msg)
+
+ layers = []
+ for i in range(num_layers):
+ kernel, padding, output_padding = \
+ self._get_deconv_cfg(num_kernels[i])
+
+ planes = num_filters[i]
+ layers.append(
+ build_upsample_layer(
+ dict(type='deconv'),
+ in_channels=self.in_channels,
+ out_channels=planes,
+ kernel_size=kernel,
+ stride=2,
+ padding=padding,
+ output_padding=output_padding,
+ bias=False))
+ layers.append(nn.BatchNorm2d(planes))
+ layers.append(nn.ReLU(inplace=True))
+ self.in_channels = planes
+
+ return nn.Sequential(*layers)
+
+ @staticmethod
+ def _get_deconv_cfg(deconv_kernel):
+ """Get configurations for deconv layers."""
+ if deconv_kernel == 4:
+ padding = 1
+ output_padding = 0
+ elif deconv_kernel == 3:
+ padding = 1
+ output_padding = 1
+ elif deconv_kernel == 2:
+ padding = 0
+ output_padding = 0
+ else:
+ raise ValueError(f'Not supported num_kernels ({deconv_kernel}).')
+
+ return deconv_kernel, padding, output_padding
+
+ def init_weights(self):
+ """Initialize model weights."""
+ for _, m in self.multi_deconv_layers.named_modules():
+ if isinstance(m, nn.ConvTranspose2d):
+ normal_init(m, std=0.001)
+ elif isinstance(m, nn.BatchNorm2d):
+ constant_init(m, 1)
+ for m in self.multi_final_layers.modules():
+ if isinstance(m, nn.Conv2d):
+ normal_init(m, std=0.001, bias=0)
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/heads/ae_simple_head.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/heads/ae_simple_head.py
new file mode 100644
index 0000000..9297f71
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/heads/ae_simple_head.py
@@ -0,0 +1,99 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from ..builder import HEADS
+from .deconv_head import DeconvHead
+
+
+@HEADS.register_module()
+class AESimpleHead(DeconvHead):
+ """Associative embedding simple head.
+ paper ref: Alejandro Newell et al. "Associative
+ Embedding: End-to-end Learning for Joint Detection
+ and Grouping"
+
+ Args:
+ in_channels (int): Number of input channels.
+ num_joints (int): Number of joints.
+ num_deconv_layers (int): Number of deconv layers.
+ num_deconv_layers should >= 0. Note that 0 means
+ no deconv layers.
+ num_deconv_filters (list|tuple): Number of filters.
+ If num_deconv_layers > 0, the length of
+ num_deconv_kernels (list|tuple): Kernel sizes.
+ tag_per_joint (bool): If tag_per_joint is True,
+ the dimension of tags equals to num_joints,
+ else the dimension of tags is 1. Default: True
+ with_ae_loss (list[bool]): Option to use ae loss or not.
+ loss_keypoint (dict): Config for loss. Default: None.
+ """
+
+ def __init__(self,
+ in_channels,
+ num_joints,
+ num_deconv_layers=3,
+ num_deconv_filters=(256, 256, 256),
+ num_deconv_kernels=(4, 4, 4),
+ tag_per_joint=True,
+ with_ae_loss=None,
+ extra=None,
+ loss_keypoint=None):
+
+ dim_tag = num_joints if tag_per_joint else 1
+ if with_ae_loss[0]:
+ out_channels = num_joints + dim_tag
+ else:
+ out_channels = num_joints
+
+ super().__init__(
+ in_channels,
+ out_channels,
+ num_deconv_layers=num_deconv_layers,
+ num_deconv_filters=num_deconv_filters,
+ num_deconv_kernels=num_deconv_kernels,
+ extra=extra,
+ loss_keypoint=loss_keypoint)
+
+ def get_loss(self, outputs, targets, masks, joints):
+ """Calculate bottom-up keypoint loss.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+ - num_outputs: O
+ - heatmaps height: H
+ - heatmaps weight: W
+
+ Args:
+ outputs (list(torch.Tensor[N,K,H,W])): Multi-scale output heatmaps.
+ targets (List(torch.Tensor[N,K,H,W])): Multi-scale target heatmaps.
+ masks (List(torch.Tensor[N,H,W])): Masks of multi-scale target
+ heatmaps
+ joints(List(torch.Tensor[N,M,K,2])): Joints of multi-scale target
+ heatmaps for ae loss
+ """
+
+ losses = dict()
+
+ heatmaps_losses, push_losses, pull_losses = self.loss(
+ outputs, targets, masks, joints)
+
+ for idx in range(len(targets)):
+ if heatmaps_losses[idx] is not None:
+ heatmaps_loss = heatmaps_losses[idx].mean(dim=0)
+ if 'heatmap_loss' not in losses:
+ losses['heatmap_loss'] = heatmaps_loss
+ else:
+ losses['heatmap_loss'] += heatmaps_loss
+ if push_losses[idx] is not None:
+ push_loss = push_losses[idx].mean(dim=0)
+ if 'push_loss' not in losses:
+ losses['push_loss'] = push_loss
+ else:
+ losses['push_loss'] += push_loss
+ if pull_losses[idx] is not None:
+ pull_loss = pull_losses[idx].mean(dim=0)
+ if 'pull_loss' not in losses:
+ losses['pull_loss'] = pull_loss
+ else:
+ losses['pull_loss'] += pull_loss
+
+ return losses
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/heads/deconv_head.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/heads/deconv_head.py
new file mode 100644
index 0000000..90846d2
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/heads/deconv_head.py
@@ -0,0 +1,295 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import torch
+import torch.nn as nn
+from mmcv.cnn import (build_conv_layer, build_norm_layer, build_upsample_layer,
+ constant_init, normal_init)
+
+from mmpose.models.builder import HEADS, build_loss
+from mmpose.models.utils.ops import resize
+
+
+@HEADS.register_module()
+class DeconvHead(nn.Module):
+ """Simple deconv head.
+
+ Args:
+ in_channels (int): Number of input channels.
+ out_channels (int): Number of output channels.
+ num_deconv_layers (int): Number of deconv layers.
+ num_deconv_layers should >= 0. Note that 0 means
+ no deconv layers.
+ num_deconv_filters (list|tuple): Number of filters.
+ If num_deconv_layers > 0, the length of
+ num_deconv_kernels (list|tuple): Kernel sizes.
+ in_index (int|Sequence[int]): Input feature index. Default: 0
+ input_transform (str|None): Transformation type of input features.
+ Options: 'resize_concat', 'multiple_select', None.
+ Default: None.
+
+ - 'resize_concat': Multiple feature maps will be resized to the
+ same size as the first one and then concat together.
+ Usually used in FCN head of HRNet.
+ - 'multiple_select': Multiple feature maps will be bundle into
+ a list and passed into decode head.
+ - None: Only one select feature map is allowed.
+ align_corners (bool): align_corners argument of F.interpolate.
+ Default: False.
+ loss_keypoint (dict): Config for loss. Default: None.
+ """
+
+ def __init__(self,
+ in_channels=3,
+ out_channels=17,
+ num_deconv_layers=3,
+ num_deconv_filters=(256, 256, 256),
+ num_deconv_kernels=(4, 4, 4),
+ extra=None,
+ in_index=0,
+ input_transform=None,
+ align_corners=False,
+ loss_keypoint=None):
+ super().__init__()
+
+ self.in_channels = in_channels
+ self.loss = build_loss(loss_keypoint)
+
+ self._init_inputs(in_channels, in_index, input_transform)
+ self.in_index = in_index
+ self.align_corners = align_corners
+
+ if extra is not None and not isinstance(extra, dict):
+ raise TypeError('extra should be dict or None.')
+
+ if num_deconv_layers > 0:
+ self.deconv_layers = self._make_deconv_layer(
+ num_deconv_layers,
+ num_deconv_filters,
+ num_deconv_kernels,
+ )
+ elif num_deconv_layers == 0:
+ self.deconv_layers = nn.Identity()
+ else:
+ raise ValueError(
+ f'num_deconv_layers ({num_deconv_layers}) should >= 0.')
+
+ identity_final_layer = False
+ if extra is not None and 'final_conv_kernel' in extra:
+ assert extra['final_conv_kernel'] in [0, 1, 3]
+ if extra['final_conv_kernel'] == 3:
+ padding = 1
+ elif extra['final_conv_kernel'] == 1:
+ padding = 0
+ else:
+ # 0 for Identity mapping.
+ identity_final_layer = True
+ kernel_size = extra['final_conv_kernel']
+ else:
+ kernel_size = 1
+ padding = 0
+
+ if identity_final_layer:
+ self.final_layer = nn.Identity()
+ else:
+ conv_channels = num_deconv_filters[
+ -1] if num_deconv_layers > 0 else self.in_channels
+
+ layers = []
+ if extra is not None:
+ num_conv_layers = extra.get('num_conv_layers', 0)
+ num_conv_kernels = extra.get('num_conv_kernels',
+ [1] * num_conv_layers)
+
+ for i in range(num_conv_layers):
+ layers.append(
+ build_conv_layer(
+ dict(type='Conv2d'),
+ in_channels=conv_channels,
+ out_channels=conv_channels,
+ kernel_size=num_conv_kernels[i],
+ stride=1,
+ padding=(num_conv_kernels[i] - 1) // 2))
+ layers.append(
+ build_norm_layer(dict(type='BN'), conv_channels)[1])
+ layers.append(nn.ReLU(inplace=True))
+
+ layers.append(
+ build_conv_layer(
+ cfg=dict(type='Conv2d'),
+ in_channels=conv_channels,
+ out_channels=out_channels,
+ kernel_size=kernel_size,
+ stride=1,
+ padding=padding))
+
+ if len(layers) > 1:
+ self.final_layer = nn.Sequential(*layers)
+ else:
+ self.final_layer = layers[0]
+
+ def _init_inputs(self, in_channels, in_index, input_transform):
+ """Check and initialize input transforms.
+
+ The in_channels, in_index and input_transform must match.
+ Specifically, when input_transform is None, only single feature map
+ will be selected. So in_channels and in_index must be of type int.
+ When input_transform is not None, in_channels and in_index must be
+ list or tuple, with the same length.
+
+ Args:
+ in_channels (int|Sequence[int]): Input channels.
+ in_index (int|Sequence[int]): Input feature index.
+ input_transform (str|None): Transformation type of input features.
+ Options: 'resize_concat', 'multiple_select', None.
+
+ - 'resize_concat': Multiple feature maps will be resize to the
+ same size as first one and than concat together.
+ Usually used in FCN head of HRNet.
+ - 'multiple_select': Multiple feature maps will be bundle into
+ a list and passed into decode head.
+ - None: Only one select feature map is allowed.
+ """
+
+ if input_transform is not None:
+ assert input_transform in ['resize_concat', 'multiple_select']
+ self.input_transform = input_transform
+ self.in_index = in_index
+ if input_transform is not None:
+ assert isinstance(in_channels, (list, tuple))
+ assert isinstance(in_index, (list, tuple))
+ assert len(in_channels) == len(in_index)
+ if input_transform == 'resize_concat':
+ self.in_channels = sum(in_channels)
+ else:
+ self.in_channels = in_channels
+ else:
+ assert isinstance(in_channels, int)
+ assert isinstance(in_index, int)
+ self.in_channels = in_channels
+
+ def _transform_inputs(self, inputs):
+ """Transform inputs for decoder.
+
+ Args:
+ inputs (list[Tensor] | Tensor): multi-level img features.
+
+ Returns:
+ Tensor: The transformed inputs
+ """
+ if not isinstance(inputs, list):
+ return inputs
+
+ if self.input_transform == 'resize_concat':
+ inputs = [inputs[i] for i in self.in_index]
+ upsampled_inputs = [
+ resize(
+ input=x,
+ size=inputs[0].shape[2:],
+ mode='bilinear',
+ align_corners=self.align_corners) for x in inputs
+ ]
+ inputs = torch.cat(upsampled_inputs, dim=1)
+ elif self.input_transform == 'multiple_select':
+ inputs = [inputs[i] for i in self.in_index]
+ else:
+ inputs = inputs[self.in_index]
+
+ return inputs
+
+ def _make_deconv_layer(self, num_layers, num_filters, num_kernels):
+ """Make deconv layers."""
+ if num_layers != len(num_filters):
+ error_msg = f'num_layers({num_layers}) ' \
+ f'!= length of num_filters({len(num_filters)})'
+ raise ValueError(error_msg)
+ if num_layers != len(num_kernels):
+ error_msg = f'num_layers({num_layers}) ' \
+ f'!= length of num_kernels({len(num_kernels)})'
+ raise ValueError(error_msg)
+
+ layers = []
+ for i in range(num_layers):
+ kernel, padding, output_padding = \
+ self._get_deconv_cfg(num_kernels[i])
+
+ planes = num_filters[i]
+ layers.append(
+ build_upsample_layer(
+ dict(type='deconv'),
+ in_channels=self.in_channels,
+ out_channels=planes,
+ kernel_size=kernel,
+ stride=2,
+ padding=padding,
+ output_padding=output_padding,
+ bias=False))
+ layers.append(nn.BatchNorm2d(planes))
+ layers.append(nn.ReLU(inplace=True))
+ self.in_channels = planes
+
+ return nn.Sequential(*layers)
+
+ @staticmethod
+ def _get_deconv_cfg(deconv_kernel):
+ """Get configurations for deconv layers."""
+ if deconv_kernel == 4:
+ padding = 1
+ output_padding = 0
+ elif deconv_kernel == 3:
+ padding = 1
+ output_padding = 1
+ elif deconv_kernel == 2:
+ padding = 0
+ output_padding = 0
+ else:
+ raise ValueError(f'Not supported num_kernels ({deconv_kernel}).')
+
+ return deconv_kernel, padding, output_padding
+
+ def get_loss(self, outputs, targets, masks):
+ """Calculate bottom-up masked mse loss.
+
+ Note:
+ - batch_size: N
+ - num_channels: C
+ - heatmaps height: H
+ - heatmaps weight: W
+
+ Args:
+ outputs (List(torch.Tensor[N,C,H,W])): Multi-scale outputs.
+ targets (List(torch.Tensor[N,C,H,W])): Multi-scale targets.
+ masks (List(torch.Tensor[N,H,W])): Masks of multi-scale targets.
+ """
+
+ losses = dict()
+
+ for idx in range(len(targets)):
+ if 'loss' not in losses:
+ losses['loss'] = self.loss(outputs[idx], targets[idx],
+ masks[idx])
+ else:
+ losses['loss'] += self.loss(outputs[idx], targets[idx],
+ masks[idx])
+
+ return losses
+
+ def forward(self, x):
+ """Forward function."""
+ x = self._transform_inputs(x)
+ final_outputs = []
+ x = self.deconv_layers(x)
+ y = self.final_layer(x)
+ final_outputs.append(y)
+ return final_outputs
+
+ def init_weights(self):
+ """Initialize model weights."""
+ for _, m in self.deconv_layers.named_modules():
+ if isinstance(m, nn.ConvTranspose2d):
+ normal_init(m, std=0.001)
+ elif isinstance(m, nn.BatchNorm2d):
+ constant_init(m, 1)
+ for m in self.final_layer.modules():
+ if isinstance(m, nn.Conv2d):
+ normal_init(m, std=0.001, bias=0)
+ elif isinstance(m, nn.BatchNorm2d):
+ constant_init(m, 1)
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/heads/deeppose_regression_head.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/heads/deeppose_regression_head.py
new file mode 100644
index 0000000..f326e26
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/heads/deeppose_regression_head.py
@@ -0,0 +1,176 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import numpy as np
+import torch.nn as nn
+from mmcv.cnn import normal_init
+
+from mmpose.core.evaluation import (keypoint_pck_accuracy,
+ keypoints_from_regression)
+from mmpose.core.post_processing import fliplr_regression
+from mmpose.models.builder import HEADS, build_loss
+
+
+@HEADS.register_module()
+class DeepposeRegressionHead(nn.Module):
+ """Deeppose regression head with fully connected layers.
+
+ "DeepPose: Human Pose Estimation via Deep Neural Networks".
+
+ Args:
+ in_channels (int): Number of input channels
+ num_joints (int): Number of joints
+ loss_keypoint (dict): Config for keypoint loss. Default: None.
+ """
+
+ def __init__(self,
+ in_channels,
+ num_joints,
+ loss_keypoint=None,
+ train_cfg=None,
+ test_cfg=None):
+ super().__init__()
+
+ self.in_channels = in_channels
+ self.num_joints = num_joints
+
+ self.loss = build_loss(loss_keypoint)
+
+ self.train_cfg = {} if train_cfg is None else train_cfg
+ self.test_cfg = {} if test_cfg is None else test_cfg
+
+ self.fc = nn.Linear(self.in_channels, self.num_joints * 2)
+
+ def forward(self, x):
+ """Forward function."""
+ output = self.fc(x)
+ N, C = output.shape
+ return output.reshape([N, C // 2, 2])
+
+ def get_loss(self, output, target, target_weight):
+ """Calculate top-down keypoint loss.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+
+ Args:
+ output (torch.Tensor[N, K, 2]): Output keypoints.
+ target (torch.Tensor[N, K, 2]): Target keypoints.
+ target_weight (torch.Tensor[N, K, 2]):
+ Weights across different joint types.
+ """
+
+ losses = dict()
+ assert not isinstance(self.loss, nn.Sequential)
+ assert target.dim() == 3 and target_weight.dim() == 3
+ losses['reg_loss'] = self.loss(output, target, target_weight)
+
+ return losses
+
+ def get_accuracy(self, output, target, target_weight):
+ """Calculate accuracy for top-down keypoint loss.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+
+ Args:
+ output (torch.Tensor[N, K, 2]): Output keypoints.
+ target (torch.Tensor[N, K, 2]): Target keypoints.
+ target_weight (torch.Tensor[N, K, 2]):
+ Weights across different joint types.
+ """
+
+ accuracy = dict()
+
+ N = output.shape[0]
+
+ _, avg_acc, cnt = keypoint_pck_accuracy(
+ output.detach().cpu().numpy(),
+ target.detach().cpu().numpy(),
+ target_weight[:, :, 0].detach().cpu().numpy() > 0,
+ thr=0.05,
+ normalize=np.ones((N, 2), dtype=np.float32))
+ accuracy['acc_pose'] = avg_acc
+
+ return accuracy
+
+ def inference_model(self, x, flip_pairs=None):
+ """Inference function.
+
+ Returns:
+ output_regression (np.ndarray): Output regression.
+
+ Args:
+ x (torch.Tensor[N, K, 2]): Input features.
+ flip_pairs (None | list[tuple()):
+ Pairs of keypoints which are mirrored.
+ """
+ output = self.forward(x)
+
+ if flip_pairs is not None:
+ output_regression = fliplr_regression(
+ output.detach().cpu().numpy(), flip_pairs)
+ else:
+ output_regression = output.detach().cpu().numpy()
+ return output_regression
+
+ def decode(self, img_metas, output, **kwargs):
+ """Decode the keypoints from output regression.
+
+ Args:
+ img_metas (list(dict)): Information about data augmentation
+ By default this includes:
+
+ - "image_file: path to the image file
+ - "center": center of the bbox
+ - "scale": scale of the bbox
+ - "rotation": rotation of the bbox
+ - "bbox_score": score of bbox
+ output (np.ndarray[N, K, 2]): predicted regression vector.
+ kwargs: dict contains 'img_size'.
+ img_size (tuple(img_width, img_height)): input image size.
+ """
+ batch_size = len(img_metas)
+
+ if 'bbox_id' in img_metas[0]:
+ bbox_ids = []
+ else:
+ bbox_ids = None
+
+ c = np.zeros((batch_size, 2), dtype=np.float32)
+ s = np.zeros((batch_size, 2), dtype=np.float32)
+ image_paths = []
+ score = np.ones(batch_size)
+ for i in range(batch_size):
+ c[i, :] = img_metas[i]['center']
+ s[i, :] = img_metas[i]['scale']
+ image_paths.append(img_metas[i]['image_file'])
+
+ if 'bbox_score' in img_metas[i]:
+ score[i] = np.array(img_metas[i]['bbox_score']).reshape(-1)
+ if bbox_ids is not None:
+ bbox_ids.append(img_metas[i]['bbox_id'])
+
+ preds, maxvals = keypoints_from_regression(output, c, s,
+ kwargs['img_size'])
+
+ all_preds = np.zeros((batch_size, preds.shape[1], 3), dtype=np.float32)
+ all_boxes = np.zeros((batch_size, 6), dtype=np.float32)
+ all_preds[:, :, 0:2] = preds[:, :, 0:2]
+ all_preds[:, :, 2:3] = maxvals
+ all_boxes[:, 0:2] = c[:, 0:2]
+ all_boxes[:, 2:4] = s[:, 0:2]
+ all_boxes[:, 4] = np.prod(s * 200.0, axis=1)
+ all_boxes[:, 5] = score
+
+ result = {}
+
+ result['preds'] = all_preds
+ result['boxes'] = all_boxes
+ result['image_paths'] = image_paths
+ result['bbox_ids'] = bbox_ids
+
+ return result
+
+ def init_weights(self):
+ normal_init(self.fc, mean=0, std=0.01, bias=0)
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/heads/hmr_head.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/heads/hmr_head.py
new file mode 100644
index 0000000..015a307
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/heads/hmr_head.py
@@ -0,0 +1,94 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import numpy as np
+import torch
+import torch.nn as nn
+from mmcv.cnn import xavier_init
+
+from ..builder import HEADS
+from ..utils.geometry import rot6d_to_rotmat
+
+
+@HEADS.register_module()
+class HMRMeshHead(nn.Module):
+ """SMPL parameters regressor head of simple baseline. "End-to-end Recovery
+ of Human Shape and Pose", CVPR'2018.
+
+ Args:
+ in_channels (int): Number of input channels
+ smpl_mean_params (str): The file name of the mean SMPL parameters
+ n_iter (int): The iterations of estimating delta parameters
+ """
+
+ def __init__(self, in_channels, smpl_mean_params=None, n_iter=3):
+ super().__init__()
+
+ self.in_channels = in_channels
+ self.n_iter = n_iter
+
+ npose = 24 * 6
+ nbeta = 10
+ ncam = 3
+ hidden_dim = 1024
+
+ self.fc1 = nn.Linear(in_channels + npose + nbeta + ncam, hidden_dim)
+ self.drop1 = nn.Dropout()
+ self.fc2 = nn.Linear(hidden_dim, hidden_dim)
+ self.drop2 = nn.Dropout()
+ self.decpose = nn.Linear(hidden_dim, npose)
+ self.decshape = nn.Linear(hidden_dim, nbeta)
+ self.deccam = nn.Linear(hidden_dim, ncam)
+
+ # Load mean SMPL parameters
+ if smpl_mean_params is None:
+ init_pose = torch.zeros([1, npose])
+ init_shape = torch.zeros([1, nbeta])
+ init_cam = torch.FloatTensor([[1, 0, 0]])
+ else:
+ mean_params = np.load(smpl_mean_params)
+ init_pose = torch.from_numpy(
+ mean_params['pose'][:]).unsqueeze(0).float()
+ init_shape = torch.from_numpy(
+ mean_params['shape'][:]).unsqueeze(0).float()
+ init_cam = torch.from_numpy(
+ mean_params['cam']).unsqueeze(0).float()
+ self.register_buffer('init_pose', init_pose)
+ self.register_buffer('init_shape', init_shape)
+ self.register_buffer('init_cam', init_cam)
+
+ def forward(self, x):
+ """Forward function.
+
+ x is the image feature map and is expected to be in shape (batch size x
+ channel number x height x width)
+ """
+ batch_size = x.shape[0]
+ # extract the global feature vector by average along
+ # spatial dimension.
+ x = x.mean(dim=-1).mean(dim=-1)
+
+ init_pose = self.init_pose.expand(batch_size, -1)
+ init_shape = self.init_shape.expand(batch_size, -1)
+ init_cam = self.init_cam.expand(batch_size, -1)
+
+ pred_pose = init_pose
+ pred_shape = init_shape
+ pred_cam = init_cam
+ for _ in range(self.n_iter):
+ xc = torch.cat([x, pred_pose, pred_shape, pred_cam], 1)
+ xc = self.fc1(xc)
+ xc = self.drop1(xc)
+ xc = self.fc2(xc)
+ xc = self.drop2(xc)
+ pred_pose = self.decpose(xc) + pred_pose
+ pred_shape = self.decshape(xc) + pred_shape
+ pred_cam = self.deccam(xc) + pred_cam
+
+ pred_rotmat = rot6d_to_rotmat(pred_pose).view(batch_size, 24, 3, 3)
+ out = (pred_rotmat, pred_shape, pred_cam)
+ return out
+
+ def init_weights(self):
+ """Initialize model weights."""
+ xavier_init(self.decpose, gain=0.01)
+ xavier_init(self.decshape, gain=0.01)
+ xavier_init(self.deccam, gain=0.01)
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/heads/interhand_3d_head.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/heads/interhand_3d_head.py
new file mode 100644
index 0000000..aebe4a5
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/heads/interhand_3d_head.py
@@ -0,0 +1,521 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import numpy as np
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+from mmcv.cnn import (build_conv_layer, build_norm_layer, build_upsample_layer,
+ constant_init, normal_init)
+
+from mmpose.core.evaluation.top_down_eval import (
+ keypoints_from_heatmaps3d, multilabel_classification_accuracy)
+from mmpose.core.post_processing import flip_back
+from mmpose.models.builder import build_loss
+from mmpose.models.necks import GlobalAveragePooling
+from ..builder import HEADS
+
+
+class Heatmap3DHead(nn.Module):
+ """Heatmap3DHead is a sub-module of Interhand3DHead, and outputs 3D
+ heatmaps. Heatmap3DHead is composed of (>=0) number of deconv layers and a
+ simple conv2d layer.
+
+ Args:
+ in_channels (int): Number of input channels
+ out_channels (int): Number of output channels
+ depth_size (int): Number of depth discretization size
+ num_deconv_layers (int): Number of deconv layers.
+ num_deconv_layers should >= 0. Note that 0 means no deconv layers.
+ num_deconv_filters (list|tuple): Number of filters.
+ num_deconv_kernels (list|tuple): Kernel sizes.
+ extra (dict): Configs for extra conv layers. Default: None
+ """
+
+ def __init__(self,
+ in_channels,
+ out_channels,
+ depth_size=64,
+ num_deconv_layers=3,
+ num_deconv_filters=(256, 256, 256),
+ num_deconv_kernels=(4, 4, 4),
+ extra=None):
+
+ super().__init__()
+
+ assert out_channels % depth_size == 0
+ self.depth_size = depth_size
+ self.in_channels = in_channels
+
+ if extra is not None and not isinstance(extra, dict):
+ raise TypeError('extra should be dict or None.')
+
+ if num_deconv_layers > 0:
+ self.deconv_layers = self._make_deconv_layer(
+ num_deconv_layers,
+ num_deconv_filters,
+ num_deconv_kernels,
+ )
+ elif num_deconv_layers == 0:
+ self.deconv_layers = nn.Identity()
+ else:
+ raise ValueError(
+ f'num_deconv_layers ({num_deconv_layers}) should >= 0.')
+
+ identity_final_layer = False
+ if extra is not None and 'final_conv_kernel' in extra:
+ assert extra['final_conv_kernel'] in [0, 1, 3]
+ if extra['final_conv_kernel'] == 3:
+ padding = 1
+ elif extra['final_conv_kernel'] == 1:
+ padding = 0
+ else:
+ # 0 for Identity mapping.
+ identity_final_layer = True
+ kernel_size = extra['final_conv_kernel']
+ else:
+ kernel_size = 1
+ padding = 0
+
+ if identity_final_layer:
+ self.final_layer = nn.Identity()
+ else:
+ conv_channels = num_deconv_filters[
+ -1] if num_deconv_layers > 0 else self.in_channels
+
+ layers = []
+ if extra is not None:
+ num_conv_layers = extra.get('num_conv_layers', 0)
+ num_conv_kernels = extra.get('num_conv_kernels',
+ [1] * num_conv_layers)
+
+ for i in range(num_conv_layers):
+ layers.append(
+ build_conv_layer(
+ dict(type='Conv2d'),
+ in_channels=conv_channels,
+ out_channels=conv_channels,
+ kernel_size=num_conv_kernels[i],
+ stride=1,
+ padding=(num_conv_kernels[i] - 1) // 2))
+ layers.append(
+ build_norm_layer(dict(type='BN'), conv_channels)[1])
+ layers.append(nn.ReLU(inplace=True))
+
+ layers.append(
+ build_conv_layer(
+ cfg=dict(type='Conv2d'),
+ in_channels=conv_channels,
+ out_channels=out_channels,
+ kernel_size=kernel_size,
+ stride=1,
+ padding=padding))
+
+ if len(layers) > 1:
+ self.final_layer = nn.Sequential(*layers)
+ else:
+ self.final_layer = layers[0]
+
+ def _make_deconv_layer(self, num_layers, num_filters, num_kernels):
+ """Make deconv layers."""
+ if num_layers != len(num_filters):
+ error_msg = f'num_layers({num_layers}) ' \
+ f'!= length of num_filters({len(num_filters)})'
+ raise ValueError(error_msg)
+ if num_layers != len(num_kernels):
+ error_msg = f'num_layers({num_layers}) ' \
+ f'!= length of num_kernels({len(num_kernels)})'
+ raise ValueError(error_msg)
+
+ layers = []
+ for i in range(num_layers):
+ kernel, padding, output_padding = \
+ self._get_deconv_cfg(num_kernels[i])
+
+ planes = num_filters[i]
+ layers.append(
+ build_upsample_layer(
+ dict(type='deconv'),
+ in_channels=self.in_channels,
+ out_channels=planes,
+ kernel_size=kernel,
+ stride=2,
+ padding=padding,
+ output_padding=output_padding,
+ bias=False))
+ layers.append(nn.BatchNorm2d(planes))
+ layers.append(nn.ReLU(inplace=True))
+ self.in_channels = planes
+
+ return nn.Sequential(*layers)
+
+ @staticmethod
+ def _get_deconv_cfg(deconv_kernel):
+ """Get configurations for deconv layers."""
+ if deconv_kernel == 4:
+ padding = 1
+ output_padding = 0
+ elif deconv_kernel == 3:
+ padding = 1
+ output_padding = 1
+ elif deconv_kernel == 2:
+ padding = 0
+ output_padding = 0
+ else:
+ raise ValueError(f'Not supported num_kernels ({deconv_kernel}).')
+
+ return deconv_kernel, padding, output_padding
+
+ def forward(self, x):
+ """Forward function."""
+ x = self.deconv_layers(x)
+ x = self.final_layer(x)
+ N, C, H, W = x.shape
+ # reshape the 2D heatmap to 3D heatmap
+ x = x.reshape(N, C // self.depth_size, self.depth_size, H, W)
+ return x
+
+ def init_weights(self):
+ """Initialize model weights."""
+ for _, m in self.deconv_layers.named_modules():
+ if isinstance(m, nn.ConvTranspose2d):
+ normal_init(m, std=0.001)
+ elif isinstance(m, nn.BatchNorm2d):
+ constant_init(m, 1)
+ for m in self.final_layer.modules():
+ if isinstance(m, nn.Conv2d):
+ normal_init(m, std=0.001, bias=0)
+ elif isinstance(m, nn.BatchNorm2d):
+ constant_init(m, 1)
+
+
+class Heatmap1DHead(nn.Module):
+ """Heatmap1DHead is a sub-module of Interhand3DHead, and outputs 1D
+ heatmaps.
+
+ Args:
+ in_channels (int): Number of input channels
+ heatmap_size (int): Heatmap size
+ hidden_dims (list|tuple): Number of feature dimension of FC layers.
+ """
+
+ def __init__(self, in_channels=2048, heatmap_size=64, hidden_dims=(512, )):
+ super().__init__()
+
+ self.in_channels = in_channels
+ self.heatmap_size = heatmap_size
+
+ feature_dims = [in_channels, *hidden_dims, heatmap_size]
+ self.fc = self._make_linear_layers(feature_dims, relu_final=False)
+
+ def soft_argmax_1d(self, heatmap1d):
+ heatmap1d = F.softmax(heatmap1d, 1)
+ accu = heatmap1d * torch.arange(
+ self.heatmap_size, dtype=heatmap1d.dtype,
+ device=heatmap1d.device)[None, :]
+ coord = accu.sum(dim=1)
+ return coord
+
+ def _make_linear_layers(self, feat_dims, relu_final=False):
+ """Make linear layers."""
+ layers = []
+ for i in range(len(feat_dims) - 1):
+ layers.append(nn.Linear(feat_dims[i], feat_dims[i + 1]))
+ if i < len(feat_dims) - 2 or \
+ (i == len(feat_dims) - 2 and relu_final):
+ layers.append(nn.ReLU(inplace=True))
+ return nn.Sequential(*layers)
+
+ def forward(self, x):
+ """Forward function."""
+ heatmap1d = self.fc(x)
+ value = self.soft_argmax_1d(heatmap1d).view(-1, 1)
+ return value
+
+ def init_weights(self):
+ """Initialize model weights."""
+ for m in self.fc.modules():
+ if isinstance(m, nn.Linear):
+ normal_init(m, mean=0, std=0.01, bias=0)
+
+
+class MultilabelClassificationHead(nn.Module):
+ """MultilabelClassificationHead is a sub-module of Interhand3DHead, and
+ outputs hand type classification.
+
+ Args:
+ in_channels (int): Number of input channels
+ num_labels (int): Number of labels
+ hidden_dims (list|tuple): Number of hidden dimension of FC layers.
+ """
+
+ def __init__(self, in_channels=2048, num_labels=2, hidden_dims=(512, )):
+ super().__init__()
+
+ self.in_channels = in_channels
+ self.num_labesl = num_labels
+
+ feature_dims = [in_channels, *hidden_dims, num_labels]
+ self.fc = self._make_linear_layers(feature_dims, relu_final=False)
+
+ def _make_linear_layers(self, feat_dims, relu_final=False):
+ """Make linear layers."""
+ layers = []
+ for i in range(len(feat_dims) - 1):
+ layers.append(nn.Linear(feat_dims[i], feat_dims[i + 1]))
+ if i < len(feat_dims) - 2 or \
+ (i == len(feat_dims) - 2 and relu_final):
+ layers.append(nn.ReLU(inplace=True))
+ return nn.Sequential(*layers)
+
+ def forward(self, x):
+ """Forward function."""
+ labels = torch.sigmoid(self.fc(x))
+ return labels
+
+ def init_weights(self):
+ for m in self.fc.modules():
+ if isinstance(m, nn.Linear):
+ normal_init(m, mean=0, std=0.01, bias=0)
+
+
+@HEADS.register_module()
+class Interhand3DHead(nn.Module):
+ """Interhand 3D head of paper ref: Gyeongsik Moon. "InterHand2.6M: A
+ Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single
+ RGB Image".
+
+ Args:
+ keypoint_head_cfg (dict): Configs of Heatmap3DHead for hand
+ keypoint estimation.
+ root_head_cfg (dict): Configs of Heatmap1DHead for relative
+ hand root depth estimation.
+ hand_type_head_cfg (dict): Configs of MultilabelClassificationHead
+ for hand type classification.
+ loss_keypoint (dict): Config for keypoint loss. Default: None.
+ loss_root_depth (dict): Config for relative root depth loss.
+ Default: None.
+ loss_hand_type (dict): Config for hand type classification
+ loss. Default: None.
+ """
+
+ def __init__(self,
+ keypoint_head_cfg,
+ root_head_cfg,
+ hand_type_head_cfg,
+ loss_keypoint=None,
+ loss_root_depth=None,
+ loss_hand_type=None,
+ train_cfg=None,
+ test_cfg=None):
+ super().__init__()
+
+ # build sub-module heads
+ self.right_hand_head = Heatmap3DHead(**keypoint_head_cfg)
+ self.left_hand_head = Heatmap3DHead(**keypoint_head_cfg)
+ self.root_head = Heatmap1DHead(**root_head_cfg)
+ self.hand_type_head = MultilabelClassificationHead(
+ **hand_type_head_cfg)
+ self.neck = GlobalAveragePooling()
+
+ # build losses
+ self.keypoint_loss = build_loss(loss_keypoint)
+ self.root_depth_loss = build_loss(loss_root_depth)
+ self.hand_type_loss = build_loss(loss_hand_type)
+ self.train_cfg = {} if train_cfg is None else train_cfg
+ self.test_cfg = {} if test_cfg is None else test_cfg
+ self.target_type = self.test_cfg.get('target_type', 'GaussianHeatmap')
+
+ def init_weights(self):
+ self.left_hand_head.init_weights()
+ self.right_hand_head.init_weights()
+ self.root_head.init_weights()
+ self.hand_type_head.init_weights()
+
+ def get_loss(self, output, target, target_weight):
+ """Calculate loss for hand keypoint heatmaps, relative root depth and
+ hand type.
+
+ Args:
+ output (list[Tensor]): a list of outputs from multiple heads.
+ target (list[Tensor]): a list of targets for multiple heads.
+ target_weight (list[Tensor]): a list of targets weight for
+ multiple heads.
+ """
+ losses = dict()
+
+ # hand keypoint loss
+ assert not isinstance(self.keypoint_loss, nn.Sequential)
+ out, tar, tar_weight = output[0], target[0], target_weight[0]
+ assert tar.dim() == 5 and tar_weight.dim() == 3
+ losses['hand_loss'] = self.keypoint_loss(out, tar, tar_weight)
+
+ # relative root depth loss
+ assert not isinstance(self.root_depth_loss, nn.Sequential)
+ out, tar, tar_weight = output[1], target[1], target_weight[1]
+ assert tar.dim() == 2 and tar_weight.dim() == 2
+ losses['rel_root_loss'] = self.root_depth_loss(out, tar, tar_weight)
+
+ # hand type loss
+ assert not isinstance(self.hand_type_loss, nn.Sequential)
+ out, tar, tar_weight = output[2], target[2], target_weight[2]
+ assert tar.dim() == 2 and tar_weight.dim() in [1, 2]
+ losses['hand_type_loss'] = self.hand_type_loss(out, tar, tar_weight)
+
+ return losses
+
+ def get_accuracy(self, output, target, target_weight):
+ """Calculate accuracy for hand type.
+
+ Args:
+ output (list[Tensor]): a list of outputs from multiple heads.
+ target (list[Tensor]): a list of targets for multiple heads.
+ target_weight (list[Tensor]): a list of targets weight for
+ multiple heads.
+ """
+ accuracy = dict()
+ avg_acc = multilabel_classification_accuracy(
+ output[2].detach().cpu().numpy(),
+ target[2].detach().cpu().numpy(),
+ target_weight[2].detach().cpu().numpy(),
+ )
+ accuracy['acc_classification'] = float(avg_acc)
+ return accuracy
+
+ def forward(self, x):
+ """Forward function."""
+ outputs = []
+ outputs.append(
+ torch.cat([self.right_hand_head(x),
+ self.left_hand_head(x)], dim=1))
+ x = self.neck(x)
+ outputs.append(self.root_head(x))
+ outputs.append(self.hand_type_head(x))
+ return outputs
+
+ def inference_model(self, x, flip_pairs=None):
+ """Inference function.
+
+ Returns:
+ output (list[np.ndarray]): list of output hand keypoint
+ heatmaps, relative root depth and hand type.
+
+ Args:
+ x (torch.Tensor[N,K,H,W]): Input features.
+ flip_pairs (None | list[tuple()):
+ Pairs of keypoints which are mirrored.
+ """
+
+ output = self.forward(x)
+
+ if flip_pairs is not None:
+ # flip 3D heatmap
+ heatmap_3d = output[0]
+ N, K, D, H, W = heatmap_3d.shape
+ # reshape 3D heatmap to 2D heatmap
+ heatmap_3d = heatmap_3d.reshape(N, K * D, H, W)
+ # 2D heatmap flip
+ heatmap_3d_flipped_back = flip_back(
+ heatmap_3d.detach().cpu().numpy(),
+ flip_pairs,
+ target_type=self.target_type)
+ # reshape back to 3D heatmap
+ heatmap_3d_flipped_back = heatmap_3d_flipped_back.reshape(
+ N, K, D, H, W)
+ # feature is not aligned, shift flipped heatmap for higher accuracy
+ if self.test_cfg.get('shift_heatmap', False):
+ heatmap_3d_flipped_back[...,
+ 1:] = heatmap_3d_flipped_back[..., :-1]
+ output[0] = heatmap_3d_flipped_back
+
+ # flip relative hand root depth
+ output[1] = -output[1].detach().cpu().numpy()
+
+ # flip hand type
+ hand_type = output[2].detach().cpu().numpy()
+ hand_type_flipped_back = hand_type.copy()
+ hand_type_flipped_back[:, 0] = hand_type[:, 1]
+ hand_type_flipped_back[:, 1] = hand_type[:, 0]
+ output[2] = hand_type_flipped_back
+ else:
+ output = [out.detach().cpu().numpy() for out in output]
+
+ return output
+
+ def decode(self, img_metas, output, **kwargs):
+ """Decode hand keypoint, relative root depth and hand type.
+
+ Args:
+ img_metas (list(dict)): Information about data augmentation
+ By default this includes:
+
+ - "image_file: path to the image file
+ - "center": center of the bbox
+ - "scale": scale of the bbox
+ - "rotation": rotation of the bbox
+ - "bbox_score": score of bbox
+ - "heatmap3d_depth_bound": depth bound of hand keypoint
+ 3D heatmap
+ - "root_depth_bound": depth bound of relative root depth
+ 1D heatmap
+ output (list[np.ndarray]): model predicted 3D heatmaps, relative
+ root depth and hand type.
+ """
+
+ batch_size = len(img_metas)
+ result = {}
+
+ heatmap3d_depth_bound = np.ones(batch_size, dtype=np.float32)
+ root_depth_bound = np.ones(batch_size, dtype=np.float32)
+ center = np.zeros((batch_size, 2), dtype=np.float32)
+ scale = np.zeros((batch_size, 2), dtype=np.float32)
+ image_paths = []
+ score = np.ones(batch_size, dtype=np.float32)
+ if 'bbox_id' in img_metas[0]:
+ bbox_ids = []
+ else:
+ bbox_ids = None
+
+ for i in range(batch_size):
+ heatmap3d_depth_bound[i] = img_metas[i]['heatmap3d_depth_bound']
+ root_depth_bound[i] = img_metas[i]['root_depth_bound']
+ center[i, :] = img_metas[i]['center']
+ scale[i, :] = img_metas[i]['scale']
+ image_paths.append(img_metas[i]['image_file'])
+
+ if 'bbox_score' in img_metas[i]:
+ score[i] = np.array(img_metas[i]['bbox_score']).reshape(-1)
+ if bbox_ids is not None:
+ bbox_ids.append(img_metas[i]['bbox_id'])
+
+ all_boxes = np.zeros((batch_size, 6), dtype=np.float32)
+ all_boxes[:, 0:2] = center[:, 0:2]
+ all_boxes[:, 2:4] = scale[:, 0:2]
+ # scale is defined as: bbox_size / 200.0, so we
+ # need multiply 200.0 to get bbox size
+ all_boxes[:, 4] = np.prod(scale * 200.0, axis=1)
+ all_boxes[:, 5] = score
+ result['boxes'] = all_boxes
+ result['image_paths'] = image_paths
+ result['bbox_ids'] = bbox_ids
+
+ # decode 3D heatmaps of hand keypoints
+ heatmap3d = output[0]
+ preds, maxvals = keypoints_from_heatmaps3d(heatmap3d, center, scale)
+ keypoints_3d = np.zeros((batch_size, preds.shape[1], 4),
+ dtype=np.float32)
+ keypoints_3d[:, :, 0:3] = preds[:, :, 0:3]
+ keypoints_3d[:, :, 3:4] = maxvals
+ # transform keypoint depth to camera space
+ keypoints_3d[:, :, 2] = \
+ (keypoints_3d[:, :, 2] / self.right_hand_head.depth_size - 0.5) \
+ * heatmap3d_depth_bound[:, np.newaxis]
+
+ result['preds'] = keypoints_3d
+
+ # decode relative hand root depth
+ # transform relative root depth to camera space
+ result['rel_root_depth'] = (output[1] / self.root_head.heatmap_size -
+ 0.5) * root_depth_bound
+
+ # decode hand type
+ result['hand_type'] = output[2] > 0.5
+ return result
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/heads/temporal_regression_head.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/heads/temporal_regression_head.py
new file mode 100644
index 0000000..97a07f9
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/heads/temporal_regression_head.py
@@ -0,0 +1,319 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import numpy as np
+import torch.nn as nn
+from mmcv.cnn import build_conv_layer, constant_init, kaiming_init
+from mmcv.utils.parrots_wrapper import _BatchNorm
+
+from mmpose.core import (WeightNormClipHook, compute_similarity_transform,
+ fliplr_regression)
+from mmpose.models.builder import HEADS, build_loss
+
+
+@HEADS.register_module()
+class TemporalRegressionHead(nn.Module):
+ """Regression head of VideoPose3D.
+
+ "3D human pose estimation in video with temporal convolutions and
+ semi-supervised training", CVPR'2019.
+
+ Args:
+ in_channels (int): Number of input channels
+ num_joints (int): Number of joints
+ loss_keypoint (dict): Config for keypoint loss. Default: None.
+ max_norm (float|None): if not None, the weight of convolution layers
+ will be clipped to have a maximum norm of max_norm.
+ is_trajectory (bool): If the model only predicts root joint
+ position, then this arg should be set to True. In this case,
+ traj_loss will be calculated. Otherwise, it should be set to
+ False. Default: False.
+ """
+
+ def __init__(self,
+ in_channels,
+ num_joints,
+ max_norm=None,
+ loss_keypoint=None,
+ is_trajectory=False,
+ train_cfg=None,
+ test_cfg=None):
+ super().__init__()
+
+ self.in_channels = in_channels
+ self.num_joints = num_joints
+ self.max_norm = max_norm
+ self.loss = build_loss(loss_keypoint)
+ self.is_trajectory = is_trajectory
+ if self.is_trajectory:
+ assert self.num_joints == 1
+
+ self.train_cfg = {} if train_cfg is None else train_cfg
+ self.test_cfg = {} if test_cfg is None else test_cfg
+
+ self.conv = build_conv_layer(
+ dict(type='Conv1d'), in_channels, num_joints * 3, 1)
+
+ if self.max_norm is not None:
+ # Apply weight norm clip to conv layers
+ weight_clip = WeightNormClipHook(self.max_norm)
+ for module in self.modules():
+ if isinstance(module, nn.modules.conv._ConvNd):
+ weight_clip.register(module)
+
+ @staticmethod
+ def _transform_inputs(x):
+ """Transform inputs for decoder.
+
+ Args:
+ inputs (tuple or list of Tensor | Tensor): multi-level features.
+
+ Returns:
+ Tensor: The transformed inputs
+ """
+ if not isinstance(x, (list, tuple)):
+ return x
+
+ assert len(x) > 0
+
+ # return the top-level feature of the 1D feature pyramid
+ return x[-1]
+
+ def forward(self, x):
+ """Forward function."""
+ x = self._transform_inputs(x)
+
+ assert x.ndim == 3 and x.shape[2] == 1, f'Invalid shape {x.shape}'
+ output = self.conv(x)
+ N = output.shape[0]
+ return output.reshape(N, self.num_joints, 3)
+
+ def get_loss(self, output, target, target_weight):
+ """Calculate keypoint loss.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+
+ Args:
+ output (torch.Tensor[N, K, 3]): Output keypoints.
+ target (torch.Tensor[N, K, 3]): Target keypoints.
+ target_weight (torch.Tensor[N, K, 3]):
+ Weights across different joint types.
+ If self.is_trajectory is True and target_weight is None,
+ target_weight will be set inversely proportional to joint
+ depth.
+ """
+ losses = dict()
+ assert not isinstance(self.loss, nn.Sequential)
+
+ # trajectory model
+ if self.is_trajectory:
+ if target.dim() == 2:
+ target.unsqueeze_(1)
+
+ if target_weight is None:
+ target_weight = (1 / target[:, :, 2:]).expand(target.shape)
+ assert target.dim() == 3 and target_weight.dim() == 3
+
+ losses['traj_loss'] = self.loss(output, target, target_weight)
+
+ # pose model
+ else:
+ if target_weight is None:
+ target_weight = target.new_ones(target.shape)
+ assert target.dim() == 3 and target_weight.dim() == 3
+ losses['reg_loss'] = self.loss(output, target, target_weight)
+
+ return losses
+
+ def get_accuracy(self, output, target, target_weight, metas):
+ """Calculate accuracy for keypoint loss.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+
+ Args:
+ output (torch.Tensor[N, K, 3]): Output keypoints.
+ target (torch.Tensor[N, K, 3]): Target keypoints.
+ target_weight (torch.Tensor[N, K, 3]):
+ Weights across different joint types.
+ metas (list(dict)): Information about data augmentation including:
+
+ - target_image_path (str): Optional, path to the image file
+ - target_mean (float): Optional, normalization parameter of
+ the target pose.
+ - target_std (float): Optional, normalization parameter of the
+ target pose.
+ - root_position (np.ndarray[3,1]): Optional, global
+ position of the root joint.
+ - root_index (torch.ndarray[1,]): Optional, original index of
+ the root joint before root-centering.
+ """
+
+ accuracy = dict()
+
+ N = output.shape[0]
+ output_ = output.detach().cpu().numpy()
+ target_ = target.detach().cpu().numpy()
+ # Denormalize the predicted pose
+ if 'target_mean' in metas[0] and 'target_std' in metas[0]:
+ target_mean = np.stack([m['target_mean'] for m in metas])
+ target_std = np.stack([m['target_std'] for m in metas])
+ output_ = self._denormalize_joints(output_, target_mean,
+ target_std)
+ target_ = self._denormalize_joints(target_, target_mean,
+ target_std)
+
+ # Restore global position
+ if self.test_cfg.get('restore_global_position', False):
+ root_pos = np.stack([m['root_position'] for m in metas])
+ root_idx = metas[0].get('root_position_index', None)
+ output_ = self._restore_global_position(output_, root_pos,
+ root_idx)
+ target_ = self._restore_global_position(target_, root_pos,
+ root_idx)
+ # Get target weight
+ if target_weight is None:
+ target_weight_ = np.ones_like(target_)
+ else:
+ target_weight_ = target_weight.detach().cpu().numpy()
+ if self.test_cfg.get('restore_global_position', False):
+ root_idx = metas[0].get('root_position_index', None)
+ root_weight = metas[0].get('root_joint_weight', 1.0)
+ target_weight_ = self._restore_root_target_weight(
+ target_weight_, root_weight, root_idx)
+
+ mpjpe = np.mean(
+ np.linalg.norm((output_ - target_) * target_weight_, axis=-1))
+
+ transformed_output = np.zeros_like(output_)
+ for i in range(N):
+ transformed_output[i, :, :] = compute_similarity_transform(
+ output_[i, :, :], target_[i, :, :])
+ p_mpjpe = np.mean(
+ np.linalg.norm(
+ (transformed_output - target_) * target_weight_, axis=-1))
+
+ accuracy['mpjpe'] = output.new_tensor(mpjpe)
+ accuracy['p_mpjpe'] = output.new_tensor(p_mpjpe)
+
+ return accuracy
+
+ def inference_model(self, x, flip_pairs=None):
+ """Inference function.
+
+ Returns:
+ output_regression (np.ndarray): Output regression.
+
+ Args:
+ x (torch.Tensor[N, K, 2]): Input features.
+ flip_pairs (None | list[tuple()):
+ Pairs of keypoints which are mirrored.
+ """
+ output = self.forward(x)
+
+ if flip_pairs is not None:
+ output_regression = fliplr_regression(
+ output.detach().cpu().numpy(),
+ flip_pairs,
+ center_mode='static',
+ center_x=0)
+ else:
+ output_regression = output.detach().cpu().numpy()
+ return output_regression
+
+ def decode(self, metas, output):
+ """Decode the keypoints from output regression.
+
+ Args:
+ metas (list(dict)): Information about data augmentation.
+ By default this includes:
+
+ - "target_image_path": path to the image file
+ output (np.ndarray[N, K, 3]): predicted regression vector.
+ metas (list(dict)): Information about data augmentation including:
+
+ - target_image_path (str): Optional, path to the image file
+ - target_mean (float): Optional, normalization parameter of
+ the target pose.
+ - target_std (float): Optional, normalization parameter of the
+ target pose.
+ - root_position (np.ndarray[3,1]): Optional, global
+ position of the root joint.
+ - root_index (torch.ndarray[1,]): Optional, original index of
+ the root joint before root-centering.
+ """
+
+ # Denormalize the predicted pose
+ if 'target_mean' in metas[0] and 'target_std' in metas[0]:
+ target_mean = np.stack([m['target_mean'] for m in metas])
+ target_std = np.stack([m['target_std'] for m in metas])
+ output = self._denormalize_joints(output, target_mean, target_std)
+
+ # Restore global position
+ if self.test_cfg.get('restore_global_position', False):
+ root_pos = np.stack([m['root_position'] for m in metas])
+ root_idx = metas[0].get('root_position_index', None)
+ output = self._restore_global_position(output, root_pos, root_idx)
+
+ target_image_paths = [m.get('target_image_path', None) for m in metas]
+ result = {'preds': output, 'target_image_paths': target_image_paths}
+
+ return result
+
+ @staticmethod
+ def _denormalize_joints(x, mean, std):
+ """Denormalize joint coordinates with given statistics mean and std.
+
+ Args:
+ x (np.ndarray[N, K, 3]): Normalized joint coordinates.
+ mean (np.ndarray[K, 3]): Mean value.
+ std (np.ndarray[K, 3]): Std value.
+ """
+ assert x.ndim == 3
+ assert x.shape == mean.shape == std.shape
+
+ return x * std + mean
+
+ @staticmethod
+ def _restore_global_position(x, root_pos, root_idx=None):
+ """Restore global position of the root-centered joints.
+
+ Args:
+ x (np.ndarray[N, K, 3]): root-centered joint coordinates
+ root_pos (np.ndarray[N,1,3]): The global position of the
+ root joint.
+ root_idx (int|None): If not none, the root joint will be inserted
+ back to the pose at the given index.
+ """
+ x = x + root_pos
+ if root_idx is not None:
+ x = np.insert(x, root_idx, root_pos.squeeze(1), axis=1)
+ return x
+
+ @staticmethod
+ def _restore_root_target_weight(target_weight, root_weight, root_idx=None):
+ """Restore the target weight of the root joint after the restoration of
+ the global position.
+
+ Args:
+ target_weight (np.ndarray[N, K, 1]): Target weight of relativized
+ joints.
+ root_weight (float): The target weight value of the root joint.
+ root_idx (int|None): If not none, the root joint weight will be
+ inserted back to the target weight at the given index.
+ """
+ if root_idx is not None:
+ root_weight = np.full(
+ target_weight.shape[0], root_weight, dtype=target_weight.dtype)
+ target_weight = np.insert(
+ target_weight, root_idx, root_weight[:, None], axis=1)
+ return target_weight
+
+ def init_weights(self):
+ """Initialize the weights."""
+ for m in self.modules():
+ if isinstance(m, nn.modules.conv._ConvNd):
+ kaiming_init(m, mode='fan_in', nonlinearity='relu')
+ elif isinstance(m, _BatchNorm):
+ constant_init(m, 1)
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/heads/topdown_heatmap_base_head.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/heads/topdown_heatmap_base_head.py
new file mode 100644
index 0000000..09646ea
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/heads/topdown_heatmap_base_head.py
@@ -0,0 +1,120 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from abc import ABCMeta, abstractmethod
+
+import numpy as np
+import torch.nn as nn
+
+from mmpose.core.evaluation.top_down_eval import keypoints_from_heatmaps
+
+
+class TopdownHeatmapBaseHead(nn.Module):
+ """Base class for top-down heatmap heads.
+
+ All top-down heatmap heads should subclass it.
+ All subclass should overwrite:
+
+ Methods:`get_loss`, supporting to calculate loss.
+ Methods:`get_accuracy`, supporting to calculate accuracy.
+ Methods:`forward`, supporting to forward model.
+ Methods:`inference_model`, supporting to inference model.
+ """
+
+ __metaclass__ = ABCMeta
+
+ @abstractmethod
+ def get_loss(self, **kwargs):
+ """Gets the loss."""
+
+ @abstractmethod
+ def get_accuracy(self, **kwargs):
+ """Gets the accuracy."""
+
+ @abstractmethod
+ def forward(self, **kwargs):
+ """Forward function."""
+
+ @abstractmethod
+ def inference_model(self, **kwargs):
+ """Inference function."""
+
+ def decode(self, img_metas, output, **kwargs):
+ """Decode keypoints from heatmaps.
+
+ Args:
+ img_metas (list(dict)): Information about data augmentation
+ By default this includes:
+
+ - "image_file: path to the image file
+ - "center": center of the bbox
+ - "scale": scale of the bbox
+ - "rotation": rotation of the bbox
+ - "bbox_score": score of bbox
+ output (np.ndarray[N, K, H, W]): model predicted heatmaps.
+ """
+ batch_size = len(img_metas)
+
+ if 'bbox_id' in img_metas[0]:
+ bbox_ids = []
+ else:
+ bbox_ids = None
+
+ c = np.zeros((batch_size, 2), dtype=np.float32)
+ s = np.zeros((batch_size, 2), dtype=np.float32)
+ image_paths = []
+ score = np.ones(batch_size)
+ for i in range(batch_size):
+ c[i, :] = img_metas[i]['center']
+ s[i, :] = img_metas[i]['scale']
+ image_paths.append(img_metas[i]['image_file'])
+
+ if 'bbox_score' in img_metas[i]:
+ score[i] = np.array(img_metas[i]['bbox_score']).reshape(-1)
+ if bbox_ids is not None:
+ bbox_ids.append(img_metas[i]['bbox_id'])
+
+ preds, maxvals = keypoints_from_heatmaps(
+ output,
+ c,
+ s,
+ unbiased=self.test_cfg.get('unbiased_decoding', False),
+ post_process=self.test_cfg.get('post_process', 'default'),
+ kernel=self.test_cfg.get('modulate_kernel', 11),
+ valid_radius_factor=self.test_cfg.get('valid_radius_factor',
+ 0.0546875),
+ use_udp=self.test_cfg.get('use_udp', False),
+ target_type=self.test_cfg.get('target_type', 'GaussianHeatmap'))
+
+ all_preds = np.zeros((batch_size, preds.shape[1], 3), dtype=np.float32)
+ all_boxes = np.zeros((batch_size, 6), dtype=np.float32)
+ all_preds[:, :, 0:2] = preds[:, :, 0:2]
+ all_preds[:, :, 2:3] = maxvals
+ all_boxes[:, 0:2] = c[:, 0:2]
+ all_boxes[:, 2:4] = s[:, 0:2]
+ all_boxes[:, 4] = np.prod(s * 200.0, axis=1)
+ all_boxes[:, 5] = score
+
+ result = {}
+
+ result['preds'] = all_preds
+ result['boxes'] = all_boxes
+ result['image_paths'] = image_paths
+ result['bbox_ids'] = bbox_ids
+
+ return result
+
+ @staticmethod
+ def _get_deconv_cfg(deconv_kernel):
+ """Get configurations for deconv layers."""
+ if deconv_kernel == 4:
+ padding = 1
+ output_padding = 0
+ elif deconv_kernel == 3:
+ padding = 1
+ output_padding = 1
+ elif deconv_kernel == 2:
+ padding = 0
+ output_padding = 0
+ else:
+ raise ValueError(f'Not supported num_kernels ({deconv_kernel}).')
+
+ return deconv_kernel, padding, output_padding
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/heads/topdown_heatmap_multi_stage_head.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/heads/topdown_heatmap_multi_stage_head.py
new file mode 100644
index 0000000..c439f5b
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/heads/topdown_heatmap_multi_stage_head.py
@@ -0,0 +1,572 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import copy as cp
+
+import torch.nn as nn
+from mmcv.cnn import (ConvModule, DepthwiseSeparableConvModule, Linear,
+ build_activation_layer, build_conv_layer,
+ build_norm_layer, build_upsample_layer, constant_init,
+ kaiming_init, normal_init)
+
+from mmpose.core.evaluation import pose_pck_accuracy
+from mmpose.core.post_processing import flip_back
+from mmpose.models.builder import build_loss
+from ..builder import HEADS
+from .topdown_heatmap_base_head import TopdownHeatmapBaseHead
+
+
+@HEADS.register_module()
+class TopdownHeatmapMultiStageHead(TopdownHeatmapBaseHead):
+ """Top-down heatmap multi-stage head.
+
+ TopdownHeatmapMultiStageHead is consisted of multiple branches,
+ each of which has num_deconv_layers(>=0) number of deconv layers
+ and a simple conv2d layer.
+
+ Args:
+ in_channels (int): Number of input channels.
+ out_channels (int): Number of output channels.
+ num_stages (int): Number of stages.
+ num_deconv_layers (int): Number of deconv layers.
+ num_deconv_layers should >= 0. Note that 0 means
+ no deconv layers.
+ num_deconv_filters (list|tuple): Number of filters.
+ If num_deconv_layers > 0, the length of
+ num_deconv_kernels (list|tuple): Kernel sizes.
+ loss_keypoint (dict): Config for keypoint loss. Default: None.
+ """
+
+ def __init__(self,
+ in_channels=512,
+ out_channels=17,
+ num_stages=1,
+ num_deconv_layers=3,
+ num_deconv_filters=(256, 256, 256),
+ num_deconv_kernels=(4, 4, 4),
+ extra=None,
+ loss_keypoint=None,
+ train_cfg=None,
+ test_cfg=None):
+ super().__init__()
+
+ self.in_channels = in_channels
+ self.num_stages = num_stages
+ self.loss = build_loss(loss_keypoint)
+
+ self.train_cfg = {} if train_cfg is None else train_cfg
+ self.test_cfg = {} if test_cfg is None else test_cfg
+ self.target_type = self.test_cfg.get('target_type', 'GaussianHeatmap')
+
+ if extra is not None and not isinstance(extra, dict):
+ raise TypeError('extra should be dict or None.')
+
+ # build multi-stage deconv layers
+ self.multi_deconv_layers = nn.ModuleList([])
+ for _ in range(self.num_stages):
+ if num_deconv_layers > 0:
+ deconv_layers = self._make_deconv_layer(
+ num_deconv_layers,
+ num_deconv_filters,
+ num_deconv_kernels,
+ )
+ elif num_deconv_layers == 0:
+ deconv_layers = nn.Identity()
+ else:
+ raise ValueError(
+ f'num_deconv_layers ({num_deconv_layers}) should >= 0.')
+ self.multi_deconv_layers.append(deconv_layers)
+
+ identity_final_layer = False
+ if extra is not None and 'final_conv_kernel' in extra:
+ assert extra['final_conv_kernel'] in [0, 1, 3]
+ if extra['final_conv_kernel'] == 3:
+ padding = 1
+ elif extra['final_conv_kernel'] == 1:
+ padding = 0
+ else:
+ # 0 for Identity mapping.
+ identity_final_layer = True
+ kernel_size = extra['final_conv_kernel']
+ else:
+ kernel_size = 1
+ padding = 0
+
+ # build multi-stage final layers
+ self.multi_final_layers = nn.ModuleList([])
+ for i in range(self.num_stages):
+ if identity_final_layer:
+ final_layer = nn.Identity()
+ else:
+ final_layer = build_conv_layer(
+ cfg=dict(type='Conv2d'),
+ in_channels=num_deconv_filters[-1]
+ if num_deconv_layers > 0 else in_channels,
+ out_channels=out_channels,
+ kernel_size=kernel_size,
+ stride=1,
+ padding=padding)
+ self.multi_final_layers.append(final_layer)
+
+ def get_loss(self, output, target, target_weight):
+ """Calculate top-down keypoint loss.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+ - num_outputs: O
+ - heatmaps height: H
+ - heatmaps weight: W
+
+ Args:
+ output (torch.Tensor[N,K,H,W]):
+ Output heatmaps.
+ target (torch.Tensor[N,K,H,W]):
+ Target heatmaps.
+ target_weight (torch.Tensor[N,K,1]):
+ Weights across different joint types.
+ """
+
+ losses = dict()
+
+ assert isinstance(output, list)
+ assert target.dim() == 4 and target_weight.dim() == 3
+
+ if isinstance(self.loss, nn.Sequential):
+ assert len(self.loss) == len(output)
+ for i in range(len(output)):
+ target_i = target
+ target_weight_i = target_weight
+ if isinstance(self.loss, nn.Sequential):
+ loss_func = self.loss[i]
+ else:
+ loss_func = self.loss
+ loss_i = loss_func(output[i], target_i, target_weight_i)
+ if 'heatmap_loss' not in losses:
+ losses['heatmap_loss'] = loss_i
+ else:
+ losses['heatmap_loss'] += loss_i
+
+ return losses
+
+ def get_accuracy(self, output, target, target_weight):
+ """Calculate accuracy for top-down keypoint loss.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+ - heatmaps height: H
+ - heatmaps weight: W
+
+ Args:
+ output (torch.Tensor[N,K,H,W]): Output heatmaps.
+ target (torch.Tensor[N,K,H,W]): Target heatmaps.
+ target_weight (torch.Tensor[N,K,1]):
+ Weights across different joint types.
+ """
+
+ accuracy = dict()
+
+ if self.target_type == 'GaussianHeatmap':
+ _, avg_acc, _ = pose_pck_accuracy(
+ output[-1].detach().cpu().numpy(),
+ target.detach().cpu().numpy(),
+ target_weight.detach().cpu().numpy().squeeze(-1) > 0)
+ accuracy['acc_pose'] = float(avg_acc)
+
+ return accuracy
+
+ def forward(self, x):
+ """Forward function.
+
+ Returns:
+ out (list[Tensor]): a list of heatmaps from multiple stages.
+ """
+ out = []
+ assert isinstance(x, list)
+ for i in range(self.num_stages):
+ y = self.multi_deconv_layers[i](x[i])
+ y = self.multi_final_layers[i](y)
+ out.append(y)
+ return out
+
+ def inference_model(self, x, flip_pairs=None):
+ """Inference function.
+
+ Returns:
+ output_heatmap (np.ndarray): Output heatmaps.
+
+ Args:
+ x (List[torch.Tensor[NxKxHxW]]): Input features.
+ flip_pairs (None | list[tuple()):
+ Pairs of keypoints which are mirrored.
+ """
+ output = self.forward(x)
+ assert isinstance(output, list)
+ output = output[-1]
+
+ if flip_pairs is not None:
+ # perform flip
+ output_heatmap = flip_back(
+ output.detach().cpu().numpy(),
+ flip_pairs,
+ target_type=self.target_type)
+ # feature is not aligned, shift flipped heatmap for higher accuracy
+ if self.test_cfg.get('shift_heatmap', False):
+ output_heatmap[:, :, :, 1:] = output_heatmap[:, :, :, :-1]
+ else:
+ output_heatmap = output.detach().cpu().numpy()
+
+ return output_heatmap
+
+ def _make_deconv_layer(self, num_layers, num_filters, num_kernels):
+ """Make deconv layers."""
+ if num_layers != len(num_filters):
+ error_msg = f'num_layers({num_layers}) ' \
+ f'!= length of num_filters({len(num_filters)})'
+ raise ValueError(error_msg)
+ if num_layers != len(num_kernels):
+ error_msg = f'num_layers({num_layers}) ' \
+ f'!= length of num_kernels({len(num_kernels)})'
+ raise ValueError(error_msg)
+
+ layers = []
+ for i in range(num_layers):
+ kernel, padding, output_padding = \
+ self._get_deconv_cfg(num_kernels[i])
+
+ planes = num_filters[i]
+ layers.append(
+ build_upsample_layer(
+ dict(type='deconv'),
+ in_channels=self.in_channels,
+ out_channels=planes,
+ kernel_size=kernel,
+ stride=2,
+ padding=padding,
+ output_padding=output_padding,
+ bias=False))
+ layers.append(nn.BatchNorm2d(planes))
+ layers.append(nn.ReLU(inplace=True))
+ self.in_channels = planes
+
+ return nn.Sequential(*layers)
+
+ def init_weights(self):
+ """Initialize model weights."""
+ for _, m in self.multi_deconv_layers.named_modules():
+ if isinstance(m, nn.ConvTranspose2d):
+ normal_init(m, std=0.001)
+ elif isinstance(m, nn.BatchNorm2d):
+ constant_init(m, 1)
+ for m in self.multi_final_layers.modules():
+ if isinstance(m, nn.Conv2d):
+ normal_init(m, std=0.001, bias=0)
+
+
+class PredictHeatmap(nn.Module):
+ """Predict the heat map for an input feature.
+
+ Args:
+ unit_channels (int): Number of input channels.
+ out_channels (int): Number of output channels.
+ out_shape (tuple): Shape of the output heatmap.
+ use_prm (bool): Whether to use pose refine machine. Default: False.
+ norm_cfg (dict): dictionary to construct and config norm layer.
+ Default: dict(type='BN')
+ """
+
+ def __init__(self,
+ unit_channels,
+ out_channels,
+ out_shape,
+ use_prm=False,
+ norm_cfg=dict(type='BN')):
+ # Protect mutable default arguments
+ norm_cfg = cp.deepcopy(norm_cfg)
+ super().__init__()
+ self.unit_channels = unit_channels
+ self.out_channels = out_channels
+ self.out_shape = out_shape
+ self.use_prm = use_prm
+ if use_prm:
+ self.prm = PRM(out_channels, norm_cfg=norm_cfg)
+ self.conv_layers = nn.Sequential(
+ ConvModule(
+ unit_channels,
+ unit_channels,
+ kernel_size=1,
+ stride=1,
+ padding=0,
+ norm_cfg=norm_cfg,
+ inplace=False),
+ ConvModule(
+ unit_channels,
+ out_channels,
+ kernel_size=3,
+ stride=1,
+ padding=1,
+ norm_cfg=norm_cfg,
+ act_cfg=None,
+ inplace=False))
+
+ def forward(self, feature):
+ feature = self.conv_layers(feature)
+ output = nn.functional.interpolate(
+ feature, size=self.out_shape, mode='bilinear', align_corners=True)
+ if self.use_prm:
+ output = self.prm(output)
+ return output
+
+
+class PRM(nn.Module):
+ """Pose Refine Machine.
+
+ Please refer to "Learning Delicate Local Representations
+ for Multi-Person Pose Estimation" (ECCV 2020).
+
+ Args:
+ out_channels (int): Channel number of the output. Equals to
+ the number of key points.
+ norm_cfg (dict): dictionary to construct and config norm layer.
+ Default: dict(type='BN')
+ """
+
+ def __init__(self, out_channels, norm_cfg=dict(type='BN')):
+ # Protect mutable default arguments
+ norm_cfg = cp.deepcopy(norm_cfg)
+ super().__init__()
+ self.out_channels = out_channels
+ self.global_pooling = nn.AdaptiveAvgPool2d((1, 1))
+ self.middle_path = nn.Sequential(
+ Linear(self.out_channels, self.out_channels),
+ build_norm_layer(dict(type='BN1d'), out_channels)[1],
+ build_activation_layer(dict(type='ReLU')),
+ Linear(self.out_channels, self.out_channels),
+ build_norm_layer(dict(type='BN1d'), out_channels)[1],
+ build_activation_layer(dict(type='ReLU')),
+ build_activation_layer(dict(type='Sigmoid')))
+
+ self.bottom_path = nn.Sequential(
+ ConvModule(
+ self.out_channels,
+ self.out_channels,
+ kernel_size=1,
+ stride=1,
+ padding=0,
+ norm_cfg=norm_cfg,
+ inplace=False),
+ DepthwiseSeparableConvModule(
+ self.out_channels,
+ 1,
+ kernel_size=9,
+ stride=1,
+ padding=4,
+ norm_cfg=norm_cfg,
+ inplace=False), build_activation_layer(dict(type='Sigmoid')))
+ self.conv_bn_relu_prm_1 = ConvModule(
+ self.out_channels,
+ self.out_channels,
+ kernel_size=3,
+ stride=1,
+ padding=1,
+ norm_cfg=norm_cfg,
+ inplace=False)
+
+ def forward(self, x):
+ out = self.conv_bn_relu_prm_1(x)
+ out_1 = out
+
+ out_2 = self.global_pooling(out_1)
+ out_2 = out_2.view(out_2.size(0), -1)
+ out_2 = self.middle_path(out_2)
+ out_2 = out_2.unsqueeze(2)
+ out_2 = out_2.unsqueeze(3)
+
+ out_3 = self.bottom_path(out_1)
+ out = out_1 * (1 + out_2 * out_3)
+
+ return out
+
+
+@HEADS.register_module()
+class TopdownHeatmapMSMUHead(TopdownHeatmapBaseHead):
+ """Heads for multi-stage multi-unit heads used in Multi-Stage Pose
+ estimation Network (MSPN), and Residual Steps Networks (RSN).
+
+ Args:
+ unit_channels (int): Number of input channels.
+ out_channels (int): Number of output channels.
+ out_shape (tuple): Shape of the output heatmap.
+ num_stages (int): Number of stages.
+ num_units (int): Number of units in each stage.
+ use_prm (bool): Whether to use pose refine machine (PRM).
+ Default: False.
+ norm_cfg (dict): dictionary to construct and config norm layer.
+ Default: dict(type='BN')
+ loss_keypoint (dict): Config for keypoint loss. Default: None.
+ """
+
+ def __init__(self,
+ out_shape,
+ unit_channels=256,
+ out_channels=17,
+ num_stages=4,
+ num_units=4,
+ use_prm=False,
+ norm_cfg=dict(type='BN'),
+ loss_keypoint=None,
+ train_cfg=None,
+ test_cfg=None):
+ # Protect mutable default arguments
+ norm_cfg = cp.deepcopy(norm_cfg)
+ super().__init__()
+
+ self.train_cfg = {} if train_cfg is None else train_cfg
+ self.test_cfg = {} if test_cfg is None else test_cfg
+ self.target_type = self.test_cfg.get('target_type', 'GaussianHeatmap')
+
+ self.out_shape = out_shape
+ self.unit_channels = unit_channels
+ self.out_channels = out_channels
+ self.num_stages = num_stages
+ self.num_units = num_units
+
+ self.loss = build_loss(loss_keypoint)
+
+ self.predict_layers = nn.ModuleList([])
+ for i in range(self.num_stages):
+ for j in range(self.num_units):
+ self.predict_layers.append(
+ PredictHeatmap(
+ unit_channels,
+ out_channels,
+ out_shape,
+ use_prm,
+ norm_cfg=norm_cfg))
+
+ def get_loss(self, output, target, target_weight):
+ """Calculate top-down keypoint loss.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+ - num_outputs: O
+ - heatmaps height: H
+ - heatmaps weight: W
+
+ Args:
+ output (torch.Tensor[N,O,K,H,W]): Output heatmaps.
+ target (torch.Tensor[N,O,K,H,W]): Target heatmaps.
+ target_weight (torch.Tensor[N,O,K,1]):
+ Weights across different joint types.
+ """
+
+ losses = dict()
+
+ assert isinstance(output, list)
+ assert target.dim() == 5 and target_weight.dim() == 4
+ assert target.size(1) == len(output)
+
+ if isinstance(self.loss, nn.Sequential):
+ assert len(self.loss) == len(output)
+ for i in range(len(output)):
+ target_i = target[:, i, :, :, :]
+ target_weight_i = target_weight[:, i, :, :]
+
+ if isinstance(self.loss, nn.Sequential):
+ loss_func = self.loss[i]
+ else:
+ loss_func = self.loss
+
+ loss_i = loss_func(output[i], target_i, target_weight_i)
+ if 'heatmap_loss' not in losses:
+ losses['heatmap_loss'] = loss_i
+ else:
+ losses['heatmap_loss'] += loss_i
+
+ return losses
+
+ def get_accuracy(self, output, target, target_weight):
+ """Calculate accuracy for top-down keypoint loss.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+ - heatmaps height: H
+ - heatmaps weight: W
+
+ Args:
+ output (torch.Tensor[N,K,H,W]): Output heatmaps.
+ target (torch.Tensor[N,K,H,W]): Target heatmaps.
+ target_weight (torch.Tensor[N,K,1]):
+ Weights across different joint types.
+ """
+
+ accuracy = dict()
+
+ if self.target_type == 'GaussianHeatmap':
+ assert isinstance(output, list)
+ assert target.dim() == 5 and target_weight.dim() == 4
+ _, avg_acc, _ = pose_pck_accuracy(
+ output[-1].detach().cpu().numpy(),
+ target[:, -1, ...].detach().cpu().numpy(),
+ target_weight[:, -1,
+ ...].detach().cpu().numpy().squeeze(-1) > 0)
+ accuracy['acc_pose'] = float(avg_acc)
+
+ return accuracy
+
+ def forward(self, x):
+ """Forward function.
+
+ Returns:
+ out (list[Tensor]): a list of heatmaps from multiple stages
+ and units.
+ """
+ out = []
+ assert isinstance(x, list)
+ assert len(x) == self.num_stages
+ assert isinstance(x[0], list)
+ assert len(x[0]) == self.num_units
+ assert x[0][0].shape[1] == self.unit_channels
+ for i in range(self.num_stages):
+ for j in range(self.num_units):
+ y = self.predict_layers[i * self.num_units + j](x[i][j])
+ out.append(y)
+
+ return out
+
+ def inference_model(self, x, flip_pairs=None):
+ """Inference function.
+
+ Returns:
+ output_heatmap (np.ndarray): Output heatmaps.
+
+ Args:
+ x (list[torch.Tensor[N,K,H,W]]): Input features.
+ flip_pairs (None | list[tuple]):
+ Pairs of keypoints which are mirrored.
+ """
+ output = self.forward(x)
+ assert isinstance(output, list)
+ output = output[-1]
+ if flip_pairs is not None:
+ output_heatmap = flip_back(
+ output.detach().cpu().numpy(),
+ flip_pairs,
+ target_type=self.target_type)
+ # feature is not aligned, shift flipped heatmap for higher accuracy
+ if self.test_cfg.get('shift_heatmap', False):
+ output_heatmap[:, :, :, 1:] = output_heatmap[:, :, :, :-1]
+ else:
+ output_heatmap = output.detach().cpu().numpy()
+ return output_heatmap
+
+ def init_weights(self):
+ """Initialize model weights."""
+ for m in self.predict_layers.modules():
+ if isinstance(m, nn.Conv2d):
+ kaiming_init(m)
+ elif isinstance(m, nn.BatchNorm2d):
+ constant_init(m, 1)
+ elif isinstance(m, nn.Linear):
+ normal_init(m, std=0.01)
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/heads/topdown_heatmap_simple_head.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/heads/topdown_heatmap_simple_head.py
new file mode 100644
index 0000000..72f3348
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/heads/topdown_heatmap_simple_head.py
@@ -0,0 +1,350 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import torch
+import torch.nn as nn
+from mmcv.cnn import (build_conv_layer, build_norm_layer, build_upsample_layer,
+ constant_init, normal_init)
+
+from mmpose.core.evaluation import pose_pck_accuracy
+from mmpose.core.post_processing import flip_back
+from mmpose.models.builder import build_loss
+from mmpose.models.utils.ops import resize
+from ..builder import HEADS
+import torch.nn.functional as F
+from .topdown_heatmap_base_head import TopdownHeatmapBaseHead
+
+
+@HEADS.register_module()
+class TopdownHeatmapSimpleHead(TopdownHeatmapBaseHead):
+ """Top-down heatmap simple head. paper ref: Bin Xiao et al. ``Simple
+ Baselines for Human Pose Estimation and Tracking``.
+
+ TopdownHeatmapSimpleHead is consisted of (>=0) number of deconv layers
+ and a simple conv2d layer.
+
+ Args:
+ in_channels (int): Number of input channels
+ out_channels (int): Number of output channels
+ num_deconv_layers (int): Number of deconv layers.
+ num_deconv_layers should >= 0. Note that 0 means
+ no deconv layers.
+ num_deconv_filters (list|tuple): Number of filters.
+ If num_deconv_layers > 0, the length of
+ num_deconv_kernels (list|tuple): Kernel sizes.
+ in_index (int|Sequence[int]): Input feature index. Default: 0
+ input_transform (str|None): Transformation type of input features.
+ Options: 'resize_concat', 'multiple_select', None.
+ Default: None.
+
+ - 'resize_concat': Multiple feature maps will be resized to the
+ same size as the first one and then concat together.
+ Usually used in FCN head of HRNet.
+ - 'multiple_select': Multiple feature maps will be bundle into
+ a list and passed into decode head.
+ - None: Only one select feature map is allowed.
+ align_corners (bool): align_corners argument of F.interpolate.
+ Default: False.
+ loss_keypoint (dict): Config for keypoint loss. Default: None.
+ """
+
+ def __init__(self,
+ in_channels,
+ out_channels,
+ num_deconv_layers=3,
+ num_deconv_filters=(256, 256, 256),
+ num_deconv_kernels=(4, 4, 4),
+ extra=None,
+ in_index=0,
+ input_transform=None,
+ align_corners=False,
+ loss_keypoint=None,
+ train_cfg=None,
+ test_cfg=None,
+ upsample=0,):
+ super().__init__()
+
+ self.in_channels = in_channels
+ self.loss = build_loss(loss_keypoint)
+ self.upsample = upsample
+
+ self.train_cfg = {} if train_cfg is None else train_cfg
+ self.test_cfg = {} if test_cfg is None else test_cfg
+ self.target_type = self.test_cfg.get('target_type', 'GaussianHeatmap')
+
+ self._init_inputs(in_channels, in_index, input_transform)
+ self.in_index = in_index
+ self.align_corners = align_corners
+
+ if extra is not None and not isinstance(extra, dict):
+ raise TypeError('extra should be dict or None.')
+
+ if num_deconv_layers > 0:
+ self.deconv_layers = self._make_deconv_layer(
+ num_deconv_layers,
+ num_deconv_filters,
+ num_deconv_kernels,
+ )
+ elif num_deconv_layers == 0:
+ self.deconv_layers = nn.Identity()
+ else:
+ raise ValueError(
+ f'num_deconv_layers ({num_deconv_layers}) should >= 0.')
+
+ identity_final_layer = False
+ if extra is not None and 'final_conv_kernel' in extra:
+ assert extra['final_conv_kernel'] in [0, 1, 3]
+ if extra['final_conv_kernel'] == 3:
+ padding = 1
+ elif extra['final_conv_kernel'] == 1:
+ padding = 0
+ else:
+ # 0 for Identity mapping.
+ identity_final_layer = True
+ kernel_size = extra['final_conv_kernel']
+ else:
+ kernel_size = 1
+ padding = 0
+
+ if identity_final_layer:
+ self.final_layer = nn.Identity()
+ else:
+ conv_channels = num_deconv_filters[
+ -1] if num_deconv_layers > 0 else self.in_channels
+
+ layers = []
+ if extra is not None:
+ num_conv_layers = extra.get('num_conv_layers', 0)
+ num_conv_kernels = extra.get('num_conv_kernels',
+ [1] * num_conv_layers)
+
+ for i in range(num_conv_layers):
+ layers.append(
+ build_conv_layer(
+ dict(type='Conv2d'),
+ in_channels=conv_channels,
+ out_channels=conv_channels,
+ kernel_size=num_conv_kernels[i],
+ stride=1,
+ padding=(num_conv_kernels[i] - 1) // 2))
+ layers.append(
+ build_norm_layer(dict(type='BN'), conv_channels)[1])
+ layers.append(nn.ReLU(inplace=True))
+
+ layers.append(
+ build_conv_layer(
+ cfg=dict(type='Conv2d'),
+ in_channels=conv_channels,
+ out_channels=out_channels,
+ kernel_size=kernel_size,
+ stride=1,
+ padding=padding))
+
+ if len(layers) > 1:
+ self.final_layer = nn.Sequential(*layers)
+ else:
+ self.final_layer = layers[0]
+
+ def get_loss(self, output, target, target_weight):
+ """Calculate top-down keypoint loss.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+ - heatmaps height: H
+ - heatmaps weight: W
+
+ Args:
+ output (torch.Tensor[N,K,H,W]): Output heatmaps.
+ target (torch.Tensor[N,K,H,W]): Target heatmaps.
+ target_weight (torch.Tensor[N,K,1]):
+ Weights across different joint types.
+ """
+
+ losses = dict()
+
+ assert not isinstance(self.loss, nn.Sequential)
+ assert target.dim() == 4 and target_weight.dim() == 3
+ losses['heatmap_loss'] = self.loss(output, target, target_weight)
+
+ return losses
+
+ def get_accuracy(self, output, target, target_weight):
+ """Calculate accuracy for top-down keypoint loss.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+ - heatmaps height: H
+ - heatmaps weight: W
+
+ Args:
+ output (torch.Tensor[N,K,H,W]): Output heatmaps.
+ target (torch.Tensor[N,K,H,W]): Target heatmaps.
+ target_weight (torch.Tensor[N,K,1]):
+ Weights across different joint types.
+ """
+
+ accuracy = dict()
+
+ if self.target_type == 'GaussianHeatmap':
+ _, avg_acc, _ = pose_pck_accuracy(
+ output.detach().cpu().numpy(),
+ target.detach().cpu().numpy(),
+ target_weight.detach().cpu().numpy().squeeze(-1) > 0)
+ accuracy['acc_pose'] = float(avg_acc)
+
+ return accuracy
+
+ def forward(self, x):
+ """Forward function."""
+ x = self._transform_inputs(x)
+ x = self.deconv_layers(x)
+ x = self.final_layer(x)
+ return x
+
+ def inference_model(self, x, flip_pairs=None):
+ """Inference function.
+
+ Returns:
+ output_heatmap (np.ndarray): Output heatmaps.
+
+ Args:
+ x (torch.Tensor[N,K,H,W]): Input features.
+ flip_pairs (None | list[tuple]):
+ Pairs of keypoints which are mirrored.
+ """
+ output = self.forward(x)
+
+ if flip_pairs is not None:
+ output_heatmap = flip_back(
+ output.detach().cpu().numpy(),
+ flip_pairs,
+ target_type=self.target_type)
+ # feature is not aligned, shift flipped heatmap for higher accuracy
+ if self.test_cfg.get('shift_heatmap', False):
+ output_heatmap[:, :, :, 1:] = output_heatmap[:, :, :, :-1]
+ else:
+ output_heatmap = output.detach().cpu().numpy()
+ return output_heatmap
+
+ def _init_inputs(self, in_channels, in_index, input_transform):
+ """Check and initialize input transforms.
+
+ The in_channels, in_index and input_transform must match.
+ Specifically, when input_transform is None, only single feature map
+ will be selected. So in_channels and in_index must be of type int.
+ When input_transform is not None, in_channels and in_index must be
+ list or tuple, with the same length.
+
+ Args:
+ in_channels (int|Sequence[int]): Input channels.
+ in_index (int|Sequence[int]): Input feature index.
+ input_transform (str|None): Transformation type of input features.
+ Options: 'resize_concat', 'multiple_select', None.
+
+ - 'resize_concat': Multiple feature maps will be resize to the
+ same size as first one and than concat together.
+ Usually used in FCN head of HRNet.
+ - 'multiple_select': Multiple feature maps will be bundle into
+ a list and passed into decode head.
+ - None: Only one select feature map is allowed.
+ """
+
+ if input_transform is not None:
+ assert input_transform in ['resize_concat', 'multiple_select']
+ self.input_transform = input_transform
+ self.in_index = in_index
+ if input_transform is not None:
+ assert isinstance(in_channels, (list, tuple))
+ assert isinstance(in_index, (list, tuple))
+ assert len(in_channels) == len(in_index)
+ if input_transform == 'resize_concat':
+ self.in_channels = sum(in_channels)
+ else:
+ self.in_channels = in_channels
+ else:
+ assert isinstance(in_channels, int)
+ assert isinstance(in_index, int)
+ self.in_channels = in_channels
+
+ def _transform_inputs(self, inputs):
+ """Transform inputs for decoder.
+
+ Args:
+ inputs (list[Tensor] | Tensor): multi-level img features.
+
+ Returns:
+ Tensor: The transformed inputs
+ """
+ if not isinstance(inputs, list):
+ if not isinstance(inputs, list):
+ if self.upsample > 0:
+ inputs = resize(
+ input=F.relu(inputs),
+ scale_factor=self.upsample,
+ mode='bilinear',
+ align_corners=self.align_corners
+ )
+ return inputs
+
+ if self.input_transform == 'resize_concat':
+ inputs = [inputs[i] for i in self.in_index]
+ upsampled_inputs = [
+ resize(
+ input=x,
+ size=inputs[0].shape[2:],
+ mode='bilinear',
+ align_corners=self.align_corners) for x in inputs
+ ]
+ inputs = torch.cat(upsampled_inputs, dim=1)
+ elif self.input_transform == 'multiple_select':
+ inputs = [inputs[i] for i in self.in_index]
+ else:
+ inputs = inputs[self.in_index]
+
+ return inputs
+
+ def _make_deconv_layer(self, num_layers, num_filters, num_kernels):
+ """Make deconv layers."""
+ if num_layers != len(num_filters):
+ error_msg = f'num_layers({num_layers}) ' \
+ f'!= length of num_filters({len(num_filters)})'
+ raise ValueError(error_msg)
+ if num_layers != len(num_kernels):
+ error_msg = f'num_layers({num_layers}) ' \
+ f'!= length of num_kernels({len(num_kernels)})'
+ raise ValueError(error_msg)
+
+ layers = []
+ for i in range(num_layers):
+ kernel, padding, output_padding = \
+ self._get_deconv_cfg(num_kernels[i])
+
+ planes = num_filters[i]
+ layers.append(
+ build_upsample_layer(
+ dict(type='deconv'),
+ in_channels=self.in_channels,
+ out_channels=planes,
+ kernel_size=kernel,
+ stride=2,
+ padding=padding,
+ output_padding=output_padding,
+ bias=False))
+ layers.append(nn.BatchNorm2d(planes))
+ layers.append(nn.ReLU(inplace=True))
+ self.in_channels = planes
+
+ return nn.Sequential(*layers)
+
+ def init_weights(self):
+ """Initialize model weights."""
+ for _, m in self.deconv_layers.named_modules():
+ if isinstance(m, nn.ConvTranspose2d):
+ normal_init(m, std=0.001)
+ elif isinstance(m, nn.BatchNorm2d):
+ constant_init(m, 1)
+ for m in self.final_layer.modules():
+ if isinstance(m, nn.Conv2d):
+ normal_init(m, std=0.001, bias=0)
+ elif isinstance(m, nn.BatchNorm2d):
+ constant_init(m, 1)
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/heads/vipnas_heatmap_simple_head.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/heads/vipnas_heatmap_simple_head.py
new file mode 100644
index 0000000..4170312
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/heads/vipnas_heatmap_simple_head.py
@@ -0,0 +1,349 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import torch
+import torch.nn as nn
+from mmcv.cnn import (build_conv_layer, build_norm_layer, build_upsample_layer,
+ constant_init, normal_init)
+
+from mmpose.core.evaluation import pose_pck_accuracy
+from mmpose.core.post_processing import flip_back
+from mmpose.models.builder import build_loss
+from mmpose.models.utils.ops import resize
+from ..builder import HEADS
+from .topdown_heatmap_base_head import TopdownHeatmapBaseHead
+
+
+@HEADS.register_module()
+class ViPNASHeatmapSimpleHead(TopdownHeatmapBaseHead):
+ """ViPNAS heatmap simple head.
+
+ ViPNAS: Efficient Video Pose Estimation via Neural Architecture Search.
+ More details can be found in the `paper
+ `__ .
+
+ TopdownHeatmapSimpleHead is consisted of (>=0) number of deconv layers
+ and a simple conv2d layer.
+
+ Args:
+ in_channels (int): Number of input channels
+ out_channels (int): Number of output channels
+ num_deconv_layers (int): Number of deconv layers.
+ num_deconv_layers should >= 0. Note that 0 means
+ no deconv layers.
+ num_deconv_filters (list|tuple): Number of filters.
+ If num_deconv_layers > 0, the length of
+ num_deconv_kernels (list|tuple): Kernel sizes.
+ num_deconv_groups (list|tuple): Group number.
+ in_index (int|Sequence[int]): Input feature index. Default: -1
+ input_transform (str|None): Transformation type of input features.
+ Options: 'resize_concat', 'multiple_select', None.
+ Default: None.
+
+ - 'resize_concat': Multiple feature maps will be resize to the
+ same size as first one and than concat together.
+ Usually used in FCN head of HRNet.
+ - 'multiple_select': Multiple feature maps will be bundle into
+ a list and passed into decode head.
+ - None: Only one select feature map is allowed.
+ align_corners (bool): align_corners argument of F.interpolate.
+ Default: False.
+ loss_keypoint (dict): Config for keypoint loss. Default: None.
+ """
+
+ def __init__(self,
+ in_channels,
+ out_channels,
+ num_deconv_layers=3,
+ num_deconv_filters=(144, 144, 144),
+ num_deconv_kernels=(4, 4, 4),
+ num_deconv_groups=(16, 16, 16),
+ extra=None,
+ in_index=0,
+ input_transform=None,
+ align_corners=False,
+ loss_keypoint=None,
+ train_cfg=None,
+ test_cfg=None):
+ super().__init__()
+
+ self.in_channels = in_channels
+ self.loss = build_loss(loss_keypoint)
+
+ self.train_cfg = {} if train_cfg is None else train_cfg
+ self.test_cfg = {} if test_cfg is None else test_cfg
+ self.target_type = self.test_cfg.get('target_type', 'GaussianHeatmap')
+
+ self._init_inputs(in_channels, in_index, input_transform)
+ self.in_index = in_index
+ self.align_corners = align_corners
+
+ if extra is not None and not isinstance(extra, dict):
+ raise TypeError('extra should be dict or None.')
+
+ if num_deconv_layers > 0:
+ self.deconv_layers = self._make_deconv_layer(
+ num_deconv_layers, num_deconv_filters, num_deconv_kernels,
+ num_deconv_groups)
+ elif num_deconv_layers == 0:
+ self.deconv_layers = nn.Identity()
+ else:
+ raise ValueError(
+ f'num_deconv_layers ({num_deconv_layers}) should >= 0.')
+
+ identity_final_layer = False
+ if extra is not None and 'final_conv_kernel' in extra:
+ assert extra['final_conv_kernel'] in [0, 1, 3]
+ if extra['final_conv_kernel'] == 3:
+ padding = 1
+ elif extra['final_conv_kernel'] == 1:
+ padding = 0
+ else:
+ # 0 for Identity mapping.
+ identity_final_layer = True
+ kernel_size = extra['final_conv_kernel']
+ else:
+ kernel_size = 1
+ padding = 0
+
+ if identity_final_layer:
+ self.final_layer = nn.Identity()
+ else:
+ conv_channels = num_deconv_filters[
+ -1] if num_deconv_layers > 0 else self.in_channels
+
+ layers = []
+ if extra is not None:
+ num_conv_layers = extra.get('num_conv_layers', 0)
+ num_conv_kernels = extra.get('num_conv_kernels',
+ [1] * num_conv_layers)
+
+ for i in range(num_conv_layers):
+ layers.append(
+ build_conv_layer(
+ dict(type='Conv2d'),
+ in_channels=conv_channels,
+ out_channels=conv_channels,
+ kernel_size=num_conv_kernels[i],
+ stride=1,
+ padding=(num_conv_kernels[i] - 1) // 2))
+ layers.append(
+ build_norm_layer(dict(type='BN'), conv_channels)[1])
+ layers.append(nn.ReLU(inplace=True))
+
+ layers.append(
+ build_conv_layer(
+ cfg=dict(type='Conv2d'),
+ in_channels=conv_channels,
+ out_channels=out_channels,
+ kernel_size=kernel_size,
+ stride=1,
+ padding=padding))
+
+ if len(layers) > 1:
+ self.final_layer = nn.Sequential(*layers)
+ else:
+ self.final_layer = layers[0]
+
+ def get_loss(self, output, target, target_weight):
+ """Calculate top-down keypoint loss.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+ - heatmaps height: H
+ - heatmaps weight: W
+
+ Args:
+ output (torch.Tensor[N,K,H,W]): Output heatmaps.
+ target (torch.Tensor[N,K,H,W]): Target heatmaps.
+ target_weight (torch.Tensor[N,K,1]):
+ Weights across different joint types.
+ """
+
+ losses = dict()
+
+ assert not isinstance(self.loss, nn.Sequential)
+ assert target.dim() == 4 and target_weight.dim() == 3
+ losses['heatmap_loss'] = self.loss(output, target, target_weight)
+
+ return losses
+
+ def get_accuracy(self, output, target, target_weight):
+ """Calculate accuracy for top-down keypoint loss.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+ - heatmaps height: H
+ - heatmaps weight: W
+
+ Args:
+ output (torch.Tensor[N,K,H,W]): Output heatmaps.
+ target (torch.Tensor[N,K,H,W]): Target heatmaps.
+ target_weight (torch.Tensor[N,K,1]):
+ Weights across different joint types.
+ """
+
+ accuracy = dict()
+
+ if self.target_type.lower() == 'GaussianHeatmap'.lower():
+ _, avg_acc, _ = pose_pck_accuracy(
+ output.detach().cpu().numpy(),
+ target.detach().cpu().numpy(),
+ target_weight.detach().cpu().numpy().squeeze(-1) > 0)
+ accuracy['acc_pose'] = float(avg_acc)
+
+ return accuracy
+
+ def forward(self, x):
+ """Forward function."""
+ x = self._transform_inputs(x)
+ x = self.deconv_layers(x)
+ x = self.final_layer(x)
+ return x
+
+ def inference_model(self, x, flip_pairs=None):
+ """Inference function.
+
+ Returns:
+ output_heatmap (np.ndarray): Output heatmaps.
+
+ Args:
+ x (torch.Tensor[N,K,H,W]): Input features.
+ flip_pairs (None | list[tuple]):
+ Pairs of keypoints which are mirrored.
+ """
+ output = self.forward(x)
+
+ if flip_pairs is not None:
+ output_heatmap = flip_back(
+ output.detach().cpu().numpy(),
+ flip_pairs,
+ target_type=self.target_type)
+ # feature is not aligned, shift flipped heatmap for higher accuracy
+ if self.test_cfg.get('shift_heatmap', False):
+ output_heatmap[:, :, :, 1:] = output_heatmap[:, :, :, :-1]
+ else:
+ output_heatmap = output.detach().cpu().numpy()
+ return output_heatmap
+
+ def _init_inputs(self, in_channels, in_index, input_transform):
+ """Check and initialize input transforms.
+
+ The in_channels, in_index and input_transform must match.
+ Specifically, when input_transform is None, only single feature map
+ will be selected. So in_channels and in_index must be of type int.
+ When input_transform is not None, in_channels and in_index must be
+ list or tuple, with the same length.
+
+ Args:
+ in_channels (int|Sequence[int]): Input channels.
+ in_index (int|Sequence[int]): Input feature index.
+ input_transform (str|None): Transformation type of input features.
+ Options: 'resize_concat', 'multiple_select', None.
+
+ - 'resize_concat': Multiple feature maps will be resize to the
+ same size as first one and than concat together.
+ Usually used in FCN head of HRNet.
+ - 'multiple_select': Multiple feature maps will be bundle into
+ a list and passed into decode head.
+ - None: Only one select feature map is allowed.
+ """
+
+ if input_transform is not None:
+ assert input_transform in ['resize_concat', 'multiple_select']
+ self.input_transform = input_transform
+ self.in_index = in_index
+ if input_transform is not None:
+ assert isinstance(in_channels, (list, tuple))
+ assert isinstance(in_index, (list, tuple))
+ assert len(in_channels) == len(in_index)
+ if input_transform == 'resize_concat':
+ self.in_channels = sum(in_channels)
+ else:
+ self.in_channels = in_channels
+ else:
+ assert isinstance(in_channels, int)
+ assert isinstance(in_index, int)
+ self.in_channels = in_channels
+
+ def _transform_inputs(self, inputs):
+ """Transform inputs for decoder.
+
+ Args:
+ inputs (list[Tensor] | Tensor): multi-level img features.
+
+ Returns:
+ Tensor: The transformed inputs
+ """
+ if not isinstance(inputs, list):
+ return inputs
+
+ if self.input_transform == 'resize_concat':
+ inputs = [inputs[i] for i in self.in_index]
+ upsampled_inputs = [
+ resize(
+ input=x,
+ size=inputs[0].shape[2:],
+ mode='bilinear',
+ align_corners=self.align_corners) for x in inputs
+ ]
+ inputs = torch.cat(upsampled_inputs, dim=1)
+ elif self.input_transform == 'multiple_select':
+ inputs = [inputs[i] for i in self.in_index]
+ else:
+ inputs = inputs[self.in_index]
+
+ return inputs
+
+ def _make_deconv_layer(self, num_layers, num_filters, num_kernels,
+ num_groups):
+ """Make deconv layers."""
+ if num_layers != len(num_filters):
+ error_msg = f'num_layers({num_layers}) ' \
+ f'!= length of num_filters({len(num_filters)})'
+ raise ValueError(error_msg)
+ if num_layers != len(num_kernels):
+ error_msg = f'num_layers({num_layers}) ' \
+ f'!= length of num_kernels({len(num_kernels)})'
+ raise ValueError(error_msg)
+ if num_layers != len(num_groups):
+ error_msg = f'num_layers({num_layers}) ' \
+ f'!= length of num_groups({len(num_groups)})'
+ raise ValueError(error_msg)
+
+ layers = []
+ for i in range(num_layers):
+ kernel, padding, output_padding = \
+ self._get_deconv_cfg(num_kernels[i])
+
+ planes = num_filters[i]
+ groups = num_groups[i]
+ layers.append(
+ build_upsample_layer(
+ dict(type='deconv'),
+ in_channels=self.in_channels,
+ out_channels=planes,
+ kernel_size=kernel,
+ groups=groups,
+ stride=2,
+ padding=padding,
+ output_padding=output_padding,
+ bias=False))
+ layers.append(nn.BatchNorm2d(planes))
+ layers.append(nn.ReLU(inplace=True))
+ self.in_channels = planes
+
+ return nn.Sequential(*layers)
+
+ def init_weights(self):
+ """Initialize model weights."""
+ for _, m in self.deconv_layers.named_modules():
+ if isinstance(m, nn.ConvTranspose2d):
+ normal_init(m, std=0.001)
+ elif isinstance(m, nn.BatchNorm2d):
+ constant_init(m, 1)
+ for m in self.final_layer.modules():
+ if isinstance(m, nn.Conv2d):
+ normal_init(m, std=0.001, bias=0)
+ elif isinstance(m, nn.BatchNorm2d):
+ constant_init(m, 1)
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/heads/voxelpose_head.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/heads/voxelpose_head.py
new file mode 100644
index 0000000..8799bdc
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/heads/voxelpose_head.py
@@ -0,0 +1,167 @@
+# ------------------------------------------------------------------------------
+# Copyright and License Information
+# https://github.com/microsoft/voxelpose-pytorch/blob/main/lib/models
+# Original Licence: MIT License
+# ------------------------------------------------------------------------------
+
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
+from ..builder import HEADS
+
+
+@HEADS.register_module()
+class CuboidCenterHead(nn.Module):
+ """Get results from the 3D human center heatmap. In this module, human 3D
+ centers are local maximums obtained from the 3D heatmap via NMS (max-
+ pooling).
+
+ Args:
+ space_size (list[3]): The size of the 3D space.
+ cube_size (list[3]): The size of the heatmap volume.
+ space_center (list[3]): The coordinate of space center.
+ max_num (int): Maximum of human center detections.
+ max_pool_kernel (int): Kernel size of the max-pool kernel in nms.
+ """
+
+ def __init__(self,
+ space_size,
+ space_center,
+ cube_size,
+ max_num=10,
+ max_pool_kernel=3):
+ super(CuboidCenterHead, self).__init__()
+ # use register_buffer
+ self.register_buffer('grid_size', torch.tensor(space_size))
+ self.register_buffer('cube_size', torch.tensor(cube_size))
+ self.register_buffer('grid_center', torch.tensor(space_center))
+
+ self.num_candidates = max_num
+ self.max_pool_kernel = max_pool_kernel
+ self.loss = nn.MSELoss()
+
+ def _get_real_locations(self, indices):
+ """
+ Args:
+ indices (torch.Tensor(NXP)): Indices of points in the 3D tensor
+
+ Returns:
+ real_locations (torch.Tensor(NXPx3)): Locations of points
+ in the world coordinate system
+ """
+ real_locations = indices.float() / (
+ self.cube_size - 1) * self.grid_size + \
+ self.grid_center - self.grid_size / 2.0
+ return real_locations
+
+ def _nms_by_max_pool(self, heatmap_volumes):
+ max_num = self.num_candidates
+ batch_size = heatmap_volumes.shape[0]
+ root_cubes_nms = self._max_pool(heatmap_volumes)
+ root_cubes_nms_reshape = root_cubes_nms.reshape(batch_size, -1)
+ topk_values, topk_index = root_cubes_nms_reshape.topk(max_num)
+ topk_unravel_index = self._get_3d_indices(topk_index,
+ heatmap_volumes[0].shape)
+
+ return topk_values, topk_unravel_index
+
+ def _max_pool(self, inputs):
+ kernel = self.max_pool_kernel
+ padding = (kernel - 1) // 2
+ max = F.max_pool3d(
+ inputs, kernel_size=kernel, stride=1, padding=padding)
+ keep = (inputs == max).float()
+ return keep * inputs
+
+ @staticmethod
+ def _get_3d_indices(indices, shape):
+ """Get indices in the 3-D tensor.
+
+ Args:
+ indices (torch.Tensor(NXp)): Indices of points in the 1D tensor
+ shape (torch.Size(3)): The shape of the original 3D tensor
+
+ Returns:
+ indices: Indices of points in the original 3D tensor
+ """
+ batch_size = indices.shape[0]
+ num_people = indices.shape[1]
+ indices_x = (indices //
+ (shape[1] * shape[2])).reshape(batch_size, num_people, -1)
+ indices_y = ((indices % (shape[1] * shape[2])) //
+ shape[2]).reshape(batch_size, num_people, -1)
+ indices_z = (indices % shape[2]).reshape(batch_size, num_people, -1)
+ indices = torch.cat([indices_x, indices_y, indices_z], dim=2)
+ return indices
+
+ def forward(self, heatmap_volumes):
+ """
+
+ Args:
+ heatmap_volumes (torch.Tensor(NXLXWXH)):
+ 3D human center heatmaps predicted by the network.
+ Returns:
+ human_centers (torch.Tensor(NXPX5)):
+ Coordinates of human centers.
+ """
+ batch_size = heatmap_volumes.shape[0]
+
+ topk_values, topk_unravel_index = self._nms_by_max_pool(
+ heatmap_volumes.detach())
+
+ topk_unravel_index = self._get_real_locations(topk_unravel_index)
+
+ human_centers = torch.zeros(
+ batch_size, self.num_candidates, 5, device=heatmap_volumes.device)
+ human_centers[:, :, 0:3] = topk_unravel_index
+ human_centers[:, :, 4] = topk_values
+
+ return human_centers
+
+ def get_loss(self, pred_cubes, gt):
+
+ return dict(loss_center=self.loss(pred_cubes, gt))
+
+
+@HEADS.register_module()
+class CuboidPoseHead(nn.Module):
+
+ def __init__(self, beta):
+ """Get results from the 3D human pose heatmap. Instead of obtaining
+ maximums on the heatmap, this module regresses the coordinates of
+ keypoints via integral pose regression. Refer to `paper.
+
+ ` for more details.
+
+ Args:
+ beta: Constant to adjust the magnification of soft-maxed heatmap.
+ """
+ super(CuboidPoseHead, self).__init__()
+ self.beta = beta
+ self.loss = nn.L1Loss()
+
+ def forward(self, heatmap_volumes, grid_coordinates):
+ """
+
+ Args:
+ heatmap_volumes (torch.Tensor(NxKxLxWxH)):
+ 3D human pose heatmaps predicted by the network.
+ grid_coordinates (torch.Tensor(Nx(LxWxH)x3)):
+ Coordinates of the grids in the heatmap volumes.
+ Returns:
+ human_poses (torch.Tensor(NxKx3)): Coordinates of human poses.
+ """
+ batch_size = heatmap_volumes.size(0)
+ channel = heatmap_volumes.size(1)
+ x = heatmap_volumes.reshape(batch_size, channel, -1, 1)
+ x = F.softmax(self.beta * x, dim=2)
+ grid_coordinates = grid_coordinates.unsqueeze(1)
+ x = torch.mul(x, grid_coordinates)
+ human_poses = torch.sum(x, dim=2)
+
+ return human_poses
+
+ def get_loss(self, preds, targets, weights):
+
+ return dict(loss_pose=self.loss(preds * weights, targets * weights))
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/losses/__init__.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/losses/__init__.py
new file mode 100644
index 0000000..d67973f
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/losses/__init__.py
@@ -0,0 +1,16 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from .classfication_loss import BCELoss
+from .heatmap_loss import AdaptiveWingLoss
+from .mesh_loss import GANLoss, MeshLoss
+from .mse_loss import JointsMSELoss, JointsOHKMMSELoss
+from .multi_loss_factory import AELoss, HeatmapLoss, MultiLossFactory
+from .regression_loss import (BoneLoss, L1Loss, MPJPELoss, MSELoss,
+ SemiSupervisionLoss, SmoothL1Loss, SoftWingLoss,
+ WingLoss)
+
+__all__ = [
+ 'JointsMSELoss', 'JointsOHKMMSELoss', 'HeatmapLoss', 'AELoss',
+ 'MultiLossFactory', 'MeshLoss', 'GANLoss', 'SmoothL1Loss', 'WingLoss',
+ 'MPJPELoss', 'MSELoss', 'L1Loss', 'BCELoss', 'BoneLoss',
+ 'SemiSupervisionLoss', 'SoftWingLoss', 'AdaptiveWingLoss'
+]
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/losses/classfication_loss.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/losses/classfication_loss.py
new file mode 100644
index 0000000..b79b69d
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/losses/classfication_loss.py
@@ -0,0 +1,41 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import torch.nn as nn
+import torch.nn.functional as F
+
+from ..builder import LOSSES
+
+
+@LOSSES.register_module()
+class BCELoss(nn.Module):
+ """Binary Cross Entropy loss."""
+
+ def __init__(self, use_target_weight=False, loss_weight=1.):
+ super().__init__()
+ self.criterion = F.binary_cross_entropy
+ self.use_target_weight = use_target_weight
+ self.loss_weight = loss_weight
+
+ def forward(self, output, target, target_weight=None):
+ """Forward function.
+
+ Note:
+ - batch_size: N
+ - num_labels: K
+
+ Args:
+ output (torch.Tensor[N, K]): Output classification.
+ target (torch.Tensor[N, K]): Target classification.
+ target_weight (torch.Tensor[N, K] or torch.Tensor[N]):
+ Weights across different labels.
+ """
+
+ if self.use_target_weight:
+ assert target_weight is not None
+ loss = self.criterion(output, target, reduction='none')
+ if target_weight.dim() == 1:
+ target_weight = target_weight[:, None]
+ loss = (loss * target_weight).mean()
+ else:
+ loss = self.criterion(output, target)
+
+ return loss * self.loss_weight
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/losses/heatmap_loss.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/losses/heatmap_loss.py
new file mode 100644
index 0000000..9471457
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/losses/heatmap_loss.py
@@ -0,0 +1,86 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import torch
+import torch.nn as nn
+
+from ..builder import LOSSES
+
+
+@LOSSES.register_module()
+class AdaptiveWingLoss(nn.Module):
+ """Adaptive wing loss. paper ref: 'Adaptive Wing Loss for Robust Face
+ Alignment via Heatmap Regression' Wang et al. ICCV'2019.
+
+ Args:
+ alpha (float), omega (float), epsilon (float), theta (float)
+ are hyper-parameters.
+ use_target_weight (bool): Option to use weighted MSE loss.
+ Different joint types may have different target weights.
+ loss_weight (float): Weight of the loss. Default: 1.0.
+ """
+
+ def __init__(self,
+ alpha=2.1,
+ omega=14,
+ epsilon=1,
+ theta=0.5,
+ use_target_weight=False,
+ loss_weight=1.):
+ super().__init__()
+ self.alpha = float(alpha)
+ self.omega = float(omega)
+ self.epsilon = float(epsilon)
+ self.theta = float(theta)
+ self.use_target_weight = use_target_weight
+ self.loss_weight = loss_weight
+
+ def criterion(self, pred, target):
+ """Criterion of wingloss.
+
+ Note:
+ batch_size: N
+ num_keypoints: K
+
+ Args:
+ pred (torch.Tensor[NxKxHxW]): Predicted heatmaps.
+ target (torch.Tensor[NxKxHxW]): Target heatmaps.
+ """
+ H, W = pred.shape[2:4]
+ delta = (target - pred).abs()
+
+ A = self.omega * (
+ 1 / (1 + torch.pow(self.theta / self.epsilon, self.alpha - target))
+ ) * (self.alpha - target) * (torch.pow(
+ self.theta / self.epsilon,
+ self.alpha - target - 1)) * (1 / self.epsilon)
+ C = self.theta * A - self.omega * torch.log(
+ 1 + torch.pow(self.theta / self.epsilon, self.alpha - target))
+
+ losses = torch.where(
+ delta < self.theta,
+ self.omega *
+ torch.log(1 +
+ torch.pow(delta / self.epsilon, self.alpha - target)),
+ A * delta - C)
+
+ return torch.mean(losses)
+
+ def forward(self, output, target, target_weight):
+ """Forward function.
+
+ Note:
+ batch_size: N
+ num_keypoints: K
+
+ Args:
+ output (torch.Tensor[NxKxHxW]): Output heatmaps.
+ target (torch.Tensor[NxKxHxW]): Target heatmaps.
+ target_weight (torch.Tensor[NxKx1]):
+ Weights across different joint types.
+ """
+ if self.use_target_weight:
+ loss = self.criterion(output * target_weight.unsqueeze(-1),
+ target * target_weight.unsqueeze(-1))
+ else:
+ loss = self.criterion(output, target)
+
+ return loss * self.loss_weight
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/losses/mesh_loss.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/losses/mesh_loss.py
new file mode 100644
index 0000000..f9d18bd
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/losses/mesh_loss.py
@@ -0,0 +1,340 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import torch
+import torch.nn as nn
+
+from ..builder import LOSSES
+from ..utils.geometry import batch_rodrigues
+
+
+def perspective_projection(points, rotation, translation, focal_length,
+ camera_center):
+ """This function computes the perspective projection of a set of 3D points.
+
+ Note:
+ - batch size: B
+ - point number: N
+
+ Args:
+ points (Tensor([B, N, 3])): A set of 3D points
+ rotation (Tensor([B, 3, 3])): Camera rotation matrix
+ translation (Tensor([B, 3])): Camera translation
+ focal_length (Tensor([B,])): Focal length
+ camera_center (Tensor([B, 2])): Camera center
+
+ Returns:
+ projected_points (Tensor([B, N, 2])): Projected 2D
+ points in image space.
+ """
+
+ batch_size = points.shape[0]
+ K = torch.zeros([batch_size, 3, 3], device=points.device)
+ K[:, 0, 0] = focal_length
+ K[:, 1, 1] = focal_length
+ K[:, 2, 2] = 1.
+ K[:, :-1, -1] = camera_center
+
+ # Transform points
+ points = torch.einsum('bij,bkj->bki', rotation, points)
+ points = points + translation.unsqueeze(1)
+
+ # Apply perspective distortion
+ projected_points = points / points[:, :, -1].unsqueeze(-1)
+
+ # Apply camera intrinsics
+ projected_points = torch.einsum('bij,bkj->bki', K, projected_points)
+ projected_points = projected_points[:, :, :-1]
+ return projected_points
+
+
+@LOSSES.register_module()
+class MeshLoss(nn.Module):
+ """Mix loss for 3D human mesh. It is composed of loss on 2D joints, 3D
+ joints, mesh vertices and smpl parameters (if any).
+
+ Args:
+ joints_2d_loss_weight (float): Weight for loss on 2D joints.
+ joints_3d_loss_weight (float): Weight for loss on 3D joints.
+ vertex_loss_weight (float): Weight for loss on 3D verteices.
+ smpl_pose_loss_weight (float): Weight for loss on SMPL
+ pose parameters.
+ smpl_beta_loss_weight (float): Weight for loss on SMPL
+ shape parameters.
+ img_res (int): Input image resolution.
+ focal_length (float): Focal length of camera model. Default=5000.
+ """
+
+ def __init__(self,
+ joints_2d_loss_weight,
+ joints_3d_loss_weight,
+ vertex_loss_weight,
+ smpl_pose_loss_weight,
+ smpl_beta_loss_weight,
+ img_res,
+ focal_length=5000):
+
+ super().__init__()
+ # Per-vertex loss on the mesh
+ self.criterion_vertex = nn.L1Loss(reduction='none')
+
+ # Joints (2D and 3D) loss
+ self.criterion_joints_2d = nn.SmoothL1Loss(reduction='none')
+ self.criterion_joints_3d = nn.SmoothL1Loss(reduction='none')
+
+ # Loss for SMPL parameter regression
+ self.criterion_regr = nn.MSELoss(reduction='none')
+
+ self.joints_2d_loss_weight = joints_2d_loss_weight
+ self.joints_3d_loss_weight = joints_3d_loss_weight
+ self.vertex_loss_weight = vertex_loss_weight
+ self.smpl_pose_loss_weight = smpl_pose_loss_weight
+ self.smpl_beta_loss_weight = smpl_beta_loss_weight
+ self.focal_length = focal_length
+ self.img_res = img_res
+
+ def joints_2d_loss(self, pred_joints_2d, gt_joints_2d, joints_2d_visible):
+ """Compute 2D reprojection loss on the joints.
+
+ The loss is weighted by joints_2d_visible.
+ """
+ conf = joints_2d_visible.float()
+ loss = (conf *
+ self.criterion_joints_2d(pred_joints_2d, gt_joints_2d)).mean()
+ return loss
+
+ def joints_3d_loss(self, pred_joints_3d, gt_joints_3d, joints_3d_visible):
+ """Compute 3D joints loss for the examples that 3D joint annotations
+ are available.
+
+ The loss is weighted by joints_3d_visible.
+ """
+ conf = joints_3d_visible.float()
+ if len(gt_joints_3d) > 0:
+ gt_pelvis = (gt_joints_3d[:, 2, :] + gt_joints_3d[:, 3, :]) / 2
+ gt_joints_3d = gt_joints_3d - gt_pelvis[:, None, :]
+ pred_pelvis = (pred_joints_3d[:, 2, :] +
+ pred_joints_3d[:, 3, :]) / 2
+ pred_joints_3d = pred_joints_3d - pred_pelvis[:, None, :]
+ return (
+ conf *
+ self.criterion_joints_3d(pred_joints_3d, gt_joints_3d)).mean()
+ return pred_joints_3d.sum() * 0
+
+ def vertex_loss(self, pred_vertices, gt_vertices, has_smpl):
+ """Compute 3D vertex loss for the examples that 3D human mesh
+ annotations are available.
+
+ The loss is weighted by the has_smpl.
+ """
+ conf = has_smpl.float()
+ loss_vertex = self.criterion_vertex(pred_vertices, gt_vertices)
+ loss_vertex = (conf[:, None, None] * loss_vertex).mean()
+ return loss_vertex
+
+ def smpl_losses(self, pred_rotmat, pred_betas, gt_pose, gt_betas,
+ has_smpl):
+ """Compute SMPL parameters loss for the examples that SMPL parameter
+ annotations are available.
+
+ The loss is weighted by has_smpl.
+ """
+ conf = has_smpl.float()
+ gt_rotmat = batch_rodrigues(gt_pose.view(-1, 3)).view(-1, 24, 3, 3)
+ loss_regr_pose = self.criterion_regr(pred_rotmat, gt_rotmat)
+ loss_regr_betas = self.criterion_regr(pred_betas, gt_betas)
+ loss_regr_pose = (conf[:, None, None, None] * loss_regr_pose).mean()
+ loss_regr_betas = (conf[:, None] * loss_regr_betas).mean()
+ return loss_regr_pose, loss_regr_betas
+
+ def project_points(self, points_3d, camera):
+ """Perform orthographic projection of 3D points using the camera
+ parameters, return projected 2D points in image plane.
+
+ Note:
+ - batch size: B
+ - point number: N
+
+ Args:
+ points_3d (Tensor([B, N, 3])): 3D points.
+ camera (Tensor([B, 3])): camera parameters with the
+ 3 channel as (scale, translation_x, translation_y)
+
+ Returns:
+ Tensor([B, N, 2]): projected 2D points \
+ in image space.
+ """
+ batch_size = points_3d.shape[0]
+ device = points_3d.device
+ cam_t = torch.stack([
+ camera[:, 1], camera[:, 2], 2 * self.focal_length /
+ (self.img_res * camera[:, 0] + 1e-9)
+ ],
+ dim=-1)
+ camera_center = camera.new_zeros([batch_size, 2])
+ rot_t = torch.eye(
+ 3, device=device,
+ dtype=points_3d.dtype).unsqueeze(0).expand(batch_size, -1, -1)
+ joints_2d = perspective_projection(
+ points_3d,
+ rotation=rot_t,
+ translation=cam_t,
+ focal_length=self.focal_length,
+ camera_center=camera_center)
+ return joints_2d
+
+ def forward(self, output, target):
+ """Forward function.
+
+ Args:
+ output (dict): dict of network predicted results.
+ Keys: 'vertices', 'joints_3d', 'camera',
+ 'pose'(optional), 'beta'(optional)
+ target (dict): dict of ground-truth labels.
+ Keys: 'vertices', 'joints_3d', 'joints_3d_visible',
+ 'joints_2d', 'joints_2d_visible', 'pose', 'beta',
+ 'has_smpl'
+
+ Returns:
+ dict: dict of losses.
+ """
+ losses = {}
+
+ # Per-vertex loss for the shape
+ pred_vertices = output['vertices']
+
+ gt_vertices = target['vertices']
+ has_smpl = target['has_smpl']
+ loss_vertex = self.vertex_loss(pred_vertices, gt_vertices, has_smpl)
+ losses['vertex_loss'] = loss_vertex * self.vertex_loss_weight
+
+ # Compute loss on SMPL parameters, if available
+ if 'pose' in output.keys() and 'beta' in output.keys():
+ pred_rotmat = output['pose']
+ pred_betas = output['beta']
+ gt_pose = target['pose']
+ gt_betas = target['beta']
+ loss_regr_pose, loss_regr_betas = self.smpl_losses(
+ pred_rotmat, pred_betas, gt_pose, gt_betas, has_smpl)
+ losses['smpl_pose_loss'] = \
+ loss_regr_pose * self.smpl_pose_loss_weight
+ losses['smpl_beta_loss'] = \
+ loss_regr_betas * self.smpl_beta_loss_weight
+
+ # Compute 3D joints loss
+ pred_joints_3d = output['joints_3d']
+ gt_joints_3d = target['joints_3d']
+ joints_3d_visible = target['joints_3d_visible']
+ loss_joints_3d = self.joints_3d_loss(pred_joints_3d, gt_joints_3d,
+ joints_3d_visible)
+ losses['joints_3d_loss'] = loss_joints_3d * self.joints_3d_loss_weight
+
+ # Compute 2D reprojection loss for the 2D joints
+ pred_camera = output['camera']
+ gt_joints_2d = target['joints_2d']
+ joints_2d_visible = target['joints_2d_visible']
+ pred_joints_2d = self.project_points(pred_joints_3d, pred_camera)
+
+ # Normalize keypoints to [-1,1]
+ # The coordinate origin of pred_joints_2d is
+ # the center of the input image.
+ pred_joints_2d = 2 * pred_joints_2d / (self.img_res - 1)
+ # The coordinate origin of gt_joints_2d is
+ # the top left corner of the input image.
+ gt_joints_2d = 2 * gt_joints_2d / (self.img_res - 1) - 1
+ loss_joints_2d = self.joints_2d_loss(pred_joints_2d, gt_joints_2d,
+ joints_2d_visible)
+ losses['joints_2d_loss'] = loss_joints_2d * self.joints_2d_loss_weight
+
+ return losses
+
+
+@LOSSES.register_module()
+class GANLoss(nn.Module):
+ """Define GAN loss.
+
+ Args:
+ gan_type (str): Support 'vanilla', 'lsgan', 'wgan', 'hinge'.
+ real_label_val (float): The value for real label. Default: 1.0.
+ fake_label_val (float): The value for fake label. Default: 0.0.
+ loss_weight (float): Loss weight. Default: 1.0.
+ Note that loss_weight is only for generators; and it is always 1.0
+ for discriminators.
+ """
+
+ def __init__(self,
+ gan_type,
+ real_label_val=1.0,
+ fake_label_val=0.0,
+ loss_weight=1.0):
+ super().__init__()
+ self.gan_type = gan_type
+ self.loss_weight = loss_weight
+ self.real_label_val = real_label_val
+ self.fake_label_val = fake_label_val
+
+ if self.gan_type == 'vanilla':
+ self.loss = nn.BCEWithLogitsLoss()
+ elif self.gan_type == 'lsgan':
+ self.loss = nn.MSELoss()
+ elif self.gan_type == 'wgan':
+ self.loss = self._wgan_loss
+ elif self.gan_type == 'hinge':
+ self.loss = nn.ReLU()
+ else:
+ raise NotImplementedError(
+ f'GAN type {self.gan_type} is not implemented.')
+
+ @staticmethod
+ def _wgan_loss(input, target):
+ """wgan loss.
+
+ Args:
+ input (Tensor): Input tensor.
+ target (bool): Target label.
+
+ Returns:
+ Tensor: wgan loss.
+ """
+ return -input.mean() if target else input.mean()
+
+ def get_target_label(self, input, target_is_real):
+ """Get target label.
+
+ Args:
+ input (Tensor): Input tensor.
+ target_is_real (bool): Whether the target is real or fake.
+
+ Returns:
+ (bool | Tensor): Target tensor. Return bool for wgan, \
+ otherwise, return Tensor.
+ """
+
+ if self.gan_type == 'wgan':
+ return target_is_real
+ target_val = (
+ self.real_label_val if target_is_real else self.fake_label_val)
+ return input.new_ones(input.size()) * target_val
+
+ def forward(self, input, target_is_real, is_disc=False):
+ """
+ Args:
+ input (Tensor): The input for the loss module, i.e., the network
+ prediction.
+ target_is_real (bool): Whether the targe is real or fake.
+ is_disc (bool): Whether the loss for discriminators or not.
+ Default: False.
+
+ Returns:
+ Tensor: GAN loss value.
+ """
+ target_label = self.get_target_label(input, target_is_real)
+ if self.gan_type == 'hinge':
+ if is_disc: # for discriminators in hinge-gan
+ input = -input if target_is_real else input
+ loss = self.loss(1 + input).mean()
+ else: # for generators in hinge-gan
+ loss = -input.mean()
+ else: # other gan types
+ loss = self.loss(input, target_label)
+
+ # loss_weight is always 1.0 for discriminators
+ return loss if is_disc else loss * self.loss_weight
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/losses/mse_loss.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/losses/mse_loss.py
new file mode 100644
index 0000000..f972efa
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/losses/mse_loss.py
@@ -0,0 +1,153 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import torch
+import torch.nn as nn
+
+from ..builder import LOSSES
+
+
+@LOSSES.register_module()
+class JointsMSELoss(nn.Module):
+ """MSE loss for heatmaps.
+
+ Args:
+ use_target_weight (bool): Option to use weighted MSE loss.
+ Different joint types may have different target weights.
+ loss_weight (float): Weight of the loss. Default: 1.0.
+ """
+
+ def __init__(self, use_target_weight=False, loss_weight=1.):
+ super().__init__()
+ self.criterion = nn.MSELoss()
+ self.use_target_weight = use_target_weight
+ self.loss_weight = loss_weight
+
+ def forward(self, output, target, target_weight):
+ """Forward function."""
+ batch_size = output.size(0)
+ num_joints = output.size(1)
+
+ heatmaps_pred = output.reshape(
+ (batch_size, num_joints, -1)).split(1, 1)
+ heatmaps_gt = target.reshape((batch_size, num_joints, -1)).split(1, 1)
+
+ loss = 0.
+
+ for idx in range(num_joints):
+ heatmap_pred = heatmaps_pred[idx].squeeze(1)
+ heatmap_gt = heatmaps_gt[idx].squeeze(1)
+ if self.use_target_weight:
+ loss += self.criterion(heatmap_pred * target_weight[:, idx],
+ heatmap_gt * target_weight[:, idx])
+ else:
+ loss += self.criterion(heatmap_pred, heatmap_gt)
+
+ return loss / num_joints * self.loss_weight
+
+
+@LOSSES.register_module()
+class CombinedTargetMSELoss(nn.Module):
+ """MSE loss for combined target.
+ CombinedTarget: The combination of classification target
+ (response map) and regression target (offset map).
+ Paper ref: Huang et al. The Devil is in the Details: Delving into
+ Unbiased Data Processing for Human Pose Estimation (CVPR 2020).
+
+ Args:
+ use_target_weight (bool): Option to use weighted MSE loss.
+ Different joint types may have different target weights.
+ loss_weight (float): Weight of the loss. Default: 1.0.
+ """
+
+ def __init__(self, use_target_weight, loss_weight=1.):
+ super().__init__()
+ self.criterion = nn.MSELoss(reduction='mean')
+ self.use_target_weight = use_target_weight
+ self.loss_weight = loss_weight
+
+ def forward(self, output, target, target_weight):
+ batch_size = output.size(0)
+ num_channels = output.size(1)
+ heatmaps_pred = output.reshape(
+ (batch_size, num_channels, -1)).split(1, 1)
+ heatmaps_gt = target.reshape(
+ (batch_size, num_channels, -1)).split(1, 1)
+ loss = 0.
+ num_joints = num_channels // 3
+ for idx in range(num_joints):
+ heatmap_pred = heatmaps_pred[idx * 3].squeeze()
+ heatmap_gt = heatmaps_gt[idx * 3].squeeze()
+ offset_x_pred = heatmaps_pred[idx * 3 + 1].squeeze()
+ offset_x_gt = heatmaps_gt[idx * 3 + 1].squeeze()
+ offset_y_pred = heatmaps_pred[idx * 3 + 2].squeeze()
+ offset_y_gt = heatmaps_gt[idx * 3 + 2].squeeze()
+ if self.use_target_weight:
+ heatmap_pred = heatmap_pred * target_weight[:, idx]
+ heatmap_gt = heatmap_gt * target_weight[:, idx]
+ # classification loss
+ loss += 0.5 * self.criterion(heatmap_pred, heatmap_gt)
+ # regression loss
+ loss += 0.5 * self.criterion(heatmap_gt * offset_x_pred,
+ heatmap_gt * offset_x_gt)
+ loss += 0.5 * self.criterion(heatmap_gt * offset_y_pred,
+ heatmap_gt * offset_y_gt)
+ return loss / num_joints * self.loss_weight
+
+
+@LOSSES.register_module()
+class JointsOHKMMSELoss(nn.Module):
+ """MSE loss with online hard keypoint mining.
+
+ Args:
+ use_target_weight (bool): Option to use weighted MSE loss.
+ Different joint types may have different target weights.
+ topk (int): Only top k joint losses are kept.
+ loss_weight (float): Weight of the loss. Default: 1.0.
+ """
+
+ def __init__(self, use_target_weight=False, topk=8, loss_weight=1.):
+ super().__init__()
+ assert topk > 0
+ self.criterion = nn.MSELoss(reduction='none')
+ self.use_target_weight = use_target_weight
+ self.topk = topk
+ self.loss_weight = loss_weight
+
+ def _ohkm(self, loss):
+ """Online hard keypoint mining."""
+ ohkm_loss = 0.
+ N = len(loss)
+ for i in range(N):
+ sub_loss = loss[i]
+ _, topk_idx = torch.topk(
+ sub_loss, k=self.topk, dim=0, sorted=False)
+ tmp_loss = torch.gather(sub_loss, 0, topk_idx)
+ ohkm_loss += torch.sum(tmp_loss) / self.topk
+ ohkm_loss /= N
+ return ohkm_loss
+
+ def forward(self, output, target, target_weight):
+ """Forward function."""
+ batch_size = output.size(0)
+ num_joints = output.size(1)
+ if num_joints < self.topk:
+ raise ValueError(f'topk ({self.topk}) should not '
+ f'larger than num_joints ({num_joints}).')
+ heatmaps_pred = output.reshape(
+ (batch_size, num_joints, -1)).split(1, 1)
+ heatmaps_gt = target.reshape((batch_size, num_joints, -1)).split(1, 1)
+
+ losses = []
+ for idx in range(num_joints):
+ heatmap_pred = heatmaps_pred[idx].squeeze(1)
+ heatmap_gt = heatmaps_gt[idx].squeeze(1)
+ if self.use_target_weight:
+ losses.append(
+ self.criterion(heatmap_pred * target_weight[:, idx],
+ heatmap_gt * target_weight[:, idx]))
+ else:
+ losses.append(self.criterion(heatmap_pred, heatmap_gt))
+
+ losses = [loss.mean(dim=1).unsqueeze(dim=1) for loss in losses]
+ losses = torch.cat(losses, dim=1)
+
+ return self._ohkm(losses) * self.loss_weight
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/losses/multi_loss_factory.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/losses/multi_loss_factory.py
new file mode 100644
index 0000000..65f90a7
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/losses/multi_loss_factory.py
@@ -0,0 +1,281 @@
+# ------------------------------------------------------------------------------
+# Adapted from https://github.com/HRNet/HigherHRNet-Human-Pose-Estimation
+# Original licence: Copyright (c) Microsoft, under the MIT License.
+# ------------------------------------------------------------------------------
+
+import torch
+import torch.nn as nn
+
+from ..builder import LOSSES
+
+
+def _make_input(t, requires_grad=False, device=torch.device('cpu')):
+ """Make zero inputs for AE loss.
+
+ Args:
+ t (torch.Tensor): input
+ requires_grad (bool): Option to use requires_grad.
+ device: torch device
+
+ Returns:
+ torch.Tensor: zero input.
+ """
+ inp = torch.autograd.Variable(t, requires_grad=requires_grad)
+ inp = inp.sum()
+ inp = inp.to(device)
+ return inp
+
+
+@LOSSES.register_module()
+class HeatmapLoss(nn.Module):
+ """Accumulate the heatmap loss for each image in the batch.
+
+ Args:
+ supervise_empty (bool): Whether to supervise empty channels.
+ """
+
+ def __init__(self, supervise_empty=True):
+ super().__init__()
+ self.supervise_empty = supervise_empty
+
+ def forward(self, pred, gt, mask):
+ """Forward function.
+
+ Note:
+ - batch_size: N
+ - heatmaps weight: W
+ - heatmaps height: H
+ - max_num_people: M
+ - num_keypoints: K
+
+ Args:
+ pred (torch.Tensor[N,K,H,W]):heatmap of output.
+ gt (torch.Tensor[N,K,H,W]): target heatmap.
+ mask (torch.Tensor[N,H,W]): mask of target.
+ """
+ assert pred.size() == gt.size(
+ ), f'pred.size() is {pred.size()}, gt.size() is {gt.size()}'
+
+ if not self.supervise_empty:
+ empty_mask = (gt.sum(dim=[2, 3], keepdim=True) > 0).float()
+ loss = ((pred - gt)**2) * empty_mask.expand_as(
+ pred) * mask[:, None, :, :].expand_as(pred)
+ else:
+ loss = ((pred - gt)**2) * mask[:, None, :, :].expand_as(pred)
+ loss = loss.mean(dim=3).mean(dim=2).mean(dim=1)
+ return loss
+
+
+@LOSSES.register_module()
+class AELoss(nn.Module):
+ """Associative Embedding loss.
+
+ `Associative Embedding: End-to-End Learning for Joint Detection and
+ Grouping `_.
+ """
+
+ def __init__(self, loss_type):
+ super().__init__()
+ self.loss_type = loss_type
+
+ def singleTagLoss(self, pred_tag, joints):
+ """Associative embedding loss for one image.
+
+ Note:
+ - heatmaps weight: W
+ - heatmaps height: H
+ - max_num_people: M
+ - num_keypoints: K
+
+ Args:
+ pred_tag (torch.Tensor[KxHxW,1]): tag of output for one image.
+ joints (torch.Tensor[M,K,2]): joints information for one image.
+ """
+ tags = []
+ pull = 0
+ for joints_per_person in joints:
+ tmp = []
+ for joint in joints_per_person:
+ if joint[1] > 0:
+ tmp.append(pred_tag[joint[0]])
+ if len(tmp) == 0:
+ continue
+ tmp = torch.stack(tmp)
+ tags.append(torch.mean(tmp, dim=0))
+ pull = pull + torch.mean((tmp - tags[-1].expand_as(tmp))**2)
+
+ num_tags = len(tags)
+ if num_tags == 0:
+ return (
+ _make_input(torch.zeros(1).float(), device=pred_tag.device),
+ _make_input(torch.zeros(1).float(), device=pred_tag.device))
+ elif num_tags == 1:
+ return (_make_input(
+ torch.zeros(1).float(), device=pred_tag.device), pull)
+
+ tags = torch.stack(tags)
+
+ size = (num_tags, num_tags)
+ A = tags.expand(*size)
+ B = A.permute(1, 0)
+
+ diff = A - B
+
+ if self.loss_type == 'exp':
+ diff = torch.pow(diff, 2)
+ push = torch.exp(-diff)
+ push = torch.sum(push) - num_tags
+ elif self.loss_type == 'max':
+ diff = 1 - torch.abs(diff)
+ push = torch.clamp(diff, min=0).sum() - num_tags
+ else:
+ raise ValueError('Unknown ae loss type')
+
+ push_loss = push / ((num_tags - 1) * num_tags) * 0.5
+ pull_loss = pull / (num_tags)
+
+ return push_loss, pull_loss
+
+ def forward(self, tags, joints):
+ """Accumulate the tag loss for each image in the batch.
+
+ Note:
+ - batch_size: N
+ - heatmaps weight: W
+ - heatmaps height: H
+ - max_num_people: M
+ - num_keypoints: K
+
+ Args:
+ tags (torch.Tensor[N,KxHxW,1]): tag channels of output.
+ joints (torch.Tensor[N,M,K,2]): joints information.
+ """
+ pushes, pulls = [], []
+ joints = joints.cpu().data.numpy()
+ batch_size = tags.size(0)
+ for i in range(batch_size):
+ push, pull = self.singleTagLoss(tags[i], joints[i])
+ pushes.append(push)
+ pulls.append(pull)
+ return torch.stack(pushes), torch.stack(pulls)
+
+
+@LOSSES.register_module()
+class MultiLossFactory(nn.Module):
+ """Loss for bottom-up models.
+
+ Args:
+ num_joints (int): Number of keypoints.
+ num_stages (int): Number of stages.
+ ae_loss_type (str): Type of ae loss.
+ with_ae_loss (list[bool]): Use ae loss or not in multi-heatmap.
+ push_loss_factor (list[float]):
+ Parameter of push loss in multi-heatmap.
+ pull_loss_factor (list[float]):
+ Parameter of pull loss in multi-heatmap.
+ with_heatmap_loss (list[bool]):
+ Use heatmap loss or not in multi-heatmap.
+ heatmaps_loss_factor (list[float]):
+ Parameter of heatmap loss in multi-heatmap.
+ supervise_empty (bool): Whether to supervise empty channels.
+ """
+
+ def __init__(self,
+ num_joints,
+ num_stages,
+ ae_loss_type,
+ with_ae_loss,
+ push_loss_factor,
+ pull_loss_factor,
+ with_heatmaps_loss,
+ heatmaps_loss_factor,
+ supervise_empty=True):
+ super().__init__()
+
+ assert isinstance(with_heatmaps_loss, (list, tuple)), \
+ 'with_heatmaps_loss should be a list or tuple'
+ assert isinstance(heatmaps_loss_factor, (list, tuple)), \
+ 'heatmaps_loss_factor should be a list or tuple'
+ assert isinstance(with_ae_loss, (list, tuple)), \
+ 'with_ae_loss should be a list or tuple'
+ assert isinstance(push_loss_factor, (list, tuple)), \
+ 'push_loss_factor should be a list or tuple'
+ assert isinstance(pull_loss_factor, (list, tuple)), \
+ 'pull_loss_factor should be a list or tuple'
+
+ self.num_joints = num_joints
+ self.num_stages = num_stages
+ self.ae_loss_type = ae_loss_type
+ self.with_ae_loss = with_ae_loss
+ self.push_loss_factor = push_loss_factor
+ self.pull_loss_factor = pull_loss_factor
+ self.with_heatmaps_loss = with_heatmaps_loss
+ self.heatmaps_loss_factor = heatmaps_loss_factor
+
+ self.heatmaps_loss = \
+ nn.ModuleList(
+ [
+ HeatmapLoss(supervise_empty)
+ if with_heatmaps_loss else None
+ for with_heatmaps_loss in self.with_heatmaps_loss
+ ]
+ )
+
+ self.ae_loss = \
+ nn.ModuleList(
+ [
+ AELoss(self.ae_loss_type) if with_ae_loss else None
+ for with_ae_loss in self.with_ae_loss
+ ]
+ )
+
+ def forward(self, outputs, heatmaps, masks, joints):
+ """Forward function to calculate losses.
+
+ Note:
+ - batch_size: N
+ - heatmaps weight: W
+ - heatmaps height: H
+ - max_num_people: M
+ - num_keypoints: K
+ - output_channel: C C=2K if use ae loss else K
+
+ Args:
+ outputs (list(torch.Tensor[N,C,H,W])): outputs of stages.
+ heatmaps (list(torch.Tensor[N,K,H,W])): target of heatmaps.
+ masks (list(torch.Tensor[N,H,W])): masks of heatmaps.
+ joints (list(torch.Tensor[N,M,K,2])): joints of ae loss.
+ """
+ heatmaps_losses = []
+ push_losses = []
+ pull_losses = []
+ for idx in range(len(outputs)):
+ offset_feat = 0
+ if self.heatmaps_loss[idx]:
+ heatmaps_pred = outputs[idx][:, :self.num_joints]
+ offset_feat = self.num_joints
+ heatmaps_loss = self.heatmaps_loss[idx](heatmaps_pred,
+ heatmaps[idx],
+ masks[idx])
+ heatmaps_loss = heatmaps_loss * self.heatmaps_loss_factor[idx]
+ heatmaps_losses.append(heatmaps_loss)
+ else:
+ heatmaps_losses.append(None)
+
+ if self.ae_loss[idx]:
+ tags_pred = outputs[idx][:, offset_feat:]
+ batch_size = tags_pred.size()[0]
+ tags_pred = tags_pred.contiguous().view(batch_size, -1, 1)
+
+ push_loss, pull_loss = self.ae_loss[idx](tags_pred,
+ joints[idx])
+ push_loss = push_loss * self.push_loss_factor[idx]
+ pull_loss = pull_loss * self.pull_loss_factor[idx]
+
+ push_losses.append(push_loss)
+ pull_losses.append(pull_loss)
+ else:
+ push_losses.append(None)
+ pull_losses.append(None)
+
+ return heatmaps_losses, push_losses, pull_losses
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/losses/regression_loss.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/losses/regression_loss.py
new file mode 100644
index 0000000..db41783
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/losses/regression_loss.py
@@ -0,0 +1,448 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import math
+
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
+from ..builder import LOSSES
+
+
+@LOSSES.register_module()
+class SmoothL1Loss(nn.Module):
+ """SmoothL1Loss loss.
+
+ Args:
+ use_target_weight (bool): Option to use weighted MSE loss.
+ Different joint types may have different target weights.
+ loss_weight (float): Weight of the loss. Default: 1.0.
+ """
+
+ def __init__(self, use_target_weight=False, loss_weight=1.):
+ super().__init__()
+ self.criterion = F.smooth_l1_loss
+ self.use_target_weight = use_target_weight
+ self.loss_weight = loss_weight
+
+ def forward(self, output, target, target_weight=None):
+ """Forward function.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+ - dimension of keypoints: D (D=2 or D=3)
+
+ Args:
+ output (torch.Tensor[N, K, D]): Output regression.
+ target (torch.Tensor[N, K, D]): Target regression.
+ target_weight (torch.Tensor[N, K, D]):
+ Weights across different joint types.
+ """
+ if self.use_target_weight:
+ assert target_weight is not None
+ loss = self.criterion(output * target_weight,
+ target * target_weight)
+ else:
+ loss = self.criterion(output, target)
+
+ return loss * self.loss_weight
+
+
+@LOSSES.register_module()
+class WingLoss(nn.Module):
+ """Wing Loss. paper ref: 'Wing Loss for Robust Facial Landmark Localisation
+ with Convolutional Neural Networks' Feng et al. CVPR'2018.
+
+ Args:
+ omega (float): Also referred to as width.
+ epsilon (float): Also referred to as curvature.
+ use_target_weight (bool): Option to use weighted MSE loss.
+ Different joint types may have different target weights.
+ loss_weight (float): Weight of the loss. Default: 1.0.
+ """
+
+ def __init__(self,
+ omega=10.0,
+ epsilon=2.0,
+ use_target_weight=False,
+ loss_weight=1.):
+ super().__init__()
+ self.omega = omega
+ self.epsilon = epsilon
+ self.use_target_weight = use_target_weight
+ self.loss_weight = loss_weight
+
+ # constant that smoothly links the piecewise-defined linear
+ # and nonlinear parts
+ self.C = self.omega * (1.0 - math.log(1.0 + self.omega / self.epsilon))
+
+ def criterion(self, pred, target):
+ """Criterion of wingloss.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+ - dimension of keypoints: D (D=2 or D=3)
+
+ Args:
+ pred (torch.Tensor[N, K, D]): Output regression.
+ target (torch.Tensor[N, K, D]): Target regression.
+ """
+ delta = (target - pred).abs()
+ losses = torch.where(
+ delta < self.omega,
+ self.omega * torch.log(1.0 + delta / self.epsilon), delta - self.C)
+ return torch.mean(torch.sum(losses, dim=[1, 2]), dim=0)
+
+ def forward(self, output, target, target_weight=None):
+ """Forward function.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+ - dimension of keypoints: D (D=2 or D=3)
+
+ Args:
+ output (torch.Tensor[N, K, D]): Output regression.
+ target (torch.Tensor[N, K, D]): Target regression.
+ target_weight (torch.Tensor[N,K,D]):
+ Weights across different joint types.
+ """
+ if self.use_target_weight:
+ assert target_weight is not None
+ loss = self.criterion(output * target_weight,
+ target * target_weight)
+ else:
+ loss = self.criterion(output, target)
+
+ return loss * self.loss_weight
+
+
+@LOSSES.register_module()
+class SoftWingLoss(nn.Module):
+ """Soft Wing Loss 'Structure-Coherent Deep Feature Learning for Robust Face
+ Alignment' Lin et al. TIP'2021.
+
+ loss =
+ 1. |x| , if |x| < omega1
+ 2. omega2*ln(1+|x|/epsilon) + B, if |x| >= omega1
+
+ Args:
+ omega1 (float): The first threshold.
+ omega2 (float): The second threshold.
+ epsilon (float): Also referred to as curvature.
+ use_target_weight (bool): Option to use weighted MSE loss.
+ Different joint types may have different target weights.
+ loss_weight (float): Weight of the loss. Default: 1.0.
+ """
+
+ def __init__(self,
+ omega1=2.0,
+ omega2=20.0,
+ epsilon=0.5,
+ use_target_weight=False,
+ loss_weight=1.):
+ super().__init__()
+ self.omega1 = omega1
+ self.omega2 = omega2
+ self.epsilon = epsilon
+ self.use_target_weight = use_target_weight
+ self.loss_weight = loss_weight
+
+ # constant that smoothly links the piecewise-defined linear
+ # and nonlinear parts
+ self.B = self.omega1 - self.omega2 * math.log(1.0 + self.omega1 /
+ self.epsilon)
+
+ def criterion(self, pred, target):
+ """Criterion of wingloss.
+
+ Note:
+ batch_size: N
+ num_keypoints: K
+ dimension of keypoints: D (D=2 or D=3)
+
+ Args:
+ pred (torch.Tensor[N, K, D]): Output regression.
+ target (torch.Tensor[N, K, D]): Target regression.
+ """
+ delta = (target - pred).abs()
+ losses = torch.where(
+ delta < self.omega1, delta,
+ self.omega2 * torch.log(1.0 + delta / self.epsilon) + self.B)
+ return torch.mean(torch.sum(losses, dim=[1, 2]), dim=0)
+
+ def forward(self, output, target, target_weight=None):
+ """Forward function.
+
+ Note:
+ batch_size: N
+ num_keypoints: K
+ dimension of keypoints: D (D=2 or D=3)
+
+ Args:
+ output (torch.Tensor[N, K, D]): Output regression.
+ target (torch.Tensor[N, K, D]): Target regression.
+ target_weight (torch.Tensor[N, K, D]):
+ Weights across different joint types.
+ """
+ if self.use_target_weight:
+ assert target_weight is not None
+ loss = self.criterion(output * target_weight,
+ target * target_weight)
+ else:
+ loss = self.criterion(output, target)
+
+ return loss * self.loss_weight
+
+
+@LOSSES.register_module()
+class MPJPELoss(nn.Module):
+ """MPJPE (Mean Per Joint Position Error) loss.
+
+ Args:
+ use_target_weight (bool): Option to use weighted MSE loss.
+ Different joint types may have different target weights.
+ loss_weight (float): Weight of the loss. Default: 1.0.
+ """
+
+ def __init__(self, use_target_weight=False, loss_weight=1.):
+ super().__init__()
+ self.use_target_weight = use_target_weight
+ self.loss_weight = loss_weight
+
+ def forward(self, output, target, target_weight=None):
+ """Forward function.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+ - dimension of keypoints: D (D=2 or D=3)
+
+ Args:
+ output (torch.Tensor[N, K, D]): Output regression.
+ target (torch.Tensor[N, K, D]): Target regression.
+ target_weight (torch.Tensor[N,K,D]):
+ Weights across different joint types.
+ """
+
+ if self.use_target_weight:
+ assert target_weight is not None
+ loss = torch.mean(
+ torch.norm((output - target) * target_weight, dim=-1))
+ else:
+ loss = torch.mean(torch.norm(output - target, dim=-1))
+
+ return loss * self.loss_weight
+
+
+@LOSSES.register_module()
+class L1Loss(nn.Module):
+ """L1Loss loss ."""
+
+ def __init__(self, use_target_weight=False, loss_weight=1.):
+ super().__init__()
+ self.criterion = F.l1_loss
+ self.use_target_weight = use_target_weight
+ self.loss_weight = loss_weight
+
+ def forward(self, output, target, target_weight=None):
+ """Forward function.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+
+ Args:
+ output (torch.Tensor[N, K, 2]): Output regression.
+ target (torch.Tensor[N, K, 2]): Target regression.
+ target_weight (torch.Tensor[N, K, 2]):
+ Weights across different joint types.
+ """
+ if self.use_target_weight:
+ assert target_weight is not None
+ loss = self.criterion(output * target_weight,
+ target * target_weight)
+ else:
+ loss = self.criterion(output, target)
+
+ return loss * self.loss_weight
+
+
+@LOSSES.register_module()
+class MSELoss(nn.Module):
+ """MSE loss for coordinate regression."""
+
+ def __init__(self, use_target_weight=False, loss_weight=1.):
+ super().__init__()
+ self.criterion = F.mse_loss
+ self.use_target_weight = use_target_weight
+ self.loss_weight = loss_weight
+
+ def forward(self, output, target, target_weight=None):
+ """Forward function.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+
+ Args:
+ output (torch.Tensor[N, K, 2]): Output regression.
+ target (torch.Tensor[N, K, 2]): Target regression.
+ target_weight (torch.Tensor[N, K, 2]):
+ Weights across different joint types.
+ """
+ if self.use_target_weight:
+ assert target_weight is not None
+ loss = self.criterion(output * target_weight,
+ target * target_weight)
+ else:
+ loss = self.criterion(output, target)
+
+ return loss * self.loss_weight
+
+
+@LOSSES.register_module()
+class BoneLoss(nn.Module):
+ """Bone length loss.
+
+ Args:
+ joint_parents (list): Indices of each joint's parent joint.
+ use_target_weight (bool): Option to use weighted bone loss.
+ Different bone types may have different target weights.
+ loss_weight (float): Weight of the loss. Default: 1.0.
+ """
+
+ def __init__(self, joint_parents, use_target_weight=False, loss_weight=1.):
+ super().__init__()
+ self.joint_parents = joint_parents
+ self.use_target_weight = use_target_weight
+ self.loss_weight = loss_weight
+
+ self.non_root_indices = []
+ for i in range(len(self.joint_parents)):
+ if i != self.joint_parents[i]:
+ self.non_root_indices.append(i)
+
+ def forward(self, output, target, target_weight=None):
+ """Forward function.
+
+ Note:
+ - batch_size: N
+ - num_keypoints: K
+ - dimension of keypoints: D (D=2 or D=3)
+
+ Args:
+ output (torch.Tensor[N, K, D]): Output regression.
+ target (torch.Tensor[N, K, D]): Target regression.
+ target_weight (torch.Tensor[N, K-1]):
+ Weights across different bone types.
+ """
+ output_bone = torch.norm(
+ output - output[:, self.joint_parents, :],
+ dim=-1)[:, self.non_root_indices]
+ target_bone = torch.norm(
+ target - target[:, self.joint_parents, :],
+ dim=-1)[:, self.non_root_indices]
+ if self.use_target_weight:
+ assert target_weight is not None
+ loss = torch.mean(
+ torch.abs((output_bone * target_weight).mean(dim=0) -
+ (target_bone * target_weight).mean(dim=0)))
+ else:
+ loss = torch.mean(
+ torch.abs(output_bone.mean(dim=0) - target_bone.mean(dim=0)))
+
+ return loss * self.loss_weight
+
+
+@LOSSES.register_module()
+class SemiSupervisionLoss(nn.Module):
+ """Semi-supervision loss for unlabeled data. It is composed of projection
+ loss and bone loss.
+
+ Paper ref: `3D human pose estimation in video with temporal convolutions
+ and semi-supervised training` Dario Pavllo et al. CVPR'2019.
+
+ Args:
+ joint_parents (list): Indices of each joint's parent joint.
+ projection_loss_weight (float): Weight for projection loss.
+ bone_loss_weight (float): Weight for bone loss.
+ warmup_iterations (int): Number of warmup iterations. In the first
+ `warmup_iterations` iterations, the model is trained only on
+ labeled data, and semi-supervision loss will be 0.
+ This is a workaround since currently we cannot access
+ epoch number in loss functions. Note that the iteration number in
+ an epoch can be changed due to different GPU numbers in multi-GPU
+ settings. So please set this parameter carefully.
+ warmup_iterations = dataset_size // samples_per_gpu // gpu_num
+ * warmup_epochs
+ """
+
+ def __init__(self,
+ joint_parents,
+ projection_loss_weight=1.,
+ bone_loss_weight=1.,
+ warmup_iterations=0):
+ super().__init__()
+ self.criterion_projection = MPJPELoss(
+ loss_weight=projection_loss_weight)
+ self.criterion_bone = BoneLoss(
+ joint_parents, loss_weight=bone_loss_weight)
+ self.warmup_iterations = warmup_iterations
+ self.num_iterations = 0
+
+ @staticmethod
+ def project_joints(x, intrinsics):
+ """Project 3D joint coordinates to 2D image plane using camera
+ intrinsic parameters.
+
+ Args:
+ x (torch.Tensor[N, K, 3]): 3D joint coordinates.
+ intrinsics (torch.Tensor[N, 4] | torch.Tensor[N, 9]): Camera
+ intrinsics: f (2), c (2), k (3), p (2).
+ """
+ while intrinsics.dim() < x.dim():
+ intrinsics.unsqueeze_(1)
+ f = intrinsics[..., :2]
+ c = intrinsics[..., 2:4]
+ _x = torch.clamp(x[:, :, :2] / x[:, :, 2:], -1, 1)
+ if intrinsics.shape[-1] == 9:
+ k = intrinsics[..., 4:7]
+ p = intrinsics[..., 7:9]
+
+ r2 = torch.sum(_x[:, :, :2]**2, dim=-1, keepdim=True)
+ radial = 1 + torch.sum(
+ k * torch.cat((r2, r2**2, r2**3), dim=-1),
+ dim=-1,
+ keepdim=True)
+ tan = torch.sum(p * _x, dim=-1, keepdim=True)
+ _x = _x * (radial + tan) + p * r2
+ _x = f * _x + c
+ return _x
+
+ def forward(self, output, target):
+ losses = dict()
+
+ self.num_iterations += 1
+ if self.num_iterations <= self.warmup_iterations:
+ return losses
+
+ labeled_pose = output['labeled_pose']
+ unlabeled_pose = output['unlabeled_pose']
+ unlabeled_traj = output['unlabeled_traj']
+ unlabeled_target_2d = target['unlabeled_target_2d']
+ intrinsics = target['intrinsics']
+
+ # projection loss
+ unlabeled_output = unlabeled_pose + unlabeled_traj
+ unlabeled_output_2d = self.project_joints(unlabeled_output, intrinsics)
+ loss_proj = self.criterion_projection(unlabeled_output_2d,
+ unlabeled_target_2d, None)
+ losses['proj_loss'] = loss_proj
+
+ # bone loss
+ loss_bone = self.criterion_bone(unlabeled_pose, labeled_pose, None)
+ losses['bone_loss'] = loss_bone
+
+ return losses
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/misc/__init__.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/misc/__init__.py
new file mode 100644
index 0000000..ef101fe
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/misc/__init__.py
@@ -0,0 +1 @@
+# Copyright (c) OpenMMLab. All rights reserved.
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/misc/discriminator.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/misc/discriminator.py
new file mode 100644
index 0000000..712f0a8
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/misc/discriminator.py
@@ -0,0 +1,307 @@
+# ------------------------------------------------------------------------------
+# Adapted from https://github.com/akanazawa/hmr
+# Original licence: Copyright (c) 2018 akanazawa, under the MIT License.
+# ------------------------------------------------------------------------------
+
+from abc import abstractmethod
+
+import torch
+import torch.nn as nn
+from mmcv.cnn import normal_init, xavier_init
+
+from mmpose.models.utils.geometry import batch_rodrigues
+
+
+class BaseDiscriminator(nn.Module):
+ """Base linear module for SMPL parameter discriminator.
+
+ Args:
+ fc_layers (Tuple): Tuple of neuron count,
+ such as (9, 32, 32, 1)
+ use_dropout (Tuple): Tuple of bool define use dropout or not
+ for each layer, such as (True, True, False)
+ drop_prob (Tuple): Tuple of float defined the drop prob,
+ such as (0.5, 0.5, 0)
+ use_activation(Tuple): Tuple of bool define use active function
+ or not, such as (True, True, False)
+ """
+
+ def __init__(self, fc_layers, use_dropout, drop_prob, use_activation):
+ super().__init__()
+ self.fc_layers = fc_layers
+ self.use_dropout = use_dropout
+ self.drop_prob = drop_prob
+ self.use_activation = use_activation
+ self._check()
+ self.create_layers()
+
+ def _check(self):
+ """Check input to avoid ValueError."""
+ if not isinstance(self.fc_layers, tuple):
+ raise TypeError(f'fc_layers require tuple, '
+ f'get {type(self.fc_layers)}')
+
+ if not isinstance(self.use_dropout, tuple):
+ raise TypeError(f'use_dropout require tuple, '
+ f'get {type(self.use_dropout)}')
+
+ if not isinstance(self.drop_prob, tuple):
+ raise TypeError(f'drop_prob require tuple, '
+ f'get {type(self.drop_prob)}')
+
+ if not isinstance(self.use_activation, tuple):
+ raise TypeError(f'use_activation require tuple, '
+ f'get {type(self.use_activation)}')
+
+ l_fc_layer = len(self.fc_layers)
+ l_use_drop = len(self.use_dropout)
+ l_drop_prob = len(self.drop_prob)
+ l_use_activation = len(self.use_activation)
+
+ pass_check = (
+ l_fc_layer >= 2 and l_use_drop < l_fc_layer
+ and l_drop_prob < l_fc_layer and l_use_activation < l_fc_layer
+ and l_drop_prob == l_use_drop)
+
+ if not pass_check:
+ msg = 'Wrong BaseDiscriminator parameters!'
+ raise ValueError(msg)
+
+ def create_layers(self):
+ """Create layers."""
+ l_fc_layer = len(self.fc_layers)
+ l_use_drop = len(self.use_dropout)
+ l_use_activation = len(self.use_activation)
+
+ self.fc_blocks = nn.Sequential()
+
+ for i in range(l_fc_layer - 1):
+ self.fc_blocks.add_module(
+ name=f'regressor_fc_{i}',
+ module=nn.Linear(
+ in_features=self.fc_layers[i],
+ out_features=self.fc_layers[i + 1]))
+
+ if i < l_use_activation and self.use_activation[i]:
+ self.fc_blocks.add_module(
+ name=f'regressor_af_{i}', module=nn.ReLU())
+
+ if i < l_use_drop and self.use_dropout[i]:
+ self.fc_blocks.add_module(
+ name=f'regressor_fc_dropout_{i}',
+ module=nn.Dropout(p=self.drop_prob[i]))
+
+ @abstractmethod
+ def forward(self, inputs):
+ """Forward function."""
+ msg = 'the base class [BaseDiscriminator] is not callable!'
+ raise NotImplementedError(msg)
+
+ def init_weights(self):
+ """Initialize model weights."""
+ for m in self.fc_blocks.named_modules():
+ if isinstance(m, nn.Linear):
+ xavier_init(m, gain=0.01)
+
+
+class ShapeDiscriminator(BaseDiscriminator):
+ """Discriminator for SMPL shape parameters, the inputs is (batch_size x 10)
+
+ Args:
+ fc_layers (Tuple): Tuple of neuron count, such as (10, 5, 1)
+ use_dropout (Tuple): Tuple of bool define use dropout or
+ not for each layer, such as (True, True, False)
+ drop_prob (Tuple): Tuple of float defined the drop prob,
+ such as (0.5, 0)
+ use_activation(Tuple): Tuple of bool define use active
+ function or not, such as (True, False)
+ """
+
+ def __init__(self, fc_layers, use_dropout, drop_prob, use_activation):
+ if fc_layers[-1] != 1:
+ msg = f'the neuron count of the last layer ' \
+ f'must be 1, but got {fc_layers[-1]}'
+ raise ValueError(msg)
+
+ super().__init__(fc_layers, use_dropout, drop_prob, use_activation)
+
+ def forward(self, inputs):
+ """Forward function."""
+ return self.fc_blocks(inputs)
+
+
+class PoseDiscriminator(nn.Module):
+ """Discriminator for SMPL pose parameters of each joint. It is composed of
+ discriminators for each joints. The inputs is (batch_size x joint_count x
+ 9)
+
+ Args:
+ channels (Tuple): Tuple of channel number,
+ such as (9, 32, 32, 1)
+ joint_count (int): Joint number, such as 23
+ """
+
+ def __init__(self, channels, joint_count):
+ super().__init__()
+ if channels[-1] != 1:
+ msg = f'the neuron count of the last layer ' \
+ f'must be 1, but got {channels[-1]}'
+ raise ValueError(msg)
+ self.joint_count = joint_count
+
+ self.conv_blocks = nn.Sequential()
+ len_channels = len(channels)
+ for idx in range(len_channels - 2):
+ self.conv_blocks.add_module(
+ name=f'conv_{idx}',
+ module=nn.Conv2d(
+ in_channels=channels[idx],
+ out_channels=channels[idx + 1],
+ kernel_size=1,
+ stride=1))
+
+ self.fc_layer = nn.ModuleList()
+ for idx in range(joint_count):
+ self.fc_layer.append(
+ nn.Linear(
+ in_features=channels[len_channels - 2], out_features=1))
+
+ def forward(self, inputs):
+ """Forward function.
+
+ The input is (batch_size x joint_count x 9).
+ """
+ # shape: batch_size x 9 x 1 x joint_count
+ inputs = inputs.transpose(1, 2).unsqueeze(2).contiguous()
+ # shape: batch_size x c x 1 x joint_count
+ internal_outputs = self.conv_blocks(inputs)
+ outputs = []
+ for idx in range(self.joint_count):
+ outputs.append(self.fc_layer[idx](internal_outputs[:, :, 0, idx]))
+
+ return torch.cat(outputs, 1), internal_outputs
+
+ def init_weights(self):
+ """Initialize model weights."""
+ for m in self.conv_blocks:
+ if isinstance(m, nn.Conv2d):
+ normal_init(m, std=0.001, bias=0)
+ for m in self.fc_layer.named_modules():
+ if isinstance(m, nn.Linear):
+ xavier_init(m, gain=0.01)
+
+
+class FullPoseDiscriminator(BaseDiscriminator):
+ """Discriminator for SMPL pose parameters of all joints.
+
+ Args:
+ fc_layers (Tuple): Tuple of neuron count,
+ such as (736, 1024, 1024, 1)
+ use_dropout (Tuple): Tuple of bool define use dropout or not
+ for each layer, such as (True, True, False)
+ drop_prob (Tuple): Tuple of float defined the drop prob,
+ such as (0.5, 0.5, 0)
+ use_activation(Tuple): Tuple of bool define use active
+ function or not, such as (True, True, False)
+ """
+
+ def __init__(self, fc_layers, use_dropout, drop_prob, use_activation):
+ if fc_layers[-1] != 1:
+ msg = f'the neuron count of the last layer must be 1,' \
+ f' but got {fc_layers[-1]}'
+ raise ValueError(msg)
+
+ super().__init__(fc_layers, use_dropout, drop_prob, use_activation)
+
+ def forward(self, inputs):
+ """Forward function."""
+ return self.fc_blocks(inputs)
+
+
+class SMPLDiscriminator(nn.Module):
+ """Discriminator for SMPL pose and shape parameters. It is composed of a
+ discriminator for SMPL shape parameters, a discriminator for SMPL pose
+ parameters of all joints and a discriminator for SMPL pose parameters of
+ each joint.
+
+ Args:
+ beta_channel (tuple of int): Tuple of neuron count of the
+ discriminator of shape parameters. Defaults to (10, 5, 1)
+ per_joint_channel (tuple of int): Tuple of neuron count of the
+ discriminator of each joint. Defaults to (9, 32, 32, 1)
+ full_pose_channel (tuple of int): Tuple of neuron count of the
+ discriminator of full pose. Defaults to (23*32, 1024, 1024, 1)
+ """
+
+ def __init__(self,
+ beta_channel=(10, 5, 1),
+ per_joint_channel=(9, 32, 32, 1),
+ full_pose_channel=(23 * 32, 1024, 1024, 1)):
+ super().__init__()
+ self.joint_count = 23
+ # The count of SMPL shape parameter is 10.
+ assert beta_channel[0] == 10
+ # Use 3 x 3 rotation matrix as the pose parameters
+ # of each joint, so the input channel is 9.
+ assert per_joint_channel[0] == 9
+ assert self.joint_count * per_joint_channel[-2] \
+ == full_pose_channel[0]
+
+ self.beta_channel = beta_channel
+ self.per_joint_channel = per_joint_channel
+ self.full_pose_channel = full_pose_channel
+ self._create_sub_modules()
+
+ def _create_sub_modules(self):
+ """Create sub discriminators."""
+
+ # create theta discriminator for each joint
+ self.pose_discriminator = PoseDiscriminator(self.per_joint_channel,
+ self.joint_count)
+
+ # create full pose discriminator for total joints
+ fc_layers = self.full_pose_channel
+ use_dropout = tuple([False] * (len(fc_layers) - 1))
+ drop_prob = tuple([0.5] * (len(fc_layers) - 1))
+ use_activation = tuple([True] * (len(fc_layers) - 2) + [False])
+
+ self.full_pose_discriminator = FullPoseDiscriminator(
+ fc_layers, use_dropout, drop_prob, use_activation)
+
+ # create shape discriminator for betas
+ fc_layers = self.beta_channel
+ use_dropout = tuple([False] * (len(fc_layers) - 1))
+ drop_prob = tuple([0.5] * (len(fc_layers) - 1))
+ use_activation = tuple([True] * (len(fc_layers) - 2) + [False])
+ self.shape_discriminator = ShapeDiscriminator(fc_layers, use_dropout,
+ drop_prob,
+ use_activation)
+
+ def forward(self, thetas):
+ """Forward function."""
+ _, poses, shapes = thetas
+
+ batch_size = poses.shape[0]
+ shape_disc_value = self.shape_discriminator(shapes)
+
+ # The first rotation matrix is global rotation
+ # and is NOT used in discriminator.
+ if poses.dim() == 2:
+ rotate_matrixs = \
+ batch_rodrigues(poses.contiguous().view(-1, 3)
+ ).view(batch_size, 24, 9)[:, 1:, :]
+ else:
+ rotate_matrixs = poses.contiguous().view(batch_size, 24,
+ 9)[:, 1:, :].contiguous()
+ pose_disc_value, pose_inter_disc_value \
+ = self.pose_discriminator(rotate_matrixs)
+ full_pose_disc_value = self.full_pose_discriminator(
+ pose_inter_disc_value.contiguous().view(batch_size, -1))
+ return torch.cat(
+ (pose_disc_value, full_pose_disc_value, shape_disc_value), 1)
+
+ def init_weights(self):
+ """Initialize model weights."""
+ self.full_pose_discriminator.init_weights()
+ self.pose_discriminator.init_weights()
+ self.shape_discriminator.init_weights()
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/necks/__init__.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/necks/__init__.py
new file mode 100644
index 0000000..0d3a5cc
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/necks/__init__.py
@@ -0,0 +1,5 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from .gap_neck import GlobalAveragePooling
+from .posewarper_neck import PoseWarperNeck
+
+__all__ = ['GlobalAveragePooling', 'PoseWarperNeck']
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/necks/gap_neck.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/necks/gap_neck.py
new file mode 100644
index 0000000..5e6ad68
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/necks/gap_neck.py
@@ -0,0 +1,37 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import torch
+import torch.nn as nn
+
+from ..builder import NECKS
+
+
+@NECKS.register_module()
+class GlobalAveragePooling(nn.Module):
+ """Global Average Pooling neck.
+
+ Note that we use `view` to remove extra channel after pooling. We do not
+ use `squeeze` as it will also remove the batch dimension when the tensor
+ has a batch dimension of size 1, which can lead to unexpected errors.
+ """
+
+ def __init__(self):
+ super().__init__()
+ self.gap = nn.AdaptiveAvgPool2d((1, 1))
+
+ def init_weights(self):
+ pass
+
+ def forward(self, inputs):
+ if isinstance(inputs, tuple):
+ outs = tuple([self.gap(x) for x in inputs])
+ outs = tuple(
+ [out.view(x.size(0), -1) for out, x in zip(outs, inputs)])
+ elif isinstance(inputs, list):
+ outs = [self.gap(x) for x in inputs]
+ outs = [out.view(x.size(0), -1) for out, x in zip(outs, inputs)]
+ elif isinstance(inputs, torch.Tensor):
+ outs = self.gap(inputs)
+ outs = outs.view(inputs.size(0), -1)
+ else:
+ raise TypeError('neck inputs should be tuple or torch.tensor')
+ return outs
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/necks/posewarper_neck.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/necks/posewarper_neck.py
new file mode 100644
index 0000000..dd4ddfb
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/necks/posewarper_neck.py
@@ -0,0 +1,329 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import mmcv
+import torch
+import torch.nn as nn
+from mmcv.cnn import (build_conv_layer, build_norm_layer, constant_init,
+ normal_init)
+from mmcv.utils import digit_version
+from torch.nn.modules.batchnorm import _BatchNorm
+
+from mmpose.models.utils.ops import resize
+from ..backbones.resnet import BasicBlock, Bottleneck
+from ..builder import NECKS
+
+try:
+ from mmcv.ops import DeformConv2d
+ has_mmcv_full = True
+except (ImportError, ModuleNotFoundError):
+ has_mmcv_full = False
+
+
+@NECKS.register_module()
+class PoseWarperNeck(nn.Module):
+ """PoseWarper neck.
+
+ `"Learning temporal pose estimation from sparsely-labeled videos"
+ `_.
+
+ Args:
+ in_channels (int): Number of input channels from backbone
+ out_channels (int): Number of output channels
+ inner_channels (int): Number of intermediate channels of the res block
+ deform_groups (int): Number of groups in the deformable conv
+ dilations (list|tuple): different dilations of the offset conv layers
+ trans_conv_kernel (int): the kernel of the trans conv layer, which is
+ used to get heatmap from the output of backbone. Default: 1
+ res_blocks_cfg (dict|None): config of residual blocks. If None,
+ use the default values. If not None, it should contain the
+ following keys:
+
+ - block (str): the type of residual block, Default: 'BASIC'.
+ - num_blocks (int): the number of blocks, Default: 20.
+
+ offsets_kernel (int): the kernel of offset conv layer.
+ deform_conv_kernel (int): the kernel of defomrable conv layer.
+ in_index (int|Sequence[int]): Input feature index. Default: 0
+ input_transform (str|None): Transformation type of input features.
+ Options: 'resize_concat', 'multiple_select', None.
+ Default: None.
+
+ - 'resize_concat': Multiple feature maps will be resize to \
+ the same size as first one and than concat together. \
+ Usually used in FCN head of HRNet.
+ - 'multiple_select': Multiple feature maps will be bundle into \
+ a list and passed into decode head.
+ - None: Only one select feature map is allowed.
+
+ freeze_trans_layer (bool): Whether to freeze the transition layer
+ (stop grad and set eval mode). Default: True.
+ norm_eval (bool): Whether to set norm layers to eval mode, namely,
+ freeze running stats (mean and var). Note: Effect on Batch Norm
+ and its variants only. Default: False.
+ im2col_step (int): the argument `im2col_step` in deformable conv,
+ Default: 80.
+ """
+ blocks_dict = {'BASIC': BasicBlock, 'BOTTLENECK': Bottleneck}
+ minimum_mmcv_version = '1.3.17'
+
+ def __init__(self,
+ in_channels,
+ out_channels,
+ inner_channels,
+ deform_groups=17,
+ dilations=(3, 6, 12, 18, 24),
+ trans_conv_kernel=1,
+ res_blocks_cfg=None,
+ offsets_kernel=3,
+ deform_conv_kernel=3,
+ in_index=0,
+ input_transform=None,
+ freeze_trans_layer=True,
+ norm_eval=False,
+ im2col_step=80):
+ super().__init__()
+ self.in_channels = in_channels
+ self.out_channels = out_channels
+ self.inner_channels = inner_channels
+ self.deform_groups = deform_groups
+ self.dilations = dilations
+ self.trans_conv_kernel = trans_conv_kernel
+ self.res_blocks_cfg = res_blocks_cfg
+ self.offsets_kernel = offsets_kernel
+ self.deform_conv_kernel = deform_conv_kernel
+ self.in_index = in_index
+ self.input_transform = input_transform
+ self.freeze_trans_layer = freeze_trans_layer
+ self.norm_eval = norm_eval
+ self.im2col_step = im2col_step
+
+ identity_trans_layer = False
+
+ assert trans_conv_kernel in [0, 1, 3]
+ kernel_size = trans_conv_kernel
+ if kernel_size == 3:
+ padding = 1
+ elif kernel_size == 1:
+ padding = 0
+ else:
+ # 0 for Identity mapping.
+ identity_trans_layer = True
+
+ if identity_trans_layer:
+ self.trans_layer = nn.Identity()
+ else:
+ self.trans_layer = build_conv_layer(
+ cfg=dict(type='Conv2d'),
+ in_channels=in_channels,
+ out_channels=out_channels,
+ kernel_size=kernel_size,
+ stride=1,
+ padding=padding)
+
+ # build chain of residual blocks
+ if res_blocks_cfg is not None and not isinstance(res_blocks_cfg, dict):
+ raise TypeError('res_blocks_cfg should be dict or None.')
+
+ if res_blocks_cfg is None:
+ block_type = 'BASIC'
+ num_blocks = 20
+ else:
+ block_type = res_blocks_cfg.get('block', 'BASIC')
+ num_blocks = res_blocks_cfg.get('num_blocks', 20)
+
+ block = self.blocks_dict[block_type]
+
+ res_layers = []
+ downsample = nn.Sequential(
+ build_conv_layer(
+ cfg=dict(type='Conv2d'),
+ in_channels=out_channels,
+ out_channels=inner_channels,
+ kernel_size=1,
+ stride=1,
+ bias=False),
+ build_norm_layer(dict(type='BN'), inner_channels)[1])
+ res_layers.append(
+ block(
+ in_channels=out_channels,
+ out_channels=inner_channels,
+ downsample=downsample))
+
+ for _ in range(1, num_blocks):
+ res_layers.append(block(inner_channels, inner_channels))
+ self.offset_feats = nn.Sequential(*res_layers)
+
+ # build offset layers
+ self.num_offset_layers = len(dilations)
+ assert self.num_offset_layers > 0, 'Number of offset layers ' \
+ 'should be larger than 0.'
+
+ target_offset_channels = 2 * offsets_kernel**2 * deform_groups
+
+ offset_layers = [
+ build_conv_layer(
+ cfg=dict(type='Conv2d'),
+ in_channels=inner_channels,
+ out_channels=target_offset_channels,
+ kernel_size=offsets_kernel,
+ stride=1,
+ dilation=dilations[i],
+ padding=dilations[i],
+ bias=False,
+ ) for i in range(self.num_offset_layers)
+ ]
+ self.offset_layers = nn.ModuleList(offset_layers)
+
+ # build deformable conv layers
+ assert digit_version(mmcv.__version__) >= \
+ digit_version(self.minimum_mmcv_version), \
+ f'Current MMCV version: {mmcv.__version__}, ' \
+ f'but MMCV >= {self.minimum_mmcv_version} is required, see ' \
+ f'https://github.com/open-mmlab/mmcv/issues/1440, ' \
+ f'Please install the latest MMCV.'
+
+ if has_mmcv_full:
+ deform_conv_layers = [
+ DeformConv2d(
+ in_channels=out_channels,
+ out_channels=out_channels,
+ kernel_size=deform_conv_kernel,
+ stride=1,
+ padding=int(deform_conv_kernel / 2) * dilations[i],
+ dilation=dilations[i],
+ deform_groups=deform_groups,
+ im2col_step=self.im2col_step,
+ ) for i in range(self.num_offset_layers)
+ ]
+ else:
+ raise ImportError('Please install the full version of mmcv '
+ 'to use `DeformConv2d`.')
+
+ self.deform_conv_layers = nn.ModuleList(deform_conv_layers)
+
+ self.freeze_layers()
+
+ def freeze_layers(self):
+ if self.freeze_trans_layer:
+ self.trans_layer.eval()
+
+ for param in self.trans_layer.parameters():
+ param.requires_grad = False
+
+ def init_weights(self):
+ for m in self.modules():
+ if isinstance(m, nn.Conv2d):
+ normal_init(m, std=0.001)
+ elif isinstance(m, (_BatchNorm, nn.GroupNorm)):
+ constant_init(m, 1)
+ elif isinstance(m, DeformConv2d):
+ filler = torch.zeros([
+ m.weight.size(0),
+ m.weight.size(1),
+ m.weight.size(2),
+ m.weight.size(3)
+ ],
+ dtype=torch.float32,
+ device=m.weight.device)
+ for k in range(m.weight.size(0)):
+ filler[k, k,
+ int(m.weight.size(2) / 2),
+ int(m.weight.size(3) / 2)] = 1.0
+ m.weight = torch.nn.Parameter(filler)
+ m.weight.requires_grad = True
+
+ # posewarper offset layer weight initialization
+ for m in self.offset_layers.modules():
+ constant_init(m, 0)
+
+ def _transform_inputs(self, inputs):
+ """Transform inputs for decoder.
+
+ Args:
+ inputs (list[Tensor] | Tensor): multi-level img features.
+
+ Returns:
+ Tensor: The transformed inputs
+ """
+ if not isinstance(inputs, list):
+ return inputs
+
+ if self.input_transform == 'resize_concat':
+ inputs = [inputs[i] for i in self.in_index]
+ upsampled_inputs = [
+ resize(
+ input=x,
+ size=inputs[0].shape[2:],
+ mode='bilinear',
+ align_corners=self.align_corners) for x in inputs
+ ]
+ inputs = torch.cat(upsampled_inputs, dim=1)
+ elif self.input_transform == 'multiple_select':
+ inputs = [inputs[i] for i in self.in_index]
+ else:
+ inputs = inputs[self.in_index]
+
+ return inputs
+
+ def forward(self, inputs, frame_weight):
+ assert isinstance(inputs, (list, tuple)), 'PoseWarperNeck inputs ' \
+ 'should be list or tuple, even though the length is 1, ' \
+ 'for unified processing.'
+
+ output_heatmap = 0
+ if len(inputs) > 1:
+ inputs = [self._transform_inputs(input) for input in inputs]
+ inputs = [self.trans_layer(input) for input in inputs]
+
+ # calculate difference features
+ diff_features = [
+ self.offset_feats(inputs[0] - input) for input in inputs
+ ]
+
+ for i in range(len(inputs)):
+ if frame_weight[i] == 0:
+ continue
+ warped_heatmap = 0
+ for j in range(self.num_offset_layers):
+ offset = (self.offset_layers[j](diff_features[i]))
+ warped_heatmap_tmp = self.deform_conv_layers[j](inputs[i],
+ offset)
+ warped_heatmap += warped_heatmap_tmp / \
+ self.num_offset_layers
+
+ output_heatmap += warped_heatmap * frame_weight[i]
+
+ else:
+ inputs = inputs[0]
+ inputs = self._transform_inputs(inputs)
+ inputs = self.trans_layer(inputs)
+
+ num_frames = len(frame_weight)
+ batch_size = inputs.size(0) // num_frames
+ ref_x = inputs[:batch_size]
+ ref_x_tiled = ref_x.repeat(num_frames, 1, 1, 1)
+
+ offset_features = self.offset_feats(ref_x_tiled - inputs)
+
+ warped_heatmap = 0
+ for j in range(self.num_offset_layers):
+ offset = self.offset_layers[j](offset_features)
+
+ warped_heatmap_tmp = self.deform_conv_layers[j](inputs, offset)
+ warped_heatmap += warped_heatmap_tmp / self.num_offset_layers
+
+ for i in range(num_frames):
+ if frame_weight[i] == 0:
+ continue
+ output_heatmap += warped_heatmap[i * batch_size:(i + 1) *
+ batch_size] * frame_weight[i]
+
+ return output_heatmap
+
+ def train(self, mode=True):
+ """Convert the model into training mode."""
+ super().train(mode)
+ self.freeze_layers()
+ if mode and self.norm_eval:
+ for m in self.modules():
+ if isinstance(m, _BatchNorm):
+ m.eval()
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/registry.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/registry.py
new file mode 100644
index 0000000..f354ae9
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/registry.py
@@ -0,0 +1,13 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import warnings
+
+from .builder import BACKBONES, HEADS, LOSSES, NECKS, POSENETS
+
+__all__ = ['BACKBONES', 'HEADS', 'LOSSES', 'NECKS', 'POSENETS']
+
+warnings.simplefilter('once', DeprecationWarning)
+warnings.warn(
+ 'Registries (BACKBONES, NECKS, HEADS, LOSSES, POSENETS) have '
+ 'been moved to mmpose.models.builder. Importing from '
+ 'mmpose.models.registry will be deprecated in the future.',
+ DeprecationWarning)
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/utils/__init__.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/utils/__init__.py
new file mode 100644
index 0000000..6871c66
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/utils/__init__.py
@@ -0,0 +1,4 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from .smpl import SMPL
+
+__all__ = ['SMPL']
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/utils/geometry.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/utils/geometry.py
new file mode 100644
index 0000000..0ceadae
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/utils/geometry.py
@@ -0,0 +1,68 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import torch
+from torch.nn import functional as F
+
+
+def rot6d_to_rotmat(x):
+ """Convert 6D rotation representation to 3x3 rotation matrix.
+
+ Based on Zhou et al., "On the Continuity of Rotation
+ Representations in Neural Networks", CVPR 2019
+ Input:
+ (B,6) Batch of 6-D rotation representations
+ Output:
+ (B,3,3) Batch of corresponding rotation matrices
+ """
+ x = x.view(-1, 3, 2)
+ a1 = x[:, :, 0]
+ a2 = x[:, :, 1]
+ b1 = F.normalize(a1)
+ b2 = F.normalize(a2 - torch.einsum('bi,bi->b', b1, a2).unsqueeze(-1) * b1)
+ b3 = torch.cross(b1, b2)
+ return torch.stack((b1, b2, b3), dim=-1)
+
+
+def batch_rodrigues(theta):
+ """Convert axis-angle representation to rotation matrix.
+ Args:
+ theta: size = [B, 3]
+ Returns:
+ Rotation matrix corresponding to the quaternion
+ -- size = [B, 3, 3]
+ """
+ l2norm = torch.norm(theta + 1e-8, p=2, dim=1)
+ angle = torch.unsqueeze(l2norm, -1)
+ normalized = torch.div(theta, angle)
+ angle = angle * 0.5
+ v_cos = torch.cos(angle)
+ v_sin = torch.sin(angle)
+ quat = torch.cat([v_cos, v_sin * normalized], dim=1)
+ return quat_to_rotmat(quat)
+
+
+def quat_to_rotmat(quat):
+ """Convert quaternion coefficients to rotation matrix.
+ Args:
+ quat: size = [B, 4] 4 <===>(w, x, y, z)
+ Returns:
+ Rotation matrix corresponding to the quaternion
+ -- size = [B, 3, 3]
+ """
+ norm_quat = quat
+ norm_quat = norm_quat / norm_quat.norm(p=2, dim=1, keepdim=True)
+ w, x, y, z = norm_quat[:, 0], norm_quat[:, 1],\
+ norm_quat[:, 2], norm_quat[:, 3]
+
+ B = quat.size(0)
+
+ w2, x2, y2, z2 = w.pow(2), x.pow(2), y.pow(2), z.pow(2)
+ wx, wy, wz = w * x, w * y, w * z
+ xy, xz, yz = x * y, x * z, y * z
+
+ rotMat = torch.stack([
+ w2 + x2 - y2 - z2, 2 * xy - 2 * wz, 2 * wy + 2 * xz, 2 * wz + 2 * xy,
+ w2 - x2 + y2 - z2, 2 * yz - 2 * wx, 2 * xz - 2 * wy, 2 * wx + 2 * yz,
+ w2 - x2 - y2 + z2
+ ],
+ dim=1).view(B, 3, 3)
+ return rotMat
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/utils/ops.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/utils/ops.py
new file mode 100644
index 0000000..858d0a9
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/utils/ops.py
@@ -0,0 +1,29 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import warnings
+
+import torch
+import torch.nn.functional as F
+
+
+def resize(input,
+ size=None,
+ scale_factor=None,
+ mode='nearest',
+ align_corners=None,
+ warning=True):
+ if warning:
+ if size is not None and align_corners:
+ input_h, input_w = tuple(int(x) for x in input.shape[2:])
+ output_h, output_w = tuple(int(x) for x in size)
+ if output_h > input_h or output_w > output_h:
+ if ((output_h > 1 and output_w > 1 and input_h > 1
+ and input_w > 1) and (output_h - 1) % (input_h - 1)
+ and (output_w - 1) % (input_w - 1)):
+ warnings.warn(
+ f'When align_corners={align_corners}, '
+ 'the output would more aligned if '
+ f'input size {(input_h, input_w)} is `x+1` and '
+ f'out size {(output_h, output_w)} is `nx+1`')
+ if isinstance(size, torch.Size):
+ size = tuple(int(x) for x in size)
+ return F.interpolate(input, size, scale_factor, mode, align_corners)
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/models/utils/smpl.py b/engine/pose_estimation/third-party/ViTPose/mmpose/models/utils/smpl.py
new file mode 100644
index 0000000..fe723d4
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/models/utils/smpl.py
@@ -0,0 +1,184 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import numpy as np
+import torch
+import torch.nn as nn
+
+from ..builder import MESH_MODELS
+
+try:
+ from smplx import SMPL as SMPL_
+ has_smpl = True
+except (ImportError, ModuleNotFoundError):
+ has_smpl = False
+
+
+@MESH_MODELS.register_module()
+class SMPL(nn.Module):
+ """SMPL 3d human mesh model of paper ref: Matthew Loper. ``SMPL: A skinned
+ multi-person linear model''. This module is based on the smplx project
+ (https://github.com/vchoutas/smplx).
+
+ Args:
+ smpl_path (str): The path to the folder where the model weights are
+ stored.
+ joints_regressor (str): The path to the file where the joints
+ regressor weight are stored.
+ """
+
+ def __init__(self, smpl_path, joints_regressor):
+ super().__init__()
+
+ assert has_smpl, 'Please install smplx to use SMPL.'
+
+ self.smpl_neutral = SMPL_(
+ model_path=smpl_path,
+ create_global_orient=False,
+ create_body_pose=False,
+ create_transl=False,
+ gender='neutral')
+
+ self.smpl_male = SMPL_(
+ model_path=smpl_path,
+ create_betas=False,
+ create_global_orient=False,
+ create_body_pose=False,
+ create_transl=False,
+ gender='male')
+
+ self.smpl_female = SMPL_(
+ model_path=smpl_path,
+ create_betas=False,
+ create_global_orient=False,
+ create_body_pose=False,
+ create_transl=False,
+ gender='female')
+
+ joints_regressor = torch.tensor(
+ np.load(joints_regressor), dtype=torch.float)[None, ...]
+ self.register_buffer('joints_regressor', joints_regressor)
+
+ self.num_verts = self.smpl_neutral.get_num_verts()
+ self.num_joints = self.joints_regressor.shape[1]
+
+ def smpl_forward(self, model, **kwargs):
+ """Apply a specific SMPL model with given model parameters.
+
+ Note:
+ B: batch size
+ V: number of vertices
+ K: number of joints
+
+ Returns:
+ outputs (dict): Dict with mesh vertices and joints.
+ - vertices: Tensor([B, V, 3]), mesh vertices
+ - joints: Tensor([B, K, 3]), 3d joints regressed
+ from mesh vertices.
+ """
+
+ betas = kwargs['betas']
+ batch_size = betas.shape[0]
+ device = betas.device
+ output = {}
+ if batch_size == 0:
+ output['vertices'] = betas.new_zeros([0, self.num_verts, 3])
+ output['joints'] = betas.new_zeros([0, self.num_joints, 3])
+ else:
+ smpl_out = model(**kwargs)
+ output['vertices'] = smpl_out.vertices
+ output['joints'] = torch.matmul(
+ self.joints_regressor.to(device), output['vertices'])
+ return output
+
+ def get_faces(self):
+ """Return mesh faces.
+
+ Note:
+ F: number of faces
+
+ Returns:
+ faces: np.ndarray([F, 3]), mesh faces
+ """
+ return self.smpl_neutral.faces
+
+ def forward(self,
+ betas,
+ body_pose,
+ global_orient,
+ transl=None,
+ gender=None):
+ """Forward function.
+
+ Note:
+ B: batch size
+ J: number of controllable joints of model, for smpl model J=23
+ K: number of joints
+
+ Args:
+ betas: Tensor([B, 10]), human body shape parameters of SMPL model.
+ body_pose: Tensor([B, J*3] or [B, J, 3, 3]), human body pose
+ parameters of SMPL model. It should be axis-angle vector
+ ([B, J*3]) or rotation matrix ([B, J, 3, 3)].
+ global_orient: Tensor([B, 3] or [B, 1, 3, 3]), global orientation
+ of human body. It should be axis-angle vector ([B, 3]) or
+ rotation matrix ([B, 1, 3, 3)].
+ transl: Tensor([B, 3]), global translation of human body.
+ gender: Tensor([B]), gender parameters of human body. -1 for
+ neutral, 0 for male , 1 for female.
+
+ Returns:
+ outputs (dict): Dict with mesh vertices and joints.
+ - vertices: Tensor([B, V, 3]), mesh vertices
+ - joints: Tensor([B, K, 3]), 3d joints regressed from
+ mesh vertices.
+ """
+
+ batch_size = betas.shape[0]
+ pose2rot = True if body_pose.dim() == 2 else False
+ if batch_size > 0 and gender is not None:
+ output = {
+ 'vertices': betas.new_zeros([batch_size, self.num_verts, 3]),
+ 'joints': betas.new_zeros([batch_size, self.num_joints, 3])
+ }
+
+ mask = gender < 0
+ _out = self.smpl_forward(
+ self.smpl_neutral,
+ betas=betas[mask],
+ body_pose=body_pose[mask],
+ global_orient=global_orient[mask],
+ transl=transl[mask] if transl is not None else None,
+ pose2rot=pose2rot)
+ output['vertices'][mask] = _out['vertices']
+ output['joints'][mask] = _out['joints']
+
+ mask = gender == 0
+ _out = self.smpl_forward(
+ self.smpl_male,
+ betas=betas[mask],
+ body_pose=body_pose[mask],
+ global_orient=global_orient[mask],
+ transl=transl[mask] if transl is not None else None,
+ pose2rot=pose2rot)
+ output['vertices'][mask] = _out['vertices']
+ output['joints'][mask] = _out['joints']
+
+ mask = gender == 1
+ _out = self.smpl_forward(
+ self.smpl_male,
+ betas=betas[mask],
+ body_pose=body_pose[mask],
+ global_orient=global_orient[mask],
+ transl=transl[mask] if transl is not None else None,
+ pose2rot=pose2rot)
+ output['vertices'][mask] = _out['vertices']
+ output['joints'][mask] = _out['joints']
+ else:
+ return self.smpl_forward(
+ self.smpl_neutral,
+ betas=betas,
+ body_pose=body_pose,
+ global_orient=global_orient,
+ transl=transl,
+ pose2rot=pose2rot)
+
+ return output
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/utils/__init__.py b/engine/pose_estimation/third-party/ViTPose/mmpose/utils/__init__.py
new file mode 100644
index 0000000..1293ca0
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/utils/__init__.py
@@ -0,0 +1,9 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from .collect_env import collect_env
+from .logger import get_root_logger
+from .setup_env import setup_multi_processes
+from .timer import StopWatch
+
+__all__ = [
+ 'get_root_logger', 'collect_env', 'StopWatch', 'setup_multi_processes'
+]
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/utils/collect_env.py b/engine/pose_estimation/third-party/ViTPose/mmpose/utils/collect_env.py
new file mode 100644
index 0000000..f75c5ea
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/utils/collect_env.py
@@ -0,0 +1,16 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from mmcv.utils import collect_env as collect_basic_env
+from mmcv.utils import get_git_hash
+
+import mmpose
+
+
+def collect_env():
+ env_info = collect_basic_env()
+ env_info['MMPose'] = (mmpose.__version__ + '+' + get_git_hash(digits=7))
+ return env_info
+
+
+if __name__ == '__main__':
+ for name, val in collect_env().items():
+ print(f'{name}: {val}')
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/utils/hooks.py b/engine/pose_estimation/third-party/ViTPose/mmpose/utils/hooks.py
new file mode 100644
index 0000000..b68940f
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/utils/hooks.py
@@ -0,0 +1,60 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import functools
+
+
+class OutputHook:
+
+ def __init__(self, module, outputs=None, as_tensor=False):
+ self.outputs = outputs
+ self.as_tensor = as_tensor
+ self.layer_outputs = {}
+ self.register(module)
+
+ def register(self, module):
+
+ def hook_wrapper(name):
+
+ def hook(model, input, output):
+ if self.as_tensor:
+ self.layer_outputs[name] = output
+ else:
+ if isinstance(output, list):
+ self.layer_outputs[name] = [
+ out.detach().cpu().numpy() for out in output
+ ]
+ else:
+ self.layer_outputs[name] = output.detach().cpu().numpy(
+ )
+
+ return hook
+
+ self.handles = []
+ if isinstance(self.outputs, (list, tuple)):
+ for name in self.outputs:
+ try:
+ layer = rgetattr(module, name)
+ h = layer.register_forward_hook(hook_wrapper(name))
+ except ModuleNotFoundError as module_not_found:
+ raise ModuleNotFoundError(
+ f'Module {name} not found') from module_not_found
+ self.handles.append(h)
+
+ def remove(self):
+ for h in self.handles:
+ h.remove()
+
+ def __enter__(self):
+ return self
+
+ def __exit__(self, exc_type, exc_val, exc_tb):
+ self.remove()
+
+
+# using wonder's beautiful simplification:
+# https://stackoverflow.com/questions/31174295/getattr-and-setattr-on-nested-objects
+def rgetattr(obj, attr, *args):
+
+ def _getattr(obj, attr):
+ return getattr(obj, attr, *args)
+
+ return functools.reduce(_getattr, [obj] + attr.split('.'))
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/utils/logger.py b/engine/pose_estimation/third-party/ViTPose/mmpose/utils/logger.py
new file mode 100644
index 0000000..294837f
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/utils/logger.py
@@ -0,0 +1,25 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import logging
+
+from mmcv.utils import get_logger
+
+
+def get_root_logger(log_file=None, log_level=logging.INFO):
+ """Use `get_logger` method in mmcv to get the root logger.
+
+ The logger will be initialized if it has not been initialized. By default a
+ StreamHandler will be added. If `log_file` is specified, a FileHandler will
+ also be added. The name of the root logger is the top-level package name,
+ e.g., "mmpose".
+
+ Args:
+ log_file (str | None): The log filename. If specified, a FileHandler
+ will be added to the root logger.
+ log_level (int): The root logger level. Note that only the process of
+ rank 0 is affected, while other processes will set the level to
+ "Error" and be silent most of the time.
+
+ Returns:
+ logging.Logger: The root logger.
+ """
+ return get_logger(__name__.split('.')[0], log_file, log_level)
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/utils/setup_env.py b/engine/pose_estimation/third-party/ViTPose/mmpose/utils/setup_env.py
new file mode 100644
index 0000000..21def2f
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/utils/setup_env.py
@@ -0,0 +1,47 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os
+import platform
+import warnings
+
+import cv2
+import torch.multiprocessing as mp
+
+
+def setup_multi_processes(cfg):
+ """Setup multi-processing environment variables."""
+ # set multi-process start method as `fork` to speed up the training
+ if platform.system() != 'Windows':
+ mp_start_method = cfg.get('mp_start_method', 'fork')
+ current_method = mp.get_start_method(allow_none=True)
+ if current_method is not None and current_method != mp_start_method:
+ warnings.warn(
+ f'Multi-processing start method `{mp_start_method}` is '
+ f'different from the previous setting `{current_method}`.'
+ f'It will be force set to `{mp_start_method}`. You can change '
+ f'this behavior by changing `mp_start_method` in your config.')
+ mp.set_start_method(mp_start_method, force=True)
+
+ # disable opencv multithreading to avoid system being overloaded
+ opencv_num_threads = cfg.get('opencv_num_threads', 0)
+ cv2.setNumThreads(opencv_num_threads)
+
+ # setup OMP threads
+ # This code is referred from https://github.com/pytorch/pytorch/blob/master/torch/distributed/run.py # noqa
+ if 'OMP_NUM_THREADS' not in os.environ and cfg.data.workers_per_gpu > 1:
+ omp_num_threads = 1
+ warnings.warn(
+ f'Setting OMP_NUM_THREADS environment variable for each process '
+ f'to be {omp_num_threads} in default, to avoid your system being '
+ f'overloaded, please further tune the variable for optimal '
+ f'performance in your application as needed.')
+ os.environ['OMP_NUM_THREADS'] = str(omp_num_threads)
+
+ # setup MKL threads
+ if 'MKL_NUM_THREADS' not in os.environ and cfg.data.workers_per_gpu > 1:
+ mkl_num_threads = 1
+ warnings.warn(
+ f'Setting MKL_NUM_THREADS environment variable for each process '
+ f'to be {mkl_num_threads} in default, to avoid your system being '
+ f'overloaded, please further tune the variable for optimal '
+ f'performance in your application as needed.')
+ os.environ['MKL_NUM_THREADS'] = str(mkl_num_threads)
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/utils/timer.py b/engine/pose_estimation/third-party/ViTPose/mmpose/utils/timer.py
new file mode 100644
index 0000000..5a3185c
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/utils/timer.py
@@ -0,0 +1,117 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from collections import defaultdict
+from contextlib import contextmanager
+from functools import partial
+
+import numpy as np
+from mmcv import Timer
+
+
+class RunningAverage():
+ r"""A helper class to calculate running average in a sliding window.
+
+ Args:
+ window (int): The size of the sliding window.
+ """
+
+ def __init__(self, window: int = 1):
+ self.window = window
+ self._data = []
+
+ def update(self, value):
+ """Update a new data sample."""
+ self._data.append(value)
+ self._data = self._data[-self.window:]
+
+ def average(self):
+ """Get the average value of current window."""
+ return np.mean(self._data)
+
+
+class StopWatch:
+ r"""A helper class to measure FPS and detailed time consuming of each phase
+ in a video processing loop or similar scenarios.
+
+ Args:
+ window (int): The sliding window size to calculate the running average
+ of the time consuming.
+
+ Example:
+ >>> from mmpose.utils import StopWatch
+ >>> import time
+ >>> stop_watch = StopWatch(window=10)
+ >>> with stop_watch.timeit('total'):
+ >>> time.sleep(0.1)
+ >>> # 'timeit' support nested use
+ >>> with stop_watch.timeit('phase1'):
+ >>> time.sleep(0.1)
+ >>> with stop_watch.timeit('phase2'):
+ >>> time.sleep(0.2)
+ >>> time.sleep(0.2)
+ >>> report = stop_watch.report()
+ """
+
+ def __init__(self, window=1):
+ self.window = window
+ self._record = defaultdict(partial(RunningAverage, window=self.window))
+ self._timer_stack = []
+
+ @contextmanager
+ def timeit(self, timer_name='_FPS_'):
+ """Timing a code snippet with an assigned name.
+
+ Args:
+ timer_name (str): The unique name of the interested code snippet to
+ handle multiple timers and generate reports. Note that '_FPS_'
+ is a special key that the measurement will be in `fps` instead
+ of `millisecond`. Also see `report` and `report_strings`.
+ Default: '_FPS_'.
+ Note:
+ This function should always be used in a `with` statement, as shown
+ in the example.
+ """
+ self._timer_stack.append((timer_name, Timer()))
+ try:
+ yield
+ finally:
+ timer_name, timer = self._timer_stack.pop()
+ self._record[timer_name].update(timer.since_start())
+
+ def report(self, key=None):
+ """Report timing information.
+
+ Returns:
+ dict: The key is the timer name and the value is the \
+ corresponding average time consuming.
+ """
+ result = {
+ name: r.average() * 1000.
+ for name, r in self._record.items()
+ }
+
+ if '_FPS_' in result:
+ result['_FPS_'] = 1000. / result.pop('_FPS_')
+
+ if key is None:
+ return result
+ return result[key]
+
+ def report_strings(self):
+ """Report timing information in texture strings.
+
+ Returns:
+ list(str): Each element is the information string of a timed \
+ event, in format of '{timer_name}: {time_in_ms}'. \
+ Specially, if timer_name is '_FPS_', the result will \
+ be converted to fps.
+ """
+ result = self.report()
+ strings = []
+ if '_FPS_' in result:
+ strings.append(f'FPS: {result["_FPS_"]:>5.1f}')
+ strings += [f'{name}: {val:>3.0f}' for name, val in result.items()]
+ return strings
+
+ def reset(self):
+ self._record = defaultdict(list)
+ self._active_timer_stack = []
diff --git a/engine/pose_estimation/third-party/ViTPose/mmpose/version.py b/engine/pose_estimation/third-party/ViTPose/mmpose/version.py
new file mode 100644
index 0000000..1a10826
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/mmpose/version.py
@@ -0,0 +1,19 @@
+# Copyright (c) Open-MMLab. All rights reserved.
+
+__version__ = '0.24.0'
+short_version = __version__
+
+
+def parse_version_info(version_str):
+ version_info = []
+ for x in version_str.split('.'):
+ if x.isdigit():
+ version_info.append(int(x))
+ elif x.find('rc') != -1:
+ patch_version = x.split('rc')
+ version_info.append(int(patch_version[0]))
+ version_info.append(f'rc{patch_version[1]}')
+ return tuple(version_info)
+
+
+version_info = parse_version_info(__version__)
diff --git a/engine/pose_estimation/third-party/ViTPose/model-index.yml b/engine/pose_estimation/third-party/ViTPose/model-index.yml
new file mode 100644
index 0000000..c5522f6
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/model-index.yml
@@ -0,0 +1,139 @@
+Import:
+- configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap/animalpose/hrnet_animalpose.yml
+- configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap/animalpose/resnet_animalpose.yml
+- configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap/ap10k/hrnet_ap10k.yml
+- configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap/ap10k/resnet_ap10k.yml
+- configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap/atrw/hrnet_atrw.yml
+- configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap/atrw/resnet_atrw.yml
+- configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap/fly/resnet_fly.yml
+- configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap/horse10/hrnet_horse10.yml
+- configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap/horse10/resnet_horse10.yml
+- configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap/locust/resnet_locust.yml
+- configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap/macaque/hrnet_macaque.yml
+- configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap/macaque/resnet_macaque.yml
+- configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap/zebra/resnet_zebra.yml
+- configs/body/2d_kpt_sview_rgb_img/associative_embedding/aic/higherhrnet_aic.yml
+- configs/body/2d_kpt_sview_rgb_img/associative_embedding/aic/hrnet_aic.yml
+- configs/body/2d_kpt_sview_rgb_img/associative_embedding/coco/higherhrnet_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/associative_embedding/coco/higherhrnet_udp_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/associative_embedding/coco/hourglass_ae_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/associative_embedding/coco/hrnet_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/associative_embedding/coco/hrnet_udp_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/associative_embedding/coco/mobilenetv2_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/associative_embedding/coco/resnet_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/associative_embedding/crowdpose/higherhrnet_crowdpose.yml
+- configs/body/2d_kpt_sview_rgb_img/associative_embedding/mhp/hrnet_mhp.yml
+- configs/body/2d_kpt_sview_rgb_img/deeppose/coco/resnet_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/deeppose/mpii/resnet_mpii.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/aic/hrnet_aic.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/aic/resnet_aic.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/alexnet_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/cpm_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/hourglass_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/hrformer_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/hrnet_augmentation_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/hrnet_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/hrnet_dark_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/hrnet_fp16_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/hrnet_udp_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/litehrnet_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/mobilenetv2_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/mspn_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/resnest_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/resnet_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/resnet_dark_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/resnet_fp16_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/resnetv1d_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/resnext_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/rsn_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/scnet_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/seresnet_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/shufflenetv1_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/shufflenetv2_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/vgg_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/vipnas_coco.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/crowdpose/hrnet_crowdpose.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/crowdpose/resnet_crowdpose.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/h36m/hrnet_h36m.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/jhmdb/cpm_jhmdb.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/jhmdb/resnet_jhmdb.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/mhp/resnet_mhp.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/mpii/cpm_mpii.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/mpii/hourglass_mpii.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/mpii/hrnet_dark_mpii.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/mpii/hrnet_mpii.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/mpii/litehrnet_mpii.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/mpii/mobilenetv2_mpii.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/mpii/resnet_mpii.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/mpii/resnetv1d_mpii.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/mpii/resnext_mpii.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/mpii/scnet_mpii.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/mpii/seresnet_mpii.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/mpii/shufflenetv1_mpii.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/mpii/shufflenetv2_mpii.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/mpii_trb/resnet_mpii_trb.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/ochuman/hrnet_ochuman.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/ochuman/resnet_ochuman.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/posetrack18/hrnet_posetrack18.yml
+- configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/posetrack18/resnet_posetrack18.yml
+- configs/body/2d_kpt_sview_rgb_vid/posewarper/posetrack18/hrnet_posetrack18_posewarper.yml
+- configs/body/3d_kpt_mview_rgb_img/voxelpose/panoptic/voxelpose_prn64x64x64_cpn80x80x20_panoptic_cam5.yml
+- configs/body/3d_kpt_sview_rgb_img/pose_lift/h36m/simplebaseline3d_h36m.yml
+- configs/body/3d_kpt_sview_rgb_img/pose_lift/mpi_inf_3dhp/simplebaseline3d_mpi-inf-3dhp.yml
+- configs/body/3d_kpt_sview_rgb_vid/video_pose_lift/h36m/videopose3d_h36m.yml
+- configs/body/3d_kpt_sview_rgb_vid/video_pose_lift/mpi_inf_3dhp/videopose3d_mpi-inf-3dhp.yml
+- configs/body/3d_mesh_sview_rgb_img/hmr/mixed/resnet_mixed.yml
+- configs/face/2d_kpt_sview_rgb_img/deeppose/wflw/resnet_softwingloss_wflw.yml
+- configs/face/2d_kpt_sview_rgb_img/deeppose/wflw/resnet_wflw.yml
+- configs/face/2d_kpt_sview_rgb_img/deeppose/wflw/resnet_wingloss_wflw.yml
+- configs/face/2d_kpt_sview_rgb_img/topdown_heatmap/300w/hrnetv2_300w.yml
+- configs/face/2d_kpt_sview_rgb_img/topdown_heatmap/aflw/hrnetv2_aflw.yml
+- configs/face/2d_kpt_sview_rgb_img/topdown_heatmap/aflw/hrnetv2_dark_aflw.yml
+- configs/face/2d_kpt_sview_rgb_img/topdown_heatmap/coco_wholebody_face/hourglass_coco_wholebody_face.yml
+- configs/face/2d_kpt_sview_rgb_img/topdown_heatmap/coco_wholebody_face/hrnetv2_coco_wholebody_face.yml
+- configs/face/2d_kpt_sview_rgb_img/topdown_heatmap/coco_wholebody_face/hrnetv2_dark_coco_wholebody_face.yml
+- configs/face/2d_kpt_sview_rgb_img/topdown_heatmap/coco_wholebody_face/mobilenetv2_coco_wholebody_face.yml
+- configs/face/2d_kpt_sview_rgb_img/topdown_heatmap/coco_wholebody_face/resnet_coco_wholebody_face.yml
+- configs/face/2d_kpt_sview_rgb_img/topdown_heatmap/coco_wholebody_face/scnet_coco_wholebody_face.yml
+- configs/face/2d_kpt_sview_rgb_img/topdown_heatmap/cofw/hrnetv2_cofw.yml
+- configs/face/2d_kpt_sview_rgb_img/topdown_heatmap/wflw/hrnetv2_awing_wflw.yml
+- configs/face/2d_kpt_sview_rgb_img/topdown_heatmap/wflw/hrnetv2_dark_wflw.yml
+- configs/face/2d_kpt_sview_rgb_img/topdown_heatmap/wflw/hrnetv2_wflw.yml
+- configs/fashion/2d_kpt_sview_rgb_img/deeppose/deepfashion/resnet_deepfashion.yml
+- configs/fashion/2d_kpt_sview_rgb_img/topdown_heatmap/deepfashion/resnet_deepfashion.yml
+- configs/hand/2d_kpt_sview_rgb_img/deeppose/onehand10k/resnet_onehand10k.yml
+- configs/hand/2d_kpt_sview_rgb_img/deeppose/panoptic2d/resnet_panoptic2d.yml
+- configs/hand/2d_kpt_sview_rgb_img/deeppose/rhd2d/resnet_rhd2d.yml
+- configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/coco_wholebody_hand/hourglass_coco_wholebody_hand.yml
+- configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/coco_wholebody_hand/hrnetv2_coco_wholebody_hand.yml
+- configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/coco_wholebody_hand/hrnetv2_dark_coco_wholebody_hand.yml
+- configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/coco_wholebody_hand/litehrnet_coco_wholebody_hand.yml
+- configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/coco_wholebody_hand/mobilenetv2_coco_wholebody_hand.yml
+- configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/coco_wholebody_hand/resnet_coco_wholebody_hand.yml
+- configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/coco_wholebody_hand/scnet_coco_wholebody_hand.yml
+- configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/freihand2d/resnet_freihand2d.yml
+- configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/interhand2d/resnet_interhand2d.yml
+- configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/onehand10k/hrnetv2_dark_onehand10k.yml
+- configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/onehand10k/hrnetv2_onehand10k.yml
+- configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/onehand10k/hrnetv2_udp_onehand10k.yml
+- configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/onehand10k/mobilenetv2_onehand10k.yml
+- configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/onehand10k/resnet_onehand10k.yml
+- configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/panoptic2d/hrnetv2_dark_panoptic2d.yml
+- configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/panoptic2d/hrnetv2_panoptic2d.yml
+- configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/panoptic2d/hrnetv2_udp_panoptic2d.yml
+- configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/panoptic2d/mobilenetv2_panoptic2d.yml
+- configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/panoptic2d/resnet_panoptic2d.yml
+- configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/rhd2d/hrnetv2_dark_rhd2d.yml
+- configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/rhd2d/hrnetv2_rhd2d.yml
+- configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/rhd2d/hrnetv2_udp_rhd2d.yml
+- configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/rhd2d/mobilenetv2_rhd2d.yml
+- configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/rhd2d/resnet_rhd2d.yml
+- configs/hand/3d_kpt_sview_rgb_img/internet/interhand3d/internet_interhand3d.yml
+- configs/wholebody/2d_kpt_sview_rgb_img/associative_embedding/coco-wholebody/higherhrnet_coco-wholebody.yml
+- configs/wholebody/2d_kpt_sview_rgb_img/associative_embedding/coco-wholebody/hrnet_coco-wholebody.yml
+- configs/wholebody/2d_kpt_sview_rgb_img/topdown_heatmap/coco-wholebody/hrnet_coco-wholebody.yml
+- configs/wholebody/2d_kpt_sview_rgb_img/topdown_heatmap/coco-wholebody/hrnet_dark_coco-wholebody.yml
+- configs/wholebody/2d_kpt_sview_rgb_img/topdown_heatmap/coco-wholebody/resnet_coco-wholebody.yml
+- configs/wholebody/2d_kpt_sview_rgb_img/topdown_heatmap/coco-wholebody/vipnas_coco-wholebody.yml
+- configs/wholebody/2d_kpt_sview_rgb_img/topdown_heatmap/coco-wholebody/vipnas_dark_coco-wholebody.yml
+- configs/wholebody/2d_kpt_sview_rgb_img/topdown_heatmap/halpe/hrnet_dark_halpe.yml
diff --git a/engine/pose_estimation/third-party/ViTPose/pytest.ini b/engine/pose_estimation/third-party/ViTPose/pytest.ini
new file mode 100644
index 0000000..9796e87
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/pytest.ini
@@ -0,0 +1,7 @@
+[pytest]
+addopts = --xdoctest --xdoctest-style=auto
+norecursedirs = .git ignore build __pycache__ data docker docs .eggs
+
+filterwarnings= default
+ ignore:.*No cfgstr given in Cacher constructor or call.*:Warning
+ ignore:.*Define the __nice__ method for.*:Warning
diff --git a/engine/pose_estimation/third-party/ViTPose/requirements.txt b/engine/pose_estimation/third-party/ViTPose/requirements.txt
new file mode 100644
index 0000000..b5b5d97
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/requirements.txt
@@ -0,0 +1,4 @@
+-r requirements/build.txt
+-r requirements/runtime.txt
+-r requirements/tests.txt
+-r requirements/optional.txt
diff --git a/engine/pose_estimation/third-party/ViTPose/requirements/build.txt b/engine/pose_estimation/third-party/ViTPose/requirements/build.txt
new file mode 100644
index 0000000..ddd3b82
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/requirements/build.txt
@@ -0,0 +1,2 @@
+# These must be installed before building mmpose
+numpy
diff --git a/engine/pose_estimation/third-party/ViTPose/requirements/docs.txt b/engine/pose_estimation/third-party/ViTPose/requirements/docs.txt
new file mode 100644
index 0000000..2017084
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/requirements/docs.txt
@@ -0,0 +1,6 @@
+docutils==0.16.0
+myst-parser
+-e git+https://github.com/gaotongxiao/pytorch_sphinx_theme.git#egg=pytorch_sphinx_theme
+sphinx==4.0.2
+sphinx_copybutton
+sphinx_markdown_tables
diff --git a/engine/pose_estimation/third-party/ViTPose/requirements/mminstall.txt b/engine/pose_estimation/third-party/ViTPose/requirements/mminstall.txt
new file mode 100644
index 0000000..89199e3
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/requirements/mminstall.txt
@@ -0,0 +1,3 @@
+mmcv-full>=1.3.8
+mmdet>=2.14.0
+mmtrack>=0.6.0
diff --git a/engine/pose_estimation/third-party/ViTPose/requirements/optional.txt b/engine/pose_estimation/third-party/ViTPose/requirements/optional.txt
new file mode 100644
index 0000000..bfb1e75
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/requirements/optional.txt
@@ -0,0 +1,8 @@
+albumentations>=0.3.2 --no-binary qudida,albumentations
+onnx
+onnxruntime
+poseval@git+https://github.com/svenkreiss/poseval.git
+pyrender
+requests
+smplx>=0.1.28
+trimesh
diff --git a/engine/pose_estimation/third-party/ViTPose/requirements/readthedocs.txt b/engine/pose_estimation/third-party/ViTPose/requirements/readthedocs.txt
new file mode 100644
index 0000000..b8b69d3
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/requirements/readthedocs.txt
@@ -0,0 +1,9 @@
+mmcv-full
+munkres
+poseval@git+https://github.com/svenkreiss/poseval.git
+regex
+scipy
+titlecase
+torch
+torchvision
+xtcocotools>=1.8
diff --git a/engine/pose_estimation/third-party/ViTPose/requirements/runtime.txt b/engine/pose_estimation/third-party/ViTPose/requirements/runtime.txt
new file mode 100644
index 0000000..2d2a96c
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/requirements/runtime.txt
@@ -0,0 +1,9 @@
+chumpy
+json_tricks
+matplotlib
+munkres
+numpy
+opencv-python
+pillow
+scipy
+xtcocotools>=1.8
diff --git a/engine/pose_estimation/third-party/ViTPose/requirements/tests.txt b/engine/pose_estimation/third-party/ViTPose/requirements/tests.txt
new file mode 100644
index 0000000..aa23e69
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/requirements/tests.txt
@@ -0,0 +1,9 @@
+coverage
+flake8
+interrogate
+isort==4.3.21
+pytest
+pytest-runner
+smplx>=0.1.28
+xdoctest>=0.10.0
+yapf
diff --git a/engine/pose_estimation/third-party/ViTPose/resources/mmpose-logo.png b/engine/pose_estimation/third-party/ViTPose/resources/mmpose-logo.png
new file mode 100644
index 0000000..128e171
Binary files /dev/null and b/engine/pose_estimation/third-party/ViTPose/resources/mmpose-logo.png differ
diff --git a/engine/pose_estimation/third-party/ViTPose/setup.cfg b/engine/pose_estimation/third-party/ViTPose/setup.cfg
new file mode 100644
index 0000000..c4d8643
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/setup.cfg
@@ -0,0 +1,24 @@
+[bdist_wheel]
+universal=1
+
+[aliases]
+test=pytest
+
+[tool:pytest]
+addopts=tests/
+
+[yapf]
+based_on_style = pep8
+blank_line_before_nested_class_or_def = true
+split_before_expression_after_opening_paren = true
+split_penalty_import_names=0
+SPLIT_PENALTY_AFTER_OPENING_BRACKET=800
+
+[isort]
+line_length = 79
+multi_line_output = 0
+extra_standard_library = pkg_resources,setuptools
+known_first_party = mmpose
+known_third_party = PIL,cv2,h5py,json_tricks,matplotlib,mmcv,munkres,numpy,pytest,pytorch_sphinx_theme,requests,scipy,seaborn,spacepy,titlecase,torch,torchvision,webcam_apis,xmltodict,xtcocotools
+no_lines_before = STDLIB,LOCALFOLDER
+default_section = THIRDPARTY
diff --git a/engine/pose_estimation/third-party/ViTPose/setup.py b/engine/pose_estimation/third-party/ViTPose/setup.py
new file mode 100644
index 0000000..c72e8ce
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/setup.py
@@ -0,0 +1,193 @@
+import os
+import os.path as osp
+import platform
+import shutil
+import sys
+import warnings
+from setuptools import find_packages, setup
+
+
+def readme():
+ with open('README.md', encoding='utf-8') as f:
+ content = f.read()
+ return content
+
+
+version_file = 'mmpose/version.py'
+
+
+def get_version():
+ with open(version_file, 'r') as f:
+ exec(compile(f.read(), version_file, 'exec'))
+ import sys
+
+ # return short version for sdist
+ if 'sdist' in sys.argv or 'bdist_wheel' in sys.argv:
+ return locals()['short_version']
+ else:
+ return locals()['__version__']
+
+
+def parse_requirements(fname='requirements.txt', with_version=True):
+ """Parse the package dependencies listed in a requirements file but strips
+ specific versioning information.
+
+ Args:
+ fname (str): path to requirements file
+ with_version (bool, default=False): if True include version specs
+
+ Returns:
+ List[str]: list of requirements items
+
+ CommandLine:
+ python -c "import setup; print(setup.parse_requirements())"
+ """
+ import re
+ import sys
+ from os.path import exists
+ require_fpath = fname
+
+ def parse_line(line):
+ """Parse information from a line in a requirements text file."""
+ if line.startswith('-r '):
+ # Allow specifying requirements in other files
+ target = line.split(' ')[1]
+ for info in parse_require_file(target):
+ yield info
+ else:
+ info = {'line': line}
+ if line.startswith('-e '):
+ info['package'] = line.split('#egg=')[1]
+ elif '@git+' in line:
+ info['package'] = line
+ else:
+ # Remove versioning from the package
+ pat = '(' + '|'.join(['>=', '==', '>']) + ')'
+ parts = re.split(pat, line, maxsplit=1)
+ parts = [p.strip() for p in parts]
+
+ info['package'] = parts[0]
+ if len(parts) > 1:
+ op, rest = parts[1:]
+ if ';' in rest:
+ # Handle platform specific dependencies
+ # http://setuptools.readthedocs.io/en/latest/setuptools.html#declaring-platform-specific-dependencies
+ version, platform_deps = map(str.strip,
+ rest.split(';'))
+ info['platform_deps'] = platform_deps
+ else:
+ version = rest # NOQA
+ info['version'] = (op, version)
+ yield info
+
+ def parse_require_file(fpath):
+ with open(fpath, 'r') as f:
+ for line in f.readlines():
+ line = line.strip()
+ if line and not line.startswith('#'):
+ for info in parse_line(line):
+ yield info
+
+ def gen_packages_items():
+ if exists(require_fpath):
+ for info in parse_require_file(require_fpath):
+ parts = [info['package']]
+ if with_version and 'version' in info:
+ parts.extend(info['version'])
+ if not sys.version.startswith('3.4'):
+ # apparently package_deps are broken in 3.4
+ platform_deps = info.get('platform_deps')
+ if platform_deps is not None:
+ parts.append(';' + platform_deps)
+ item = ''.join(parts)
+ yield item
+
+ packages = list(gen_packages_items())
+ return packages
+
+
+def add_mim_extension():
+ """Add extra files that are required to support MIM into the package.
+
+ These files will be added by creating a symlink to the originals if the
+ package is installed in `editable` mode (e.g. pip install -e .), or by
+ copying from the originals otherwise.
+ """
+
+ # parse installment mode
+ if 'develop' in sys.argv:
+ # installed by `pip install -e .`
+ if platform.system() == 'Windows':
+ mode = 'copy'
+ else:
+ mode = 'symlink'
+ elif 'sdist' in sys.argv or 'bdist_wheel' in sys.argv:
+ # installed by `pip install .`
+ # or create source distribution by `python setup.py sdist`
+ mode = 'copy'
+ else:
+ return
+
+ filenames = ['tools', 'configs', 'demo', 'model-index.yml']
+ repo_path = osp.dirname(__file__)
+ mim_path = osp.join(repo_path, 'mmpose', '.mim')
+ os.makedirs(mim_path, exist_ok=True)
+
+ for filename in filenames:
+ if osp.exists(filename):
+ src_path = osp.join(repo_path, filename)
+ tar_path = osp.join(mim_path, filename)
+
+ if osp.isfile(tar_path) or osp.islink(tar_path):
+ os.remove(tar_path)
+ elif osp.isdir(tar_path):
+ shutil.rmtree(tar_path)
+
+ if mode == 'symlink':
+ src_relpath = osp.relpath(src_path, osp.dirname(tar_path))
+ os.symlink(src_relpath, tar_path)
+ elif mode == 'copy':
+ if osp.isfile(src_path):
+ shutil.copyfile(src_path, tar_path)
+ elif osp.isdir(src_path):
+ shutil.copytree(src_path, tar_path)
+ else:
+ warnings.warn(f'Cannot copy file {src_path}.')
+ else:
+ raise ValueError(f'Invalid mode {mode}')
+
+
+if __name__ == '__main__':
+ add_mim_extension()
+ setup(
+ name='mmpose',
+ version=get_version(),
+ description='OpenMMLab Pose Estimation Toolbox and Benchmark.',
+ author='MMPose Contributors',
+ author_email='openmmlab@gmail.com',
+ keywords='computer vision, pose estimation',
+ long_description=readme(),
+ long_description_content_type='text/markdown',
+ packages=find_packages(exclude=('configs', 'tools', 'demo')),
+ include_package_data=True,
+ package_data={'mmpose.ops': ['*/*.so']},
+ classifiers=[
+ 'Development Status :: 4 - Beta',
+ 'License :: OSI Approved :: Apache Software License',
+ 'Operating System :: OS Independent',
+ 'Programming Language :: Python :: 3',
+ 'Programming Language :: Python :: 3.5',
+ 'Programming Language :: Python :: 3.6',
+ 'Programming Language :: Python :: 3.7',
+ 'Programming Language :: Python :: 3.8',
+ 'Programming Language :: Python :: 3.9',
+ ],
+ url='https://github.com/open-mmlab/mmpose',
+ license='Apache License 2.0',
+ install_requires=parse_requirements('requirements/runtime.txt'),
+ extras_require={
+ 'tests': parse_requirements('requirements/tests.txt'),
+ 'build': parse_requirements('requirements/build.txt'),
+ 'runtime': parse_requirements('requirements/runtime.txt')
+ },
+ zip_safe=False)
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/__init__.py b/engine/pose_estimation/third-party/ViTPose/tests/__init__.py
new file mode 100644
index 0000000..ef101fe
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/__init__.py
@@ -0,0 +1 @@
+# Copyright (c) OpenMMLab. All rights reserved.
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_apis/test_inference.py b/engine/pose_estimation/third-party/ViTPose/tests/test_apis/test_inference.py
new file mode 100644
index 0000000..fbdb614
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_apis/test_inference.py
@@ -0,0 +1,198 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import copy
+
+import numpy as np
+
+from mmpose.apis import (inference_bottom_up_pose_model,
+ inference_top_down_pose_model, init_pose_model,
+ process_mmdet_results, vis_pose_result)
+from mmpose.datasets import DatasetInfo
+
+
+def test_top_down_demo():
+ # COCO demo
+ # build the pose model from a config file and a checkpoint file
+ pose_model = init_pose_model(
+ 'configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/'
+ 'coco/res50_coco_256x192.py',
+ None,
+ device='cpu')
+ image_name = 'tests/data/coco/000000000785.jpg'
+ dataset_info = DatasetInfo(pose_model.cfg.data['test'].get(
+ 'dataset_info', None))
+
+ person_result = []
+ person_result.append({'bbox': [50, 50, 50, 100]})
+ # test a single image, with a list of bboxes.
+ pose_results, _ = inference_top_down_pose_model(
+ pose_model,
+ image_name,
+ person_result,
+ format='xywh',
+ dataset_info=dataset_info)
+ # show the results
+ vis_pose_result(
+ pose_model, image_name, pose_results, dataset_info=dataset_info)
+
+ # AIC demo
+ pose_model = init_pose_model(
+ 'configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/'
+ 'aic/res50_aic_256x192.py',
+ None,
+ device='cpu')
+ image_name = 'tests/data/aic/054d9ce9201beffc76e5ff2169d2af2f027002ca.jpg'
+ dataset_info = DatasetInfo(pose_model.cfg.data['test'].get(
+ 'dataset_info', None))
+ # test a single image, with a list of bboxes.
+ pose_results, _ = inference_top_down_pose_model(
+ pose_model,
+ image_name,
+ person_result,
+ format='xywh',
+ dataset_info=dataset_info)
+ # show the results
+ vis_pose_result(
+ pose_model, image_name, pose_results, dataset_info=dataset_info)
+
+ # OneHand10K demo
+ # build the pose model from a config file and a checkpoint file
+ pose_model = init_pose_model(
+ 'configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/'
+ 'onehand10k/res50_onehand10k_256x256.py',
+ None,
+ device='cpu')
+ image_name = 'tests/data/onehand10k/9.jpg'
+ dataset_info = DatasetInfo(pose_model.cfg.data['test'].get(
+ 'dataset_info', None))
+ # test a single image, with a list of bboxes.
+ pose_results, _ = inference_top_down_pose_model(
+ pose_model,
+ image_name,
+ person_result,
+ format='xywh',
+ dataset_info=dataset_info)
+ # show the results
+ vis_pose_result(
+ pose_model, image_name, pose_results, dataset_info=dataset_info)
+
+ # InterHand2DDataset demo
+ # build the pose model from a config file and a checkpoint file
+ pose_model = init_pose_model(
+ 'configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/'
+ 'interhand2d/res50_interhand2d_all_256x256.py',
+ None,
+ device='cpu')
+ image_name = 'tests/data/interhand2.6m/image2017.jpg'
+ dataset_info = DatasetInfo(pose_model.cfg.data['test'].get(
+ 'dataset_info', None))
+ # test a single image, with a list of bboxes.
+ pose_results, _ = inference_top_down_pose_model(
+ pose_model,
+ image_name,
+ person_result,
+ format='xywh',
+ dataset_info=dataset_info)
+ # show the results
+ vis_pose_result(
+ pose_model, image_name, pose_results, dataset_info=dataset_info)
+
+ # Face300WDataset demo
+ # build the pose model from a config file and a checkpoint file
+ pose_model = init_pose_model(
+ 'configs/face/2d_kpt_sview_rgb_img/topdown_heatmap/'
+ '300w/res50_300w_256x256.py',
+ None,
+ device='cpu')
+ image_name = 'tests/data/300w/indoor_020.png'
+ dataset_info = DatasetInfo(pose_model.cfg.data['test'].get(
+ 'dataset_info', None))
+ # test a single image, with a list of bboxes.
+ pose_results, _ = inference_top_down_pose_model(
+ pose_model,
+ image_name,
+ person_result,
+ format='xywh',
+ dataset_info=dataset_info)
+ # show the results
+ vis_pose_result(
+ pose_model, image_name, pose_results, dataset_info=dataset_info)
+
+ # FaceAFLWDataset demo
+ # build the pose model from a config file and a checkpoint file
+ pose_model = init_pose_model(
+ 'configs/face/2d_kpt_sview_rgb_img/topdown_heatmap/'
+ 'aflw/res50_aflw_256x256.py',
+ None,
+ device='cpu')
+ image_name = 'tests/data/aflw/image04476.jpg'
+ dataset_info = DatasetInfo(pose_model.cfg.data['test'].get(
+ 'dataset_info', None))
+ # test a single image, with a list of bboxes.
+ pose_results, _ = inference_top_down_pose_model(
+ pose_model,
+ image_name,
+ person_result,
+ format='xywh',
+ dataset_info=dataset_info)
+ # show the results
+ vis_pose_result(
+ pose_model, image_name, pose_results, dataset_info=dataset_info)
+
+ # FaceCOFWDataset demo
+ # build the pose model from a config file and a checkpoint file
+ pose_model = init_pose_model(
+ 'configs/face/2d_kpt_sview_rgb_img/topdown_heatmap/'
+ 'cofw/res50_cofw_256x256.py',
+ None,
+ device='cpu')
+ image_name = 'tests/data/cofw/001766.jpg'
+ dataset_info = DatasetInfo(pose_model.cfg.data['test'].get(
+ 'dataset_info', None))
+ # test a single image, with a list of bboxes.
+ pose_results, _ = inference_top_down_pose_model(
+ pose_model,
+ image_name,
+ person_result,
+ format='xywh',
+ dataset_info=dataset_info)
+ # show the results
+ vis_pose_result(
+ pose_model, image_name, pose_results, dataset_info=dataset_info)
+
+
+def test_bottom_up_demo():
+
+ # build the pose model from a config file and a checkpoint file
+ pose_model = init_pose_model(
+ 'configs/body/2d_kpt_sview_rgb_img/associative_embedding/'
+ 'coco/res50_coco_512x512.py',
+ None,
+ device='cpu')
+
+ image_name = 'tests/data/coco/000000000785.jpg'
+ dataset_info = DatasetInfo(pose_model.cfg.data['test'].get(
+ 'dataset_info', None))
+
+ pose_results, _ = inference_bottom_up_pose_model(
+ pose_model, image_name, dataset_info=dataset_info)
+
+ # show the results
+ vis_pose_result(
+ pose_model, image_name, pose_results, dataset_info=dataset_info)
+
+ # test dataset_info without sigmas
+ pose_model_copy = copy.deepcopy(pose_model)
+
+ pose_model_copy.cfg.data.test.dataset_info.pop('sigmas')
+ pose_results, _ = inference_bottom_up_pose_model(
+ pose_model_copy, image_name, dataset_info=dataset_info)
+
+
+def test_process_mmdet_results():
+ det_results = [np.array([0, 0, 100, 100])]
+ det_mask_results = None
+
+ _ = process_mmdet_results(
+ mmdet_results=(det_results, det_mask_results), cat_id=1)
+
+ _ = process_mmdet_results(mmdet_results=det_results, cat_id=1)
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_apis/test_inference_3d.py b/engine/pose_estimation/third-party/ViTPose/tests/test_apis/test_inference_3d.py
new file mode 100644
index 0000000..350acd7
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_apis/test_inference_3d.py
@@ -0,0 +1,210 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os.path as osp
+import tempfile
+
+import mmcv
+import numpy as np
+import pytest
+import torch
+
+from mmpose.apis import (extract_pose_sequence, inference_interhand_3d_model,
+ inference_mesh_model, inference_pose_lifter_model,
+ init_pose_model, vis_3d_mesh_result,
+ vis_3d_pose_result)
+from mmpose.datasets.dataset_info import DatasetInfo
+from tests.utils.mesh_utils import generate_smpl_weight_file
+
+
+def test_pose_lifter_demo():
+ # H36M demo
+ pose_model = init_pose_model(
+ 'configs/body/3d_kpt_sview_rgb_img/pose_lift/'
+ 'h36m/simplebaseline3d_h36m.py',
+ None,
+ device='cpu')
+
+ pose_det_result = {
+ 'keypoints': np.zeros((17, 3)),
+ 'bbox': [50, 50, 50, 50],
+ 'track_id': 0,
+ 'image_name': 'tests/data/h36m/S1_Directions_1.54138969_000001.jpg',
+ }
+
+ pose_results_2d = [[pose_det_result]]
+
+ dataset_info = DatasetInfo(pose_model.cfg.data['test']['dataset_info'])
+
+ pose_results_2d = extract_pose_sequence(
+ pose_results_2d, frame_idx=0, causal=False, seq_len=1, step=1)
+
+ _ = inference_pose_lifter_model(
+ pose_model,
+ pose_results_2d,
+ dataset_info=dataset_info,
+ with_track_id=False)
+
+ pose_lift_results = inference_pose_lifter_model(
+ pose_model,
+ pose_results_2d,
+ dataset_info=dataset_info,
+ with_track_id=True)
+
+ for res in pose_lift_results:
+ res['title'] = 'title'
+ vis_3d_pose_result(
+ pose_model,
+ pose_lift_results,
+ img=pose_results_2d[0][0]['image_name'],
+ dataset_info=dataset_info)
+
+ # test special cases
+ # Empty 2D results
+ _ = inference_pose_lifter_model(
+ pose_model, [[]], dataset_info=dataset_info, with_track_id=False)
+
+ if torch.cuda.is_available():
+ _ = inference_pose_lifter_model(
+ pose_model.cuda(),
+ pose_results_2d,
+ dataset_info=dataset_info,
+ with_track_id=False)
+
+ # test videopose3d
+ pose_model = init_pose_model(
+ 'configs/body/3d_kpt_sview_rgb_vid/video_pose_lift/h36m/'
+ 'videopose3d_h36m_243frames_fullconv_supervised_cpn_ft.py',
+ None,
+ device='cpu')
+
+ pose_det_result_0 = {
+ 'keypoints': np.ones((17, 3)),
+ 'bbox': [50, 50, 100, 100],
+ 'track_id': 0,
+ 'image_name': 'tests/data/h36m/S1_Directions_1.54138969_000001.jpg',
+ }
+ pose_det_result_1 = {
+ 'keypoints': np.ones((17, 3)),
+ 'bbox': [50, 50, 100, 100],
+ 'track_id': 1,
+ 'image_name': 'tests/data/h36m/S5_SittingDown.54138969_002061.jpg',
+ }
+ pose_det_result_2 = {
+ 'keypoints': np.ones((17, 3)),
+ 'bbox': [50, 50, 100, 100],
+ 'track_id': 2,
+ 'image_name': 'tests/data/h36m/S7_Greeting.55011271_000396.jpg',
+ }
+
+ pose_results_2d = [[pose_det_result_0], [pose_det_result_1],
+ [pose_det_result_2]]
+
+ dataset_info = DatasetInfo(pose_model.cfg.data['test']['dataset_info'])
+
+ seq_len = pose_model.cfg.test_data_cfg.seq_len
+ pose_results_2d_seq = extract_pose_sequence(
+ pose_results_2d, 1, causal=False, seq_len=seq_len, step=1)
+
+ pose_lift_results = inference_pose_lifter_model(
+ pose_model,
+ pose_results_2d_seq,
+ dataset_info=dataset_info,
+ with_track_id=True,
+ image_size=[1000, 1000],
+ norm_pose_2d=True)
+
+ for res in pose_lift_results:
+ res['title'] = 'title'
+ vis_3d_pose_result(
+ pose_model,
+ pose_lift_results,
+ img=pose_results_2d[0][0]['image_name'],
+ dataset_info=dataset_info,
+ )
+
+
+def test_interhand3d_demo():
+ # H36M demo
+ pose_model = init_pose_model(
+ 'configs/hand/3d_kpt_sview_rgb_img/internet/interhand3d/'
+ 'res50_interhand3d_all_256x256.py',
+ None,
+ device='cpu')
+
+ image_name = 'tests/data/interhand2.6m/image2017.jpg'
+ det_result = {
+ 'image_name': image_name,
+ 'bbox': [50, 50, 50, 50], # bbox format is 'xywh'
+ 'camera_param': None,
+ 'keypoints_3d_gt': None
+ }
+ det_results = [det_result]
+ dataset = pose_model.cfg.data['test']['type']
+ dataset_info = DatasetInfo(pose_model.cfg.data['test']['dataset_info'])
+
+ pose_results = inference_interhand_3d_model(
+ pose_model, image_name, det_results, dataset=dataset)
+
+ for res in pose_results:
+ res['title'] = 'title'
+
+ vis_3d_pose_result(
+ pose_model,
+ result=pose_results,
+ img=det_results[0]['image_name'],
+ dataset_info=dataset_info,
+ )
+
+ # test special cases
+ # Empty det results
+ _ = inference_interhand_3d_model(
+ pose_model, image_name, [], dataset=dataset)
+
+ if torch.cuda.is_available():
+ _ = inference_interhand_3d_model(
+ pose_model.cuda(), image_name, det_results, dataset=dataset)
+
+ with pytest.raises(NotImplementedError):
+ _ = inference_interhand_3d_model(
+ pose_model, image_name, det_results, dataset='test')
+
+
+def test_body_mesh_demo():
+ # H36M demo
+ config = 'configs/body/3d_mesh_sview_rgb_img/hmr' \
+ '/mixed/res50_mixed_224x224.py'
+ config = mmcv.Config.fromfile(config)
+ config.model.mesh_head.smpl_mean_params = \
+ 'tests/data/smpl/smpl_mean_params.npz'
+
+ pose_model = None
+ with tempfile.TemporaryDirectory() as tmpdir:
+ config.model.smpl.smpl_path = tmpdir
+ config.model.smpl.joints_regressor = osp.join(
+ tmpdir, 'test_joint_regressor.npy')
+ # generate weight file for SMPL model.
+ generate_smpl_weight_file(tmpdir)
+ pose_model = init_pose_model(config, device='cpu')
+
+ assert pose_model is not None, 'Fail to build pose model'
+
+ image_name = 'tests/data/h36m/S1_Directions_1.54138969_000001.jpg'
+ det_result = {
+ 'keypoints': np.zeros((17, 3)),
+ 'bbox': [50, 50, 50, 50],
+ 'image_name': image_name,
+ }
+
+ # make person bounding boxes
+ person_results = [det_result]
+ dataset = pose_model.cfg.data['test']['type']
+
+ # test a single image, with a list of bboxes
+ pose_results = inference_mesh_model(
+ pose_model,
+ image_name,
+ person_results,
+ bbox_thr=None,
+ format='xywh',
+ dataset=dataset)
+
+ vis_3d_mesh_result(pose_model, pose_results, image_name)
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_apis/test_inference_tracking.py b/engine/pose_estimation/third-party/ViTPose/tests/test_apis/test_inference_tracking.py
new file mode 100644
index 0000000..1ef62b7
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_apis/test_inference_tracking.py
@@ -0,0 +1,157 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from mmpose.apis import (get_track_id, inference_bottom_up_pose_model,
+ inference_top_down_pose_model, init_pose_model,
+ vis_pose_tracking_result)
+from mmpose.datasets.dataset_info import DatasetInfo
+
+
+def test_top_down_pose_tracking_demo():
+ # COCO demo
+ # build the pose model from a config file and a checkpoint file
+ pose_model = init_pose_model(
+ 'configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/'
+ 'coco/res50_coco_256x192.py',
+ None,
+ device='cpu')
+ image_name = 'tests/data/coco/000000000785.jpg'
+ dataset_info = DatasetInfo(pose_model.cfg.data['test']['dataset_info'])
+ person_result = [{'bbox': [50, 50, 50, 100]}]
+
+ # test a single image, with a list of bboxes.
+ pose_results, _ = inference_top_down_pose_model(
+ pose_model,
+ image_name,
+ person_result,
+ format='xywh',
+ dataset_info=dataset_info)
+ pose_results, next_id = get_track_id(pose_results, [], next_id=0)
+ # show the results
+ vis_pose_tracking_result(
+ pose_model, image_name, pose_results, dataset_info=dataset_info)
+ pose_results_last = pose_results
+
+ # AIC demo
+ pose_model = init_pose_model(
+ 'configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/'
+ 'aic/res50_aic_256x192.py',
+ None,
+ device='cpu')
+ image_name = 'tests/data/aic/054d9ce9201beffc76e5ff2169d2af2f027002ca.jpg'
+ dataset_info = DatasetInfo(pose_model.cfg.data['test']['dataset_info'])
+ # test a single image, with a list of bboxes.
+ pose_results, _ = inference_top_down_pose_model(
+ pose_model,
+ image_name,
+ person_result,
+ format='xywh',
+ dataset_info=dataset_info)
+ pose_results, next_id = get_track_id(pose_results, pose_results_last,
+ next_id)
+ for pose_result in pose_results:
+ del pose_result['bbox']
+ pose_results, next_id = get_track_id(pose_results, pose_results_last,
+ next_id)
+
+ # show the results
+ vis_pose_tracking_result(
+ pose_model, image_name, pose_results, dataset_info=dataset_info)
+
+ # OneHand10K demo
+ # build the pose model from a config file and a checkpoint file
+ pose_model = init_pose_model(
+ 'configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/'
+ 'onehand10k/res50_onehand10k_256x256.py',
+ None,
+ device='cpu')
+ image_name = 'tests/data/onehand10k/9.jpg'
+ dataset_info = DatasetInfo(pose_model.cfg.data['test']['dataset_info'])
+ # test a single image, with a list of bboxes.
+ pose_results, _ = inference_top_down_pose_model(
+ pose_model,
+ image_name, [{
+ 'bbox': [10, 10, 30, 30]
+ }],
+ format='xywh',
+ dataset_info=dataset_info)
+ pose_results, next_id = get_track_id(pose_results, pose_results_last,
+ next_id)
+ # show the results
+ vis_pose_tracking_result(
+ pose_model, image_name, pose_results, dataset_info=dataset_info)
+
+ # InterHand2D demo
+ pose_model = init_pose_model(
+ 'configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/'
+ 'interhand2d/res50_interhand2d_all_256x256.py',
+ None,
+ device='cpu')
+ image_name = 'tests/data/interhand2.6m/image2017.jpg'
+ dataset_info = DatasetInfo(pose_model.cfg.data['test']['dataset_info'])
+ # test a single image, with a list of bboxes.
+ pose_results, _ = inference_top_down_pose_model(
+ pose_model,
+ image_name, [{
+ 'bbox': [50, 50, 0, 0]
+ }],
+ format='xywh',
+ dataset_info=dataset_info)
+ pose_results, next_id = get_track_id(pose_results, [], next_id=0)
+ # show the results
+ vis_pose_tracking_result(
+ pose_model, image_name, pose_results, dataset_info=dataset_info)
+ pose_results_last = pose_results
+
+ # MPII demo
+ pose_model = init_pose_model(
+ 'configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/'
+ 'mpii/res50_mpii_256x256.py',
+ None,
+ device='cpu')
+ image_name = 'tests/data/mpii/004645041.jpg'
+ dataset_info = DatasetInfo(pose_model.cfg.data['test']['dataset_info'])
+ # test a single image, with a list of bboxes.
+ pose_results, _ = inference_top_down_pose_model(
+ pose_model,
+ image_name, [{
+ 'bbox': [50, 50, 0, 0]
+ }],
+ format='xywh',
+ dataset_info=dataset_info)
+ pose_results, next_id = get_track_id(pose_results, pose_results_last,
+ next_id)
+ # show the results
+ vis_pose_tracking_result(
+ pose_model, image_name, pose_results, dataset_info=dataset_info)
+
+
+def test_bottom_up_pose_tracking_demo():
+ # COCO demo
+ # build the pose model from a config file and a checkpoint file
+ pose_model = init_pose_model(
+ 'configs/body/2d_kpt_sview_rgb_img/associative_embedding/'
+ 'coco/res50_coco_512x512.py',
+ None,
+ device='cpu')
+
+ image_name = 'tests/data/coco/000000000785.jpg'
+ dataset_info = DatasetInfo(pose_model.cfg.data['test']['dataset_info'])
+
+ pose_results, _ = inference_bottom_up_pose_model(
+ pose_model, image_name, dataset_info=dataset_info)
+
+ pose_results, next_id = get_track_id(pose_results, [], next_id=0)
+
+ # show the results
+ vis_pose_tracking_result(
+ pose_model, image_name, pose_results, dataset_info=dataset_info)
+
+ pose_results_last = pose_results
+
+ # oks
+ pose_results, next_id = get_track_id(
+ pose_results, pose_results_last, next_id=next_id, use_oks=True)
+
+ pose_results_last = pose_results
+ # one_euro
+ pose_results, next_id = get_track_id(
+ pose_results, pose_results_last, next_id=next_id, use_one_euro=True)
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_alexnet.py b/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_alexnet.py
new file mode 100644
index 0000000..a01f3e8
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_alexnet.py
@@ -0,0 +1,21 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import torch
+
+from mmpose.models.backbones import AlexNet
+
+
+def test_alexnet_backbone():
+ """Test alexnet backbone."""
+ model = AlexNet(-1)
+ model.train()
+
+ imgs = torch.randn(1, 3, 256, 192)
+ feat = model(imgs)
+ assert feat.shape == (1, 256, 7, 5)
+
+ model = AlexNet(1)
+ model.train()
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert feat.shape == (1, 1)
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_backbones_utils.py b/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_backbones_utils.py
new file mode 100644
index 0000000..9b2769e
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_backbones_utils.py
@@ -0,0 +1,117 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import pytest
+import torch
+from torch.nn.modules import GroupNorm
+from torch.nn.modules.batchnorm import _BatchNorm
+
+from mmpose.models.backbones.utils import (InvertedResidual, SELayer,
+ channel_shuffle, make_divisible)
+
+
+def is_norm(modules):
+ """Check if is one of the norms."""
+ if isinstance(modules, (GroupNorm, _BatchNorm)):
+ return True
+ return False
+
+
+def test_make_divisible():
+ # test min_value is None
+ result = make_divisible(34, 8, None)
+ assert result == 32
+
+ # test when new_value > min_ratio * value
+ result = make_divisible(10, 8, min_ratio=0.9)
+ assert result == 16
+
+ # test min_value = 0.8
+ result = make_divisible(33, 8, min_ratio=0.8)
+ assert result == 32
+
+
+def test_channel_shuffle():
+ x = torch.randn(1, 24, 56, 56)
+ with pytest.raises(AssertionError):
+ # num_channels should be divisible by groups
+ channel_shuffle(x, 7)
+
+ groups = 3
+ batch_size, num_channels, height, width = x.size()
+ channels_per_group = num_channels // groups
+ out = channel_shuffle(x, groups)
+ # test the output value when groups = 3
+ for b in range(batch_size):
+ for c in range(num_channels):
+ c_out = c % channels_per_group * groups + c // channels_per_group
+ for i in range(height):
+ for j in range(width):
+ assert x[b, c, i, j] == out[b, c_out, i, j]
+
+
+def test_inverted_residual():
+
+ with pytest.raises(AssertionError):
+ # stride must be in [1, 2]
+ InvertedResidual(16, 16, 32, stride=3)
+
+ with pytest.raises(AssertionError):
+ # se_cfg must be None or dict
+ InvertedResidual(16, 16, 32, se_cfg=list())
+
+ with pytest.raises(AssertionError):
+ # in_channeld and out_channels must be the same if
+ # with_expand_conv is False
+ InvertedResidual(16, 16, 32, with_expand_conv=False)
+
+ # Test InvertedResidual forward, stride=1
+ block = InvertedResidual(16, 16, 32, stride=1)
+ x = torch.randn(1, 16, 56, 56)
+ x_out = block(x)
+ assert getattr(block, 'se', None) is None
+ assert block.with_res_shortcut
+ assert x_out.shape == torch.Size((1, 16, 56, 56))
+
+ # Test InvertedResidual forward, stride=2
+ block = InvertedResidual(16, 16, 32, stride=2)
+ x = torch.randn(1, 16, 56, 56)
+ x_out = block(x)
+ assert not block.with_res_shortcut
+ assert x_out.shape == torch.Size((1, 16, 28, 28))
+
+ # Test InvertedResidual forward with se layer
+ se_cfg = dict(channels=32)
+ block = InvertedResidual(16, 16, 32, stride=1, se_cfg=se_cfg)
+ x = torch.randn(1, 16, 56, 56)
+ x_out = block(x)
+ assert isinstance(block.se, SELayer)
+ assert x_out.shape == torch.Size((1, 16, 56, 56))
+
+ # Test InvertedResidual forward, with_expand_conv=False
+ block = InvertedResidual(32, 16, 32, with_expand_conv=False)
+ x = torch.randn(1, 32, 56, 56)
+ x_out = block(x)
+ assert getattr(block, 'expand_conv', None) is None
+ assert x_out.shape == torch.Size((1, 16, 56, 56))
+
+ # Test InvertedResidual forward with GroupNorm
+ block = InvertedResidual(
+ 16, 16, 32, norm_cfg=dict(type='GN', num_groups=2))
+ x = torch.randn(1, 16, 56, 56)
+ x_out = block(x)
+ for m in block.modules():
+ if is_norm(m):
+ assert isinstance(m, GroupNorm)
+ assert x_out.shape == torch.Size((1, 16, 56, 56))
+
+ # Test InvertedResidual forward with HSigmoid
+ block = InvertedResidual(16, 16, 32, act_cfg=dict(type='HSigmoid'))
+ x = torch.randn(1, 16, 56, 56)
+ x_out = block(x)
+ assert x_out.shape == torch.Size((1, 16, 56, 56))
+
+ # Test InvertedResidual forward with checkpoint
+ block = InvertedResidual(16, 16, 32, with_cp=True)
+ x = torch.randn(1, 16, 56, 56)
+ x_out = block(x)
+ assert block.with_cp
+ assert x_out.shape == torch.Size((1, 16, 56, 56))
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_cpm.py b/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_cpm.py
new file mode 100644
index 0000000..a8ce354
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_cpm.py
@@ -0,0 +1,64 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import pytest
+import torch
+
+from mmpose.models import CPM
+from mmpose.models.backbones.cpm import CpmBlock
+
+
+def test_cpm_block():
+ with pytest.raises(AssertionError):
+ # len(channels) == len(kernels)
+ CpmBlock(
+ 3, channels=[3, 3, 3], kernels=[
+ 1,
+ ])
+
+ # Test CPM Block
+ model = CpmBlock(3, channels=[3, 3, 3], kernels=[1, 1, 1])
+ model.train()
+
+ imgs = torch.randn(1, 3, 10, 10)
+ feat = model(imgs)
+ assert feat.shape == torch.Size([1, 3, 10, 10])
+
+
+def test_cpm_backbone():
+ with pytest.raises(AssertionError):
+ # CPM's num_stacks should larger than 0
+ CPM(in_channels=3, out_channels=17, num_stages=-1)
+
+ with pytest.raises(AssertionError):
+ # CPM's in_channels should be 3
+ CPM(in_channels=2, out_channels=17)
+
+ # Test CPM
+ model = CPM(in_channels=3, out_channels=17, num_stages=1)
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(1, 3, 256, 192)
+ feat = model(imgs)
+ assert len(feat) == 1
+ assert feat[0].shape == torch.Size([1, 17, 32, 24])
+
+ imgs = torch.randn(1, 3, 384, 288)
+ feat = model(imgs)
+ assert len(feat) == 1
+ assert feat[0].shape == torch.Size([1, 17, 48, 36])
+
+ imgs = torch.randn(1, 3, 368, 368)
+ feat = model(imgs)
+ assert len(feat) == 1
+ assert feat[0].shape == torch.Size([1, 17, 46, 46])
+
+ # Test CPM multi-stages
+ model = CPM(in_channels=3, out_channels=17, num_stages=2)
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(1, 3, 368, 368)
+ feat = model(imgs)
+ assert len(feat) == 2
+ assert feat[0].shape == torch.Size([1, 17, 46, 46])
+ assert feat[1].shape == torch.Size([1, 17, 46, 46])
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_hourglass.py b/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_hourglass.py
new file mode 100644
index 0000000..3a85610
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_hourglass.py
@@ -0,0 +1,77 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import pytest
+import torch
+
+from mmpose.models import HourglassAENet, HourglassNet
+
+
+def test_hourglass_backbone():
+ with pytest.raises(AssertionError):
+ # HourglassNet's num_stacks should larger than 0
+ HourglassNet(num_stacks=0)
+
+ with pytest.raises(AssertionError):
+ # len(stage_channels) should equal len(stage_blocks)
+ HourglassNet(
+ stage_channels=[256, 256, 384, 384, 384],
+ stage_blocks=[2, 2, 2, 2, 2, 4])
+
+ with pytest.raises(AssertionError):
+ # len(stage_channels) should larger than downsample_times
+ HourglassNet(
+ downsample_times=5,
+ stage_channels=[256, 256, 384, 384, 384],
+ stage_blocks=[2, 2, 2, 2, 2])
+
+ # Test HourglassNet-52
+ model = HourglassNet(num_stacks=1)
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(1, 3, 256, 256)
+ feat = model(imgs)
+ assert len(feat) == 1
+ assert feat[0].shape == torch.Size([1, 256, 64, 64])
+
+ # Test HourglassNet-104
+ model = HourglassNet(num_stacks=2)
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(1, 3, 256, 256)
+ feat = model(imgs)
+ assert len(feat) == 2
+ assert feat[0].shape == torch.Size([1, 256, 64, 64])
+ assert feat[1].shape == torch.Size([1, 256, 64, 64])
+
+
+def test_hourglass_ae_backbone():
+ with pytest.raises(AssertionError):
+ # HourglassAENet's num_stacks should larger than 0
+ HourglassAENet(num_stacks=0)
+
+ with pytest.raises(AssertionError):
+ # len(stage_channels) should larger than downsample_times
+ HourglassAENet(
+ downsample_times=5, stage_channels=[256, 256, 384, 384, 384])
+
+ # num_stack=1
+ model = HourglassAENet(num_stacks=1)
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(1, 3, 256, 256)
+ feat = model(imgs)
+ assert len(feat) == 1
+ assert feat[0].shape == torch.Size([1, 34, 64, 64])
+
+ # num_stack=2
+ model = HourglassAENet(num_stacks=2)
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(1, 3, 256, 256)
+ feat = model(imgs)
+ assert len(feat) == 2
+ assert feat[0].shape == torch.Size([1, 34, 64, 64])
+ assert feat[1].shape == torch.Size([1, 34, 64, 64])
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_hrformer.py b/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_hrformer.py
new file mode 100644
index 0000000..9b91754
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_hrformer.py
@@ -0,0 +1,187 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import pytest
+import torch
+
+from mmpose.models.backbones.hrformer import (HRFomerModule, HRFormer,
+ HRFormerBlock)
+
+
+def test_hrformer_module():
+ norm_cfg = dict(type='BN')
+ block = HRFormerBlock
+ # Test multiscale forward
+ num_channles = (32, 64)
+ num_inchannels = [c * block.expansion for c in num_channles]
+ hrmodule = HRFomerModule(
+ num_branches=2,
+ block=block,
+ num_blocks=(2, 2),
+ num_inchannels=num_inchannels,
+ num_channels=num_channles,
+ num_heads=(1, 2),
+ num_window_sizes=(7, 7),
+ num_mlp_ratios=(4, 4),
+ drop_paths=(0., 0.),
+ norm_cfg=norm_cfg)
+
+ feats = [
+ torch.randn(1, num_inchannels[0], 64, 64),
+ torch.randn(1, num_inchannels[1], 32, 32)
+ ]
+ feats = hrmodule(feats)
+
+ assert len(str(hrmodule)) > 0
+ assert len(feats) == 2
+ assert feats[0].shape == torch.Size([1, num_inchannels[0], 64, 64])
+ assert feats[1].shape == torch.Size([1, num_inchannels[1], 32, 32])
+
+ # Test single scale forward
+ num_channles = (32, 64)
+ in_channels = [c * block.expansion for c in num_channles]
+ hrmodule = HRFomerModule(
+ num_branches=2,
+ block=block,
+ num_blocks=(2, 2),
+ num_inchannels=num_inchannels,
+ num_channels=num_channles,
+ num_heads=(1, 2),
+ num_window_sizes=(7, 7),
+ num_mlp_ratios=(4, 4),
+ drop_paths=(0., 0.),
+ norm_cfg=norm_cfg,
+ multiscale_output=False,
+ )
+
+ feats = [
+ torch.randn(1, in_channels[0], 64, 64),
+ torch.randn(1, in_channels[1], 32, 32)
+ ]
+ feats = hrmodule(feats)
+
+ assert len(feats) == 1
+ assert feats[0].shape == torch.Size([1, in_channels[0], 64, 64])
+
+ # Test single branch HRFormer module
+ hrmodule = HRFomerModule(
+ num_branches=1,
+ block=block,
+ num_blocks=(1, ),
+ num_inchannels=[num_inchannels[0]],
+ num_channels=[num_channles[0]],
+ num_heads=(1, ),
+ num_window_sizes=(7, ),
+ num_mlp_ratios=(4, ),
+ drop_paths=(0.1, ),
+ norm_cfg=norm_cfg,
+ )
+
+ feats = [
+ torch.randn(1, in_channels[0], 64, 64),
+ ]
+ feats = hrmodule(feats)
+
+ assert len(feats) == 1
+ assert feats[0].shape == torch.Size([1, in_channels[0], 64, 64])
+
+ # Value tests
+ kwargs = dict(
+ num_branches=2,
+ block=block,
+ num_blocks=(2, 2),
+ num_inchannels=num_inchannels,
+ num_channels=num_channles,
+ num_heads=(1, 2),
+ num_window_sizes=(7, 7),
+ num_mlp_ratios=(4, 4),
+ drop_paths=(0.1, 0.1),
+ norm_cfg=norm_cfg,
+ )
+
+ with pytest.raises(ValueError):
+ # len(num_blocks) should equal num_branches
+ kwargs['num_blocks'] = [2, 2, 2]
+ HRFomerModule(**kwargs)
+ kwargs['num_blocks'] = [2, 2]
+
+ with pytest.raises(ValueError):
+ # len(num_blocks) should equal num_branches
+ kwargs['num_channels'] = [2]
+ HRFomerModule(**kwargs)
+ kwargs['num_channels'] = [2, 2]
+
+ with pytest.raises(ValueError):
+ # len(num_blocks) should equal num_branches
+ kwargs['num_inchannels'] = [2]
+ HRFomerModule(**kwargs)
+ kwargs['num_inchannels'] = [2, 2]
+
+
+def test_hrformer_backbone():
+ norm_cfg = dict(type='BN')
+ # only have 3 stages
+ extra = dict(
+ drop_path_rate=0.2,
+ stage1=dict(
+ num_modules=1,
+ num_branches=1,
+ block='BOTTLENECK',
+ num_blocks=(2, ),
+ num_channels=(64, )),
+ stage2=dict(
+ num_modules=1,
+ num_branches=2,
+ block='HRFORMERBLOCK',
+ window_sizes=(7, 7),
+ num_heads=(1, 2),
+ mlp_ratios=(4, 4),
+ num_blocks=(2, 2),
+ num_channels=(32, 64)),
+ stage3=dict(
+ num_modules=4,
+ num_branches=3,
+ block='HRFORMERBLOCK',
+ window_sizes=(7, 7, 7),
+ num_heads=(1, 2, 4),
+ mlp_ratios=(4, 4, 4),
+ num_blocks=(2, 2, 2),
+ num_channels=(32, 64, 128)),
+ stage4=dict(
+ num_modules=3,
+ num_branches=4,
+ block='HRFORMERBLOCK',
+ window_sizes=(7, 7, 7, 7),
+ num_heads=(1, 2, 4, 8),
+ mlp_ratios=(4, 4, 4, 4),
+ num_blocks=(2, 2, 2, 2),
+ num_channels=(32, 64, 128, 256),
+ multiscale_output=True))
+
+ with pytest.raises(ValueError):
+ # len(num_blocks) should equal num_branches
+ extra['stage4']['num_branches'] = 3
+ HRFormer(extra=extra)
+ extra['stage4']['num_branches'] = 4
+
+ # Test HRFormer-S
+ model = HRFormer(extra=extra, norm_cfg=norm_cfg)
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(1, 3, 64, 64)
+ feats = model(imgs)
+ assert len(feats) == 4
+ assert feats[0].shape == torch.Size([1, 32, 16, 16])
+ assert feats[3].shape == torch.Size([1, 256, 2, 2])
+
+ # Test single scale output and model
+ # without relative position bias
+ extra['stage4']['multiscale_output'] = False
+ extra['with_rpe'] = False
+ model = HRFormer(extra=extra, norm_cfg=norm_cfg)
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(1, 3, 64, 64)
+ feats = model(imgs)
+ assert len(feats) == 1
+ assert feats[0].shape == torch.Size([1, 32, 16, 16])
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_hrnet.py b/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_hrnet.py
new file mode 100644
index 0000000..cb87880
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_hrnet.py
@@ -0,0 +1,129 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import torch
+from torch.nn.modules.batchnorm import _BatchNorm
+
+from mmpose.models.backbones import HRNet
+from mmpose.models.backbones.hrnet import HRModule
+from mmpose.models.backbones.resnet import BasicBlock, Bottleneck
+
+
+def is_block(modules):
+ """Check if is HRModule building block."""
+ if isinstance(modules, (HRModule, )):
+ return True
+ return False
+
+
+def is_norm(modules):
+ """Check if is one of the norms."""
+ if isinstance(modules, (_BatchNorm, )):
+ return True
+ return False
+
+
+def all_zeros(modules):
+ """Check if the weight(and bias) is all zero."""
+ weight_zero = torch.equal(modules.weight.data,
+ torch.zeros_like(modules.weight.data))
+ if hasattr(modules, 'bias'):
+ bias_zero = torch.equal(modules.bias.data,
+ torch.zeros_like(modules.bias.data))
+ else:
+ bias_zero = True
+
+ return weight_zero and bias_zero
+
+
+def test_hrmodule():
+ # Test HRModule forward
+ block = HRModule(
+ num_branches=1,
+ blocks=BasicBlock,
+ num_blocks=(4, ),
+ in_channels=[
+ 64,
+ ],
+ num_channels=(64, ))
+
+ x = torch.randn(2, 64, 56, 56)
+ x_out = block([x])
+ assert x_out[0].shape == torch.Size([2, 64, 56, 56])
+
+
+def test_hrnet_backbone():
+ extra = dict(
+ stage1=dict(
+ num_modules=1,
+ num_branches=1,
+ block='BOTTLENECK',
+ num_blocks=(4, ),
+ num_channels=(64, )),
+ stage2=dict(
+ num_modules=1,
+ num_branches=2,
+ block='BASIC',
+ num_blocks=(4, 4),
+ num_channels=(32, 64)),
+ stage3=dict(
+ num_modules=4,
+ num_branches=3,
+ block='BASIC',
+ num_blocks=(4, 4, 4),
+ num_channels=(32, 64, 128)),
+ stage4=dict(
+ num_modules=3,
+ num_branches=4,
+ block='BASIC',
+ num_blocks=(4, 4, 4, 4),
+ num_channels=(32, 64, 128, 256)))
+
+ model = HRNet(extra, in_channels=3)
+
+ imgs = torch.randn(2, 3, 224, 224)
+ feat = model(imgs)
+ assert len(feat) == 1
+ assert feat[0].shape == torch.Size([2, 32, 56, 56])
+
+ # Test HRNet zero initialization of residual
+ model = HRNet(extra, in_channels=3, zero_init_residual=True)
+ model.init_weights()
+ for m in model.modules():
+ if isinstance(m, Bottleneck):
+ assert all_zeros(m.norm3)
+ model.train()
+
+ imgs = torch.randn(2, 3, 224, 224)
+ feat = model(imgs)
+ assert len(feat) == 1
+ assert feat[0].shape == torch.Size([2, 32, 56, 56])
+
+ # Test HRNet with the first three stages frozen
+ frozen_stages = 3
+ model = HRNet(extra, in_channels=3, frozen_stages=frozen_stages)
+ model.init_weights()
+ model.train()
+ if frozen_stages >= 0:
+ assert model.norm1.training is False
+ assert model.norm2.training is False
+ for layer in [model.conv1, model.norm1, model.conv2, model.norm2]:
+ for param in layer.parameters():
+ assert param.requires_grad is False
+
+ for i in range(1, frozen_stages + 1):
+ if i == 1:
+ layer = getattr(model, 'layer1')
+ else:
+ layer = getattr(model, f'stage{i}')
+ for mod in layer.modules():
+ if isinstance(mod, _BatchNorm):
+ assert mod.training is False
+ for param in layer.parameters():
+ assert param.requires_grad is False
+
+ if i < 4:
+ layer = getattr(model, f'transition{i}')
+ for mod in layer.modules():
+ if isinstance(mod, _BatchNorm):
+ assert mod.training is False
+ for param in layer.parameters():
+ assert param.requires_grad is False
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_litehrnet.py b/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_litehrnet.py
new file mode 100644
index 0000000..de2b6db
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_litehrnet.py
@@ -0,0 +1,143 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import pytest
+import torch
+from torch.nn.modules.batchnorm import _BatchNorm
+
+from mmpose.models.backbones import LiteHRNet
+from mmpose.models.backbones.litehrnet import LiteHRModule
+from mmpose.models.backbones.resnet import Bottleneck
+
+
+def is_norm(modules):
+ """Check if is one of the norms."""
+ if isinstance(modules, (_BatchNorm, )):
+ return True
+ return False
+
+
+def all_zeros(modules):
+ """Check if the weight(and bias) is all zero."""
+ weight_zero = torch.equal(modules.weight.data,
+ torch.zeros_like(modules.weight.data))
+ if hasattr(modules, 'bias'):
+ bias_zero = torch.equal(modules.bias.data,
+ torch.zeros_like(modules.bias.data))
+ else:
+ bias_zero = True
+
+ return weight_zero and bias_zero
+
+
+def test_litehrmodule():
+ # Test LiteHRModule forward
+ block = LiteHRModule(
+ num_branches=1,
+ num_blocks=1,
+ in_channels=[
+ 40,
+ ],
+ reduce_ratio=8,
+ module_type='LITE')
+
+ x = torch.randn(2, 40, 56, 56)
+ x_out = block([[x]])
+ assert x_out[0][0].shape == torch.Size([2, 40, 56, 56])
+
+ block = LiteHRModule(
+ num_branches=1,
+ num_blocks=1,
+ in_channels=[
+ 40,
+ ],
+ reduce_ratio=8,
+ module_type='NAIVE')
+
+ x = torch.randn(2, 40, 56, 56)
+ x_out = block([x])
+ assert x_out[0].shape == torch.Size([2, 40, 56, 56])
+
+ with pytest.raises(ValueError):
+ block = LiteHRModule(
+ num_branches=1,
+ num_blocks=1,
+ in_channels=[
+ 40,
+ ],
+ reduce_ratio=8,
+ module_type='none')
+
+
+def test_litehrnet_backbone():
+ extra = dict(
+ stem=dict(stem_channels=32, out_channels=32, expand_ratio=1),
+ num_stages=3,
+ stages_spec=dict(
+ num_modules=(2, 4, 2),
+ num_branches=(2, 3, 4),
+ num_blocks=(2, 2, 2),
+ module_type=('LITE', 'LITE', 'LITE'),
+ with_fuse=(True, True, True),
+ reduce_ratios=(8, 8, 8),
+ num_channels=(
+ (40, 80),
+ (40, 80, 160),
+ (40, 80, 160, 320),
+ )),
+ with_head=True)
+
+ model = LiteHRNet(extra, in_channels=3)
+
+ imgs = torch.randn(2, 3, 224, 224)
+ feat = model(imgs)
+ assert len(feat) == 1
+ assert feat[0].shape == torch.Size([2, 40, 56, 56])
+
+ # Test HRNet zero initialization of residual
+ model = LiteHRNet(extra, in_channels=3)
+ model.init_weights()
+ for m in model.modules():
+ if isinstance(m, Bottleneck):
+ assert all_zeros(m.norm3)
+ model.train()
+
+ imgs = torch.randn(2, 3, 224, 224)
+ feat = model(imgs)
+ assert len(feat) == 1
+ assert feat[0].shape == torch.Size([2, 40, 56, 56])
+
+ extra = dict(
+ stem=dict(stem_channels=32, out_channels=32, expand_ratio=1),
+ num_stages=3,
+ stages_spec=dict(
+ num_modules=(2, 4, 2),
+ num_branches=(2, 3, 4),
+ num_blocks=(2, 2, 2),
+ module_type=('NAIVE', 'NAIVE', 'NAIVE'),
+ with_fuse=(True, True, True),
+ reduce_ratios=(8, 8, 8),
+ num_channels=(
+ (40, 80),
+ (40, 80, 160),
+ (40, 80, 160, 320),
+ )),
+ with_head=True)
+
+ model = LiteHRNet(extra, in_channels=3)
+
+ imgs = torch.randn(2, 3, 224, 224)
+ feat = model(imgs)
+ assert len(feat) == 1
+ assert feat[0].shape == torch.Size([2, 40, 56, 56])
+
+ # Test HRNet zero initialization of residual
+ model = LiteHRNet(extra, in_channels=3)
+ model.init_weights()
+ for m in model.modules():
+ if isinstance(m, Bottleneck):
+ assert all_zeros(m.norm3)
+ model.train()
+
+ imgs = torch.randn(2, 3, 224, 224)
+ feat = model(imgs)
+ assert len(feat) == 1
+ assert feat[0].shape == torch.Size([2, 40, 56, 56])
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_mobilenet_v2.py b/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_mobilenet_v2.py
new file mode 100644
index 0000000..1381ec2
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_mobilenet_v2.py
@@ -0,0 +1,257 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import pytest
+import torch
+from torch.nn.modules import GroupNorm
+from torch.nn.modules.batchnorm import _BatchNorm
+
+from mmpose.models.backbones import MobileNetV2
+from mmpose.models.backbones.mobilenet_v2 import InvertedResidual
+
+
+def is_block(modules):
+ """Check if is ResNet building block."""
+ if isinstance(modules, (InvertedResidual, )):
+ return True
+ return False
+
+
+def is_norm(modules):
+ """Check if is one of the norms."""
+ if isinstance(modules, (GroupNorm, _BatchNorm)):
+ return True
+ return False
+
+
+def check_norm_state(modules, train_state):
+ """Check if norm layer is in correct train state."""
+ for mod in modules:
+ if isinstance(mod, _BatchNorm):
+ if mod.training != train_state:
+ return False
+ return True
+
+
+def test_mobilenetv2_invertedresidual():
+
+ with pytest.raises(AssertionError):
+ # stride must be in [1, 2]
+ InvertedResidual(16, 24, stride=3, expand_ratio=6)
+
+ # Test InvertedResidual with checkpoint forward, stride=1
+ block = InvertedResidual(16, 24, stride=1, expand_ratio=6)
+ x = torch.randn(1, 16, 56, 56)
+ x_out = block(x)
+ assert x_out.shape == torch.Size((1, 24, 56, 56))
+
+ # Test InvertedResidual with expand_ratio=1
+ block = InvertedResidual(16, 16, stride=1, expand_ratio=1)
+ assert len(block.conv) == 2
+
+ # Test InvertedResidual with use_res_connect
+ block = InvertedResidual(16, 16, stride=1, expand_ratio=6)
+ x = torch.randn(1, 16, 56, 56)
+ x_out = block(x)
+ assert block.use_res_connect is True
+ assert x_out.shape == torch.Size((1, 16, 56, 56))
+
+ # Test InvertedResidual with checkpoint forward, stride=2
+ block = InvertedResidual(16, 24, stride=2, expand_ratio=6)
+ x = torch.randn(1, 16, 56, 56)
+ x_out = block(x)
+ assert x_out.shape == torch.Size((1, 24, 28, 28))
+
+ # Test InvertedResidual with checkpoint forward
+ block = InvertedResidual(16, 24, stride=1, expand_ratio=6, with_cp=True)
+ assert block.with_cp
+ x = torch.randn(1, 16, 56, 56)
+ x_out = block(x)
+ assert x_out.shape == torch.Size((1, 24, 56, 56))
+
+ # Test InvertedResidual with act_cfg=dict(type='ReLU')
+ block = InvertedResidual(
+ 16, 24, stride=1, expand_ratio=6, act_cfg=dict(type='ReLU'))
+ x = torch.randn(1, 16, 56, 56)
+ x_out = block(x)
+ assert x_out.shape == torch.Size((1, 24, 56, 56))
+
+
+def test_mobilenetv2_backbone():
+ with pytest.raises(TypeError):
+ # pretrained must be a string path
+ model = MobileNetV2()
+ model.init_weights(pretrained=0)
+
+ with pytest.raises(ValueError):
+ # frozen_stages must in range(1, 8)
+ MobileNetV2(frozen_stages=8)
+
+ with pytest.raises(ValueError):
+ # tout_indices in range(-1, 8)
+ MobileNetV2(out_indices=[8])
+
+ # Test MobileNetV2 with first stage frozen
+ frozen_stages = 1
+ model = MobileNetV2(frozen_stages=frozen_stages)
+ model.init_weights()
+ model.train()
+
+ for mod in model.conv1.modules():
+ for param in mod.parameters():
+ assert param.requires_grad is False
+ for i in range(1, frozen_stages + 1):
+ layer = getattr(model, f'layer{i}')
+ for mod in layer.modules():
+ if isinstance(mod, _BatchNorm):
+ assert mod.training is False
+ for param in layer.parameters():
+ assert param.requires_grad is False
+
+ # Test MobileNetV2 with norm_eval=True
+ model = MobileNetV2(norm_eval=True)
+ model.init_weights()
+ model.train()
+
+ assert check_norm_state(model.modules(), False)
+
+ # Test MobileNetV2 forward with widen_factor=1.0
+ model = MobileNetV2(widen_factor=1.0, out_indices=range(0, 8))
+ model.init_weights()
+ model.train()
+
+ assert check_norm_state(model.modules(), True)
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert len(feat) == 8
+ assert feat[0].shape == torch.Size((1, 16, 112, 112))
+ assert feat[1].shape == torch.Size((1, 24, 56, 56))
+ assert feat[2].shape == torch.Size((1, 32, 28, 28))
+ assert feat[3].shape == torch.Size((1, 64, 14, 14))
+ assert feat[4].shape == torch.Size((1, 96, 14, 14))
+ assert feat[5].shape == torch.Size((1, 160, 7, 7))
+ assert feat[6].shape == torch.Size((1, 320, 7, 7))
+ assert feat[7].shape == torch.Size((1, 1280, 7, 7))
+
+ # Test MobileNetV2 forward with widen_factor=0.5
+ model = MobileNetV2(widen_factor=0.5, out_indices=range(0, 7))
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert len(feat) == 7
+ assert feat[0].shape == torch.Size((1, 8, 112, 112))
+ assert feat[1].shape == torch.Size((1, 16, 56, 56))
+ assert feat[2].shape == torch.Size((1, 16, 28, 28))
+ assert feat[3].shape == torch.Size((1, 32, 14, 14))
+ assert feat[4].shape == torch.Size((1, 48, 14, 14))
+ assert feat[5].shape == torch.Size((1, 80, 7, 7))
+ assert feat[6].shape == torch.Size((1, 160, 7, 7))
+
+ # Test MobileNetV2 forward with widen_factor=2.0
+ model = MobileNetV2(widen_factor=2.0)
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert feat.shape == torch.Size((1, 2560, 7, 7))
+
+ # Test MobileNetV2 forward with out_indices=None
+ model = MobileNetV2(widen_factor=1.0)
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert feat.shape == torch.Size((1, 1280, 7, 7))
+
+ # Test MobileNetV2 forward with dict(type='ReLU')
+ model = MobileNetV2(
+ widen_factor=1.0, act_cfg=dict(type='ReLU'), out_indices=range(0, 7))
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert len(feat) == 7
+ assert feat[0].shape == torch.Size((1, 16, 112, 112))
+ assert feat[1].shape == torch.Size((1, 24, 56, 56))
+ assert feat[2].shape == torch.Size((1, 32, 28, 28))
+ assert feat[3].shape == torch.Size((1, 64, 14, 14))
+ assert feat[4].shape == torch.Size((1, 96, 14, 14))
+ assert feat[5].shape == torch.Size((1, 160, 7, 7))
+ assert feat[6].shape == torch.Size((1, 320, 7, 7))
+
+ # Test MobileNetV2 with GroupNorm forward
+ model = MobileNetV2(widen_factor=1.0, out_indices=range(0, 7))
+ for m in model.modules():
+ if is_norm(m):
+ assert isinstance(m, _BatchNorm)
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert len(feat) == 7
+ assert feat[0].shape == torch.Size((1, 16, 112, 112))
+ assert feat[1].shape == torch.Size((1, 24, 56, 56))
+ assert feat[2].shape == torch.Size((1, 32, 28, 28))
+ assert feat[3].shape == torch.Size((1, 64, 14, 14))
+ assert feat[4].shape == torch.Size((1, 96, 14, 14))
+ assert feat[5].shape == torch.Size((1, 160, 7, 7))
+ assert feat[6].shape == torch.Size((1, 320, 7, 7))
+
+ # Test MobileNetV2 with BatchNorm forward
+ model = MobileNetV2(
+ widen_factor=1.0,
+ norm_cfg=dict(type='GN', num_groups=2, requires_grad=True),
+ out_indices=range(0, 7))
+ for m in model.modules():
+ if is_norm(m):
+ assert isinstance(m, GroupNorm)
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert len(feat) == 7
+ assert feat[0].shape == torch.Size((1, 16, 112, 112))
+ assert feat[1].shape == torch.Size((1, 24, 56, 56))
+ assert feat[2].shape == torch.Size((1, 32, 28, 28))
+ assert feat[3].shape == torch.Size((1, 64, 14, 14))
+ assert feat[4].shape == torch.Size((1, 96, 14, 14))
+ assert feat[5].shape == torch.Size((1, 160, 7, 7))
+ assert feat[6].shape == torch.Size((1, 320, 7, 7))
+
+ # Test MobileNetV2 with layers 1, 3, 5 out forward
+ model = MobileNetV2(widen_factor=1.0, out_indices=(0, 2, 4))
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert len(feat) == 3
+ assert feat[0].shape == torch.Size((1, 16, 112, 112))
+ assert feat[1].shape == torch.Size((1, 32, 28, 28))
+ assert feat[2].shape == torch.Size((1, 96, 14, 14))
+
+ # Test MobileNetV2 with checkpoint forward
+ model = MobileNetV2(
+ widen_factor=1.0, with_cp=True, out_indices=range(0, 7))
+ for m in model.modules():
+ if is_block(m):
+ assert m.with_cp
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert len(feat) == 7
+ assert feat[0].shape == torch.Size((1, 16, 112, 112))
+ assert feat[1].shape == torch.Size((1, 24, 56, 56))
+ assert feat[2].shape == torch.Size((1, 32, 28, 28))
+ assert feat[3].shape == torch.Size((1, 64, 14, 14))
+ assert feat[4].shape == torch.Size((1, 96, 14, 14))
+ assert feat[5].shape == torch.Size((1, 160, 7, 7))
+ assert feat[6].shape == torch.Size((1, 320, 7, 7))
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_mobilenet_v3.py b/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_mobilenet_v3.py
new file mode 100644
index 0000000..1cc00ea
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_mobilenet_v3.py
@@ -0,0 +1,169 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import pytest
+import torch
+from torch.nn.modules import GroupNorm
+from torch.nn.modules.batchnorm import _BatchNorm
+
+from mmpose.models.backbones import MobileNetV3
+from mmpose.models.backbones.utils import InvertedResidual
+
+
+def is_norm(modules):
+ """Check if is one of the norms."""
+ if isinstance(modules, (GroupNorm, _BatchNorm)):
+ return True
+ return False
+
+
+def check_norm_state(modules, train_state):
+ """Check if norm layer is in correct train state."""
+ for mod in modules:
+ if isinstance(mod, _BatchNorm):
+ if mod.training != train_state:
+ return False
+ return True
+
+
+def test_mobilenetv3_backbone():
+ with pytest.raises(TypeError):
+ # pretrained must be a string path
+ model = MobileNetV3()
+ model.init_weights(pretrained=0)
+
+ with pytest.raises(AssertionError):
+ # arch must in [small, big]
+ MobileNetV3(arch='others')
+
+ with pytest.raises(ValueError):
+ # frozen_stages must less than 12 when arch is small
+ MobileNetV3(arch='small', frozen_stages=12)
+
+ with pytest.raises(ValueError):
+ # frozen_stages must less than 16 when arch is big
+ MobileNetV3(arch='big', frozen_stages=16)
+
+ with pytest.raises(ValueError):
+ # max out_indices must less than 11 when arch is small
+ MobileNetV3(arch='small', out_indices=(11, ))
+
+ with pytest.raises(ValueError):
+ # max out_indices must less than 15 when arch is big
+ MobileNetV3(arch='big', out_indices=(15, ))
+
+ # Test MobileNetv3
+ model = MobileNetV3()
+ model.init_weights()
+ model.train()
+
+ # Test MobileNetv3 with first stage frozen
+ frozen_stages = 1
+ model = MobileNetV3(frozen_stages=frozen_stages)
+ model.init_weights()
+ model.train()
+ for param in model.conv1.parameters():
+ assert param.requires_grad is False
+ for i in range(1, frozen_stages + 1):
+ layer = getattr(model, f'layer{i}')
+ for mod in layer.modules():
+ if isinstance(mod, _BatchNorm):
+ assert mod.training is False
+ for param in layer.parameters():
+ assert param.requires_grad is False
+
+ # Test MobileNetv3 with norm eval
+ model = MobileNetV3(norm_eval=True, out_indices=range(0, 11))
+ model.init_weights()
+ model.train()
+ assert check_norm_state(model.modules(), False)
+
+ # Test MobileNetv3 forward with small arch
+ model = MobileNetV3(out_indices=(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10))
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert len(feat) == 11
+ assert feat[0].shape == torch.Size([1, 16, 56, 56])
+ assert feat[1].shape == torch.Size([1, 24, 28, 28])
+ assert feat[2].shape == torch.Size([1, 24, 28, 28])
+ assert feat[3].shape == torch.Size([1, 40, 14, 14])
+ assert feat[4].shape == torch.Size([1, 40, 14, 14])
+ assert feat[5].shape == torch.Size([1, 40, 14, 14])
+ assert feat[6].shape == torch.Size([1, 48, 14, 14])
+ assert feat[7].shape == torch.Size([1, 48, 14, 14])
+ assert feat[8].shape == torch.Size([1, 96, 7, 7])
+ assert feat[9].shape == torch.Size([1, 96, 7, 7])
+ assert feat[10].shape == torch.Size([1, 96, 7, 7])
+
+ # Test MobileNetv3 forward with small arch and GroupNorm
+ model = MobileNetV3(
+ out_indices=(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10),
+ norm_cfg=dict(type='GN', num_groups=2, requires_grad=True))
+ for m in model.modules():
+ if is_norm(m):
+ assert isinstance(m, GroupNorm)
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert len(feat) == 11
+ assert feat[0].shape == torch.Size([1, 16, 56, 56])
+ assert feat[1].shape == torch.Size([1, 24, 28, 28])
+ assert feat[2].shape == torch.Size([1, 24, 28, 28])
+ assert feat[3].shape == torch.Size([1, 40, 14, 14])
+ assert feat[4].shape == torch.Size([1, 40, 14, 14])
+ assert feat[5].shape == torch.Size([1, 40, 14, 14])
+ assert feat[6].shape == torch.Size([1, 48, 14, 14])
+ assert feat[7].shape == torch.Size([1, 48, 14, 14])
+ assert feat[8].shape == torch.Size([1, 96, 7, 7])
+ assert feat[9].shape == torch.Size([1, 96, 7, 7])
+ assert feat[10].shape == torch.Size([1, 96, 7, 7])
+
+ # Test MobileNetv3 forward with big arch
+ model = MobileNetV3(
+ arch='big',
+ out_indices=(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14))
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert len(feat) == 15
+ assert feat[0].shape == torch.Size([1, 16, 112, 112])
+ assert feat[1].shape == torch.Size([1, 24, 56, 56])
+ assert feat[2].shape == torch.Size([1, 24, 56, 56])
+ assert feat[3].shape == torch.Size([1, 40, 28, 28])
+ assert feat[4].shape == torch.Size([1, 40, 28, 28])
+ assert feat[5].shape == torch.Size([1, 40, 28, 28])
+ assert feat[6].shape == torch.Size([1, 80, 14, 14])
+ assert feat[7].shape == torch.Size([1, 80, 14, 14])
+ assert feat[8].shape == torch.Size([1, 80, 14, 14])
+ assert feat[9].shape == torch.Size([1, 80, 14, 14])
+ assert feat[10].shape == torch.Size([1, 112, 14, 14])
+ assert feat[11].shape == torch.Size([1, 112, 14, 14])
+ assert feat[12].shape == torch.Size([1, 160, 14, 14])
+ assert feat[13].shape == torch.Size([1, 160, 7, 7])
+ assert feat[14].shape == torch.Size([1, 160, 7, 7])
+
+ # Test MobileNetv3 forward with big arch
+ model = MobileNetV3(arch='big', out_indices=(0, ))
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert feat.shape == torch.Size([1, 16, 112, 112])
+
+ # Test MobileNetv3 with checkpoint forward
+ model = MobileNetV3(with_cp=True)
+ for m in model.modules():
+ if isinstance(m, InvertedResidual):
+ assert m.with_cp
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert feat.shape == torch.Size([1, 96, 7, 7])
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_mspn.py b/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_mspn.py
new file mode 100644
index 0000000..6aca441
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_mspn.py
@@ -0,0 +1,32 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import pytest
+import torch
+
+from mmpose.models import MSPN
+
+
+def test_mspn_backbone():
+ with pytest.raises(AssertionError):
+ # MSPN's num_stages should larger than 0
+ MSPN(num_stages=0)
+ with pytest.raises(AssertionError):
+ # MSPN's num_units should larger than 1
+ MSPN(num_units=1)
+ with pytest.raises(AssertionError):
+ # len(num_blocks) should equal num_units
+ MSPN(num_units=2, num_blocks=[2, 2, 2])
+
+ # Test MSPN's outputs
+ model = MSPN(num_stages=2, num_units=2, num_blocks=[2, 2])
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(1, 3, 511, 511)
+ feat = model(imgs)
+ assert len(feat) == 2
+ assert len(feat[0]) == 2
+ assert len(feat[1]) == 2
+ assert feat[0][0].shape == torch.Size([1, 256, 64, 64])
+ assert feat[0][1].shape == torch.Size([1, 256, 128, 128])
+ assert feat[1][0].shape == torch.Size([1, 256, 64, 64])
+ assert feat[1][1].shape == torch.Size([1, 256, 128, 128])
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_regnet.py b/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_regnet.py
new file mode 100644
index 0000000..165aad7
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_regnet.py
@@ -0,0 +1,92 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import pytest
+import torch
+
+from mmpose.models.backbones import RegNet
+
+regnet_test_data = [
+ ('regnetx_400mf',
+ dict(w0=24, wa=24.48, wm=2.54, group_w=16, depth=22,
+ bot_mul=1.0), [32, 64, 160, 384]),
+ ('regnetx_800mf',
+ dict(w0=56, wa=35.73, wm=2.28, group_w=16, depth=16,
+ bot_mul=1.0), [64, 128, 288, 672]),
+ ('regnetx_1.6gf',
+ dict(w0=80, wa=34.01, wm=2.25, group_w=24, depth=18,
+ bot_mul=1.0), [72, 168, 408, 912]),
+ ('regnetx_3.2gf',
+ dict(w0=88, wa=26.31, wm=2.25, group_w=48, depth=25,
+ bot_mul=1.0), [96, 192, 432, 1008]),
+ ('regnetx_4.0gf',
+ dict(w0=96, wa=38.65, wm=2.43, group_w=40, depth=23,
+ bot_mul=1.0), [80, 240, 560, 1360]),
+ ('regnetx_6.4gf',
+ dict(w0=184, wa=60.83, wm=2.07, group_w=56, depth=17,
+ bot_mul=1.0), [168, 392, 784, 1624]),
+ ('regnetx_8.0gf',
+ dict(w0=80, wa=49.56, wm=2.88, group_w=120, depth=23,
+ bot_mul=1.0), [80, 240, 720, 1920]),
+ ('regnetx_12gf',
+ dict(w0=168, wa=73.36, wm=2.37, group_w=112, depth=19,
+ bot_mul=1.0), [224, 448, 896, 2240]),
+]
+
+
+@pytest.mark.parametrize('arch_name,arch,out_channels', regnet_test_data)
+def test_regnet_backbone(arch_name, arch, out_channels):
+ with pytest.raises(AssertionError):
+ # ResNeXt depth should be in [50, 101, 152]
+ RegNet(arch_name + '233')
+
+ # output the last feature map
+ model = RegNet(arch_name)
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert isinstance(feat, torch.Tensor)
+ assert feat.shape == (1, out_channels[-1], 7, 7)
+
+ # output feature map of all stages
+ model = RegNet(arch_name, out_indices=(0, 1, 2, 3))
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert len(feat) == 4
+ assert feat[0].shape == (1, out_channels[0], 56, 56)
+ assert feat[1].shape == (1, out_channels[1], 28, 28)
+ assert feat[2].shape == (1, out_channels[2], 14, 14)
+ assert feat[3].shape == (1, out_channels[3], 7, 7)
+
+
+@pytest.mark.parametrize('arch_name,arch,out_channels', regnet_test_data)
+def test_custom_arch(arch_name, arch, out_channels):
+ # output the last feature map
+ model = RegNet(arch)
+ model.init_weights()
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert isinstance(feat, torch.Tensor)
+ assert feat.shape == (1, out_channels[-1], 7, 7)
+
+ # output feature map of all stages
+ model = RegNet(arch, out_indices=(0, 1, 2, 3))
+ model.init_weights()
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert len(feat) == 4
+ assert feat[0].shape == (1, out_channels[0], 56, 56)
+ assert feat[1].shape == (1, out_channels[1], 28, 28)
+ assert feat[2].shape == (1, out_channels[2], 14, 14)
+ assert feat[3].shape == (1, out_channels[3], 7, 7)
+
+
+def test_exception():
+ # arch must be a str or dict
+ with pytest.raises(TypeError):
+ _ = RegNet(50)
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_resnest.py b/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_resnest.py
new file mode 100644
index 0000000..3bb41b1
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_resnest.py
@@ -0,0 +1,44 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import pytest
+import torch
+
+from mmpose.models.backbones import ResNeSt
+from mmpose.models.backbones.resnest import Bottleneck as BottleneckS
+
+
+def test_bottleneck():
+ with pytest.raises(AssertionError):
+ # Style must be in ['pytorch', 'caffe']
+ BottleneckS(64, 64, radix=2, reduction_factor=4, style='tensorflow')
+
+ # Test ResNeSt Bottleneck structure
+ block = BottleneckS(
+ 64, 256, radix=2, reduction_factor=4, stride=2, style='pytorch')
+ assert block.avd_layer.stride == 2
+ assert block.conv2.channels == 64
+
+ # Test ResNeSt Bottleneck forward
+ block = BottleneckS(64, 64, radix=2, reduction_factor=4)
+ x = torch.randn(2, 64, 56, 56)
+ x_out = block(x)
+ assert x_out.shape == torch.Size([2, 64, 56, 56])
+
+
+def test_resnest():
+ with pytest.raises(KeyError):
+ # ResNeSt depth should be in [50, 101, 152, 200]
+ ResNeSt(depth=18)
+
+ # Test ResNeSt with radix 2, reduction_factor 4
+ model = ResNeSt(
+ depth=50, radix=2, reduction_factor=4, out_indices=(0, 1, 2, 3))
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(2, 3, 224, 224)
+ feat = model(imgs)
+ assert len(feat) == 4
+ assert feat[0].shape == torch.Size([2, 256, 56, 56])
+ assert feat[1].shape == torch.Size([2, 512, 28, 28])
+ assert feat[2].shape == torch.Size([2, 1024, 14, 14])
+ assert feat[3].shape == torch.Size([2, 2048, 7, 7])
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_resnet.py b/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_resnet.py
new file mode 100644
index 0000000..036a76c
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_resnet.py
@@ -0,0 +1,562 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import pytest
+import torch
+import torch.nn as nn
+from mmcv.cnn import ConvModule
+from mmcv.utils.parrots_wrapper import _BatchNorm
+
+from mmpose.models.backbones import ResNet, ResNetV1d
+from mmpose.models.backbones.resnet import (BasicBlock, Bottleneck, ResLayer,
+ get_expansion)
+
+
+def is_block(modules):
+ """Check if is ResNet building block."""
+ if isinstance(modules, (BasicBlock, Bottleneck)):
+ return True
+ return False
+
+
+def all_zeros(modules):
+ """Check if the weight(and bias) is all zero."""
+ weight_zero = torch.equal(modules.weight.data,
+ torch.zeros_like(modules.weight.data))
+ if hasattr(modules, 'bias'):
+ bias_zero = torch.equal(modules.bias.data,
+ torch.zeros_like(modules.bias.data))
+ else:
+ bias_zero = True
+
+ return weight_zero and bias_zero
+
+
+def check_norm_state(modules, train_state):
+ """Check if norm layer is in correct train state."""
+ for mod in modules:
+ if isinstance(mod, _BatchNorm):
+ if mod.training != train_state:
+ return False
+ return True
+
+
+def test_get_expansion():
+ assert get_expansion(Bottleneck, 2) == 2
+ assert get_expansion(BasicBlock) == 1
+ assert get_expansion(Bottleneck) == 4
+
+ class MyResBlock(nn.Module):
+
+ expansion = 8
+
+ assert get_expansion(MyResBlock) == 8
+
+ # expansion must be an integer or None
+ with pytest.raises(TypeError):
+ get_expansion(Bottleneck, '0')
+
+ # expansion is not specified and cannot be inferred
+ with pytest.raises(TypeError):
+
+ class SomeModule(nn.Module):
+ pass
+
+ get_expansion(SomeModule)
+
+
+def test_basic_block():
+ # expansion must be 1
+ with pytest.raises(AssertionError):
+ BasicBlock(64, 64, expansion=2)
+
+ # BasicBlock with stride 1, out_channels == in_channels
+ block = BasicBlock(64, 64)
+ assert block.in_channels == 64
+ assert block.mid_channels == 64
+ assert block.out_channels == 64
+ assert block.conv1.in_channels == 64
+ assert block.conv1.out_channels == 64
+ assert block.conv1.kernel_size == (3, 3)
+ assert block.conv1.stride == (1, 1)
+ assert block.conv2.in_channels == 64
+ assert block.conv2.out_channels == 64
+ assert block.conv2.kernel_size == (3, 3)
+ x = torch.randn(1, 64, 56, 56)
+ x_out = block(x)
+ assert x_out.shape == torch.Size([1, 64, 56, 56])
+
+ # BasicBlock with stride 1 and downsample
+ downsample = nn.Sequential(
+ nn.Conv2d(64, 128, kernel_size=1, bias=False), nn.BatchNorm2d(128))
+ block = BasicBlock(64, 128, downsample=downsample)
+ assert block.in_channels == 64
+ assert block.mid_channels == 128
+ assert block.out_channels == 128
+ assert block.conv1.in_channels == 64
+ assert block.conv1.out_channels == 128
+ assert block.conv1.kernel_size == (3, 3)
+ assert block.conv1.stride == (1, 1)
+ assert block.conv2.in_channels == 128
+ assert block.conv2.out_channels == 128
+ assert block.conv2.kernel_size == (3, 3)
+ x = torch.randn(1, 64, 56, 56)
+ x_out = block(x)
+ assert x_out.shape == torch.Size([1, 128, 56, 56])
+
+ # BasicBlock with stride 2 and downsample
+ downsample = nn.Sequential(
+ nn.Conv2d(64, 128, kernel_size=1, stride=2, bias=False),
+ nn.BatchNorm2d(128))
+ block = BasicBlock(64, 128, stride=2, downsample=downsample)
+ assert block.in_channels == 64
+ assert block.mid_channels == 128
+ assert block.out_channels == 128
+ assert block.conv1.in_channels == 64
+ assert block.conv1.out_channels == 128
+ assert block.conv1.kernel_size == (3, 3)
+ assert block.conv1.stride == (2, 2)
+ assert block.conv2.in_channels == 128
+ assert block.conv2.out_channels == 128
+ assert block.conv2.kernel_size == (3, 3)
+ x = torch.randn(1, 64, 56, 56)
+ x_out = block(x)
+ assert x_out.shape == torch.Size([1, 128, 28, 28])
+
+ # forward with checkpointing
+ block = BasicBlock(64, 64, with_cp=True)
+ assert block.with_cp
+ x = torch.randn(1, 64, 56, 56, requires_grad=True)
+ x_out = block(x)
+ assert x_out.shape == torch.Size([1, 64, 56, 56])
+
+
+def test_bottleneck():
+ # style must be in ['pytorch', 'caffe']
+ with pytest.raises(AssertionError):
+ Bottleneck(64, 64, style='tensorflow')
+
+ # expansion must be divisible by out_channels
+ with pytest.raises(AssertionError):
+ Bottleneck(64, 64, expansion=3)
+
+ # Test Bottleneck style
+ block = Bottleneck(64, 64, stride=2, style='pytorch')
+ assert block.conv1.stride == (1, 1)
+ assert block.conv2.stride == (2, 2)
+ block = Bottleneck(64, 64, stride=2, style='caffe')
+ assert block.conv1.stride == (2, 2)
+ assert block.conv2.stride == (1, 1)
+
+ # Bottleneck with stride 1
+ block = Bottleneck(64, 64, style='pytorch')
+ assert block.in_channels == 64
+ assert block.mid_channels == 16
+ assert block.out_channels == 64
+ assert block.conv1.in_channels == 64
+ assert block.conv1.out_channels == 16
+ assert block.conv1.kernel_size == (1, 1)
+ assert block.conv2.in_channels == 16
+ assert block.conv2.out_channels == 16
+ assert block.conv2.kernel_size == (3, 3)
+ assert block.conv3.in_channels == 16
+ assert block.conv3.out_channels == 64
+ assert block.conv3.kernel_size == (1, 1)
+ x = torch.randn(1, 64, 56, 56)
+ x_out = block(x)
+ assert x_out.shape == (1, 64, 56, 56)
+
+ # Bottleneck with stride 1 and downsample
+ downsample = nn.Sequential(
+ nn.Conv2d(64, 128, kernel_size=1), nn.BatchNorm2d(128))
+ block = Bottleneck(64, 128, style='pytorch', downsample=downsample)
+ assert block.in_channels == 64
+ assert block.mid_channels == 32
+ assert block.out_channels == 128
+ assert block.conv1.in_channels == 64
+ assert block.conv1.out_channels == 32
+ assert block.conv1.kernel_size == (1, 1)
+ assert block.conv2.in_channels == 32
+ assert block.conv2.out_channels == 32
+ assert block.conv2.kernel_size == (3, 3)
+ assert block.conv3.in_channels == 32
+ assert block.conv3.out_channels == 128
+ assert block.conv3.kernel_size == (1, 1)
+ x = torch.randn(1, 64, 56, 56)
+ x_out = block(x)
+ assert x_out.shape == (1, 128, 56, 56)
+
+ # Bottleneck with stride 2 and downsample
+ downsample = nn.Sequential(
+ nn.Conv2d(64, 128, kernel_size=1, stride=2), nn.BatchNorm2d(128))
+ block = Bottleneck(
+ 64, 128, stride=2, style='pytorch', downsample=downsample)
+ x = torch.randn(1, 64, 56, 56)
+ x_out = block(x)
+ assert x_out.shape == (1, 128, 28, 28)
+
+ # Bottleneck with expansion 2
+ block = Bottleneck(64, 64, style='pytorch', expansion=2)
+ assert block.in_channels == 64
+ assert block.mid_channels == 32
+ assert block.out_channels == 64
+ assert block.conv1.in_channels == 64
+ assert block.conv1.out_channels == 32
+ assert block.conv1.kernel_size == (1, 1)
+ assert block.conv2.in_channels == 32
+ assert block.conv2.out_channels == 32
+ assert block.conv2.kernel_size == (3, 3)
+ assert block.conv3.in_channels == 32
+ assert block.conv3.out_channels == 64
+ assert block.conv3.kernel_size == (1, 1)
+ x = torch.randn(1, 64, 56, 56)
+ x_out = block(x)
+ assert x_out.shape == (1, 64, 56, 56)
+
+ # Test Bottleneck with checkpointing
+ block = Bottleneck(64, 64, with_cp=True)
+ block.train()
+ assert block.with_cp
+ x = torch.randn(1, 64, 56, 56, requires_grad=True)
+ x_out = block(x)
+ assert x_out.shape == torch.Size([1, 64, 56, 56])
+
+
+def test_basicblock_reslayer():
+ # 3 BasicBlock w/o downsample
+ layer = ResLayer(BasicBlock, 3, 32, 32)
+ assert len(layer) == 3
+ for i in range(3):
+ assert layer[i].in_channels == 32
+ assert layer[i].out_channels == 32
+ assert layer[i].downsample is None
+ x = torch.randn(1, 32, 56, 56)
+ x_out = layer(x)
+ assert x_out.shape == (1, 32, 56, 56)
+
+ # 3 BasicBlock w/ stride 1 and downsample
+ layer = ResLayer(BasicBlock, 3, 32, 64)
+ assert len(layer) == 3
+ assert layer[0].in_channels == 32
+ assert layer[0].out_channels == 64
+ assert layer[0].downsample is not None and len(layer[0].downsample) == 2
+ assert isinstance(layer[0].downsample[0], nn.Conv2d)
+ assert layer[0].downsample[0].stride == (1, 1)
+ for i in range(1, 3):
+ assert layer[i].in_channels == 64
+ assert layer[i].out_channels == 64
+ assert layer[i].downsample is None
+ x = torch.randn(1, 32, 56, 56)
+ x_out = layer(x)
+ assert x_out.shape == (1, 64, 56, 56)
+
+ # 3 BasicBlock w/ stride 2 and downsample
+ layer = ResLayer(BasicBlock, 3, 32, 64, stride=2)
+ assert len(layer) == 3
+ assert layer[0].in_channels == 32
+ assert layer[0].out_channels == 64
+ assert layer[0].stride == 2
+ assert layer[0].downsample is not None and len(layer[0].downsample) == 2
+ assert isinstance(layer[0].downsample[0], nn.Conv2d)
+ assert layer[0].downsample[0].stride == (2, 2)
+ for i in range(1, 3):
+ assert layer[i].in_channels == 64
+ assert layer[i].out_channels == 64
+ assert layer[i].stride == 1
+ assert layer[i].downsample is None
+ x = torch.randn(1, 32, 56, 56)
+ x_out = layer(x)
+ assert x_out.shape == (1, 64, 28, 28)
+
+ # 3 BasicBlock w/ stride 2 and downsample with avg pool
+ layer = ResLayer(BasicBlock, 3, 32, 64, stride=2, avg_down=True)
+ assert len(layer) == 3
+ assert layer[0].in_channels == 32
+ assert layer[0].out_channels == 64
+ assert layer[0].stride == 2
+ assert layer[0].downsample is not None and len(layer[0].downsample) == 3
+ assert isinstance(layer[0].downsample[0], nn.AvgPool2d)
+ assert layer[0].downsample[0].stride == 2
+ for i in range(1, 3):
+ assert layer[i].in_channels == 64
+ assert layer[i].out_channels == 64
+ assert layer[i].stride == 1
+ assert layer[i].downsample is None
+ x = torch.randn(1, 32, 56, 56)
+ x_out = layer(x)
+ assert x_out.shape == (1, 64, 28, 28)
+
+
+def test_bottleneck_reslayer():
+ # 3 Bottleneck w/o downsample
+ layer = ResLayer(Bottleneck, 3, 32, 32)
+ assert len(layer) == 3
+ for i in range(3):
+ assert layer[i].in_channels == 32
+ assert layer[i].out_channels == 32
+ assert layer[i].downsample is None
+ x = torch.randn(1, 32, 56, 56)
+ x_out = layer(x)
+ assert x_out.shape == (1, 32, 56, 56)
+
+ # 3 Bottleneck w/ stride 1 and downsample
+ layer = ResLayer(Bottleneck, 3, 32, 64)
+ assert len(layer) == 3
+ assert layer[0].in_channels == 32
+ assert layer[0].out_channels == 64
+ assert layer[0].stride == 1
+ assert layer[0].conv1.out_channels == 16
+ assert layer[0].downsample is not None and len(layer[0].downsample) == 2
+ assert isinstance(layer[0].downsample[0], nn.Conv2d)
+ assert layer[0].downsample[0].stride == (1, 1)
+ for i in range(1, 3):
+ assert layer[i].in_channels == 64
+ assert layer[i].out_channels == 64
+ assert layer[i].conv1.out_channels == 16
+ assert layer[i].stride == 1
+ assert layer[i].downsample is None
+ x = torch.randn(1, 32, 56, 56)
+ x_out = layer(x)
+ assert x_out.shape == (1, 64, 56, 56)
+
+ # 3 Bottleneck w/ stride 2 and downsample
+ layer = ResLayer(Bottleneck, 3, 32, 64, stride=2)
+ assert len(layer) == 3
+ assert layer[0].in_channels == 32
+ assert layer[0].out_channels == 64
+ assert layer[0].stride == 2
+ assert layer[0].conv1.out_channels == 16
+ assert layer[0].downsample is not None and len(layer[0].downsample) == 2
+ assert isinstance(layer[0].downsample[0], nn.Conv2d)
+ assert layer[0].downsample[0].stride == (2, 2)
+ for i in range(1, 3):
+ assert layer[i].in_channels == 64
+ assert layer[i].out_channels == 64
+ assert layer[i].conv1.out_channels == 16
+ assert layer[i].stride == 1
+ assert layer[i].downsample is None
+ x = torch.randn(1, 32, 56, 56)
+ x_out = layer(x)
+ assert x_out.shape == (1, 64, 28, 28)
+
+ # 3 Bottleneck w/ stride 2 and downsample with avg pool
+ layer = ResLayer(Bottleneck, 3, 32, 64, stride=2, avg_down=True)
+ assert len(layer) == 3
+ assert layer[0].in_channels == 32
+ assert layer[0].out_channels == 64
+ assert layer[0].stride == 2
+ assert layer[0].conv1.out_channels == 16
+ assert layer[0].downsample is not None and len(layer[0].downsample) == 3
+ assert isinstance(layer[0].downsample[0], nn.AvgPool2d)
+ assert layer[0].downsample[0].stride == 2
+ for i in range(1, 3):
+ assert layer[i].in_channels == 64
+ assert layer[i].out_channels == 64
+ assert layer[i].conv1.out_channels == 16
+ assert layer[i].stride == 1
+ assert layer[i].downsample is None
+ x = torch.randn(1, 32, 56, 56)
+ x_out = layer(x)
+ assert x_out.shape == (1, 64, 28, 28)
+
+ # 3 Bottleneck with custom expansion
+ layer = ResLayer(Bottleneck, 3, 32, 32, expansion=2)
+ assert len(layer) == 3
+ for i in range(3):
+ assert layer[i].in_channels == 32
+ assert layer[i].out_channels == 32
+ assert layer[i].stride == 1
+ assert layer[i].conv1.out_channels == 16
+ assert layer[i].downsample is None
+ x = torch.randn(1, 32, 56, 56)
+ x_out = layer(x)
+ assert x_out.shape == (1, 32, 56, 56)
+
+
+def test_resnet():
+ """Test resnet backbone."""
+ with pytest.raises(KeyError):
+ # ResNet depth should be in [18, 34, 50, 101, 152]
+ ResNet(20)
+
+ with pytest.raises(AssertionError):
+ # In ResNet: 1 <= num_stages <= 4
+ ResNet(50, num_stages=0)
+
+ with pytest.raises(AssertionError):
+ # In ResNet: 1 <= num_stages <= 4
+ ResNet(50, num_stages=5)
+
+ with pytest.raises(AssertionError):
+ # len(strides) == len(dilations) == num_stages
+ ResNet(50, strides=(1, ), dilations=(1, 1), num_stages=3)
+
+ with pytest.raises(TypeError):
+ # pretrained must be a string path
+ model = ResNet(50)
+ model.init_weights(pretrained=0)
+
+ with pytest.raises(AssertionError):
+ # Style must be in ['pytorch', 'caffe']
+ ResNet(50, style='tensorflow')
+
+ # Test ResNet50 norm_eval=True
+ model = ResNet(50, norm_eval=True)
+ model.init_weights()
+ model.train()
+ assert check_norm_state(model.modules(), False)
+
+ # Test ResNet50 with torchvision pretrained weight
+ model = ResNet(depth=50, norm_eval=True)
+ model.init_weights('torchvision://resnet50')
+ model.train()
+ assert check_norm_state(model.modules(), False)
+
+ # Test ResNet50 with first stage frozen
+ frozen_stages = 1
+ model = ResNet(50, frozen_stages=frozen_stages)
+ model.init_weights()
+ model.train()
+ assert model.norm1.training is False
+ for layer in [model.conv1, model.norm1]:
+ for param in layer.parameters():
+ assert param.requires_grad is False
+ for i in range(1, frozen_stages + 1):
+ layer = getattr(model, f'layer{i}')
+ for mod in layer.modules():
+ if isinstance(mod, _BatchNorm):
+ assert mod.training is False
+ for param in layer.parameters():
+ assert param.requires_grad is False
+
+ # Test ResNet18 forward
+ model = ResNet(18, out_indices=(0, 1, 2, 3))
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert len(feat) == 4
+ assert feat[0].shape == (1, 64, 56, 56)
+ assert feat[1].shape == (1, 128, 28, 28)
+ assert feat[2].shape == (1, 256, 14, 14)
+ assert feat[3].shape == (1, 512, 7, 7)
+
+ # Test ResNet50 with BatchNorm forward
+ model = ResNet(50, out_indices=(0, 1, 2, 3))
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert len(feat) == 4
+ assert feat[0].shape == (1, 256, 56, 56)
+ assert feat[1].shape == (1, 512, 28, 28)
+ assert feat[2].shape == (1, 1024, 14, 14)
+ assert feat[3].shape == (1, 2048, 7, 7)
+
+ # Test ResNet50 with layers 1, 2, 3 out forward
+ model = ResNet(50, out_indices=(0, 1, 2))
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert len(feat) == 3
+ assert feat[0].shape == (1, 256, 56, 56)
+ assert feat[1].shape == (1, 512, 28, 28)
+ assert feat[2].shape == (1, 1024, 14, 14)
+
+ # Test ResNet50 with layers 3 (top feature maps) out forward
+ model = ResNet(50, out_indices=(3, ))
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert feat.shape == (1, 2048, 7, 7)
+
+ # Test ResNet50 with checkpoint forward
+ model = ResNet(50, out_indices=(0, 1, 2, 3), with_cp=True)
+ for m in model.modules():
+ if is_block(m):
+ assert m.with_cp
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert len(feat) == 4
+ assert feat[0].shape == (1, 256, 56, 56)
+ assert feat[1].shape == (1, 512, 28, 28)
+ assert feat[2].shape == (1, 1024, 14, 14)
+ assert feat[3].shape == (1, 2048, 7, 7)
+
+ # zero initialization of residual blocks
+ model = ResNet(50, out_indices=(0, 1, 2, 3), zero_init_residual=True)
+ model.init_weights()
+ for m in model.modules():
+ if isinstance(m, Bottleneck):
+ assert all_zeros(m.norm3)
+ elif isinstance(m, BasicBlock):
+ assert all_zeros(m.norm2)
+
+ # non-zero initialization of residual blocks
+ model = ResNet(50, out_indices=(0, 1, 2, 3), zero_init_residual=False)
+ model.init_weights()
+ for m in model.modules():
+ if isinstance(m, Bottleneck):
+ assert not all_zeros(m.norm3)
+ elif isinstance(m, BasicBlock):
+ assert not all_zeros(m.norm2)
+
+
+def test_resnet_v1d():
+ model = ResNetV1d(depth=50, out_indices=(0, 1, 2, 3))
+ model.init_weights()
+ model.train()
+
+ assert len(model.stem) == 3
+ for i in range(3):
+ assert isinstance(model.stem[i], ConvModule)
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model.stem(imgs)
+ assert feat.shape == (1, 64, 112, 112)
+ feat = model(imgs)
+ assert len(feat) == 4
+ assert feat[0].shape == (1, 256, 56, 56)
+ assert feat[1].shape == (1, 512, 28, 28)
+ assert feat[2].shape == (1, 1024, 14, 14)
+ assert feat[3].shape == (1, 2048, 7, 7)
+
+ # Test ResNet50V1d with first stage frozen
+ frozen_stages = 1
+ model = ResNetV1d(depth=50, frozen_stages=frozen_stages)
+ assert len(model.stem) == 3
+ for i in range(3):
+ assert isinstance(model.stem[i], ConvModule)
+ model.init_weights()
+ model.train()
+ check_norm_state(model.stem, False)
+ for param in model.stem.parameters():
+ assert param.requires_grad is False
+ for i in range(1, frozen_stages + 1):
+ layer = getattr(model, f'layer{i}')
+ for mod in layer.modules():
+ if isinstance(mod, _BatchNorm):
+ assert mod.training is False
+ for param in layer.parameters():
+ assert param.requires_grad is False
+
+
+def test_resnet_half_channel():
+ model = ResNet(50, base_channels=32, out_indices=(0, 1, 2, 3))
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert len(feat) == 4
+ assert feat[0].shape == (1, 128, 56, 56)
+ assert feat[1].shape == (1, 256, 28, 28)
+ assert feat[2].shape == (1, 512, 14, 14)
+ assert feat[3].shape == (1, 1024, 7, 7)
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_resnext.py b/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_resnext.py
new file mode 100644
index 0000000..88191e1
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_resnext.py
@@ -0,0 +1,60 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import pytest
+import torch
+
+from mmpose.models.backbones import ResNeXt
+from mmpose.models.backbones.resnext import Bottleneck as BottleneckX
+
+
+def test_bottleneck():
+ with pytest.raises(AssertionError):
+ # Style must be in ['pytorch', 'caffe']
+ BottleneckX(64, 64, groups=32, width_per_group=4, style='tensorflow')
+
+ # Test ResNeXt Bottleneck structure
+ block = BottleneckX(
+ 64, 256, groups=32, width_per_group=4, stride=2, style='pytorch')
+ assert block.conv2.stride == (2, 2)
+ assert block.conv2.groups == 32
+ assert block.conv2.out_channels == 128
+
+ # Test ResNeXt Bottleneck forward
+ block = BottleneckX(64, 64, base_channels=16, groups=32, width_per_group=4)
+ x = torch.randn(1, 64, 56, 56)
+ x_out = block(x)
+ assert x_out.shape == torch.Size([1, 64, 56, 56])
+
+
+def test_resnext():
+ with pytest.raises(KeyError):
+ # ResNeXt depth should be in [50, 101, 152]
+ ResNeXt(depth=18)
+
+ # Test ResNeXt with group 32, width_per_group 4
+ model = ResNeXt(
+ depth=50, groups=32, width_per_group=4, out_indices=(0, 1, 2, 3))
+ for m in model.modules():
+ if isinstance(m, BottleneckX):
+ assert m.conv2.groups == 32
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert len(feat) == 4
+ assert feat[0].shape == torch.Size([1, 256, 56, 56])
+ assert feat[1].shape == torch.Size([1, 512, 28, 28])
+ assert feat[2].shape == torch.Size([1, 1024, 14, 14])
+ assert feat[3].shape == torch.Size([1, 2048, 7, 7])
+
+ # Test ResNeXt with group 32, width_per_group 4 and layers 3 out forward
+ model = ResNeXt(depth=50, groups=32, width_per_group=4, out_indices=(3, ))
+ for m in model.modules():
+ if isinstance(m, BottleneckX):
+ assert m.conv2.groups == 32
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert feat.shape == torch.Size([1, 2048, 7, 7])
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_rsn.py b/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_rsn.py
new file mode 100644
index 0000000..617dd9e
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_rsn.py
@@ -0,0 +1,35 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import pytest
+import torch
+
+from mmpose.models import RSN
+
+
+def test_rsn_backbone():
+ with pytest.raises(AssertionError):
+ # RSN's num_stages should larger than 0
+ RSN(num_stages=0)
+ with pytest.raises(AssertionError):
+ # RSN's num_steps should larger than 1
+ RSN(num_steps=1)
+ with pytest.raises(AssertionError):
+ # RSN's num_units should larger than 1
+ RSN(num_units=1)
+ with pytest.raises(AssertionError):
+ # len(num_blocks) should equal num_units
+ RSN(num_units=2, num_blocks=[2, 2, 2])
+
+ # Test RSN's outputs
+ model = RSN(num_stages=2, num_units=2, num_blocks=[2, 2])
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(1, 3, 511, 511)
+ feat = model(imgs)
+ assert len(feat) == 2
+ assert len(feat[0]) == 2
+ assert len(feat[1]) == 2
+ assert feat[0][0].shape == torch.Size([1, 256, 64, 64])
+ assert feat[0][1].shape == torch.Size([1, 256, 128, 128])
+ assert feat[1][0].shape == torch.Size([1, 256, 64, 64])
+ assert feat[1][1].shape == torch.Size([1, 256, 128, 128])
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_scnet.py b/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_scnet.py
new file mode 100644
index 0000000..e03a87b
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_scnet.py
@@ -0,0 +1,163 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import pytest
+import torch
+from torch.nn.modules.batchnorm import _BatchNorm
+
+from mmpose.models.backbones import SCNet
+from mmpose.models.backbones.scnet import SCBottleneck, SCConv
+
+
+def is_block(modules):
+ """Check if is SCNet building block."""
+ if isinstance(modules, (SCBottleneck, )):
+ return True
+ return False
+
+
+def is_norm(modules):
+ """Check if is one of the norms."""
+ if isinstance(modules, (_BatchNorm, )):
+ return True
+ return False
+
+
+def all_zeros(modules):
+ """Check if the weight(and bias) is all zero."""
+ weight_zero = torch.equal(modules.weight.data,
+ torch.zeros_like(modules.weight.data))
+ if hasattr(modules, 'bias'):
+ bias_zero = torch.equal(modules.bias.data,
+ torch.zeros_like(modules.bias.data))
+ else:
+ bias_zero = True
+
+ return weight_zero and bias_zero
+
+
+def check_norm_state(modules, train_state):
+ """Check if norm layer is in correct train state."""
+ for mod in modules:
+ if isinstance(mod, _BatchNorm):
+ if mod.training != train_state:
+ return False
+ return True
+
+
+def test_scnet_scconv():
+ # Test scconv forward
+ layer = SCConv(64, 64, 1, 4)
+ x = torch.randn(1, 64, 56, 56)
+ x_out = layer(x)
+ assert x_out.shape == torch.Size([1, 64, 56, 56])
+
+
+def test_scnet_bottleneck():
+ # Test Bottleneck forward
+ block = SCBottleneck(64, 64)
+ x = torch.randn(1, 64, 56, 56)
+ x_out = block(x)
+ assert x_out.shape == torch.Size([1, 64, 56, 56])
+
+
+def test_scnet_backbone():
+ """Test scnet backbone."""
+ with pytest.raises(KeyError):
+ # SCNet depth should be in [50, 101]
+ SCNet(20)
+
+ with pytest.raises(TypeError):
+ # pretrained must be a string path
+ model = SCNet(50)
+ model.init_weights(pretrained=0)
+
+ # Test SCNet norm_eval=True
+ model = SCNet(50, norm_eval=True)
+ model.init_weights()
+ model.train()
+ assert check_norm_state(model.modules(), False)
+
+ # Test SCNet50 with first stage frozen
+ frozen_stages = 1
+ model = SCNet(50, frozen_stages=frozen_stages)
+ model.init_weights()
+ model.train()
+ assert model.norm1.training is False
+ for layer in [model.conv1, model.norm1]:
+ for param in layer.parameters():
+ assert param.requires_grad is False
+ for i in range(1, frozen_stages + 1):
+ layer = getattr(model, f'layer{i}')
+ for mod in layer.modules():
+ if isinstance(mod, _BatchNorm):
+ assert mod.training is False
+ for param in layer.parameters():
+ assert param.requires_grad is False
+
+ # Test SCNet with BatchNorm forward
+ model = SCNet(50, out_indices=(0, 1, 2, 3))
+ for m in model.modules():
+ if is_norm(m):
+ assert isinstance(m, _BatchNorm)
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(2, 3, 224, 224)
+ feat = model(imgs)
+ assert len(feat) == 4
+ assert feat[0].shape == torch.Size([2, 256, 56, 56])
+ assert feat[1].shape == torch.Size([2, 512, 28, 28])
+ assert feat[2].shape == torch.Size([2, 1024, 14, 14])
+ assert feat[3].shape == torch.Size([2, 2048, 7, 7])
+
+ # Test SCNet with layers 1, 2, 3 out forward
+ model = SCNet(50, out_indices=(0, 1, 2))
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(2, 3, 224, 224)
+ feat = model(imgs)
+ assert len(feat) == 3
+ assert feat[0].shape == torch.Size([2, 256, 56, 56])
+ assert feat[1].shape == torch.Size([2, 512, 28, 28])
+ assert feat[2].shape == torch.Size([2, 1024, 14, 14])
+
+ # Test SEResNet50 with layers 3 (top feature maps) out forward
+ model = SCNet(50, out_indices=(3, ))
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(2, 3, 224, 224)
+ feat = model(imgs)
+ assert feat.shape == torch.Size([2, 2048, 7, 7])
+
+ # Test SEResNet50 with checkpoint forward
+ model = SCNet(50, out_indices=(0, 1, 2, 3), with_cp=True)
+ for m in model.modules():
+ if is_block(m):
+ assert m.with_cp
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(2, 3, 224, 224)
+ feat = model(imgs)
+ assert len(feat) == 4
+ assert feat[0].shape == torch.Size([2, 256, 56, 56])
+ assert feat[1].shape == torch.Size([2, 512, 28, 28])
+ assert feat[2].shape == torch.Size([2, 1024, 14, 14])
+ assert feat[3].shape == torch.Size([2, 2048, 7, 7])
+
+ # Test SCNet zero initialization of residual
+ model = SCNet(50, out_indices=(0, 1, 2, 3), zero_init_residual=True)
+ model.init_weights()
+ for m in model.modules():
+ if isinstance(m, SCBottleneck):
+ assert all_zeros(m.norm3)
+ model.train()
+
+ imgs = torch.randn(2, 3, 224, 224)
+ feat = model(imgs)
+ assert len(feat) == 4
+ assert feat[0].shape == torch.Size([2, 256, 56, 56])
+ assert feat[1].shape == torch.Size([2, 512, 28, 28])
+ assert feat[2].shape == torch.Size([2, 1024, 14, 14])
+ assert feat[3].shape == torch.Size([2, 2048, 7, 7])
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_seresnet.py b/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_seresnet.py
new file mode 100644
index 0000000..4484c66
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_seresnet.py
@@ -0,0 +1,243 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import pytest
+import torch
+from torch.nn.modules import AvgPool2d
+from torch.nn.modules.batchnorm import _BatchNorm
+
+from mmpose.models.backbones import SEResNet
+from mmpose.models.backbones.resnet import ResLayer
+from mmpose.models.backbones.seresnet import SEBottleneck, SELayer
+
+
+def all_zeros(modules):
+ """Check if the weight(and bias) is all zero."""
+ weight_zero = torch.equal(modules.weight.data,
+ torch.zeros_like(modules.weight.data))
+ if hasattr(modules, 'bias'):
+ bias_zero = torch.equal(modules.bias.data,
+ torch.zeros_like(modules.bias.data))
+ else:
+ bias_zero = True
+
+ return weight_zero and bias_zero
+
+
+def check_norm_state(modules, train_state):
+ """Check if norm layer is in correct train state."""
+ for mod in modules:
+ if isinstance(mod, _BatchNorm):
+ if mod.training != train_state:
+ return False
+ return True
+
+
+def test_selayer():
+ # Test selayer forward
+ layer = SELayer(64)
+ x = torch.randn(1, 64, 56, 56)
+ x_out = layer(x)
+ assert x_out.shape == torch.Size([1, 64, 56, 56])
+
+ # Test selayer forward with different ratio
+ layer = SELayer(64, ratio=8)
+ x = torch.randn(1, 64, 56, 56)
+ x_out = layer(x)
+ assert x_out.shape == torch.Size([1, 64, 56, 56])
+
+
+def test_bottleneck():
+
+ with pytest.raises(AssertionError):
+ # Style must be in ['pytorch', 'caffe']
+ SEBottleneck(64, 64, style='tensorflow')
+
+ # Test SEBottleneck with checkpoint forward
+ block = SEBottleneck(64, 64, with_cp=True)
+ assert block.with_cp
+ x = torch.randn(1, 64, 56, 56)
+ x_out = block(x)
+ assert x_out.shape == torch.Size([1, 64, 56, 56])
+
+ # Test Bottleneck style
+ block = SEBottleneck(64, 256, stride=2, style='pytorch')
+ assert block.conv1.stride == (1, 1)
+ assert block.conv2.stride == (2, 2)
+ block = SEBottleneck(64, 256, stride=2, style='caffe')
+ assert block.conv1.stride == (2, 2)
+ assert block.conv2.stride == (1, 1)
+
+ # Test Bottleneck forward
+ block = SEBottleneck(64, 64)
+ x = torch.randn(1, 64, 56, 56)
+ x_out = block(x)
+ assert x_out.shape == torch.Size([1, 64, 56, 56])
+
+
+def test_res_layer():
+ # Test ResLayer of 3 Bottleneck w\o downsample
+ layer = ResLayer(SEBottleneck, 3, 64, 64, se_ratio=16)
+ assert len(layer) == 3
+ assert layer[0].conv1.in_channels == 64
+ assert layer[0].conv1.out_channels == 16
+ for i in range(1, len(layer)):
+ assert layer[i].conv1.in_channels == 64
+ assert layer[i].conv1.out_channels == 16
+ for i in range(len(layer)):
+ assert layer[i].downsample is None
+ x = torch.randn(1, 64, 56, 56)
+ x_out = layer(x)
+ assert x_out.shape == torch.Size([1, 64, 56, 56])
+
+ # Test ResLayer of 3 SEBottleneck with downsample
+ layer = ResLayer(SEBottleneck, 3, 64, 256, se_ratio=16)
+ assert layer[0].downsample[0].out_channels == 256
+ for i in range(1, len(layer)):
+ assert layer[i].downsample is None
+ x = torch.randn(1, 64, 56, 56)
+ x_out = layer(x)
+ assert x_out.shape == torch.Size([1, 256, 56, 56])
+
+ # Test ResLayer of 3 SEBottleneck with stride=2
+ layer = ResLayer(SEBottleneck, 3, 64, 256, stride=2, se_ratio=8)
+ assert layer[0].downsample[0].out_channels == 256
+ assert layer[0].downsample[0].stride == (2, 2)
+ for i in range(1, len(layer)):
+ assert layer[i].downsample is None
+ x = torch.randn(1, 64, 56, 56)
+ x_out = layer(x)
+ assert x_out.shape == torch.Size([1, 256, 28, 28])
+
+ # Test ResLayer of 3 SEBottleneck with stride=2 and average downsample
+ layer = ResLayer(
+ SEBottleneck, 3, 64, 256, stride=2, avg_down=True, se_ratio=8)
+ assert isinstance(layer[0].downsample[0], AvgPool2d)
+ assert layer[0].downsample[1].out_channels == 256
+ assert layer[0].downsample[1].stride == (1, 1)
+ for i in range(1, len(layer)):
+ assert layer[i].downsample is None
+ x = torch.randn(1, 64, 56, 56)
+ x_out = layer(x)
+ assert x_out.shape == torch.Size([1, 256, 28, 28])
+
+
+def test_seresnet():
+ """Test resnet backbone."""
+ with pytest.raises(KeyError):
+ # SEResNet depth should be in [50, 101, 152]
+ SEResNet(20)
+
+ with pytest.raises(AssertionError):
+ # In SEResNet: 1 <= num_stages <= 4
+ SEResNet(50, num_stages=0)
+
+ with pytest.raises(AssertionError):
+ # In SEResNet: 1 <= num_stages <= 4
+ SEResNet(50, num_stages=5)
+
+ with pytest.raises(AssertionError):
+ # len(strides) == len(dilations) == num_stages
+ SEResNet(50, strides=(1, ), dilations=(1, 1), num_stages=3)
+
+ with pytest.raises(TypeError):
+ # pretrained must be a string path
+ model = SEResNet(50)
+ model.init_weights(pretrained=0)
+
+ with pytest.raises(AssertionError):
+ # Style must be in ['pytorch', 'caffe']
+ SEResNet(50, style='tensorflow')
+
+ # Test SEResNet50 norm_eval=True
+ model = SEResNet(50, norm_eval=True)
+ model.init_weights()
+ model.train()
+ assert check_norm_state(model.modules(), False)
+
+ # Test SEResNet50 with torchvision pretrained weight
+ model = SEResNet(depth=50, norm_eval=True)
+ model.init_weights('torchvision://resnet50')
+ model.train()
+ assert check_norm_state(model.modules(), False)
+
+ # Test SEResNet50 with first stage frozen
+ frozen_stages = 1
+ model = SEResNet(50, frozen_stages=frozen_stages)
+ model.init_weights()
+ model.train()
+ assert model.norm1.training is False
+ for layer in [model.conv1, model.norm1]:
+ for param in layer.parameters():
+ assert param.requires_grad is False
+ for i in range(1, frozen_stages + 1):
+ layer = getattr(model, f'layer{i}')
+ for mod in layer.modules():
+ if isinstance(mod, _BatchNorm):
+ assert mod.training is False
+ for param in layer.parameters():
+ assert param.requires_grad is False
+
+ # Test SEResNet50 with BatchNorm forward
+ model = SEResNet(50, out_indices=(0, 1, 2, 3))
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert len(feat) == 4
+ assert feat[0].shape == torch.Size([1, 256, 56, 56])
+ assert feat[1].shape == torch.Size([1, 512, 28, 28])
+ assert feat[2].shape == torch.Size([1, 1024, 14, 14])
+ assert feat[3].shape == torch.Size([1, 2048, 7, 7])
+
+ # Test SEResNet50 with layers 1, 2, 3 out forward
+ model = SEResNet(50, out_indices=(0, 1, 2))
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert len(feat) == 3
+ assert feat[0].shape == torch.Size([1, 256, 56, 56])
+ assert feat[1].shape == torch.Size([1, 512, 28, 28])
+ assert feat[2].shape == torch.Size([1, 1024, 14, 14])
+
+ # Test SEResNet50 with layers 3 (top feature maps) out forward
+ model = SEResNet(50, out_indices=(3, ))
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert feat.shape == torch.Size([1, 2048, 7, 7])
+
+ # Test SEResNet50 with checkpoint forward
+ model = SEResNet(50, out_indices=(0, 1, 2, 3), with_cp=True)
+ for m in model.modules():
+ if isinstance(m, SEBottleneck):
+ assert m.with_cp
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert len(feat) == 4
+ assert feat[0].shape == torch.Size([1, 256, 56, 56])
+ assert feat[1].shape == torch.Size([1, 512, 28, 28])
+ assert feat[2].shape == torch.Size([1, 1024, 14, 14])
+ assert feat[3].shape == torch.Size([1, 2048, 7, 7])
+
+ # Test SEResNet50 zero initialization of residual
+ model = SEResNet(50, out_indices=(0, 1, 2, 3), zero_init_residual=True)
+ model.init_weights()
+ for m in model.modules():
+ if isinstance(m, SEBottleneck):
+ assert all_zeros(m.norm3)
+ model.train()
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert len(feat) == 4
+ assert feat[0].shape == torch.Size([1, 256, 56, 56])
+ assert feat[1].shape == torch.Size([1, 512, 28, 28])
+ assert feat[2].shape == torch.Size([1, 1024, 14, 14])
+ assert feat[3].shape == torch.Size([1, 2048, 7, 7])
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_seresnext.py b/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_seresnext.py
new file mode 100644
index 0000000..2c15605
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_seresnext.py
@@ -0,0 +1,73 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import pytest
+import torch
+
+from mmpose.models.backbones import SEResNeXt
+from mmpose.models.backbones.seresnext import SEBottleneck as SEBottleneckX
+
+
+def test_bottleneck():
+ with pytest.raises(AssertionError):
+ # Style must be in ['pytorch', 'caffe']
+ SEBottleneckX(64, 64, groups=32, width_per_group=4, style='tensorflow')
+
+ # Test SEResNeXt Bottleneck structure
+ block = SEBottleneckX(
+ 64, 256, groups=32, width_per_group=4, stride=2, style='pytorch')
+ assert block.width_per_group == 4
+ assert block.conv2.stride == (2, 2)
+ assert block.conv2.groups == 32
+ assert block.conv2.out_channels == 128
+ assert block.conv2.out_channels == block.mid_channels
+
+ # Test SEResNeXt Bottleneck structure (groups=1)
+ block = SEBottleneckX(
+ 64, 256, groups=1, width_per_group=4, stride=2, style='pytorch')
+ assert block.conv2.stride == (2, 2)
+ assert block.conv2.groups == 1
+ assert block.conv2.out_channels == 64
+ assert block.mid_channels == 64
+ assert block.conv2.out_channels == block.mid_channels
+
+ # Test SEResNeXt Bottleneck forward
+ block = SEBottleneckX(
+ 64, 64, base_channels=16, groups=32, width_per_group=4)
+ x = torch.randn(1, 64, 56, 56)
+ x_out = block(x)
+ assert x_out.shape == torch.Size([1, 64, 56, 56])
+
+
+def test_seresnext():
+ with pytest.raises(KeyError):
+ # SEResNeXt depth should be in [50, 101, 152]
+ SEResNeXt(depth=18)
+
+ # Test SEResNeXt with group 32, width_per_group 4
+ model = SEResNeXt(
+ depth=50, groups=32, width_per_group=4, out_indices=(0, 1, 2, 3))
+ for m in model.modules():
+ if isinstance(m, SEBottleneckX):
+ assert m.conv2.groups == 32
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert len(feat) == 4
+ assert feat[0].shape == torch.Size([1, 256, 56, 56])
+ assert feat[1].shape == torch.Size([1, 512, 28, 28])
+ assert feat[2].shape == torch.Size([1, 1024, 14, 14])
+ assert feat[3].shape == torch.Size([1, 2048, 7, 7])
+
+ # Test SEResNeXt with group 32, width_per_group 4 and layers 3 out forward
+ model = SEResNeXt(
+ depth=50, groups=32, width_per_group=4, out_indices=(3, ))
+ for m in model.modules():
+ if isinstance(m, SEBottleneckX):
+ assert m.conv2.groups == 32
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert feat.shape == torch.Size([1, 2048, 7, 7])
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_shufflenet_v1.py b/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_shufflenet_v1.py
new file mode 100644
index 0000000..302d52f
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_shufflenet_v1.py
@@ -0,0 +1,245 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import pytest
+import torch
+from torch.nn.modules import GroupNorm
+from torch.nn.modules.batchnorm import _BatchNorm
+
+from mmpose.models.backbones import ShuffleNetV1
+from mmpose.models.backbones.shufflenet_v1 import ShuffleUnit
+
+
+def is_block(modules):
+ """Check if is ResNet building block."""
+ if isinstance(modules, (ShuffleUnit, )):
+ return True
+ return False
+
+
+def is_norm(modules):
+ """Check if is one of the norms."""
+ if isinstance(modules, (GroupNorm, _BatchNorm)):
+ return True
+ return False
+
+
+def check_norm_state(modules, train_state):
+ """Check if norm layer is in correct train state."""
+ for mod in modules:
+ if isinstance(mod, _BatchNorm):
+ if mod.training != train_state:
+ return False
+ return True
+
+
+def test_shufflenetv1_shuffleuint():
+
+ with pytest.raises(ValueError):
+ # combine must be in ['add', 'concat']
+ ShuffleUnit(24, 16, groups=3, first_block=True, combine='test')
+
+ with pytest.raises(AssertionError):
+ # inplanes must be equal tp = outplanes when combine='add'
+ ShuffleUnit(64, 24, groups=4, first_block=True, combine='add')
+
+ # Test ShuffleUnit with combine='add'
+ block = ShuffleUnit(24, 24, groups=3, first_block=True, combine='add')
+ x = torch.randn(1, 24, 56, 56)
+ x_out = block(x)
+ assert x_out.shape == torch.Size((1, 24, 56, 56))
+
+ # Test ShuffleUnit with combine='concat'
+ block = ShuffleUnit(24, 240, groups=3, first_block=True, combine='concat')
+ x = torch.randn(1, 24, 56, 56)
+ x_out = block(x)
+ assert x_out.shape == torch.Size((1, 240, 28, 28))
+
+ # Test ShuffleUnit with checkpoint forward
+ block = ShuffleUnit(
+ 24, 24, groups=3, first_block=True, combine='add', with_cp=True)
+ assert block.with_cp
+ x = torch.randn(1, 24, 56, 56)
+ x.requires_grad = True
+ x_out = block(x)
+ assert x_out.shape == torch.Size((1, 24, 56, 56))
+
+
+def test_shufflenetv1_backbone():
+
+ with pytest.raises(ValueError):
+ # frozen_stages must be in range(-1, 4)
+ ShuffleNetV1(frozen_stages=10)
+
+ with pytest.raises(ValueError):
+ # the item in out_indices must be in range(0, 4)
+ ShuffleNetV1(out_indices=[5])
+
+ with pytest.raises(ValueError):
+ # groups must be in [1, 2, 3, 4, 8]
+ ShuffleNetV1(groups=10)
+
+ with pytest.raises(TypeError):
+ # pretrained must be str or None
+ model = ShuffleNetV1()
+ model.init_weights(pretrained=1)
+
+ # Test ShuffleNetV1 norm state
+ model = ShuffleNetV1()
+ model.init_weights()
+ model.train()
+ assert check_norm_state(model.modules(), True)
+
+ # Test ShuffleNetV1 with first stage frozen
+ frozen_stages = 1
+ model = ShuffleNetV1(frozen_stages=frozen_stages, out_indices=(0, 1, 2))
+ model.init_weights()
+ model.train()
+ for param in model.conv1.parameters():
+ assert param.requires_grad is False
+ for i in range(frozen_stages):
+ layer = model.layers[i]
+ for mod in layer.modules():
+ if isinstance(mod, _BatchNorm):
+ assert mod.training is False
+ for param in layer.parameters():
+ assert param.requires_grad is False
+
+ # Test ShuffleNetV1 forward with groups=1
+ model = ShuffleNetV1(groups=1, out_indices=(0, 1, 2))
+ model.init_weights()
+ model.train()
+
+ for m in model.modules():
+ if is_norm(m):
+ assert isinstance(m, _BatchNorm)
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert len(feat) == 3
+ assert feat[0].shape == torch.Size((1, 144, 28, 28))
+ assert feat[1].shape == torch.Size((1, 288, 14, 14))
+ assert feat[2].shape == torch.Size((1, 576, 7, 7))
+
+ # Test ShuffleNetV1 forward with groups=2
+ model = ShuffleNetV1(groups=2, out_indices=(0, 1, 2))
+ model.init_weights()
+ model.train()
+
+ for m in model.modules():
+ if is_norm(m):
+ assert isinstance(m, _BatchNorm)
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert len(feat) == 3
+ assert feat[0].shape == torch.Size((1, 200, 28, 28))
+ assert feat[1].shape == torch.Size((1, 400, 14, 14))
+ assert feat[2].shape == torch.Size((1, 800, 7, 7))
+
+ # Test ShuffleNetV1 forward with groups=3
+ model = ShuffleNetV1(groups=3, out_indices=(0, 1, 2))
+ model.init_weights()
+ model.train()
+
+ for m in model.modules():
+ if is_norm(m):
+ assert isinstance(m, _BatchNorm)
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert len(feat) == 3
+ assert feat[0].shape == torch.Size((1, 240, 28, 28))
+ assert feat[1].shape == torch.Size((1, 480, 14, 14))
+ assert feat[2].shape == torch.Size((1, 960, 7, 7))
+
+ # Test ShuffleNetV1 forward with groups=4
+ model = ShuffleNetV1(groups=4, out_indices=(0, 1, 2))
+ model.init_weights()
+ model.train()
+
+ for m in model.modules():
+ if is_norm(m):
+ assert isinstance(m, _BatchNorm)
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert len(feat) == 3
+ assert feat[0].shape == torch.Size((1, 272, 28, 28))
+ assert feat[1].shape == torch.Size((1, 544, 14, 14))
+ assert feat[2].shape == torch.Size((1, 1088, 7, 7))
+
+ # Test ShuffleNetV1 forward with groups=8
+ model = ShuffleNetV1(groups=8, out_indices=(0, 1, 2))
+ model.init_weights()
+ model.train()
+
+ for m in model.modules():
+ if is_norm(m):
+ assert isinstance(m, _BatchNorm)
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert len(feat) == 3
+ assert feat[0].shape == torch.Size((1, 384, 28, 28))
+ assert feat[1].shape == torch.Size((1, 768, 14, 14))
+ assert feat[2].shape == torch.Size((1, 1536, 7, 7))
+
+ # Test ShuffleNetV1 forward with GroupNorm forward
+ model = ShuffleNetV1(
+ groups=3,
+ norm_cfg=dict(type='GN', num_groups=2, requires_grad=True),
+ out_indices=(0, 1, 2))
+ model.init_weights()
+ model.train()
+
+ for m in model.modules():
+ if is_norm(m):
+ assert isinstance(m, GroupNorm)
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert len(feat) == 3
+ assert feat[0].shape == torch.Size((1, 240, 28, 28))
+ assert feat[1].shape == torch.Size((1, 480, 14, 14))
+ assert feat[2].shape == torch.Size((1, 960, 7, 7))
+
+ # Test ShuffleNetV1 forward with layers 1, 2 forward
+ model = ShuffleNetV1(groups=3, out_indices=(1, 2))
+ model.init_weights()
+ model.train()
+
+ for m in model.modules():
+ if is_norm(m):
+ assert isinstance(m, _BatchNorm)
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert len(feat) == 2
+ assert feat[0].shape == torch.Size((1, 480, 14, 14))
+ assert feat[1].shape == torch.Size((1, 960, 7, 7))
+
+ # Test ShuffleNetV1 forward with layers 2 forward
+ model = ShuffleNetV1(groups=3, out_indices=(2, ))
+ model.init_weights()
+ model.train()
+
+ for m in model.modules():
+ if is_norm(m):
+ assert isinstance(m, _BatchNorm)
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert isinstance(feat, torch.Tensor)
+ assert feat.shape == torch.Size((1, 960, 7, 7))
+
+ # Test ShuffleNetV1 forward with checkpoint forward
+ model = ShuffleNetV1(groups=3, with_cp=True)
+ for m in model.modules():
+ if is_block(m):
+ assert m.with_cp
+
+ # Test ShuffleNetV1 with norm_eval
+ model = ShuffleNetV1(norm_eval=True)
+ model.init_weights()
+ model.train()
+
+ assert check_norm_state(model.modules(), False)
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_shufflenet_v2.py b/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_shufflenet_v2.py
new file mode 100644
index 0000000..2af5254
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_shufflenet_v2.py
@@ -0,0 +1,204 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import pytest
+import torch
+from torch.nn.modules import GroupNorm
+from torch.nn.modules.batchnorm import _BatchNorm
+
+from mmpose.models.backbones import ShuffleNetV2
+from mmpose.models.backbones.shufflenet_v2 import InvertedResidual
+
+
+def is_block(modules):
+ """Check if is ResNet building block."""
+ if isinstance(modules, (InvertedResidual, )):
+ return True
+ return False
+
+
+def is_norm(modules):
+ """Check if is one of the norms."""
+ if isinstance(modules, (GroupNorm, _BatchNorm)):
+ return True
+ return False
+
+
+def check_norm_state(modules, train_state):
+ """Check if norm layer is in correct train state."""
+ for mod in modules:
+ if isinstance(mod, _BatchNorm):
+ if mod.training != train_state:
+ return False
+ return True
+
+
+def test_shufflenetv2_invertedresidual():
+
+ with pytest.raises(AssertionError):
+ # when stride==1, in_channels should be equal to out_channels // 2 * 2
+ InvertedResidual(24, 32, stride=1)
+
+ with pytest.raises(AssertionError):
+ # when in_channels != out_channels // 2 * 2, stride should not be
+ # equal to 1.
+ InvertedResidual(24, 32, stride=1)
+
+ # Test InvertedResidual forward
+ block = InvertedResidual(24, 48, stride=2)
+ x = torch.randn(1, 24, 56, 56)
+ x_out = block(x)
+ assert x_out.shape == torch.Size((1, 48, 28, 28))
+
+ # Test InvertedResidual with checkpoint forward
+ block = InvertedResidual(48, 48, stride=1, with_cp=True)
+ assert block.with_cp
+ x = torch.randn(1, 48, 56, 56)
+ x.requires_grad = True
+ x_out = block(x)
+ assert x_out.shape == torch.Size((1, 48, 56, 56))
+
+
+def test_shufflenetv2_backbone():
+
+ with pytest.raises(ValueError):
+ # groups must be in 0.5, 1.0, 1.5, 2.0]
+ ShuffleNetV2(widen_factor=3.0)
+
+ with pytest.raises(ValueError):
+ # frozen_stages must be in [0, 1, 2, 3]
+ ShuffleNetV2(widen_factor=1.0, frozen_stages=4)
+
+ with pytest.raises(ValueError):
+ # out_indices must be in [0, 1, 2, 3]
+ ShuffleNetV2(widen_factor=1.0, out_indices=(4, ))
+
+ with pytest.raises(TypeError):
+ # pretrained must be str or None
+ model = ShuffleNetV2()
+ model.init_weights(pretrained=1)
+
+ # Test ShuffleNetV2 norm state
+ model = ShuffleNetV2()
+ model.init_weights()
+ model.train()
+ assert check_norm_state(model.modules(), True)
+
+ # Test ShuffleNetV2 with first stage frozen
+ frozen_stages = 1
+ model = ShuffleNetV2(frozen_stages=frozen_stages)
+ model.init_weights()
+ model.train()
+ for param in model.conv1.parameters():
+ assert param.requires_grad is False
+ for i in range(0, frozen_stages):
+ layer = model.layers[i]
+ for mod in layer.modules():
+ if isinstance(mod, _BatchNorm):
+ assert mod.training is False
+ for param in layer.parameters():
+ assert param.requires_grad is False
+
+ # Test ShuffleNetV2 with norm_eval
+ model = ShuffleNetV2(norm_eval=True)
+ model.init_weights()
+ model.train()
+
+ assert check_norm_state(model.modules(), False)
+
+ # Test ShuffleNetV2 forward with widen_factor=0.5
+ model = ShuffleNetV2(widen_factor=0.5, out_indices=(0, 1, 2, 3))
+ model.init_weights()
+ model.train()
+
+ for m in model.modules():
+ if is_norm(m):
+ assert isinstance(m, _BatchNorm)
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert len(feat) == 4
+ assert feat[0].shape == torch.Size((1, 48, 28, 28))
+ assert feat[1].shape == torch.Size((1, 96, 14, 14))
+ assert feat[2].shape == torch.Size((1, 192, 7, 7))
+
+ # Test ShuffleNetV2 forward with widen_factor=1.0
+ model = ShuffleNetV2(widen_factor=1.0, out_indices=(0, 1, 2, 3))
+ model.init_weights()
+ model.train()
+
+ for m in model.modules():
+ if is_norm(m):
+ assert isinstance(m, _BatchNorm)
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert len(feat) == 4
+ assert feat[0].shape == torch.Size((1, 116, 28, 28))
+ assert feat[1].shape == torch.Size((1, 232, 14, 14))
+ assert feat[2].shape == torch.Size((1, 464, 7, 7))
+
+ # Test ShuffleNetV2 forward with widen_factor=1.5
+ model = ShuffleNetV2(widen_factor=1.5, out_indices=(0, 1, 2, 3))
+ model.init_weights()
+ model.train()
+
+ for m in model.modules():
+ if is_norm(m):
+ assert isinstance(m, _BatchNorm)
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert len(feat) == 4
+ assert feat[0].shape == torch.Size((1, 176, 28, 28))
+ assert feat[1].shape == torch.Size((1, 352, 14, 14))
+ assert feat[2].shape == torch.Size((1, 704, 7, 7))
+
+ # Test ShuffleNetV2 forward with widen_factor=2.0
+ model = ShuffleNetV2(widen_factor=2.0, out_indices=(0, 1, 2, 3))
+ model.init_weights()
+ model.train()
+
+ for m in model.modules():
+ if is_norm(m):
+ assert isinstance(m, _BatchNorm)
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert len(feat) == 4
+ assert feat[0].shape == torch.Size((1, 244, 28, 28))
+ assert feat[1].shape == torch.Size((1, 488, 14, 14))
+ assert feat[2].shape == torch.Size((1, 976, 7, 7))
+
+ # Test ShuffleNetV2 forward with layers 3 forward
+ model = ShuffleNetV2(widen_factor=1.0, out_indices=(2, ))
+ model.init_weights()
+ model.train()
+
+ for m in model.modules():
+ if is_norm(m):
+ assert isinstance(m, _BatchNorm)
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert isinstance(feat, torch.Tensor)
+ assert feat.shape == torch.Size((1, 464, 7, 7))
+
+ # Test ShuffleNetV2 forward with layers 1 2 forward
+ model = ShuffleNetV2(widen_factor=1.0, out_indices=(1, 2))
+ model.init_weights()
+ model.train()
+
+ for m in model.modules():
+ if is_norm(m):
+ assert isinstance(m, _BatchNorm)
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert len(feat) == 2
+ assert feat[0].shape == torch.Size((1, 232, 14, 14))
+ assert feat[1].shape == torch.Size((1, 464, 7, 7))
+
+ # Test ShuffleNetV2 forward with checkpoint forward
+ model = ShuffleNetV2(widen_factor=1.0, with_cp=True)
+ for m in model.modules():
+ if is_block(m):
+ assert m.with_cp
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_tcn.py b/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_tcn.py
new file mode 100644
index 0000000..be66a0a
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_tcn.py
@@ -0,0 +1,153 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import numpy as np
+import pytest
+import torch
+import torch.nn as nn
+
+from mmpose.models.backbones import TCN
+from mmpose.models.backbones.tcn import BasicTemporalBlock
+
+
+def test_basic_temporal_block():
+ with pytest.raises(AssertionError):
+ # padding( + shift) should not be larger than x.shape[2]
+ block = BasicTemporalBlock(1024, 1024, dilation=81)
+ x = torch.rand(2, 1024, 150)
+ x_out = block(x)
+
+ with pytest.raises(AssertionError):
+ # when use_stride_conv is True, shift + kernel_size // 2 should
+ # not be larger than x.shape[2]
+ block = BasicTemporalBlock(
+ 1024, 1024, kernel_size=5, causal=True, use_stride_conv=True)
+ x = torch.rand(2, 1024, 3)
+ x_out = block(x)
+
+ # BasicTemporalBlock with causal == False
+ block = BasicTemporalBlock(1024, 1024)
+ x = torch.rand(2, 1024, 241)
+ x_out = block(x)
+ assert x_out.shape == torch.Size([2, 1024, 235])
+
+ # BasicTemporalBlock with causal == True
+ block = BasicTemporalBlock(1024, 1024, causal=True)
+ x = torch.rand(2, 1024, 241)
+ x_out = block(x)
+ assert x_out.shape == torch.Size([2, 1024, 235])
+
+ # BasicTemporalBlock with residual == False
+ block = BasicTemporalBlock(1024, 1024, residual=False)
+ x = torch.rand(2, 1024, 241)
+ x_out = block(x)
+ assert x_out.shape == torch.Size([2, 1024, 235])
+
+ # BasicTemporalBlock, use_stride_conv == True
+ block = BasicTemporalBlock(1024, 1024, use_stride_conv=True)
+ x = torch.rand(2, 1024, 81)
+ x_out = block(x)
+ assert x_out.shape == torch.Size([2, 1024, 27])
+
+ # BasicTemporalBlock with use_stride_conv == True and causal == True
+ block = BasicTemporalBlock(1024, 1024, use_stride_conv=True, causal=True)
+ x = torch.rand(2, 1024, 81)
+ x_out = block(x)
+ assert x_out.shape == torch.Size([2, 1024, 27])
+
+
+def test_tcn_backbone():
+ with pytest.raises(AssertionError):
+ # num_blocks should equal len(kernel_sizes) - 1
+ TCN(in_channels=34, num_blocks=3, kernel_sizes=(3, 3, 3))
+
+ with pytest.raises(AssertionError):
+ # kernel size should be odd
+ TCN(in_channels=34, kernel_sizes=(3, 4, 3))
+
+ # Test TCN with 2 blocks (use_stride_conv == False)
+ model = TCN(in_channels=34, num_blocks=2, kernel_sizes=(3, 3, 3))
+ pose2d = torch.rand((2, 34, 243))
+ feat = model(pose2d)
+ assert len(feat) == 2
+ assert feat[0].shape == (2, 1024, 235)
+ assert feat[1].shape == (2, 1024, 217)
+
+ # Test TCN with 4 blocks and weight norm clip
+ max_norm = 0.1
+ model = TCN(
+ in_channels=34,
+ num_blocks=4,
+ kernel_sizes=(3, 3, 3, 3, 3),
+ max_norm=max_norm)
+ pose2d = torch.rand((2, 34, 243))
+ feat = model(pose2d)
+ assert len(feat) == 4
+ assert feat[0].shape == (2, 1024, 235)
+ assert feat[1].shape == (2, 1024, 217)
+ assert feat[2].shape == (2, 1024, 163)
+ assert feat[3].shape == (2, 1024, 1)
+
+ for module in model.modules():
+ if isinstance(module, torch.nn.modules.conv._ConvNd):
+ norm = module.weight.norm().item()
+ np.testing.assert_allclose(
+ np.maximum(norm, max_norm), max_norm, rtol=1e-4)
+
+ # Test TCN with 4 blocks (use_stride_conv == True)
+ model = TCN(
+ in_channels=34,
+ num_blocks=4,
+ kernel_sizes=(3, 3, 3, 3, 3),
+ use_stride_conv=True)
+ pose2d = torch.rand((2, 34, 243))
+ feat = model(pose2d)
+ assert len(feat) == 4
+ assert feat[0].shape == (2, 1024, 27)
+ assert feat[1].shape == (2, 1024, 9)
+ assert feat[2].shape == (2, 1024, 3)
+ assert feat[3].shape == (2, 1024, 1)
+
+ # Check that the model w. or w/o use_stride_conv will have the same
+ # output and gradient after a forward+backward pass
+ model1 = TCN(
+ in_channels=34,
+ stem_channels=4,
+ num_blocks=1,
+ kernel_sizes=(3, 3),
+ dropout=0,
+ residual=False,
+ norm_cfg=None)
+ model2 = TCN(
+ in_channels=34,
+ stem_channels=4,
+ num_blocks=1,
+ kernel_sizes=(3, 3),
+ dropout=0,
+ residual=False,
+ norm_cfg=None,
+ use_stride_conv=True)
+ for m in model1.modules():
+ if isinstance(m, nn.Conv1d):
+ nn.init.constant_(m.weight, 0.5)
+ if m.bias is not None:
+ nn.init.constant_(m.bias, 0)
+ for m in model2.modules():
+ if isinstance(m, nn.Conv1d):
+ nn.init.constant_(m.weight, 0.5)
+ if m.bias is not None:
+ nn.init.constant_(m.bias, 0)
+ input1 = torch.rand((1, 34, 9))
+ input2 = input1.clone()
+ outputs1 = model1(input1)
+ outputs2 = model2(input2)
+ for output1, output2 in zip(outputs1, outputs2):
+ assert torch.isclose(output1, output2).all()
+
+ criterion = nn.MSELoss()
+ target = torch.rand(output1.shape)
+ loss1 = criterion(output1, target)
+ loss2 = criterion(output2, target)
+ loss1.backward()
+ loss2.backward()
+ for m1, m2 in zip(model1.modules(), model2.modules()):
+ if isinstance(m1, nn.Conv1d):
+ assert torch.isclose(m1.weight.grad, m2.weight.grad).all()
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_v2v_net.py b/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_v2v_net.py
new file mode 100644
index 0000000..33c467a
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_v2v_net.py
@@ -0,0 +1,13 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import torch
+
+from mmpose.models import builder
+
+
+def test_v2v_net():
+ """Test V2VNet."""
+ cfg = dict(type='V2VNet', input_channels=17, output_channels=15),
+ model = builder.build_backbone(*cfg)
+ input = torch.randn(2, 17, 32, 32, 32)
+ output = model(input)
+ assert output.shape == (2, 15, 32, 32, 32)
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_vgg.py b/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_vgg.py
new file mode 100644
index 0000000..f69e38b
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_vgg.py
@@ -0,0 +1,137 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import pytest
+import torch
+from mmcv.utils.parrots_wrapper import _BatchNorm
+
+from mmpose.models.backbones import VGG
+
+
+def check_norm_state(modules, train_state):
+ """Check if norm layer is in correct train state."""
+ for mod in modules:
+ if isinstance(mod, _BatchNorm):
+ if mod.training != train_state:
+ return False
+ return True
+
+
+def test_vgg():
+ """Test VGG backbone."""
+ with pytest.raises(KeyError):
+ # VGG depth should be in [11, 13, 16, 19]
+ VGG(18)
+
+ with pytest.raises(AssertionError):
+ # In VGG: 1 <= num_stages <= 5
+ VGG(11, num_stages=0)
+
+ with pytest.raises(AssertionError):
+ # In VGG: 1 <= num_stages <= 5
+ VGG(11, num_stages=6)
+
+ with pytest.raises(AssertionError):
+ # len(dilations) == num_stages
+ VGG(11, dilations=(1, 1), num_stages=3)
+
+ with pytest.raises(TypeError):
+ # pretrained must be a string path
+ model = VGG(11)
+ model.init_weights(pretrained=0)
+
+ # Test VGG11 norm_eval=True
+ model = VGG(11, norm_eval=True)
+ model.init_weights()
+ model.train()
+ assert check_norm_state(model.modules(), False)
+
+ # Test VGG11 forward without classifiers
+ model = VGG(11, out_indices=(0, 1, 2, 3, 4))
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert len(feat) == 5
+ assert feat[0].shape == (1, 64, 112, 112)
+ assert feat[1].shape == (1, 128, 56, 56)
+ assert feat[2].shape == (1, 256, 28, 28)
+ assert feat[3].shape == (1, 512, 14, 14)
+ assert feat[4].shape == (1, 512, 7, 7)
+
+ # Test VGG11 forward with classifiers
+ model = VGG(11, num_classes=10, out_indices=(0, 1, 2, 3, 4, 5))
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert len(feat) == 6
+ assert feat[0].shape == (1, 64, 112, 112)
+ assert feat[1].shape == (1, 128, 56, 56)
+ assert feat[2].shape == (1, 256, 28, 28)
+ assert feat[3].shape == (1, 512, 14, 14)
+ assert feat[4].shape == (1, 512, 7, 7)
+ assert feat[5].shape == (1, 10)
+
+ # Test VGG11BN forward
+ model = VGG(11, norm_cfg=dict(type='BN'), out_indices=(0, 1, 2, 3, 4))
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert len(feat) == 5
+ assert feat[0].shape == (1, 64, 112, 112)
+ assert feat[1].shape == (1, 128, 56, 56)
+ assert feat[2].shape == (1, 256, 28, 28)
+ assert feat[3].shape == (1, 512, 14, 14)
+ assert feat[4].shape == (1, 512, 7, 7)
+
+ # Test VGG11BN forward with classifiers
+ model = VGG(
+ 11,
+ num_classes=10,
+ norm_cfg=dict(type='BN'),
+ out_indices=(0, 1, 2, 3, 4, 5))
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert len(feat) == 6
+ assert feat[0].shape == (1, 64, 112, 112)
+ assert feat[1].shape == (1, 128, 56, 56)
+ assert feat[2].shape == (1, 256, 28, 28)
+ assert feat[3].shape == (1, 512, 14, 14)
+ assert feat[4].shape == (1, 512, 7, 7)
+ assert feat[5].shape == (1, 10)
+
+ # Test VGG13 with layers 1, 2, 3 out forward
+ model = VGG(13, out_indices=(0, 1, 2))
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert len(feat) == 3
+ assert feat[0].shape == (1, 64, 112, 112)
+ assert feat[1].shape == (1, 128, 56, 56)
+ assert feat[2].shape == (1, 256, 28, 28)
+
+ # Test VGG16 with top feature maps out forward
+ model = VGG(16)
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert feat.shape == (1, 512, 7, 7)
+
+ # Test VGG19 with classification score out forward
+ model = VGG(19, num_classes=10)
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert feat.shape == (1, 10)
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_vipnas_mbv3.py b/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_vipnas_mbv3.py
new file mode 100644
index 0000000..83011da
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_vipnas_mbv3.py
@@ -0,0 +1,99 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import pytest
+import torch
+from torch.nn.modules import GroupNorm
+from torch.nn.modules.batchnorm import _BatchNorm
+
+from mmpose.models.backbones import ViPNAS_MobileNetV3
+from mmpose.models.backbones.utils import InvertedResidual
+
+
+def is_norm(modules):
+ """Check if is one of the norms."""
+ if isinstance(modules, (GroupNorm, _BatchNorm)):
+ return True
+ return False
+
+
+def check_norm_state(modules, train_state):
+ """Check if norm layer is in correct train state."""
+ for mod in modules:
+ if isinstance(mod, _BatchNorm):
+ if mod.training != train_state:
+ return False
+ return True
+
+
+def test_mobilenetv3_backbone():
+ with pytest.raises(TypeError):
+ # pretrained must be a string path
+ model = ViPNAS_MobileNetV3()
+ model.init_weights(pretrained=0)
+
+ with pytest.raises(AttributeError):
+ # frozen_stages must no more than 21
+ model = ViPNAS_MobileNetV3(frozen_stages=22)
+ model.train()
+
+ # Test MobileNetv3
+ model = ViPNAS_MobileNetV3()
+ model.init_weights()
+ model.train()
+
+ # Test MobileNetv3 with first stage frozen
+ frozen_stages = 1
+ model = ViPNAS_MobileNetV3(frozen_stages=frozen_stages)
+ model.init_weights()
+ model.train()
+ for param in model.conv1.parameters():
+ assert param.requires_grad is False
+ for i in range(1, frozen_stages + 1):
+ layer = getattr(model, f'layer{i}')
+ for mod in layer.modules():
+ if isinstance(mod, _BatchNorm):
+ assert mod.training is False
+ for param in layer.parameters():
+ assert param.requires_grad is False
+
+ # Test MobileNetv3 with norm eval
+ model = ViPNAS_MobileNetV3(norm_eval=True)
+ model.init_weights()
+ model.train()
+ assert check_norm_state(model.modules(), False)
+
+ # Test MobileNetv3 forward
+ model = ViPNAS_MobileNetV3()
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert feat.shape == torch.Size([1, 160, 7, 7])
+
+ # Test MobileNetv3 forward with GroupNorm
+ model = ViPNAS_MobileNetV3(
+ norm_cfg=dict(type='GN', num_groups=2, requires_grad=True))
+ for m in model.modules():
+ if is_norm(m):
+ assert isinstance(m, GroupNorm)
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert feat.shape == torch.Size([1, 160, 7, 7])
+
+ # Test MobileNetv3 with checkpoint forward
+ model = ViPNAS_MobileNetV3(with_cp=True)
+ for m in model.modules():
+ if isinstance(m, InvertedResidual):
+ assert m.with_cp
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert feat.shape == torch.Size([1, 160, 7, 7])
+
+
+test_mobilenetv3_backbone()
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_vipnas_resnet.py b/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_vipnas_resnet.py
new file mode 100644
index 0000000..2793589
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_backbones/test_vipnas_resnet.py
@@ -0,0 +1,341 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import pytest
+import torch
+import torch.nn as nn
+from mmcv.utils.parrots_wrapper import _BatchNorm
+
+from mmpose.models.backbones import ViPNAS_ResNet
+from mmpose.models.backbones.vipnas_resnet import (ViPNAS_Bottleneck,
+ ViPNAS_ResLayer,
+ get_expansion)
+
+
+def is_block(modules):
+ """Check if is ViPNAS_ResNet building block."""
+ if isinstance(modules, (ViPNAS_Bottleneck)):
+ return True
+ return False
+
+
+def all_zeros(modules):
+ """Check if the weight(and bias) is all zero."""
+ weight_zero = torch.equal(modules.weight.data,
+ torch.zeros_like(modules.weight.data))
+ if hasattr(modules, 'bias'):
+ bias_zero = torch.equal(modules.bias.data,
+ torch.zeros_like(modules.bias.data))
+ else:
+ bias_zero = True
+
+ return weight_zero and bias_zero
+
+
+def check_norm_state(modules, train_state):
+ """Check if norm layer is in correct train state."""
+ for mod in modules:
+ if isinstance(mod, _BatchNorm):
+ if mod.training != train_state:
+ return False
+ return True
+
+
+def test_get_expansion():
+ assert get_expansion(ViPNAS_Bottleneck, 2) == 2
+ assert get_expansion(ViPNAS_Bottleneck) == 1
+
+ class MyResBlock(nn.Module):
+
+ expansion = 8
+
+ assert get_expansion(MyResBlock) == 8
+
+ # expansion must be an integer or None
+ with pytest.raises(TypeError):
+ get_expansion(ViPNAS_Bottleneck, '0')
+
+ # expansion is not specified and cannot be inferred
+ with pytest.raises(TypeError):
+
+ class SomeModule(nn.Module):
+ pass
+
+ get_expansion(SomeModule)
+
+
+def test_vipnas_bottleneck():
+ # style must be in ['pytorch', 'caffe']
+ with pytest.raises(AssertionError):
+ ViPNAS_Bottleneck(64, 64, style='tensorflow')
+
+ # expansion must be divisible by out_channels
+ with pytest.raises(AssertionError):
+ ViPNAS_Bottleneck(64, 64, expansion=3)
+
+ # Test ViPNAS_Bottleneck style
+ block = ViPNAS_Bottleneck(64, 64, stride=2, style='pytorch')
+ assert block.conv1.stride == (1, 1)
+ assert block.conv2.stride == (2, 2)
+ block = ViPNAS_Bottleneck(64, 64, stride=2, style='caffe')
+ assert block.conv1.stride == (2, 2)
+ assert block.conv2.stride == (1, 1)
+
+ # ViPNAS_Bottleneck with stride 1
+ block = ViPNAS_Bottleneck(64, 64, style='pytorch')
+ assert block.in_channels == 64
+ assert block.mid_channels == 16
+ assert block.out_channels == 64
+ assert block.conv1.in_channels == 64
+ assert block.conv1.out_channels == 16
+ assert block.conv1.kernel_size == (1, 1)
+ assert block.conv2.in_channels == 16
+ assert block.conv2.out_channels == 16
+ assert block.conv2.kernel_size == (3, 3)
+ assert block.conv3.in_channels == 16
+ assert block.conv3.out_channels == 64
+ assert block.conv3.kernel_size == (1, 1)
+ x = torch.randn(1, 64, 56, 56)
+ x_out = block(x)
+ assert x_out.shape == (1, 64, 56, 56)
+
+ # ViPNAS_Bottleneck with stride 1 and downsample
+ downsample = nn.Sequential(
+ nn.Conv2d(64, 128, kernel_size=1), nn.BatchNorm2d(128))
+ block = ViPNAS_Bottleneck(64, 128, style='pytorch', downsample=downsample)
+ assert block.in_channels == 64
+ assert block.mid_channels == 32
+ assert block.out_channels == 128
+ assert block.conv1.in_channels == 64
+ assert block.conv1.out_channels == 32
+ assert block.conv1.kernel_size == (1, 1)
+ assert block.conv2.in_channels == 32
+ assert block.conv2.out_channels == 32
+ assert block.conv2.kernel_size == (3, 3)
+ assert block.conv3.in_channels == 32
+ assert block.conv3.out_channels == 128
+ assert block.conv3.kernel_size == (1, 1)
+ x = torch.randn(1, 64, 56, 56)
+ x_out = block(x)
+ assert x_out.shape == (1, 128, 56, 56)
+
+ # ViPNAS_Bottleneck with stride 2 and downsample
+ downsample = nn.Sequential(
+ nn.Conv2d(64, 128, kernel_size=1, stride=2), nn.BatchNorm2d(128))
+ block = ViPNAS_Bottleneck(
+ 64, 128, stride=2, style='pytorch', downsample=downsample)
+ x = torch.randn(1, 64, 56, 56)
+ x_out = block(x)
+ assert x_out.shape == (1, 128, 28, 28)
+
+ # ViPNAS_Bottleneck with expansion 2
+ block = ViPNAS_Bottleneck(64, 64, style='pytorch', expansion=2)
+ assert block.in_channels == 64
+ assert block.mid_channels == 32
+ assert block.out_channels == 64
+ assert block.conv1.in_channels == 64
+ assert block.conv1.out_channels == 32
+ assert block.conv1.kernel_size == (1, 1)
+ assert block.conv2.in_channels == 32
+ assert block.conv2.out_channels == 32
+ assert block.conv2.kernel_size == (3, 3)
+ assert block.conv3.in_channels == 32
+ assert block.conv3.out_channels == 64
+ assert block.conv3.kernel_size == (1, 1)
+ x = torch.randn(1, 64, 56, 56)
+ x_out = block(x)
+ assert x_out.shape == (1, 64, 56, 56)
+
+ # Test ViPNAS_Bottleneck with checkpointing
+ block = ViPNAS_Bottleneck(64, 64, with_cp=True)
+ block.train()
+ assert block.with_cp
+ x = torch.randn(1, 64, 56, 56, requires_grad=True)
+ x_out = block(x)
+ assert x_out.shape == torch.Size([1, 64, 56, 56])
+
+
+def test_vipnas_bottleneck_reslayer():
+ # 3 Bottleneck w/o downsample
+ layer = ViPNAS_ResLayer(ViPNAS_Bottleneck, 3, 32, 32)
+ assert len(layer) == 3
+ for i in range(3):
+ assert layer[i].in_channels == 32
+ assert layer[i].out_channels == 32
+ assert layer[i].downsample is None
+ x = torch.randn(1, 32, 56, 56)
+ x_out = layer(x)
+ assert x_out.shape == (1, 32, 56, 56)
+
+ # 3 ViPNAS_Bottleneck w/ stride 1 and downsample
+ layer = ViPNAS_ResLayer(ViPNAS_Bottleneck, 3, 32, 64)
+ assert len(layer) == 3
+ assert layer[0].in_channels == 32
+ assert layer[0].out_channels == 64
+ assert layer[0].stride == 1
+ assert layer[0].conv1.out_channels == 64
+ assert layer[0].downsample is not None and len(layer[0].downsample) == 2
+ assert isinstance(layer[0].downsample[0], nn.Conv2d)
+ assert layer[0].downsample[0].stride == (1, 1)
+ for i in range(1, 3):
+ assert layer[i].in_channels == 64
+ assert layer[i].out_channels == 64
+ assert layer[i].conv1.out_channels == 64
+ assert layer[i].stride == 1
+ assert layer[i].downsample is None
+ x = torch.randn(1, 32, 56, 56)
+ x_out = layer(x)
+ assert x_out.shape == (1, 64, 56, 56)
+
+ # 3 ViPNAS_Bottleneck w/ stride 2 and downsample
+ layer = ViPNAS_ResLayer(ViPNAS_Bottleneck, 3, 32, 64, stride=2)
+ assert len(layer) == 3
+ assert layer[0].in_channels == 32
+ assert layer[0].out_channels == 64
+ assert layer[0].stride == 2
+ assert layer[0].conv1.out_channels == 64
+ assert layer[0].downsample is not None and len(layer[0].downsample) == 2
+ assert isinstance(layer[0].downsample[0], nn.Conv2d)
+ assert layer[0].downsample[0].stride == (2, 2)
+ for i in range(1, 3):
+ assert layer[i].in_channels == 64
+ assert layer[i].out_channels == 64
+ assert layer[i].conv1.out_channels == 64
+ assert layer[i].stride == 1
+ assert layer[i].downsample is None
+ x = torch.randn(1, 32, 56, 56)
+ x_out = layer(x)
+ assert x_out.shape == (1, 64, 28, 28)
+
+ # 3 ViPNAS_Bottleneck w/ stride 2 and downsample with avg pool
+ layer = ViPNAS_ResLayer(
+ ViPNAS_Bottleneck, 3, 32, 64, stride=2, avg_down=True)
+ assert len(layer) == 3
+ assert layer[0].in_channels == 32
+ assert layer[0].out_channels == 64
+ assert layer[0].stride == 2
+ assert layer[0].conv1.out_channels == 64
+ assert layer[0].downsample is not None and len(layer[0].downsample) == 3
+ assert isinstance(layer[0].downsample[0], nn.AvgPool2d)
+ assert layer[0].downsample[0].stride == 2
+ for i in range(1, 3):
+ assert layer[i].in_channels == 64
+ assert layer[i].out_channels == 64
+ assert layer[i].conv1.out_channels == 64
+ assert layer[i].stride == 1
+ assert layer[i].downsample is None
+ x = torch.randn(1, 32, 56, 56)
+ x_out = layer(x)
+ assert x_out.shape == (1, 64, 28, 28)
+
+ # 3 ViPNAS_Bottleneck with custom expansion
+ layer = ViPNAS_ResLayer(ViPNAS_Bottleneck, 3, 32, 32, expansion=2)
+ assert len(layer) == 3
+ for i in range(3):
+ assert layer[i].in_channels == 32
+ assert layer[i].out_channels == 32
+ assert layer[i].stride == 1
+ assert layer[i].conv1.out_channels == 16
+ assert layer[i].downsample is None
+ x = torch.randn(1, 32, 56, 56)
+ x_out = layer(x)
+ assert x_out.shape == (1, 32, 56, 56)
+
+
+def test_resnet():
+ """Test ViPNAS_ResNet backbone."""
+ with pytest.raises(KeyError):
+ # ViPNAS_ResNet depth should be in [50]
+ ViPNAS_ResNet(20)
+
+ with pytest.raises(AssertionError):
+ # In ViPNAS_ResNet: 1 <= num_stages <= 4
+ ViPNAS_ResNet(50, num_stages=0)
+
+ with pytest.raises(AssertionError):
+ # In ViPNAS_ResNet: 1 <= num_stages <= 4
+ ViPNAS_ResNet(50, num_stages=5)
+
+ with pytest.raises(AssertionError):
+ # len(strides) == len(dilations) == num_stages
+ ViPNAS_ResNet(50, strides=(1, ), dilations=(1, 1), num_stages=3)
+
+ with pytest.raises(TypeError):
+ # pretrained must be a string path
+ model = ViPNAS_ResNet(50)
+ model.init_weights(pretrained=0)
+
+ with pytest.raises(AssertionError):
+ # Style must be in ['pytorch', 'caffe']
+ ViPNAS_ResNet(50, style='tensorflow')
+
+ # Test ViPNAS_ResNet50 norm_eval=True
+ model = ViPNAS_ResNet(50, norm_eval=True)
+ model.init_weights()
+ model.train()
+ assert check_norm_state(model.modules(), False)
+
+ # Test ViPNAS_ResNet50 with first stage frozen
+ frozen_stages = 1
+ model = ViPNAS_ResNet(50, frozen_stages=frozen_stages)
+ model.init_weights()
+ model.train()
+ assert model.norm1.training is False
+ for layer in [model.conv1, model.norm1]:
+ for param in layer.parameters():
+ assert param.requires_grad is False
+ for i in range(1, frozen_stages + 1):
+ layer = getattr(model, f'layer{i}')
+ for mod in layer.modules():
+ if isinstance(mod, _BatchNorm):
+ assert mod.training is False
+ for param in layer.parameters():
+ assert param.requires_grad is False
+
+ # Test ViPNAS_ResNet50 with BatchNorm forward
+ model = ViPNAS_ResNet(50, out_indices=(0, 1, 2, 3))
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert len(feat) == 4
+ assert feat[0].shape == (1, 80, 56, 56)
+ assert feat[1].shape == (1, 160, 28, 28)
+ assert feat[2].shape == (1, 304, 14, 14)
+ assert feat[3].shape == (1, 608, 7, 7)
+
+ # Test ViPNAS_ResNet50 with layers 1, 2, 3 out forward
+ model = ViPNAS_ResNet(50, out_indices=(0, 1, 2))
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert len(feat) == 3
+ assert feat[0].shape == (1, 80, 56, 56)
+ assert feat[1].shape == (1, 160, 28, 28)
+ assert feat[2].shape == (1, 304, 14, 14)
+
+ # Test ViPNAS_ResNet50 with layers 3 (top feature maps) out forward
+ model = ViPNAS_ResNet(50, out_indices=(3, ))
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert feat.shape == (1, 608, 7, 7)
+
+ # Test ViPNAS_ResNet50 with checkpoint forward
+ model = ViPNAS_ResNet(50, out_indices=(0, 1, 2, 3), with_cp=True)
+ for m in model.modules():
+ if is_block(m):
+ assert m.with_cp
+ model.init_weights()
+ model.train()
+
+ imgs = torch.randn(1, 3, 224, 224)
+ feat = model(imgs)
+ assert len(feat) == 4
+ assert feat[0].shape == (1, 80, 56, 56)
+ assert feat[1].shape == (1, 160, 28, 28)
+ assert feat[2].shape == (1, 304, 14, 14)
+ assert feat[3].shape == (1, 608, 7, 7)
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_backward_compatibility/test_dataset_info_compatibility/test_animal_dataset_compatibility.py b/engine/pose_estimation/third-party/ViTPose/tests/test_backward_compatibility/test_dataset_info_compatibility/test_animal_dataset_compatibility.py
new file mode 100644
index 0000000..3933612
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_backward_compatibility/test_dataset_info_compatibility/test_animal_dataset_compatibility.py
@@ -0,0 +1,415 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import copy
+import tempfile
+
+import pytest
+from numpy.testing import assert_almost_equal
+
+from mmpose.datasets import DATASETS
+from tests.utils.data_utils import convert_db_to_output
+
+
+def test_animal_horse10_dataset_compatibility():
+ dataset = 'AnimalHorse10Dataset'
+ dataset_class = DATASETS.get(dataset)
+
+ channel_cfg = dict(
+ num_output_channels=22,
+ dataset_joints=22,
+ dataset_channel=[
+ [
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17,
+ 18, 19, 21
+ ],
+ ],
+ inference_channel=[
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18,
+ 19, 21
+ ])
+
+ data_cfg = dict(
+ image_size=[256, 256],
+ heatmap_size=[64, 64],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'])
+ # Test
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ with pytest.warns(DeprecationWarning):
+ _ = dataset_class(
+ ann_file='tests/data/horse10/test_horse10.json',
+ img_prefix='tests/data/horse10/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ test_mode=True)
+
+ with pytest.warns(DeprecationWarning):
+ custom_dataset = dataset_class(
+ ann_file='tests/data/horse10/test_horse10.json',
+ img_prefix='tests/data/horse10/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ test_mode=False)
+
+ assert custom_dataset.test_mode is False
+ assert custom_dataset.num_images == 3
+ _ = custom_dataset[0]
+
+ outputs = convert_db_to_output(custom_dataset.db)
+ with tempfile.TemporaryDirectory() as tmpdir:
+ infos = custom_dataset.evaluate(outputs, tmpdir, ['PCK'])
+ assert_almost_equal(infos['PCK'], 1.0)
+
+ with pytest.raises(KeyError):
+ infos = custom_dataset.evaluate(outputs, tmpdir, 'mAP')
+
+
+def test_animal_fly_dataset_compatibility():
+ dataset = 'AnimalFlyDataset'
+ dataset_class = DATASETS.get(dataset)
+
+ channel_cfg = dict(
+ num_output_channels=32,
+ dataset_joints=32,
+ dataset_channel=[
+ [
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17,
+ 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31
+ ],
+ ],
+ inference_channel=[
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18,
+ 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31
+ ])
+
+ data_cfg = dict(
+ image_size=[192, 192],
+ heatmap_size=[48, 48],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'])
+
+ # Test
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ with pytest.warns(DeprecationWarning):
+ _ = dataset_class(
+ ann_file='tests/data/fly/test_fly.json',
+ img_prefix='tests/data/fly/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ test_mode=True)
+
+ with pytest.warns(DeprecationWarning):
+ custom_dataset = dataset_class(
+ ann_file='tests/data/fly/test_fly.json',
+ img_prefix='tests/data/fly/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ test_mode=False)
+
+ assert custom_dataset.test_mode is False
+ assert custom_dataset.num_images == 2
+ _ = custom_dataset[0]
+
+ outputs = convert_db_to_output(custom_dataset.db)
+ with tempfile.TemporaryDirectory() as tmpdir:
+ infos = custom_dataset.evaluate(outputs, tmpdir, ['PCK'])
+ assert_almost_equal(infos['PCK'], 1.0)
+
+ with pytest.raises(KeyError):
+ infos = custom_dataset.evaluate(outputs, tmpdir, 'mAP')
+
+
+def test_animal_locust_dataset_compatibility():
+ dataset = 'AnimalLocustDataset'
+ dataset_class = DATASETS.get(dataset)
+
+ channel_cfg = dict(
+ num_output_channels=35,
+ dataset_joints=35,
+ dataset_channel=[
+ [
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17,
+ 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33,
+ 34
+ ],
+ ],
+ inference_channel=[
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18,
+ 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34
+ ])
+
+ data_cfg = dict(
+ image_size=[160, 160],
+ heatmap_size=[40, 40],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'])
+
+ # Test
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ with pytest.warns(DeprecationWarning):
+ _ = dataset_class(
+ ann_file='tests/data/locust/test_locust.json',
+ img_prefix='tests/data/locust/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ test_mode=True)
+
+ with pytest.warns(DeprecationWarning):
+ custom_dataset = dataset_class(
+ ann_file='tests/data/locust/test_locust.json',
+ img_prefix='tests/data/locust/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ test_mode=False)
+
+ assert custom_dataset.test_mode is False
+ assert custom_dataset.num_images == 2
+ _ = custom_dataset[0]
+
+ outputs = convert_db_to_output(custom_dataset.db)
+ with tempfile.TemporaryDirectory() as tmpdir:
+ infos = custom_dataset.evaluate(outputs, tmpdir, ['PCK'])
+ assert_almost_equal(infos['PCK'], 1.0)
+
+ with pytest.raises(KeyError):
+ infos = custom_dataset.evaluate(outputs, tmpdir, 'mAP')
+
+
+def test_animal_zebra_dataset_compatibility():
+ dataset = 'AnimalZebraDataset'
+ dataset_class = DATASETS.get(dataset)
+
+ channel_cfg = dict(
+ num_output_channels=9,
+ dataset_joints=9,
+ dataset_channel=[
+ [0, 1, 2, 3, 4, 5, 6, 7, 8],
+ ],
+ inference_channel=[0, 1, 2, 3, 4, 5, 6, 7, 8])
+
+ data_cfg = dict(
+ image_size=[160, 160],
+ heatmap_size=[40, 40],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'])
+
+ # Test
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ with pytest.warns(DeprecationWarning):
+ _ = dataset_class(
+ ann_file='tests/data/zebra/test_zebra.json',
+ img_prefix='tests/data/zebra/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ test_mode=True)
+
+ with pytest.warns(DeprecationWarning):
+ custom_dataset = dataset_class(
+ ann_file='tests/data/zebra/test_zebra.json',
+ img_prefix='tests/data/zebra/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ test_mode=False)
+
+ assert custom_dataset.test_mode is False
+ assert custom_dataset.num_images == 2
+ _ = custom_dataset[0]
+
+ outputs = convert_db_to_output(custom_dataset.db)
+ with tempfile.TemporaryDirectory() as tmpdir:
+ infos = custom_dataset.evaluate(outputs, tmpdir, ['PCK'])
+ assert_almost_equal(infos['PCK'], 1.0)
+
+ with pytest.raises(KeyError):
+ infos = custom_dataset.evaluate(outputs, tmpdir, 'mAP')
+
+
+def test_animal_ATRW_dataset_compatibility():
+ dataset = 'AnimalATRWDataset'
+ dataset_class = DATASETS.get(dataset)
+
+ channel_cfg = dict(
+ num_output_channels=15,
+ dataset_joints=15,
+ dataset_channel=[
+ [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14],
+ ],
+ inference_channel=[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14])
+
+ data_cfg = dict(
+ image_size=[256, 256],
+ heatmap_size=[64, 64],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'],
+ soft_nms=False,
+ nms_thr=1.0,
+ oks_thr=0.9,
+ vis_thr=0.2,
+ use_gt_bbox=True,
+ det_bbox_thr=0.0,
+ bbox_file='',
+ )
+
+ # Test
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ with pytest.warns(DeprecationWarning):
+ _ = dataset_class(
+ ann_file='tests/data/atrw/test_atrw.json',
+ img_prefix='tests/data/atrw/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ test_mode=True)
+
+ with pytest.warns(DeprecationWarning):
+ custom_dataset = dataset_class(
+ ann_file='tests/data/atrw/test_atrw.json',
+ img_prefix='tests/data/atrw/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ test_mode=False)
+
+ assert custom_dataset.test_mode is False
+ assert custom_dataset.num_images == 2
+ _ = custom_dataset[0]
+
+ outputs = convert_db_to_output(custom_dataset.db)
+ with tempfile.TemporaryDirectory() as tmpdir:
+ infos = custom_dataset.evaluate(outputs, tmpdir, 'mAP')
+ assert_almost_equal(infos['AP'], 1.0)
+
+ with pytest.raises(KeyError):
+ infos = custom_dataset.evaluate(outputs, tmpdir, ['PCK'])
+
+
+def test_animal_Macaque_dataset_compatibility():
+ dataset = 'AnimalMacaqueDataset'
+ dataset_class = DATASETS.get(dataset)
+
+ channel_cfg = dict(
+ num_output_channels=17,
+ dataset_joints=17,
+ dataset_channel=[
+ [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16],
+ ],
+ inference_channel=[
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16
+ ])
+
+ data_cfg = dict(
+ image_size=[192, 256],
+ heatmap_size=[48, 64],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'],
+ soft_nms=False,
+ nms_thr=1.0,
+ oks_thr=0.9,
+ vis_thr=0.2,
+ use_gt_bbox=True,
+ det_bbox_thr=0.0,
+ bbox_file='',
+ )
+
+ # Test
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ with pytest.warns(DeprecationWarning):
+ _ = dataset_class(
+ ann_file='tests/data/macaque/test_macaque.json',
+ img_prefix='tests/data/macaque/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ test_mode=True)
+ with pytest.warns(DeprecationWarning):
+ custom_dataset = dataset_class(
+ ann_file='tests/data/macaque/test_macaque.json',
+ img_prefix='tests/data/macaque/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ test_mode=False)
+
+ assert custom_dataset.test_mode is False
+ assert custom_dataset.num_images == 2
+ _ = custom_dataset[0]
+
+ outputs = convert_db_to_output(custom_dataset.db)
+ with tempfile.TemporaryDirectory() as tmpdir:
+ infos = custom_dataset.evaluate(outputs, tmpdir, 'mAP')
+ assert_almost_equal(infos['AP'], 1.0)
+
+ with pytest.raises(KeyError):
+ infos = custom_dataset.evaluate(outputs, tmpdir, ['PCK'])
+
+
+def test_animalpose_dataset_compatibility():
+ dataset = 'AnimalPoseDataset'
+ dataset_class = DATASETS.get(dataset)
+
+ channel_cfg = dict(
+ num_output_channels=20,
+ dataset_joints=20,
+ dataset_channel=[
+ [
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17,
+ 18, 19
+ ],
+ ],
+ inference_channel=[
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18,
+ 19
+ ])
+
+ data_cfg = dict(
+ image_size=[256, 256],
+ heatmap_size=[64, 64],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'],
+ soft_nms=False,
+ nms_thr=1.0,
+ oks_thr=0.9,
+ vis_thr=0.2,
+ use_gt_bbox=True,
+ det_bbox_thr=0.0,
+ bbox_file='',
+ )
+
+ # Test
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ with pytest.warns(DeprecationWarning):
+ _ = dataset_class(
+ ann_file='tests/data/animalpose/test_animalpose.json',
+ img_prefix='tests/data/animalpose/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ test_mode=True)
+
+ with pytest.warns(DeprecationWarning):
+ custom_dataset = dataset_class(
+ ann_file='tests/data/animalpose/test_animalpose.json',
+ img_prefix='tests/data/animalpose/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ test_mode=False)
+
+ assert custom_dataset.test_mode is False
+ assert custom_dataset.num_images == 2
+ _ = custom_dataset[0]
+
+ outputs = convert_db_to_output(custom_dataset.db)
+ with tempfile.TemporaryDirectory() as tmpdir:
+ infos = custom_dataset.evaluate(outputs, tmpdir, 'mAP')
+ assert_almost_equal(infos['AP'], 1.0)
+
+ with pytest.raises(KeyError):
+ infos = custom_dataset.evaluate(outputs, tmpdir, ['PCK'])
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_backward_compatibility/test_dataset_info_compatibility/test_body3d_dataset_compatibility.py b/engine/pose_estimation/third-party/ViTPose/tests/test_backward_compatibility/test_dataset_info_compatibility/test_body3d_dataset_compatibility.py
new file mode 100644
index 0000000..a7e4b71
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_backward_compatibility/test_dataset_info_compatibility/test_body3d_dataset_compatibility.py
@@ -0,0 +1,266 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import tempfile
+
+import numpy as np
+import pytest
+
+from mmpose.datasets import DATASETS
+from mmpose.datasets.builder import build_dataset
+
+
+def test_body3d_h36m_dataset_compatibility():
+ # Test Human3.6M dataset
+ dataset = 'Body3DH36MDataset'
+ dataset_class = DATASETS.get(dataset)
+
+ # test single-frame input
+ data_cfg = dict(
+ num_joints=17,
+ seq_len=1,
+ seq_frame_interval=1,
+ joint_2d_src='pipeline',
+ joint_2d_det_file=None,
+ causal=False,
+ need_camera_param=True,
+ camera_param_file='tests/data/h36m/cameras.pkl')
+
+ with pytest.warns(DeprecationWarning):
+ _ = dataset_class(
+ ann_file='tests/data/h36m/test_h36m_body3d.npz',
+ img_prefix='tests/data/h36m',
+ data_cfg=data_cfg,
+ pipeline=[],
+ test_mode=False)
+
+ with pytest.warns(DeprecationWarning):
+ custom_dataset = dataset_class(
+ ann_file='tests/data/h36m/test_h36m_body3d.npz',
+ img_prefix='tests/data/h36m',
+ data_cfg=data_cfg,
+ pipeline=[],
+ test_mode=True)
+
+ assert custom_dataset.test_mode is True
+ _ = custom_dataset[0]
+
+ with tempfile.TemporaryDirectory() as tmpdir:
+ outputs = []
+ for result in custom_dataset:
+ outputs.append({
+ 'preds': result['target'][None, ...],
+ 'target_image_paths': [result['target_image_path']],
+ })
+
+ metrics = ['mpjpe', 'p-mpjpe', 'n-mpjpe']
+ infos = custom_dataset.evaluate(outputs, tmpdir, metrics)
+
+ np.testing.assert_almost_equal(infos['MPJPE'], 0.0)
+ np.testing.assert_almost_equal(infos['P-MPJPE'], 0.0)
+ np.testing.assert_almost_equal(infos['N-MPJPE'], 0.0)
+
+ # test multi-frame input with joint_2d_src = 'detection'
+ data_cfg = dict(
+ num_joints=17,
+ seq_len=27,
+ seq_frame_interval=1,
+ causal=True,
+ temporal_padding=True,
+ joint_2d_src='detection',
+ joint_2d_det_file='tests/data/h36m/test_h36m_2d_detection.npy',
+ need_camera_param=True,
+ camera_param_file='tests/data/h36m/cameras.pkl')
+
+ with pytest.warns(DeprecationWarning):
+ _ = dataset_class(
+ ann_file='tests/data/h36m/test_h36m_body3d.npz',
+ img_prefix='tests/data/h36m',
+ data_cfg=data_cfg,
+ pipeline=[],
+ test_mode=False)
+
+ with pytest.warns(DeprecationWarning):
+ custom_dataset = dataset_class(
+ ann_file='tests/data/h36m/test_h36m_body3d.npz',
+ img_prefix='tests/data/h36m',
+ data_cfg=data_cfg,
+ pipeline=[],
+ test_mode=True)
+
+ assert custom_dataset.test_mode is True
+ _ = custom_dataset[0]
+
+ with tempfile.TemporaryDirectory() as tmpdir:
+ outputs = []
+ for result in custom_dataset:
+ outputs.append({
+ 'preds': result['target'][None, ...],
+ 'target_image_paths': [result['target_image_path']],
+ })
+
+ metrics = ['mpjpe', 'p-mpjpe', 'n-mpjpe']
+ infos = custom_dataset.evaluate(outputs, tmpdir, metrics)
+
+ np.testing.assert_almost_equal(infos['MPJPE'], 0.0)
+ np.testing.assert_almost_equal(infos['P-MPJPE'], 0.0)
+ np.testing.assert_almost_equal(infos['N-MPJPE'], 0.0)
+
+
+def test_body3d_semi_supervision_dataset_compatibility():
+ # Test Body3d Semi-supervision Dataset
+
+ # load labeled dataset
+ labeled_data_cfg = dict(
+ num_joints=17,
+ seq_len=27,
+ seq_frame_interval=1,
+ causall=False,
+ temporal_padding=True,
+ joint_2d_src='gt',
+ subset=1,
+ subjects=['S1'],
+ need_camera_param=True,
+ camera_param_file='tests/data/h36m/cameras.pkl')
+ labeled_dataset = dict(
+ type='Body3DH36MDataset',
+ ann_file='tests/data/h36m/test_h36m_body3d.npz',
+ img_prefix='tests/data/h36m',
+ data_cfg=labeled_data_cfg,
+ pipeline=[])
+
+ # load unlabled data
+ unlabeled_data_cfg = dict(
+ num_joints=17,
+ seq_len=27,
+ seq_frame_interval=1,
+ causal=False,
+ temporal_padding=True,
+ joint_2d_src='gt',
+ subjects=['S5', 'S7', 'S8'],
+ need_camera_param=True,
+ camera_param_file='tests/data/h36m/cameras.pkl',
+ need_2d_label=True)
+ unlabeled_dataset = dict(
+ type='Body3DH36MDataset',
+ ann_file='tests/data/h36m/test_h36m_body3d.npz',
+ img_prefix='tests/data/h36m',
+ data_cfg=unlabeled_data_cfg,
+ pipeline=[
+ dict(
+ type='Collect',
+ keys=[('input_2d', 'unlabeled_input')],
+ meta_name='metas',
+ meta_keys=[])
+ ])
+
+ # combine labeled and unlabeled dataset to form a new dataset
+ dataset = 'Body3DSemiSupervisionDataset'
+ dataset_class = DATASETS.get(dataset)
+ with pytest.warns(DeprecationWarning):
+ custom_dataset = dataset_class(labeled_dataset, unlabeled_dataset)
+ item = custom_dataset[0]
+ assert 'unlabeled_input' in item.keys()
+
+ unlabeled_dataset = build_dataset(unlabeled_dataset)
+ assert len(unlabeled_dataset) == len(custom_dataset)
+
+
+def test_body3d_mpi_inf_3dhp_dataset_compatibility():
+ # Test MPI-INF-3DHP dataset
+ dataset = 'Body3DMpiInf3dhpDataset'
+ dataset_class = DATASETS.get(dataset)
+
+ # Test single-frame input on trainset
+ single_frame_train_data_cfg = dict(
+ num_joints=17,
+ seq_len=1,
+ seq_frame_interval=1,
+ joint_2d_src='pipeline',
+ joint_2d_det_file=None,
+ causal=False,
+ need_camera_param=True,
+ camera_param_file='tests/data/mpi_inf_3dhp/cameras_train.pkl')
+
+ # Test single-frame input on testset
+ single_frame_test_data_cfg = dict(
+ num_joints=17,
+ seq_len=1,
+ seq_frame_interval=1,
+ joint_2d_src='gt',
+ joint_2d_det_file=None,
+ causal=False,
+ need_camera_param=True,
+ camera_param_file='tests/data/mpi_inf_3dhp/cameras_test.pkl')
+
+ # Test multi-frame input on trainset
+ multi_frame_train_data_cfg = dict(
+ num_joints=17,
+ seq_len=27,
+ seq_frame_interval=1,
+ joint_2d_src='gt',
+ joint_2d_det_file=None,
+ causal=True,
+ temporal_padding=True,
+ need_camera_param=True,
+ camera_param_file='tests/data/mpi_inf_3dhp/cameras_train.pkl')
+
+ # Test multi-frame input on testset
+ multi_frame_test_data_cfg = dict(
+ num_joints=17,
+ seq_len=27,
+ seq_frame_interval=1,
+ joint_2d_src='pipeline',
+ joint_2d_det_file=None,
+ causal=False,
+ temporal_padding=True,
+ need_camera_param=True,
+ camera_param_file='tests/data/mpi_inf_3dhp/cameras_test.pkl')
+
+ ann_files = [
+ 'tests/data/mpi_inf_3dhp/test_3dhp_train.npz',
+ 'tests/data/mpi_inf_3dhp/test_3dhp_test.npz'
+ ] * 2
+ data_cfgs = [
+ single_frame_train_data_cfg, single_frame_test_data_cfg,
+ multi_frame_train_data_cfg, multi_frame_test_data_cfg
+ ]
+
+ for ann_file, data_cfg in zip(ann_files, data_cfgs):
+ with pytest.warns(DeprecationWarning):
+ _ = dataset_class(
+ ann_file=ann_file,
+ img_prefix='tests/data/mpi_inf_3dhp',
+ data_cfg=data_cfg,
+ pipeline=[],
+ test_mode=False)
+
+ with pytest.warns(DeprecationWarning):
+ custom_dataset = dataset_class(
+ ann_file=ann_file,
+ img_prefix='tests/data/mpi_inf_3dhp',
+ data_cfg=data_cfg,
+ pipeline=[],
+ test_mode=True)
+
+ assert custom_dataset.test_mode is True
+ _ = custom_dataset[0]
+
+ with tempfile.TemporaryDirectory() as tmpdir:
+ outputs = []
+ for result in custom_dataset:
+ outputs.append({
+ 'preds':
+ result['target'][None, ...],
+ 'target_image_paths': [result['target_image_path']],
+ })
+
+ metrics = [
+ 'mpjpe', 'p-mpjpe', '3dpck', 'p-3dpck', '3dauc', 'p-3dauc'
+ ]
+ infos = custom_dataset.evaluate(outputs, tmpdir, metrics)
+
+ np.testing.assert_almost_equal(infos['MPJPE'], 0.0)
+ np.testing.assert_almost_equal(infos['P-MPJPE'], 0.0)
+ np.testing.assert_almost_equal(infos['3DPCK'], 100.)
+ np.testing.assert_almost_equal(infos['P-3DPCK'], 100.)
+ np.testing.assert_almost_equal(infos['3DAUC'], 30 / 31 * 100)
+ np.testing.assert_almost_equal(infos['P-3DAUC'], 30 / 31 * 100)
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_backward_compatibility/test_dataset_info_compatibility/test_bottom_up_dataset_compatibility.py b/engine/pose_estimation/third-party/ViTPose/tests/test_backward_compatibility/test_dataset_info_compatibility/test_bottom_up_dataset_compatibility.py
new file mode 100644
index 0000000..366fcfe
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_backward_compatibility/test_dataset_info_compatibility/test_bottom_up_dataset_compatibility.py
@@ -0,0 +1,325 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import tempfile
+
+import numpy as np
+import pytest
+from numpy.testing import assert_almost_equal
+
+from mmpose.datasets import DATASETS
+
+
+def convert_coco_to_output(coco, is_wholebody=False):
+ outputs = []
+ for img_id in coco.getImgIds():
+ preds = []
+ scores = []
+ image = coco.imgs[img_id]
+ ann_ids = coco.getAnnIds(img_id)
+ for ann_id in ann_ids:
+ obj = coco.anns[ann_id]
+ if is_wholebody:
+ keypoints = np.array(obj['keypoints'] + obj['foot_kpts'] +
+ obj['face_kpts'] + obj['lefthand_kpts'] +
+ obj['righthand_kpts']).reshape(-1, 3)
+ else:
+ keypoints = np.array(obj['keypoints']).reshape((-1, 3))
+ K = keypoints.shape[0]
+ if sum(keypoints[:, 2]) == 0:
+ continue
+ preds.append(
+ np.concatenate((keypoints[:, :2], np.ones(
+ [K, 1]), np.ones([K, 1]) * ann_id),
+ axis=1))
+ scores.append(1)
+ image_paths = []
+ image_paths.append(image['file_name'])
+
+ output = {}
+ output['preds'] = np.stack(preds)
+ output['scores'] = scores
+ output['image_paths'] = image_paths
+ output['output_heatmap'] = None
+
+ outputs.append(output)
+
+ return outputs
+
+
+def test_bottom_up_COCO_dataset_compatibility():
+ dataset = 'BottomUpCocoDataset'
+ # test COCO datasets
+ dataset_class = DATASETS.get(dataset)
+
+ channel_cfg = dict(
+ dataset_joints=17,
+ dataset_channel=[
+ [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17],
+ ],
+ inference_channel=[
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17
+ ])
+
+ data_cfg = dict(
+ image_size=512,
+ base_size=256,
+ base_sigma=2,
+ heatmap_size=[128, 256],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'],
+ num_scales=2,
+ scale_aware_sigma=False,
+ use_nms=True)
+
+ with pytest.warns(DeprecationWarning):
+ _ = dataset_class(
+ ann_file='tests/data/coco/test_coco.json',
+ img_prefix='tests/data/coco/',
+ data_cfg=data_cfg,
+ pipeline=[],
+ test_mode=False)
+
+ with pytest.warns(DeprecationWarning):
+ custom_dataset = dataset_class(
+ ann_file='tests/data/coco/test_coco.json',
+ img_prefix='tests/data/coco/',
+ data_cfg=data_cfg,
+ pipeline=[],
+ test_mode=True)
+
+ assert custom_dataset.num_images == 4
+ _ = custom_dataset[0]
+ assert custom_dataset.dataset_name == 'coco'
+
+ outputs = convert_coco_to_output(custom_dataset.coco)
+ with tempfile.TemporaryDirectory() as tmpdir:
+ infos = custom_dataset.evaluate(outputs, tmpdir, 'mAP')
+ assert_almost_equal(infos['AP'], 1.0)
+
+ with pytest.raises(KeyError):
+ _ = custom_dataset.evaluate(outputs, tmpdir, 'PCK')
+
+
+def test_bottom_up_CrowdPose_dataset_compatibility():
+ dataset = 'BottomUpCrowdPoseDataset'
+ # test CrowdPose datasets
+ dataset_class = DATASETS.get(dataset)
+
+ channel_cfg = dict(
+ num_output_channels=14,
+ dataset_joints=14,
+ dataset_channel=[
+ [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13],
+ ],
+ inference_channel=[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13])
+
+ data_cfg = dict(
+ image_size=512,
+ base_size=256,
+ base_sigma=2,
+ heatmap_size=[128, 256],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'],
+ num_scales=2,
+ scale_aware_sigma=False)
+
+ with pytest.warns(DeprecationWarning):
+ _ = dataset_class(
+ ann_file='tests/data/crowdpose/test_crowdpose.json',
+ img_prefix='tests/data/crowdpose/',
+ data_cfg=data_cfg,
+ pipeline=[],
+ test_mode=False)
+
+ with pytest.warns(DeprecationWarning):
+ custom_dataset = dataset_class(
+ ann_file='tests/data/crowdpose/test_crowdpose.json',
+ img_prefix='tests/data/crowdpose/',
+ data_cfg=data_cfg,
+ pipeline=[],
+ test_mode=True)
+
+ image_id = 103319
+ assert image_id in custom_dataset.img_ids
+ assert len(custom_dataset.img_ids) == 2
+ _ = custom_dataset[0]
+ assert custom_dataset.dataset_name == 'crowdpose'
+
+ outputs = convert_coco_to_output(custom_dataset.coco)
+ with tempfile.TemporaryDirectory() as tmpdir:
+ infos = custom_dataset.evaluate(outputs, tmpdir, 'mAP')
+ assert_almost_equal(infos['AP'], 1.0)
+
+ with pytest.raises(KeyError):
+ _ = custom_dataset.evaluate(outputs, tmpdir, 'PCK')
+
+
+def test_bottom_up_MHP_dataset_compatibility():
+ dataset = 'BottomUpMhpDataset'
+ # test MHP datasets
+ dataset_class = DATASETS.get(dataset)
+
+ channel_cfg = dict(
+ dataset_joints=16,
+ dataset_channel=[
+ [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15],
+ ],
+ inference_channel=[
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15
+ ])
+
+ data_cfg = dict(
+ image_size=512,
+ base_size=256,
+ base_sigma=2,
+ heatmap_size=[128],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'],
+ num_scales=1,
+ scale_aware_sigma=False,
+ )
+
+ with pytest.warns(DeprecationWarning):
+ _ = dataset_class(
+ ann_file='tests/data/mhp/test_mhp.json',
+ img_prefix='tests/data/mhp/',
+ data_cfg=data_cfg,
+ pipeline=[],
+ test_mode=False)
+
+ with pytest.warns(DeprecationWarning):
+ custom_dataset = dataset_class(
+ ann_file='tests/data/mhp/test_mhp.json',
+ img_prefix='tests/data/mhp/',
+ data_cfg=data_cfg,
+ pipeline=[],
+ test_mode=True)
+
+ image_id = 2889
+ assert image_id in custom_dataset.img_ids
+ assert len(custom_dataset.img_ids) == 2
+ _ = custom_dataset[0]
+ assert custom_dataset.dataset_name == 'mhp'
+
+ outputs = convert_coco_to_output(custom_dataset.coco)
+ with tempfile.TemporaryDirectory() as tmpdir:
+ infos = custom_dataset.evaluate(outputs, tmpdir, 'mAP')
+ assert_almost_equal(infos['AP'], 1.0)
+
+ with pytest.raises(KeyError):
+ _ = custom_dataset.evaluate(outputs, tmpdir, 'PCK')
+
+
+def test_bottom_up_AIC_dataset_compatibility():
+ dataset = 'BottomUpAicDataset'
+ # test MHP datasets
+ dataset_class = DATASETS.get(dataset)
+
+ channel_cfg = dict(
+ num_output_channels=14,
+ dataset_joints=14,
+ dataset_channel=[
+ [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13],
+ ],
+ inference_channel=[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13])
+
+ data_cfg = dict(
+ image_size=512,
+ base_size=256,
+ base_sigma=2,
+ heatmap_size=[128],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'],
+ num_scales=1,
+ scale_aware_sigma=False,
+ )
+
+ with pytest.warns(DeprecationWarning):
+ _ = dataset_class(
+ ann_file='tests/data/aic/test_aic.json',
+ img_prefix='tests/data/aic/',
+ data_cfg=data_cfg,
+ pipeline=[],
+ test_mode=False)
+
+ with pytest.warns(DeprecationWarning):
+ custom_dataset = dataset_class(
+ ann_file='tests/data/aic/test_aic.json',
+ img_prefix='tests/data/aic/',
+ data_cfg=data_cfg,
+ pipeline=[],
+ test_mode=True)
+
+ image_id = 1
+ assert image_id in custom_dataset.img_ids
+ assert len(custom_dataset.img_ids) == 3
+ _ = custom_dataset[0]
+
+ outputs = convert_coco_to_output(custom_dataset.coco)
+ with tempfile.TemporaryDirectory() as tmpdir:
+ infos = custom_dataset.evaluate(outputs, tmpdir, 'mAP')
+ assert_almost_equal(infos['AP'], 1.0)
+
+ with pytest.raises(KeyError):
+ _ = custom_dataset.evaluate(outputs, tmpdir, 'PCK')
+
+
+def test_bottom_up_COCO_wholebody_dataset_compatibility():
+ dataset = 'BottomUpCocoWholeBodyDataset'
+ # test COCO-wholebody datasets
+ dataset_class = DATASETS.get(dataset)
+
+ channel_cfg = dict(
+ num_output_channels=133,
+ dataset_joints=133,
+ dataset_channel=[
+ list(range(133)),
+ ],
+ inference_channel=list(range(133)))
+
+ data_cfg = dict(
+ image_size=512,
+ base_size=256,
+ base_sigma=2,
+ heatmap_size=[128, 256],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'],
+ num_scales=2,
+ scale_aware_sigma=False,
+ )
+
+ with pytest.warns(DeprecationWarning):
+ _ = dataset_class(
+ ann_file='tests/data/coco/test_coco_wholebody.json',
+ img_prefix='tests/data/coco/',
+ data_cfg=data_cfg,
+ pipeline=[],
+ test_mode=False)
+
+ with pytest.warns(DeprecationWarning):
+ custom_dataset = dataset_class(
+ ann_file='tests/data/coco/test_coco_wholebody.json',
+ img_prefix='tests/data/coco/',
+ data_cfg=data_cfg,
+ pipeline=[],
+ test_mode=True)
+
+ assert custom_dataset.test_mode is True
+ assert custom_dataset.dataset_name == 'coco_wholebody'
+
+ image_id = 785
+ assert image_id in custom_dataset.img_ids
+ assert len(custom_dataset.img_ids) == 4
+ _ = custom_dataset[0]
+
+ outputs = convert_coco_to_output(custom_dataset.coco, is_wholebody=True)
+ with tempfile.TemporaryDirectory() as tmpdir:
+ infos = custom_dataset.evaluate(outputs, tmpdir, 'mAP')
+ assert_almost_equal(infos['AP'], 1.0)
+
+ with pytest.raises(KeyError):
+ _ = custom_dataset.evaluate(outputs, tmpdir, 'PCK')
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_backward_compatibility/test_dataset_info_compatibility/test_deprecated_dataset_base.py b/engine/pose_estimation/third-party/ViTPose/tests/test_backward_compatibility/test_dataset_info_compatibility/test_deprecated_dataset_base.py
new file mode 100644
index 0000000..c5aad98
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_backward_compatibility/test_dataset_info_compatibility/test_deprecated_dataset_base.py
@@ -0,0 +1,28 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import pytest
+
+from mmpose.datasets.datasets.animal.animal_base_dataset import \
+ AnimalBaseDataset
+from mmpose.datasets.datasets.body3d.body3d_base_dataset import \
+ Body3DBaseDataset
+from mmpose.datasets.datasets.bottom_up.bottom_up_base_dataset import \
+ BottomUpBaseDataset
+from mmpose.datasets.datasets.face.face_base_dataset import FaceBaseDataset
+from mmpose.datasets.datasets.fashion.fashion_base_dataset import \
+ FashionBaseDataset
+from mmpose.datasets.datasets.hand.hand_base_dataset import HandBaseDataset
+from mmpose.datasets.datasets.top_down.topdown_base_dataset import \
+ TopDownBaseDataset
+
+
+@pytest.mark.parametrize('BaseDataset',
+ (AnimalBaseDataset, BottomUpBaseDataset,
+ FaceBaseDataset, FashionBaseDataset, HandBaseDataset,
+ TopDownBaseDataset, Body3DBaseDataset))
+def test_dataset_base_class(BaseDataset):
+ with pytest.raises(ImportError):
+
+ class Dataset(BaseDataset):
+ pass
+
+ _ = Dataset()
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_backward_compatibility/test_dataset_info_compatibility/test_face_dataset_compatibility.py b/engine/pose_estimation/third-party/ViTPose/tests/test_backward_compatibility/test_dataset_info_compatibility/test_face_dataset_compatibility.py
new file mode 100644
index 0000000..056845b
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_backward_compatibility/test_dataset_info_compatibility/test_face_dataset_compatibility.py
@@ -0,0 +1,170 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import copy
+import tempfile
+from unittest.mock import MagicMock
+
+import pytest
+from numpy.testing import assert_almost_equal
+
+from mmpose.datasets import DATASETS
+from tests.utils.data_utils import convert_db_to_output
+
+
+def test_face_300W_dataset_compatibility():
+ dataset = 'Face300WDataset'
+ # test Face 300W datasets
+ dataset_class = DATASETS.get(dataset)
+ dataset_class.load_annotations = MagicMock()
+ dataset_class.coco = MagicMock()
+
+ channel_cfg = dict(
+ num_output_channels=68,
+ dataset_joints=68,
+ dataset_channel=[
+ list(range(68)),
+ ],
+ inference_channel=list(range(68)))
+
+ data_cfg = dict(
+ image_size=[256, 256],
+ heatmap_size=[64, 64],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'])
+ # Test
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ with pytest.warns(DeprecationWarning):
+ _ = dataset_class(
+ ann_file='tests/data/300w/test_300w.json',
+ img_prefix='tests/data/300w/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ test_mode=True)
+
+ with pytest.warns(DeprecationWarning):
+ custom_dataset = dataset_class(
+ ann_file='tests/data/300w/test_300w.json',
+ img_prefix='tests/data/300w/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ test_mode=False)
+
+ assert custom_dataset.test_mode is False
+ assert custom_dataset.num_images == 2
+ _ = custom_dataset[0]
+
+ outputs = convert_db_to_output(custom_dataset.db)
+ with tempfile.TemporaryDirectory() as tmpdir:
+ infos = custom_dataset.evaluate(outputs, tmpdir, ['NME'])
+ assert_almost_equal(infos['NME'], 0.0)
+
+ with pytest.raises(KeyError):
+ _ = custom_dataset.evaluate(outputs, tmpdir, 'mAP')
+
+
+def test_face_AFLW_dataset_compatibility():
+ dataset = 'FaceAFLWDataset'
+ # test Face AFLW datasets
+ dataset_class = DATASETS.get(dataset)
+ dataset_class.load_annotations = MagicMock()
+ dataset_class.coco = MagicMock()
+
+ channel_cfg = dict(
+ num_output_channels=19,
+ dataset_joints=19,
+ dataset_channel=[
+ list(range(19)),
+ ],
+ inference_channel=list(range(19)))
+
+ data_cfg = dict(
+ image_size=[256, 256],
+ heatmap_size=[64, 64],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'])
+ # Test
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ with pytest.warns(DeprecationWarning):
+ _ = dataset_class(
+ ann_file='tests/data/aflw/test_aflw.json',
+ img_prefix='tests/data/aflw/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ test_mode=True)
+
+ with pytest.warns(DeprecationWarning):
+ custom_dataset = dataset_class(
+ ann_file='tests/data/aflw/test_aflw.json',
+ img_prefix='tests/data/aflw/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ test_mode=False)
+
+ assert custom_dataset.test_mode is False
+ assert custom_dataset.num_images == 2
+ _ = custom_dataset[0]
+
+ outputs = convert_db_to_output(custom_dataset.db)
+ with tempfile.TemporaryDirectory() as tmpdir:
+ infos = custom_dataset.evaluate(outputs, tmpdir, ['NME'])
+ assert_almost_equal(infos['NME'], 0.0)
+
+ with pytest.raises(KeyError):
+ _ = custom_dataset.evaluate(outputs, tmpdir, 'mAP')
+
+
+def test_face_WFLW_dataset_compatibility():
+ dataset = 'FaceWFLWDataset'
+ # test Face WFLW datasets
+ dataset_class = DATASETS.get(dataset)
+ dataset_class.load_annotations = MagicMock()
+ dataset_class.coco = MagicMock()
+
+ channel_cfg = dict(
+ num_output_channels=98,
+ dataset_joints=98,
+ dataset_channel=[
+ list(range(98)),
+ ],
+ inference_channel=list(range(98)))
+
+ data_cfg = dict(
+ image_size=[256, 256],
+ heatmap_size=[64, 64],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'])
+ # Test
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ with pytest.warns(DeprecationWarning):
+ _ = dataset_class(
+ ann_file='tests/data/wflw/test_wflw.json',
+ img_prefix='tests/data/wflw/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ test_mode=True)
+
+ with pytest.warns(DeprecationWarning):
+ custom_dataset = dataset_class(
+ ann_file='tests/data/wflw/test_wflw.json',
+ img_prefix='tests/data/wflw/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ test_mode=False)
+
+ assert custom_dataset.test_mode is False
+ assert custom_dataset.num_images == 2
+ _ = custom_dataset[0]
+
+ outputs = convert_db_to_output(custom_dataset.db)
+
+ with tempfile.TemporaryDirectory() as tmpdir:
+ infos = custom_dataset.evaluate(outputs, tmpdir, ['NME'])
+ assert_almost_equal(infos['NME'], 0.0)
+
+ with pytest.raises(KeyError):
+ _ = custom_dataset.evaluate(outputs, tmpdir, 'mAP')
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_backward_compatibility/test_dataset_info_compatibility/test_fashion_dataset_compatibility.py b/engine/pose_estimation/third-party/ViTPose/tests/test_backward_compatibility/test_dataset_info_compatibility/test_fashion_dataset_compatibility.py
new file mode 100644
index 0000000..b647156
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_backward_compatibility/test_dataset_info_compatibility/test_fashion_dataset_compatibility.py
@@ -0,0 +1,69 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import tempfile
+from unittest.mock import MagicMock
+
+import pytest
+from numpy.testing import assert_almost_equal
+
+from mmpose.datasets import DATASETS
+from tests.utils.data_utils import convert_db_to_output
+
+
+def test_deepfashion_dataset_compatibility():
+ dataset = 'DeepFashionDataset'
+ # test JHMDB datasets
+ dataset_class = DATASETS.get(dataset)
+ dataset_class.load_annotations = MagicMock()
+ dataset_class.coco = MagicMock()
+
+ channel_cfg = dict(
+ num_output_channels=8,
+ dataset_joints=8,
+ dataset_channel=[
+ [0, 1, 2, 3, 4, 5, 6, 7],
+ ],
+ inference_channel=[0, 1, 2, 3, 4, 5, 6, 7])
+
+ data_cfg = dict(
+ image_size=[192, 256],
+ heatmap_size=[48, 64],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'],
+ soft_nms=False,
+ nms_thr=1.0,
+ oks_thr=0.9,
+ vis_thr=0.2,
+ use_gt_bbox=True,
+ det_bbox_thr=0.0,
+ image_thr=0.0,
+ bbox_file='')
+
+ # Test gt bbox
+ with pytest.warns(DeprecationWarning):
+ custom_dataset = dataset_class(
+ ann_file='tests/data/fld/test_fld.json',
+ img_prefix='tests/data/fld/',
+ subset='full',
+ data_cfg=data_cfg,
+ pipeline=[],
+ test_mode=True)
+
+ assert custom_dataset.test_mode is True
+ assert custom_dataset.dataset_name == 'deepfashion_full'
+
+ image_id = 128
+ assert image_id in custom_dataset.img_ids
+ assert len(custom_dataset.img_ids) == 2
+ _ = custom_dataset[0]
+
+ outputs = convert_db_to_output(custom_dataset.db)
+ with tempfile.TemporaryDirectory() as tmpdir:
+ infos = custom_dataset.evaluate(outputs, tmpdir, ['PCK', 'EPE', 'AUC'])
+ assert_almost_equal(infos['PCK'], 1.0)
+ assert_almost_equal(infos['AUC'], 0.95)
+ assert_almost_equal(infos['EPE'], 0.0)
+
+ with pytest.raises(KeyError):
+ infos = custom_dataset.evaluate(outputs, tmpdir, 'mAP')
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_backward_compatibility/test_dataset_info_compatibility/test_hand_dataset_compatibility.py b/engine/pose_estimation/third-party/ViTPose/tests/test_backward_compatibility/test_dataset_info_compatibility/test_hand_dataset_compatibility.py
new file mode 100644
index 0000000..af11f24
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_backward_compatibility/test_dataset_info_compatibility/test_hand_dataset_compatibility.py
@@ -0,0 +1,388 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import copy
+import tempfile
+
+import pytest
+from numpy.testing import assert_almost_equal
+
+from mmpose.datasets import DATASETS
+from tests.utils.data_utils import convert_db_to_output
+
+
+def test_top_down_OneHand10K_dataset_compatibility():
+ dataset = 'OneHand10KDataset'
+ dataset_class = DATASETS.get(dataset)
+
+ channel_cfg = dict(
+ num_output_channels=21,
+ dataset_joints=21,
+ dataset_channel=[
+ [
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17,
+ 18, 19, 20
+ ],
+ ],
+ inference_channel=[
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18,
+ 19, 20
+ ])
+
+ data_cfg = dict(
+ image_size=[256, 256],
+ heatmap_size=[64, 64],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'])
+ # Test
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ with pytest.warns(DeprecationWarning):
+ _ = dataset_class(
+ ann_file='tests/data/onehand10k/test_onehand10k.json',
+ img_prefix='tests/data/onehand10k/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ test_mode=True)
+
+ with pytest.warns(DeprecationWarning):
+ custom_dataset = dataset_class(
+ ann_file='tests/data/onehand10k/test_onehand10k.json',
+ img_prefix='tests/data/onehand10k/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ test_mode=False)
+
+ assert custom_dataset.test_mode is False
+ assert custom_dataset.num_images == 4
+ _ = custom_dataset[0]
+
+ outputs = convert_db_to_output(custom_dataset.db)
+ with tempfile.TemporaryDirectory() as tmpdir:
+ infos = custom_dataset.evaluate(outputs, tmpdir, ['PCK', 'EPE', 'AUC'])
+ assert_almost_equal(infos['PCK'], 1.0)
+ assert_almost_equal(infos['AUC'], 0.95)
+ assert_almost_equal(infos['EPE'], 0.0)
+
+ with pytest.raises(KeyError):
+ infos = custom_dataset.evaluate(outputs, tmpdir, 'mAP')
+
+
+def test_top_down_FreiHand_dataset_compatibility():
+ dataset = 'FreiHandDataset'
+ dataset_class = DATASETS.get(dataset)
+
+ channel_cfg = dict(
+ num_output_channels=21,
+ dataset_joints=21,
+ dataset_channel=[
+ [
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17,
+ 18, 19, 20
+ ],
+ ],
+ inference_channel=[
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18,
+ 19, 20
+ ])
+
+ data_cfg = dict(
+ image_size=[224, 224],
+ heatmap_size=[56, 56],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'])
+ # Test
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ with pytest.warns(DeprecationWarning):
+ _ = dataset_class(
+ ann_file='tests/data/freihand/test_freihand.json',
+ img_prefix='tests/data/freihand/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ test_mode=True)
+
+ with pytest.warns(DeprecationWarning):
+ custom_dataset = dataset_class(
+ ann_file='tests/data/freihand/test_freihand.json',
+ img_prefix='tests/data/freihand/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ test_mode=False)
+
+ assert custom_dataset.test_mode is False
+ assert custom_dataset.num_images == 8
+ _ = custom_dataset[0]
+
+ outputs = convert_db_to_output(custom_dataset.db)
+ with tempfile.TemporaryDirectory() as tmpdir:
+ infos = custom_dataset.evaluate(outputs, tmpdir, ['PCK', 'EPE', 'AUC'])
+ assert_almost_equal(infos['PCK'], 1.0)
+ assert_almost_equal(infos['AUC'], 0.95)
+ assert_almost_equal(infos['EPE'], 0.0)
+
+ with pytest.raises(KeyError):
+ infos = custom_dataset.evaluate(outputs, tmpdir, 'mAP')
+
+
+def test_top_down_RHD_dataset_compatibility():
+ dataset = 'Rhd2DDataset'
+ dataset_class = DATASETS.get(dataset)
+
+ channel_cfg = dict(
+ num_output_channels=21,
+ dataset_joints=21,
+ dataset_channel=[
+ [
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17,
+ 18, 19, 20
+ ],
+ ],
+ inference_channel=[
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18,
+ 19, 20
+ ])
+
+ data_cfg = dict(
+ image_size=[256, 256],
+ heatmap_size=[64, 64],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'])
+ # Test
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ with pytest.warns(DeprecationWarning):
+ _ = dataset_class(
+ ann_file='tests/data/rhd/test_rhd.json',
+ img_prefix='tests/data/rhd/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ test_mode=True)
+
+ with pytest.warns(DeprecationWarning):
+ custom_dataset = dataset_class(
+ ann_file='tests/data/rhd/test_rhd.json',
+ img_prefix='tests/data/rhd/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ test_mode=False)
+
+ assert custom_dataset.test_mode is False
+ assert custom_dataset.num_images == 3
+ _ = custom_dataset[0]
+
+ outputs = convert_db_to_output(custom_dataset.db)
+ with tempfile.TemporaryDirectory() as tmpdir:
+ infos = custom_dataset.evaluate(outputs, tmpdir, ['PCK', 'EPE', 'AUC'])
+ assert_almost_equal(infos['PCK'], 1.0)
+ assert_almost_equal(infos['AUC'], 0.95)
+ assert_almost_equal(infos['EPE'], 0.0)
+
+ with pytest.raises(KeyError):
+ infos = custom_dataset.evaluate(outputs, tmpdir, 'mAP')
+
+
+def test_top_down_Panoptic_dataset_compatibility():
+ dataset = 'PanopticDataset'
+ dataset_class = DATASETS.get(dataset)
+
+ channel_cfg = dict(
+ num_output_channels=21,
+ dataset_joints=21,
+ dataset_channel=[
+ [
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17,
+ 18, 19, 20
+ ],
+ ],
+ inference_channel=[
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18,
+ 19, 20
+ ])
+
+ data_cfg = dict(
+ image_size=[256, 256],
+ heatmap_size=[64, 64],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'])
+ # Test
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ with pytest.warns(DeprecationWarning):
+ _ = dataset_class(
+ ann_file='tests/data/panoptic/test_panoptic.json',
+ img_prefix='tests/data/panoptic/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ test_mode=True)
+
+ with pytest.warns(DeprecationWarning):
+ custom_dataset = dataset_class(
+ ann_file='tests/data/panoptic/test_panoptic.json',
+ img_prefix='tests/data/panoptic/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ test_mode=False)
+
+ assert custom_dataset.test_mode is False
+ assert custom_dataset.num_images == 4
+ _ = custom_dataset[0]
+
+ outputs = convert_db_to_output(custom_dataset.db)
+ with tempfile.TemporaryDirectory() as tmpdir:
+ infos = custom_dataset.evaluate(outputs, tmpdir,
+ ['PCKh', 'EPE', 'AUC'])
+ assert_almost_equal(infos['PCKh'], 1.0)
+ assert_almost_equal(infos['AUC'], 0.95)
+ assert_almost_equal(infos['EPE'], 0.0)
+
+ with pytest.raises(KeyError):
+ infos = custom_dataset.evaluate(outputs, tmpdir, 'mAP')
+
+
+def test_top_down_InterHand2D_dataset_compatibility():
+ dataset = 'InterHand2DDataset'
+ dataset_class = DATASETS.get(dataset)
+
+ channel_cfg = dict(
+ num_output_channels=21,
+ dataset_joints=21,
+ dataset_channel=[
+ [
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17,
+ 18, 19, 20
+ ],
+ ],
+ inference_channel=[
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18,
+ 19, 20
+ ])
+
+ data_cfg = dict(
+ image_size=[256, 256],
+ heatmap_size=[64, 64],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'])
+ # Test
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ with pytest.warns(DeprecationWarning):
+ _ = dataset_class(
+ ann_file='tests/data/interhand2.6m/test_interhand2.6m_data.json',
+ camera_file='tests/data/interhand2.6m/'
+ 'test_interhand2.6m_camera.json',
+ joint_file='tests/data/interhand2.6m/'
+ 'test_interhand2.6m_joint_3d.json',
+ img_prefix='tests/data/interhand2.6m/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ test_mode=True)
+
+ with pytest.warns(DeprecationWarning):
+ custom_dataset = dataset_class(
+ ann_file='tests/data/interhand2.6m/test_interhand2.6m_data.json',
+ camera_file='tests/data/interhand2.6m/'
+ 'test_interhand2.6m_camera.json',
+ joint_file='tests/data/interhand2.6m/'
+ 'test_interhand2.6m_joint_3d.json',
+ img_prefix='tests/data/interhand2.6m/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ test_mode=False)
+
+ assert custom_dataset.test_mode is False
+ assert custom_dataset.num_images == 4
+ assert len(custom_dataset.db) == 6
+
+ _ = custom_dataset[0]
+
+ outputs = convert_db_to_output(custom_dataset.db)
+ with tempfile.TemporaryDirectory() as tmpdir:
+ infos = custom_dataset.evaluate(outputs, tmpdir, ['PCK', 'EPE', 'AUC'])
+ print(infos, flush=True)
+ assert_almost_equal(infos['PCK'], 1.0)
+ assert_almost_equal(infos['AUC'], 0.95)
+ assert_almost_equal(infos['EPE'], 0.0)
+
+ with pytest.raises(KeyError):
+ infos = custom_dataset.evaluate(outputs, tmpdir, 'mAP')
+
+
+def test_top_down_InterHand3D_dataset_compatibility():
+ dataset = 'InterHand3DDataset'
+ dataset_class = DATASETS.get(dataset)
+
+ channel_cfg = dict(
+ num_output_channels=42,
+ dataset_joints=42,
+ dataset_channel=[
+ [
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17,
+ 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33,
+ 34, 35, 36, 37, 38, 39, 40, 41
+ ],
+ ],
+ inference_channel=[
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18,
+ 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35,
+ 36, 37, 38, 39, 40, 41
+ ])
+
+ data_cfg = dict(
+ image_size=[256, 256],
+ heatmap_size=[64, 64, 64],
+ heatmap3d_depth_bound=400.0,
+ heatmap_size_root=64,
+ root_depth_bound=400.0,
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'])
+ # Test
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ with pytest.warns(DeprecationWarning):
+ _ = dataset_class(
+ ann_file='tests/data/interhand2.6m/test_interhand2.6m_data.json',
+ camera_file='tests/data/interhand2.6m/'
+ 'test_interhand2.6m_camera.json',
+ joint_file='tests/data/interhand2.6m/'
+ 'test_interhand2.6m_joint_3d.json',
+ img_prefix='tests/data/interhand2.6m/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ test_mode=True)
+
+ with pytest.warns(DeprecationWarning):
+ custom_dataset = dataset_class(
+ ann_file='tests/data/interhand2.6m/test_interhand2.6m_data.json',
+ camera_file='tests/data/interhand2.6m/'
+ 'test_interhand2.6m_camera.json',
+ joint_file='tests/data/interhand2.6m/'
+ 'test_interhand2.6m_joint_3d.json',
+ img_prefix='tests/data/interhand2.6m/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ test_mode=False)
+
+ assert custom_dataset.test_mode is False
+ assert custom_dataset.num_images == 4
+ assert len(custom_dataset.db) == 4
+
+ _ = custom_dataset[0]
+
+ outputs = convert_db_to_output(
+ custom_dataset.db, keys=['rel_root_depth', 'hand_type'], is_3d=True)
+ with tempfile.TemporaryDirectory() as tmpdir:
+ infos = custom_dataset.evaluate(outputs, tmpdir,
+ ['MRRPE', 'MPJPE', 'Handedness_acc'])
+ assert_almost_equal(infos['MRRPE'], 0.0, decimal=5)
+ assert_almost_equal(infos['MPJPE_all'], 0.0, decimal=5)
+ assert_almost_equal(infos['MPJPE_single'], 0.0, decimal=5)
+ assert_almost_equal(infos['MPJPE_interacting'], 0.0, decimal=5)
+ assert_almost_equal(infos['Handedness_acc'], 1.0)
+
+ with pytest.raises(KeyError):
+ infos = custom_dataset.evaluate(outputs, tmpdir, 'mAP')
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_backward_compatibility/test_dataset_info_compatibility/test_inference_compatibility.py b/engine/pose_estimation/third-party/ViTPose/tests/test_backward_compatibility/test_dataset_info_compatibility/test_inference_compatibility.py
new file mode 100644
index 0000000..fb0988d
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_backward_compatibility/test_dataset_info_compatibility/test_inference_compatibility.py
@@ -0,0 +1,156 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import numpy as np
+import pytest
+
+from mmpose.apis import (extract_pose_sequence, get_track_id,
+ inference_bottom_up_pose_model,
+ inference_pose_lifter_model,
+ inference_top_down_pose_model, init_pose_model,
+ vis_3d_pose_result, vis_pose_result,
+ vis_pose_tracking_result)
+
+
+def test_inference_without_dataset_info():
+ # Top down
+ pose_model = init_pose_model(
+ 'configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/'
+ 'coco/res50_coco_256x192.py',
+ None,
+ device='cpu')
+
+ if 'dataset_info' in pose_model.cfg:
+ _ = pose_model.cfg.pop('dataset_info')
+
+ image_name = 'tests/data/coco/000000000785.jpg'
+ person_result = []
+ person_result.append({'bbox': [50, 50, 50, 100]})
+
+ with pytest.warns(DeprecationWarning):
+ pose_results, _ = inference_top_down_pose_model(
+ pose_model, image_name, person_result, format='xywh')
+
+ with pytest.warns(DeprecationWarning):
+ vis_pose_result(pose_model, image_name, pose_results)
+
+ with pytest.raises(NotImplementedError):
+ with pytest.warns(DeprecationWarning):
+ pose_results, _ = inference_top_down_pose_model(
+ pose_model,
+ image_name,
+ person_result,
+ format='xywh',
+ dataset='test')
+
+ # Bottom up
+ pose_model = init_pose_model(
+ 'configs/body/2d_kpt_sview_rgb_img/associative_embedding/'
+ 'coco/res50_coco_512x512.py',
+ None,
+ device='cpu')
+ if 'dataset_info' in pose_model.cfg:
+ _ = pose_model.cfg.pop('dataset_info')
+
+ image_name = 'tests/data/coco/000000000785.jpg'
+
+ with pytest.warns(DeprecationWarning):
+ pose_results, _ = inference_bottom_up_pose_model(
+ pose_model, image_name)
+ with pytest.warns(DeprecationWarning):
+ vis_pose_result(pose_model, image_name, pose_results)
+
+ # Top down tracking
+ pose_model = init_pose_model(
+ 'configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/'
+ 'coco/res50_coco_256x192.py',
+ None,
+ device='cpu')
+
+ if 'dataset_info' in pose_model.cfg:
+ _ = pose_model.cfg.pop('dataset_info')
+
+ image_name = 'tests/data/coco/000000000785.jpg'
+ person_result = [{'bbox': [50, 50, 50, 100]}]
+
+ with pytest.warns(DeprecationWarning):
+ pose_results, _ = inference_top_down_pose_model(
+ pose_model, image_name, person_result, format='xywh')
+
+ pose_results, _ = get_track_id(pose_results, [], next_id=0)
+
+ with pytest.warns(DeprecationWarning):
+ vis_pose_tracking_result(pose_model, image_name, pose_results)
+
+ with pytest.raises(NotImplementedError):
+ with pytest.warns(DeprecationWarning):
+ vis_pose_tracking_result(
+ pose_model, image_name, pose_results, dataset='test')
+
+ # Bottom up tracking
+ pose_model = init_pose_model(
+ 'configs/body/2d_kpt_sview_rgb_img/associative_embedding/'
+ 'coco/res50_coco_512x512.py',
+ None,
+ device='cpu')
+
+ if 'dataset_info' in pose_model.cfg:
+ _ = pose_model.cfg.pop('dataset_info')
+
+ image_name = 'tests/data/coco/000000000785.jpg'
+ with pytest.warns(DeprecationWarning):
+ pose_results, _ = inference_bottom_up_pose_model(
+ pose_model, image_name)
+
+ pose_results, next_id = get_track_id(pose_results, [], next_id=0)
+
+ with pytest.warns(DeprecationWarning):
+ vis_pose_tracking_result(
+ pose_model,
+ image_name,
+ pose_results,
+ dataset='BottomUpCocoDataset')
+
+ # Pose lifting
+ pose_model = init_pose_model(
+ 'configs/body/3d_kpt_sview_rgb_img/pose_lift/'
+ 'h36m/simplebaseline3d_h36m.py',
+ None,
+ device='cpu')
+
+ pose_det_result = {
+ 'keypoints': np.zeros((17, 3)),
+ 'bbox': [50, 50, 50, 50],
+ 'track_id': 0,
+ 'image_name': 'tests/data/h36m/S1_Directions_1.54138969_000001.jpg',
+ }
+
+ if 'dataset_info' in pose_model.cfg:
+ _ = pose_model.cfg.pop('dataset_info')
+
+ pose_results_2d = [[pose_det_result]]
+
+ dataset = pose_model.cfg.data['test']['type']
+
+ pose_results_2d = extract_pose_sequence(
+ pose_results_2d, frame_idx=0, causal=False, seq_len=1, step=1)
+
+ with pytest.warns(DeprecationWarning):
+ _ = inference_pose_lifter_model(
+ pose_model, pose_results_2d, dataset, with_track_id=False)
+
+ with pytest.warns(DeprecationWarning):
+ pose_lift_results = inference_pose_lifter_model(
+ pose_model, pose_results_2d, dataset, with_track_id=True)
+
+ for res in pose_lift_results:
+ res['title'] = 'title'
+ with pytest.warns(DeprecationWarning):
+ vis_3d_pose_result(
+ pose_model,
+ pose_lift_results,
+ img=pose_results_2d[0][0]['image_name'],
+ dataset=dataset)
+
+ with pytest.raises(NotImplementedError):
+ with pytest.warns(DeprecationWarning):
+ _ = inference_pose_lifter_model(
+ pose_model, pose_results_2d, dataset='test')
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_backward_compatibility/test_dataset_info_compatibility/test_top_down_dataset_compatibility.py b/engine/pose_estimation/third-party/ViTPose/tests/test_backward_compatibility/test_dataset_info_compatibility/test_top_down_dataset_compatibility.py
new file mode 100644
index 0000000..0a4333f
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_backward_compatibility/test_dataset_info_compatibility/test_top_down_dataset_compatibility.py
@@ -0,0 +1,748 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import copy
+import tempfile
+from unittest.mock import MagicMock
+
+import pytest
+from numpy.testing import assert_almost_equal
+
+from mmpose.datasets import DATASETS
+from tests.utils.data_utils import convert_db_to_output
+
+
+def test_top_down_COCO_dataset_compatibility():
+ dataset = 'TopDownCocoDataset'
+ # test COCO datasets
+ dataset_class = DATASETS.get(dataset)
+ dataset_class.load_annotations = MagicMock()
+ dataset_class.coco = MagicMock()
+
+ channel_cfg = dict(
+ num_output_channels=17,
+ dataset_joints=17,
+ dataset_channel=[
+ [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16],
+ ],
+ inference_channel=[
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16
+ ])
+
+ data_cfg = dict(
+ image_size=[192, 256],
+ heatmap_size=[48, 64],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'],
+ soft_nms=False,
+ nms_thr=1.0,
+ oks_thr=0.9,
+ vis_thr=0.2,
+ use_gt_bbox=True,
+ det_bbox_thr=0.0,
+ bbox_file='tests/data/coco/test_coco_det_AP_H_56.json',
+ )
+ # Test det bbox
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ data_cfg_copy['use_gt_bbox'] = False
+ with pytest.warns(DeprecationWarning):
+ _ = dataset_class(
+ ann_file='tests/data/coco/test_coco.json',
+ img_prefix='tests/data/coco/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ test_mode=True)
+
+ with pytest.warns(DeprecationWarning):
+ _ = dataset_class(
+ ann_file='tests/data/coco/test_coco.json',
+ img_prefix='tests/data/coco/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ test_mode=False)
+
+ # Test gt bbox
+ with pytest.warns(DeprecationWarning):
+ custom_dataset = dataset_class(
+ ann_file='tests/data/coco/test_coco.json',
+ img_prefix='tests/data/coco/',
+ data_cfg=data_cfg,
+ pipeline=[],
+ test_mode=True)
+
+ assert custom_dataset.test_mode is True
+ assert custom_dataset.dataset_name == 'coco'
+
+ image_id = 785
+ assert image_id in custom_dataset.img_ids
+ assert len(custom_dataset.img_ids) == 4
+ _ = custom_dataset[0]
+
+ outputs = convert_db_to_output(custom_dataset.db)
+ with tempfile.TemporaryDirectory() as tmpdir:
+ infos = custom_dataset.evaluate(outputs, tmpdir, 'mAP')
+ assert_almost_equal(infos['AP'], 1.0)
+
+ with pytest.raises(KeyError):
+ _ = custom_dataset.evaluate(outputs, tmpdir, 'PCK')
+
+
+def test_top_down_MHP_dataset_compatibility():
+ dataset = 'TopDownMhpDataset'
+ # test MHP datasets
+ dataset_class = DATASETS.get(dataset)
+ dataset_class.load_annotations = MagicMock()
+ dataset_class.coco = MagicMock()
+
+ channel_cfg = dict(
+ num_output_channels=16,
+ dataset_joints=16,
+ dataset_channel=[
+ [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15],
+ ],
+ inference_channel=[
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15
+ ])
+
+ data_cfg = dict(
+ image_size=[192, 256],
+ heatmap_size=[48, 64],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'],
+ soft_nms=False,
+ nms_thr=1.0,
+ oks_thr=0.9,
+ vis_thr=0.2,
+ bbox_thr=1.0,
+ use_gt_bbox=True,
+ det_bbox_thr=0.0,
+ bbox_file='',
+ )
+
+ # Test det bbox
+ with pytest.raises(AssertionError):
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ data_cfg_copy['use_gt_bbox'] = False
+
+ with pytest.warns(DeprecationWarning):
+ _ = dataset_class(
+ ann_file='tests/data/mhp/test_mhp.json',
+ img_prefix='tests/data/mhp/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ test_mode=True)
+
+ # Test gt bbox
+ with pytest.warns(DeprecationWarning):
+ _ = dataset_class(
+ ann_file='tests/data/mhp/test_mhp.json',
+ img_prefix='tests/data/mhp/',
+ data_cfg=data_cfg,
+ pipeline=[],
+ test_mode=False)
+
+ with pytest.warns(DeprecationWarning):
+ custom_dataset = dataset_class(
+ ann_file='tests/data/mhp/test_mhp.json',
+ img_prefix='tests/data/mhp/',
+ data_cfg=data_cfg,
+ pipeline=[],
+ test_mode=True)
+
+ assert custom_dataset.test_mode is True
+ assert custom_dataset.dataset_name == 'mhp'
+
+ image_id = 2889
+ assert image_id in custom_dataset.img_ids
+ assert len(custom_dataset.img_ids) == 2
+ _ = custom_dataset[0]
+
+ outputs = convert_db_to_output(custom_dataset.db)
+ with tempfile.TemporaryDirectory() as tmpdir:
+ infos = custom_dataset.evaluate(outputs, tmpdir, 'mAP')
+ assert_almost_equal(infos['AP'], 1.0)
+
+ with pytest.raises(KeyError):
+ _ = custom_dataset.evaluate(outputs, tmpdir, 'PCK')
+
+
+def test_top_down_PoseTrack18_dataset_compatibility():
+ dataset = 'TopDownPoseTrack18Dataset'
+ # test PoseTrack datasets
+ dataset_class = DATASETS.get(dataset)
+ dataset_class.load_annotations = MagicMock()
+ dataset_class.coco = MagicMock()
+
+ channel_cfg = dict(
+ num_output_channels=17,
+ dataset_joints=17,
+ dataset_channel=[
+ [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16],
+ ],
+ inference_channel=[
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16
+ ])
+
+ data_cfg = dict(
+ image_size=[192, 256],
+ heatmap_size=[48, 64],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'],
+ soft_nms=False,
+ nms_thr=1.0,
+ oks_thr=0.9,
+ vis_thr=0.2,
+ use_gt_bbox=True,
+ det_bbox_thr=0.0,
+ bbox_file='tests/data/posetrack18/annotations/'
+ 'test_posetrack18_human_detections.json',
+ )
+ # Test det bbox
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ data_cfg_copy['use_gt_bbox'] = False
+ with pytest.warns(DeprecationWarning):
+ _ = dataset_class(
+ ann_file='tests/data/posetrack18/annotations/'
+ 'test_posetrack18_val.json',
+ img_prefix='tests/data/posetrack18/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ test_mode=True)
+
+ with pytest.warns(DeprecationWarning):
+ _ = dataset_class(
+ ann_file='tests/data/posetrack18/annotations/'
+ 'test_posetrack18_val.json',
+ img_prefix='tests/data/posetrack18/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ test_mode=False)
+
+ # Test gt bbox
+ with pytest.warns(DeprecationWarning):
+ custom_dataset = dataset_class(
+ ann_file='tests/data/posetrack18/annotations/'
+ 'test_posetrack18_val.json',
+ img_prefix='tests/data/posetrack18/',
+ data_cfg=data_cfg,
+ pipeline=[],
+ test_mode=True)
+
+ assert custom_dataset.test_mode is True
+ assert custom_dataset.dataset_name == 'posetrack18'
+
+ image_id = 10128340000
+ assert image_id in custom_dataset.img_ids
+ assert len(custom_dataset.img_ids) == 3
+ _ = custom_dataset[0]
+
+
+def test_top_down_CrowdPose_dataset_compatibility():
+ dataset = 'TopDownCrowdPoseDataset'
+ # test CrowdPose datasets
+ dataset_class = DATASETS.get(dataset)
+ dataset_class.load_annotations = MagicMock()
+ dataset_class.coco = MagicMock()
+
+ channel_cfg = dict(
+ num_output_channels=14,
+ dataset_joints=14,
+ dataset_channel=[
+ [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13],
+ ],
+ inference_channel=[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13])
+
+ data_cfg = dict(
+ image_size=[192, 256],
+ heatmap_size=[48, 64],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'],
+ soft_nms=False,
+ nms_thr=1.0,
+ oks_thr=0.9,
+ vis_thr=0.2,
+ use_gt_bbox=True,
+ det_bbox_thr=0.0,
+ bbox_file='tests/data/crowdpose/test_crowdpose_det_AP_40.json',
+ )
+ # Test det bbox
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ data_cfg_copy['use_gt_bbox'] = False
+ with pytest.warns(DeprecationWarning):
+ _ = dataset_class(
+ ann_file='tests/data/crowdpose/test_crowdpose.json',
+ img_prefix='tests/data/crowdpose/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ test_mode=True)
+
+ with pytest.warns(DeprecationWarning):
+ _ = dataset_class(
+ ann_file='tests/data/crowdpose/test_crowdpose.json',
+ img_prefix='tests/data/crowdpose/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ test_mode=False)
+
+ # Test gt bbox
+ with pytest.warns(DeprecationWarning):
+ custom_dataset = dataset_class(
+ ann_file='tests/data/crowdpose/test_crowdpose.json',
+ img_prefix='tests/data/crowdpose/',
+ data_cfg=data_cfg,
+ pipeline=[],
+ test_mode=True)
+
+ assert custom_dataset.test_mode is True
+ assert custom_dataset.dataset_name == 'crowdpose'
+
+ image_id = 103319
+ assert image_id in custom_dataset.img_ids
+ assert len(custom_dataset.img_ids) == 2
+ _ = custom_dataset[0]
+
+ outputs = convert_db_to_output(custom_dataset.db)
+ with tempfile.TemporaryDirectory() as tmpdir:
+ infos = custom_dataset.evaluate(outputs, tmpdir, 'mAP')
+ assert_almost_equal(infos['AP'], 1.0)
+
+ with pytest.raises(KeyError):
+ _ = custom_dataset.evaluate(outputs, tmpdir, 'PCK')
+
+
+def test_top_down_COCO_wholebody_dataset_compatibility():
+ dataset = 'TopDownCocoWholeBodyDataset'
+ # test COCO datasets
+ dataset_class = DATASETS.get(dataset)
+ dataset_class.load_annotations = MagicMock()
+ dataset_class.coco = MagicMock()
+
+ channel_cfg = dict(
+ num_output_channels=133,
+ dataset_joints=133,
+ dataset_channel=[
+ list(range(133)),
+ ],
+ inference_channel=list(range(133)))
+
+ data_cfg = dict(
+ image_size=[192, 256],
+ heatmap_size=[48, 64],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'],
+ soft_nms=False,
+ nms_thr=1.0,
+ oks_thr=0.9,
+ vis_thr=0.2,
+ use_gt_bbox=True,
+ det_bbox_thr=0.0,
+ bbox_file='tests/data/coco/test_coco_det_AP_H_56.json',
+ )
+ # Test det bbox
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ data_cfg_copy['use_gt_bbox'] = False
+ with pytest.warns(DeprecationWarning):
+ _ = dataset_class(
+ ann_file='tests/data/coco/test_coco_wholebody.json',
+ img_prefix='tests/data/coco/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ test_mode=True)
+
+ with pytest.warns(DeprecationWarning):
+ _ = dataset_class(
+ ann_file='tests/data/coco/test_coco_wholebody.json',
+ img_prefix='tests/data/coco/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ test_mode=False)
+
+ # Test gt bbox
+ with pytest.warns(DeprecationWarning):
+ custom_dataset = dataset_class(
+ ann_file='tests/data/coco/test_coco_wholebody.json',
+ img_prefix='tests/data/coco/',
+ data_cfg=data_cfg,
+ pipeline=[],
+ test_mode=True)
+
+ assert custom_dataset.test_mode is True
+ assert custom_dataset.dataset_name == 'coco_wholebody'
+
+ image_id = 785
+ assert image_id in custom_dataset.img_ids
+ assert len(custom_dataset.img_ids) == 4
+ _ = custom_dataset[0]
+
+ outputs = convert_db_to_output(custom_dataset.db)
+ with tempfile.TemporaryDirectory() as tmpdir:
+ infos = custom_dataset.evaluate(outputs, tmpdir, 'mAP')
+ assert_almost_equal(infos['AP'], 1.0)
+
+ with pytest.raises(KeyError):
+ _ = custom_dataset.evaluate(outputs, tmpdir, 'PCK')
+
+
+def test_top_down_OCHuman_dataset_compatibility():
+ dataset = 'TopDownOCHumanDataset'
+ # test OCHuman datasets
+ dataset_class = DATASETS.get(dataset)
+ dataset_class.load_annotations = MagicMock()
+ dataset_class.coco = MagicMock()
+
+ channel_cfg = dict(
+ num_output_channels=17,
+ dataset_joints=17,
+ dataset_channel=[
+ [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16],
+ ],
+ inference_channel=[
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16
+ ])
+
+ data_cfg = dict(
+ image_size=[192, 256],
+ heatmap_size=[48, 64],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'],
+ soft_nms=False,
+ nms_thr=1.0,
+ oks_thr=0.9,
+ vis_thr=0.2,
+ use_gt_bbox=True,
+ det_bbox_thr=0.0,
+ bbox_file='',
+ )
+
+ with pytest.raises(AssertionError):
+ # Test det bbox
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ data_cfg_copy['use_gt_bbox'] = False
+ with pytest.warns(DeprecationWarning):
+ _ = dataset_class(
+ ann_file='tests/data/ochuman/test_ochuman.json',
+ img_prefix='tests/data/ochuman/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ test_mode=True)
+
+ # Test gt bbox
+ with pytest.warns(DeprecationWarning):
+ custom_dataset = dataset_class(
+ ann_file='tests/data/ochuman/test_ochuman.json',
+ img_prefix='tests/data/ochuman/',
+ data_cfg=data_cfg,
+ pipeline=[],
+ test_mode=True)
+
+ assert custom_dataset.test_mode is True
+ assert custom_dataset.dataset_name == 'ochuman'
+
+ image_id = 1
+ assert image_id in custom_dataset.img_ids
+ assert len(custom_dataset.img_ids) == 3
+ _ = custom_dataset[0]
+
+ outputs = convert_db_to_output(custom_dataset.db)
+ with tempfile.TemporaryDirectory() as tmpdir:
+ infos = custom_dataset.evaluate(outputs, tmpdir, 'mAP')
+ assert_almost_equal(infos['AP'], 1.0)
+
+ with pytest.raises(KeyError):
+ _ = custom_dataset.evaluate(outputs, tmpdir, 'PCK')
+
+
+def test_top_down_MPII_dataset_compatibility():
+ dataset = 'TopDownMpiiDataset'
+ # test COCO datasets
+ dataset_class = DATASETS.get(dataset)
+ dataset_class.load_annotations = MagicMock()
+ dataset_class.coco = MagicMock()
+
+ channel_cfg = dict(
+ num_output_channels=16,
+ dataset_joints=16,
+ dataset_channel=[
+ [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15],
+ ],
+ inference_channel=[
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15
+ ])
+
+ data_cfg = dict(
+ image_size=[256, 256],
+ heatmap_size=[64, 64],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'],
+ )
+
+ # Test det bbox
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ with pytest.warns(DeprecationWarning):
+ custom_dataset = dataset_class(
+ ann_file='tests/data/mpii/test_mpii.json',
+ img_prefix='tests/data/mpii/',
+ data_cfg=data_cfg_copy,
+ pipeline=[])
+
+ assert len(custom_dataset) == 5
+ assert custom_dataset.dataset_name == 'mpii'
+ _ = custom_dataset[0]
+
+
+def test_top_down_MPII_TRB_dataset_compatibility():
+ dataset = 'TopDownMpiiTrbDataset'
+ # test MPII TRB datasets
+ dataset_class = DATASETS.get(dataset)
+
+ channel_cfg = dict(
+ num_output_channels=40,
+ dataset_joints=40,
+ dataset_channel=[list(range(40))],
+ inference_channel=list(range(40)))
+
+ data_cfg = dict(
+ image_size=[256, 256],
+ heatmap_size=[64, 64],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'])
+
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ with pytest.warns(DeprecationWarning):
+ _ = dataset_class(
+ ann_file='tests/data/mpii/test_mpii_trb.json',
+ img_prefix='tests/data/mpii/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ test_mode=False)
+
+ with pytest.warns(DeprecationWarning):
+ custom_dataset = dataset_class(
+ ann_file='tests/data/mpii/test_mpii_trb.json',
+ img_prefix='tests/data/mpii/',
+ data_cfg=data_cfg,
+ pipeline=[],
+ test_mode=True)
+
+ assert custom_dataset.test_mode is True
+ assert custom_dataset.dataset_name == 'mpii_trb'
+ _ = custom_dataset[0]
+
+
+def test_top_down_AIC_dataset_compatibility():
+ dataset = 'TopDownAicDataset'
+ # test AIC datasets
+ dataset_class = DATASETS.get(dataset)
+ dataset_class.load_annotations = MagicMock()
+ dataset_class.coco = MagicMock()
+
+ channel_cfg = dict(
+ num_output_channels=14,
+ dataset_joints=14,
+ dataset_channel=[
+ [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13],
+ ],
+ inference_channel=[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13])
+
+ data_cfg = dict(
+ image_size=[192, 256],
+ heatmap_size=[48, 64],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'],
+ soft_nms=False,
+ nms_thr=1.0,
+ oks_thr=0.9,
+ vis_thr=0.2,
+ use_gt_bbox=True,
+ det_bbox_thr=0.0,
+ bbox_file='')
+
+ with pytest.raises(AssertionError):
+ # Test det bbox
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ data_cfg_copy['use_gt_bbox'] = False
+ with pytest.warns(DeprecationWarning):
+ _ = dataset_class(
+ ann_file='tests/data/aic/test_aic.json',
+ img_prefix='tests/data/aic/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ test_mode=True)
+
+ with pytest.warns(DeprecationWarning):
+ _ = dataset_class(
+ ann_file='tests/data/aic/test_aic.json',
+ img_prefix='tests/data/aic/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ test_mode=False)
+
+ # Test gt bbox
+ with pytest.warns(DeprecationWarning):
+ custom_dataset = dataset_class(
+ ann_file='tests/data/aic/test_aic.json',
+ img_prefix='tests/data/aic/',
+ data_cfg=data_cfg,
+ pipeline=[],
+ test_mode=True)
+
+ assert custom_dataset.test_mode is True
+ assert custom_dataset.dataset_name == 'aic'
+
+ image_id = 1
+ assert image_id in custom_dataset.img_ids
+ assert len(custom_dataset.img_ids) == 3
+ _ = custom_dataset[0]
+
+ outputs = convert_db_to_output(custom_dataset.db)
+ with tempfile.TemporaryDirectory() as tmpdir:
+ infos = custom_dataset.evaluate(outputs, tmpdir, 'mAP')
+ assert_almost_equal(infos['AP'], 1.0)
+
+ with pytest.raises(KeyError):
+ _ = custom_dataset.evaluate(outputs, tmpdir, 'PCK')
+
+
+def test_top_down_JHMDB_dataset_compatibility():
+ dataset = 'TopDownJhmdbDataset'
+ # test JHMDB datasets
+ dataset_class = DATASETS.get(dataset)
+ dataset_class.load_annotations = MagicMock()
+ dataset_class.coco = MagicMock()
+
+ channel_cfg = dict(
+ num_output_channels=15,
+ dataset_joints=15,
+ dataset_channel=[
+ [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14],
+ ],
+ inference_channel=[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14])
+
+ data_cfg = dict(
+ image_size=[192, 256],
+ heatmap_size=[48, 64],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'],
+ soft_nms=False,
+ nms_thr=1.0,
+ oks_thr=0.9,
+ vis_thr=0.2,
+ use_gt_bbox=True,
+ det_bbox_thr=0.0,
+ bbox_file='')
+
+ with pytest.raises(AssertionError):
+ # Test det bbox
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ data_cfg_copy['use_gt_bbox'] = False
+ with pytest.warns(DeprecationWarning):
+ _ = dataset_class(
+ ann_file='tests/data/jhmdb/test_jhmdb_sub1.json',
+ img_prefix='tests/data/jhmdb/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ test_mode=True)
+
+ with pytest.warns(DeprecationWarning):
+ _ = dataset_class(
+ ann_file='tests/data/jhmdb/test_jhmdb_sub1.json',
+ img_prefix='tests/data/jhmdb/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ test_mode=False)
+
+ # Test gt bbox
+ with pytest.warns(DeprecationWarning):
+ custom_dataset = dataset_class(
+ ann_file='tests/data/jhmdb/test_jhmdb_sub1.json',
+ img_prefix='tests/data/jhmdb/',
+ data_cfg=data_cfg,
+ pipeline=[],
+ test_mode=True)
+
+ assert custom_dataset.test_mode is True
+ assert custom_dataset.dataset_name == 'jhmdb'
+
+ image_id = 2290001
+ assert image_id in custom_dataset.img_ids
+ assert len(custom_dataset.img_ids) == 3
+ _ = custom_dataset[0]
+
+ outputs = convert_db_to_output(custom_dataset.db)
+ with tempfile.TemporaryDirectory() as tmpdir:
+ infos = custom_dataset.evaluate(outputs, tmpdir, ['PCK'])
+ assert_almost_equal(infos['Mean PCK'], 1.0)
+
+ infos = custom_dataset.evaluate(outputs, tmpdir, ['tPCK'])
+ assert_almost_equal(infos['Mean tPCK'], 1.0)
+
+ with pytest.raises(KeyError):
+ _ = custom_dataset.evaluate(outputs, tmpdir, 'mAP')
+
+
+def test_top_down_h36m_dataset_compatibility():
+ dataset = 'TopDownH36MDataset'
+ # test AIC datasets
+ dataset_class = DATASETS.get(dataset)
+ dataset_class.load_annotations = MagicMock()
+ dataset_class.coco = MagicMock()
+
+ channel_cfg = dict(
+ num_output_channels=17,
+ dataset_joints=17,
+ dataset_channel=[
+ [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16],
+ ],
+ inference_channel=[
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16
+ ])
+
+ data_cfg = dict(
+ image_size=[256, 256],
+ heatmap_size=[64, 64],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'])
+
+ # Test gt bbox
+ with pytest.warns(DeprecationWarning):
+ custom_dataset = dataset_class(
+ ann_file='tests/data/h36m/h36m_coco.json',
+ img_prefix='tests/data/h36m/',
+ data_cfg=data_cfg,
+ pipeline=[],
+ test_mode=True)
+
+ assert custom_dataset.test_mode is True
+ assert custom_dataset.dataset_name == 'h36m'
+
+ image_id = 1
+ assert image_id in custom_dataset.img_ids
+ _ = custom_dataset[0]
+
+ outputs = convert_db_to_output(custom_dataset.db)
+ with tempfile.TemporaryDirectory() as tmpdir:
+ infos = custom_dataset.evaluate(outputs, tmpdir, 'EPE')
+ assert_almost_equal(infos['EPE'], 0.0)
+
+ with pytest.raises(KeyError):
+ _ = custom_dataset.evaluate(outputs, tmpdir, 'AUC')
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_backward_compatibility/test_eval_hook_compatibility.py b/engine/pose_estimation/third-party/ViTPose/tests/test_backward_compatibility/test_eval_hook_compatibility.py
new file mode 100644
index 0000000..f62f586
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_backward_compatibility/test_eval_hook_compatibility.py
@@ -0,0 +1,46 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import unittest.mock as mock
+
+import pytest
+import torch
+from torch.utils.data import DataLoader, Dataset
+
+from mmpose.core import DistEvalHook, EvalHook
+
+
+class ExampleDataset(Dataset):
+
+ def __init__(self):
+ self.index = 0
+ self.eval_result = [0.1, 0.4, 0.3, 0.7, 0.2, 0.05, 0.4, 0.6]
+
+ def __getitem__(self, idx):
+ results = dict(imgs=torch.tensor([1]))
+ return results
+
+ def __len__(self):
+ return 1
+
+ @mock.create_autospec
+ def evaluate(self, results, res_folder=None, logger=None):
+ pass
+
+
+def test_old_fashion_eval_hook_parameters():
+
+ data_loader = DataLoader(
+ ExampleDataset(),
+ batch_size=1,
+ sampler=None,
+ num_workers=0,
+ shuffle=False)
+
+ # test argument "key_indicator"
+ with pytest.warns(DeprecationWarning):
+ _ = EvalHook(data_loader, key_indicator='AP')
+ with pytest.warns(DeprecationWarning):
+ _ = DistEvalHook(data_loader, key_indicator='AP')
+
+ # test argument "gpu_collect"
+ with pytest.warns(DeprecationWarning):
+ _ = EvalHook(data_loader, save_best='AP', gpu_collect=False)
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_backward_compatibility/test_registry_compatibility.py b/engine/pose_estimation/third-party/ViTPose/tests/test_backward_compatibility/test_registry_compatibility.py
new file mode 100644
index 0000000..68a487b
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_backward_compatibility/test_registry_compatibility.py
@@ -0,0 +1,10 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+# flake8: noqa
+import pytest
+
+
+def test_old_fashion_registry_importing():
+ with pytest.warns(DeprecationWarning):
+ from mmpose.models.registry import BACKBONES, HEADS, LOSSES, NECKS, POSENETS # isort: skip
+ with pytest.warns(DeprecationWarning):
+ from mmpose.datasets.registry import DATASETS, PIPELINES # noqa: F401
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_config.py b/engine/pose_estimation/third-party/ViTPose/tests/test_config.py
new file mode 100644
index 0000000..cbcc599
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_config.py
@@ -0,0 +1,54 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from os.path import dirname, exists, join, relpath
+
+import torch
+from mmcv.runner import build_optimizer
+
+
+def _get_config_directory():
+ """Find the predefined detector config directory."""
+ try:
+ # Assume we are running in the source mmdetection repo
+ repo_dpath = dirname(dirname(__file__))
+ except NameError:
+ # For IPython development when this __file__ is not defined
+ import mmpose
+ repo_dpath = dirname(dirname(mmpose.__file__))
+ config_dpath = join(repo_dpath, 'configs')
+ if not exists(config_dpath):
+ raise Exception('Cannot find config path')
+ return config_dpath
+
+
+def test_config_build_detector():
+ """Test that all detection models defined in the configs can be
+ initialized."""
+ from mmcv import Config
+
+ from mmpose.models import build_posenet
+
+ config_dpath = _get_config_directory()
+ print(f'Found config_dpath = {config_dpath}')
+
+ import glob
+ config_fpaths = list(glob.glob(join(config_dpath, '**', '*.py')))
+ config_fpaths = [p for p in config_fpaths if p.find('_base_') == -1]
+ config_names = [relpath(p, config_dpath) for p in config_fpaths]
+
+ print(f'Using {len(config_names)} config files')
+
+ for config_fname in config_names:
+ config_fpath = join(config_dpath, config_fname)
+ config_mod = Config.fromfile(config_fpath)
+
+ print(f'Building detector, config_fpath = {config_fpath}')
+
+ # Remove pretrained keys to allow for testing in an offline environment
+ if 'pretrained' in config_mod.model:
+ config_mod.model['pretrained'] = None
+
+ detector = build_posenet(config_mod.model)
+ assert detector is not None
+
+ optimizer = build_optimizer(detector, config_mod.optimizer)
+ assert isinstance(optimizer, torch.optim.Optimizer)
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_datasets/test_animal_dataset.py b/engine/pose_estimation/third-party/ViTPose/tests/test_datasets/test_animal_dataset.py
new file mode 100644
index 0000000..328c8d5
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_datasets/test_animal_dataset.py
@@ -0,0 +1,500 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import copy
+
+import pytest
+from mmcv import Config
+from numpy.testing import assert_almost_equal
+
+from mmpose.datasets import DATASETS
+from tests.utils.data_utils import convert_db_to_output
+
+
+def test_animal_horse10_dataset():
+ dataset = 'AnimalHorse10Dataset'
+ dataset_class = DATASETS.get(dataset)
+ dataset_info = Config.fromfile(
+ 'configs/_base_/datasets/horse10.py').dataset_info
+
+ channel_cfg = dict(
+ num_output_channels=22,
+ dataset_joints=22,
+ dataset_channel=[
+ [
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17,
+ 18, 19, 21
+ ],
+ ],
+ inference_channel=[
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18,
+ 19, 21
+ ])
+
+ data_cfg = dict(
+ image_size=[256, 256],
+ heatmap_size=[64, 64],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'])
+ # Test
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ _ = dataset_class(
+ ann_file='tests/data/horse10/test_horse10.json',
+ img_prefix='tests/data/horse10/',
+ data_cfg=data_cfg_copy,
+ dataset_info=dataset_info,
+ pipeline=[],
+ test_mode=True)
+
+ custom_dataset = dataset_class(
+ ann_file='tests/data/horse10/test_horse10.json',
+ img_prefix='tests/data/horse10/',
+ data_cfg=data_cfg_copy,
+ dataset_info=dataset_info,
+ pipeline=[],
+ test_mode=False)
+
+ assert custom_dataset.dataset_name == 'horse10'
+ assert custom_dataset.test_mode is False
+ assert custom_dataset.num_images == 3
+ _ = custom_dataset[0]
+
+ results = convert_db_to_output(custom_dataset.db)
+ infos = custom_dataset.evaluate(results, metric=['PCK'])
+ assert_almost_equal(infos['PCK'], 1.0)
+
+ with pytest.raises(KeyError):
+ infos = custom_dataset.evaluate(results, metric='mAP')
+
+
+def test_animal_fly_dataset():
+ dataset = 'AnimalFlyDataset'
+ dataset_class = DATASETS.get(dataset)
+ dataset_info = Config.fromfile(
+ 'configs/_base_/datasets/fly.py').dataset_info
+
+ channel_cfg = dict(
+ num_output_channels=32,
+ dataset_joints=32,
+ dataset_channel=[
+ [
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17,
+ 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31
+ ],
+ ],
+ inference_channel=[
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18,
+ 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31
+ ])
+
+ data_cfg = dict(
+ image_size=[192, 192],
+ heatmap_size=[48, 48],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'])
+
+ # Test
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ _ = dataset_class(
+ ann_file='tests/data/fly/test_fly.json',
+ img_prefix='tests/data/fly/',
+ data_cfg=data_cfg_copy,
+ dataset_info=dataset_info,
+ pipeline=[],
+ test_mode=True)
+
+ custom_dataset = dataset_class(
+ ann_file='tests/data/fly/test_fly.json',
+ img_prefix='tests/data/fly/',
+ data_cfg=data_cfg_copy,
+ dataset_info=dataset_info,
+ pipeline=[],
+ test_mode=False)
+
+ assert custom_dataset.dataset_name == 'fly'
+ assert custom_dataset.test_mode is False
+ assert custom_dataset.num_images == 2
+ _ = custom_dataset[0]
+
+ results = convert_db_to_output(custom_dataset.db)
+
+ infos = custom_dataset.evaluate(results, metric=['PCK'])
+ assert_almost_equal(infos['PCK'], 1.0)
+
+ with pytest.raises(KeyError):
+ infos = custom_dataset.evaluate(results, metric='mAP')
+
+
+def test_animal_locust_dataset():
+ dataset = 'AnimalLocustDataset'
+ dataset_class = DATASETS.get(dataset)
+ dataset_info = Config.fromfile(
+ 'configs/_base_/datasets/locust.py').dataset_info
+
+ channel_cfg = dict(
+ num_output_channels=35,
+ dataset_joints=35,
+ dataset_channel=[
+ [
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17,
+ 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33,
+ 34
+ ],
+ ],
+ inference_channel=[
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18,
+ 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34
+ ])
+
+ data_cfg = dict(
+ image_size=[160, 160],
+ heatmap_size=[40, 40],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'])
+
+ # Test
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ _ = dataset_class(
+ ann_file='tests/data/locust/test_locust.json',
+ img_prefix='tests/data/locust/',
+ data_cfg=data_cfg_copy,
+ dataset_info=dataset_info,
+ pipeline=[],
+ test_mode=True)
+
+ custom_dataset = dataset_class(
+ ann_file='tests/data/locust/test_locust.json',
+ img_prefix='tests/data/locust/',
+ data_cfg=data_cfg_copy,
+ dataset_info=dataset_info,
+ pipeline=[],
+ test_mode=False)
+
+ assert custom_dataset.dataset_name == 'locust'
+ assert custom_dataset.test_mode is False
+ assert custom_dataset.num_images == 2
+ _ = custom_dataset[0]
+
+ results = convert_db_to_output(custom_dataset.db)
+
+ infos = custom_dataset.evaluate(results, metric=['PCK'])
+ assert_almost_equal(infos['PCK'], 1.0)
+
+ with pytest.raises(KeyError):
+ infos = custom_dataset.evaluate(results, metric='mAP')
+
+
+def test_animal_zebra_dataset():
+ dataset = 'AnimalZebraDataset'
+ dataset_class = DATASETS.get(dataset)
+ dataset_info = Config.fromfile(
+ 'configs/_base_/datasets/zebra.py').dataset_info
+
+ channel_cfg = dict(
+ num_output_channels=9,
+ dataset_joints=9,
+ dataset_channel=[
+ [0, 1, 2, 3, 4, 5, 6, 7, 8],
+ ],
+ inference_channel=[0, 1, 2, 3, 4, 5, 6, 7, 8])
+
+ data_cfg = dict(
+ image_size=[160, 160],
+ heatmap_size=[40, 40],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'])
+
+ # Test
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ _ = dataset_class(
+ ann_file='tests/data/zebra/test_zebra.json',
+ img_prefix='tests/data/zebra/',
+ data_cfg=data_cfg_copy,
+ dataset_info=dataset_info,
+ pipeline=[],
+ test_mode=True)
+
+ custom_dataset = dataset_class(
+ ann_file='tests/data/zebra/test_zebra.json',
+ img_prefix='tests/data/zebra/',
+ data_cfg=data_cfg_copy,
+ dataset_info=dataset_info,
+ pipeline=[],
+ test_mode=False)
+
+ assert custom_dataset.dataset_name == 'zebra'
+ assert custom_dataset.test_mode is False
+ assert custom_dataset.num_images == 2
+ _ = custom_dataset[0]
+
+ results = convert_db_to_output(custom_dataset.db)
+ infos = custom_dataset.evaluate(results, metric=['PCK'])
+ assert_almost_equal(infos['PCK'], 1.0)
+
+ with pytest.raises(KeyError):
+ infos = custom_dataset.evaluate(results, metric='mAP')
+
+
+def test_animal_ATRW_dataset():
+ dataset = 'AnimalATRWDataset'
+ dataset_class = DATASETS.get(dataset)
+ dataset_info = Config.fromfile(
+ 'configs/_base_/datasets/atrw.py').dataset_info
+
+ channel_cfg = dict(
+ num_output_channels=15,
+ dataset_joints=15,
+ dataset_channel=[
+ [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14],
+ ],
+ inference_channel=[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14])
+
+ data_cfg = dict(
+ image_size=[256, 256],
+ heatmap_size=[64, 64],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'],
+ soft_nms=False,
+ nms_thr=1.0,
+ oks_thr=0.9,
+ vis_thr=0.2,
+ use_gt_bbox=True,
+ det_bbox_thr=0.0,
+ bbox_file='',
+ )
+
+ # Test
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ _ = dataset_class(
+ ann_file='tests/data/atrw/test_atrw.json',
+ img_prefix='tests/data/atrw/',
+ data_cfg=data_cfg_copy,
+ dataset_info=dataset_info,
+ pipeline=[],
+ test_mode=True)
+
+ custom_dataset = dataset_class(
+ ann_file='tests/data/atrw/test_atrw.json',
+ img_prefix='tests/data/atrw/',
+ data_cfg=data_cfg_copy,
+ dataset_info=dataset_info,
+ pipeline=[],
+ test_mode=False)
+
+ assert custom_dataset.dataset_name == 'atrw'
+ assert custom_dataset.test_mode is False
+ assert custom_dataset.num_images == 2
+ _ = custom_dataset[0]
+
+ results = convert_db_to_output(custom_dataset.db)
+ infos = custom_dataset.evaluate(results, metric='mAP')
+ assert_almost_equal(infos['AP'], 1.0)
+
+ with pytest.raises(KeyError):
+ infos = custom_dataset.evaluate(results, metric=['PCK'])
+
+
+def test_animal_Macaque_dataset():
+ dataset = 'AnimalMacaqueDataset'
+ dataset_class = DATASETS.get(dataset)
+ dataset_info = Config.fromfile(
+ 'configs/_base_/datasets/macaque.py').dataset_info
+
+ channel_cfg = dict(
+ num_output_channels=17,
+ dataset_joints=17,
+ dataset_channel=[
+ [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16],
+ ],
+ inference_channel=[
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16
+ ])
+
+ data_cfg = dict(
+ image_size=[192, 256],
+ heatmap_size=[48, 64],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'],
+ soft_nms=False,
+ nms_thr=1.0,
+ oks_thr=0.9,
+ vis_thr=0.2,
+ use_gt_bbox=True,
+ det_bbox_thr=0.0,
+ bbox_file='',
+ )
+
+ # Test
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ _ = dataset_class(
+ ann_file='tests/data/macaque/test_macaque.json',
+ img_prefix='tests/data/macaque/',
+ data_cfg=data_cfg_copy,
+ dataset_info=dataset_info,
+ pipeline=[],
+ test_mode=True)
+
+ custom_dataset = dataset_class(
+ ann_file='tests/data/macaque/test_macaque.json',
+ img_prefix='tests/data/macaque/',
+ data_cfg=data_cfg_copy,
+ dataset_info=dataset_info,
+ pipeline=[],
+ test_mode=False)
+
+ assert custom_dataset.dataset_name == 'macaque'
+ assert custom_dataset.test_mode is False
+ assert custom_dataset.num_images == 2
+ _ = custom_dataset[0]
+
+ results = convert_db_to_output(custom_dataset.db)
+ infos = custom_dataset.evaluate(results, metric='mAP')
+ assert_almost_equal(infos['AP'], 1.0)
+
+ with pytest.raises(KeyError):
+ infos = custom_dataset.evaluate(results, metric=['PCK'])
+
+
+def test_animalpose_dataset():
+ dataset = 'AnimalPoseDataset'
+ dataset_class = DATASETS.get(dataset)
+ dataset_info = Config.fromfile(
+ 'configs/_base_/datasets/animalpose.py').dataset_info
+
+ channel_cfg = dict(
+ num_output_channels=20,
+ dataset_joints=20,
+ dataset_channel=[
+ [
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17,
+ 18, 19
+ ],
+ ],
+ inference_channel=[
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18,
+ 19
+ ])
+
+ data_cfg = dict(
+ image_size=[256, 256],
+ heatmap_size=[64, 64],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'],
+ soft_nms=False,
+ nms_thr=1.0,
+ oks_thr=0.9,
+ vis_thr=0.2,
+ use_gt_bbox=True,
+ det_bbox_thr=0.0,
+ bbox_file='',
+ )
+
+ # Test
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ _ = dataset_class(
+ ann_file='tests/data/animalpose/test_animalpose.json',
+ img_prefix='tests/data/animalpose/',
+ data_cfg=data_cfg_copy,
+ dataset_info=dataset_info,
+ pipeline=[],
+ test_mode=True)
+
+ custom_dataset = dataset_class(
+ ann_file='tests/data/animalpose/test_animalpose.json',
+ img_prefix='tests/data/animalpose/',
+ data_cfg=data_cfg_copy,
+ dataset_info=dataset_info,
+ pipeline=[],
+ test_mode=False)
+
+ assert custom_dataset.dataset_name == 'animalpose'
+ assert custom_dataset.test_mode is False
+ assert custom_dataset.num_images == 2
+ _ = custom_dataset[0]
+
+ results = convert_db_to_output(custom_dataset.db)
+ infos = custom_dataset.evaluate(results, metric='mAP')
+ assert_almost_equal(infos['AP'], 1.0)
+
+ with pytest.raises(KeyError):
+ infos = custom_dataset.evaluate(results, metric=['PCK'])
+
+
+def test_ap10k_dataset():
+ dataset = 'AnimalAP10KDataset'
+ dataset_class = DATASETS.get(dataset)
+ dataset_info = Config.fromfile(
+ 'configs/_base_/datasets/ap10k.py').dataset_info
+
+ channel_cfg = dict(
+ num_output_channels=17,
+ dataset_joints=17,
+ dataset_channel=[
+ [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16],
+ ],
+ inference_channel=[
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16
+ ])
+
+ data_cfg = dict(
+ image_size=[256, 256],
+ heatmap_size=[64, 64],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'],
+ soft_nms=False,
+ nms_thr=1.0,
+ oks_thr=0.9,
+ vis_thr=0.2,
+ use_gt_bbox=True,
+ det_bbox_thr=0.0,
+ bbox_file='',
+ )
+
+ # Test
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ _ = dataset_class(
+ ann_file='tests/data/ap10k/test_ap10k.json',
+ img_prefix='tests/data/ap10k/',
+ data_cfg=data_cfg_copy,
+ dataset_info=dataset_info,
+ pipeline=[],
+ test_mode=True)
+
+ custom_dataset = dataset_class(
+ ann_file='tests/data/ap10k/test_ap10k.json',
+ img_prefix='tests/data/ap10k/',
+ data_cfg=data_cfg_copy,
+ dataset_info=dataset_info,
+ pipeline=[],
+ test_mode=False)
+
+ assert custom_dataset.dataset_name == 'ap10k'
+ assert custom_dataset.test_mode is False
+ assert custom_dataset.num_images == 2
+ _ = custom_dataset[0]
+
+ results = convert_db_to_output(custom_dataset.db)
+
+ for output in results:
+ # as there is only one box in each image for test
+ output['bbox_ids'] = [0 for _ in range(len(output['bbox_ids']))]
+
+ infos = custom_dataset.evaluate(results, metric='mAP')
+ assert_almost_equal(infos['AP'], 1.0)
+
+ with pytest.raises(KeyError):
+ infos = custom_dataset.evaluate(results, metric=['PCK'])
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_datasets/test_body3d_dataset.py b/engine/pose_estimation/third-party/ViTPose/tests/test_datasets/test_body3d_dataset.py
new file mode 100644
index 0000000..a9cd94e
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_datasets/test_body3d_dataset.py
@@ -0,0 +1,347 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import tempfile
+
+import numpy as np
+from mmcv import Config
+
+from mmpose.datasets import DATASETS
+from mmpose.datasets.builder import build_dataset
+
+
+def test_body3d_h36m_dataset():
+ # Test Human3.6M dataset
+ dataset = 'Body3DH36MDataset'
+ dataset_class = DATASETS.get(dataset)
+ dataset_info = Config.fromfile(
+ 'configs/_base_/datasets/h36m.py').dataset_info
+
+ # test single-frame input
+ data_cfg = dict(
+ num_joints=17,
+ seq_len=1,
+ seq_frame_interval=1,
+ joint_2d_src='pipeline',
+ joint_2d_det_file=None,
+ causal=False,
+ need_camera_param=True,
+ camera_param_file='tests/data/h36m/cameras.pkl')
+
+ _ = dataset_class(
+ ann_file='tests/data/h36m/test_h36m_body3d.npz',
+ img_prefix='tests/data/h36m',
+ data_cfg=data_cfg,
+ dataset_info=dataset_info,
+ pipeline=[],
+ test_mode=False)
+
+ custom_dataset = dataset_class(
+ ann_file='tests/data/h36m/test_h36m_body3d.npz',
+ img_prefix='tests/data/h36m',
+ data_cfg=data_cfg,
+ dataset_info=dataset_info,
+ pipeline=[],
+ test_mode=True)
+
+ assert custom_dataset.dataset_name == 'h36m'
+ assert custom_dataset.test_mode is True
+ _ = custom_dataset[0]
+
+ results = []
+ for result in custom_dataset:
+ results.append({
+ 'preds': result['target'][None, ...],
+ 'target_image_paths': [result['target_image_path']],
+ })
+
+ metrics = ['mpjpe', 'p-mpjpe', 'n-mpjpe']
+ infos = custom_dataset.evaluate(results, metric=metrics)
+
+ np.testing.assert_almost_equal(infos['MPJPE'], 0.0)
+ np.testing.assert_almost_equal(infos['P-MPJPE'], 0.0)
+ np.testing.assert_almost_equal(infos['N-MPJPE'], 0.0)
+
+ # test multi-frame input with joint_2d_src = 'detection'
+ data_cfg = dict(
+ num_joints=17,
+ seq_len=27,
+ seq_frame_interval=1,
+ causal=True,
+ temporal_padding=True,
+ joint_2d_src='detection',
+ joint_2d_det_file='tests/data/h36m/test_h36m_2d_detection.npy',
+ need_camera_param=True,
+ camera_param_file='tests/data/h36m/cameras.pkl')
+
+ _ = dataset_class(
+ ann_file='tests/data/h36m/test_h36m_body3d.npz',
+ img_prefix='tests/data/h36m',
+ data_cfg=data_cfg,
+ dataset_info=dataset_info,
+ pipeline=[],
+ test_mode=False)
+
+ custom_dataset = dataset_class(
+ ann_file='tests/data/h36m/test_h36m_body3d.npz',
+ img_prefix='tests/data/h36m',
+ data_cfg=data_cfg,
+ dataset_info=dataset_info,
+ pipeline=[],
+ test_mode=True)
+
+ assert custom_dataset.test_mode is True
+ _ = custom_dataset[0]
+
+ results = []
+ for result in custom_dataset:
+ results.append({
+ 'preds': result['target'][None, ...],
+ 'target_image_paths': [result['target_image_path']],
+ })
+
+ metrics = ['mpjpe', 'p-mpjpe', 'n-mpjpe']
+ infos = custom_dataset.evaluate(results, metric=metrics)
+
+ np.testing.assert_almost_equal(infos['MPJPE'], 0.0)
+ np.testing.assert_almost_equal(infos['P-MPJPE'], 0.0)
+ np.testing.assert_almost_equal(infos['N-MPJPE'], 0.0)
+
+
+def test_body3d_semi_supervision_dataset():
+ # Test Body3d Semi-supervision Dataset
+ dataset_info = Config.fromfile(
+ 'configs/_base_/datasets/h36m.py').dataset_info
+
+ # load labeled dataset
+ labeled_data_cfg = dict(
+ num_joints=17,
+ seq_len=27,
+ seq_frame_interval=1,
+ causall=False,
+ temporal_padding=True,
+ joint_2d_src='gt',
+ subset=1,
+ subjects=['S1'],
+ need_camera_param=True,
+ camera_param_file='tests/data/h36m/cameras.pkl')
+ labeled_dataset_cfg = dict(
+ type='Body3DH36MDataset',
+ ann_file='tests/data/h36m/test_h36m_body3d.npz',
+ img_prefix='tests/data/h36m',
+ data_cfg=labeled_data_cfg,
+ dataset_info=dataset_info,
+ pipeline=[])
+
+ # load unlabled data
+ unlabeled_data_cfg = dict(
+ num_joints=17,
+ seq_len=27,
+ seq_frame_interval=1,
+ causal=False,
+ temporal_padding=True,
+ joint_2d_src='gt',
+ subjects=['S5', 'S7', 'S8'],
+ need_camera_param=True,
+ camera_param_file='tests/data/h36m/cameras.pkl',
+ need_2d_label=True)
+ unlabeled_dataset_cfg = dict(
+ type='Body3DH36MDataset',
+ ann_file='tests/data/h36m/test_h36m_body3d.npz',
+ img_prefix='tests/data/h36m',
+ data_cfg=unlabeled_data_cfg,
+ dataset_info=dataset_info,
+ pipeline=[
+ dict(
+ type='Collect',
+ keys=[('input_2d', 'unlabeled_input')],
+ meta_name='metas',
+ meta_keys=[])
+ ])
+
+ # combine labeled and unlabeled dataset to form a new dataset
+ dataset = 'Body3DSemiSupervisionDataset'
+ dataset_class = DATASETS.get(dataset)
+ custom_dataset = dataset_class(labeled_dataset_cfg, unlabeled_dataset_cfg)
+ item = custom_dataset[0]
+ assert custom_dataset.labeled_dataset.dataset_name == 'h36m'
+ assert 'unlabeled_input' in item.keys()
+
+ unlabeled_dataset = build_dataset(unlabeled_dataset_cfg)
+ assert len(unlabeled_dataset) == len(custom_dataset)
+
+
+def test_body3d_mpi_inf_3dhp_dataset():
+ # Test MPI-INF-3DHP dataset
+ dataset = 'Body3DMpiInf3dhpDataset'
+ dataset_class = DATASETS.get(dataset)
+ dataset_info = Config.fromfile(
+ 'configs/_base_/datasets/mpi_inf_3dhp.py').dataset_info
+
+ # Test single-frame input on trainset
+ single_frame_train_data_cfg = dict(
+ num_joints=17,
+ seq_len=1,
+ seq_frame_interval=1,
+ joint_2d_src='pipeline',
+ joint_2d_det_file=None,
+ causal=False,
+ need_camera_param=True,
+ camera_param_file='tests/data/mpi_inf_3dhp/cameras_train.pkl')
+
+ # Test single-frame input on testset
+ single_frame_test_data_cfg = dict(
+ num_joints=17,
+ seq_len=1,
+ seq_frame_interval=1,
+ joint_2d_src='gt',
+ joint_2d_det_file=None,
+ causal=False,
+ need_camera_param=True,
+ camera_param_file='tests/data/mpi_inf_3dhp/cameras_test.pkl')
+
+ # Test multi-frame input on trainset
+ multi_frame_train_data_cfg = dict(
+ num_joints=17,
+ seq_len=27,
+ seq_frame_interval=1,
+ joint_2d_src='gt',
+ joint_2d_det_file=None,
+ causal=True,
+ temporal_padding=True,
+ need_camera_param=True,
+ camera_param_file='tests/data/mpi_inf_3dhp/cameras_train.pkl')
+
+ # Test multi-frame input on testset
+ multi_frame_test_data_cfg = dict(
+ num_joints=17,
+ seq_len=27,
+ seq_frame_interval=1,
+ joint_2d_src='pipeline',
+ joint_2d_det_file=None,
+ causal=False,
+ temporal_padding=True,
+ need_camera_param=True,
+ camera_param_file='tests/data/mpi_inf_3dhp/cameras_test.pkl')
+
+ ann_files = [
+ 'tests/data/mpi_inf_3dhp/test_3dhp_train.npz',
+ 'tests/data/mpi_inf_3dhp/test_3dhp_test.npz'
+ ] * 2
+ data_cfgs = [
+ single_frame_train_data_cfg, single_frame_test_data_cfg,
+ multi_frame_train_data_cfg, multi_frame_test_data_cfg
+ ]
+
+ for ann_file, data_cfg in zip(ann_files, data_cfgs):
+ _ = dataset_class(
+ ann_file=ann_file,
+ img_prefix='tests/data/mpi_inf_3dhp',
+ data_cfg=data_cfg,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=False)
+
+ custom_dataset = dataset_class(
+ ann_file=ann_file,
+ img_prefix='tests/data/mpi_inf_3dhp',
+ data_cfg=data_cfg,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=True)
+
+ assert custom_dataset.test_mode is True
+ _ = custom_dataset[0]
+
+ results = []
+ for result in custom_dataset:
+ results.append({
+ 'preds': result['target'][None, ...],
+ 'target_image_paths': [result['target_image_path']],
+ })
+
+ metrics = ['mpjpe', 'p-mpjpe', '3dpck', 'p-3dpck', '3dauc', 'p-3dauc']
+ infos = custom_dataset.evaluate(results, metric=metrics)
+
+ np.testing.assert_almost_equal(infos['MPJPE'], 0.0)
+ np.testing.assert_almost_equal(infos['P-MPJPE'], 0.0)
+ np.testing.assert_almost_equal(infos['3DPCK'], 100.)
+ np.testing.assert_almost_equal(infos['P-3DPCK'], 100.)
+ np.testing.assert_almost_equal(infos['3DAUC'], 30 / 31 * 100)
+ np.testing.assert_almost_equal(infos['P-3DAUC'], 30 / 31 * 100)
+
+
+def test_body3dmview_direct_panoptic_dataset():
+ # Test Mview-Panoptic dataset
+ dataset = 'Body3DMviewDirectPanopticDataset'
+ dataset_class = DATASETS.get(dataset)
+ dataset_info = Config.fromfile(
+ 'configs/_base_/datasets/panoptic_body3d.py').dataset_info
+ space_size = [8000, 8000, 2000]
+ space_center = [0, -500, 800]
+ cube_size = [80, 80, 20]
+ train_data_cfg = dict(
+ image_size=[960, 512],
+ heatmap_size=[[240, 128]],
+ space_size=space_size,
+ space_center=space_center,
+ cube_size=cube_size,
+ num_joints=15,
+ seq_list=['160906_band1', '160906_band2'],
+ cam_list=[(0, 12), (0, 6)],
+ num_cameras=2,
+ seq_frame_interval=1,
+ subset='train',
+ need_2d_label=True,
+ need_camera_param=True,
+ root_id=2)
+
+ test_data_cfg = dict(
+ image_size=[960, 512],
+ heatmap_size=[[240, 128]],
+ num_joints=15,
+ space_size=space_size,
+ space_center=space_center,
+ cube_size=cube_size,
+ seq_list=['160906_band1', '160906_band2'],
+ cam_list=[(0, 12), (0, 6)],
+ num_cameras=2,
+ seq_frame_interval=1,
+ subset='validation',
+ need_2d_label=True,
+ need_camera_param=True,
+ root_id=2)
+ with tempfile.TemporaryDirectory() as tmpdir:
+ _ = dataset_class(
+ ann_file=tmpdir + '/tmp_train.pkl',
+ img_prefix='tests/data/panoptic_body3d/',
+ data_cfg=train_data_cfg,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=False)
+
+ with tempfile.TemporaryDirectory() as tmpdir:
+ test_dataset = dataset_class(
+ ann_file=tmpdir + '/tmp_validation.pkl',
+ img_prefix='tests/data/panoptic_body3d',
+ data_cfg=test_data_cfg,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=False)
+
+ import copy
+ gt_num = test_dataset.db_size // test_dataset.num_cameras
+ results = []
+ for i in range(gt_num):
+ index = test_dataset.num_cameras * i
+ db_rec = copy.deepcopy(test_dataset.db[index])
+ joints_3d = db_rec['joints_3d']
+ joints_3d_vis = db_rec['joints_3d_visible']
+ num_gts = len(joints_3d)
+ gt_pose = -np.ones((1, 10, test_dataset.num_joints, 5))
+
+ if num_gts > 0:
+ gt_pose[0, :num_gts, :, :3] = np.array(joints_3d)
+ gt_pose[0, :num_gts, :, 3] = np.array(joints_3d_vis)[:, :, 0] - 1.0
+ gt_pose[0, :num_gts, :, 4] = 1.0
+
+ results.append(dict(pose_3d=gt_pose, sample_id=[i]))
+ _ = test_dataset.evaluate(results, metric=['mAP', 'mpjpe'])
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_datasets/test_bottom_up_dataset.py b/engine/pose_estimation/third-party/ViTPose/tests/test_datasets/test_bottom_up_dataset.py
new file mode 100644
index 0000000..ceb2bac
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_datasets/test_bottom_up_dataset.py
@@ -0,0 +1,334 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import numpy as np
+import pytest
+from mmcv import Config
+from numpy.testing import assert_almost_equal
+
+from mmpose.datasets import DATASETS
+
+
+def convert_coco_to_output(coco, is_wholebody=False):
+ results = []
+ for img_id in coco.getImgIds():
+ preds = []
+ scores = []
+ image = coco.imgs[img_id]
+ ann_ids = coco.getAnnIds(img_id)
+ for ann_id in ann_ids:
+ obj = coco.anns[ann_id]
+ if is_wholebody:
+ keypoints = np.array(obj['keypoints'] + obj['foot_kpts'] +
+ obj['face_kpts'] + obj['lefthand_kpts'] +
+ obj['righthand_kpts']).reshape(-1, 3)
+ else:
+ keypoints = np.array(obj['keypoints']).reshape((-1, 3))
+ K = keypoints.shape[0]
+ if sum(keypoints[:, 2]) == 0:
+ continue
+ preds.append(
+ np.concatenate((keypoints[:, :2], np.ones(
+ [K, 1]), np.ones([K, 1]) * ann_id),
+ axis=1))
+ scores.append(1)
+ image_paths = []
+ image_paths.append(image['file_name'])
+
+ output = {}
+ output['preds'] = np.stack(preds)
+ output['scores'] = scores
+ output['image_paths'] = image_paths
+ output['output_heatmap'] = None
+
+ results.append(output)
+
+ return results
+
+
+def test_bottom_up_COCO_dataset():
+ dataset = 'BottomUpCocoDataset'
+ dataset_info = Config.fromfile(
+ 'configs/_base_/datasets/coco.py').dataset_info
+ # test COCO datasets
+ dataset_class = DATASETS.get(dataset)
+
+ channel_cfg = dict(
+ dataset_joints=17,
+ dataset_channel=[
+ [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17],
+ ],
+ inference_channel=[
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17
+ ])
+
+ data_cfg = dict(
+ image_size=512,
+ base_size=256,
+ base_sigma=2,
+ heatmap_size=[128, 256],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'],
+ num_scales=2,
+ scale_aware_sigma=False,
+ use_nms=True)
+
+ _ = dataset_class(
+ ann_file='tests/data/coco/test_coco.json',
+ img_prefix='tests/data/coco/',
+ data_cfg=data_cfg,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=False)
+
+ custom_dataset = dataset_class(
+ ann_file='tests/data/coco/test_coco.json',
+ img_prefix='tests/data/coco/',
+ data_cfg=data_cfg,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=True)
+
+ assert custom_dataset.dataset_name == 'coco'
+ assert custom_dataset.num_images == 4
+ _ = custom_dataset[0]
+
+ results = convert_coco_to_output(custom_dataset.coco)
+
+ infos = custom_dataset.evaluate(results, metric='mAP')
+ assert_almost_equal(infos['AP'], 1.0)
+
+ with pytest.raises(KeyError):
+ _ = custom_dataset.evaluate(results, metric='PCK')
+
+
+def test_bottom_up_CrowdPose_dataset():
+ dataset = 'BottomUpCrowdPoseDataset'
+ dataset_info = Config.fromfile(
+ 'configs/_base_/datasets/crowdpose.py').dataset_info
+ # test CrowdPose datasets
+ dataset_class = DATASETS.get(dataset)
+
+ channel_cfg = dict(
+ num_output_channels=14,
+ dataset_joints=14,
+ dataset_channel=[
+ [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13],
+ ],
+ inference_channel=[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13])
+
+ data_cfg = dict(
+ image_size=512,
+ base_size=256,
+ base_sigma=2,
+ heatmap_size=[128, 256],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'],
+ num_scales=2,
+ scale_aware_sigma=False)
+
+ _ = dataset_class(
+ ann_file='tests/data/crowdpose/test_crowdpose.json',
+ img_prefix='tests/data/crowdpose/',
+ data_cfg=data_cfg,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=False)
+
+ custom_dataset = dataset_class(
+ ann_file='tests/data/crowdpose/test_crowdpose.json',
+ img_prefix='tests/data/crowdpose/',
+ data_cfg=data_cfg,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=True)
+
+ assert custom_dataset.dataset_name == 'crowdpose'
+
+ image_id = 103319
+ assert image_id in custom_dataset.img_ids
+ assert len(custom_dataset.img_ids) == 2
+ _ = custom_dataset[0]
+
+ results = convert_coco_to_output(custom_dataset.coco)
+ infos = custom_dataset.evaluate(results, metric='mAP')
+ assert_almost_equal(infos['AP'], 1.0)
+
+ with pytest.raises(KeyError):
+ _ = custom_dataset.evaluate(results, metric='PCK')
+
+
+def test_bottom_up_MHP_dataset():
+ dataset = 'BottomUpMhpDataset'
+ dataset_info = Config.fromfile(
+ 'configs/_base_/datasets/mhp.py').dataset_info
+ # test MHP datasets
+ dataset_class = DATASETS.get(dataset)
+
+ channel_cfg = dict(
+ dataset_joints=16,
+ dataset_channel=[
+ [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15],
+ ],
+ inference_channel=[
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15
+ ])
+
+ data_cfg = dict(
+ image_size=512,
+ base_size=256,
+ base_sigma=2,
+ heatmap_size=[128],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'],
+ num_scales=1,
+ scale_aware_sigma=False,
+ )
+
+ _ = dataset_class(
+ ann_file='tests/data/mhp/test_mhp.json',
+ img_prefix='tests/data/mhp/',
+ data_cfg=data_cfg,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=False)
+
+ custom_dataset = dataset_class(
+ ann_file='tests/data/mhp/test_mhp.json',
+ img_prefix='tests/data/mhp/',
+ data_cfg=data_cfg,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=True)
+
+ assert custom_dataset.dataset_name == 'mhp'
+
+ image_id = 2889
+ assert image_id in custom_dataset.img_ids
+ assert len(custom_dataset.img_ids) == 2
+ _ = custom_dataset[0]
+
+ results = convert_coco_to_output(custom_dataset.coco)
+ infos = custom_dataset.evaluate(results, metric='mAP')
+ assert_almost_equal(infos['AP'], 1.0)
+
+ with pytest.raises(KeyError):
+ _ = custom_dataset.evaluate(results, metric='PCK')
+
+
+def test_bottom_up_AIC_dataset():
+ dataset = 'BottomUpAicDataset'
+ dataset_info = Config.fromfile(
+ 'configs/_base_/datasets/aic.py').dataset_info
+ # test MHP datasets
+ dataset_class = DATASETS.get(dataset)
+
+ channel_cfg = dict(
+ num_output_channels=14,
+ dataset_joints=14,
+ dataset_channel=[
+ [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13],
+ ],
+ inference_channel=[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13])
+
+ data_cfg = dict(
+ image_size=512,
+ base_size=256,
+ base_sigma=2,
+ heatmap_size=[128],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'],
+ num_scales=1,
+ scale_aware_sigma=False,
+ )
+
+ _ = dataset_class(
+ ann_file='tests/data/aic/test_aic.json',
+ img_prefix='tests/data/aic/',
+ data_cfg=data_cfg,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=False)
+
+ custom_dataset = dataset_class(
+ ann_file='tests/data/aic/test_aic.json',
+ img_prefix='tests/data/aic/',
+ data_cfg=data_cfg,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=True)
+
+ assert custom_dataset.dataset_name == 'aic'
+
+ image_id = 1
+ assert image_id in custom_dataset.img_ids
+ assert len(custom_dataset.img_ids) == 3
+ _ = custom_dataset[0]
+
+ results = convert_coco_to_output(custom_dataset.coco)
+ infos = custom_dataset.evaluate(results, metric='mAP')
+ assert_almost_equal(infos['AP'], 1.0)
+
+ with pytest.raises(KeyError):
+ _ = custom_dataset.evaluate(results, metric='PCK')
+
+
+def test_bottom_up_COCO_wholebody_dataset():
+ dataset = 'BottomUpCocoWholeBodyDataset'
+ dataset_info = Config.fromfile(
+ 'configs/_base_/datasets/coco_wholebody.py').dataset_info
+ # test COCO-wholebody datasets
+ dataset_class = DATASETS.get(dataset)
+
+ channel_cfg = dict(
+ num_output_channels=133,
+ dataset_joints=133,
+ dataset_channel=[
+ list(range(133)),
+ ],
+ inference_channel=list(range(133)))
+
+ data_cfg = dict(
+ image_size=512,
+ base_size=256,
+ base_sigma=2,
+ heatmap_size=[128, 256],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'],
+ num_scales=2,
+ scale_aware_sigma=False,
+ )
+
+ _ = dataset_class(
+ ann_file='tests/data/coco/test_coco_wholebody.json',
+ img_prefix='tests/data/coco/',
+ data_cfg=data_cfg,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=False)
+
+ custom_dataset = dataset_class(
+ ann_file='tests/data/coco/test_coco_wholebody.json',
+ img_prefix='tests/data/coco/',
+ data_cfg=data_cfg,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=True)
+
+ assert custom_dataset.test_mode is True
+ assert custom_dataset.dataset_name == 'coco_wholebody'
+
+ image_id = 785
+ assert image_id in custom_dataset.img_ids
+ assert len(custom_dataset.img_ids) == 4
+ _ = custom_dataset[0]
+
+ results = convert_coco_to_output(custom_dataset.coco, is_wholebody=True)
+ infos = custom_dataset.evaluate(results, metric='mAP')
+ assert_almost_equal(infos['AP'], 1.0)
+
+ with pytest.raises(KeyError):
+ _ = custom_dataset.evaluate(results, metric='PCK')
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_datasets/test_dataset_info.py b/engine/pose_estimation/third-party/ViTPose/tests/test_datasets/test_dataset_info.py
new file mode 100644
index 0000000..d939b9d
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_datasets/test_dataset_info.py
@@ -0,0 +1,77 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from mmpose.datasets import DatasetInfo
+
+
+def test_dataset_info():
+ dataset_info = dict(
+ dataset_name='zebra',
+ paper_info=dict(
+ author='Graving, Jacob M and Chae, Daniel and Naik, Hemal and '
+ 'Li, Liang and Koger, Benjamin and Costelloe, Blair R and '
+ 'Couzin, Iain D',
+ title='DeepPoseKit, a software toolkit for fast and robust '
+ 'animal pose estimation using deep learning',
+ container='Elife',
+ year='2019',
+ homepage='https://github.com/jgraving/DeepPoseKit-Data',
+ ),
+ keypoint_info={
+ 0:
+ dict(name='snout', id=0, color=[255, 255, 255], type='', swap=''),
+ 1:
+ dict(name='head', id=1, color=[255, 255, 255], type='', swap=''),
+ 2:
+ dict(name='neck', id=2, color=[255, 255, 255], type='', swap=''),
+ 3:
+ dict(
+ name='forelegL1',
+ id=3,
+ color=[255, 255, 255],
+ type='',
+ swap='forelegR1'),
+ 4:
+ dict(
+ name='forelegR1',
+ id=4,
+ color=[255, 255, 255],
+ type='',
+ swap='forelegL1'),
+ 5:
+ dict(
+ name='hindlegL1',
+ id=5,
+ color=[255, 255, 255],
+ type='',
+ swap='hindlegR1'),
+ 6:
+ dict(
+ name='hindlegR1',
+ id=6,
+ color=[255, 255, 255],
+ type='',
+ swap='hindlegL1'),
+ 7:
+ dict(
+ name='tailbase', id=7, color=[255, 255, 255], type='',
+ swap=''),
+ 8:
+ dict(
+ name='tailtip', id=8, color=[255, 255, 255], type='', swap='')
+ },
+ skeleton_info={
+ 0: dict(link=('head', 'snout'), id=0, color=[255, 255, 255]),
+ 1: dict(link=('neck', 'head'), id=1, color=[255, 255, 255]),
+ 2: dict(link=('forelegL1', 'neck'), id=2, color=[255, 255, 255]),
+ 3: dict(link=('forelegR1', 'neck'), id=3, color=[255, 255, 255]),
+ 4:
+ dict(link=('hindlegL1', 'tailbase'), id=4, color=[255, 255, 255]),
+ 5:
+ dict(link=('hindlegR1', 'tailbase'), id=5, color=[255, 255, 255]),
+ 6: dict(link=('tailbase', 'neck'), id=6, color=[255, 255, 255]),
+ 7: dict(link=('tailtip', 'tailbase'), id=7, color=[255, 255, 255])
+ },
+ joint_weights=[1.] * 9,
+ sigmas=[])
+
+ dataset_info = DatasetInfo(dataset_info)
+ assert dataset_info.keypoint_num == len(dataset_info.flip_index)
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_datasets/test_dataset_wrapper.py b/engine/pose_estimation/third-party/ViTPose/tests/test_datasets/test_dataset_wrapper.py
new file mode 100644
index 0000000..f724d25
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_datasets/test_dataset_wrapper.py
@@ -0,0 +1,67 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from mmcv import Config
+
+from mmpose.datasets.builder import build_dataset
+
+
+def test_concat_dataset():
+ # build COCO-like dataset config
+ dataset_info = Config.fromfile(
+ 'configs/_base_/datasets/coco.py').dataset_info
+
+ channel_cfg = dict(
+ num_output_channels=17,
+ dataset_joints=17,
+ dataset_channel=[
+ [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16],
+ ],
+ inference_channel=[
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16
+ ])
+
+ data_cfg = dict(
+ image_size=[192, 256],
+ heatmap_size=[48, 64],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'],
+ soft_nms=False,
+ nms_thr=1.0,
+ oks_thr=0.9,
+ vis_thr=0.2,
+ use_gt_bbox=True,
+ det_bbox_thr=0.0,
+ bbox_file='tests/data/coco/test_coco_det_AP_H_56.json',
+ )
+
+ dataset_cfg = dict(
+ type='TopDownCocoDataset',
+ ann_file='tests/data/coco/test_coco.json',
+ img_prefix='tests/data/coco/',
+ data_cfg=data_cfg,
+ pipeline=[],
+ dataset_info=dataset_info)
+
+ dataset = build_dataset(dataset_cfg)
+
+ # Case 1: build ConcatDataset explicitly
+ concat_dataset_cfg = dict(
+ type='ConcatDataset', datasets=[dataset_cfg, dataset_cfg])
+ concat_dataset = build_dataset(concat_dataset_cfg)
+ assert len(concat_dataset) == 2 * len(dataset)
+
+ # Case 2: build ConcatDataset from cfg sequence
+ concat_dataset = build_dataset([dataset_cfg, dataset_cfg])
+ assert len(concat_dataset) == 2 * len(dataset)
+
+ # Case 3: build ConcatDataset from ann_file sequence
+ concat_dataset_cfg = dataset_cfg.copy()
+ for key in ['ann_file', 'type', 'img_prefix', 'dataset_info']:
+ val = concat_dataset_cfg[key]
+ concat_dataset_cfg[key] = [val] * 2
+ for key in ['num_joints', 'dataset_channel']:
+ val = concat_dataset_cfg['data_cfg'][key]
+ concat_dataset_cfg['data_cfg'][key] = [val] * 2
+ concat_dataset = build_dataset(concat_dataset_cfg)
+ assert len(concat_dataset) == 2 * len(dataset)
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_datasets/test_face_dataset.py b/engine/pose_estimation/third-party/ViTPose/tests/test_datasets/test_face_dataset.py
new file mode 100644
index 0000000..4fa30b2
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_datasets/test_face_dataset.py
@@ -0,0 +1,284 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import copy
+from unittest.mock import MagicMock
+
+import pytest
+from mmcv import Config
+from numpy.testing import assert_almost_equal
+
+from mmpose.datasets import DATASETS
+from tests.utils.data_utils import convert_db_to_output
+
+
+def test_face_300W_dataset():
+ dataset = 'Face300WDataset'
+ dataset_info = Config.fromfile(
+ 'configs/_base_/datasets/300w.py').dataset_info
+ # test Face 300W datasets
+ dataset_class = DATASETS.get(dataset)
+ dataset_class.load_annotations = MagicMock()
+ dataset_class.coco = MagicMock()
+
+ channel_cfg = dict(
+ num_output_channels=68,
+ dataset_joints=68,
+ dataset_channel=[
+ list(range(68)),
+ ],
+ inference_channel=list(range(68)))
+
+ data_cfg = dict(
+ image_size=[256, 256],
+ heatmap_size=[64, 64],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'])
+ # Test
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ _ = dataset_class(
+ ann_file='tests/data/300w/test_300w.json',
+ img_prefix='tests/data/300w/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=True)
+
+ custom_dataset = dataset_class(
+ ann_file='tests/data/300w/test_300w.json',
+ img_prefix='tests/data/300w/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=False)
+
+ assert custom_dataset.dataset_name == '300w'
+ assert custom_dataset.test_mode is False
+ assert custom_dataset.num_images == 2
+ _ = custom_dataset[0]
+
+ results = convert_db_to_output(custom_dataset.db)
+ infos = custom_dataset.evaluate(results, metric=['NME'])
+ assert_almost_equal(infos['NME'], 0.0)
+
+ with pytest.raises(KeyError):
+ _ = custom_dataset.evaluate(results, metric='mAP')
+
+
+def test_face_coco_wholebody_dataset():
+ dataset = 'FaceCocoWholeBodyDataset'
+ dataset_info = Config.fromfile(
+ 'configs/_base_/datasets/coco_wholebody_face.py').dataset_info
+ # test Face wholebody datasets
+ dataset_class = DATASETS.get(dataset)
+ dataset_class.load_annotations = MagicMock()
+ dataset_class.coco = MagicMock()
+
+ channel_cfg = dict(
+ num_output_channels=68,
+ dataset_joints=68,
+ dataset_channel=[
+ list(range(68)),
+ ],
+ inference_channel=list(range(68)))
+
+ data_cfg = dict(
+ image_size=[256, 256],
+ heatmap_size=[64, 64],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'])
+ # Test
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ _ = dataset_class(
+ ann_file='tests/data/coco/test_coco_wholebody.json',
+ img_prefix='tests/data/coco/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=True)
+
+ custom_dataset = dataset_class(
+ ann_file='tests/data/coco/test_coco_wholebody.json',
+ img_prefix='tests/data/coco/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=False)
+
+ assert custom_dataset.test_mode is False
+ assert custom_dataset.num_images == 4
+ _ = custom_dataset[0]
+
+ results = convert_db_to_output(custom_dataset.db)
+ infos = custom_dataset.evaluate(results, metric=['NME'])
+ assert_almost_equal(infos['NME'], 0.0)
+
+ with pytest.raises(KeyError):
+ _ = custom_dataset.evaluate(results, metric='mAP')
+
+
+def test_face_AFLW_dataset():
+ dataset = 'FaceAFLWDataset'
+ dataset_info = Config.fromfile(
+ 'configs/_base_/datasets/aflw.py').dataset_info
+ # test Face AFLW datasets
+ dataset_class = DATASETS.get(dataset)
+ dataset_class.load_annotations = MagicMock()
+ dataset_class.coco = MagicMock()
+
+ channel_cfg = dict(
+ num_output_channels=19,
+ dataset_joints=19,
+ dataset_channel=[
+ list(range(19)),
+ ],
+ inference_channel=list(range(19)))
+
+ data_cfg = dict(
+ image_size=[256, 256],
+ heatmap_size=[64, 64],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'])
+ # Test
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ _ = dataset_class(
+ ann_file='tests/data/aflw/test_aflw.json',
+ img_prefix='tests/data/aflw/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=True)
+
+ custom_dataset = dataset_class(
+ ann_file='tests/data/aflw/test_aflw.json',
+ img_prefix='tests/data/aflw/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=False)
+
+ assert custom_dataset.dataset_name == 'aflw'
+ assert custom_dataset.test_mode is False
+ assert custom_dataset.num_images == 2
+ _ = custom_dataset[0]
+
+ results = convert_db_to_output(custom_dataset.db)
+ infos = custom_dataset.evaluate(results, metric=['NME'])
+ assert_almost_equal(infos['NME'], 0.0)
+
+ with pytest.raises(KeyError):
+ _ = custom_dataset.evaluate(results, metric='mAP')
+
+
+def test_face_WFLW_dataset():
+ dataset = 'FaceWFLWDataset'
+ dataset_info = Config.fromfile(
+ 'configs/_base_/datasets/wflw.py').dataset_info
+ # test Face WFLW datasets
+ dataset_class = DATASETS.get(dataset)
+ dataset_class.load_annotations = MagicMock()
+ dataset_class.coco = MagicMock()
+
+ channel_cfg = dict(
+ num_output_channels=98,
+ dataset_joints=98,
+ dataset_channel=[
+ list(range(98)),
+ ],
+ inference_channel=list(range(98)))
+
+ data_cfg = dict(
+ image_size=[256, 256],
+ heatmap_size=[64, 64],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'])
+ # Test
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ _ = dataset_class(
+ ann_file='tests/data/wflw/test_wflw.json',
+ img_prefix='tests/data/wflw/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=True)
+
+ custom_dataset = dataset_class(
+ ann_file='tests/data/wflw/test_wflw.json',
+ img_prefix='tests/data/wflw/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=False)
+
+ assert custom_dataset.dataset_name == 'wflw'
+ assert custom_dataset.test_mode is False
+ assert custom_dataset.num_images == 2
+ _ = custom_dataset[0]
+
+ results = convert_db_to_output(custom_dataset.db)
+ infos = custom_dataset.evaluate(results, metric=['NME'])
+ assert_almost_equal(infos['NME'], 0.0)
+
+ with pytest.raises(KeyError):
+ _ = custom_dataset.evaluate(results, metric='mAP')
+
+
+def test_face_COFW_dataset():
+ dataset = 'FaceCOFWDataset'
+ dataset_info = Config.fromfile(
+ 'configs/_base_/datasets/cofw.py').dataset_info
+ # test Face COFW datasets
+ dataset_class = DATASETS.get(dataset)
+ dataset_class.load_annotations = MagicMock()
+ dataset_class.coco = MagicMock()
+
+ channel_cfg = dict(
+ num_output_channels=29,
+ dataset_joints=29,
+ dataset_channel=[
+ list(range(29)),
+ ],
+ inference_channel=list(range(29)))
+
+ data_cfg = dict(
+ image_size=[256, 256],
+ heatmap_size=[64, 64],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'])
+ # Test
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ _ = dataset_class(
+ ann_file='tests/data/cofw/test_cofw.json',
+ img_prefix='tests/data/cofw/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=True)
+
+ custom_dataset = dataset_class(
+ ann_file='tests/data/cofw/test_cofw.json',
+ img_prefix='tests/data/cofw/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=False)
+
+ assert custom_dataset.dataset_name == 'cofw'
+ assert custom_dataset.test_mode is False
+ assert custom_dataset.num_images == 2
+ _ = custom_dataset[0]
+
+ results = convert_db_to_output(custom_dataset.db)
+ infos = custom_dataset.evaluate(results, metric=['NME'])
+ assert_almost_equal(infos['NME'], 0.0)
+
+ with pytest.raises(KeyError):
+ _ = custom_dataset.evaluate(results, metric='mAP')
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_datasets/test_fashion_dataset.py b/engine/pose_estimation/third-party/ViTPose/tests/test_datasets/test_fashion_dataset.py
new file mode 100644
index 0000000..8f5cdc8
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_datasets/test_fashion_dataset.py
@@ -0,0 +1,70 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from unittest.mock import MagicMock
+
+import pytest
+from mmcv import Config
+from numpy.testing import assert_almost_equal
+
+from mmpose.datasets import DATASETS
+from tests.utils.data_utils import convert_db_to_output
+
+
+def test_deepfashion_dataset():
+ dataset = 'DeepFashionDataset'
+ dataset_info = Config.fromfile(
+ 'configs/_base_/datasets/deepfashion_full.py').dataset_info
+ # test JHMDB datasets
+ dataset_class = DATASETS.get(dataset)
+ dataset_class.load_annotations = MagicMock()
+ dataset_class.coco = MagicMock()
+
+ channel_cfg = dict(
+ num_output_channels=8,
+ dataset_joints=8,
+ dataset_channel=[
+ [0, 1, 2, 3, 4, 5, 6, 7],
+ ],
+ inference_channel=[0, 1, 2, 3, 4, 5, 6, 7])
+
+ data_cfg = dict(
+ image_size=[192, 256],
+ heatmap_size=[48, 64],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'],
+ soft_nms=False,
+ nms_thr=1.0,
+ oks_thr=0.9,
+ vis_thr=0.2,
+ use_gt_bbox=True,
+ det_bbox_thr=0.0,
+ image_thr=0.0,
+ bbox_file='')
+
+ # Test gt bbox
+ custom_dataset = dataset_class(
+ ann_file='tests/data/fld/test_fld.json',
+ img_prefix='tests/data/fld/',
+ subset='full',
+ data_cfg=data_cfg,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=True)
+
+ assert custom_dataset.test_mode is True
+ assert custom_dataset.dataset_name == 'deepfashion_full'
+
+ image_id = 128
+ assert image_id in custom_dataset.img_ids
+ assert len(custom_dataset.img_ids) == 2
+ _ = custom_dataset[0]
+
+ results = convert_db_to_output(custom_dataset.db)
+ infos = custom_dataset.evaluate(results, metric=['PCK', 'EPE', 'AUC'])
+ assert_almost_equal(infos['PCK'], 1.0)
+ assert_almost_equal(infos['AUC'], 0.95)
+ assert_almost_equal(infos['EPE'], 0.0)
+
+ with pytest.raises(KeyError):
+ infos = custom_dataset.evaluate(results, metric='mAP')
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_datasets/test_hand_dataset.py b/engine/pose_estimation/third-party/ViTPose/tests/test_datasets/test_hand_dataset.py
new file mode 100644
index 0000000..6f4bb1c
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_datasets/test_hand_dataset.py
@@ -0,0 +1,456 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import copy
+
+import pytest
+from mmcv import Config
+from numpy.testing import assert_almost_equal
+
+from mmpose.datasets import DATASETS
+from tests.utils.data_utils import convert_db_to_output
+
+
+def test_OneHand10K_dataset():
+ dataset = 'OneHand10KDataset'
+ dataset_info = Config.fromfile(
+ 'configs/_base_/datasets/onehand10k.py').dataset_info
+
+ dataset_class = DATASETS.get(dataset)
+
+ channel_cfg = dict(
+ num_output_channels=21,
+ dataset_joints=21,
+ dataset_channel=[
+ [
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17,
+ 18, 19, 20
+ ],
+ ],
+ inference_channel=[
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18,
+ 19, 20
+ ])
+
+ data_cfg = dict(
+ image_size=[256, 256],
+ heatmap_size=[64, 64],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'])
+ # Test
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ _ = dataset_class(
+ ann_file='tests/data/onehand10k/test_onehand10k.json',
+ img_prefix='tests/data/onehand10k/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=True)
+
+ custom_dataset = dataset_class(
+ ann_file='tests/data/onehand10k/test_onehand10k.json',
+ img_prefix='tests/data/onehand10k/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=False)
+
+ assert custom_dataset.dataset_name == 'onehand10k'
+ assert custom_dataset.test_mode is False
+ assert custom_dataset.num_images == 4
+ _ = custom_dataset[0]
+
+ results = convert_db_to_output(custom_dataset.db)
+ infos = custom_dataset.evaluate(results, metric=['PCK', 'EPE', 'AUC'])
+ assert_almost_equal(infos['PCK'], 1.0)
+ assert_almost_equal(infos['AUC'], 0.95)
+ assert_almost_equal(infos['EPE'], 0.0)
+
+ with pytest.raises(KeyError):
+ infos = custom_dataset.evaluate(results, metric='mAP')
+
+
+def test_hand_coco_wholebody_dataset():
+ dataset = 'HandCocoWholeBodyDataset'
+ dataset_info = Config.fromfile(
+ 'configs/_base_/datasets/coco_wholebody_hand.py').dataset_info
+ dataset_class = DATASETS.get(dataset)
+
+ channel_cfg = dict(
+ num_output_channels=21,
+ dataset_joints=21,
+ dataset_channel=[
+ [
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17,
+ 18, 19, 20
+ ],
+ ],
+ inference_channel=[
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18,
+ 19, 20
+ ])
+
+ data_cfg = dict(
+ image_size=[256, 256],
+ heatmap_size=[64, 64],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'])
+ # Test
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ _ = dataset_class(
+ ann_file='tests/data/coco/test_coco_wholebody.json',
+ img_prefix='tests/data/coco/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=True)
+
+ custom_dataset = dataset_class(
+ ann_file='tests/data/coco/test_coco_wholebody.json',
+ img_prefix='tests/data/coco/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=False)
+
+ assert custom_dataset.test_mode is False
+ assert custom_dataset.num_images == 4
+ _ = custom_dataset[0]
+
+ results = convert_db_to_output(custom_dataset.db)
+ infos = custom_dataset.evaluate(results, metric=['PCK', 'EPE', 'AUC'])
+ assert_almost_equal(infos['PCK'], 1.0)
+ assert_almost_equal(infos['AUC'], 0.95)
+ assert_almost_equal(infos['EPE'], 0.0)
+
+ with pytest.raises(KeyError):
+ infos = custom_dataset.evaluate(results, metric='mAP')
+
+
+def test_FreiHand2D_dataset():
+ dataset = 'FreiHandDataset'
+ dataset_info = Config.fromfile(
+ 'configs/_base_/datasets/freihand2d.py').dataset_info
+
+ dataset_class = DATASETS.get(dataset)
+
+ channel_cfg = dict(
+ num_output_channels=21,
+ dataset_joints=21,
+ dataset_channel=[
+ [
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17,
+ 18, 19, 20
+ ],
+ ],
+ inference_channel=[
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18,
+ 19, 20
+ ])
+
+ data_cfg = dict(
+ image_size=[224, 224],
+ heatmap_size=[56, 56],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'])
+ # Test
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ _ = dataset_class(
+ ann_file='tests/data/freihand/test_freihand.json',
+ img_prefix='tests/data/freihand/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=True)
+
+ custom_dataset = dataset_class(
+ ann_file='tests/data/freihand/test_freihand.json',
+ img_prefix='tests/data/freihand/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=False)
+
+ assert custom_dataset.dataset_name == 'freihand'
+ assert custom_dataset.test_mode is False
+ assert custom_dataset.num_images == 8
+ _ = custom_dataset[0]
+
+ results = convert_db_to_output(custom_dataset.db)
+ infos = custom_dataset.evaluate(results, metric=['PCK', 'EPE', 'AUC'])
+ assert_almost_equal(infos['PCK'], 1.0)
+ assert_almost_equal(infos['AUC'], 0.95)
+ assert_almost_equal(infos['EPE'], 0.0)
+
+ with pytest.raises(KeyError):
+ infos = custom_dataset.evaluate(results, metric='mAP')
+
+
+def test_RHD2D_dataset():
+ dataset = 'Rhd2DDataset'
+ dataset_info = Config.fromfile(
+ 'configs/_base_/datasets/rhd2d.py').dataset_info
+
+ dataset_class = DATASETS.get(dataset)
+
+ channel_cfg = dict(
+ num_output_channels=21,
+ dataset_joints=21,
+ dataset_channel=[
+ [
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17,
+ 18, 19, 20
+ ],
+ ],
+ inference_channel=[
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18,
+ 19, 20
+ ])
+
+ data_cfg = dict(
+ image_size=[256, 256],
+ heatmap_size=[64, 64],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'])
+ # Test
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ _ = dataset_class(
+ ann_file='tests/data/rhd/test_rhd.json',
+ img_prefix='tests/data/rhd/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=True)
+
+ custom_dataset = dataset_class(
+ ann_file='tests/data/rhd/test_rhd.json',
+ img_prefix='tests/data/rhd/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=False)
+
+ assert custom_dataset.dataset_name == 'rhd2d'
+ assert custom_dataset.test_mode is False
+ assert custom_dataset.num_images == 3
+ _ = custom_dataset[0]
+
+ results = convert_db_to_output(custom_dataset.db)
+ infos = custom_dataset.evaluate(results, metric=['PCK', 'EPE', 'AUC'])
+ assert_almost_equal(infos['PCK'], 1.0)
+ assert_almost_equal(infos['AUC'], 0.95)
+ assert_almost_equal(infos['EPE'], 0.0)
+
+ with pytest.raises(KeyError):
+ infos = custom_dataset.evaluate(results, metric='mAP')
+
+
+def test_Panoptic2D_dataset():
+ dataset = 'PanopticDataset'
+ dataset_info = Config.fromfile(
+ 'configs/_base_/datasets/panoptic_hand2d.py').dataset_info
+
+ dataset_class = DATASETS.get(dataset)
+
+ channel_cfg = dict(
+ num_output_channels=21,
+ dataset_joints=21,
+ dataset_channel=[
+ [
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17,
+ 18, 19, 20
+ ],
+ ],
+ inference_channel=[
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18,
+ 19, 20
+ ])
+
+ data_cfg = dict(
+ image_size=[256, 256],
+ heatmap_size=[64, 64],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'])
+ # Test
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ _ = dataset_class(
+ ann_file='tests/data/panoptic/test_panoptic.json',
+ img_prefix='tests/data/panoptic/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=True)
+
+ custom_dataset = dataset_class(
+ ann_file='tests/data/panoptic/test_panoptic.json',
+ img_prefix='tests/data/panoptic/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=False)
+
+ assert custom_dataset.dataset_name == 'panoptic_hand2d'
+ assert custom_dataset.test_mode is False
+ assert custom_dataset.num_images == 4
+ _ = custom_dataset[0]
+
+ results = convert_db_to_output(custom_dataset.db)
+ infos = custom_dataset.evaluate(results, metric=['PCKh', 'EPE', 'AUC'])
+ assert_almost_equal(infos['PCKh'], 1.0)
+ assert_almost_equal(infos['AUC'], 0.95)
+ assert_almost_equal(infos['EPE'], 0.0)
+
+ with pytest.raises(KeyError):
+ infos = custom_dataset.evaluate(results, metric='mAP')
+
+
+def test_InterHand2D_dataset():
+ dataset = 'InterHand2DDataset'
+ dataset_info = Config.fromfile(
+ 'configs/_base_/datasets/interhand2d.py').dataset_info
+
+ dataset_class = DATASETS.get(dataset)
+
+ channel_cfg = dict(
+ num_output_channels=21,
+ dataset_joints=21,
+ dataset_channel=[
+ [
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17,
+ 18, 19, 20
+ ],
+ ],
+ inference_channel=[
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18,
+ 19, 20
+ ])
+
+ data_cfg = dict(
+ image_size=[256, 256],
+ heatmap_size=[64, 64],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'])
+ # Test
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ _ = dataset_class(
+ ann_file='tests/data/interhand2.6m/test_interhand2.6m_data.json',
+ camera_file='tests/data/interhand2.6m/test_interhand2.6m_camera.json',
+ joint_file='tests/data/interhand2.6m/test_interhand2.6m_joint_3d.json',
+ img_prefix='tests/data/interhand2.6m/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=True)
+
+ custom_dataset = dataset_class(
+ ann_file='tests/data/interhand2.6m/test_interhand2.6m_data.json',
+ camera_file='tests/data/interhand2.6m/test_interhand2.6m_camera.json',
+ joint_file='tests/data/interhand2.6m/test_interhand2.6m_joint_3d.json',
+ img_prefix='tests/data/interhand2.6m/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=False)
+
+ assert custom_dataset.dataset_name == 'interhand2d'
+ assert custom_dataset.test_mode is False
+ assert custom_dataset.num_images == 4
+ assert len(custom_dataset.db) == 6
+
+ _ = custom_dataset[0]
+
+ results = convert_db_to_output(custom_dataset.db)
+ infos = custom_dataset.evaluate(results, metric=['PCK', 'EPE', 'AUC'])
+ print(infos, flush=True)
+ assert_almost_equal(infos['PCK'], 1.0)
+ assert_almost_equal(infos['AUC'], 0.95)
+ assert_almost_equal(infos['EPE'], 0.0)
+
+ with pytest.raises(KeyError):
+ infos = custom_dataset.evaluate(results, metric='mAP')
+
+
+def test_InterHand3D_dataset():
+ dataset = 'InterHand3DDataset'
+ dataset_info = Config.fromfile(
+ 'configs/_base_/datasets/interhand3d.py').dataset_info
+
+ dataset_class = DATASETS.get(dataset)
+
+ channel_cfg = dict(
+ num_output_channels=42,
+ dataset_joints=42,
+ dataset_channel=[
+ [
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17,
+ 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33,
+ 34, 35, 36, 37, 38, 39, 40, 41
+ ],
+ ],
+ inference_channel=[
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18,
+ 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35,
+ 36, 37, 38, 39, 40, 41
+ ])
+
+ data_cfg = dict(
+ image_size=[256, 256],
+ heatmap_size=[64, 64, 64],
+ heatmap3d_depth_bound=400.0,
+ heatmap_size_root=64,
+ root_depth_bound=400.0,
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'])
+ # Test
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ _ = dataset_class(
+ ann_file='tests/data/interhand2.6m/test_interhand2.6m_data.json',
+ camera_file='tests/data/interhand2.6m/test_interhand2.6m_camera.json',
+ joint_file='tests/data/interhand2.6m/test_interhand2.6m_joint_3d.json',
+ img_prefix='tests/data/interhand2.6m/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=True)
+
+ custom_dataset = dataset_class(
+ ann_file='tests/data/interhand2.6m/test_interhand2.6m_data.json',
+ camera_file='tests/data/interhand2.6m/test_interhand2.6m_camera.json',
+ joint_file='tests/data/interhand2.6m/test_interhand2.6m_joint_3d.json',
+ img_prefix='tests/data/interhand2.6m/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=False)
+
+ assert custom_dataset.dataset_name == 'interhand3d'
+ assert custom_dataset.test_mode is False
+ assert custom_dataset.num_images == 4
+ assert len(custom_dataset.db) == 4
+
+ _ = custom_dataset[0]
+
+ results = convert_db_to_output(
+ custom_dataset.db, keys=['rel_root_depth', 'hand_type'], is_3d=True)
+ infos = custom_dataset.evaluate(
+ results, metric=['MRRPE', 'MPJPE', 'Handedness_acc'])
+ assert_almost_equal(infos['MRRPE'], 0.0, decimal=5)
+ assert_almost_equal(infos['MPJPE_all'], 0.0, decimal=5)
+ assert_almost_equal(infos['MPJPE_single'], 0.0, decimal=5)
+ assert_almost_equal(infos['MPJPE_interacting'], 0.0, decimal=5)
+ assert_almost_equal(infos['Handedness_acc'], 1.0)
+
+ with pytest.raises(KeyError):
+ infos = custom_dataset.evaluate(results, metric='mAP')
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_datasets/test_mesh_dataset.py b/engine/pose_estimation/third-party/ViTPose/tests/test_datasets/test_mesh_dataset.py
new file mode 100644
index 0000000..59938a0
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_datasets/test_mesh_dataset.py
@@ -0,0 +1,127 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import tempfile
+
+from mmpose.datasets import DATASETS
+
+
+def test_mesh_Mosh_dataset():
+ # test Mosh dataset
+ dataset = 'MoshDataset'
+ dataset_class = DATASETS.get(dataset)
+
+ custom_dataset = dataset_class(
+ ann_file='tests/data/mosh/test_mosh.npz', pipeline=[])
+
+ _ = custom_dataset[0]
+
+
+def test_mesh_H36M_dataset():
+ # test H36M dataset
+ dataset = 'MeshH36MDataset'
+ dataset_class = DATASETS.get(dataset)
+
+ data_cfg = dict(
+ image_size=[256, 256],
+ iuv_size=[64, 64],
+ num_joints=24,
+ use_IUV=True,
+ uv_type='BF')
+ _ = dataset_class(
+ ann_file='tests/data/h36m/test_h36m.npz',
+ img_prefix='tests/data/h36m',
+ data_cfg=data_cfg,
+ pipeline=[],
+ test_mode=False)
+
+ custom_dataset = dataset_class(
+ ann_file='tests/data/h36m/test_h36m.npz',
+ img_prefix='tests/data/h36m',
+ data_cfg=data_cfg,
+ pipeline=[],
+ test_mode=True)
+
+ assert custom_dataset.test_mode is True
+ _ = custom_dataset[0]
+
+ # test evaluation
+ outputs = []
+ for item in custom_dataset:
+ pred = dict(
+ keypoints_3d=item['joints_3d'][None, ...],
+ image_path=item['image_file'])
+ outputs.append(pred)
+ with tempfile.TemporaryDirectory() as tmpdir:
+ eval_result = custom_dataset.evaluate(outputs, tmpdir)
+ assert 'MPJPE' in eval_result
+ assert 'MPJPE-PA' in eval_result
+
+
+def test_mesh_Mix_dataset():
+ # test mesh Mix dataset
+
+ dataset = 'MeshMixDataset'
+ dataset_class = DATASETS.get(dataset)
+
+ data_cfg = dict(
+ image_size=[256, 256],
+ iuv_size=[64, 64],
+ num_joints=24,
+ use_IUV=True,
+ uv_type='BF')
+
+ custom_dataset = dataset_class(
+ configs=[
+ dict(
+ ann_file='tests/data/h36m/test_h36m.npz',
+ img_prefix='tests/data/h36m',
+ data_cfg=data_cfg,
+ pipeline=[]),
+ dict(
+ ann_file='tests/data/h36m/test_h36m.npz',
+ img_prefix='tests/data/h36m',
+ data_cfg=data_cfg,
+ pipeline=[]),
+ ],
+ partition=[0.6, 0.4])
+
+ _ = custom_dataset[0]
+
+
+def test_mesh_Adversarial_dataset():
+ # test mesh Adversarial dataset
+
+ # load train dataset
+ data_cfg = dict(
+ image_size=[256, 256],
+ iuv_size=[64, 64],
+ num_joints=24,
+ use_IUV=True,
+ uv_type='BF')
+ train_dataset = dict(
+ type='MeshMixDataset',
+ configs=[
+ dict(
+ ann_file='tests/data/h36m/test_h36m.npz',
+ img_prefix='tests/data/h36m',
+ data_cfg=data_cfg,
+ pipeline=[]),
+ dict(
+ ann_file='tests/data/h36m/test_h36m.npz',
+ img_prefix='tests/data/h36m',
+ data_cfg=data_cfg,
+ pipeline=[]),
+ ],
+ partition=[0.6, 0.4])
+
+ # load adversarial dataset
+ adversarial_dataset = dict(
+ type='MoshDataset',
+ ann_file='tests/data/mosh/test_mosh.npz',
+ pipeline=[])
+
+ # combine train and adversarial dataset to form a new dataset
+ dataset = 'MeshAdversarialDataset'
+ dataset_class = DATASETS.get(dataset)
+ custom_dataset = dataset_class(train_dataset, adversarial_dataset)
+ item = custom_dataset[0]
+ assert 'mosh_theta' in item.keys()
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_datasets/test_top_down_dataset.py b/engine/pose_estimation/third-party/ViTPose/tests/test_datasets/test_top_down_dataset.py
new file mode 100644
index 0000000..35c1a99
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_datasets/test_top_down_dataset.py
@@ -0,0 +1,1022 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import copy
+from unittest.mock import MagicMock
+
+import pytest
+from mmcv import Config
+from numpy.testing import assert_almost_equal
+
+from mmpose.datasets import DATASETS
+from tests.utils.data_utils import convert_db_to_output
+
+
+def test_top_down_COCO_dataset():
+ dataset = 'TopDownCocoDataset'
+ dataset_info = Config.fromfile(
+ 'configs/_base_/datasets/coco.py').dataset_info
+ # test COCO datasets
+ dataset_class = DATASETS.get(dataset)
+ dataset_class.load_annotations = MagicMock()
+ dataset_class.coco = MagicMock()
+
+ channel_cfg = dict(
+ num_output_channels=17,
+ dataset_joints=17,
+ dataset_channel=[
+ [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16],
+ ],
+ inference_channel=[
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16
+ ])
+
+ data_cfg = dict(
+ image_size=[192, 256],
+ heatmap_size=[48, 64],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'],
+ soft_nms=False,
+ nms_thr=1.0,
+ oks_thr=0.9,
+ vis_thr=0.2,
+ use_gt_bbox=True,
+ det_bbox_thr=0.0,
+ bbox_file='tests/data/coco/test_coco_det_AP_H_56.json',
+ )
+ # Test det bbox
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ data_cfg_copy['use_gt_bbox'] = False
+ _ = dataset_class(
+ ann_file='tests/data/coco/test_coco.json',
+ img_prefix='tests/data/coco/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=True)
+
+ _ = dataset_class(
+ ann_file='tests/data/coco/test_coco.json',
+ img_prefix='tests/data/coco/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=False)
+
+ # Test gt bbox
+ custom_dataset = dataset_class(
+ ann_file='tests/data/coco/test_coco.json',
+ img_prefix='tests/data/coco/',
+ data_cfg=data_cfg,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=True)
+
+ assert custom_dataset.test_mode is True
+ assert custom_dataset.dataset_name == 'coco'
+
+ image_id = 785
+ assert image_id in custom_dataset.img_ids
+ assert len(custom_dataset.img_ids) == 4
+ _ = custom_dataset[0]
+
+ results = convert_db_to_output(custom_dataset.db)
+ infos = custom_dataset.evaluate(results, metric='mAP')
+ assert_almost_equal(infos['AP'], 1.0)
+
+ with pytest.raises(KeyError):
+ _ = custom_dataset.evaluate(results, metric='PCK')
+
+
+def test_top_down_MHP_dataset():
+ dataset = 'TopDownMhpDataset'
+ dataset_info = Config.fromfile(
+ 'configs/_base_/datasets/mhp.py').dataset_info
+ # test MHP datasets
+ dataset_class = DATASETS.get(dataset)
+ dataset_class.load_annotations = MagicMock()
+ dataset_class.coco = MagicMock()
+
+ channel_cfg = dict(
+ num_output_channels=16,
+ dataset_joints=16,
+ dataset_channel=[
+ [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15],
+ ],
+ inference_channel=[
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15
+ ])
+
+ data_cfg = dict(
+ image_size=[192, 256],
+ heatmap_size=[48, 64],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'],
+ soft_nms=False,
+ nms_thr=1.0,
+ oks_thr=0.9,
+ vis_thr=0.2,
+ bbox_thr=1.0,
+ use_gt_bbox=True,
+ det_bbox_thr=0.0,
+ bbox_file='',
+ )
+
+ # Test det bbox
+ with pytest.raises(AssertionError):
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ data_cfg_copy['use_gt_bbox'] = False
+
+ _ = dataset_class(
+ ann_file='tests/data/mhp/test_mhp.json',
+ img_prefix='tests/data/mhp/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=True)
+
+ # Test gt bbox
+ _ = dataset_class(
+ ann_file='tests/data/mhp/test_mhp.json',
+ img_prefix='tests/data/mhp/',
+ data_cfg=data_cfg,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=False)
+
+ custom_dataset = dataset_class(
+ ann_file='tests/data/mhp/test_mhp.json',
+ img_prefix='tests/data/mhp/',
+ data_cfg=data_cfg,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=True)
+
+ assert custom_dataset.test_mode is True
+ assert custom_dataset.dataset_name == 'mhp'
+
+ image_id = 2889
+ assert image_id in custom_dataset.img_ids
+ assert len(custom_dataset.img_ids) == 2
+ _ = custom_dataset[0]
+
+ results = convert_db_to_output(custom_dataset.db)
+ infos = custom_dataset.evaluate(results, metric='mAP')
+ assert_almost_equal(infos['AP'], 1.0)
+
+ with pytest.raises(KeyError):
+ _ = custom_dataset.evaluate(results, metric='PCK')
+
+
+def test_top_down_PoseTrack18_dataset():
+ dataset = 'TopDownPoseTrack18Dataset'
+ dataset_info = Config.fromfile(
+ 'configs/_base_/datasets/posetrack18.py').dataset_info
+ # test PoseTrack datasets
+ dataset_class = DATASETS.get(dataset)
+ dataset_class.load_annotations = MagicMock()
+ dataset_class.coco = MagicMock()
+
+ channel_cfg = dict(
+ num_output_channels=17,
+ dataset_joints=17,
+ dataset_channel=[
+ [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16],
+ ],
+ inference_channel=[
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16
+ ])
+
+ data_cfg = dict(
+ image_size=[192, 256],
+ heatmap_size=[48, 64],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'],
+ soft_nms=False,
+ nms_thr=1.0,
+ oks_thr=0.9,
+ vis_thr=0.2,
+ use_gt_bbox=True,
+ det_bbox_thr=0.0,
+ bbox_file='tests/data/posetrack18/annotations/'
+ 'test_posetrack18_human_detections.json',
+ )
+ # Test det bbox
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ data_cfg_copy['use_gt_bbox'] = False
+ _ = dataset_class(
+ ann_file='tests/data/posetrack18/annotations/'
+ 'test_posetrack18_val.json',
+ img_prefix='tests/data/posetrack18/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=True)
+
+ _ = dataset_class(
+ ann_file='tests/data/posetrack18/annotations/'
+ 'test_posetrack18_val.json',
+ img_prefix='tests/data/posetrack18/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=False)
+
+ # Test gt bbox
+ custom_dataset = dataset_class(
+ ann_file='tests/data/posetrack18/annotations/'
+ 'test_posetrack18_val.json',
+ img_prefix='tests/data/posetrack18/',
+ data_cfg=data_cfg,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=True)
+
+ assert custom_dataset.test_mode is True
+ assert custom_dataset.dataset_name == 'posetrack18'
+
+ image_id = 10128340000
+ assert image_id in custom_dataset.img_ids
+ assert len(custom_dataset.img_ids) == 3
+ assert len(custom_dataset) == 14
+ _ = custom_dataset[0]
+
+ # Test evaluate function, use gt bbox
+ results = convert_db_to_output(custom_dataset.db)
+ infos = custom_dataset.evaluate(results, metric='mAP')
+ assert_almost_equal(infos['Total AP'], 100)
+
+ with pytest.raises(KeyError):
+ _ = custom_dataset.evaluate(results, metric='PCK')
+
+ # Test evaluate function, use det bbox
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ data_cfg_copy['use_gt_bbox'] = False
+
+ custom_dataset = dataset_class(
+ ann_file='tests/data/posetrack18/annotations/'
+ 'test_posetrack18_val.json',
+ img_prefix='tests/data/posetrack18/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=True)
+
+ assert len(custom_dataset) == 278
+
+ results = convert_db_to_output(custom_dataset.db)
+ infos = custom_dataset.evaluate(results, metric='mAP')
+ # since the det box input assume each keypoint position to be (0,0)
+ # the Total AP will be zero.
+ assert_almost_equal(infos['Total AP'], 0.)
+
+ with pytest.raises(KeyError):
+ _ = custom_dataset.evaluate(results, metric='PCK')
+
+
+def test_top_down_PoseTrack18Video_dataset():
+ dataset = 'TopDownPoseTrack18VideoDataset'
+ dataset_info = Config.fromfile(
+ 'configs/_base_/datasets/posetrack18.py').dataset_info
+ # test PoseTrack18Video dataset
+ dataset_class = DATASETS.get(dataset)
+ dataset_class.load_annotations = MagicMock()
+ dataset_class.coco = MagicMock()
+
+ channel_cfg = dict(
+ num_output_channels=17,
+ dataset_joints=17,
+ dataset_channel=[
+ [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16],
+ ],
+ inference_channel=[
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16
+ ])
+
+ data_cfg = dict(
+ image_size=[288, 384],
+ heatmap_size=[72, 96],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'],
+ use_nms=True,
+ soft_nms=False,
+ nms_thr=1.0,
+ oks_thr=0.9,
+ vis_thr=0.2,
+ use_gt_bbox=True,
+ det_bbox_thr=0.0,
+ bbox_file='tests/data/posetrack18/annotations/'
+ 'test_posetrack18_human_detections.json',
+ # frame-related arguments
+ frame_index_rand=True,
+ frame_index_range=[-2, 2],
+ num_adj_frames=1,
+ frame_indices_test=[-2, 2, -1, 1, 0],
+ frame_weight_train=(0.0, 1.0),
+ frame_weight_test=(0.3, 0.1, 0.25, 0.25, 0.1),
+ )
+
+ # Test value of dataset_info
+ with pytest.raises(ValueError):
+ _ = dataset_class(
+ ann_file='tests/data/posetrack18/annotations/'
+ 'test_posetrack18_val.json',
+ img_prefix='tests/data/posetrack18/',
+ data_cfg=data_cfg,
+ pipeline=[],
+ dataset_info=None,
+ test_mode=False)
+
+ # Test train mode (must use gt bbox)
+ with pytest.warns(UserWarning):
+ _ = dataset_class(
+ ann_file='tests/data/posetrack18/annotations/'
+ 'test_posetrack18_val.json',
+ img_prefix='tests/data/posetrack18/',
+ data_cfg=data_cfg,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=False)
+
+ # # Test gt bbox + test mode
+ with pytest.warns(UserWarning):
+ custom_dataset = dataset_class(
+ ann_file='tests/data/posetrack18/annotations/'
+ 'test_posetrack18_val.json',
+ img_prefix='tests/data/posetrack18/',
+ data_cfg=data_cfg,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=True)
+
+ assert custom_dataset.test_mode is True
+ assert custom_dataset.dataset_name == 'posetrack18'
+ assert custom_dataset.ph_fill_len == 6
+
+ image_id = 10128340000
+ assert image_id in custom_dataset.img_ids
+ assert len(custom_dataset.img_ids) == 3
+ assert len(custom_dataset) == 14
+ _ = custom_dataset[0]
+
+ # Test det bbox + test mode
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ data_cfg_copy['use_gt_bbox'] = False
+ with pytest.warns(UserWarning):
+ custom_dataset = dataset_class(
+ ann_file='tests/data/posetrack18/annotations/'
+ 'test_posetrack18_val.json',
+ img_prefix='tests/data/posetrack18/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=True)
+
+ assert custom_dataset.frame_indices_test == [-2, -1, 0, 1, 2]
+ assert len(custom_dataset) == 278
+
+ # Test non-random index
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ data_cfg_copy['frame_index_rand'] = False
+ data_cfg_copy['frame_indices_train'] = [0, -1]
+
+ custom_dataset = dataset_class(
+ ann_file='tests/data/posetrack18/annotations/'
+ 'test_posetrack18_val.json',
+ img_prefix='tests/data/posetrack18/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=False)
+
+ assert custom_dataset.frame_indices_train == [-1, 0]
+
+ # Test evaluate function, use gt bbox
+ results = convert_db_to_output(custom_dataset.db)
+ infos = custom_dataset.evaluate(results, metric='mAP')
+ assert_almost_equal(infos['Total AP'], 100)
+
+ with pytest.raises(KeyError):
+ _ = custom_dataset.evaluate(results, metric='PCK')
+
+ # Test evaluate function, use det bbox
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ data_cfg_copy['use_gt_bbox'] = False
+ with pytest.warns(UserWarning):
+ custom_dataset = dataset_class(
+ ann_file='tests/data/posetrack18/annotations/'
+ 'test_posetrack18_val.json',
+ img_prefix='tests/data/posetrack18/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=True)
+
+ results = convert_db_to_output(custom_dataset.db)
+ infos = custom_dataset.evaluate(results, metric='mAP')
+ # since the det box input assume each keypoint position to be (0,0),
+ # the Total AP will be zero.
+ assert_almost_equal(infos['Total AP'], 0)
+
+ with pytest.raises(KeyError):
+ _ = custom_dataset.evaluate(results, metric='PCK')
+
+
+def test_top_down_CrowdPose_dataset():
+ dataset = 'TopDownCrowdPoseDataset'
+ dataset_info = Config.fromfile(
+ 'configs/_base_/datasets/crowdpose.py').dataset_info
+ # test CrowdPose datasets
+ dataset_class = DATASETS.get(dataset)
+ dataset_class.load_annotations = MagicMock()
+ dataset_class.coco = MagicMock()
+
+ channel_cfg = dict(
+ num_output_channels=14,
+ dataset_joints=14,
+ dataset_channel=[
+ [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13],
+ ],
+ inference_channel=[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13])
+
+ data_cfg = dict(
+ image_size=[192, 256],
+ heatmap_size=[48, 64],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'],
+ soft_nms=False,
+ nms_thr=1.0,
+ oks_thr=0.9,
+ vis_thr=0.2,
+ use_gt_bbox=True,
+ det_bbox_thr=0.0,
+ bbox_file='tests/data/crowdpose/test_crowdpose_det_AP_40.json',
+ )
+ # Test det bbox
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ data_cfg_copy['use_gt_bbox'] = False
+ _ = dataset_class(
+ ann_file='tests/data/crowdpose/test_crowdpose.json',
+ img_prefix='tests/data/crowdpose/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=True)
+
+ _ = dataset_class(
+ ann_file='tests/data/crowdpose/test_crowdpose.json',
+ img_prefix='tests/data/crowdpose/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=False)
+
+ # Test gt bbox
+ custom_dataset = dataset_class(
+ ann_file='tests/data/crowdpose/test_crowdpose.json',
+ img_prefix='tests/data/crowdpose/',
+ data_cfg=data_cfg,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=True)
+
+ assert custom_dataset.test_mode is True
+ assert custom_dataset.dataset_name == 'crowdpose'
+
+ image_id = 103319
+ assert image_id in custom_dataset.img_ids
+ assert len(custom_dataset.img_ids) == 2
+ _ = custom_dataset[0]
+
+ results = convert_db_to_output(custom_dataset.db)
+ infos = custom_dataset.evaluate(results, metric='mAP')
+ assert_almost_equal(infos['AP'], 1.0)
+
+ with pytest.raises(KeyError):
+ _ = custom_dataset.evaluate(results, metric='PCK')
+
+
+def test_top_down_COCO_wholebody_dataset():
+ dataset = 'TopDownCocoWholeBodyDataset'
+ dataset_info = Config.fromfile(
+ 'configs/_base_/datasets/coco_wholebody.py').dataset_info
+ # test COCO datasets
+ dataset_class = DATASETS.get(dataset)
+ dataset_class.load_annotations = MagicMock()
+ dataset_class.coco = MagicMock()
+
+ channel_cfg = dict(
+ num_output_channels=133,
+ dataset_joints=133,
+ dataset_channel=[
+ list(range(133)),
+ ],
+ inference_channel=list(range(133)))
+
+ data_cfg = dict(
+ image_size=[192, 256],
+ heatmap_size=[48, 64],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'],
+ soft_nms=False,
+ nms_thr=1.0,
+ oks_thr=0.9,
+ vis_thr=0.2,
+ use_gt_bbox=True,
+ det_bbox_thr=0.0,
+ bbox_file='tests/data/coco/test_coco_det_AP_H_56.json',
+ )
+ # Test det bbox
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ data_cfg_copy['use_gt_bbox'] = False
+ _ = dataset_class(
+ ann_file='tests/data/coco/test_coco_wholebody.json',
+ img_prefix='tests/data/coco/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=True)
+
+ _ = dataset_class(
+ ann_file='tests/data/coco/test_coco_wholebody.json',
+ img_prefix='tests/data/coco/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=False)
+
+ # Test gt bbox
+ custom_dataset = dataset_class(
+ ann_file='tests/data/coco/test_coco_wholebody.json',
+ img_prefix='tests/data/coco/',
+ data_cfg=data_cfg,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=True)
+
+ assert custom_dataset.test_mode is True
+ assert custom_dataset.dataset_name == 'coco_wholebody'
+
+ image_id = 785
+ assert image_id in custom_dataset.img_ids
+ assert len(custom_dataset.img_ids) == 4
+ _ = custom_dataset[0]
+
+ results = convert_db_to_output(custom_dataset.db)
+ infos = custom_dataset.evaluate(results, metric='mAP')
+ assert_almost_equal(infos['AP'], 1.0)
+
+ with pytest.raises(KeyError):
+ _ = custom_dataset.evaluate(results, metric='PCK')
+
+
+def test_top_down_halpe_dataset():
+ dataset = 'TopDownHalpeDataset'
+ dataset_info = Config.fromfile(
+ 'configs/_base_/datasets/halpe.py').dataset_info
+ # test Halpe datasets
+ dataset_class = DATASETS.get(dataset)
+ dataset_class.load_annotations = MagicMock()
+ dataset_class.coco = MagicMock()
+
+ channel_cfg = dict(
+ num_output_channels=136,
+ dataset_joints=136,
+ dataset_channel=[
+ list(range(136)),
+ ],
+ inference_channel=list(range(136)))
+
+ data_cfg = dict(
+ image_size=[192, 256],
+ heatmap_size=[48, 64],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'],
+ soft_nms=False,
+ nms_thr=1.0,
+ oks_thr=0.9,
+ vis_thr=0.2,
+ use_gt_bbox=True,
+ det_bbox_thr=0.0,
+ bbox_file='tests/data/coco/test_coco_det_AP_H_56.json',
+ )
+ # Test det bbox
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ data_cfg_copy['use_gt_bbox'] = False
+ _ = dataset_class(
+ ann_file='tests/data/halpe/test_halpe.json',
+ img_prefix='tests/data/coco/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=True)
+
+ _ = dataset_class(
+ ann_file='tests/data/halpe/test_halpe.json',
+ img_prefix='tests/data/coco/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=False)
+
+ # Test gt bbox
+ custom_dataset = dataset_class(
+ ann_file='tests/data/halpe/test_halpe.json',
+ img_prefix='tests/data/coco/',
+ data_cfg=data_cfg,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=True)
+
+ assert custom_dataset.test_mode is True
+ assert custom_dataset.dataset_name == 'halpe'
+
+ image_id = 785
+ assert image_id in custom_dataset.img_ids
+ assert len(custom_dataset.img_ids) == 4
+ _ = custom_dataset[0]
+
+ results = convert_db_to_output(custom_dataset.db)
+ infos = custom_dataset.evaluate(results, metric='mAP')
+ assert_almost_equal(infos['AP'], 1.0)
+
+ with pytest.raises(KeyError):
+ _ = custom_dataset.evaluate(results, metric='PCK')
+
+
+def test_top_down_OCHuman_dataset():
+ dataset = 'TopDownOCHumanDataset'
+ dataset_info = Config.fromfile(
+ 'configs/_base_/datasets/ochuman.py').dataset_info
+ # test OCHuman datasets
+ dataset_class = DATASETS.get(dataset)
+ dataset_class.load_annotations = MagicMock()
+ dataset_class.coco = MagicMock()
+
+ channel_cfg = dict(
+ num_output_channels=17,
+ dataset_joints=17,
+ dataset_channel=[
+ [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16],
+ ],
+ inference_channel=[
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16
+ ])
+
+ data_cfg = dict(
+ image_size=[192, 256],
+ heatmap_size=[48, 64],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'],
+ soft_nms=False,
+ nms_thr=1.0,
+ oks_thr=0.9,
+ vis_thr=0.2,
+ use_gt_bbox=True,
+ det_bbox_thr=0.0,
+ bbox_file='',
+ )
+
+ with pytest.raises(AssertionError):
+ # Test det bbox
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ data_cfg_copy['use_gt_bbox'] = False
+ _ = dataset_class(
+ ann_file='tests/data/ochuman/test_ochuman.json',
+ img_prefix='tests/data/ochuman/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=True)
+
+ # Test gt bbox
+ custom_dataset = dataset_class(
+ ann_file='tests/data/ochuman/test_ochuman.json',
+ img_prefix='tests/data/ochuman/',
+ data_cfg=data_cfg,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=True)
+
+ assert custom_dataset.test_mode is True
+ assert custom_dataset.dataset_name == 'ochuman'
+
+ image_id = 1
+ assert image_id in custom_dataset.img_ids
+ assert len(custom_dataset.img_ids) == 3
+ _ = custom_dataset[0]
+
+ results = convert_db_to_output(custom_dataset.db)
+ infos = custom_dataset.evaluate(results, metric='mAP')
+ assert_almost_equal(infos['AP'], 1.0)
+
+ with pytest.raises(KeyError):
+ _ = custom_dataset.evaluate(results, metric='PCK')
+
+
+def test_top_down_MPII_dataset():
+ dataset = 'TopDownMpiiDataset'
+ dataset_info = Config.fromfile(
+ 'configs/_base_/datasets/mpii.py').dataset_info
+ # test COCO datasets
+ dataset_class = DATASETS.get(dataset)
+ dataset_class.load_annotations = MagicMock()
+ dataset_class.coco = MagicMock()
+
+ channel_cfg = dict(
+ num_output_channels=16,
+ dataset_joints=16,
+ dataset_channel=[
+ [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15],
+ ],
+ inference_channel=[
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15
+ ])
+
+ data_cfg = dict(
+ image_size=[256, 256],
+ heatmap_size=[64, 64],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'],
+ )
+
+ # Test det bbox
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ custom_dataset = dataset_class(
+ ann_file='tests/data/mpii/test_mpii.json',
+ img_prefix='tests/data/mpii/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ dataset_info=dataset_info,
+ )
+
+ assert len(custom_dataset) == 5
+ assert custom_dataset.dataset_name == 'mpii'
+ _ = custom_dataset[0]
+
+
+def test_top_down_MPII_TRB_dataset():
+ dataset = 'TopDownMpiiTrbDataset'
+ dataset_info = Config.fromfile(
+ 'configs/_base_/datasets/mpii_trb.py').dataset_info
+ # test MPII TRB datasets
+ dataset_class = DATASETS.get(dataset)
+
+ channel_cfg = dict(
+ num_output_channels=40,
+ dataset_joints=40,
+ dataset_channel=[list(range(40))],
+ inference_channel=list(range(40)))
+
+ data_cfg = dict(
+ image_size=[256, 256],
+ heatmap_size=[64, 64],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'])
+
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ _ = dataset_class(
+ ann_file='tests/data/mpii/test_mpii_trb.json',
+ img_prefix='tests/data/mpii/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=False)
+
+ custom_dataset = dataset_class(
+ ann_file='tests/data/mpii/test_mpii_trb.json',
+ img_prefix='tests/data/mpii/',
+ data_cfg=data_cfg,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=True)
+
+ assert custom_dataset.test_mode is True
+ assert custom_dataset.dataset_name == 'mpii_trb'
+ _ = custom_dataset[0]
+
+
+def test_top_down_AIC_dataset():
+ dataset = 'TopDownAicDataset'
+ dataset_info = Config.fromfile(
+ 'configs/_base_/datasets/aic.py').dataset_info
+ # test AIC datasets
+ dataset_class = DATASETS.get(dataset)
+ dataset_class.load_annotations = MagicMock()
+ dataset_class.coco = MagicMock()
+
+ channel_cfg = dict(
+ num_output_channels=14,
+ dataset_joints=14,
+ dataset_channel=[
+ [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13],
+ ],
+ inference_channel=[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13])
+
+ data_cfg = dict(
+ image_size=[192, 256],
+ heatmap_size=[48, 64],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'],
+ soft_nms=False,
+ nms_thr=1.0,
+ oks_thr=0.9,
+ vis_thr=0.2,
+ use_gt_bbox=True,
+ det_bbox_thr=0.0,
+ bbox_file='')
+
+ with pytest.raises(AssertionError):
+ # Test det bbox
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ data_cfg_copy['use_gt_bbox'] = False
+ _ = dataset_class(
+ ann_file='tests/data/aic/test_aic.json',
+ img_prefix='tests/data/aic/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=True)
+
+ _ = dataset_class(
+ ann_file='tests/data/aic/test_aic.json',
+ img_prefix='tests/data/aic/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=False)
+
+ # Test gt bbox
+ custom_dataset = dataset_class(
+ ann_file='tests/data/aic/test_aic.json',
+ img_prefix='tests/data/aic/',
+ data_cfg=data_cfg,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=True)
+
+ assert custom_dataset.test_mode is True
+ assert custom_dataset.dataset_name == 'aic'
+
+ image_id = 1
+ assert image_id in custom_dataset.img_ids
+ assert len(custom_dataset.img_ids) == 3
+ _ = custom_dataset[0]
+
+ results = convert_db_to_output(custom_dataset.db)
+ infos = custom_dataset.evaluate(results, metric='mAP')
+ assert_almost_equal(infos['AP'], 1.0)
+
+ with pytest.raises(KeyError):
+ _ = custom_dataset.evaluate(results, metric='PCK')
+
+
+def test_top_down_JHMDB_dataset():
+ dataset = 'TopDownJhmdbDataset'
+ dataset_info = Config.fromfile(
+ 'configs/_base_/datasets/jhmdb.py').dataset_info
+ # test JHMDB datasets
+ dataset_class = DATASETS.get(dataset)
+ dataset_class.load_annotations = MagicMock()
+ dataset_class.coco = MagicMock()
+
+ channel_cfg = dict(
+ num_output_channels=15,
+ dataset_joints=15,
+ dataset_channel=[
+ [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14],
+ ],
+ inference_channel=[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14])
+
+ data_cfg = dict(
+ image_size=[192, 256],
+ heatmap_size=[48, 64],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'],
+ soft_nms=False,
+ nms_thr=1.0,
+ oks_thr=0.9,
+ vis_thr=0.2,
+ use_gt_bbox=True,
+ det_bbox_thr=0.0,
+ bbox_file='')
+
+ with pytest.raises(AssertionError):
+ # Test det bbox
+ data_cfg_copy = copy.deepcopy(data_cfg)
+ data_cfg_copy['use_gt_bbox'] = False
+ _ = dataset_class(
+ ann_file='tests/data/jhmdb/test_jhmdb_sub1.json',
+ img_prefix='tests/data/jhmdb/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=True)
+
+ _ = dataset_class(
+ ann_file='tests/data/jhmdb/test_jhmdb_sub1.json',
+ img_prefix='tests/data/jhmdb/',
+ data_cfg=data_cfg_copy,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=False)
+
+ # Test gt bbox
+ custom_dataset = dataset_class(
+ ann_file='tests/data/jhmdb/test_jhmdb_sub1.json',
+ img_prefix='tests/data/jhmdb/',
+ data_cfg=data_cfg,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=True)
+
+ assert custom_dataset.test_mode is True
+ assert custom_dataset.dataset_name == 'jhmdb'
+
+ image_id = 2290001
+ assert image_id in custom_dataset.img_ids
+ assert len(custom_dataset.img_ids) == 3
+ _ = custom_dataset[0]
+
+ results = convert_db_to_output(custom_dataset.db)
+ infos = custom_dataset.evaluate(results, metric=['PCK'])
+ assert_almost_equal(infos['Mean PCK'], 1.0)
+
+ infos = custom_dataset.evaluate(results, metric=['tPCK'])
+ assert_almost_equal(infos['Mean tPCK'], 1.0)
+
+ with pytest.raises(KeyError):
+ _ = custom_dataset.evaluate(results, metric='mAP')
+
+
+def test_top_down_h36m_dataset():
+ dataset = 'TopDownH36MDataset'
+ dataset_info = Config.fromfile(
+ 'configs/_base_/datasets/h36m.py').dataset_info
+ # test AIC datasets
+ dataset_class = DATASETS.get(dataset)
+ dataset_class.load_annotations = MagicMock()
+ dataset_class.coco = MagicMock()
+
+ channel_cfg = dict(
+ num_output_channels=17,
+ dataset_joints=17,
+ dataset_channel=[
+ [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16],
+ ],
+ inference_channel=[
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16
+ ])
+
+ data_cfg = dict(
+ image_size=[256, 256],
+ heatmap_size=[64, 64],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'])
+
+ # Test gt bbox
+ custom_dataset = dataset_class(
+ ann_file='tests/data/h36m/h36m_coco.json',
+ img_prefix='tests/data/h36m/',
+ data_cfg=data_cfg,
+ pipeline=[],
+ dataset_info=dataset_info,
+ test_mode=True)
+
+ assert custom_dataset.test_mode is True
+ assert custom_dataset.dataset_name == 'h36m'
+
+ image_id = 1
+ assert image_id in custom_dataset.img_ids
+ _ = custom_dataset[0]
+
+ results = convert_db_to_output(custom_dataset.db)
+ infos = custom_dataset.evaluate(results, metric='EPE')
+ assert_almost_equal(infos['EPE'], 0.0)
+
+ with pytest.raises(KeyError):
+ _ = custom_dataset.evaluate(results, metric='AUC')
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_eval_hook.py b/engine/pose_estimation/third-party/ViTPose/tests/test_eval_hook.py
new file mode 100644
index 0000000..f472541
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_eval_hook.py
@@ -0,0 +1,258 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os.path as osp
+import tempfile
+import unittest.mock as mock
+from collections import OrderedDict
+from unittest.mock import MagicMock, patch
+
+import pytest
+import torch
+import torch.nn as nn
+from mmcv.runner import EpochBasedRunner, build_optimizer
+from mmcv.utils import get_logger
+from torch.utils.data import DataLoader, Dataset
+
+from mmpose.core import DistEvalHook, EvalHook
+
+
+class ExampleDataset(Dataset):
+
+ def __init__(self):
+ self.index = 0
+ self.eval_result = [0.1, 0.4, 0.3, 0.7, 0.2, 0.05, 0.4, 0.6]
+
+ def __getitem__(self, idx):
+ results = dict(imgs=torch.tensor([1]))
+ return results
+
+ def __len__(self):
+ return 1
+
+ @mock.create_autospec
+ def evaluate(self, results, res_folder=None, logger=None):
+ pass
+
+
+class EvalDataset(ExampleDataset):
+
+ def evaluate(self, results, res_folder=None, logger=None):
+ acc = self.eval_result[self.index]
+ output = OrderedDict(acc=acc, index=self.index, score=acc)
+ self.index += 1
+ return output
+
+
+class ExampleModel(nn.Module):
+
+ def __init__(self):
+ super().__init__()
+ self.conv = nn.Linear(1, 1)
+ self.test_cfg = None
+
+ def forward(self, imgs, return_loss=False):
+ return imgs
+
+ def train_step(self, data_batch, optimizer, **kwargs):
+ outputs = {
+ 'loss': 0.5,
+ 'log_vars': {
+ 'accuracy': 0.98
+ },
+ 'num_samples': 1
+ }
+ return outputs
+
+
+@pytest.mark.skipif(
+ not torch.cuda.is_available(), reason='requires CUDA support')
+@patch('mmpose.apis.single_gpu_test', MagicMock)
+@patch('mmpose.apis.multi_gpu_test', MagicMock)
+@pytest.mark.parametrize('EvalHookCls', (EvalHook, DistEvalHook))
+def test_eval_hook(EvalHookCls):
+ with pytest.raises(TypeError):
+ # dataloader must be a pytorch DataLoader
+ test_dataset = ExampleDataset()
+ data_loader = [
+ DataLoader(
+ test_dataset,
+ batch_size=1,
+ sampler=None,
+ num_worker=0,
+ shuffle=False)
+ ]
+ EvalHookCls(data_loader)
+
+ with pytest.raises(KeyError):
+ # rule must be in keys of rule_map
+ test_dataset = ExampleDataset()
+ data_loader = DataLoader(
+ test_dataset,
+ batch_size=1,
+ sampler=None,
+ num_workers=0,
+ shuffle=False)
+ EvalHookCls(data_loader, save_best='auto', rule='unsupport')
+
+ with pytest.raises(ValueError):
+ # save_best must be valid when rule_map is None
+ test_dataset = ExampleDataset()
+ data_loader = DataLoader(
+ test_dataset,
+ batch_size=1,
+ sampler=None,
+ num_workers=0,
+ shuffle=False)
+ EvalHookCls(data_loader, save_best='unsupport')
+
+ optimizer_cfg = dict(
+ type='SGD', lr=0.01, momentum=0.9, weight_decay=0.0001)
+
+ test_dataset = ExampleDataset()
+ loader = DataLoader(test_dataset, batch_size=1)
+ model = ExampleModel()
+ optimizer = build_optimizer(model, optimizer_cfg)
+
+ data_loader = DataLoader(test_dataset, batch_size=1)
+ eval_hook = EvalHookCls(data_loader, save_best=None)
+ with tempfile.TemporaryDirectory() as tmpdir:
+ logger = get_logger('test_eval')
+ runner = EpochBasedRunner(
+ model=model,
+ batch_processor=None,
+ optimizer=optimizer,
+ work_dir=tmpdir,
+ logger=logger,
+ max_epochs=1)
+ runner.register_hook(eval_hook)
+ runner.run([loader], [('train', 1)])
+ assert runner.meta is None or 'best_score' not in runner.meta[
+ 'hook_msgs']
+ assert runner.meta is None or 'best_ckpt' not in runner.meta[
+ 'hook_msgs']
+
+ # when `save_best` is set to 'auto', first metric will be used.
+ loader = DataLoader(EvalDataset(), batch_size=1)
+ model = ExampleModel()
+ data_loader = DataLoader(EvalDataset(), batch_size=1)
+ eval_hook = EvalHookCls(data_loader, interval=1, save_best='auto')
+
+ with tempfile.TemporaryDirectory() as tmpdir:
+ logger = get_logger('test_eval')
+ runner = EpochBasedRunner(
+ model=model,
+ batch_processor=None,
+ optimizer=optimizer,
+ work_dir=tmpdir,
+ logger=logger,
+ max_epochs=8)
+ runner.register_checkpoint_hook(dict(interval=1))
+ runner.register_hook(eval_hook)
+ runner.run([loader], [('train', 1)])
+
+ real_path = osp.join(tmpdir, 'best_acc_epoch_4.pth')
+
+ assert runner.meta['hook_msgs']['best_ckpt'] == osp.realpath(real_path)
+ assert runner.meta['hook_msgs']['best_score'] == 0.7
+
+ loader = DataLoader(EvalDataset(), batch_size=1)
+ model = ExampleModel()
+ data_loader = DataLoader(EvalDataset(), batch_size=1)
+ eval_hook = EvalHookCls(data_loader, interval=1, save_best='acc')
+
+ with tempfile.TemporaryDirectory() as tmpdir:
+ logger = get_logger('test_eval')
+ runner = EpochBasedRunner(
+ model=model,
+ batch_processor=None,
+ optimizer=optimizer,
+ work_dir=tmpdir,
+ logger=logger,
+ max_epochs=8)
+ runner.register_checkpoint_hook(dict(interval=1))
+ runner.register_hook(eval_hook)
+ runner.run([loader], [('train', 1)])
+
+ real_path = osp.join(tmpdir, 'best_acc_epoch_4.pth')
+
+ assert runner.meta['hook_msgs']['best_ckpt'] == osp.realpath(real_path)
+ assert runner.meta['hook_msgs']['best_score'] == 0.7
+
+ data_loader = DataLoader(EvalDataset(), batch_size=1)
+ eval_hook = EvalHookCls(
+ data_loader, interval=1, save_best='score', rule='greater')
+ with tempfile.TemporaryDirectory() as tmpdir:
+ logger = get_logger('test_eval')
+ runner = EpochBasedRunner(
+ model=model,
+ batch_processor=None,
+ optimizer=optimizer,
+ work_dir=tmpdir,
+ logger=logger)
+ runner.register_checkpoint_hook(dict(interval=1))
+ runner.register_hook(eval_hook)
+ runner.run([loader], [('train', 1)], 8)
+
+ real_path = osp.join(tmpdir, 'best_score_epoch_4.pth')
+
+ assert runner.meta['hook_msgs']['best_ckpt'] == osp.realpath(real_path)
+ assert runner.meta['hook_msgs']['best_score'] == 0.7
+
+ data_loader = DataLoader(EvalDataset(), batch_size=1)
+ eval_hook = EvalHookCls(data_loader, save_best='acc', rule='less')
+ with tempfile.TemporaryDirectory() as tmpdir:
+ logger = get_logger('test_eval')
+ runner = EpochBasedRunner(
+ model=model,
+ batch_processor=None,
+ optimizer=optimizer,
+ work_dir=tmpdir,
+ logger=logger,
+ max_epochs=8)
+ runner.register_checkpoint_hook(dict(interval=1))
+ runner.register_hook(eval_hook)
+ runner.run([loader], [('train', 1)])
+
+ real_path = osp.join(tmpdir, 'best_acc_epoch_6.pth')
+
+ assert runner.meta['hook_msgs']['best_ckpt'] == osp.realpath(real_path)
+ assert runner.meta['hook_msgs']['best_score'] == 0.05
+
+ data_loader = DataLoader(EvalDataset(), batch_size=1)
+ eval_hook = EvalHookCls(data_loader, save_best='acc')
+ with tempfile.TemporaryDirectory() as tmpdir:
+ logger = get_logger('test_eval')
+ runner = EpochBasedRunner(
+ model=model,
+ batch_processor=None,
+ optimizer=optimizer,
+ work_dir=tmpdir,
+ logger=logger,
+ max_epochs=2)
+ runner.register_checkpoint_hook(dict(interval=1))
+ runner.register_hook(eval_hook)
+ runner.run([loader], [('train', 1)])
+
+ real_path = osp.join(tmpdir, 'best_acc_epoch_2.pth')
+
+ assert runner.meta['hook_msgs']['best_ckpt'] == osp.realpath(real_path)
+ assert runner.meta['hook_msgs']['best_score'] == 0.4
+
+ resume_from = osp.join(tmpdir, 'latest.pth')
+ loader = DataLoader(ExampleDataset(), batch_size=1)
+ eval_hook = EvalHookCls(data_loader, save_best='acc')
+ runner = EpochBasedRunner(
+ model=model,
+ batch_processor=None,
+ optimizer=optimizer,
+ work_dir=tmpdir,
+ logger=logger,
+ max_epochs=8)
+ runner.register_checkpoint_hook(dict(interval=1))
+ runner.register_hook(eval_hook)
+ runner.resume(resume_from)
+ runner.run([loader], [('train', 1)])
+
+ real_path = osp.join(tmpdir, 'best_acc_epoch_4.pth')
+
+ assert runner.meta['hook_msgs']['best_ckpt'] == osp.realpath(real_path)
+ assert runner.meta['hook_msgs']['best_score'] == 0.7
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_evaluation/test_bottom_up_eval.py b/engine/pose_estimation/third-party/ViTPose/tests/test_evaluation/test_bottom_up_eval.py
new file mode 100644
index 0000000..0459ae1
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_evaluation/test_bottom_up_eval.py
@@ -0,0 +1,102 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import numpy as np
+import torch
+
+from mmpose.core import (aggregate_scale, aggregate_stage_flip,
+ flip_feature_maps, get_group_preds, split_ae_outputs)
+
+
+def test_split_ae_outputs():
+ fake_outputs = [torch.zeros((1, 4, 2, 2))]
+ heatmaps, tags = split_ae_outputs(
+ fake_outputs,
+ num_joints=4,
+ with_heatmaps=[False],
+ with_ae=[True],
+ select_output_index=[0])
+
+
+def test_flip_feature_maps():
+ fake_outputs = [torch.zeros((1, 4, 2, 2))]
+ _ = flip_feature_maps(fake_outputs, None)
+ _ = flip_feature_maps(fake_outputs, flip_index=[1, 0])
+
+
+def test_aggregate_stage_flip():
+ fake_outputs = [torch.zeros((1, 4, 2, 2))]
+ fake_flip_outputs = [torch.ones((1, 4, 2, 2))]
+ output = aggregate_stage_flip(
+ fake_outputs,
+ fake_flip_outputs,
+ index=-1,
+ project2image=True,
+ size_projected=(4, 4),
+ align_corners=False,
+ aggregate_stage='concat',
+ aggregate_flip='average')
+ assert isinstance(output, list)
+
+ output = aggregate_stage_flip(
+ fake_outputs,
+ fake_flip_outputs,
+ index=-1,
+ project2image=True,
+ size_projected=(4, 4),
+ align_corners=False,
+ aggregate_stage='average',
+ aggregate_flip='average')
+ assert isinstance(output, list)
+
+ output = aggregate_stage_flip(
+ fake_outputs,
+ fake_flip_outputs,
+ index=-1,
+ project2image=True,
+ size_projected=(4, 4),
+ align_corners=False,
+ aggregate_stage='average',
+ aggregate_flip='concat')
+ assert isinstance(output, list)
+
+ output = aggregate_stage_flip(
+ fake_outputs,
+ fake_flip_outputs,
+ index=-1,
+ project2image=True,
+ size_projected=(4, 4),
+ align_corners=False,
+ aggregate_stage='concat',
+ aggregate_flip='concat')
+ assert isinstance(output, list)
+
+
+def test_aggregate_scale():
+ fake_outputs = [torch.zeros((1, 4, 2, 2)), torch.zeros((1, 4, 2, 2))]
+ output = aggregate_scale(
+ fake_outputs, align_corners=False, aggregate_scale='average')
+ assert isinstance(output, torch.Tensor)
+ assert output.shape == fake_outputs[0].shape
+
+ output = aggregate_scale(
+ fake_outputs, align_corners=False, aggregate_scale='unsqueeze_concat')
+
+ assert isinstance(output, torch.Tensor)
+ assert len(output.shape) == len(fake_outputs[0].shape) + 1
+
+
+def test_get_group_preds():
+ fake_grouped_joints = [np.array([[[0, 0], [1, 1]]])]
+ results = get_group_preds(
+ fake_grouped_joints,
+ center=np.array([0, 0]),
+ scale=np.array([1, 1]),
+ heatmap_size=np.array([2, 2]))
+ assert not results == []
+
+ results = get_group_preds(
+ fake_grouped_joints,
+ center=np.array([0, 0]),
+ scale=np.array([1, 1]),
+ heatmap_size=np.array([2, 2]),
+ use_udp=True)
+ assert not results == []
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_evaluation/test_mesh_eval.py b/engine/pose_estimation/third-party/ViTPose/tests/test_evaluation/test_mesh_eval.py
new file mode 100644
index 0000000..9ff4fa2
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_evaluation/test_mesh_eval.py
@@ -0,0 +1,14 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import numpy as np
+from numpy.testing import assert_array_almost_equal
+
+from mmpose.core import compute_similarity_transform
+
+
+def test_compute_similarity_transform():
+ source = np.random.rand(14, 3)
+ tran = np.random.rand(1, 3)
+ scale = 0.5
+ target = source * scale + tran
+ source_transformed = compute_similarity_transform(source, target)
+ assert_array_almost_equal(source_transformed, target)
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_evaluation/test_pose3d_eval.py b/engine/pose_estimation/third-party/ViTPose/tests/test_evaluation/test_pose3d_eval.py
new file mode 100644
index 0000000..80aaba5
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_evaluation/test_pose3d_eval.py
@@ -0,0 +1,49 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import numpy as np
+import pytest
+
+from mmpose.core import keypoint_3d_auc, keypoint_3d_pck
+
+
+def test_keypoint_3d_pck():
+ target = np.random.rand(2, 5, 3)
+ output = np.copy(target)
+ mask = np.ones((output.shape[0], output.shape[1]), dtype=bool)
+
+ with pytest.raises(ValueError):
+ _ = keypoint_3d_pck(output, target, mask, alignment='norm')
+
+ pck = keypoint_3d_pck(output, target, mask, alignment='none')
+ np.testing.assert_almost_equal(pck, 100)
+
+ output[0, 0, :] = target[0, 0, :] + 1
+ pck = keypoint_3d_pck(output, target, mask, alignment='none')
+ np.testing.assert_almost_equal(pck, 90, 5)
+
+ output = target * 2
+ pck = keypoint_3d_pck(output, target, mask, alignment='scale')
+ np.testing.assert_almost_equal(pck, 100)
+
+ output = target + 2
+ pck = keypoint_3d_pck(output, target, mask, alignment='procrustes')
+ np.testing.assert_almost_equal(pck, 100)
+
+
+def test_keypoint_3d_auc():
+ target = np.random.rand(2, 5, 3)
+ output = np.copy(target)
+ mask = np.ones((output.shape[0], output.shape[1]), dtype=bool)
+
+ with pytest.raises(ValueError):
+ _ = keypoint_3d_auc(output, target, mask, alignment='norm')
+
+ auc = keypoint_3d_auc(output, target, mask, alignment='none')
+ np.testing.assert_almost_equal(auc, 30 / 31 * 100)
+
+ output = target * 2
+ auc = keypoint_3d_auc(output, target, mask, alignment='scale')
+ np.testing.assert_almost_equal(auc, 30 / 31 * 100)
+
+ output = target + 2
+ auc = keypoint_3d_auc(output, target, mask, alignment='procrustes')
+ np.testing.assert_almost_equal(auc, 30 / 31 * 100)
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_evaluation/test_top_down_eval.py b/engine/pose_estimation/third-party/ViTPose/tests/test_evaluation/test_top_down_eval.py
new file mode 100644
index 0000000..5cda7e1
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_evaluation/test_top_down_eval.py
@@ -0,0 +1,213 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import numpy as np
+import pytest
+from numpy.testing import assert_array_almost_equal
+
+from mmpose.core import (keypoint_auc, keypoint_epe, keypoint_pck_accuracy,
+ keypoints_from_heatmaps, keypoints_from_heatmaps3d,
+ multilabel_classification_accuracy, pose_pck_accuracy)
+
+
+def test_pose_pck_accuracy():
+ output = np.zeros((1, 5, 64, 64), dtype=np.float32)
+ target = np.zeros((1, 5, 64, 64), dtype=np.float32)
+ mask = np.array([[True, True, False, False, False]])
+ # first channel
+ output[0, 0, 20, 20] = 1
+ target[0, 0, 10, 10] = 1
+ # second channel
+ output[0, 1, 30, 30] = 1
+ target[0, 1, 30, 30] = 1
+
+ acc, avg_acc, cnt = pose_pck_accuracy(output, target, mask)
+
+ assert_array_almost_equal(acc, np.array([0, 1, -1, -1, -1]), decimal=4)
+ assert abs(avg_acc - 0.5) < 1e-4
+ assert abs(cnt - 2) < 1e-4
+
+
+def test_keypoints_from_heatmaps():
+ heatmaps = np.ones((1, 1, 64, 64), dtype=np.float32)
+ heatmaps[0, 0, 31, 31] = 2
+ center = np.array([[127, 127]])
+ scale = np.array([[64 / 200.0, 64 / 200.0]])
+
+ udp_heatmaps = np.ones((32, 17, 64, 64), dtype=np.float32)
+ udp_heatmaps[:, :, 31, 31] = 2
+ udp_center = np.tile([127, 127], (32, 1))
+ udp_scale = np.tile([32, 32], (32, 1))
+
+ preds, maxvals = keypoints_from_heatmaps(heatmaps, center, scale)
+
+ assert_array_almost_equal(preds, np.array([[[126, 126]]]), decimal=4)
+ assert_array_almost_equal(maxvals, np.array([[[2]]]), decimal=4)
+ assert isinstance(preds, np.ndarray)
+ assert isinstance(maxvals, np.ndarray)
+
+ with pytest.raises(AssertionError):
+ # kernel should > 0
+ _ = keypoints_from_heatmaps(
+ heatmaps, center, scale, post_process='unbiased', kernel=0)
+
+ preds, maxvals = keypoints_from_heatmaps(
+ heatmaps, center, scale, post_process='unbiased')
+ assert_array_almost_equal(preds, np.array([[[126, 126]]]), decimal=4)
+ assert_array_almost_equal(maxvals, np.array([[[2]]]), decimal=4)
+ assert isinstance(preds, np.ndarray)
+ assert isinstance(maxvals, np.ndarray)
+
+ # test for udp dimension problem
+ preds, maxvals = keypoints_from_heatmaps(
+ udp_heatmaps,
+ udp_center,
+ udp_scale,
+ post_process='default',
+ target_type='GaussianHeatMap',
+ use_udp=True)
+ assert_array_almost_equal(preds, np.tile([76, 76], (32, 17, 1)), decimal=0)
+ assert_array_almost_equal(maxvals, np.tile([2], (32, 17, 1)), decimal=4)
+ assert isinstance(preds, np.ndarray)
+ assert isinstance(maxvals, np.ndarray)
+
+ preds1, maxvals1 = keypoints_from_heatmaps(
+ heatmaps,
+ center,
+ scale,
+ post_process='default',
+ target_type='GaussianHeatMap',
+ use_udp=True)
+ preds2, maxvals2 = keypoints_from_heatmaps(
+ heatmaps,
+ center,
+ scale,
+ post_process='default',
+ target_type='GaussianHeatmap',
+ use_udp=True)
+ assert_array_almost_equal(preds1, preds2, decimal=4)
+ assert_array_almost_equal(maxvals1, maxvals2, decimal=4)
+ assert isinstance(preds2, np.ndarray)
+ assert isinstance(maxvals2, np.ndarray)
+
+
+def test_keypoint_pck_accuracy():
+ output = np.zeros((2, 5, 2))
+ target = np.zeros((2, 5, 2))
+ mask = np.array([[True, True, False, True, True],
+ [True, True, False, True, True]])
+ thr = np.full((2, 2), 10, dtype=np.float32)
+ # first channel
+ output[0, 0] = [10, 0]
+ target[0, 0] = [10, 0]
+ # second channel
+ output[0, 1] = [20, 20]
+ target[0, 1] = [10, 10]
+ # third channel
+ output[0, 2] = [0, 0]
+ target[0, 2] = [-1, 0]
+ # fourth channel
+ output[0, 3] = [30, 30]
+ target[0, 3] = [30, 30]
+ # fifth channel
+ output[0, 4] = [0, 10]
+ target[0, 4] = [0, 10]
+
+ acc, avg_acc, cnt = keypoint_pck_accuracy(output, target, mask, 0.5, thr)
+
+ assert_array_almost_equal(acc, np.array([1, 0.5, -1, 1, 1]), decimal=4)
+ assert abs(avg_acc - 0.875) < 1e-4
+ assert abs(cnt - 4) < 1e-4
+
+ acc, avg_acc, cnt = keypoint_pck_accuracy(output, target, mask, 0.5,
+ np.zeros((2, 2)))
+ assert_array_almost_equal(acc, np.array([-1, -1, -1, -1, -1]), decimal=4)
+ assert abs(avg_acc) < 1e-4
+ assert abs(cnt) < 1e-4
+
+ acc, avg_acc, cnt = keypoint_pck_accuracy(output, target, mask, 0.5,
+ np.array([[0, 0], [10, 10]]))
+ assert_array_almost_equal(acc, np.array([1, 1, -1, 1, 1]), decimal=4)
+ assert abs(avg_acc - 1) < 1e-4
+ assert abs(cnt - 4) < 1e-4
+
+
+def test_keypoint_auc():
+ output = np.zeros((1, 5, 2))
+ target = np.zeros((1, 5, 2))
+ mask = np.array([[True, True, False, True, True]])
+ # first channel
+ output[0, 0] = [10, 4]
+ target[0, 0] = [10, 0]
+ # second channel
+ output[0, 1] = [10, 18]
+ target[0, 1] = [10, 10]
+ # third channel
+ output[0, 2] = [0, 0]
+ target[0, 2] = [0, -1]
+ # fourth channel
+ output[0, 3] = [40, 40]
+ target[0, 3] = [30, 30]
+ # fifth channel
+ output[0, 4] = [20, 10]
+ target[0, 4] = [0, 10]
+
+ auc = keypoint_auc(output, target, mask, 20, 4)
+ assert abs(auc - 0.375) < 1e-4
+
+
+def test_keypoint_epe():
+ output = np.zeros((1, 5, 2))
+ target = np.zeros((1, 5, 2))
+ mask = np.array([[True, True, False, True, True]])
+ # first channel
+ output[0, 0] = [10, 4]
+ target[0, 0] = [10, 0]
+ # second channel
+ output[0, 1] = [10, 18]
+ target[0, 1] = [10, 10]
+ # third channel
+ output[0, 2] = [0, 0]
+ target[0, 2] = [-1, -1]
+ # fourth channel
+ output[0, 3] = [40, 40]
+ target[0, 3] = [30, 30]
+ # fifth channel
+ output[0, 4] = [20, 10]
+ target[0, 4] = [0, 10]
+
+ epe = keypoint_epe(output, target, mask)
+ assert abs(epe - 11.5355339) < 1e-4
+
+
+def test_keypoints_from_heatmaps3d():
+ heatmaps = np.ones((1, 1, 64, 64, 64), dtype=np.float32)
+ heatmaps[0, 0, 10, 31, 40] = 2
+ center = np.array([[127, 127]])
+ scale = np.array([[64 / 200.0, 64 / 200.0]])
+ preds, maxvals = keypoints_from_heatmaps3d(heatmaps, center, scale)
+
+ assert_array_almost_equal(preds, np.array([[[135, 126, 10]]]), decimal=4)
+ assert_array_almost_equal(maxvals, np.array([[[2]]]), decimal=4)
+ assert isinstance(preds, np.ndarray)
+ assert isinstance(maxvals, np.ndarray)
+
+
+def test_multilabel_classification_accuracy():
+ output = np.array([[0.7, 0.8, 0.4], [0.8, 0.1, 0.1]])
+ target = np.array([[1, 0, 0], [1, 0, 1]])
+ mask = np.array([[True, True, True], [True, True, True]])
+ thr = 0.5
+ acc = multilabel_classification_accuracy(output, target, mask, thr)
+ assert acc == 0
+
+ output = np.array([[0.7, 0.2, 0.4], [0.8, 0.1, 0.9]])
+ thr = 0.5
+ acc = multilabel_classification_accuracy(output, target, mask, thr)
+ assert acc == 1
+
+ thr = 0.3
+ acc = multilabel_classification_accuracy(output, target, mask, thr)
+ assert acc == 0.5
+
+ mask = np.array([[True, True, False], [True, True, True]])
+ acc = multilabel_classification_accuracy(output, target, mask, thr)
+ assert acc == 1
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_external/test_smpl.py b/engine/pose_estimation/third-party/ViTPose/tests/test_external/test_smpl.py
new file mode 100644
index 0000000..e3e2482
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_external/test_smpl.py
@@ -0,0 +1,78 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os.path as osp
+import tempfile
+
+import numpy as np
+import torch
+
+from mmpose.models.utils import SMPL
+from tests.utils.mesh_utils import generate_smpl_weight_file
+
+
+def test_smpl():
+ """Test smpl model."""
+
+ # build smpl model
+ smpl = None
+ with tempfile.TemporaryDirectory() as tmpdir:
+ # generate weight file for SMPL model.
+ generate_smpl_weight_file(tmpdir)
+
+ smpl_cfg = dict(
+ smpl_path=tmpdir,
+ joints_regressor=osp.join(tmpdir, 'test_joint_regressor.npy'))
+ smpl = SMPL(**smpl_cfg)
+
+ assert smpl is not None, 'Fail to build SMPL model'
+
+ # test get face function
+ faces = smpl.get_faces()
+ assert isinstance(faces, np.ndarray)
+
+ betas = torch.zeros(3, 10)
+ body_pose = torch.zeros(3, 23 * 3)
+ global_orient = torch.zeros(3, 3)
+ transl = torch.zeros(3, 3)
+ gender = torch.LongTensor([-1, 0, 1])
+
+ # test forward with body_pose and global_orient in axis-angle format
+ smpl_out = smpl(
+ betas=betas, body_pose=body_pose, global_orient=global_orient)
+ assert isinstance(smpl_out, dict)
+ assert smpl_out['vertices'].shape == torch.Size([3, 6890, 3])
+ assert smpl_out['joints'].shape == torch.Size([3, 24, 3])
+
+ # test forward with body_pose and global_orient in rotation matrix format
+ body_pose = torch.eye(3).repeat([3, 23, 1, 1])
+ global_orient = torch.eye(3).repeat([3, 1, 1, 1])
+ _ = smpl(betas=betas, body_pose=body_pose, global_orient=global_orient)
+
+ # test forward with translation
+ _ = smpl(
+ betas=betas,
+ body_pose=body_pose,
+ global_orient=global_orient,
+ transl=transl)
+
+ # test forward with gender
+ _ = smpl(
+ betas=betas,
+ body_pose=body_pose,
+ global_orient=global_orient,
+ transl=transl,
+ gender=gender)
+
+ # test forward when all samples in the same gender
+ gender = torch.LongTensor([0, 0, 0])
+ _ = smpl(
+ betas=betas,
+ body_pose=body_pose,
+ global_orient=global_orient,
+ transl=transl,
+ gender=gender)
+
+ # test forward when batch size = 0
+ _ = smpl(
+ betas=torch.zeros(0, 10),
+ body_pose=torch.zeros(0, 23 * 3),
+ global_orient=torch.zeros(0, 3))
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_losses/test_bottom_up_losses.py b/engine/pose_estimation/third-party/ViTPose/tests/test_losses/test_bottom_up_losses.py
new file mode 100644
index 0000000..803c19f
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_losses/test_bottom_up_losses.py
@@ -0,0 +1,168 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import pytest
+import torch
+
+
+def test_multi_loss_factory():
+ from mmpose.models import build_loss
+
+ # test heatmap loss
+ loss_cfg = dict(type='HeatmapLoss')
+ loss = build_loss(loss_cfg)
+
+ with pytest.raises(AssertionError):
+ fake_pred = torch.zeros((2, 3, 64, 64))
+ fake_label = torch.zeros((1, 3, 64, 64))
+ fake_mask = torch.zeros((1, 64, 64))
+ loss(fake_pred, fake_label, fake_mask)
+
+ fake_pred = torch.zeros((1, 3, 64, 64))
+ fake_label = torch.zeros((1, 3, 64, 64))
+ fake_mask = torch.zeros((1, 64, 64))
+ assert torch.allclose(
+ loss(fake_pred, fake_label, fake_mask), torch.tensor(0.))
+
+ fake_pred = torch.ones((1, 3, 64, 64))
+ fake_label = torch.zeros((1, 3, 64, 64))
+ fake_mask = torch.zeros((1, 64, 64))
+ assert torch.allclose(
+ loss(fake_pred, fake_label, fake_mask), torch.tensor(0.))
+
+ fake_pred = torch.ones((1, 3, 64, 64))
+ fake_label = torch.zeros((1, 3, 64, 64))
+ fake_mask = torch.ones((1, 64, 64))
+ assert torch.allclose(
+ loss(fake_pred, fake_label, fake_mask), torch.tensor(1.))
+
+ # test AE loss
+ fake_tags = torch.zeros((1, 18, 1))
+ fake_joints = torch.zeros((1, 3, 2, 2), dtype=torch.int)
+
+ loss_cfg = dict(type='AELoss', loss_type='exp')
+ loss = build_loss(loss_cfg)
+ assert torch.allclose(loss(fake_tags, fake_joints)[0], torch.tensor(0.))
+ assert torch.allclose(loss(fake_tags, fake_joints)[1], torch.tensor(0.))
+
+ fake_tags[0, 0, 0] = 1.
+ fake_tags[0, 10, 0] = 0.
+ fake_joints[0, 0, 0, :] = torch.IntTensor((0, 1))
+ fake_joints[0, 0, 1, :] = torch.IntTensor((10, 1))
+ loss_cfg = dict(type='AELoss', loss_type='exp')
+ loss = build_loss(loss_cfg)
+ assert torch.allclose(loss(fake_tags, fake_joints)[0], torch.tensor(0.))
+ assert torch.allclose(loss(fake_tags, fake_joints)[1], torch.tensor(0.25))
+
+ fake_tags[0, 0, 0] = 0
+ fake_tags[0, 7, 0] = 1.
+ fake_tags[0, 17, 0] = 1.
+ fake_joints[0, 1, 0, :] = torch.IntTensor((7, 1))
+ fake_joints[0, 1, 1, :] = torch.IntTensor((17, 1))
+
+ loss_cfg = dict(type='AELoss', loss_type='exp')
+ loss = build_loss(loss_cfg)
+ assert torch.allclose(loss(fake_tags, fake_joints)[1], torch.tensor(0.))
+
+ loss_cfg = dict(type='AELoss', loss_type='max')
+ loss = build_loss(loss_cfg)
+ assert torch.allclose(loss(fake_tags, fake_joints)[0], torch.tensor(0.))
+
+ with pytest.raises(ValueError):
+ loss_cfg = dict(type='AELoss', loss_type='min')
+ loss = build_loss(loss_cfg)
+ loss(fake_tags, fake_joints)
+
+ # test MultiLossFactory
+ with pytest.raises(AssertionError):
+ loss_cfg = dict(
+ type='MultiLossFactory',
+ num_joints=2,
+ num_stages=1,
+ ae_loss_type='exp',
+ with_ae_loss=True,
+ push_loss_factor=[0.001],
+ pull_loss_factor=[0.001],
+ with_heatmaps_loss=[True],
+ heatmaps_loss_factor=[1.0])
+ loss = build_loss(loss_cfg)
+ with pytest.raises(AssertionError):
+ loss_cfg = dict(
+ type='MultiLossFactory',
+ num_joints=2,
+ num_stages=1,
+ ae_loss_type='exp',
+ with_ae_loss=[True],
+ push_loss_factor=0.001,
+ pull_loss_factor=[0.001],
+ with_heatmaps_loss=[True],
+ heatmaps_loss_factor=[1.0])
+ loss = build_loss(loss_cfg)
+ with pytest.raises(AssertionError):
+ loss_cfg = dict(
+ type='MultiLossFactory',
+ num_joints=2,
+ num_stages=1,
+ ae_loss_type='exp',
+ with_ae_loss=[True],
+ push_loss_factor=[0.001],
+ pull_loss_factor=0.001,
+ with_heatmaps_loss=[True],
+ heatmaps_loss_factor=[1.0])
+ loss = build_loss(loss_cfg)
+ with pytest.raises(AssertionError):
+ loss_cfg = dict(
+ type='MultiLossFactory',
+ num_joints=2,
+ num_stages=1,
+ ae_loss_type='exp',
+ with_ae_loss=[True],
+ push_loss_factor=[0.001],
+ pull_loss_factor=[0.001],
+ with_heatmaps_loss=True,
+ heatmaps_loss_factor=[1.0])
+ loss = build_loss(loss_cfg)
+ with pytest.raises(AssertionError):
+ loss_cfg = dict(
+ type='MultiLossFactory',
+ num_joints=2,
+ num_stages=1,
+ ae_loss_type='exp',
+ with_ae_loss=[True],
+ push_loss_factor=[0.001],
+ pull_loss_factor=[0.001],
+ with_heatmaps_loss=[True],
+ heatmaps_loss_factor=1.0)
+ loss = build_loss(loss_cfg)
+ loss_cfg = dict(
+ type='MultiLossFactory',
+ num_joints=17,
+ num_stages=1,
+ ae_loss_type='exp',
+ with_ae_loss=[False],
+ push_loss_factor=[0.001],
+ pull_loss_factor=[0.001],
+ with_heatmaps_loss=[False],
+ heatmaps_loss_factor=[1.0])
+ loss = build_loss(loss_cfg)
+ fake_outputs = [torch.zeros((1, 34, 64, 64))]
+ fake_heatmaps = [torch.zeros((1, 17, 64, 64))]
+ fake_masks = [torch.ones((1, 64, 64))]
+ fake_joints = [torch.zeros((1, 30, 17, 2))]
+ heatmaps_losses, push_losses, pull_losses = \
+ loss(fake_outputs, fake_heatmaps, fake_masks, fake_joints)
+ assert heatmaps_losses == [None]
+ assert pull_losses == [None]
+ assert push_losses == [None]
+ loss_cfg = dict(
+ type='MultiLossFactory',
+ num_joints=17,
+ num_stages=1,
+ ae_loss_type='exp',
+ with_ae_loss=[True],
+ push_loss_factor=[0.001],
+ pull_loss_factor=[0.001],
+ with_heatmaps_loss=[True],
+ heatmaps_loss_factor=[1.0])
+ loss = build_loss(loss_cfg)
+ heatmaps_losses, push_losses, pull_losses = \
+ loss(fake_outputs, fake_heatmaps, fake_masks, fake_joints)
+ assert len(heatmaps_losses) == 1
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_losses/test_classification_loss.py b/engine/pose_estimation/third-party/ViTPose/tests/test_losses/test_classification_loss.py
new file mode 100644
index 0000000..3cda4d6
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_losses/test_classification_loss.py
@@ -0,0 +1,40 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import torch
+
+
+def test_bce_loss():
+ from mmpose.models import build_loss
+
+ # test BCE loss without target weight(None)
+ loss_cfg = dict(type='BCELoss')
+ loss = build_loss(loss_cfg)
+
+ fake_pred = torch.zeros((1, 2))
+ fake_label = torch.zeros((1, 2))
+ assert torch.allclose(loss(fake_pred, fake_label), torch.tensor(0.))
+
+ fake_pred = torch.ones((1, 2)) * 0.5
+ fake_label = torch.zeros((1, 2))
+ assert torch.allclose(
+ loss(fake_pred, fake_label), -torch.log(torch.tensor(0.5)))
+
+ # test BCE loss with target weight
+ loss_cfg = dict(type='BCELoss', use_target_weight=True)
+ loss = build_loss(loss_cfg)
+
+ fake_pred = torch.ones((1, 2)) * 0.5
+ fake_label = torch.zeros((1, 2))
+ fake_weight = torch.ones((1, 2))
+ assert torch.allclose(
+ loss(fake_pred, fake_label, fake_weight),
+ -torch.log(torch.tensor(0.5)))
+
+ fake_weight[:, 0] = 0
+ assert torch.allclose(
+ loss(fake_pred, fake_label, fake_weight),
+ -0.5 * torch.log(torch.tensor(0.5)))
+
+ fake_weight = torch.ones(1)
+ assert torch.allclose(
+ loss(fake_pred, fake_label, fake_weight),
+ -torch.log(torch.tensor(0.5)))
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_losses/test_mesh_losses.py b/engine/pose_estimation/third-party/ViTPose/tests/test_losses/test_mesh_losses.py
new file mode 100644
index 0000000..9890767
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_losses/test_mesh_losses.py
@@ -0,0 +1,163 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import pytest
+import torch
+from numpy.testing import assert_almost_equal
+
+from mmpose.models import build_loss
+from mmpose.models.utils.geometry import batch_rodrigues
+
+
+def test_mesh_loss():
+ """test mesh loss."""
+ loss_cfg = dict(
+ type='MeshLoss',
+ joints_2d_loss_weight=1,
+ joints_3d_loss_weight=1,
+ vertex_loss_weight=1,
+ smpl_pose_loss_weight=1,
+ smpl_beta_loss_weight=1,
+ img_res=256,
+ focal_length=5000)
+
+ loss = build_loss(loss_cfg)
+
+ smpl_pose = torch.zeros([1, 72], dtype=torch.float32)
+ smpl_rotmat = batch_rodrigues(smpl_pose.view(-1, 3)).view(-1, 24, 3, 3)
+ smpl_beta = torch.zeros([1, 10], dtype=torch.float32)
+ camera = torch.tensor([[1, 0, 0]], dtype=torch.float32)
+ vertices = torch.rand([1, 6890, 3], dtype=torch.float32)
+ joints_3d = torch.ones([1, 24, 3], dtype=torch.float32)
+ joints_2d = loss.project_points(joints_3d, camera) + (256 - 1) / 2
+
+ fake_pred = {}
+ fake_pred['pose'] = smpl_rotmat
+ fake_pred['beta'] = smpl_beta
+ fake_pred['camera'] = camera
+ fake_pred['vertices'] = vertices
+ fake_pred['joints_3d'] = joints_3d
+
+ fake_gt = {}
+ fake_gt['pose'] = smpl_pose
+ fake_gt['beta'] = smpl_beta
+ fake_gt['vertices'] = vertices
+ fake_gt['has_smpl'] = torch.ones(1, dtype=torch.float32)
+ fake_gt['joints_3d'] = joints_3d
+ fake_gt['joints_3d_visible'] = torch.ones([1, 24, 1], dtype=torch.float32)
+ fake_gt['joints_2d'] = joints_2d
+ fake_gt['joints_2d_visible'] = torch.ones([1, 24, 1], dtype=torch.float32)
+
+ losses = loss(fake_pred, fake_gt)
+ assert torch.allclose(losses['vertex_loss'], torch.tensor(0.))
+ assert torch.allclose(losses['smpl_pose_loss'], torch.tensor(0.))
+ assert torch.allclose(losses['smpl_beta_loss'], torch.tensor(0.))
+ assert torch.allclose(losses['joints_3d_loss'], torch.tensor(0.))
+ assert torch.allclose(losses['joints_2d_loss'], torch.tensor(0.))
+
+ fake_pred = {}
+ fake_pred['pose'] = smpl_rotmat + 1
+ fake_pred['beta'] = smpl_beta + 1
+ fake_pred['camera'] = camera
+ fake_pred['vertices'] = vertices + 1
+ fake_pred['joints_3d'] = joints_3d.clone()
+
+ joints_3d_t = joints_3d.clone()
+ joints_3d_t[:, 0] = joints_3d_t[:, 0] + 1
+ fake_gt = {}
+ fake_gt['pose'] = smpl_pose
+ fake_gt['beta'] = smpl_beta
+ fake_gt['vertices'] = vertices
+ fake_gt['has_smpl'] = torch.ones(1, dtype=torch.float32)
+ fake_gt['joints_3d'] = joints_3d_t
+ fake_gt['joints_3d_visible'] = torch.ones([1, 24, 1], dtype=torch.float32)
+ fake_gt['joints_2d'] = joints_2d + (256 - 1) / 2
+ fake_gt['joints_2d_visible'] = torch.ones([1, 24, 1], dtype=torch.float32)
+
+ losses = loss(fake_pred, fake_gt)
+ assert torch.allclose(losses['vertex_loss'], torch.tensor(1.))
+ assert torch.allclose(losses['smpl_pose_loss'], torch.tensor(1.))
+ assert torch.allclose(losses['smpl_beta_loss'], torch.tensor(1.))
+ assert torch.allclose(losses['joints_3d_loss'], torch.tensor(0.5 / 24))
+ assert torch.allclose(losses['joints_2d_loss'], torch.tensor(0.5))
+
+
+def test_gan_loss():
+ """test gan loss."""
+ with pytest.raises(NotImplementedError):
+ loss_cfg = dict(
+ type='GANLoss',
+ gan_type='test',
+ real_label_val=1.0,
+ fake_label_val=0.0,
+ loss_weight=1)
+ _ = build_loss(loss_cfg)
+
+ input_1 = torch.ones(1, 1)
+ input_2 = torch.ones(1, 3, 6, 6) * 2
+
+ # vanilla
+ loss_cfg = dict(
+ type='GANLoss',
+ gan_type='vanilla',
+ real_label_val=1.0,
+ fake_label_val=0.0,
+ loss_weight=2.0)
+ gan_loss = build_loss(loss_cfg)
+ loss = gan_loss(input_1, True, is_disc=False)
+ assert_almost_equal(loss.item(), 0.6265233)
+ loss = gan_loss(input_1, False, is_disc=False)
+ assert_almost_equal(loss.item(), 2.6265232)
+ loss = gan_loss(input_1, True, is_disc=True)
+ assert_almost_equal(loss.item(), 0.3132616)
+ loss = gan_loss(input_1, False, is_disc=True)
+ assert_almost_equal(loss.item(), 1.3132616)
+
+ # lsgan
+ loss_cfg = dict(
+ type='GANLoss',
+ gan_type='lsgan',
+ real_label_val=1.0,
+ fake_label_val=0.0,
+ loss_weight=2.0)
+ gan_loss = build_loss(loss_cfg)
+ loss = gan_loss(input_2, True, is_disc=False)
+ assert_almost_equal(loss.item(), 2.0)
+ loss = gan_loss(input_2, False, is_disc=False)
+ assert_almost_equal(loss.item(), 8.0)
+ loss = gan_loss(input_2, True, is_disc=True)
+ assert_almost_equal(loss.item(), 1.0)
+ loss = gan_loss(input_2, False, is_disc=True)
+ assert_almost_equal(loss.item(), 4.0)
+
+ # wgan
+ loss_cfg = dict(
+ type='GANLoss',
+ gan_type='wgan',
+ real_label_val=1.0,
+ fake_label_val=0.0,
+ loss_weight=2.0)
+ gan_loss = build_loss(loss_cfg)
+ loss = gan_loss(input_2, True, is_disc=False)
+ assert_almost_equal(loss.item(), -4.0)
+ loss = gan_loss(input_2, False, is_disc=False)
+ assert_almost_equal(loss.item(), 4)
+ loss = gan_loss(input_2, True, is_disc=True)
+ assert_almost_equal(loss.item(), -2.0)
+ loss = gan_loss(input_2, False, is_disc=True)
+ assert_almost_equal(loss.item(), 2.0)
+
+ # hinge
+ loss_cfg = dict(
+ type='GANLoss',
+ gan_type='hinge',
+ real_label_val=1.0,
+ fake_label_val=0.0,
+ loss_weight=2.0)
+ gan_loss = build_loss(loss_cfg)
+ loss = gan_loss(input_2, True, is_disc=False)
+ assert_almost_equal(loss.item(), -4.0)
+ loss = gan_loss(input_2, False, is_disc=False)
+ assert_almost_equal(loss.item(), -4.0)
+ loss = gan_loss(input_2, True, is_disc=True)
+ assert_almost_equal(loss.item(), 0.0)
+ loss = gan_loss(input_2, False, is_disc=True)
+ assert_almost_equal(loss.item(), 3.0)
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_losses/test_regression_losses.py b/engine/pose_estimation/third-party/ViTPose/tests/test_losses/test_regression_losses.py
new file mode 100644
index 0000000..df710ba
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_losses/test_regression_losses.py
@@ -0,0 +1,185 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import torch
+
+from mmpose.models import build_loss
+
+
+def test_smooth_l1_loss():
+ # test SmoothL1Loss without target weight(default None)
+ loss_cfg = dict(type='SmoothL1Loss')
+ loss = build_loss(loss_cfg)
+
+ fake_pred = torch.zeros((1, 3, 2))
+ fake_label = torch.zeros((1, 3, 2))
+ assert torch.allclose(loss(fake_pred, fake_label), torch.tensor(0.))
+
+ fake_pred = torch.ones((1, 3, 2))
+ fake_label = torch.zeros((1, 3, 2))
+ assert torch.allclose(loss(fake_pred, fake_label), torch.tensor(.5))
+
+ # test SmoothL1Loss with target weight
+ loss_cfg = dict(type='SmoothL1Loss', use_target_weight=True)
+ loss = build_loss(loss_cfg)
+
+ fake_pred = torch.zeros((1, 3, 2))
+ fake_label = torch.zeros((1, 3, 2))
+ assert torch.allclose(
+ loss(fake_pred, fake_label, torch.ones_like(fake_label)),
+ torch.tensor(0.))
+
+ fake_pred = torch.ones((1, 3, 2))
+ fake_label = torch.zeros((1, 3, 2))
+ assert torch.allclose(
+ loss(fake_pred, fake_label, torch.ones_like(fake_label)),
+ torch.tensor(.5))
+
+
+def test_wing_loss():
+ # test WingLoss without target weight(default None)
+ loss_cfg = dict(type='WingLoss')
+ loss = build_loss(loss_cfg)
+
+ fake_pred = torch.zeros((1, 3, 2))
+ fake_label = torch.zeros((1, 3, 2))
+ assert torch.allclose(loss(fake_pred, fake_label), torch.tensor(0.))
+
+ fake_pred = torch.ones((1, 3, 2))
+ fake_label = torch.zeros((1, 3, 2))
+ assert torch.gt(loss(fake_pred, fake_label), torch.tensor(.5))
+
+ # test WingLoss with target weight
+ loss_cfg = dict(type='WingLoss', use_target_weight=True)
+ loss = build_loss(loss_cfg)
+
+ fake_pred = torch.zeros((1, 3, 2))
+ fake_label = torch.zeros((1, 3, 2))
+ assert torch.allclose(
+ loss(fake_pred, fake_label, torch.ones_like(fake_label)),
+ torch.tensor(0.))
+
+ fake_pred = torch.ones((1, 3, 2))
+ fake_label = torch.zeros((1, 3, 2))
+ assert torch.gt(
+ loss(fake_pred, fake_label, torch.ones_like(fake_label)),
+ torch.tensor(.5))
+
+
+def test_soft_wing_loss():
+ # test SoftWingLoss without target weight(default None)
+ loss_cfg = dict(type='SoftWingLoss')
+ loss = build_loss(loss_cfg)
+
+ fake_pred = torch.zeros((1, 3, 2))
+ fake_label = torch.zeros((1, 3, 2))
+ assert torch.allclose(loss(fake_pred, fake_label), torch.tensor(0.))
+
+ fake_pred = torch.ones((1, 3, 2))
+ fake_label = torch.zeros((1, 3, 2))
+ assert torch.gt(loss(fake_pred, fake_label), torch.tensor(.5))
+
+ # test SoftWingLoss with target weight
+ loss_cfg = dict(type='SoftWingLoss', use_target_weight=True)
+ loss = build_loss(loss_cfg)
+
+ fake_pred = torch.zeros((1, 3, 2))
+ fake_label = torch.zeros((1, 3, 2))
+ assert torch.allclose(
+ loss(fake_pred, fake_label, torch.ones_like(fake_label)),
+ torch.tensor(0.))
+
+ fake_pred = torch.ones((1, 3, 2))
+ fake_label = torch.zeros((1, 3, 2))
+ assert torch.gt(
+ loss(fake_pred, fake_label, torch.ones_like(fake_label)),
+ torch.tensor(.5))
+
+
+def test_mse_regression_loss():
+ # w/o target weight(default None)
+ loss_cfg = dict(type='MSELoss')
+ loss = build_loss(loss_cfg)
+ fake_pred = torch.zeros((1, 3, 3))
+ fake_label = torch.zeros((1, 3, 3))
+ assert torch.allclose(loss(fake_pred, fake_label), torch.tensor(0.))
+
+ fake_pred = torch.ones((1, 3, 3))
+ fake_label = torch.zeros((1, 3, 3))
+ assert torch.allclose(loss(fake_pred, fake_label), torch.tensor(1.))
+
+ # w/ target weight
+ loss_cfg = dict(type='MSELoss', use_target_weight=True)
+ loss = build_loss(loss_cfg)
+ fake_pred = torch.zeros((1, 3, 3))
+ fake_label = torch.zeros((1, 3, 3))
+ assert torch.allclose(
+ loss(fake_pred, fake_label, torch.ones_like(fake_label)),
+ torch.tensor(0.))
+
+ fake_pred = torch.ones((1, 3, 3))
+ fake_label = torch.zeros((1, 3, 3))
+ assert torch.allclose(
+ loss(fake_pred, fake_label, torch.ones_like(fake_label)),
+ torch.tensor(1.))
+
+
+def test_bone_loss():
+ # w/o target weight(default None)
+ loss_cfg = dict(type='BoneLoss', joint_parents=[0, 0, 1])
+ loss = build_loss(loss_cfg)
+ fake_pred = torch.zeros((1, 3, 3))
+ fake_label = torch.zeros((1, 3, 3))
+ assert torch.allclose(loss(fake_pred, fake_label), torch.tensor(0.))
+
+ fake_pred = torch.tensor([[[0, 0, 0], [1, 1, 1], [2, 2, 2]]],
+ dtype=torch.float32)
+ fake_label = fake_pred * 2
+ assert torch.allclose(loss(fake_pred, fake_label), torch.tensor(3**0.5))
+
+ # w/ target weight
+ loss_cfg = dict(
+ type='BoneLoss', joint_parents=[0, 0, 1], use_target_weight=True)
+ loss = build_loss(loss_cfg)
+ fake_pred = torch.zeros((1, 3, 3))
+ fake_label = torch.zeros((1, 3, 3))
+ fake_weight = torch.ones((1, 2))
+ assert torch.allclose(
+ loss(fake_pred, fake_label, fake_weight), torch.tensor(0.))
+
+ fake_pred = torch.tensor([[[0, 0, 0], [1, 1, 1], [2, 2, 2]]],
+ dtype=torch.float32)
+ fake_label = fake_pred * 2
+ fake_weight = torch.ones((1, 2))
+ assert torch.allclose(
+ loss(fake_pred, fake_label, fake_weight), torch.tensor(3**0.5))
+
+
+def test_semi_supervision_loss():
+ loss_cfg = dict(
+ type='SemiSupervisionLoss',
+ joint_parents=[0, 0, 1],
+ warmup_iterations=1)
+ loss = build_loss(loss_cfg)
+
+ unlabeled_pose = torch.rand((1, 3, 3))
+ unlabeled_traj = torch.ones((1, 1, 3))
+ labeled_pose = unlabeled_pose.clone()
+ fake_pred = dict(
+ labeled_pose=labeled_pose,
+ unlabeled_pose=unlabeled_pose,
+ unlabeled_traj=unlabeled_traj)
+
+ intrinsics = torch.tensor([[1, 1, 1, 1, 0.1, 0.1, 0.1, 0, 0]],
+ dtype=torch.float32)
+ unlabled_target_2d = loss.project_joints(unlabeled_pose + unlabeled_traj,
+ intrinsics)
+ fake_label = dict(
+ unlabeled_target_2d=unlabled_target_2d, intrinsics=intrinsics)
+
+ # test warmup
+ losses = loss(fake_pred, fake_label)
+ assert not losses
+
+ # test semi-supervised loss
+ losses = loss(fake_pred, fake_label)
+ assert torch.allclose(losses['proj_loss'], torch.tensor(0.))
+ assert torch.allclose(losses['bone_loss'], torch.tensor(0.))
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_losses/test_top_down_losses.py b/engine/pose_estimation/third-party/ViTPose/tests/test_losses/test_top_down_losses.py
new file mode 100644
index 0000000..a02595f
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_losses/test_top_down_losses.py
@@ -0,0 +1,98 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import pytest
+import torch
+
+from mmpose.models import build_loss
+
+
+def test_adaptive_wing_loss():
+ # test Adaptive WingLoss without target weight
+ loss_cfg = dict(type='AdaptiveWingLoss')
+ loss = build_loss(loss_cfg)
+
+ fake_pred = torch.zeros((1, 3, 64, 64))
+ fake_label = torch.zeros((1, 3, 64, 64))
+ assert torch.allclose(loss(fake_pred, fake_label, None), torch.tensor(0.))
+
+ # test WingLoss with target weight
+ loss_cfg = dict(type='AdaptiveWingLoss', use_target_weight=True)
+ loss = build_loss(loss_cfg)
+
+ fake_pred = torch.ones((1, 3, 64, 64))
+ fake_label = torch.ones((1, 3, 64, 64))
+ assert torch.allclose(
+ loss(fake_pred, fake_label, torch.ones([1, 3, 1])), torch.tensor(0.))
+
+
+def test_mse_loss():
+ # test MSE loss without target weight
+ loss_cfg = dict(type='JointsMSELoss')
+ loss = build_loss(loss_cfg)
+
+ fake_pred = torch.zeros((1, 3, 64, 64))
+ fake_label = torch.zeros((1, 3, 64, 64))
+ assert torch.allclose(loss(fake_pred, fake_label, None), torch.tensor(0.))
+
+ fake_pred = torch.ones((1, 3, 64, 64))
+ fake_label = torch.zeros((1, 3, 64, 64))
+ assert torch.allclose(loss(fake_pred, fake_label, None), torch.tensor(1.))
+
+ fake_pred = torch.zeros((1, 2, 64, 64))
+ fake_pred[0, 0] += 1
+ fake_label = torch.zeros((1, 2, 64, 64))
+ assert torch.allclose(loss(fake_pred, fake_label, None), torch.tensor(0.5))
+
+ with pytest.raises(ValueError):
+ loss_cfg = dict(type='JointsOHKMMSELoss')
+ loss = build_loss(loss_cfg)
+ fake_pred = torch.zeros((1, 3, 64, 64))
+ fake_label = torch.zeros((1, 3, 64, 64))
+ assert torch.allclose(
+ loss(fake_pred, fake_label, None), torch.tensor(0.))
+
+ with pytest.raises(AssertionError):
+ loss_cfg = dict(type='JointsOHKMMSELoss', topk=-1)
+ loss = build_loss(loss_cfg)
+ fake_pred = torch.zeros((1, 3, 64, 64))
+ fake_label = torch.zeros((1, 3, 64, 64))
+ assert torch.allclose(
+ loss(fake_pred, fake_label, None), torch.tensor(0.))
+
+ loss_cfg = dict(type='JointsOHKMMSELoss', topk=2)
+ loss = build_loss(loss_cfg)
+ fake_pred = torch.ones((1, 3, 64, 64))
+ fake_label = torch.zeros((1, 3, 64, 64))
+ assert torch.allclose(loss(fake_pred, fake_label, None), torch.tensor(1.))
+
+ loss_cfg = dict(type='JointsOHKMMSELoss', topk=2)
+ loss = build_loss(loss_cfg)
+ fake_pred = torch.zeros((1, 3, 64, 64))
+ fake_pred[0, 0] += 1
+ fake_label = torch.zeros((1, 3, 64, 64))
+ assert torch.allclose(loss(fake_pred, fake_label, None), torch.tensor(0.5))
+
+ loss_cfg = dict(type='CombinedTargetMSELoss', use_target_weight=True)
+ loss = build_loss(loss_cfg)
+ fake_pred = torch.ones((1, 3, 64, 64))
+ fake_label = torch.zeros((1, 3, 64, 64))
+ target_weight = torch.ones((1, 1, 1))
+ assert torch.allclose(
+ loss(fake_pred, fake_label, target_weight), torch.tensor(0.5))
+
+ loss_cfg = dict(type='CombinedTargetMSELoss', use_target_weight=True)
+ loss = build_loss(loss_cfg)
+ fake_pred = torch.ones((1, 3, 64, 64))
+ fake_label = torch.zeros((1, 3, 64, 64))
+ target_weight = torch.zeros((1, 1, 1))
+ assert torch.allclose(
+ loss(fake_pred, fake_label, target_weight), torch.tensor(0.))
+
+
+def test_smoothl1_loss():
+ # test MSE loss without target weight
+ loss_cfg = dict(type='SmoothL1Loss')
+ loss = build_loss(loss_cfg)
+
+ fake_pred = torch.zeros((1, 3))
+ fake_label = torch.zeros((1, 3))
+ assert torch.allclose(loss(fake_pred, fake_label, None), torch.tensor(0.))
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_models/test_bottom_up_forward.py b/engine/pose_estimation/third-party/ViTPose/tests/test_models/test_bottom_up_forward.py
new file mode 100644
index 0000000..37e6c5e
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_models/test_bottom_up_forward.py
@@ -0,0 +1,122 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import numpy as np
+import torch
+
+from mmpose.models.detectors import AssociativeEmbedding
+
+
+def test_ae_forward():
+ model_cfg = dict(
+ type='AssociativeEmbedding',
+ pretrained=None,
+ backbone=dict(type='ResNet', depth=18),
+ keypoint_head=dict(
+ type='AESimpleHead',
+ in_channels=512,
+ num_joints=17,
+ num_deconv_layers=0,
+ tag_per_joint=True,
+ with_ae_loss=[True],
+ extra=dict(final_conv_kernel=1, ),
+ loss_keypoint=dict(
+ type='MultiLossFactory',
+ num_joints=17,
+ num_stages=1,
+ ae_loss_type='exp',
+ with_ae_loss=[True],
+ push_loss_factor=[0.001],
+ pull_loss_factor=[0.001],
+ with_heatmaps_loss=[True],
+ heatmaps_loss_factor=[1.0])),
+ train_cfg=dict(),
+ test_cfg=dict(
+ num_joints=17,
+ max_num_people=30,
+ scale_factor=[1],
+ with_heatmaps=[True],
+ with_ae=[True],
+ project2image=True,
+ nms_kernel=5,
+ nms_padding=2,
+ tag_per_joint=True,
+ detection_threshold=0.1,
+ tag_threshold=1,
+ use_detection_val=True,
+ ignore_too_much=False,
+ adjust=True,
+ refine=True,
+ soft_nms=False,
+ flip_test=True,
+ post_process=True,
+ shift_heatmap=True,
+ use_gt_bbox=True,
+ flip_pairs=[[1, 2], [3, 4], [5, 6], [7, 8], [9, 10], [11, 12],
+ [13, 14], [15, 16]],
+ ))
+
+ detector = AssociativeEmbedding(model_cfg['backbone'],
+ model_cfg['keypoint_head'],
+ model_cfg['train_cfg'],
+ model_cfg['test_cfg'],
+ model_cfg['pretrained'])
+
+ detector.init_weights()
+
+ input_shape = (1, 3, 256, 256)
+ mm_inputs = _demo_mm_inputs(input_shape)
+
+ imgs = mm_inputs.pop('imgs')
+ target = mm_inputs.pop('target')
+ mask = mm_inputs.pop('mask')
+ joints = mm_inputs.pop('joints')
+ img_metas = mm_inputs.pop('img_metas')
+
+ # Test forward train
+ losses = detector.forward(
+ imgs, target, mask, joints, img_metas, return_loss=True)
+ assert isinstance(losses, dict)
+
+ # Test forward test
+ with torch.no_grad():
+ _ = detector.forward(imgs, img_metas=img_metas, return_loss=False)
+ _ = detector.forward_dummy(imgs)
+
+
+def _demo_mm_inputs(input_shape=(1, 3, 256, 256)):
+ """Create a superset of inputs needed to run test or train batches.
+
+ Args:
+ input_shape (tuple):
+ input batch dimensions
+ """
+ (N, C, H, W) = input_shape
+
+ rng = np.random.RandomState(0)
+
+ imgs = rng.rand(*input_shape)
+ target = np.zeros([N, 17, H // 32, W // 32], dtype=np.float32)
+ mask = np.ones([N, H // 32, W // 32], dtype=np.float32)
+ joints = np.zeros([N, 30, 17, 2], dtype=np.float32)
+
+ img_metas = [{
+ 'image_file':
+ 'test.jpg',
+ 'aug_data': [torch.zeros(1, 3, 256, 256)],
+ 'test_scale_factor': [1],
+ 'base_size': (256, 256),
+ 'center':
+ np.array([128, 128]),
+ 'scale':
+ np.array([1.28, 1.28]),
+ 'flip_index':
+ [0, 2, 1, 4, 3, 6, 5, 8, 7, 10, 9, 12, 11, 14, 13, 16, 15]
+ } for _ in range(N)]
+
+ mm_inputs = {
+ 'imgs': torch.FloatTensor(imgs).requires_grad_(True),
+ 'target': [torch.FloatTensor(target)],
+ 'mask': [torch.FloatTensor(mask)],
+ 'joints': [torch.FloatTensor(joints)],
+ 'img_metas': img_metas
+ }
+ return mm_inputs
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_models/test_bottom_up_head.py b/engine/pose_estimation/third-party/ViTPose/tests/test_models/test_bottom_up_head.py
new file mode 100644
index 0000000..4748f31
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_models/test_bottom_up_head.py
@@ -0,0 +1,483 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import numpy as np
+import pytest
+import torch
+
+from mmpose.models import AEHigherResolutionHead, AESimpleHead
+
+
+def test_ae_simple_head():
+ """test bottom up AE simple head."""
+
+ with pytest.raises(TypeError):
+ # extra
+ _ = AESimpleHead(
+ in_channels=512,
+ num_joints=17,
+ with_ae_loss=[True],
+ extra=[],
+ loss_keypoint=dict(
+ type='MultiLossFactory',
+ num_joints=17,
+ num_stages=1,
+ ae_loss_type='exp',
+ with_ae_loss=[True],
+ push_loss_factor=[0.001],
+ pull_loss_factor=[0.001],
+ with_heatmaps_loss=[True],
+ heatmaps_loss_factor=[1.0]))
+ # test final_conv_kernel
+ with pytest.raises(AssertionError):
+ _ = AESimpleHead(
+ in_channels=512,
+ num_joints=17,
+ with_ae_loss=[True],
+ extra={'final_conv_kernel': -1},
+ loss_keypoint=dict(
+ type='MultiLossFactory',
+ num_joints=17,
+ num_stages=1,
+ ae_loss_type='exp',
+ with_ae_loss=[True],
+ push_loss_factor=[0.001],
+ pull_loss_factor=[0.001],
+ with_heatmaps_loss=[True],
+ heatmaps_loss_factor=[1.0]))
+ head = AESimpleHead(
+ in_channels=512,
+ num_joints=17,
+ with_ae_loss=[True],
+ extra={'final_conv_kernel': 3},
+ loss_keypoint=dict(
+ type='MultiLossFactory',
+ num_joints=17,
+ num_stages=1,
+ ae_loss_type='exp',
+ with_ae_loss=[True],
+ push_loss_factor=[0.001],
+ pull_loss_factor=[0.001],
+ with_heatmaps_loss=[True],
+ heatmaps_loss_factor=[1.0]))
+ head.init_weights()
+ assert head.final_layer.padding == (1, 1)
+ head = AESimpleHead(
+ in_channels=512,
+ num_joints=17,
+ with_ae_loss=[True],
+ extra={'final_conv_kernel': 1},
+ loss_keypoint=dict(
+ type='MultiLossFactory',
+ num_joints=17,
+ num_stages=1,
+ ae_loss_type='exp',
+ with_ae_loss=[True],
+ push_loss_factor=[0.001],
+ pull_loss_factor=[0.001],
+ with_heatmaps_loss=[True],
+ heatmaps_loss_factor=[1.0]))
+ head.init_weights()
+ assert head.final_layer.padding == (0, 0)
+ head = AESimpleHead(
+ in_channels=512,
+ num_joints=17,
+ with_ae_loss=[True],
+ loss_keypoint=dict(
+ type='MultiLossFactory',
+ num_joints=17,
+ num_stages=1,
+ ae_loss_type='exp',
+ with_ae_loss=[True],
+ push_loss_factor=[0.001],
+ pull_loss_factor=[0.001],
+ with_heatmaps_loss=[True],
+ heatmaps_loss_factor=[1.0]))
+ head.init_weights()
+ assert head.final_layer.padding == (0, 0)
+ # test with_ae_loss
+ head = AESimpleHead(
+ in_channels=512,
+ num_joints=17,
+ num_deconv_layers=0,
+ with_ae_loss=[True],
+ extra={'final_conv_kernel': 3},
+ loss_keypoint=dict(
+ type='MultiLossFactory',
+ num_joints=17,
+ num_stages=1,
+ ae_loss_type='exp',
+ with_ae_loss=[True],
+ push_loss_factor=[0.001],
+ pull_loss_factor=[0.001],
+ with_heatmaps_loss=[True],
+ heatmaps_loss_factor=[1.0]))
+ head.init_weights()
+ input_shape = (1, 512, 32, 32)
+ inputs = _demo_inputs(input_shape)
+ out = head(inputs)
+ assert out[0].shape == torch.Size([1, 34, 32, 32])
+ head = AESimpleHead(
+ in_channels=512,
+ num_joints=17,
+ num_deconv_layers=0,
+ with_ae_loss=[False],
+ extra={'final_conv_kernel': 3},
+ loss_keypoint=dict(
+ type='MultiLossFactory',
+ num_joints=17,
+ num_stages=1,
+ ae_loss_type='exp',
+ with_ae_loss=[True],
+ push_loss_factor=[0.001],
+ pull_loss_factor=[0.001],
+ with_heatmaps_loss=[True],
+ heatmaps_loss_factor=[1.0]))
+ head.init_weights()
+ input_shape = (1, 512, 32, 32)
+ inputs = _demo_inputs(input_shape)
+ out = head(inputs)
+ assert out[0].shape == torch.Size([1, 17, 32, 32])
+ # test tag_per_joint
+ head = AESimpleHead(
+ in_channels=512,
+ num_joints=17,
+ num_deconv_layers=0,
+ tag_per_joint=False,
+ with_ae_loss=[False],
+ extra={'final_conv_kernel': 3},
+ loss_keypoint=dict(
+ type='MultiLossFactory',
+ num_joints=17,
+ num_stages=1,
+ ae_loss_type='exp',
+ with_ae_loss=[True],
+ push_loss_factor=[0.001],
+ pull_loss_factor=[0.001],
+ with_heatmaps_loss=[True],
+ heatmaps_loss_factor=[1.0]))
+ head.init_weights()
+ input_shape = (1, 512, 32, 32)
+ inputs = _demo_inputs(input_shape)
+ out = head(inputs)
+ assert out[0].shape == torch.Size([1, 17, 32, 32])
+ head = AESimpleHead(
+ in_channels=512,
+ num_joints=17,
+ num_deconv_layers=0,
+ tag_per_joint=False,
+ with_ae_loss=[True],
+ extra={'final_conv_kernel': 3},
+ loss_keypoint=dict(
+ type='MultiLossFactory',
+ num_joints=17,
+ num_stages=1,
+ ae_loss_type='exp',
+ with_ae_loss=[True],
+ push_loss_factor=[0.001],
+ pull_loss_factor=[0.001],
+ with_heatmaps_loss=[True],
+ heatmaps_loss_factor=[1.0]))
+ head.init_weights()
+ input_shape = (1, 512, 32, 32)
+ inputs = _demo_inputs(input_shape)
+ out = head(inputs)
+ assert out[0].shape == torch.Size([1, 18, 32, 32])
+ head = AESimpleHead(
+ in_channels=512,
+ num_joints=17,
+ num_deconv_layers=0,
+ tag_per_joint=False,
+ with_ae_loss=[True],
+ extra={'final_conv_kernel': 3},
+ loss_keypoint=dict(
+ type='MultiLossFactory',
+ num_joints=17,
+ num_stages=1,
+ ae_loss_type='exp',
+ with_ae_loss=[True],
+ push_loss_factor=[0.001],
+ pull_loss_factor=[0.001],
+ with_heatmaps_loss=[True],
+ heatmaps_loss_factor=[1.0]))
+ head.init_weights()
+ input_shape = (1, 512, 32, 32)
+ inputs = _demo_inputs(input_shape)
+ out = head([inputs])
+ assert out[0].shape == torch.Size([1, 18, 32, 32])
+
+
+def test_ae_higherresolution_head():
+ """test bottom up AE higherresolution head."""
+
+ # test final_conv_kernel
+ with pytest.raises(AssertionError):
+ _ = AEHigherResolutionHead(
+ in_channels=512,
+ num_joints=17,
+ with_ae_loss=[True, False],
+ extra={'final_conv_kernel': 0},
+ loss_keypoint=dict(
+ type='MultiLossFactory',
+ num_joints=17,
+ num_stages=2,
+ ae_loss_type='exp',
+ with_ae_loss=[True, False],
+ push_loss_factor=[0.001, 0.001],
+ pull_loss_factor=[0.001, 0.001],
+ with_heatmaps_loss=[True, True],
+ heatmaps_loss_factor=[1.0, 1.0]))
+ head = AEHigherResolutionHead(
+ in_channels=512,
+ num_joints=17,
+ with_ae_loss=[True, False],
+ extra={'final_conv_kernel': 3},
+ cat_output=[True],
+ loss_keypoint=dict(
+ type='MultiLossFactory',
+ num_joints=17,
+ num_stages=2,
+ ae_loss_type='exp',
+ with_ae_loss=[True, False],
+ push_loss_factor=[0.001, 0.001],
+ pull_loss_factor=[0.001, 0.001],
+ with_heatmaps_loss=[True, True],
+ heatmaps_loss_factor=[1.0, 1.0]))
+ head.init_weights()
+ assert head.final_layers[0].padding == (1, 1)
+ head = AEHigherResolutionHead(
+ in_channels=512,
+ num_joints=17,
+ with_ae_loss=[True, False],
+ extra={'final_conv_kernel': 1},
+ cat_output=[True],
+ loss_keypoint=dict(
+ type='MultiLossFactory',
+ num_joints=17,
+ num_stages=2,
+ ae_loss_type='exp',
+ with_ae_loss=[True, False],
+ push_loss_factor=[0.001, 0.001],
+ pull_loss_factor=[0.001, 0.001],
+ with_heatmaps_loss=[True, True],
+ heatmaps_loss_factor=[1.0, 1.0]))
+ head.init_weights()
+ assert head.final_layers[0].padding == (0, 0)
+ head = AEHigherResolutionHead(
+ in_channels=512,
+ num_joints=17,
+ with_ae_loss=[True, False],
+ cat_output=[True],
+ loss_keypoint=dict(
+ type='MultiLossFactory',
+ num_joints=17,
+ num_stages=2,
+ ae_loss_type='exp',
+ with_ae_loss=[True, False],
+ push_loss_factor=[0.001, 0.001],
+ pull_loss_factor=[0.001, 0.001],
+ with_heatmaps_loss=[True, True],
+ heatmaps_loss_factor=[1.0, 1.0]))
+ head.init_weights()
+ assert head.final_layers[0].padding == (0, 0)
+ # test deconv layers
+ with pytest.raises(ValueError):
+ _ = AEHigherResolutionHead(
+ in_channels=512,
+ num_joints=17,
+ with_ae_loss=[True, False],
+ num_deconv_kernels=[1],
+ cat_output=[True],
+ loss_keypoint=dict(
+ type='MultiLossFactory',
+ num_joints=17,
+ num_stages=2,
+ ae_loss_type='exp',
+ with_ae_loss=[True, False],
+ push_loss_factor=[0.001, 0.001],
+ pull_loss_factor=[0.001, 0.001],
+ with_heatmaps_loss=[True, True],
+ heatmaps_loss_factor=[1.0, 1.0]))
+ head = AEHigherResolutionHead(
+ in_channels=512,
+ num_joints=17,
+ with_ae_loss=[True, False],
+ num_deconv_kernels=[4],
+ cat_output=[True],
+ loss_keypoint=dict(
+ type='MultiLossFactory',
+ num_joints=17,
+ num_stages=2,
+ ae_loss_type='exp',
+ with_ae_loss=[True, False],
+ push_loss_factor=[0.001, 0.001],
+ pull_loss_factor=[0.001, 0.001],
+ with_heatmaps_loss=[True, True],
+ heatmaps_loss_factor=[1.0, 1.0]))
+ head.init_weights()
+ assert head.deconv_layers[0][0][0].output_padding == (0, 0)
+ head = AEHigherResolutionHead(
+ in_channels=512,
+ num_joints=17,
+ with_ae_loss=[True, False],
+ num_deconv_kernels=[3],
+ cat_output=[True],
+ loss_keypoint=dict(
+ type='MultiLossFactory',
+ num_joints=17,
+ num_stages=2,
+ ae_loss_type='exp',
+ with_ae_loss=[True, False],
+ push_loss_factor=[0.001, 0.001],
+ pull_loss_factor=[0.001, 0.001],
+ with_heatmaps_loss=[True, True],
+ heatmaps_loss_factor=[1.0, 1.0]))
+ head.init_weights()
+ assert head.deconv_layers[0][0][0].output_padding == (1, 1)
+ head = AEHigherResolutionHead(
+ in_channels=512,
+ num_joints=17,
+ with_ae_loss=[True, False],
+ num_deconv_kernels=[2],
+ cat_output=[True],
+ loss_keypoint=dict(
+ type='MultiLossFactory',
+ num_joints=17,
+ num_stages=2,
+ ae_loss_type='exp',
+ with_ae_loss=[True, False],
+ push_loss_factor=[0.001, 0.001],
+ pull_loss_factor=[0.001, 0.001],
+ with_heatmaps_loss=[True, True],
+ heatmaps_loss_factor=[1.0, 1.0]))
+ head.init_weights()
+ assert head.deconv_layers[0][0][0].output_padding == (0, 0)
+ # test tag_per_joint & ae loss
+ head = AEHigherResolutionHead(
+ in_channels=512,
+ num_joints=17,
+ tag_per_joint=False,
+ with_ae_loss=[False, False],
+ extra={'final_conv_kernel': 3},
+ cat_output=[True],
+ loss_keypoint=dict(
+ type='MultiLossFactory',
+ num_joints=17,
+ num_stages=2,
+ ae_loss_type='exp',
+ with_ae_loss=[False, False],
+ push_loss_factor=[0.001, 0.001],
+ pull_loss_factor=[0.001, 0.001],
+ with_heatmaps_loss=[True, True],
+ heatmaps_loss_factor=[1.0, 1.0]))
+ head.init_weights()
+ input_shape = (1, 512, 32, 32)
+ inputs = _demo_inputs(input_shape)
+ out = head(inputs)
+ assert out[0].shape == torch.Size([1, 17, 32, 32])
+ assert out[1].shape == torch.Size([1, 17, 64, 64])
+ head = AEHigherResolutionHead(
+ in_channels=512,
+ num_joints=17,
+ tag_per_joint=False,
+ with_ae_loss=[True, False],
+ extra={'final_conv_kernel': 3},
+ cat_output=[True],
+ loss_keypoint=dict(
+ type='MultiLossFactory',
+ num_joints=17,
+ num_stages=2,
+ ae_loss_type='exp',
+ with_ae_loss=[True, False],
+ push_loss_factor=[0.001, 0.001],
+ pull_loss_factor=[0.001, 0.001],
+ with_heatmaps_loss=[True, True],
+ heatmaps_loss_factor=[1.0, 1.0]))
+ head.init_weights()
+ input_shape = (1, 512, 32, 32)
+ inputs = _demo_inputs(input_shape)
+ out = head(inputs)
+ assert out[0].shape == torch.Size([1, 18, 32, 32])
+ assert out[1].shape == torch.Size([1, 17, 64, 64])
+ head = AEHigherResolutionHead(
+ in_channels=512,
+ num_joints=17,
+ tag_per_joint=True,
+ with_ae_loss=[True, True],
+ extra={'final_conv_kernel': 3},
+ cat_output=[True],
+ loss_keypoint=dict(
+ type='MultiLossFactory',
+ num_joints=17,
+ num_stages=2,
+ ae_loss_type='exp',
+ with_ae_loss=[True, True],
+ push_loss_factor=[0.001, 0.001],
+ pull_loss_factor=[0.001, 0.001],
+ with_heatmaps_loss=[True, True],
+ heatmaps_loss_factor=[1.0, 1.0]))
+ head.init_weights()
+ input_shape = (1, 512, 32, 32)
+ inputs = _demo_inputs(input_shape)
+ out = head(inputs)
+ assert out[0].shape == torch.Size([1, 34, 32, 32])
+ assert out[1].shape == torch.Size([1, 34, 64, 64])
+ # cat_output
+ head = AEHigherResolutionHead(
+ in_channels=512,
+ num_joints=17,
+ tag_per_joint=True,
+ with_ae_loss=[True, True],
+ extra={'final_conv_kernel': 3},
+ cat_output=[False],
+ loss_keypoint=dict(
+ type='MultiLossFactory',
+ num_joints=17,
+ num_stages=2,
+ ae_loss_type='exp',
+ with_ae_loss=[True, True],
+ push_loss_factor=[0.001, 0.001],
+ pull_loss_factor=[0.001, 0.001],
+ with_heatmaps_loss=[True, True],
+ heatmaps_loss_factor=[1.0, 1.0]))
+ head.init_weights()
+ input_shape = (1, 512, 32, 32)
+ inputs = _demo_inputs(input_shape)
+ out = head(inputs)
+ assert out[0].shape == torch.Size([1, 34, 32, 32])
+ assert out[1].shape == torch.Size([1, 34, 64, 64])
+ head = AEHigherResolutionHead(
+ in_channels=512,
+ num_joints=17,
+ tag_per_joint=True,
+ with_ae_loss=[True, True],
+ extra={'final_conv_kernel': 3},
+ cat_output=[False],
+ loss_keypoint=dict(
+ type='MultiLossFactory',
+ num_joints=17,
+ num_stages=2,
+ ae_loss_type='exp',
+ with_ae_loss=[True, True],
+ push_loss_factor=[0.001, 0.001],
+ pull_loss_factor=[0.001, 0.001],
+ with_heatmaps_loss=[True, True],
+ heatmaps_loss_factor=[1.0, 1.0]))
+ head.init_weights()
+ input_shape = (1, 512, 32, 32)
+ inputs = _demo_inputs(input_shape)
+ out = head([inputs])
+ assert out[0].shape == torch.Size([1, 34, 32, 32])
+ assert out[1].shape == torch.Size([1, 34, 64, 64])
+
+
+def _demo_inputs(input_shape=(1, 3, 64, 64)):
+ """Create a superset of inputs needed to run backbone.
+
+ Args:
+ input_shape (tuple): input batch dimensions.
+ Default: (1, 3, 64, 64).
+ Returns:
+ Random input tensor with the size of input_shape.
+ """
+ inps = np.random.random(input_shape)
+ inps = torch.FloatTensor(inps)
+ return inps
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_models/test_interhand_3d_forward.py b/engine/pose_estimation/third-party/ViTPose/tests/test_models/test_interhand_3d_forward.py
new file mode 100644
index 0000000..a2b2724
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_models/test_interhand_3d_forward.py
@@ -0,0 +1,107 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import numpy as np
+import torch
+
+from mmpose.models import build_posenet
+
+
+def test_interhand3d_forward():
+ # model settings
+ model_cfg = dict(
+ type='Interhand3D',
+ pretrained='torchvision://resnet50',
+ backbone=dict(type='ResNet', depth=50),
+ keypoint_head=dict(
+ type='Interhand3DHead',
+ keypoint_head_cfg=dict(
+ in_channels=2048,
+ out_channels=21 * 64,
+ depth_size=64,
+ num_deconv_layers=3,
+ num_deconv_filters=(256, 256, 256),
+ num_deconv_kernels=(4, 4, 4),
+ ),
+ root_head_cfg=dict(
+ in_channels=2048,
+ heatmap_size=64,
+ hidden_dims=(512, ),
+ ),
+ hand_type_head_cfg=dict(
+ in_channels=2048,
+ num_labels=2,
+ hidden_dims=(512, ),
+ ),
+ loss_keypoint=dict(type='JointsMSELoss', use_target_weight=True),
+ loss_root_depth=dict(type='L1Loss'),
+ loss_hand_type=dict(type='BCELoss', use_target_weight=True),
+ ),
+ train_cfg={},
+ test_cfg=dict(flip_test=True, shift_heatmap=True))
+
+ detector = build_posenet(model_cfg)
+ detector.init_weights()
+
+ input_shape = (2, 3, 256, 256)
+ mm_inputs = _demo_mm_inputs(input_shape)
+
+ imgs = mm_inputs.pop('imgs')
+ target = mm_inputs.pop('target')
+ target_weight = mm_inputs.pop('target_weight')
+ img_metas = mm_inputs.pop('img_metas')
+
+ # Test forward train
+ losses = detector.forward(
+ imgs, target, target_weight, img_metas, return_loss=True)
+ assert isinstance(losses, dict)
+
+ # Test forward test
+ with torch.no_grad():
+ _ = detector.forward(imgs, img_metas=img_metas, return_loss=False)
+ _ = detector.forward_dummy(imgs)
+
+
+def _demo_mm_inputs(input_shape=(1, 3, 256, 256), num_outputs=None):
+ """Create a superset of inputs needed to run test or train batches.
+
+ Args:
+ input_shape (tuple):
+ input batch dimensions
+ """
+ (N, C, H, W) = input_shape
+
+ rng = np.random.RandomState(0)
+
+ imgs = rng.rand(*input_shape)
+ imgs = torch.FloatTensor(imgs)
+
+ target = [
+ imgs.new_zeros(N, 42, 64, H // 4, W // 4),
+ imgs.new_zeros(N, 1),
+ imgs.new_zeros(N, 2),
+ ]
+ target_weight = [
+ imgs.new_ones(N, 42, 1),
+ imgs.new_ones(N, 1),
+ imgs.new_ones(N),
+ ]
+
+ img_metas = [{
+ 'img_shape': (H, W, C),
+ 'center': np.array([W / 2, H / 2]),
+ 'scale': np.array([0.5, 0.5]),
+ 'bbox_score': 1.0,
+ 'bbox_id': 0,
+ 'flip_pairs': [],
+ 'inference_channel': np.arange(42),
+ 'image_file': '.png',
+ 'heatmap3d_depth_bound': 400.0,
+ 'root_depth_bound': 400.0,
+ } for _ in range(N)]
+
+ mm_inputs = {
+ 'imgs': imgs.requires_grad_(True),
+ 'target': target,
+ 'target_weight': target_weight,
+ 'img_metas': img_metas
+ }
+ return mm_inputs
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_models/test_interhand_3d_head.py b/engine/pose_estimation/third-party/ViTPose/tests/test_models/test_interhand_3d_head.py
new file mode 100644
index 0000000..6924232
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_models/test_interhand_3d_head.py
@@ -0,0 +1,91 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import numpy as np
+import torch
+
+from mmpose.models import Interhand3DHead
+
+
+def test_interhand_3d_head():
+ """Test interhand 3d head."""
+ N = 4
+ input_shape = (N, 2048, 8, 8)
+ inputs = torch.rand(input_shape, dtype=torch.float32)
+ target = [
+ inputs.new_zeros(N, 42, 64, 64, 64),
+ inputs.new_zeros(N, 1),
+ inputs.new_zeros(N, 2),
+ ]
+ target_weight = [
+ inputs.new_ones(N, 42, 1),
+ inputs.new_ones(N, 1),
+ inputs.new_ones(N),
+ ]
+
+ img_metas = [{
+ 'img_shape': (256, 256, 3),
+ 'center': np.array([112, 112]),
+ 'scale': np.array([0.5, 0.5]),
+ 'bbox_score': 1.0,
+ 'bbox_id': 0,
+ 'flip_pairs': [],
+ 'inference_channel': np.arange(42),
+ 'image_file': '.png',
+ 'heatmap3d_depth_bound': 400.0,
+ 'root_depth_bound': 400.0,
+ } for _ in range(N)]
+
+ head = Interhand3DHead(
+ keypoint_head_cfg=dict(
+ in_channels=2048,
+ out_channels=21 * 64,
+ depth_size=64,
+ num_deconv_layers=3,
+ num_deconv_filters=(256, 256, 256),
+ num_deconv_kernels=(4, 4, 4),
+ ),
+ root_head_cfg=dict(
+ in_channels=2048,
+ heatmap_size=64,
+ hidden_dims=(512, ),
+ ),
+ hand_type_head_cfg=dict(
+ in_channels=2048,
+ num_labels=2,
+ hidden_dims=(512, ),
+ ),
+ loss_keypoint=dict(type='JointsMSELoss', use_target_weight=True),
+ loss_root_depth=dict(type='L1Loss'),
+ loss_hand_type=dict(type='BCELoss', use_target_weight=True),
+ train_cfg={},
+ test_cfg={},
+ )
+ head.init_weights()
+
+ # test forward
+ output = head(inputs)
+ assert isinstance(output, list)
+ assert len(output) == 3
+ assert output[0].shape == (N, 42, 64, 64, 64)
+ assert output[1].shape == (N, 1)
+ assert output[2].shape == (N, 2)
+
+ # test loss computation
+ losses = head.get_loss(output, target, target_weight)
+ assert 'hand_loss' in losses
+ assert 'rel_root_loss' in losses
+ assert 'hand_type_loss' in losses
+
+ # test inference model
+ flip_pairs = [[i, 21 + i] for i in range(21)]
+ output = head.inference_model(inputs, flip_pairs)
+ assert isinstance(output, list)
+ assert len(output) == 3
+ assert output[0].shape == (N, 42, 64, 64, 64)
+ assert output[1].shape == (N, 1)
+ assert output[2].shape == (N, 2)
+
+ # test decode
+ result = head.decode(img_metas, output)
+ assert 'preds' in result
+ assert 'rel_root_depth' in result
+ assert 'hand_type' in result
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_models/test_layer.py b/engine/pose_estimation/third-party/ViTPose/tests/test_models/test_layer.py
new file mode 100644
index 0000000..b88fd1b
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_models/test_layer.py
@@ -0,0 +1,68 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import numpy as np
+import torch
+import torch.nn as nn
+from mmcv.cnn import build_conv_layer, build_upsample_layer
+
+
+def test_build_upsample_layer():
+ layer1 = nn.ConvTranspose2d(
+ in_channels=3,
+ out_channels=10,
+ kernel_size=3,
+ stride=2,
+ padding=1,
+ output_padding=1,
+ bias=False)
+
+ layer2 = build_upsample_layer(
+ dict(type='deconv'),
+ in_channels=3,
+ out_channels=10,
+ kernel_size=3,
+ stride=2,
+ padding=1,
+ output_padding=1,
+ bias=False)
+ layer2.load_state_dict(layer1.state_dict())
+
+ input_shape = (1, 3, 32, 32)
+ inputs = _demo_inputs(input_shape)
+ out1 = layer1(inputs)
+ out2 = layer2(inputs)
+ assert torch.equal(out1, out2)
+
+
+def test_build_conv_layer():
+ layer1 = nn.Conv2d(
+ in_channels=3, out_channels=10, kernel_size=3, stride=1, padding=1)
+
+ layer2 = build_conv_layer(
+ cfg=dict(type='Conv2d'),
+ in_channels=3,
+ out_channels=10,
+ kernel_size=3,
+ stride=1,
+ padding=1)
+
+ layer2.load_state_dict(layer1.state_dict())
+
+ input_shape = (1, 3, 32, 32)
+ inputs = _demo_inputs(input_shape)
+ out1 = layer1(inputs)
+ out2 = layer2(inputs)
+ assert torch.equal(out1, out2)
+
+
+def _demo_inputs(input_shape=(1, 3, 64, 64)):
+ """Create a superset of inputs needed to run backbone.
+
+ Args:
+ input_shape (tuple): input batch dimensions.
+ Default: (1, 3, 64, 64).
+ Returns:
+ Random input tensor with the size of input_shape.
+ """
+ inps = np.random.random(input_shape)
+ inps = torch.FloatTensor(inps)
+ return inps
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_models/test_mesh_forward.py b/engine/pose_estimation/third-party/ViTPose/tests/test_models/test_mesh_forward.py
new file mode 100644
index 0000000..f08f769
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_models/test_mesh_forward.py
@@ -0,0 +1,153 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os.path as osp
+import tempfile
+
+import numpy as np
+import torch
+
+from mmpose.core.optimizer import build_optimizers
+from mmpose.models.detectors.mesh import ParametricMesh
+from tests.utils.mesh_utils import generate_smpl_weight_file
+
+
+def test_parametric_mesh_forward():
+ """Test parametric mesh forward."""
+
+ tmpdir = tempfile.TemporaryDirectory()
+ # generate weight file for SMPL model.
+ generate_smpl_weight_file(tmpdir.name)
+
+ # Test ParametricMesh without discriminator
+ model_cfg = dict(
+ pretrained=None,
+ backbone=dict(type='ResNet', depth=50),
+ mesh_head=dict(
+ type='HMRMeshHead',
+ in_channels=2048,
+ smpl_mean_params='tests/data/smpl/smpl_mean_params.npz'),
+ disc=None,
+ smpl=dict(
+ type='SMPL',
+ smpl_path=tmpdir.name,
+ joints_regressor=osp.join(tmpdir.name,
+ 'test_joint_regressor.npy')),
+ train_cfg=dict(disc_step=1),
+ test_cfg=dict(
+ flip_test=False,
+ post_process='default',
+ shift_heatmap=True,
+ modulate_kernel=11),
+ loss_mesh=dict(
+ type='MeshLoss',
+ joints_2d_loss_weight=1,
+ joints_3d_loss_weight=1,
+ vertex_loss_weight=1,
+ smpl_pose_loss_weight=1,
+ smpl_beta_loss_weight=1,
+ focal_length=5000,
+ img_res=256),
+ loss_gan=None)
+
+ detector = ParametricMesh(**model_cfg)
+ detector.init_weights()
+
+ optimizers_config = dict(generator=dict(type='Adam', lr=0.0001))
+ optims = build_optimizers(detector, optimizers_config)
+
+ input_shape = (1, 3, 256, 256)
+ mm_inputs = _demo_mm_inputs(input_shape)
+ # Test forward train
+ output = detector.train_step(mm_inputs, optims)
+ assert isinstance(output, dict)
+
+ # Test forward test
+ with torch.no_grad():
+ output = detector.val_step(data_batch=mm_inputs)
+ assert isinstance(output, dict)
+
+ imgs = mm_inputs.pop('img')
+ img_metas = mm_inputs.pop('img_metas')
+ output = detector.forward(imgs, img_metas=img_metas, return_loss=False)
+ assert isinstance(output, dict)
+
+ # Test ParametricMesh with discriminator
+ model_cfg['disc'] = dict()
+ model_cfg['loss_gan'] = dict(
+ type='GANLoss',
+ gan_type='lsgan',
+ real_label_val=1.0,
+ fake_label_val=0.0,
+ loss_weight=1)
+
+ optimizers_config['discriminator'] = dict(type='Adam', lr=0.0001)
+
+ detector = ParametricMesh(**model_cfg)
+ detector.init_weights()
+ optims = build_optimizers(detector, optimizers_config)
+
+ input_shape = (1, 3, 256, 256)
+ mm_inputs = _demo_mm_inputs(input_shape)
+ # Test forward train
+ output = detector.train_step(mm_inputs, optims)
+ assert isinstance(output, dict)
+
+ # Test forward test
+ with torch.no_grad():
+ output = detector.val_step(data_batch=mm_inputs)
+ assert isinstance(output, dict)
+
+ imgs = mm_inputs.pop('img')
+ img_metas = mm_inputs.pop('img_metas')
+ output = detector.forward(imgs, img_metas=img_metas, return_loss=False)
+ assert isinstance(output, dict)
+
+ _ = detector.forward_dummy(imgs)
+
+ tmpdir.cleanup()
+
+
+def _demo_mm_inputs(input_shape=(1, 3, 256, 256)):
+ """Create a superset of inputs needed to run test or train batches.
+
+ Args:
+ input_shape (tuple):
+ input batch dimensions
+ """
+ (N, C, H, W) = input_shape
+
+ rng = np.random.RandomState(0)
+
+ imgs = rng.rand(*input_shape)
+ joints_2d = np.zeros([N, 24, 2])
+ joints_2d_visible = np.ones([N, 24, 1])
+ joints_3d = np.zeros([N, 24, 3])
+ joints_3d_visible = np.ones([N, 24, 1])
+ pose = np.zeros([N, 72])
+ beta = np.zeros([N, 10])
+ has_smpl = np.ones([N])
+ mosh_theta = np.zeros([N, 3 + 72 + 10])
+
+ img_metas = [{
+ 'img_shape': (H, W, C),
+ 'center': np.array([W / 2, H / 2]),
+ 'scale': np.array([0.5, 0.5]),
+ 'bbox_score': 1.0,
+ 'flip_pairs': [],
+ 'inference_channel': np.arange(17),
+ 'image_file': '.png',
+ } for _ in range(N)]
+
+ mm_inputs = {
+ 'img': torch.FloatTensor(imgs).requires_grad_(True),
+ 'joints_2d': torch.FloatTensor(joints_2d),
+ 'joints_2d_visible': torch.FloatTensor(joints_2d_visible),
+ 'joints_3d': torch.FloatTensor(joints_3d),
+ 'joints_3d_visible': torch.FloatTensor(joints_3d_visible),
+ 'pose': torch.FloatTensor(pose),
+ 'beta': torch.FloatTensor(beta),
+ 'has_smpl': torch.FloatTensor(has_smpl),
+ 'img_metas': img_metas,
+ 'mosh_theta': torch.FloatTensor(mosh_theta)
+ }
+
+ return mm_inputs
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_models/test_mesh_head.py b/engine/pose_estimation/third-party/ViTPose/tests/test_models/test_mesh_head.py
new file mode 100644
index 0000000..4d1fc0e
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_models/test_mesh_head.py
@@ -0,0 +1,76 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import numpy as np
+import pytest
+import torch
+
+from mmpose.models import HMRMeshHead
+from mmpose.models.misc.discriminator import SMPLDiscriminator
+
+
+def test_mesh_hmr_head():
+ """Test hmr mesh head."""
+ head = HMRMeshHead(in_channels=512)
+ head.init_weights()
+
+ input_shape = (1, 512, 8, 8)
+ inputs = _demo_inputs(input_shape)
+ out = head(inputs)
+ smpl_rotmat, smpl_shape, camera = out
+ assert smpl_rotmat.shape == torch.Size([1, 24, 3, 3])
+ assert smpl_shape.shape == torch.Size([1, 10])
+ assert camera.shape == torch.Size([1, 3])
+ """Test hmr mesh head with assigned mean parameters and n_iter """
+ head = HMRMeshHead(
+ in_channels=512,
+ smpl_mean_params='tests/data/smpl/smpl_mean_params.npz',
+ n_iter=3)
+ head.init_weights()
+ input_shape = (1, 512, 8, 8)
+ inputs = _demo_inputs(input_shape)
+ out = head(inputs)
+ smpl_rotmat, smpl_shape, camera = out
+ assert smpl_rotmat.shape == torch.Size([1, 24, 3, 3])
+ assert smpl_shape.shape == torch.Size([1, 10])
+ assert camera.shape == torch.Size([1, 3])
+
+ # test discriminator with SMPL pose parameters
+ # in rotation matrix representation
+ disc = SMPLDiscriminator(
+ beta_channel=(10, 10, 5, 1),
+ per_joint_channel=(9, 32, 32, 16, 1),
+ full_pose_channel=(23 * 16, 256, 1))
+ pred_theta = (camera, smpl_rotmat, smpl_shape)
+ pred_score = disc(pred_theta)
+ assert pred_score.shape[1] == 25
+
+ # test discriminator with SMPL pose parameters
+ # in axis-angle representation
+ pred_theta = (camera, camera.new_zeros([1, 72]), smpl_shape)
+ pred_score = disc(pred_theta)
+ assert pred_score.shape[1] == 25
+
+ with pytest.raises(TypeError):
+ _ = SMPLDiscriminator(
+ beta_channel=[10, 10, 5, 1],
+ per_joint_channel=(9, 32, 32, 16, 1),
+ full_pose_channel=(23 * 16, 256, 1))
+
+ with pytest.raises(ValueError):
+ _ = SMPLDiscriminator(
+ beta_channel=(10, ),
+ per_joint_channel=(9, 32, 32, 16, 1),
+ full_pose_channel=(23 * 16, 256, 1))
+
+
+def _demo_inputs(input_shape=(1, 3, 64, 64)):
+ """Create a superset of inputs needed to run mesh head.
+
+ Args:
+ input_shape (tuple): input batch dimensions.
+ Default: (1, 3, 64, 64).
+ Returns:
+ Random input tensor with the size of input_shape.
+ """
+ inps = np.random.random(input_shape)
+ inps = torch.FloatTensor(inps)
+ return inps
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_models/test_multitask_forward.py b/engine/pose_estimation/third-party/ViTPose/tests/test_models/test_multitask_forward.py
new file mode 100644
index 0000000..97cfd7d
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_models/test_multitask_forward.py
@@ -0,0 +1,116 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import numpy as np
+import torch
+
+from mmpose.models.detectors import MultiTask
+
+
+def test_multitask_forward():
+ """Test multitask forward."""
+
+ # build MultiTask detector
+ model_cfg = dict(
+ backbone=dict(type='ResNet', depth=50),
+ heads=[
+ dict(
+ type='DeepposeRegressionHead',
+ in_channels=2048,
+ num_joints=17,
+ loss_keypoint=dict(
+ type='SmoothL1Loss', use_target_weight=False)),
+ ],
+ necks=[dict(type='GlobalAveragePooling')],
+ head2neck={0: 0},
+ pretrained=None,
+ )
+ model = MultiTask(**model_cfg)
+
+ # build inputs and target
+ mm_inputs = _demo_mm_inputs()
+ inputs = mm_inputs['img']
+ target = [mm_inputs['target_keypoints']]
+ target_weight = [mm_inputs['target_weight']]
+ img_metas = mm_inputs['img_metas']
+
+ # Test forward train
+ losses = model(inputs, target, target_weight, return_loss=True)
+ assert 'reg_loss' in losses and 'acc_pose' in losses
+
+ # Test forward test
+ outputs = model(inputs, img_metas=img_metas, return_loss=False)
+ assert 'preds' in outputs
+
+ # Test dummy forward
+ outputs = model.forward_dummy(inputs)
+ assert outputs[0].shape == torch.Size([1, 17, 2])
+
+ # Build multitask detector with no neck
+ model_cfg = dict(
+ backbone=dict(type='ResNet', depth=50),
+ heads=[
+ dict(
+ type='TopdownHeatmapSimpleHead',
+ in_channels=2048,
+ out_channels=17,
+ num_deconv_layers=3,
+ num_deconv_filters=(256, 256, 256),
+ num_deconv_kernels=(4, 4, 4),
+ loss_keypoint=dict(
+ type='JointsMSELoss', use_target_weight=True))
+ ],
+ pretrained=None,
+ )
+ model = MultiTask(**model_cfg)
+
+ # build inputs and target
+ target = [mm_inputs['target_heatmap']]
+
+ # Test forward train
+ losses = model(inputs, target, target_weight, return_loss=True)
+ assert 'heatmap_loss' in losses and 'acc_pose' in losses
+
+ # Test forward test
+ outputs = model(inputs, img_metas=img_metas, return_loss=False)
+ assert 'preds' in outputs
+
+ # Test dummy forward
+ outputs = model.forward_dummy(inputs)
+ assert outputs[0].shape == torch.Size([1, 17, 64, 64])
+
+
+def _demo_mm_inputs(input_shape=(1, 3, 256, 256)):
+ """Create a superset of inputs needed to run test or train.
+
+ Args:
+ input_shape (tuple):
+ input batch dimensions
+ """
+ (N, C, H, W) = input_shape
+
+ rng = np.random.RandomState(0)
+
+ imgs = rng.rand(*input_shape)
+
+ target_keypoints = np.zeros([N, 17, 2])
+ target_heatmap = np.zeros([N, 17, H // 4, W // 4])
+ target_weight = np.ones([N, 17, 1])
+
+ img_metas = [{
+ 'img_shape': (H, W, C),
+ 'center': np.array([W / 2, H / 2]),
+ 'scale': np.array([0.5, 0.5]),
+ 'bbox_score': 1.0,
+ 'bbox_id': 0,
+ 'flip_pairs': [],
+ 'inference_channel': np.arange(17),
+ 'image_file': '.png',
+ } for _ in range(N)]
+
+ mm_inputs = {
+ 'img': torch.FloatTensor(imgs).requires_grad_(True),
+ 'target_keypoints': torch.FloatTensor(target_keypoints),
+ 'target_heatmap': torch.FloatTensor(target_heatmap),
+ 'target_weight': torch.FloatTensor(target_weight),
+ 'img_metas': img_metas,
+ }
+ return mm_inputs
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_models/test_multiview_pose.py b/engine/pose_estimation/third-party/ViTPose/tests/test_models/test_multiview_pose.py
new file mode 100644
index 0000000..ad89777
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_models/test_multiview_pose.py
@@ -0,0 +1,129 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import tempfile
+
+from mmcv import Config
+
+from mmpose.datasets import DATASETS, build_dataloader
+from mmpose.models import builder
+
+
+def test_voxelpose_forward():
+ dataset = 'Body3DMviewDirectPanopticDataset'
+ dataset_class = DATASETS.get(dataset)
+ dataset_info = Config.fromfile(
+ 'configs/_base_/datasets/panoptic_body3d.py').dataset_info
+ space_size = [8000, 8000, 2000]
+ space_center = [0, -500, 800]
+ cube_size = [20, 20, 8]
+ data_cfg = dict(
+ image_size=[960, 512],
+ heatmap_size=[[240, 128]],
+ space_size=space_size,
+ space_center=space_center,
+ cube_size=cube_size,
+ num_joints=15,
+ seq_list=['160906_band1'],
+ cam_list=[(0, 12), (0, 6)],
+ num_cameras=2,
+ seq_frame_interval=1,
+ subset='train',
+ need_2d_label=True,
+ need_camera_param=True,
+ root_id=2)
+
+ pipeline = [
+ dict(
+ type='MultiItemProcess',
+ pipeline=[
+ dict(
+ type='BottomUpGenerateTarget', sigma=3, max_num_people=20)
+ ]),
+ dict(
+ type='DiscardDuplicatedItems',
+ keys_list=[
+ 'joints_3d', 'joints_3d_visible', 'ann_info', 'roots_3d',
+ 'num_persons', 'sample_id'
+ ]),
+ dict(
+ type='GenerateVoxel3DHeatmapTarget',
+ sigma=200.0,
+ joint_indices=[2]),
+ dict(type='RenameKeys', key_pairs=[('targets', 'input_heatmaps')]),
+ dict(
+ type='Collect',
+ keys=['targets_3d', 'input_heatmaps'],
+ meta_keys=[
+ 'camera', 'center', 'scale', 'joints_3d', 'num_persons',
+ 'joints_3d_visible', 'roots_3d', 'sample_id'
+ ]),
+ ]
+
+ model_cfg = dict(
+ type='DetectAndRegress',
+ backbone=None,
+ human_detector=dict(
+ type='VoxelCenterDetector',
+ image_size=[960, 512],
+ heatmap_size=[240, 128],
+ space_size=space_size,
+ cube_size=cube_size,
+ space_center=space_center,
+ center_net=dict(
+ type='V2VNet', input_channels=15, output_channels=1),
+ center_head=dict(
+ type='CuboidCenterHead',
+ space_size=space_size,
+ space_center=space_center,
+ cube_size=cube_size,
+ max_num=3,
+ max_pool_kernel=3),
+ train_cfg=dict(dist_threshold=500000000.0),
+ test_cfg=dict(center_threshold=0.0),
+ ),
+ pose_regressor=dict(
+ type='VoxelSinglePose',
+ image_size=[960, 512],
+ heatmap_size=[240, 128],
+ sub_space_size=[2000, 2000, 2000],
+ sub_cube_size=[20, 20, 8],
+ num_joints=15,
+ pose_net=dict(
+ type='V2VNet', input_channels=15, output_channels=15),
+ pose_head=dict(type='CuboidPoseHead', beta=100.0),
+ train_cfg=None,
+ test_cfg=None))
+
+ model = builder.build_posenet(model_cfg)
+ with tempfile.TemporaryDirectory() as tmpdir:
+ dataset = dataset_class(
+ ann_file=tmpdir + '/tmp_train.pkl',
+ img_prefix='tests/data/panoptic_body3d/',
+ data_cfg=data_cfg,
+ pipeline=pipeline,
+ dataset_info=dataset_info,
+ test_mode=False)
+
+ data_loader = build_dataloader(
+ dataset,
+ seed=None,
+ dist=False,
+ shuffle=False,
+ drop_last=False,
+ workers_per_gpu=1,
+ samples_per_gpu=1)
+
+ for data in data_loader:
+ # test forward_train
+ _ = model(
+ img=None,
+ img_metas=data['img_metas'].data[0],
+ return_loss=True,
+ targets_3d=data['targets_3d'],
+ input_heatmaps=data['input_heatmaps'])
+
+ # test forward_test
+ _ = model(
+ img=None,
+ img_metas=data['img_metas'].data[0],
+ return_loss=False,
+ input_heatmaps=data['input_heatmaps'])
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_models/test_pose_lifter_forward.py b/engine/pose_estimation/third-party/ViTPose/tests/test_models/test_pose_lifter_forward.py
new file mode 100644
index 0000000..04ebc65
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_models/test_pose_lifter_forward.py
@@ -0,0 +1,197 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import mmcv
+import numpy as np
+import torch
+
+from mmpose.models import build_posenet
+
+
+def _create_inputs(joint_num_in,
+ joint_channel_in,
+ joint_num_out,
+ joint_channel_out,
+ seq_len,
+ batch_size,
+ semi=False):
+ rng = np.random.RandomState(0)
+ pose_in = rng.rand(batch_size, joint_num_in * joint_channel_in, seq_len)
+ target = np.zeros((batch_size, joint_num_out, joint_channel_out),
+ dtype=np.float32)
+ target_weight = np.ones((batch_size, joint_num_out, joint_channel_out),
+ dtype=np.float32)
+
+ meta_info = {
+ 'root_position': np.zeros((1, joint_channel_out), np.float32),
+ 'root_position_index': 0,
+ 'target_mean': np.zeros((joint_num_out, joint_channel_out),
+ np.float32),
+ 'target_std': np.ones((joint_num_out, joint_channel_out), np.float32)
+ }
+ metas = [meta_info.copy() for _ in range(batch_size)]
+ inputs = {
+ 'input': torch.FloatTensor(pose_in).requires_grad_(True),
+ 'target': torch.FloatTensor(target),
+ 'target_weight': torch.FloatTensor(target_weight),
+ 'metas': metas,
+ }
+
+ if semi:
+ traj_target = np.zeros((batch_size, 1, joint_channel_out), np.float32)
+ unlabeled_pose_in = rng.rand(batch_size,
+ joint_num_in * joint_channel_in, seq_len)
+ unlabeled_target_2d = np.zeros(
+ (batch_size, joint_num_out, joint_channel_in), dtype=np.float32)
+ intrinsics = np.ones((batch_size, 4))
+
+ inputs['traj_target'] = torch.FloatTensor(traj_target)
+ inputs['unlabeled_input'] = torch.FloatTensor(
+ unlabeled_pose_in).requires_grad_(True)
+ inputs['unlabeled_target_2d'] = torch.FloatTensor(unlabeled_target_2d)
+ inputs['intrinsics'] = torch.FloatTensor(intrinsics)
+
+ return inputs
+
+
+def test_pose_lifter_forward():
+ # Test forward train for supervised learning with pose model only
+ model_cfg = dict(
+ type='PoseLifter',
+ pretrained=None,
+ backbone=dict(type='TCN', in_channels=2 * 17),
+ keypoint_head=dict(
+ type='TemporalRegressionHead',
+ in_channels=1024,
+ num_joints=16,
+ max_norm=1.0,
+ loss_keypoint=dict(type='MPJPELoss'),
+ test_cfg=dict(restore_global_position=True)),
+ train_cfg=dict(),
+ test_cfg=dict())
+
+ cfg = mmcv.Config({'model': model_cfg})
+ detector = build_posenet(cfg.model)
+
+ detector.init_weights()
+
+ inputs = _create_inputs(
+ joint_num_in=17,
+ joint_channel_in=2,
+ joint_num_out=16,
+ joint_channel_out=3,
+ seq_len=27,
+ batch_size=8)
+
+ losses = detector.forward(
+ inputs['input'],
+ inputs['target'],
+ inputs['target_weight'],
+ inputs['metas'],
+ return_loss=True)
+
+ assert isinstance(losses, dict)
+
+ # Test forward test for supervised learning with pose model only
+ with torch.no_grad():
+ _ = detector.forward(
+ inputs['input'],
+ inputs['target'],
+ inputs['target_weight'],
+ inputs['metas'],
+ return_loss=False)
+ _ = detector.forward_dummy(inputs['input'])
+
+ # Test forward train for semi-supervised learning
+ model_cfg = dict(
+ type='PoseLifter',
+ pretrained=None,
+ backbone=dict(type='TCN', in_channels=2 * 17),
+ keypoint_head=dict(
+ type='TemporalRegressionHead',
+ in_channels=1024,
+ num_joints=17,
+ loss_keypoint=dict(type='MPJPELoss'),
+ test_cfg=dict(restore_global_position=True)),
+ traj_backbone=dict(type='TCN', in_channels=2 * 17),
+ traj_head=dict(
+ type='TemporalRegressionHead',
+ in_channels=1024,
+ num_joints=1,
+ loss_keypoint=dict(type='MPJPELoss'),
+ is_trajectory=True),
+ loss_semi=dict(
+ type='SemiSupervisionLoss',
+ joint_parents=[
+ 0, 0, 1, 2, 0, 4, 5, 0, 7, 8, 9, 8, 11, 12, 8, 14, 15
+ ]),
+ train_cfg=dict(),
+ test_cfg=dict())
+
+ cfg = mmcv.Config({'model': model_cfg})
+ detector = build_posenet(cfg.model)
+
+ detector.init_weights()
+
+ inputs = _create_inputs(
+ joint_num_in=17,
+ joint_channel_in=2,
+ joint_num_out=17,
+ joint_channel_out=3,
+ seq_len=27,
+ batch_size=8,
+ semi=True)
+
+ losses = detector.forward(**inputs, return_loss=True)
+
+ assert isinstance(losses, dict)
+ assert 'proj_loss' in losses
+
+ # Test forward test for semi-supervised learning
+ with torch.no_grad():
+ _ = detector.forward(**inputs, return_loss=False)
+ _ = detector.forward_dummy(inputs['input'])
+
+ # Test forward train for supervised learning with pose model and trajectory
+ # model sharing one backbone
+ model_cfg = dict(
+ type='PoseLifter',
+ pretrained=None,
+ backbone=dict(type='TCN', in_channels=2 * 17),
+ keypoint_head=dict(
+ type='TemporalRegressionHead',
+ in_channels=1024,
+ num_joints=17,
+ loss_keypoint=dict(type='MPJPELoss'),
+ test_cfg=dict(restore_global_position=True)),
+ traj_head=dict(
+ type='TemporalRegressionHead',
+ in_channels=1024,
+ num_joints=1,
+ loss_keypoint=dict(type='MPJPELoss'),
+ is_trajectory=True),
+ train_cfg=dict(),
+ test_cfg=dict())
+
+ cfg = mmcv.Config({'model': model_cfg})
+ detector = build_posenet(cfg.model)
+
+ detector.init_weights()
+
+ inputs = _create_inputs(
+ joint_num_in=17,
+ joint_channel_in=2,
+ joint_num_out=17,
+ joint_channel_out=3,
+ seq_len=27,
+ batch_size=8,
+ semi=True)
+
+ losses = detector.forward(**inputs, return_loss=True)
+
+ assert isinstance(losses, dict)
+ assert 'traj_loss' in losses
+
+ # Test forward test for semi-supervised learning with pose model and
+ # trajectory model sharing one backbone
+ with torch.no_grad():
+ _ = detector.forward(**inputs, return_loss=False)
+ _ = detector.forward_dummy(inputs['input'])
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_models/test_temporal_regression_head.py b/engine/pose_estimation/third-party/ViTPose/tests/test_models/test_temporal_regression_head.py
new file mode 100644
index 0000000..65f7d78
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_models/test_temporal_regression_head.py
@@ -0,0 +1,104 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import numpy as np
+import pytest
+import torch
+
+from mmpose.models import TemporalRegressionHead
+
+
+def test_temporal_regression_head():
+ """Test temporal head."""
+
+ # w/o global position restoration
+ head = TemporalRegressionHead(
+ in_channels=1024,
+ num_joints=17,
+ loss_keypoint=dict(type='MPJPELoss', use_target_weight=True),
+ test_cfg=dict(restore_global_position=False))
+
+ head.init_weights()
+
+ with pytest.raises(AssertionError):
+ # ndim of the input tensor should be 3
+ input_shape = (1, 1024, 1, 1)
+ inputs = _demo_inputs(input_shape)
+ _ = head(inputs)
+
+ with pytest.raises(AssertionError):
+ # size of the last dim should be 1
+ input_shape = (1, 1024, 3)
+ inputs = _demo_inputs(input_shape)
+ _ = head(inputs)
+
+ input_shape = (1, 1024, 1)
+ inputs = _demo_inputs(input_shape)
+ out = head(inputs)
+ assert out.shape == torch.Size([1, 17, 3])
+
+ loss = head.get_loss(out, out, None)
+ assert torch.allclose(loss['reg_loss'], torch.tensor(0.))
+
+ _ = head.inference_model(inputs)
+ _ = head.inference_model(inputs, [(0, 1), (2, 3)])
+ metas = [{}]
+
+ acc = head.get_accuracy(out, out, None, metas=metas)
+ assert acc['mpjpe'] == 0.
+ np.testing.assert_almost_equal(acc['p_mpjpe'], 0., decimal=6)
+
+ # w/ global position restoration
+ head = TemporalRegressionHead(
+ in_channels=1024,
+ num_joints=16,
+ loss_keypoint=dict(type='MPJPELoss', use_target_weight=True),
+ test_cfg=dict(restore_global_position=True))
+ head.init_weights()
+
+ input_shape = (1, 1024, 1)
+ inputs = _demo_inputs(input_shape)
+ metas = [{
+ 'root_position': np.zeros((1, 3)),
+ 'root_position_index': 0,
+ 'root_weight': 1.
+ }]
+ out = head(inputs)
+ assert out.shape == torch.Size([1, 16, 3])
+
+ inference_out = head.inference_model(inputs)
+ acc = head.get_accuracy(out, out, torch.ones_like(out), metas)
+ assert acc['mpjpe'] == 0.
+ np.testing.assert_almost_equal(acc['p_mpjpe'], 0.)
+
+ _ = head.decode(metas, inference_out)
+
+ # trajectory model (only predict root position)
+ head = TemporalRegressionHead(
+ in_channels=1024,
+ num_joints=1,
+ loss_keypoint=dict(type='MPJPELoss', use_target_weight=True),
+ is_trajectory=True,
+ test_cfg=dict(restore_global_position=False))
+
+ head.init_weights()
+
+ input_shape = (1, 1024, 1)
+ inputs = _demo_inputs(input_shape)
+ out = head(inputs)
+ assert out.shape == torch.Size([1, 1, 3])
+
+ loss = head.get_loss(out, out.squeeze(1), torch.ones_like(out))
+ assert torch.allclose(loss['traj_loss'], torch.tensor(0.))
+
+
+def _demo_inputs(input_shape=(1, 1024, 1)):
+ """Create a superset of inputs needed to run head.
+
+ Args:
+ input_shape (tuple): input batch dimensions.
+ Default: (1, 1024, 1).
+ Returns:
+ Random input tensor with the size of input_shape.
+ """
+ inps = np.random.random(input_shape)
+ inps = torch.FloatTensor(inps)
+ return inps
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_models/test_top_down_forward.py b/engine/pose_estimation/third-party/ViTPose/tests/test_models/test_top_down_forward.py
new file mode 100644
index 0000000..eda2b8f
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_models/test_top_down_forward.py
@@ -0,0 +1,517 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import copy
+
+import numpy as np
+import torch
+
+from mmpose.models.detectors import PoseWarper, TopDown
+
+
+def test_vipnas_forward():
+ # model settings
+
+ channel_cfg = dict(
+ num_output_channels=17,
+ dataset_joints=17,
+ dataset_channel=[
+ [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16],
+ ],
+ inference_channel=[
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16
+ ])
+
+ model_cfg = dict(
+ type='TopDown',
+ pretrained=None,
+ backbone=dict(type='ViPNAS_ResNet', depth=50),
+ keypoint_head=dict(
+ type='ViPNASHeatmapSimpleHead',
+ in_channels=608,
+ out_channels=channel_cfg['num_output_channels'],
+ loss_keypoint=dict(type='JointsMSELoss', use_target_weight=True)),
+ train_cfg=dict(),
+ test_cfg=dict(
+ flip_test=True,
+ post_process='default',
+ shift_heatmap=True,
+ modulate_kernel=11))
+
+ detector = TopDown(model_cfg['backbone'], None, model_cfg['keypoint_head'],
+ model_cfg['train_cfg'], model_cfg['test_cfg'],
+ model_cfg['pretrained'])
+
+ input_shape = (1, 3, 256, 256)
+ mm_inputs = _demo_mm_inputs(input_shape)
+
+ imgs = mm_inputs.pop('imgs')
+ target = mm_inputs.pop('target')
+ target_weight = mm_inputs.pop('target_weight')
+ img_metas = mm_inputs.pop('img_metas')
+
+ # Test forward train
+ losses = detector.forward(
+ imgs, target, target_weight, img_metas, return_loss=True)
+ assert isinstance(losses, dict)
+
+ # Test forward test
+ with torch.no_grad():
+ _ = detector.forward(imgs, img_metas=img_metas, return_loss=False)
+
+
+def test_topdown_forward():
+ model_cfg = dict(
+ type='TopDown',
+ pretrained=None,
+ backbone=dict(type='ResNet', depth=18),
+ keypoint_head=dict(
+ type='TopdownHeatmapSimpleHead',
+ in_channels=512,
+ out_channels=17,
+ loss_keypoint=dict(type='JointsMSELoss', use_target_weight=True)),
+ train_cfg=dict(),
+ test_cfg=dict(
+ flip_test=True,
+ post_process='default',
+ shift_heatmap=True,
+ modulate_kernel=11))
+
+ detector = TopDown(model_cfg['backbone'], None, model_cfg['keypoint_head'],
+ model_cfg['train_cfg'], model_cfg['test_cfg'],
+ model_cfg['pretrained'])
+
+ detector.init_weights()
+
+ input_shape = (1, 3, 256, 256)
+ mm_inputs = _demo_mm_inputs(input_shape)
+
+ imgs = mm_inputs.pop('imgs')
+ target = mm_inputs.pop('target')
+ target_weight = mm_inputs.pop('target_weight')
+ img_metas = mm_inputs.pop('img_metas')
+
+ # Test forward train
+ losses = detector.forward(
+ imgs, target, target_weight, img_metas, return_loss=True)
+ assert isinstance(losses, dict)
+
+ # Test forward test
+ with torch.no_grad():
+ _ = detector.forward(imgs, img_metas=img_metas, return_loss=False)
+
+ # flip test
+ model_cfg = dict(
+ type='TopDown',
+ pretrained=None,
+ backbone=dict(
+ type='HourglassNet',
+ num_stacks=1,
+ ),
+ keypoint_head=dict(
+ type='TopdownHeatmapMultiStageHead',
+ in_channels=256,
+ out_channels=17,
+ num_stages=1,
+ num_deconv_layers=0,
+ extra=dict(final_conv_kernel=1, ),
+ loss_keypoint=dict(type='JointsMSELoss', use_target_weight=False)),
+ train_cfg=dict(),
+ test_cfg=dict(
+ flip_test=True,
+ post_process='default',
+ shift_heatmap=True,
+ modulate_kernel=11))
+
+ detector = TopDown(model_cfg['backbone'], None, model_cfg['keypoint_head'],
+ model_cfg['train_cfg'], model_cfg['test_cfg'],
+ model_cfg['pretrained'])
+
+ # Test forward train
+ losses = detector.forward(
+ imgs, target, target_weight, img_metas, return_loss=True)
+ assert isinstance(losses, dict)
+
+ # Test forward test
+ with torch.no_grad():
+ _ = detector.forward(imgs, img_metas=img_metas, return_loss=False)
+
+ model_cfg = dict(
+ type='TopDown',
+ pretrained=None,
+ backbone=dict(
+ type='HourglassNet',
+ num_stacks=1,
+ ),
+ keypoint_head=dict(
+ type='TopdownHeatmapMultiStageHead',
+ in_channels=256,
+ out_channels=17,
+ num_stages=1,
+ num_deconv_layers=0,
+ extra=dict(final_conv_kernel=1, ),
+ loss_keypoint=[
+ dict(
+ type='JointsMSELoss',
+ use_target_weight=True,
+ loss_weight=1.)
+ ]),
+ train_cfg=dict(),
+ test_cfg=dict(
+ flip_test=True,
+ post_process='default',
+ shift_heatmap=True,
+ modulate_kernel=11))
+
+ detector = TopDown(model_cfg['backbone'], None, model_cfg['keypoint_head'],
+ model_cfg['train_cfg'], model_cfg['test_cfg'],
+ model_cfg['pretrained'])
+
+ detector.init_weights()
+
+ input_shape = (1, 3, 256, 256)
+ mm_inputs = _demo_mm_inputs(input_shape, num_outputs=None)
+
+ imgs = mm_inputs.pop('imgs')
+ target = mm_inputs.pop('target')
+ target_weight = mm_inputs.pop('target_weight')
+ img_metas = mm_inputs.pop('img_metas')
+
+ # Test forward train
+ losses = detector.forward(
+ imgs, target, target_weight, img_metas, return_loss=True)
+ assert isinstance(losses, dict)
+ # Test forward test
+ with torch.no_grad():
+ _ = detector.forward(imgs, img_metas=img_metas, return_loss=False)
+
+ model_cfg = dict(
+ type='TopDown',
+ pretrained=None,
+ backbone=dict(
+ type='RSN',
+ unit_channels=256,
+ num_stages=1,
+ num_units=4,
+ num_blocks=[2, 2, 2, 2],
+ num_steps=4,
+ norm_cfg=dict(type='BN')),
+ keypoint_head=dict(
+ type='TopdownHeatmapMSMUHead',
+ out_shape=(64, 48),
+ unit_channels=256,
+ out_channels=17,
+ num_stages=1,
+ num_units=4,
+ use_prm=False,
+ norm_cfg=dict(type='BN'),
+ loss_keypoint=[dict(type='JointsMSELoss', use_target_weight=True)]
+ * 3 + [dict(type='JointsOHKMMSELoss', use_target_weight=True)]),
+ train_cfg=dict(num_units=4),
+ test_cfg=dict(
+ flip_test=True,
+ post_process='default',
+ shift_heatmap=False,
+ unbiased_decoding=False,
+ modulate_kernel=5))
+
+ detector = TopDown(model_cfg['backbone'], None, model_cfg['keypoint_head'],
+ model_cfg['train_cfg'], model_cfg['test_cfg'],
+ model_cfg['pretrained'])
+
+ detector.init_weights()
+
+ input_shape = (1, 3, 256, 192)
+ mm_inputs = _demo_mm_inputs(input_shape, num_outputs=4)
+
+ imgs = mm_inputs.pop('imgs')
+ target = mm_inputs.pop('target')
+ target_weight = mm_inputs.pop('target_weight')
+ img_metas = mm_inputs.pop('img_metas')
+
+ # Test forward train
+ losses = detector.forward(
+ imgs, target, target_weight, img_metas, return_loss=True)
+ assert isinstance(losses, dict)
+ # Test forward test
+ with torch.no_grad():
+ _ = detector.forward(imgs, img_metas=img_metas, return_loss=False)
+ _ = detector.forward_dummy(imgs)
+
+
+def test_posewarper_forward():
+ # test PoseWarper
+ model_cfg = dict(
+ type='PoseWarper',
+ pretrained=None,
+ backbone=dict(
+ type='HRNet',
+ in_channels=3,
+ extra=dict(
+ stage1=dict(
+ num_modules=1,
+ num_branches=1,
+ block='BOTTLENECK',
+ num_blocks=(4, ),
+ num_channels=(64, )),
+ stage2=dict(
+ num_modules=1,
+ num_branches=2,
+ block='BASIC',
+ num_blocks=(4, 4),
+ num_channels=(48, 96)),
+ stage3=dict(
+ num_modules=4,
+ num_branches=3,
+ block='BASIC',
+ num_blocks=(4, 4, 4),
+ num_channels=(48, 96, 192)),
+ stage4=dict(
+ num_modules=3,
+ num_branches=4,
+ block='BASIC',
+ num_blocks=(4, 4, 4, 4),
+ num_channels=(48, 96, 192, 384))),
+ frozen_stages=4,
+ ),
+ concat_tensors=True,
+ neck=dict(
+ type='PoseWarperNeck',
+ in_channels=48,
+ freeze_trans_layer=True,
+ out_channels=17,
+ inner_channels=128,
+ deform_groups=17,
+ dilations=(3, 6, 12, 18, 24),
+ trans_conv_kernel=1,
+ res_blocks_cfg=dict(block='BASIC', num_blocks=20),
+ offsets_kernel=3,
+ deform_conv_kernel=3),
+ keypoint_head=dict(
+ type='TopdownHeatmapSimpleHead',
+ in_channels=17,
+ out_channels=17,
+ num_deconv_layers=0,
+ extra=dict(final_conv_kernel=0, ),
+ loss_keypoint=dict(type='JointsMSELoss', use_target_weight=True)),
+ train_cfg=dict(),
+ test_cfg=dict(
+ flip_test=False,
+ post_process='default',
+ shift_heatmap=True,
+ modulate_kernel=11))
+
+ detector = PoseWarper(model_cfg['backbone'], model_cfg['neck'],
+ model_cfg['keypoint_head'], model_cfg['train_cfg'],
+ model_cfg['test_cfg'], model_cfg['pretrained'], None,
+ model_cfg['concat_tensors'])
+ assert detector.concat_tensors
+
+ detector.init_weights()
+
+ input_shape = (2, 3, 64, 64)
+ num_frames = 2
+ mm_inputs = _demo_mm_inputs(input_shape, None, num_frames)
+
+ imgs = mm_inputs.pop('imgs')
+ target = mm_inputs.pop('target')
+ target_weight = mm_inputs.pop('target_weight')
+ img_metas = mm_inputs.pop('img_metas')
+
+ # Test forward train
+ losses = detector.forward(
+ imgs, target, target_weight, img_metas, return_loss=True)
+ assert isinstance(losses, dict)
+
+ # Test forward test
+ with torch.no_grad():
+ _ = detector.forward(imgs, img_metas=img_metas, return_loss=False)
+ _ = detector.forward_dummy(imgs)
+
+ # test argument 'concat_tensors'
+ model_cfg_copy = copy.deepcopy(model_cfg)
+ model_cfg_copy['concat_tensors'] = False
+
+ detector = PoseWarper(model_cfg_copy['backbone'], model_cfg_copy['neck'],
+ model_cfg_copy['keypoint_head'],
+ model_cfg_copy['train_cfg'],
+ model_cfg_copy['test_cfg'],
+ model_cfg_copy['pretrained'], None,
+ model_cfg_copy['concat_tensors'])
+ assert not detector.concat_tensors
+
+ detector.init_weights()
+
+ input_shape = (2, 3, 64, 64)
+ num_frames = 2
+ mm_inputs = _demo_mm_inputs(input_shape, None, num_frames)
+
+ imgs = mm_inputs.pop('imgs')
+ target = mm_inputs.pop('target')
+ target_weight = mm_inputs.pop('target_weight')
+ img_metas = mm_inputs.pop('img_metas')
+
+ # Test forward train
+ losses = detector.forward(
+ imgs, target, target_weight, img_metas, return_loss=True)
+ assert isinstance(losses, dict)
+
+ # Test forward test
+ with torch.no_grad():
+ _ = detector.forward(imgs, img_metas=img_metas, return_loss=False)
+ _ = detector.forward_dummy(imgs)
+
+ # flip test
+ model_cfg_copy = copy.deepcopy(model_cfg)
+ model_cfg_copy['test_cfg']['flip_test'] = True
+
+ detector = PoseWarper(model_cfg_copy['backbone'], model_cfg_copy['neck'],
+ model_cfg_copy['keypoint_head'],
+ model_cfg_copy['train_cfg'],
+ model_cfg_copy['test_cfg'],
+ model_cfg_copy['pretrained'], None,
+ model_cfg_copy['concat_tensors'])
+
+ detector.init_weights()
+
+ input_shape = (1, 3, 64, 64)
+ num_frames = 2
+ mm_inputs = _demo_mm_inputs(input_shape, None, num_frames)
+
+ imgs = mm_inputs.pop('imgs')
+ target = mm_inputs.pop('target')
+ target_weight = mm_inputs.pop('target_weight')
+ img_metas = mm_inputs.pop('img_metas')
+
+ # Test forward train
+ losses = detector.forward(
+ imgs, target, target_weight, img_metas, return_loss=True)
+ assert isinstance(losses, dict)
+
+ # Test forward test
+ with torch.no_grad():
+ _ = detector.forward(imgs, img_metas=img_metas, return_loss=False)
+ _ = detector.forward_dummy(imgs)
+
+ # test different number of dilations
+ model_cfg_copy = copy.deepcopy(model_cfg)
+ model_cfg_copy['neck']['dilations'] = (3, 6, 12)
+
+ detector = PoseWarper(model_cfg_copy['backbone'], model_cfg_copy['neck'],
+ model_cfg_copy['keypoint_head'],
+ model_cfg_copy['train_cfg'],
+ model_cfg_copy['test_cfg'],
+ model_cfg_copy['pretrained'], None,
+ model_cfg_copy['concat_tensors'])
+
+ detector.init_weights()
+
+ input_shape = (2, 3, 64, 64)
+ num_frames = 2
+ mm_inputs = _demo_mm_inputs(input_shape, None, num_frames)
+
+ imgs = mm_inputs.pop('imgs')
+ target = mm_inputs.pop('target')
+ target_weight = mm_inputs.pop('target_weight')
+ img_metas = mm_inputs.pop('img_metas')
+
+ # Test forward train
+ losses = detector.forward(
+ imgs, target, target_weight, img_metas, return_loss=True)
+ assert isinstance(losses, dict)
+
+ # Test forward test
+ with torch.no_grad():
+ _ = detector.forward(imgs, img_metas=img_metas, return_loss=False)
+ _ = detector.forward_dummy(imgs)
+
+ # test different backbone, change head accordingly
+ model_cfg_copy = copy.deepcopy(model_cfg)
+ model_cfg_copy['backbone'] = dict(type='ResNet', depth=18)
+ model_cfg_copy['neck']['in_channels'] = 512
+ model_cfg_copy['keypoint_head'] = dict(
+ type='TopdownHeatmapSimpleHead',
+ in_channels=17,
+ out_channels=17,
+ loss_keypoint=dict(type='JointsMSELoss', use_target_weight=True))
+
+ detector = PoseWarper(model_cfg_copy['backbone'], model_cfg_copy['neck'],
+ model_cfg_copy['keypoint_head'],
+ model_cfg_copy['train_cfg'],
+ model_cfg_copy['test_cfg'],
+ model_cfg_copy['pretrained'], None,
+ model_cfg_copy['concat_tensors'])
+
+ detector.init_weights()
+
+ input_shape = (1, 3, 64, 64)
+ num_frames = 2
+ mm_inputs = _demo_mm_inputs(input_shape, None, num_frames)
+
+ imgs = mm_inputs.pop('imgs')
+ target = mm_inputs.pop('target')
+ target_weight = mm_inputs.pop('target_weight')
+ img_metas = mm_inputs.pop('img_metas')
+
+ # Test forward train
+ losses = detector.forward(
+ imgs, target, target_weight, img_metas, return_loss=True)
+ assert isinstance(losses, dict)
+
+ # Test forward test
+ with torch.no_grad():
+ _ = detector.forward(imgs, img_metas=img_metas, return_loss=False)
+ _ = detector.forward_dummy(imgs)
+
+
+def _demo_mm_inputs(
+ input_shape=(1, 3, 256, 256), num_outputs=None, num_frames=1):
+ """Create a superset of inputs needed to run test or train batches.
+
+ Args:
+ input_shape (tuple):
+ input batch dimensions
+ num_frames (int):
+ number of frames for each sample, default: 1,
+ if larger than 1, return a list of tensors
+ """
+ (N, C, H, W) = input_shape
+
+ rng = np.random.RandomState(0)
+
+ imgs = rng.rand(*input_shape)
+ if num_outputs is not None:
+ target = np.zeros([N, num_outputs, 17, H // 4, W // 4],
+ dtype=np.float32)
+ target_weight = np.ones([N, num_outputs, 17, 1], dtype=np.float32)
+ else:
+ target = np.zeros([N, 17, H // 4, W // 4], dtype=np.float32)
+ target_weight = np.ones([N, 17, 1], dtype=np.float32)
+
+ img_metas = [{
+ 'img_shape': (H, W, C),
+ 'center': np.array([W / 2, H / 2]),
+ 'scale': np.array([0.5, 0.5]),
+ 'bbox_score': 1.0,
+ 'bbox_id': 0,
+ 'flip_pairs': [],
+ 'inference_channel': np.arange(17),
+ 'image_file': '.png',
+ 'frame_weight': np.random.uniform(0, 1, num_frames),
+ } for _ in range(N)]
+
+ mm_inputs = {
+ 'target': torch.FloatTensor(target),
+ 'target_weight': torch.FloatTensor(target_weight),
+ 'img_metas': img_metas
+ }
+
+ if num_frames == 1:
+ imgs = torch.FloatTensor(rng.rand(*input_shape)).requires_grad_(True)
+ else:
+
+ imgs = [
+ torch.FloatTensor(rng.rand(*input_shape)).requires_grad_(True)
+ for _ in range(num_frames)
+ ]
+
+ mm_inputs['imgs'] = imgs
+ return mm_inputs
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_models/test_top_down_head.py b/engine/pose_estimation/third-party/ViTPose/tests/test_models/test_top_down_head.py
new file mode 100644
index 0000000..2558e33
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_models/test_top_down_head.py
@@ -0,0 +1,518 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import numpy as np
+import pytest
+import torch
+
+from mmpose.models import (DeepposeRegressionHead, TopdownHeatmapMSMUHead,
+ TopdownHeatmapMultiStageHead,
+ TopdownHeatmapSimpleHead, ViPNASHeatmapSimpleHead)
+
+
+def test_vipnas_simple_head():
+ """Test simple head."""
+ with pytest.raises(TypeError):
+ # extra
+ _ = ViPNASHeatmapSimpleHead(
+ out_channels=3,
+ in_channels=512,
+ extra=[],
+ loss_keypoint=dict(type='JointsMSELoss', use_target_weight=True))
+
+ with pytest.raises(TypeError):
+ head = ViPNASHeatmapSimpleHead(
+ out_channels=3, in_channels=512, extra={'final_conv_kernel': 1})
+
+ # test num deconv layers
+ with pytest.raises(ValueError):
+ _ = ViPNASHeatmapSimpleHead(
+ out_channels=3,
+ in_channels=512,
+ num_deconv_layers=-1,
+ loss_keypoint=dict(type='JointsMSELoss', use_target_weight=True))
+
+ _ = ViPNASHeatmapSimpleHead(
+ out_channels=3,
+ in_channels=512,
+ num_deconv_layers=0,
+ loss_keypoint=dict(type='JointsMSELoss', use_target_weight=True))
+
+ with pytest.raises(ValueError):
+ # the number of layers should match
+ _ = ViPNASHeatmapSimpleHead(
+ out_channels=3,
+ in_channels=512,
+ num_deconv_layers=3,
+ num_deconv_filters=(256, 256),
+ num_deconv_kernels=(4, 4),
+ loss_keypoint=dict(type='JointsMSELoss', use_target_weight=True))
+
+ with pytest.raises(ValueError):
+ # the number of kernels should match
+ _ = ViPNASHeatmapSimpleHead(
+ out_channels=3,
+ in_channels=512,
+ num_deconv_layers=3,
+ num_deconv_filters=(256, 256, 256),
+ num_deconv_kernels=(4, 4),
+ loss_keypoint=dict(type='JointsMSELoss', use_target_weight=True))
+
+ with pytest.raises(ValueError):
+ # the deconv kernels should be 4, 3, 2
+ _ = ViPNASHeatmapSimpleHead(
+ out_channels=3,
+ in_channels=512,
+ num_deconv_layers=3,
+ num_deconv_filters=(256, 256, 256),
+ num_deconv_kernels=(3, 2, 0),
+ loss_keypoint=dict(type='JointsMSELoss', use_target_weight=True))
+
+ with pytest.raises(ValueError):
+ # the deconv kernels should be 4, 3, 2
+ _ = ViPNASHeatmapSimpleHead(
+ out_channels=3,
+ in_channels=512,
+ num_deconv_layers=3,
+ num_deconv_filters=(256, 256, 256),
+ num_deconv_kernels=(4, 4, -1),
+ loss_keypoint=dict(type='JointsMSELoss', use_target_weight=True))
+
+ # test final_conv_kernel
+ head = ViPNASHeatmapSimpleHead(
+ out_channels=3,
+ in_channels=512,
+ extra={'final_conv_kernel': 3},
+ loss_keypoint=dict(type='JointsMSELoss', use_target_weight=True))
+ head.init_weights()
+ assert head.final_layer.padding == (1, 1)
+ head = ViPNASHeatmapSimpleHead(
+ out_channels=3,
+ in_channels=512,
+ extra={'final_conv_kernel': 1},
+ loss_keypoint=dict(type='JointsMSELoss', use_target_weight=True))
+ assert head.final_layer.padding == (0, 0)
+ _ = ViPNASHeatmapSimpleHead(
+ out_channels=3,
+ in_channels=512,
+ extra={'final_conv_kernel': 0},
+ loss_keypoint=dict(type='JointsMSELoss', use_target_weight=True))
+
+ head = ViPNASHeatmapSimpleHead(
+ out_channels=3,
+ in_channels=512,
+ loss_keypoint=dict(type='JointsMSELoss', use_target_weight=True),
+ extra=dict(
+ final_conv_kernel=1, num_conv_layers=1, num_conv_kernels=(1, )))
+ assert len(head.final_layer) == 4
+
+ head = ViPNASHeatmapSimpleHead(
+ out_channels=3,
+ in_channels=512,
+ loss_keypoint=dict(type='JointsMSELoss', use_target_weight=True))
+ input_shape = (1, 512, 32, 32)
+ inputs = _demo_inputs(input_shape)
+ out = head(inputs)
+ assert out.shape == torch.Size([1, 3, 256, 256])
+
+ head = ViPNASHeatmapSimpleHead(
+ out_channels=3,
+ in_channels=512,
+ num_deconv_layers=0,
+ loss_keypoint=dict(type='JointsMSELoss', use_target_weight=True))
+ input_shape = (1, 512, 32, 32)
+ inputs = _demo_inputs(input_shape)
+ out = head(inputs)
+ assert out.shape == torch.Size([1, 3, 32, 32])
+
+ head = ViPNASHeatmapSimpleHead(
+ out_channels=3,
+ in_channels=512,
+ num_deconv_layers=0,
+ loss_keypoint=dict(type='JointsMSELoss', use_target_weight=True))
+ input_shape = (1, 512, 32, 32)
+ inputs = _demo_inputs(input_shape)
+ out = head([inputs])
+ assert out.shape == torch.Size([1, 3, 32, 32])
+
+ head.init_weights()
+
+
+def test_top_down_simple_head():
+ """Test simple head."""
+ with pytest.raises(TypeError):
+ # extra
+ _ = TopdownHeatmapSimpleHead(
+ out_channels=3,
+ in_channels=512,
+ extra=[],
+ loss_keypoint=dict(type='JointsMSELoss', use_target_weight=True))
+
+ with pytest.raises(TypeError):
+ head = TopdownHeatmapSimpleHead(
+ out_channels=3, in_channels=512, extra={'final_conv_kernel': 1})
+
+ # test num deconv layers
+ with pytest.raises(ValueError):
+ _ = TopdownHeatmapSimpleHead(
+ out_channels=3,
+ in_channels=512,
+ num_deconv_layers=-1,
+ loss_keypoint=dict(type='JointsMSELoss', use_target_weight=True))
+
+ _ = TopdownHeatmapSimpleHead(
+ out_channels=3,
+ in_channels=512,
+ num_deconv_layers=0,
+ loss_keypoint=dict(type='JointsMSELoss', use_target_weight=True))
+
+ with pytest.raises(ValueError):
+ # the number of layers should match
+ _ = TopdownHeatmapSimpleHead(
+ out_channels=3,
+ in_channels=512,
+ num_deconv_layers=3,
+ num_deconv_filters=(256, 256),
+ num_deconv_kernels=(4, 4),
+ loss_keypoint=dict(type='JointsMSELoss', use_target_weight=True))
+
+ with pytest.raises(ValueError):
+ # the number of kernels should match
+ _ = TopdownHeatmapSimpleHead(
+ out_channels=3,
+ in_channels=512,
+ num_deconv_layers=3,
+ num_deconv_filters=(256, 256, 256),
+ num_deconv_kernels=(4, 4),
+ loss_keypoint=dict(type='JointsMSELoss', use_target_weight=True))
+
+ with pytest.raises(ValueError):
+ # the deconv kernels should be 4, 3, 2
+ _ = TopdownHeatmapSimpleHead(
+ out_channels=3,
+ in_channels=512,
+ num_deconv_layers=3,
+ num_deconv_filters=(256, 256, 256),
+ num_deconv_kernels=(3, 2, 0),
+ loss_keypoint=dict(type='JointsMSELoss', use_target_weight=True))
+
+ with pytest.raises(ValueError):
+ # the deconv kernels should be 4, 3, 2
+ _ = TopdownHeatmapSimpleHead(
+ out_channels=3,
+ in_channels=512,
+ num_deconv_layers=3,
+ num_deconv_filters=(256, 256, 256),
+ num_deconv_kernels=(4, 4, -1),
+ loss_keypoint=dict(type='JointsMSELoss', use_target_weight=True))
+
+ # test final_conv_kernel
+ head = TopdownHeatmapSimpleHead(
+ out_channels=3,
+ in_channels=512,
+ extra={'final_conv_kernel': 3},
+ loss_keypoint=dict(type='JointsMSELoss', use_target_weight=True))
+ head.init_weights()
+ assert head.final_layer.padding == (1, 1)
+ head = TopdownHeatmapSimpleHead(
+ out_channels=3,
+ in_channels=512,
+ extra={'final_conv_kernel': 1},
+ loss_keypoint=dict(type='JointsMSELoss', use_target_weight=True))
+ assert head.final_layer.padding == (0, 0)
+ _ = TopdownHeatmapSimpleHead(
+ out_channels=3,
+ in_channels=512,
+ extra={'final_conv_kernel': 0},
+ loss_keypoint=dict(type='JointsMSELoss', use_target_weight=True))
+
+ head = TopdownHeatmapSimpleHead(
+ out_channels=3,
+ in_channels=512,
+ loss_keypoint=dict(type='JointsMSELoss', use_target_weight=True),
+ extra=dict(
+ final_conv_kernel=1, num_conv_layers=1, num_conv_kernels=(1, )))
+ assert len(head.final_layer) == 4
+
+ head = TopdownHeatmapSimpleHead(
+ out_channels=3,
+ in_channels=512,
+ loss_keypoint=dict(type='JointsMSELoss', use_target_weight=True))
+ input_shape = (1, 512, 32, 32)
+ inputs = _demo_inputs(input_shape)
+ out = head(inputs)
+ assert out.shape == torch.Size([1, 3, 256, 256])
+
+ head = TopdownHeatmapSimpleHead(
+ out_channels=3,
+ in_channels=512,
+ num_deconv_layers=0,
+ loss_keypoint=dict(type='JointsMSELoss', use_target_weight=True))
+ input_shape = (1, 512, 32, 32)
+ inputs = _demo_inputs(input_shape)
+ out = head(inputs)
+ assert out.shape == torch.Size([1, 3, 32, 32])
+
+ head = TopdownHeatmapSimpleHead(
+ out_channels=3,
+ in_channels=512,
+ num_deconv_layers=0,
+ loss_keypoint=dict(type='JointsMSELoss', use_target_weight=True))
+ input_shape = (1, 512, 32, 32)
+ inputs = _demo_inputs(input_shape)
+ out = head([inputs])
+ assert out.shape == torch.Size([1, 3, 32, 32])
+
+ head.init_weights()
+
+
+def test_top_down_multistage_head():
+ """Test multistage head."""
+ with pytest.raises(TypeError):
+ # the number of layers should match
+ _ = TopdownHeatmapMultiStageHead(
+ out_channels=3,
+ in_channels=512,
+ num_stages=1,
+ extra=[],
+ loss_keypoint=dict(type='JointsMSELoss', use_target_weight=True))
+
+ # test num deconv layers
+ with pytest.raises(ValueError):
+ _ = TopdownHeatmapMultiStageHead(
+ out_channels=3,
+ in_channels=512,
+ num_deconv_layers=-1,
+ loss_keypoint=dict(type='JointsMSELoss', use_target_weight=True))
+
+ _ = TopdownHeatmapMultiStageHead(
+ out_channels=3,
+ in_channels=512,
+ num_deconv_layers=0,
+ loss_keypoint=dict(type='JointsMSELoss', use_target_weight=True))
+
+ with pytest.raises(ValueError):
+ # the number of layers should match
+ _ = TopdownHeatmapMultiStageHead(
+ out_channels=3,
+ in_channels=512,
+ num_stages=1,
+ num_deconv_layers=3,
+ num_deconv_filters=(256, 256),
+ num_deconv_kernels=(4, 4),
+ loss_keypoint=dict(type='JointsMSELoss', use_target_weight=True))
+
+ with pytest.raises(ValueError):
+ # the number of kernels should match
+ _ = TopdownHeatmapMultiStageHead(
+ out_channels=3,
+ in_channels=512,
+ num_stages=1,
+ num_deconv_layers=3,
+ num_deconv_filters=(256, 256, 256),
+ num_deconv_kernels=(4, 4),
+ loss_keypoint=dict(type='JointsMSELoss', use_target_weight=True))
+
+ with pytest.raises(ValueError):
+ # the deconv kernels should be 4, 3, 2
+ _ = TopdownHeatmapMultiStageHead(
+ out_channels=3,
+ in_channels=512,
+ num_stages=1,
+ num_deconv_layers=3,
+ num_deconv_filters=(256, 256, 256),
+ num_deconv_kernels=(3, 2, 0),
+ loss_keypoint=dict(type='JointsMSELoss', use_target_weight=True))
+
+ with pytest.raises(ValueError):
+ # the deconv kernels should be 4, 3, 2
+ _ = TopdownHeatmapMultiStageHead(
+ out_channels=3,
+ in_channels=512,
+ num_deconv_layers=3,
+ num_deconv_filters=(256, 256, 256),
+ num_deconv_kernels=(4, 4, -1),
+ loss_keypoint=dict(type='JointsMSELoss', use_target_weight=True))
+
+ with pytest.raises(AssertionError):
+ # inputs should be list
+ head = TopdownHeatmapMultiStageHead(
+ out_channels=3,
+ in_channels=512,
+ loss_keypoint=dict(type='JointsMSELoss', use_target_weight=True))
+ input_shape = (1, 512, 32, 32)
+ inputs = _demo_inputs(input_shape)
+ out = head(inputs)
+
+ # test final_conv_kernel
+ head = TopdownHeatmapMultiStageHead(
+ out_channels=3,
+ in_channels=512,
+ extra={'final_conv_kernel': 3},
+ loss_keypoint=dict(type='JointsMSELoss', use_target_weight=True))
+ head.init_weights()
+ assert head.multi_final_layers[0].padding == (1, 1)
+ head = TopdownHeatmapMultiStageHead(
+ out_channels=3,
+ in_channels=512,
+ extra={'final_conv_kernel': 1},
+ loss_keypoint=dict(type='JointsMSELoss', use_target_weight=True))
+ assert head.multi_final_layers[0].padding == (0, 0)
+ _ = TopdownHeatmapMultiStageHead(
+ out_channels=3,
+ in_channels=512,
+ extra={'final_conv_kernel': 0},
+ loss_keypoint=dict(type='JointsMSELoss', use_target_weight=True))
+
+ head = TopdownHeatmapMultiStageHead(
+ out_channels=3,
+ in_channels=512,
+ loss_keypoint=dict(type='JointsMSELoss', use_target_weight=True))
+ input_shape = (1, 512, 32, 32)
+ inputs = _demo_inputs(input_shape)
+ out = head([inputs])
+ assert len(out) == 1
+ assert out[0].shape == torch.Size([1, 3, 256, 256])
+
+ head = TopdownHeatmapMultiStageHead(
+ out_channels=3,
+ in_channels=512,
+ num_deconv_layers=0,
+ loss_keypoint=dict(type='JointsMSELoss', use_target_weight=True))
+ input_shape = (1, 512, 32, 32)
+ inputs = _demo_inputs(input_shape)
+ out = head([inputs])
+ assert out[0].shape == torch.Size([1, 3, 32, 32])
+
+ head.init_weights()
+
+
+def test_top_down_msmu_head():
+ """Test multi-stage multi-unit head."""
+ with pytest.raises(AssertionError):
+ # inputs should be list
+ head = TopdownHeatmapMSMUHead(
+ out_shape=(64, 48),
+ unit_channels=256,
+ num_stages=2,
+ num_units=2,
+ loss_keypoint=(
+ [dict(type='JointsMSELoss', use_target_weight=True)] * 2 +
+ [dict(type='JointsOHKMMSELoss', use_target_weight=True)]) * 2)
+ input_shape = (1, 256, 32, 32)
+ inputs = _demo_inputs(input_shape)
+ _ = head(inputs)
+
+ with pytest.raises(AssertionError):
+ # inputs should be list[list, ...]
+ head = TopdownHeatmapMSMUHead(
+ out_shape=(64, 48),
+ unit_channels=256,
+ num_stages=2,
+ num_units=2,
+ loss_keypoint=(
+ [dict(type='JointsMSELoss', use_target_weight=True)] * 2 +
+ [dict(type='JointsOHKMMSELoss', use_target_weight=True)]) * 2)
+ input_shape = (1, 256, 32, 32)
+ inputs = _demo_inputs(input_shape)
+ inputs = [inputs] * 2
+ _ = head(inputs)
+
+ with pytest.raises(AssertionError):
+ # len(inputs) should equal to num_stages
+ head = TopdownHeatmapMSMUHead(
+ out_shape=(64, 48),
+ unit_channels=256,
+ num_stages=2,
+ num_units=2,
+ loss_keypoint=(
+ [dict(type='JointsMSELoss', use_target_weight=True)] * 2 +
+ [dict(type='JointsOHKMMSELoss', use_target_weight=True)]) * 2)
+ input_shape = (1, 256, 32, 32)
+ inputs = _demo_inputs(input_shape)
+ inputs = [[inputs] * 2] * 3
+ _ = head(inputs)
+
+ with pytest.raises(AssertionError):
+ # len(inputs[0]) should equal to num_units
+ head = TopdownHeatmapMSMUHead(
+ out_shape=(64, 48),
+ unit_channels=256,
+ num_stages=2,
+ num_units=2,
+ loss_keypoint=(
+ [dict(type='JointsMSELoss', use_target_weight=True)] * 2 +
+ [dict(type='JointsOHKMMSELoss', use_target_weight=True)]) * 2)
+ input_shape = (1, 256, 32, 32)
+ inputs = _demo_inputs(input_shape)
+ inputs = [[inputs] * 3] * 2
+ _ = head(inputs)
+
+ with pytest.raises(AssertionError):
+ # input channels should equal to param unit_channels
+ head = TopdownHeatmapMSMUHead(
+ out_shape=(64, 48),
+ unit_channels=256,
+ num_stages=2,
+ num_units=2,
+ loss_keypoint=(
+ [dict(type='JointsMSELoss', use_target_weight=True)] * 2 +
+ [dict(type='JointsOHKMMSELoss', use_target_weight=True)]) * 2)
+ input_shape = (1, 128, 32, 32)
+ inputs = _demo_inputs(input_shape)
+ inputs = [[inputs] * 2] * 2
+ _ = head(inputs)
+
+ head = TopdownHeatmapMSMUHead(
+ out_shape=(64, 48),
+ unit_channels=256,
+ out_channels=17,
+ num_stages=2,
+ num_units=2,
+ loss_keypoint=(
+ [dict(type='JointsMSELoss', use_target_weight=True)] * 2 +
+ [dict(type='JointsOHKMMSELoss', use_target_weight=True)]) * 2)
+ input_shape = (1, 256, 32, 32)
+ inputs = _demo_inputs(input_shape)
+ inputs = [[inputs] * 2] * 2
+ out = head(inputs)
+ assert len(out) == 2 * 2
+ assert out[0].shape == torch.Size([1, 17, 64, 48])
+
+ head.init_weights()
+
+
+def test_fc_head():
+ """Test fc head."""
+ head = DeepposeRegressionHead(
+ in_channels=2048,
+ num_joints=17,
+ loss_keypoint=dict(type='SmoothL1Loss', use_target_weight=True))
+
+ head.init_weights()
+
+ input_shape = (1, 2048)
+ inputs = _demo_inputs(input_shape)
+ out = head(inputs)
+ assert out.shape == torch.Size([1, 17, 2])
+
+ loss = head.get_loss(out, out, torch.ones_like(out))
+ assert torch.allclose(loss['reg_loss'], torch.tensor(0.))
+
+ _ = head.inference_model(inputs)
+ _ = head.inference_model(inputs, [])
+
+ acc = head.get_accuracy(out, out, torch.ones_like(out))
+ assert acc['acc_pose'] == 1.
+
+
+def _demo_inputs(input_shape=(1, 3, 64, 64)):
+ """Create a superset of inputs needed to run backbone.
+
+ Args:
+ input_shape (tuple): input batch dimensions.
+ Default: (1, 3, 64, 64).
+ Returns:
+ Random input tensor with the size of input_shape.
+ """
+ inps = np.random.random(input_shape)
+ inps = torch.FloatTensor(inps)
+ return inps
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_necks/test_gap_neck.py b/engine/pose_estimation/third-party/ViTPose/tests/test_necks/test_gap_neck.py
new file mode 100644
index 0000000..57d26cb
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_necks/test_gap_neck.py
@@ -0,0 +1,43 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import numpy as np
+import pytest
+import torch
+
+from mmpose.models.necks import GlobalAveragePooling
+
+
+def test_gap():
+ """Test GlobalAveragePooling neck."""
+ gap = GlobalAveragePooling()
+
+ with pytest.raises(TypeError):
+ gap(1)
+
+ x0_shape = (32, 1024, 4, 4)
+ x1_shape = (32, 2048, 2, 2)
+ x0 = _demo_inputs(x0_shape)
+ x1 = _demo_inputs(x1_shape)
+
+ y = gap(x0)
+ assert y.shape == torch.Size([32, 1024])
+
+ y = gap([x0, x1])
+ assert y[0].shape == torch.Size([32, 1024])
+ assert y[1].shape == torch.Size([32, 2048])
+
+ y = gap((x0, x1))
+ assert y[0].shape == torch.Size([32, 1024])
+ assert y[1].shape == torch.Size([32, 2048])
+
+
+def _demo_inputs(input_shape=(1, 3, 64, 64)):
+ """Create a superset of inputs needed to run backbone.
+
+ Args:
+ input_shape (tuple): input batch dimensions.
+ Default: (1, 3, 64, 64).
+ """
+ imgs = np.random.random(input_shape)
+ imgs = torch.FloatTensor(imgs)
+
+ return imgs
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_necks/test_posewarper_neck.py b/engine/pose_estimation/third-party/ViTPose/tests/test_necks/test_posewarper_neck.py
new file mode 100644
index 0000000..45faabf
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_necks/test_posewarper_neck.py
@@ -0,0 +1,143 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import numpy as np
+import pytest
+import torch
+
+from mmpose.models.necks import PoseWarperNeck
+
+
+def test_posewarper_neck():
+ """Test PoseWarperNeck."""
+ with pytest.raises(AssertionError):
+ # test value of trans_conv_kernel
+ _ = PoseWarperNeck(
+ out_channels=3,
+ in_channels=512,
+ inner_channels=128,
+ trans_conv_kernel=2)
+
+ with pytest.raises(TypeError):
+ # test type of res_blocks_cfg
+ _ = PoseWarperNeck(
+ out_channels=3,
+ in_channels=512,
+ inner_channels=128,
+ res_blocks_cfg=2)
+
+ with pytest.raises(AssertionError):
+ # test value of dilations
+ neck = PoseWarperNeck(
+ out_channels=3, in_channels=512, inner_channels=128, dilations=[])
+
+ in_channels = 48
+ out_channels = 17
+ inner_channels = 128
+
+ neck = PoseWarperNeck(
+ in_channels=in_channels,
+ out_channels=out_channels,
+ inner_channels=inner_channels)
+
+ with pytest.raises(TypeError):
+ # the forward require two arguments: inputs and frame_weight
+ _ = neck(1)
+
+ with pytest.raises(AssertionError):
+ # the inputs to PoseWarperNeck must be list or tuple
+ _ = neck(1, [0.1])
+
+ # test the case when num_frames * batch_size if larger than
+ # the default value of 'im2col_step' but can not be divided
+ # by it in mmcv.ops.deform_conv
+ b_0 = 8 # batch_size
+ b_1 = 16
+ h_0 = 4 # image height
+ h_1 = 2
+
+ num_frame_0 = 2
+ num_frame_1 = 5
+
+ # test input format
+ # B, C, H, W
+ x0_shape = (b_0, in_channels, h_0, h_0)
+ x1_shape = (b_1, in_channels, h_1, h_1)
+
+ # test concat_tensors case
+ # at the same time, features output from backbone like ResNet is Tensors
+ x0_shape = (b_0 * num_frame_0, in_channels, h_0, h_0)
+ x0 = _demo_inputs(x0_shape, length=1)
+ frame_weight_0 = np.random.uniform(0, 1, num_frame_0)
+
+ # test forward
+ y = neck(x0, frame_weight_0)
+ assert y.shape == torch.Size([b_0, out_channels, h_0, h_0])
+
+ # test concat_tensors case
+ # this time, features output from backbone like HRNet
+ # is list of Tensors rather than Tensors
+ x0_shape = (b_0 * num_frame_0, in_channels, h_0, h_0)
+ x0 = _demo_inputs(x0_shape, length=2)
+ x0 = [x0]
+ frame_weight_0 = np.random.uniform(0, 1, num_frame_0)
+
+ # test forward
+ y = neck(x0, frame_weight_0)
+ assert y.shape == torch.Size([b_0, out_channels, h_0, h_0])
+
+ # test not concat_tensors case
+ # at the same time, features output from backbone like ResNet is Tensors
+ x1_shape = (b_1, in_channels, h_1, h_1)
+ x1 = _demo_inputs(x1_shape, length=num_frame_1)
+ frame_weight_1 = np.random.uniform(0, 1, num_frame_1)
+
+ # test forward
+ y = neck(x1, frame_weight_1)
+ assert y.shape == torch.Size([b_1, out_channels, h_1, h_1])
+
+ # test not concat_tensors case
+ # this time, features output from backbone like HRNet
+ # is list of Tensors rather than Tensors
+ x1_shape = (b_1, in_channels, h_1, h_1)
+ x1 = _demo_inputs(x1_shape, length=2)
+ x1 = [x1 for _ in range(num_frame_1)]
+ frame_weight_1 = np.random.uniform(0, 1, num_frame_1)
+
+ # test forward
+ y = neck(x1, frame_weight_1)
+ assert y.shape == torch.Size([b_1, out_channels, h_1, h_1])
+
+ # test special case that when in concat_tensors case,
+ # batch_size * num_frames is larger than the default value
+ # 'im2col_step' in mmcv.ops.deform_conv, but can not be divided by it
+ # see https://github.com/open-mmlab/mmcv/issues/1440
+ x1_shape = (b_1 * num_frame_1, in_channels, h_1, h_1)
+ x1 = _demo_inputs(x1_shape, length=2)
+ x1 = [x1]
+ frame_weight_0 = np.random.uniform(0, 1, num_frame_1)
+
+ y = neck(x1, frame_weight_1)
+ assert y.shape == torch.Size([b_1, out_channels, h_1, h_1])
+
+ # test the inappropriate value of `im2col_step`
+ neck = PoseWarperNeck(
+ in_channels=in_channels,
+ out_channels=out_channels,
+ inner_channels=inner_channels,
+ im2col_step=32)
+ with pytest.raises(AssertionError):
+ _ = neck(x1, frame_weight_1)
+
+
+def _demo_inputs(input_shape=(80, 48, 4, 4), length=1):
+ """Create a superset of inputs needed to run backbone.
+
+ Args:
+ input_shape (tuple): input batch dimensions.
+ Default: (1, 3, 64, 64).
+ length (int): the length of output list
+ nested (bool): whether the output Tensor is double-nested list.
+ """
+ imgs = [
+ torch.FloatTensor(np.random.random(input_shape)) for _ in range(length)
+ ]
+ return imgs
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_onnx.py b/engine/pose_estimation/third-party/ViTPose/tests/test_onnx.py
new file mode 100644
index 0000000..c0179c2
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_onnx.py
@@ -0,0 +1,30 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os.path as osp
+import tempfile
+
+import torch.nn as nn
+
+from tools.deployment.pytorch2onnx import _convert_batchnorm, pytorch2onnx
+
+
+class DummyModel(nn.Module):
+
+ def __init__(self):
+ super().__init__()
+ self.conv = nn.Conv3d(1, 2, 1)
+ self.bn = nn.SyncBatchNorm(2)
+
+ def forward(self, x):
+ return self.bn(self.conv(x))
+
+ def forward_dummy(self, x):
+ return (self.forward(x), )
+
+
+def test_onnx_exporting():
+ with tempfile.TemporaryDirectory() as tmpdir:
+ out_file = osp.join(tmpdir, 'tmp.onnx')
+ model = DummyModel()
+ model = _convert_batchnorm(model)
+ # test exporting
+ pytorch2onnx(model, (1, 1, 1, 1, 1), output_file=out_file)
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_optimizer.py b/engine/pose_estimation/third-party/ViTPose/tests/test_optimizer.py
new file mode 100644
index 0000000..2379f61
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_optimizer.py
@@ -0,0 +1,101 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import torch
+import torch.nn as nn
+
+from mmpose.core import build_optimizers
+
+
+class ExampleModel(nn.Module):
+
+ def __init__(self):
+ super().__init__()
+ self.model1 = nn.Conv2d(3, 8, kernel_size=3)
+ self.model2 = nn.Conv2d(3, 4, kernel_size=3)
+
+ def forward(self, x):
+ return x
+
+
+def test_build_optimizers():
+ base_lr = 0.0001
+ base_wd = 0.0002
+ momentum = 0.9
+
+ # basic config with ExampleModel
+ optimizer_cfg = dict(
+ model1=dict(
+ type='SGD', lr=base_lr, weight_decay=base_wd, momentum=momentum),
+ model2=dict(
+ type='SGD', lr=base_lr, weight_decay=base_wd, momentum=momentum))
+ model = ExampleModel()
+ optimizers = build_optimizers(model, optimizer_cfg)
+ param_dict = dict(model.named_parameters())
+ assert isinstance(optimizers, dict)
+ for i in range(2):
+ optimizer = optimizers[f'model{i+1}']
+ param_groups = optimizer.param_groups[0]
+ assert isinstance(optimizer, torch.optim.SGD)
+ assert optimizer.defaults['lr'] == base_lr
+ assert optimizer.defaults['momentum'] == momentum
+ assert optimizer.defaults['weight_decay'] == base_wd
+ assert len(param_groups['params']) == 2
+ assert torch.equal(param_groups['params'][0],
+ param_dict[f'model{i+1}.weight'])
+ assert torch.equal(param_groups['params'][1],
+ param_dict[f'model{i+1}.bias'])
+
+ # basic config with Parallel model
+ model = torch.nn.DataParallel(ExampleModel())
+ optimizers = build_optimizers(model, optimizer_cfg)
+ param_dict = dict(model.named_parameters())
+ assert isinstance(optimizers, dict)
+ for i in range(2):
+ optimizer = optimizers[f'model{i+1}']
+ param_groups = optimizer.param_groups[0]
+ assert isinstance(optimizer, torch.optim.SGD)
+ assert optimizer.defaults['lr'] == base_lr
+ assert optimizer.defaults['momentum'] == momentum
+ assert optimizer.defaults['weight_decay'] == base_wd
+ assert len(param_groups['params']) == 2
+ assert torch.equal(param_groups['params'][0],
+ param_dict[f'module.model{i+1}.weight'])
+ assert torch.equal(param_groups['params'][1],
+ param_dict[f'module.model{i+1}.bias'])
+
+ # basic config with ExampleModel (one optimizer)
+ optimizer_cfg = dict(
+ type='SGD', lr=base_lr, weight_decay=base_wd, momentum=momentum)
+ model = ExampleModel()
+ optimizer = build_optimizers(model, optimizer_cfg)
+ param_dict = dict(model.named_parameters())
+ assert isinstance(optimizers, dict)
+ param_groups = optimizer.param_groups[0]
+ assert isinstance(optimizer, torch.optim.SGD)
+ assert optimizer.defaults['lr'] == base_lr
+ assert optimizer.defaults['momentum'] == momentum
+ assert optimizer.defaults['weight_decay'] == base_wd
+ assert len(param_groups['params']) == 4
+ assert torch.equal(param_groups['params'][0], param_dict['model1.weight'])
+ assert torch.equal(param_groups['params'][1], param_dict['model1.bias'])
+ assert torch.equal(param_groups['params'][2], param_dict['model2.weight'])
+ assert torch.equal(param_groups['params'][3], param_dict['model2.bias'])
+
+ # basic config with Parallel model (one optimizer)
+ model = torch.nn.DataParallel(ExampleModel())
+ optimizer = build_optimizers(model, optimizer_cfg)
+ param_dict = dict(model.named_parameters())
+ assert isinstance(optimizers, dict)
+ param_groups = optimizer.param_groups[0]
+ assert isinstance(optimizer, torch.optim.SGD)
+ assert optimizer.defaults['lr'] == base_lr
+ assert optimizer.defaults['momentum'] == momentum
+ assert optimizer.defaults['weight_decay'] == base_wd
+ assert len(param_groups['params']) == 4
+ assert torch.equal(param_groups['params'][0],
+ param_dict['module.model1.weight'])
+ assert torch.equal(param_groups['params'][1],
+ param_dict['module.model1.bias'])
+ assert torch.equal(param_groups['params'][2],
+ param_dict['module.model2.weight'])
+ assert torch.equal(param_groups['params'][3],
+ param_dict['module.model2.bias'])
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_pipelines/test_bottom_up_pipelines.py b/engine/pose_estimation/third-party/ViTPose/tests/test_pipelines/test_bottom_up_pipelines.py
new file mode 100644
index 0000000..6d05c63
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_pipelines/test_bottom_up_pipelines.py
@@ -0,0 +1,427 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import copy
+import os.path as osp
+
+import numpy as np
+import pytest
+import xtcocotools
+from xtcocotools.coco import COCO
+
+from mmpose.datasets.pipelines import (BottomUpGenerateHeatmapTarget,
+ BottomUpGeneratePAFTarget,
+ BottomUpGenerateTarget,
+ BottomUpGetImgSize,
+ BottomUpRandomAffine,
+ BottomUpRandomFlip, BottomUpResizeAlign,
+ LoadImageFromFile)
+
+
+def _get_mask(coco, anno, img_id):
+ img_info = coco.loadImgs(img_id)[0]
+
+ m = np.zeros((img_info['height'], img_info['width']), dtype=np.float32)
+
+ for obj in anno:
+ if obj['iscrowd']:
+ rle = xtcocotools.mask.frPyObjects(obj['segmentation'],
+ img_info['height'],
+ img_info['width'])
+ m += xtcocotools.mask.decode(rle)
+ elif obj['num_keypoints'] == 0:
+ rles = xtcocotools.mask.frPyObjects(obj['segmentation'],
+ img_info['height'],
+ img_info['width'])
+ for rle in rles:
+ m += xtcocotools.mask.decode(rle)
+
+ return m < 0.5
+
+
+def _get_joints(anno, ann_info, int_sigma):
+ num_people = len(anno)
+
+ if ann_info['scale_aware_sigma']:
+ joints = np.zeros((num_people, ann_info['num_joints'], 4),
+ dtype=np.float32)
+ else:
+ joints = np.zeros((num_people, ann_info['num_joints'], 3),
+ dtype=np.float32)
+
+ for i, obj in enumerate(anno):
+ joints[i, :ann_info['num_joints'], :3] = \
+ np.array(obj['keypoints']).reshape([-1, 3])
+ if ann_info['scale_aware_sigma']:
+ # get person box
+ box = obj['bbox']
+ size = max(box[2], box[3])
+ sigma = size / 256 * 2
+ if int_sigma:
+ sigma = int(np.ceil(sigma))
+ assert sigma > 0, sigma
+ joints[i, :, 3] = sigma
+
+ return joints
+
+
+def _check_flip(origin_imgs, result_imgs):
+ """Check if the origin_imgs are flipped correctly."""
+ h, w, c = origin_imgs.shape
+ for i in range(h):
+ for j in range(w):
+ for k in range(c):
+ if result_imgs[i, j, k] != origin_imgs[i, w - 1 - j, k]:
+ return False
+ return True
+
+
+def test_bottomup_pipeline():
+
+ data_prefix = 'tests/data/coco/'
+ ann_file = osp.join(data_prefix, 'test_coco.json')
+ coco = COCO(ann_file)
+
+ ann_info = {}
+ ann_info['flip_pairs'] = [[1, 2], [3, 4], [5, 6], [7, 8], [9, 10],
+ [11, 12], [13, 14], [15, 16]]
+ ann_info['flip_index'] = [
+ 0, 2, 1, 4, 3, 6, 5, 8, 7, 10, 9, 12, 11, 14, 13, 16, 15
+ ]
+
+ ann_info['use_different_joint_weights'] = False
+ ann_info['joint_weights'] = np.array([
+ 1., 1., 1., 1., 1., 1., 1., 1.2, 1.2, 1.5, 1.5, 1., 1., 1.2, 1.2, 1.5,
+ 1.5
+ ],
+ dtype=np.float32).reshape((17, 1))
+ ann_info['image_size'] = np.array([384, 512])
+ ann_info['heatmap_size'] = np.array([[96, 128], [192, 256]])
+ ann_info['num_joints'] = 17
+ ann_info['num_scales'] = 2
+ ann_info['scale_aware_sigma'] = False
+
+ ann_ids = coco.getAnnIds(785)
+ anno = coco.loadAnns(ann_ids)
+ mask = _get_mask(coco, anno, 785)
+
+ anno = [
+ obj for obj in anno if obj['iscrowd'] == 0 or obj['num_keypoints'] > 0
+ ]
+ joints = _get_joints(anno, ann_info, False)
+
+ mask_list = [mask.copy() for _ in range(ann_info['num_scales'])]
+ joints_list = [joints.copy() for _ in range(ann_info['num_scales'])]
+
+ results = {}
+ results['dataset'] = 'coco'
+ results['image_file'] = osp.join(data_prefix, '000000000785.jpg')
+ results['mask'] = mask_list
+ results['joints'] = joints_list
+ results['ann_info'] = ann_info
+
+ transform = LoadImageFromFile()
+ results = transform(copy.deepcopy(results))
+ assert results['img'].shape == (425, 640, 3)
+
+ # test HorizontalFlip
+ random_horizontal_flip = BottomUpRandomFlip(flip_prob=1.)
+ results_horizontal_flip = random_horizontal_flip(copy.deepcopy(results))
+ assert _check_flip(results['img'], results_horizontal_flip['img'])
+
+ random_horizontal_flip = BottomUpRandomFlip(flip_prob=0.)
+ results_horizontal_flip = random_horizontal_flip(copy.deepcopy(results))
+ assert (results['img'] == results_horizontal_flip['img']).all()
+
+ results_copy = copy.deepcopy(results)
+ results_copy['mask'] = mask_list[0]
+ with pytest.raises(AssertionError):
+ results_horizontal_flip = random_horizontal_flip(
+ copy.deepcopy(results_copy))
+
+ results_copy = copy.deepcopy(results)
+ results_copy['joints'] = joints_list[0]
+ with pytest.raises(AssertionError):
+ results_horizontal_flip = random_horizontal_flip(
+ copy.deepcopy(results_copy))
+
+ results_copy = copy.deepcopy(results)
+ results_copy['joints'] = joints_list[:1]
+ with pytest.raises(AssertionError):
+ results_horizontal_flip = random_horizontal_flip(
+ copy.deepcopy(results_copy))
+
+ results_copy = copy.deepcopy(results)
+ results_copy['mask'] = mask_list[:1]
+ with pytest.raises(AssertionError):
+ results_horizontal_flip = random_horizontal_flip(
+ copy.deepcopy(results_copy))
+
+ # test TopDownAffine
+ random_affine_transform = BottomUpRandomAffine(30, [0.75, 1.5], 'short', 0)
+ results_affine_transform = random_affine_transform(copy.deepcopy(results))
+ assert results_affine_transform['img'].shape == (512, 384, 3)
+
+ random_affine_transform = BottomUpRandomAffine(30, [0.75, 1.5], 'short',
+ 40)
+ results_affine_transform = random_affine_transform(copy.deepcopy(results))
+ assert results_affine_transform['img'].shape == (512, 384, 3)
+
+ results_copy = copy.deepcopy(results)
+ results_copy['ann_info']['scale_aware_sigma'] = True
+ joints = _get_joints(anno, results_copy['ann_info'], False)
+ results_copy['joints'] = \
+ [joints.copy() for _ in range(results_copy['ann_info']['num_scales'])]
+ results_affine_transform = random_affine_transform(results_copy)
+ assert results_affine_transform['img'].shape == (512, 384, 3)
+
+ results_copy = copy.deepcopy(results)
+ results_copy['mask'] = mask_list[0]
+ with pytest.raises(AssertionError):
+ results_horizontal_flip = random_affine_transform(
+ copy.deepcopy(results_copy))
+
+ results_copy = copy.deepcopy(results)
+ results_copy['joints'] = joints_list[0]
+ with pytest.raises(AssertionError):
+ results_horizontal_flip = random_affine_transform(
+ copy.deepcopy(results_copy))
+
+ results_copy = copy.deepcopy(results)
+ results_copy['joints'] = joints_list[:1]
+ with pytest.raises(AssertionError):
+ results_horizontal_flip = random_affine_transform(
+ copy.deepcopy(results_copy))
+
+ results_copy = copy.deepcopy(results)
+ results_copy['mask'] = mask_list[:1]
+ with pytest.raises(AssertionError):
+ results_horizontal_flip = random_affine_transform(
+ copy.deepcopy(results_copy))
+
+ random_affine_transform = BottomUpRandomAffine(30, [0.75, 1.5], 'long', 40)
+ results_affine_transform = random_affine_transform(copy.deepcopy(results))
+ assert results_affine_transform['img'].shape == (512, 384, 3)
+
+ with pytest.raises(ValueError):
+ random_affine_transform = BottomUpRandomAffine(30, [0.75, 1.5],
+ 'short-long', 40)
+ results_affine_transform = random_affine_transform(
+ copy.deepcopy(results))
+
+ # test BottomUpGenerateTarget
+ generate_multi_target = BottomUpGenerateTarget(2, 30)
+ results_generate_multi_target = generate_multi_target(
+ copy.deepcopy(results))
+ assert 'targets' in results_generate_multi_target
+ assert len(results_generate_multi_target['targets']
+ ) == results['ann_info']['num_scales']
+
+ # test BottomUpGetImgSize when W > H
+ get_multi_scale_size = BottomUpGetImgSize([1])
+ results_get_multi_scale_size = get_multi_scale_size(copy.deepcopy(results))
+ assert 'test_scale_factor' in results_get_multi_scale_size['ann_info']
+ assert 'base_size' in results_get_multi_scale_size['ann_info']
+ assert 'center' in results_get_multi_scale_size['ann_info']
+ assert 'scale' in results_get_multi_scale_size['ann_info']
+ assert results_get_multi_scale_size['ann_info']['base_size'][1] == 512
+
+ # test BottomUpResizeAlign
+ transforms = [
+ dict(type='ToTensor'),
+ dict(
+ type='NormalizeTensor',
+ mean=[0.485, 0.456, 0.406],
+ std=[0.229, 0.224, 0.225]),
+ ]
+ resize_align_multi_scale = BottomUpResizeAlign(transforms=transforms)
+ results_copy = copy.deepcopy(results_get_multi_scale_size)
+ results_resize_align_multi_scale = resize_align_multi_scale(results_copy)
+ assert 'aug_data' in results_resize_align_multi_scale['ann_info']
+
+ # test when W < H
+ ann_info['image_size'] = np.array([512, 384])
+ ann_info['heatmap_size'] = np.array([[128, 96], [256, 192]])
+ results = {}
+ results['dataset'] = 'coco'
+ results['image_file'] = osp.join(data_prefix, '000000000785.jpg')
+ results['mask'] = mask_list
+ results['joints'] = joints_list
+ results['ann_info'] = ann_info
+ results['img'] = np.random.rand(640, 425, 3)
+
+ # test HorizontalFlip
+ random_horizontal_flip = BottomUpRandomFlip(flip_prob=1.)
+ results_horizontal_flip = random_horizontal_flip(copy.deepcopy(results))
+ assert _check_flip(results['img'], results_horizontal_flip['img'])
+
+ random_horizontal_flip = BottomUpRandomFlip(flip_prob=0.)
+ results_horizontal_flip = random_horizontal_flip(copy.deepcopy(results))
+ assert (results['img'] == results_horizontal_flip['img']).all()
+
+ results_copy = copy.deepcopy(results)
+ results_copy['mask'] = mask_list[0]
+ with pytest.raises(AssertionError):
+ results_horizontal_flip = random_horizontal_flip(
+ copy.deepcopy(results_copy))
+
+ results_copy = copy.deepcopy(results)
+ results_copy['joints'] = joints_list[0]
+ with pytest.raises(AssertionError):
+ results_horizontal_flip = random_horizontal_flip(
+ copy.deepcopy(results_copy))
+
+ results_copy = copy.deepcopy(results)
+ results_copy['joints'] = joints_list[:1]
+ with pytest.raises(AssertionError):
+ results_horizontal_flip = random_horizontal_flip(
+ copy.deepcopy(results_copy))
+
+ results_copy = copy.deepcopy(results)
+ results_copy['mask'] = mask_list[:1]
+ with pytest.raises(AssertionError):
+ results_horizontal_flip = random_horizontal_flip(
+ copy.deepcopy(results_copy))
+
+ # test TopDownAffine
+ random_affine_transform = BottomUpRandomAffine(30, [0.75, 1.5], 'short', 0)
+ results_affine_transform = random_affine_transform(copy.deepcopy(results))
+ assert results_affine_transform['img'].shape == (384, 512, 3)
+
+ random_affine_transform = BottomUpRandomAffine(30, [0.75, 1.5], 'short',
+ 40)
+ results_affine_transform = random_affine_transform(copy.deepcopy(results))
+ assert results_affine_transform['img'].shape == (384, 512, 3)
+
+ results_copy = copy.deepcopy(results)
+ results_copy['ann_info']['scale_aware_sigma'] = True
+ joints = _get_joints(anno, results_copy['ann_info'], False)
+ results_copy['joints'] = \
+ [joints.copy() for _ in range(results_copy['ann_info']['num_scales'])]
+ results_affine_transform = random_affine_transform(results_copy)
+ assert results_affine_transform['img'].shape == (384, 512, 3)
+
+ results_copy = copy.deepcopy(results)
+ results_copy['mask'] = mask_list[0]
+ with pytest.raises(AssertionError):
+ results_horizontal_flip = random_affine_transform(
+ copy.deepcopy(results_copy))
+
+ results_copy = copy.deepcopy(results)
+ results_copy['joints'] = joints_list[0]
+ with pytest.raises(AssertionError):
+ results_horizontal_flip = random_affine_transform(
+ copy.deepcopy(results_copy))
+
+ results_copy = copy.deepcopy(results)
+ results_copy['joints'] = joints_list[:1]
+ with pytest.raises(AssertionError):
+ results_horizontal_flip = random_affine_transform(
+ copy.deepcopy(results_copy))
+
+ results_copy = copy.deepcopy(results)
+ results_copy['mask'] = mask_list[:1]
+ with pytest.raises(AssertionError):
+ results_horizontal_flip = random_affine_transform(
+ copy.deepcopy(results_copy))
+
+ random_affine_transform = BottomUpRandomAffine(30, [0.75, 1.5], 'long', 40)
+ results_affine_transform = random_affine_transform(copy.deepcopy(results))
+ assert results_affine_transform['img'].shape == (384, 512, 3)
+
+ with pytest.raises(ValueError):
+ random_affine_transform = BottomUpRandomAffine(30, [0.75, 1.5],
+ 'short-long', 40)
+ results_affine_transform = random_affine_transform(
+ copy.deepcopy(results))
+
+ # test BottomUpGenerateTarget
+ generate_multi_target = BottomUpGenerateTarget(2, 30)
+ results_generate_multi_target = generate_multi_target(
+ copy.deepcopy(results))
+ assert 'targets' in results_generate_multi_target
+ assert len(results_generate_multi_target['targets']
+ ) == results['ann_info']['num_scales']
+
+ # test BottomUpGetImgSize when W < H
+ get_multi_scale_size = BottomUpGetImgSize([1])
+ results_get_multi_scale_size = get_multi_scale_size(copy.deepcopy(results))
+ assert 'test_scale_factor' in results_get_multi_scale_size['ann_info']
+ assert 'base_size' in results_get_multi_scale_size['ann_info']
+ assert 'center' in results_get_multi_scale_size['ann_info']
+ assert 'scale' in results_get_multi_scale_size['ann_info']
+ assert results_get_multi_scale_size['ann_info']['base_size'][0] == 512
+
+
+def test_BottomUpGenerateHeatmapTarget():
+
+ data_prefix = 'tests/data/coco/'
+ ann_file = osp.join(data_prefix, 'test_coco.json')
+ coco = COCO(ann_file)
+
+ ann_info = {}
+ ann_info['heatmap_size'] = np.array([128, 256])
+ ann_info['num_joints'] = 17
+ ann_info['num_scales'] = 2
+ ann_info['scale_aware_sigma'] = False
+
+ ann_ids = coco.getAnnIds(785)
+ anno = coco.loadAnns(ann_ids)
+ mask = _get_mask(coco, anno, 785)
+
+ anno = [
+ obj for obj in anno if obj['iscrowd'] == 0 or obj['num_keypoints'] > 0
+ ]
+ joints = _get_joints(anno, ann_info, False)
+
+ mask_list = [mask.copy() for _ in range(ann_info['num_scales'])]
+ joints_list = [joints.copy() for _ in range(ann_info['num_scales'])]
+
+ results = {}
+ results['dataset'] = 'coco'
+ results['image_file'] = osp.join(data_prefix, '000000000785.jpg')
+ results['mask'] = mask_list
+ results['joints'] = joints_list
+ results['ann_info'] = ann_info
+
+ generate_heatmap_target = BottomUpGenerateHeatmapTarget(2)
+ results_generate_heatmap_target = generate_heatmap_target(results)
+ assert 'target' in results_generate_heatmap_target
+ assert len(results_generate_heatmap_target['target']
+ ) == results['ann_info']['num_scales']
+
+
+def test_BottomUpGeneratePAFTarget():
+
+ ann_info = {}
+ ann_info['skeleton'] = [[0, 1], [2, 3]]
+ ann_info['heatmap_size'] = np.array([5])
+ ann_info['num_joints'] = 4
+ ann_info['num_scales'] = 1
+
+ mask = np.ones((5, 5), dtype=bool)
+ joints = np.array([[[1, 1, 2], [3, 3, 2], [0, 0, 0], [0, 0, 0]],
+ [[1, 3, 2], [3, 1, 2], [0, 0, 0], [0, 0, 0]]])
+
+ mask_list = [mask.copy() for _ in range(ann_info['num_scales'])]
+ joints_list = [joints.copy() for _ in range(ann_info['num_scales'])]
+
+ results = {}
+ results['dataset'] = 'coco'
+ results['mask'] = mask_list
+ results['joints'] = joints_list
+ results['ann_info'] = ann_info
+
+ generate_paf_target = BottomUpGeneratePAFTarget(1)
+ results_generate_paf_target = generate_paf_target(results)
+ sqrt = np.sqrt(2) / 2
+ assert (results_generate_paf_target['target'] == np.array(
+ [[[sqrt, sqrt, 0, sqrt, sqrt], [sqrt, sqrt, sqrt, sqrt, sqrt],
+ [0, sqrt, sqrt, sqrt, 0], [sqrt, sqrt, sqrt, sqrt, sqrt],
+ [sqrt, sqrt, 0, sqrt, sqrt]],
+ [[sqrt, sqrt, 0, -sqrt, -sqrt], [sqrt, sqrt, 0, -sqrt, -sqrt],
+ [0, 0, 0, 0, 0], [-sqrt, -sqrt, 0, sqrt, sqrt],
+ [-sqrt, -sqrt, 0, sqrt, sqrt]],
+ [[0, 0, 0, 0, 0], [0, 0, 0, 0, 0], [0, 0, 0, 0, 0], [0, 0, 0, 0, 0],
+ [0, 0, 0, 0, 0]],
+ [[0, 0, 0, 0, 0], [0, 0, 0, 0, 0], [0, 0, 0, 0, 0], [0, 0, 0, 0, 0],
+ [0, 0, 0, 0, 0]]],
+ dtype=np.float32)).all()
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_pipelines/test_hand_transform.py b/engine/pose_estimation/third-party/ViTPose/tests/test_pipelines/test_hand_transform.py
new file mode 100644
index 0000000..2225b87
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_pipelines/test_hand_transform.py
@@ -0,0 +1,68 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import copy
+
+import numpy as np
+
+from mmpose.datasets.pipelines import Compose
+
+
+def _check_flip(origin_imgs, result_imgs):
+ """Check if the origin_imgs are flipped correctly."""
+ h, w, c = origin_imgs.shape
+ for i in range(h):
+ for j in range(w):
+ for k in range(c):
+ if result_imgs[i, j, k] != origin_imgs[i, w - 1 - j, k]:
+ return False
+ return True
+
+
+def get_sample_data():
+ ann_info = {}
+ ann_info['image_size'] = np.array([256, 256])
+ ann_info['heatmap_size'] = np.array([64, 64, 64])
+ ann_info['heatmap3d_depth_bound'] = 400.0
+ ann_info['heatmap_size_root'] = 64
+ ann_info['root_depth_bound'] = 400.0
+ ann_info['num_joints'] = 42
+ ann_info['joint_weights'] = np.ones((ann_info['num_joints'], 1),
+ dtype=np.float32)
+ ann_info['use_different_joint_weights'] = False
+ ann_info['flip_pairs'] = [[i, 21 + i] for i in range(21)]
+ ann_info['inference_channel'] = list(range(42))
+ ann_info['num_output_channels'] = 42
+ ann_info['dataset_channel'] = list(range(42))
+
+ results = {
+ 'image_file': 'tests/data/interhand2.6m/image69148.jpg',
+ 'center': np.asarray([200, 200], dtype=np.float32),
+ 'scale': 1.0,
+ 'rotation': 0,
+ 'joints_3d': np.zeros([42, 3], dtype=np.float32),
+ 'joints_3d_visible': np.ones([42, 3], dtype=np.float32),
+ 'hand_type': np.asarray([1, 0], dtype=np.float32),
+ 'hand_type_valid': 1,
+ 'rel_root_depth': 50.0,
+ 'rel_root_valid': 1,
+ 'ann_info': ann_info
+ }
+ return results
+
+
+def test_hand_transforms():
+ results = get_sample_data()
+
+ # load image
+ pipeline = Compose([dict(type='LoadImageFromFile')])
+ results = pipeline(results)
+
+ # test random flip
+ pipeline = Compose([dict(type='HandRandomFlip', flip_prob=1)])
+ results_flip = pipeline(copy.deepcopy(results))
+ assert _check_flip(results['img'], results_flip['img'])
+
+ # test root depth target generation
+ pipeline = Compose([dict(type='HandGenerateRelDepthTarget')])
+ results_depth = pipeline(copy.deepcopy(results))
+ assert results_depth['target'].shape == (1, )
+ assert results_depth['target_weight'].shape == (1, )
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_pipelines/test_mesh_pipelines.py b/engine/pose_estimation/third-party/ViTPose/tests/test_pipelines/test_mesh_pipelines.py
new file mode 100644
index 0000000..9c2c8d1
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_pipelines/test_mesh_pipelines.py
@@ -0,0 +1,255 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import copy
+import os
+
+import numpy as np
+import torch
+from numpy.testing import assert_array_almost_equal
+
+from mmpose.datasets.pipelines import (Collect, IUVToTensor, LoadImageFromFile,
+ LoadIUVFromFile, MeshAffine,
+ MeshGetRandomScaleRotation,
+ MeshRandomChannelNoise, MeshRandomFlip,
+ NormalizeTensor, ToTensor)
+
+
+def _check_keys_contain(result_keys, target_keys):
+ """Check if all elements in target_keys is in result_keys."""
+ return set(target_keys).issubset(set(result_keys))
+
+
+def _check_flip(origin_imgs, result_imgs):
+ """Check if the origin_imgs are flipped correctly."""
+ h, w, c = origin_imgs.shape
+ for i in range(h):
+ for j in range(w):
+ for k in range(c):
+ if result_imgs[i, j, k] != origin_imgs[i, w - 1 - j, k]:
+ return False
+ return True
+
+
+def _check_rot90(origin_imgs, result_imgs):
+ if origin_imgs.shape[0] == result_imgs.shape[1] and \
+ origin_imgs.shape[1] == result_imgs.shape[0]:
+ return True
+ else:
+ return False
+
+
+def _check_normalize(origin_imgs, result_imgs, norm_cfg):
+ """Check if the origin_imgs are normalized correctly into result_imgs in a
+ given norm_cfg."""
+ target_imgs = result_imgs.copy()
+ for i in range(3):
+ target_imgs[i] *= norm_cfg['std'][i]
+ target_imgs[i] += norm_cfg['mean'][i]
+ assert_array_almost_equal(origin_imgs, target_imgs, decimal=4)
+
+
+def _box2cs(box, image_size):
+ x, y, w, h = box[:4]
+
+ aspect_ratio = 1. * image_size[0] / image_size[1]
+ center = np.zeros((2), dtype=np.float32)
+ center[0] = x + w * 0.5
+ center[1] = y + h * 0.5
+
+ if w > aspect_ratio * h:
+ h = w * 1.0 / aspect_ratio
+ elif w < aspect_ratio * h:
+ w = h * aspect_ratio
+ scale = np.array([w * 1.0 / 200.0, h * 1.0 / 200.0], dtype=np.float32)
+ scale = scale * 1.25
+ return center, scale
+
+
+def _load_test_data():
+ data_cfg = dict(
+ image_size=[256, 256],
+ iuv_size=[64, 64],
+ num_joints=24,
+ use_IUV=True,
+ uv_type='BF')
+ ann_file = 'tests/data/h36m/test_h36m.npz'
+ img_prefix = 'tests/data/h36m'
+ index = 0
+
+ ann_info = dict(image_size=np.array(data_cfg['image_size']))
+ ann_info['iuv_size'] = np.array(data_cfg['iuv_size'])
+ ann_info['num_joints'] = data_cfg['num_joints']
+ ann_info['flip_pairs'] = [[0, 5], [1, 4], [2, 3], [6, 11], [7, 10], [8, 9],
+ [20, 21], [22, 23]]
+ ann_info['use_different_joint_weights'] = False
+ ann_info['joint_weights'] = \
+ np.ones(ann_info['num_joints'], dtype=np.float32
+ ).reshape(ann_info['num_joints'], 1)
+ ann_info['uv_type'] = data_cfg['uv_type']
+ ann_info['use_IUV'] = data_cfg['use_IUV']
+ uv_type = ann_info['uv_type']
+ iuv_prefix = os.path.join(img_prefix, f'{uv_type}_IUV_gt')
+
+ ann_data = np.load(ann_file)
+
+ results = dict(ann_info=ann_info)
+ results['rotation'] = 0
+ results['image_file'] = os.path.join(img_prefix,
+ ann_data['imgname'][index])
+ scale = ann_data['scale'][index]
+ results['scale'] = np.array([scale, scale]).astype(np.float32)
+ results['center'] = ann_data['center'][index].astype(np.float32)
+
+ # Get gt 2D joints, if available
+ if 'part' in ann_data.keys():
+ keypoints = ann_data['part'][index].astype(np.float32)
+ results['joints_2d'] = keypoints[:, :2]
+ results['joints_2d_visible'] = keypoints[:, -1][:, np.newaxis]
+ else:
+ results['joints_2d'] = np.zeros((24, 2), dtype=np.float32)
+ results['joints_2d_visible'] = np.zeros((24, 1), dtype=np.float32)
+
+ # Get gt 3D joints, if available
+ if 'S' in ann_data.keys():
+ joints_3d = ann_data['S'][index].astype(np.float32)
+ results['joints_3d'] = joints_3d[:, :3]
+ results['joints_3d_visible'] = joints_3d[:, -1][:, np.newaxis]
+ else:
+ results['joints_3d'] = np.zeros((24, 3), dtype=np.float32)
+ results['joints_3d_visible'] = np.zeros((24, 1), dtype=np.float32)
+
+ # Get gt SMPL parameters, if available
+ if 'pose' in ann_data.keys() and 'shape' in ann_data.keys():
+ results['pose'] = ann_data['pose'][index].astype(np.float32)
+ results['beta'] = ann_data['shape'][index].astype(np.float32)
+ results['has_smpl'] = 1
+ else:
+ results['pose'] = np.zeros(72, dtype=np.float32)
+ results['beta'] = np.zeros(10, dtype=np.float32)
+ results['has_smpl'] = 0
+
+ # Get gender data, if available
+ if 'gender' in ann_data.keys():
+ gender = ann_data['gender'][index]
+ results['gender'] = 0 if str(gender) == 'm' else 1
+ else:
+ results['gender'] = -1
+
+ # Get IUV image, if available
+ if 'iuv_names' in ann_data.keys():
+ results['iuv_file'] = os.path.join(iuv_prefix,
+ ann_data['iuv_names'][index])
+ results['has_iuv'] = results['has_smpl']
+ else:
+ results['iuv_file'] = ''
+ results['has_iuv'] = 0
+
+ return copy.deepcopy(results)
+
+
+def test_mesh_pipeline():
+ # load data
+ results = _load_test_data()
+
+ # data_prefix = 'tests/data/coco/'
+ # ann_file = osp.join(data_prefix, 'test_coco.json')
+ # coco = COCO(ann_file)
+ #
+ # results = dict(image_file=osp.join(data_prefix, '000000000785.jpg'))
+
+ # test loading image
+ transform = LoadImageFromFile()
+ results = transform(copy.deepcopy(results))
+ assert results['img'].shape == (1002, 1000, 3)
+
+ # test loading densepose IUV image without GT iuv image
+ transform = LoadIUVFromFile()
+ results_no_iuv = copy.deepcopy(results)
+ results_no_iuv['has_iuv'] = 0
+ results_no_iuv = transform(results_no_iuv)
+ assert results_no_iuv['iuv'] is None
+
+ # test loading densepose IUV image
+ results = transform(results)
+ assert results['iuv'].shape == (1002, 1000, 3)
+ assert results['iuv'][:, :, 0].max() <= 1
+
+ # test flip
+ random_flip = MeshRandomFlip(flip_prob=1.)
+ results_flip = random_flip(copy.deepcopy(results))
+ assert _check_flip(results['img'], results_flip['img'])
+ flip_iuv = results_flip['iuv']
+ flip_iuv[:, :, 1] = 255 - flip_iuv[:, :, 1]
+ assert _check_flip(results['iuv'], flip_iuv)
+ results = results_flip
+
+ # test flip without IUV image
+ results_no_iuv = random_flip(copy.deepcopy(results_no_iuv))
+ assert results_no_iuv['iuv'] is None
+
+ # test random scale and rotation
+ random_scale_rotation = MeshGetRandomScaleRotation()
+ results = random_scale_rotation(results)
+
+ # test affine
+ affine_transform = MeshAffine()
+ results_affine = affine_transform(copy.deepcopy(results))
+ assert results_affine['img'].shape == (256, 256, 3)
+ assert results_affine['iuv'].shape == (64, 64, 3)
+ results = results_affine
+
+ # test affine without IUV image
+ results_no_iuv['rotation'] = 30
+ results_no_iuv = affine_transform(copy.deepcopy(results_no_iuv))
+ assert results_no_iuv['iuv'] is None
+
+ # test channel noise
+ random_noise = MeshRandomChannelNoise()
+ results_noise = random_noise(copy.deepcopy(results))
+ results = results_noise
+
+ # transfer image to tensor
+ to_tensor = ToTensor()
+ results_tensor = to_tensor(copy.deepcopy(results))
+ assert isinstance(results_tensor['img'], torch.Tensor)
+ assert results_tensor['img'].shape == torch.Size([3, 256, 256])
+
+ # transfer IUV image to tensor
+ iuv_to_tensor = IUVToTensor()
+ results_tensor = iuv_to_tensor(results_tensor)
+ assert isinstance(results_tensor['part_index'], torch.LongTensor)
+ assert results_tensor['part_index'].shape == torch.Size([1, 64, 64])
+ max_I = results_tensor['part_index'].max().item()
+ assert (max_I == 0 or max_I == 1)
+ assert isinstance(results_tensor['uv_coordinates'], torch.FloatTensor)
+ assert results_tensor['uv_coordinates'].shape == torch.Size([2, 64, 64])
+
+ # transfer IUV image to tensor without GT IUV image
+ results_no_iuv = iuv_to_tensor(results_no_iuv)
+ assert isinstance(results_no_iuv['part_index'], torch.LongTensor)
+ assert results_no_iuv['part_index'].shape == torch.Size([1, 64, 64])
+ max_I = results_no_iuv['part_index'].max().item()
+ assert (max_I == 0)
+ assert isinstance(results_no_iuv['uv_coordinates'], torch.FloatTensor)
+ assert results_no_iuv['uv_coordinates'].shape == torch.Size([2, 64, 64])
+
+ # test norm
+ norm_cfg = {}
+ norm_cfg['mean'] = [0.485, 0.456, 0.406]
+ norm_cfg['std'] = [0.229, 0.224, 0.225]
+ normalize = NormalizeTensor(mean=norm_cfg['mean'], std=norm_cfg['std'])
+
+ results_normalize = normalize(copy.deepcopy(results_tensor))
+ _check_normalize(results_tensor['img'].data.numpy(),
+ results_normalize['img'].data.numpy(), norm_cfg)
+
+ # test collect
+ collect = Collect(
+ keys=[
+ 'img', 'joints_2d', 'joints_2d_visible', 'joints_3d',
+ 'joints_3d_visible', 'pose', 'beta', 'part_index', 'uv_coordinates'
+ ],
+ meta_keys=['image_file', 'center', 'scale', 'rotation', 'iuv_file'])
+ results_final = collect(results_normalize)
+
+ assert 'img_size' not in results_final['img_metas'].data
+ assert 'image_file' in results_final['img_metas'].data
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_pipelines/test_pose3d_transform.py b/engine/pose_estimation/third-party/ViTPose/tests/test_pipelines/test_pose3d_transform.py
new file mode 100644
index 0000000..b6a52d9
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_pipelines/test_pose3d_transform.py
@@ -0,0 +1,336 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import copy
+import os.path as osp
+import tempfile
+
+import mmcv
+import numpy as np
+import pytest
+from numpy.testing import assert_array_almost_equal
+
+from mmpose.core import SimpleCamera
+from mmpose.datasets.pipelines import Compose
+
+H36M_JOINT_IDX = [14, 2, 1, 0, 3, 4, 5, 16, 12, 17, 18, 9, 10, 11, 8, 7, 6]
+
+
+def get_data_sample():
+
+ def _parse_h36m_imgname(imgname):
+ """Parse imgname to get information of subject, action and camera.
+
+ A typical h36m image filename is like:
+ S1_Directions_1.54138969_000001.jpg
+ """
+ subj, rest = osp.basename(imgname).split('_', 1)
+ action, rest = rest.split('.', 1)
+ camera, rest = rest.split('_', 1)
+ return subj, action, camera
+
+ ann_flle = 'tests/data/h36m/test_h36m.npz'
+ camera_param_file = 'tests/data/h36m/cameras.pkl'
+
+ data = np.load(ann_flle)
+ cameras = mmcv.load(camera_param_file)
+
+ _imgnames = data['imgname']
+ _joints_2d = data['part'][:, H36M_JOINT_IDX].astype(np.float32)
+ _joints_3d = data['S'][:, H36M_JOINT_IDX].astype(np.float32)
+ _centers = data['center'].astype(np.float32)
+ _scales = data['scale'].astype(np.float32)
+
+ frame_ids = [0]
+ target_frame_id = 0
+
+ results = {
+ 'frame_ids': frame_ids,
+ 'target_frame_id': target_frame_id,
+ 'input_2d': _joints_2d[frame_ids, :, :2],
+ 'input_2d_visible': _joints_2d[frame_ids, :, -1:],
+ 'input_3d': _joints_3d[frame_ids, :, :3],
+ 'input_3d_visible': _joints_3d[frame_ids, :, -1:],
+ 'target': _joints_3d[target_frame_id, :, :3],
+ 'target_visible': _joints_3d[target_frame_id, :, -1:],
+ 'imgnames': _imgnames[frame_ids],
+ 'scales': _scales[frame_ids],
+ 'centers': _centers[frame_ids],
+ }
+
+ # add camera parameters
+ subj, _, camera = _parse_h36m_imgname(_imgnames[frame_ids[0]])
+ results['camera_param'] = cameras[(subj, camera)]
+
+ # add image size
+ results['image_width'] = results['camera_param']['w']
+ results['image_height'] = results['camera_param']['h']
+
+ # add ann_info
+ ann_info = {}
+ ann_info['num_joints'] = 17
+ ann_info['joint_weights'] = np.full(17, 1.0, dtype=np.float32)
+ ann_info['flip_pairs'] = [[1, 4], [2, 5], [3, 6], [11, 14], [12, 15],
+ [13, 16]]
+ ann_info['upper_body_ids'] = (0, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16)
+ ann_info['lower_body_ids'] = (1, 2, 3, 4, 5, 6)
+ ann_info['use_different_joint_weights'] = False
+
+ results['ann_info'] = ann_info
+
+ return results
+
+
+def test_joint_transforms():
+ results = get_data_sample()
+
+ mean = np.random.rand(16, 3).astype(np.float32)
+ std = np.random.rand(16, 3).astype(np.float32) + 1e-6
+
+ pipeline = [
+ dict(
+ type='RelativeJointRandomFlip',
+ item='target',
+ flip_cfg=dict(center_mode='root', center_index=0),
+ visible_item='target_visible',
+ flip_prob=1.,
+ flip_camera=True),
+ dict(
+ type='GetRootCenteredPose',
+ item='target',
+ root_index=0,
+ root_name='global_position',
+ remove_root=True),
+ dict(
+ type='NormalizeJointCoordinate', item='target', mean=mean,
+ std=std),
+ dict(type='PoseSequenceToTensor', item='target'),
+ dict(
+ type='ImageCoordinateNormalization',
+ item='input_2d',
+ norm_camera=True),
+ dict(type='CollectCameraIntrinsics'),
+ dict(
+ type='Collect',
+ keys=[('input_2d', 'input'), ('target', 'output'), 'flip_pairs',
+ 'intrinsics'],
+ meta_name='metas',
+ meta_keys=['camera_param'])
+ ]
+
+ pipeline = Compose(pipeline)
+ output = pipeline(copy.deepcopy(results))
+
+ # test transformation of target
+ joints_0 = results['target']
+ joints_1 = output['output'].numpy()
+ # manually do transformations
+ flip_pairs = output['flip_pairs']
+ _joints_0_flipped = joints_0.copy()
+ for _l, _r in flip_pairs:
+ _joints_0_flipped[..., _l, :] = joints_0[..., _r, :]
+ _joints_0_flipped[..., _r, :] = joints_0[..., _l, :]
+ _joints_0_flipped[...,
+ 0] = 2 * joints_0[..., 0:1, 0] - _joints_0_flipped[...,
+ 0]
+ joints_0 = _joints_0_flipped
+ joints_0 = (joints_0[..., 1:, :] - joints_0[..., 0:1, :] - mean) / std
+ # convert to [K*C, T]
+ joints_0 = joints_0.reshape(-1)[..., None]
+ np.testing.assert_array_almost_equal(joints_0, joints_1)
+
+ # test transformation of input
+ joints_0 = results['input_2d']
+ joints_1 = output['input']
+ # manually do transformations
+ center = np.array(
+ [0.5 * results['image_width'], 0.5 * results['image_height']],
+ dtype=np.float32)
+ scale = np.array(0.5 * results['image_width'], dtype=np.float32)
+ joints_0 = (joints_0 - center) / scale
+ np.testing.assert_array_almost_equal(joints_0, joints_1)
+
+ # test transformation of camera parameters
+ camera_param_0 = results['camera_param']
+ camera_param_1 = output['metas'].data['camera_param']
+ # manually flip and normalization
+ camera_param_0['c'][0] *= -1
+ camera_param_0['p'][0] *= -1
+ camera_param_0['c'] = (camera_param_0['c'] -
+ np.array(center)[:, None]) / scale
+ camera_param_0['f'] = camera_param_0['f'] / scale
+ np.testing.assert_array_almost_equal(camera_param_0['c'],
+ camera_param_1['c'])
+ np.testing.assert_array_almost_equal(camera_param_0['f'],
+ camera_param_1['f'])
+
+ # test CollectCameraIntrinsics
+ intrinsics_0 = np.concatenate([
+ results['camera_param']['f'].reshape(2),
+ results['camera_param']['c'].reshape(2),
+ results['camera_param']['k'].reshape(3),
+ results['camera_param']['p'].reshape(2)
+ ])
+ intrinsics_1 = output['intrinsics']
+ np.testing.assert_array_almost_equal(intrinsics_0, intrinsics_1)
+
+ # test load mean/std from file
+ with tempfile.TemporaryDirectory() as tmpdir:
+ norm_param = {'mean': mean, 'std': std}
+ norm_param_file = osp.join(tmpdir, 'norm_param.pkl')
+ mmcv.dump(norm_param, norm_param_file)
+
+ pipeline = [
+ dict(
+ type='NormalizeJointCoordinate',
+ item='target',
+ norm_param_file=norm_param_file),
+ ]
+ pipeline = Compose(pipeline)
+
+
+def test_camera_projection():
+ results = get_data_sample()
+ pipeline_1 = [
+ dict(
+ type='CameraProjection',
+ item='input_3d',
+ output_name='input_3d_w',
+ camera_type='SimpleCamera',
+ mode='camera_to_world'),
+ dict(
+ type='CameraProjection',
+ item='input_3d_w',
+ output_name='input_3d_wp',
+ camera_type='SimpleCamera',
+ mode='world_to_pixel'),
+ dict(
+ type='CameraProjection',
+ item='input_3d',
+ output_name='input_3d_p',
+ camera_type='SimpleCamera',
+ mode='camera_to_pixel'),
+ dict(type='Collect', keys=['input_3d_wp', 'input_3d_p'], meta_keys=[])
+ ]
+ camera_param = results['camera_param'].copy()
+ camera_param['K'] = np.concatenate(
+ (np.diagflat(camera_param['f']), camera_param['c']), axis=-1)
+ pipeline_2 = [
+ dict(
+ type='CameraProjection',
+ item='input_3d',
+ output_name='input_3d_w',
+ camera_type='SimpleCamera',
+ camera_param=camera_param,
+ mode='camera_to_world'),
+ dict(
+ type='CameraProjection',
+ item='input_3d_w',
+ output_name='input_3d_wp',
+ camera_type='SimpleCamera',
+ camera_param=camera_param,
+ mode='world_to_pixel'),
+ dict(
+ type='CameraProjection',
+ item='input_3d',
+ output_name='input_3d_p',
+ camera_type='SimpleCamera',
+ camera_param=camera_param,
+ mode='camera_to_pixel'),
+ dict(
+ type='CameraProjection',
+ item='input_3d_w',
+ output_name='input_3d_wc',
+ camera_type='SimpleCamera',
+ camera_param=camera_param,
+ mode='world_to_camera'),
+ dict(
+ type='Collect',
+ keys=['input_3d_wp', 'input_3d_p', 'input_2d'],
+ meta_keys=[])
+ ]
+
+ output1 = Compose(pipeline_1)(results)
+ output2 = Compose(pipeline_2)(results)
+
+ np.testing.assert_allclose(
+ output1['input_3d_wp'], output1['input_3d_p'], rtol=1e-6)
+
+ np.testing.assert_allclose(
+ output2['input_3d_wp'], output2['input_3d_p'], rtol=1e-6)
+
+ np.testing.assert_allclose(
+ output2['input_3d_p'], output2['input_2d'], rtol=1e-3, atol=1e-1)
+
+ # test invalid camera parameters
+ with pytest.raises(ValueError):
+ # missing intrinsic parameters
+ camera_param_wo_intrinsic = camera_param.copy()
+ camera_param_wo_intrinsic.pop('K')
+ camera_param_wo_intrinsic.pop('f')
+ camera_param_wo_intrinsic.pop('c')
+ _ = Compose([
+ dict(
+ type='CameraProjection',
+ item='input_3d',
+ camera_type='SimpleCamera',
+ camera_param=camera_param_wo_intrinsic,
+ mode='camera_to_pixel')
+ ])
+
+ with pytest.raises(ValueError):
+ # invalid mode
+ _ = Compose([
+ dict(
+ type='CameraProjection',
+ item='input_3d',
+ camera_type='SimpleCamera',
+ camera_param=camera_param,
+ mode='dummy')
+ ])
+
+ # test camera without undistortion
+ camera_param_wo_undistortion = camera_param.copy()
+ camera_param_wo_undistortion.pop('k')
+ camera_param_wo_undistortion.pop('p')
+ _ = Compose([
+ dict(
+ type='CameraProjection',
+ item='input_3d',
+ camera_type='SimpleCamera',
+ camera_param=camera_param_wo_undistortion,
+ mode='camera_to_pixel')
+ ])
+
+ # test pixel to camera transformation
+ camera = SimpleCamera(camera_param_wo_undistortion)
+ kpt_camera = np.random.rand(14, 3)
+ kpt_pixel = camera.camera_to_pixel(kpt_camera)
+ _kpt_camera = camera.pixel_to_camera(
+ np.concatenate([kpt_pixel, kpt_camera[:, [2]]], -1))
+ assert_array_almost_equal(_kpt_camera, kpt_camera, decimal=4)
+
+
+def test_3d_heatmap_generation():
+ ann_info = dict(
+ image_size=np.array([256, 256]),
+ heatmap_size=np.array([64, 64, 64]),
+ heatmap3d_depth_bound=400.0,
+ num_joints=17,
+ joint_weights=np.ones((17, 1), dtype=np.float32),
+ use_different_joint_weights=False)
+
+ results = dict(
+ joints_3d=np.zeros([17, 3]),
+ joints_3d_visible=np.ones([17, 3]),
+ ann_info=ann_info)
+
+ pipeline = Compose([dict(type='Generate3DHeatmapTarget')])
+ results_3d = pipeline(results)
+ assert results_3d['target'].shape == (17, 64, 64, 64)
+ assert results_3d['target_weight'].shape == (17, 1)
+
+ # test joint_indices
+ pipeline = Compose(
+ [dict(type='Generate3DHeatmapTarget', joint_indices=[0, 8, 16])])
+ results_3d = pipeline(results)
+ assert results_3d['target'].shape == (3, 64, 64, 64)
+ assert results_3d['target_weight'].shape == (3, 1)
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_pipelines/test_shared_transform.py b/engine/pose_estimation/third-party/ViTPose/tests/test_pipelines/test_shared_transform.py
new file mode 100644
index 0000000..684a103
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_pipelines/test_shared_transform.py
@@ -0,0 +1,218 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os.path as osp
+
+import numpy as np
+import pytest
+from mmcv import bgr2rgb, build_from_cfg
+
+from mmpose.datasets import PIPELINES
+from mmpose.datasets.pipelines import Compose
+
+
+def check_keys_equal(result_keys, target_keys):
+ """Check if all elements in target_keys is in result_keys."""
+ return set(target_keys) == set(result_keys)
+
+
+def check_keys_contain(result_keys, target_keys):
+ """Check if elements in target_keys is in result_keys."""
+ return set(target_keys).issubset(set(result_keys))
+
+
+def test_compose():
+ with pytest.raises(TypeError):
+ # transform must be callable or a dict
+ Compose('LoadImageFromFile')
+
+ target_keys = ['img', 'img_rename', 'img_metas']
+
+ # test Compose given a data pipeline
+ img = np.random.randn(256, 256, 3)
+ results = dict(img=img, img_file='test_image.png')
+ test_pipeline = [
+ dict(
+ type='Collect',
+ keys=['img', ('img', 'img_rename')],
+ meta_keys=['img_file'])
+ ]
+ compose = Compose(test_pipeline)
+ compose_results = compose(results)
+ assert check_keys_equal(compose_results.keys(), target_keys)
+ assert check_keys_equal(compose_results['img_metas'].data.keys(),
+ ['img_file'])
+
+ # test Compose when forward data is None
+ results = None
+
+ class ExamplePipeline:
+
+ def __call__(self, results):
+ return None
+
+ nonePipeline = ExamplePipeline()
+ test_pipeline = [nonePipeline]
+ compose = Compose(test_pipeline)
+ compose_results = compose(results)
+ assert compose_results is None
+
+ assert repr(compose) == compose.__class__.__name__ + \
+ f'(\n {nonePipeline}\n)'
+
+
+def test_load_image_from_file():
+ # Define simple pipeline
+ load = dict(type='LoadImageFromFile')
+ load = build_from_cfg(load, PIPELINES)
+
+ data_prefix = 'tests/data/coco/'
+ image_file = osp.join(data_prefix, '00000000078.jpg')
+ results = dict(image_file=image_file)
+
+ # load an image that doesn't exist
+ with pytest.raises(FileNotFoundError):
+ results = load(results)
+
+ # mormal loading
+ image_file = osp.join(data_prefix, '000000000785.jpg')
+ results = dict(image_file=image_file)
+ results = load(results)
+ assert results['img'].shape == (425, 640, 3)
+
+ # load a single image from a list
+ image_file = [osp.join(data_prefix, '000000000785.jpg')]
+ results = dict(image_file=image_file)
+ results = load(results)
+ assert len(results['img']) == 1
+
+ # test loading multi images from a list
+ image_file = [
+ osp.join(data_prefix, '000000000785.jpg'),
+ osp.join(data_prefix, '00000004008.jpg'),
+ ]
+ results = dict(image_file=image_file)
+
+ with pytest.raises(FileNotFoundError):
+ results = load(results)
+
+ image_file = [
+ osp.join(data_prefix, '000000000785.jpg'),
+ osp.join(data_prefix, '000000040083.jpg'),
+ ]
+ results = dict(image_file=image_file)
+
+ results = load(results)
+ assert len(results['img']) == 2
+
+ # manually set image outside the pipeline
+ img = np.random.randint(0, 255, (32, 32, 3), dtype=np.uint8)
+ results = load(dict(img=img))
+ np.testing.assert_equal(results['img'], bgr2rgb(img))
+
+ imgs = np.random.randint(0, 255, (2, 32, 32, 3), dtype=np.uint8)
+ desired = np.concatenate([bgr2rgb(img) for img in imgs], axis=0)
+ results = load(dict(img=imgs))
+ np.testing.assert_equal(results['img'], desired)
+
+ # neither 'image_file' or valid 'img' is given
+ results = dict()
+ with pytest.raises(KeyError):
+ _ = load(results)
+
+ results = dict(img=np.random.randint(0, 255, (32, 32), dtype=np.uint8))
+ with pytest.raises(ValueError):
+ _ = load(results)
+
+
+def test_albu_transform():
+ data_prefix = 'tests/data/coco/'
+ results = dict(image_file=osp.join(data_prefix, '000000000785.jpg'))
+
+ # Define simple pipeline
+ load = dict(type='LoadImageFromFile')
+ load = build_from_cfg(load, PIPELINES)
+
+ albu_transform = dict(
+ type='Albumentation',
+ transforms=[
+ dict(type='RandomBrightnessContrast', p=0.2),
+ dict(type='ToFloat')
+ ])
+ albu_transform = build_from_cfg(albu_transform, PIPELINES)
+
+ # Execute transforms
+ results = load(results)
+
+ results = albu_transform(results)
+
+ assert results['img'].dtype == np.float32
+
+
+def test_photometric_distortion_transform():
+ data_prefix = 'tests/data/coco/'
+ results = dict(image_file=osp.join(data_prefix, '000000000785.jpg'))
+
+ # Define simple pipeline
+ load = dict(type='LoadImageFromFile')
+ load = build_from_cfg(load, PIPELINES)
+
+ photo_transform = dict(type='PhotometricDistortion')
+ photo_transform = build_from_cfg(photo_transform, PIPELINES)
+
+ # Execute transforms
+ results = load(results)
+
+ results = photo_transform(results)
+
+ assert results['img'].dtype == np.uint8
+
+
+def test_multitask_gather():
+ ann_info = dict(
+ image_size=np.array([256, 256]),
+ heatmap_size=np.array([64, 64]),
+ num_joints=17,
+ joint_weights=np.ones((17, 1), dtype=np.float32),
+ use_different_joint_weights=False)
+
+ results = dict(
+ joints_3d=np.zeros([17, 3]),
+ joints_3d_visible=np.ones([17, 3]),
+ ann_info=ann_info)
+
+ pipeline_list = [[dict(type='TopDownGenerateTarget', sigma=2)],
+ [dict(type='TopDownGenerateTargetRegression')]]
+ pipeline = dict(
+ type='MultitaskGatherTarget',
+ pipeline_list=pipeline_list,
+ pipeline_indices=[0, 1, 0],
+ )
+ pipeline = build_from_cfg(pipeline, PIPELINES)
+
+ results = pipeline(results)
+ target = results['target']
+ target_weight = results['target_weight']
+ assert isinstance(target, list)
+ assert isinstance(target_weight, list)
+ assert target[0].shape == (17, 64, 64)
+ assert target_weight[0].shape == (17, 1)
+ assert target[1].shape == (17, 2)
+ assert target_weight[1].shape == (17, 2)
+ assert target[2].shape == (17, 64, 64)
+ assert target_weight[2].shape == (17, 1)
+
+
+def test_rename_keys():
+ results = dict(
+ joints_3d=np.ones([17, 3]), joints_3d_visible=np.ones([17, 3]))
+ pipeline = dict(
+ type='RenameKeys',
+ key_pairs=[('joints_3d', 'target'),
+ ('joints_3d_visible', 'target_weight')])
+ pipeline = build_from_cfg(pipeline, PIPELINES)
+ results = pipeline(results)
+ assert 'joints_3d' not in results
+ assert 'joints_3d_visible' not in results
+ assert 'target' in results
+ assert 'target_weight' in results
+ assert results['target'].shape == (17, 3)
+ assert results['target_weight'].shape == (17, 3)
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_pipelines/test_top_down_pipelines.py b/engine/pose_estimation/third-party/ViTPose/tests/test_pipelines/test_top_down_pipelines.py
new file mode 100644
index 0000000..f4ca1fb
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_pipelines/test_top_down_pipelines.py
@@ -0,0 +1,243 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import copy
+import os.path as osp
+
+import numpy as np
+import torch
+from numpy.testing import assert_array_almost_equal
+from xtcocotools.coco import COCO
+
+from mmpose.datasets.pipelines import (Collect, LoadImageFromFile,
+ NormalizeTensor, TopDownAffine,
+ TopDownGenerateTarget,
+ TopDownGetRandomScaleRotation,
+ TopDownHalfBodyTransform,
+ TopDownRandomFlip,
+ TopDownRandomTranslation, ToTensor)
+
+
+def _check_keys_contain(result_keys, target_keys):
+ """Check if all elements in target_keys is in result_keys."""
+ return set(target_keys).issubset(set(result_keys))
+
+
+def _check_flip(origin_imgs, result_imgs):
+ """Check if the origin_imgs are flipped correctly."""
+ h, w, c = origin_imgs.shape
+ for i in range(h):
+ for j in range(w):
+ for k in range(c):
+ if result_imgs[i, j, k] != origin_imgs[i, w - 1 - j, k]:
+ return False
+ return True
+
+
+def _check_rot90(origin_imgs, result_imgs):
+ if origin_imgs.shape[0] == result_imgs.shape[1] and \
+ origin_imgs.shape[1] == result_imgs.shape[0]:
+ return True
+ else:
+ return False
+
+
+def _check_normalize(origin_imgs, result_imgs, norm_cfg):
+ """Check if the origin_imgs are normalized correctly into result_imgs in a
+ given norm_cfg."""
+ target_imgs = result_imgs.copy()
+ for i in range(3):
+ target_imgs[i] *= norm_cfg['std'][i]
+ target_imgs[i] += norm_cfg['mean'][i]
+ assert_array_almost_equal(origin_imgs, target_imgs, decimal=4)
+
+
+def _box2cs(box, image_size):
+ x, y, w, h = box[:4]
+
+ aspect_ratio = 1. * image_size[0] / image_size[1]
+ center = np.zeros((2), dtype=np.float32)
+ center[0] = x + w * 0.5
+ center[1] = y + h * 0.5
+
+ if w > aspect_ratio * h:
+ h = w * 1.0 / aspect_ratio
+ elif w < aspect_ratio * h:
+ w = h * aspect_ratio
+ scale = np.array([w * 1.0 / 200.0, h * 1.0 / 200.0], dtype=np.float32)
+ scale = scale * 1.25
+ return center, scale
+
+
+def test_top_down_pipeline():
+ # test loading
+ data_prefix = 'tests/data/coco/'
+ ann_file = osp.join(data_prefix, 'test_coco.json')
+ coco = COCO(ann_file)
+
+ results = dict(image_file=osp.join(data_prefix, '000000000785.jpg'))
+ transform = LoadImageFromFile()
+ results = transform(copy.deepcopy(results))
+ assert results['image_file'] == osp.join(data_prefix, '000000000785.jpg')
+
+ assert results['img'].shape == (425, 640, 3)
+ image_size = (425, 640)
+
+ ann_ids = coco.getAnnIds(785)
+ ann = coco.anns[ann_ids[0]]
+
+ num_joints = 17
+ joints_3d = np.zeros((num_joints, 3), dtype=np.float32)
+ joints_3d_visible = np.zeros((num_joints, 3), dtype=np.float32)
+ for ipt in range(num_joints):
+ joints_3d[ipt, 0] = ann['keypoints'][ipt * 3 + 0]
+ joints_3d[ipt, 1] = ann['keypoints'][ipt * 3 + 1]
+ joints_3d[ipt, 2] = 0
+ t_vis = ann['keypoints'][ipt * 3 + 2]
+ if t_vis > 1:
+ t_vis = 1
+ joints_3d_visible[ipt, 0] = t_vis
+ joints_3d_visible[ipt, 1] = t_vis
+ joints_3d_visible[ipt, 2] = 0
+
+ center, scale = _box2cs(ann['bbox'][:4], image_size)
+
+ results['joints_3d'] = joints_3d
+ results['joints_3d_visible'] = joints_3d_visible
+ results['center'] = center
+ results['scale'] = scale
+ results['bbox_score'] = 1
+ results['bbox_id'] = 0
+
+ results['ann_info'] = {}
+ results['ann_info']['flip_pairs'] = [[1, 2], [3, 4], [5, 6], [7, 8],
+ [9, 10], [11, 12], [13, 14], [15, 16]]
+ results['ann_info']['num_joints'] = num_joints
+ results['ann_info']['upper_body_ids'] = (0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10)
+ results['ann_info']['lower_body_ids'] = (11, 12, 13, 14, 15, 16)
+ results['ann_info']['use_different_joint_weights'] = False
+ results['ann_info']['joint_weights'] = np.array([
+ 1., 1., 1., 1., 1., 1., 1., 1.2, 1.2, 1.5, 1.5, 1., 1., 1.2, 1.2, 1.5,
+ 1.5
+ ],
+ dtype=np.float32).reshape(
+ (num_joints, 1))
+ results['ann_info']['image_size'] = np.array([192, 256])
+ results['ann_info']['heatmap_size'] = np.array([48, 64])
+
+ # test flip
+ random_flip = TopDownRandomFlip(flip_prob=1.)
+ results_flip = random_flip(copy.deepcopy(results))
+ assert _check_flip(results['img'], results_flip['img'])
+
+ # test random scale and rotate
+ random_scale_rotate = TopDownGetRandomScaleRotation(90, 0.3, 1.0)
+ results_scale_rotate = random_scale_rotate(copy.deepcopy(results))
+ assert results_scale_rotate['rotation'] <= 180
+ assert results_scale_rotate['rotation'] >= -180
+ assert (results_scale_rotate['scale'] / results['scale'] <= 1.3).all()
+ assert (results_scale_rotate['scale'] / results['scale'] >= 0.7).all()
+
+ # test halfbody transform
+ halfbody_transform = TopDownHalfBodyTransform(
+ num_joints_half_body=8, prob_half_body=1.)
+ results_halfbody = halfbody_transform(copy.deepcopy(results))
+ assert (results_halfbody['scale'] <= results['scale']).all()
+
+ affine_transform = TopDownAffine()
+ results['rotation'] = 90
+ results_affine = affine_transform(copy.deepcopy(results))
+ assert results_affine['img'].shape == (256, 192, 3)
+
+ results = results_affine
+ to_tensor = ToTensor()
+ results_tensor = to_tensor(copy.deepcopy(results))
+ assert isinstance(results_tensor['img'], torch.Tensor)
+ assert results_tensor['img'].shape == torch.Size([3, 256, 192])
+
+ norm_cfg = {}
+ norm_cfg['mean'] = [0.485, 0.456, 0.406]
+ norm_cfg['std'] = [0.229, 0.224, 0.225]
+
+ normalize = NormalizeTensor(mean=norm_cfg['mean'], std=norm_cfg['std'])
+
+ results_normalize = normalize(copy.deepcopy(results_tensor))
+ _check_normalize(results_tensor['img'].data.numpy(),
+ results_normalize['img'].data.numpy(), norm_cfg)
+
+ generate_target = TopDownGenerateTarget(
+ sigma=2, target_type='GaussianHeatMap', unbiased_encoding=True)
+ results_target = generate_target(copy.deepcopy(results_tensor))
+ assert 'target' in results_target
+ assert results_target['target'].shape == (
+ num_joints, results['ann_info']['heatmap_size'][1],
+ results['ann_info']['heatmap_size'][0])
+ assert 'target_weight' in results_target
+ assert results_target['target_weight'].shape == (num_joints, 1)
+
+ generate_target = TopDownGenerateTarget(
+ sigma=2, target_type='GaussianHeatmap', unbiased_encoding=True)
+ results_target = generate_target(copy.deepcopy(results_tensor))
+ assert 'target' in results_target
+ assert results_target['target'].shape == (
+ num_joints, results['ann_info']['heatmap_size'][1],
+ results['ann_info']['heatmap_size'][0])
+ assert 'target_weight' in results_target
+ assert results_target['target_weight'].shape == (num_joints, 1)
+
+ generate_target = TopDownGenerateTarget(sigma=2, unbiased_encoding=False)
+ results_target = generate_target(copy.deepcopy(results_tensor))
+ assert 'target' in results_target
+ assert results_target['target'].shape == (
+ num_joints, results['ann_info']['heatmap_size'][1],
+ results['ann_info']['heatmap_size'][0])
+ assert 'target_weight' in results_target
+ assert results_target['target_weight'].shape == (num_joints, 1)
+
+ generate_target = TopDownGenerateTarget(
+ sigma=[2, 3], unbiased_encoding=False)
+ results_target = generate_target(copy.deepcopy(results_tensor))
+ assert 'target' in results_target
+ assert results_target['target'].shape == (
+ 2, num_joints, results['ann_info']['heatmap_size'][1],
+ results['ann_info']['heatmap_size'][0])
+ assert 'target_weight' in results_target
+ assert results_target['target_weight'].shape == (2, num_joints, 1)
+
+ generate_target = TopDownGenerateTarget(
+ kernel=(11, 11), encoding='Megvii', unbiased_encoding=False)
+ results_target = generate_target(copy.deepcopy(results_tensor))
+ assert 'target' in results_target
+ assert results_target['target'].shape == (
+ num_joints, results['ann_info']['heatmap_size'][1],
+ results['ann_info']['heatmap_size'][0])
+ assert 'target_weight' in results_target
+ assert results_target['target_weight'].shape == (num_joints, 1)
+
+ generate_target = TopDownGenerateTarget(
+ kernel=[(11, 11), (7, 7)], encoding='Megvii', unbiased_encoding=False)
+ results_target = generate_target(copy.deepcopy(results_tensor))
+ assert 'target' in results_target
+ assert results_target['target'].shape == (
+ 2, num_joints, results['ann_info']['heatmap_size'][1],
+ results['ann_info']['heatmap_size'][0])
+ assert 'target_weight' in results_target
+ assert results_target['target_weight'].shape == (2, num_joints, 1)
+
+ collect = Collect(
+ keys=['img', 'target', 'target_weight'],
+ meta_keys=[
+ 'image_file', 'center', 'scale', 'rotation', 'bbox_score',
+ 'flip_pairs'
+ ])
+ results_final = collect(results_target)
+ assert 'img_size' not in results_final['img_metas'].data
+ assert 'image_file' in results_final['img_metas'].data
+
+
+def test_random_translation():
+ results = dict(
+ center=np.zeros([2]),
+ scale=1,
+ )
+ pipeline = TopDownRandomTranslation()
+ results = pipeline(results)
+ assert results['center'].shape == (2, )
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_post_processing.py b/engine/pose_estimation/third-party/ViTPose/tests/test_post_processing.py
new file mode 100644
index 0000000..79c8c2a
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_post_processing.py
@@ -0,0 +1,94 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import numpy as np
+from numpy.testing import assert_array_almost_equal
+
+from mmpose.core import (affine_transform, flip_back, fliplr_joints,
+ fliplr_regression, get_affine_transform, rotate_point,
+ transform_preds)
+
+
+def test_affine_transform():
+ pt = np.array([0, 1])
+ trans = np.array([[1, 0, 1], [0, 1, 0]])
+ result = affine_transform(pt, trans)
+ assert_array_almost_equal(result, np.array([1, 1]), decimal=4)
+ assert isinstance(result, np.ndarray)
+
+
+def test_rotate_point():
+ src_point = np.array([0, 1])
+ rot_rad = np.pi / 2.
+ result = rotate_point(src_point, rot_rad)
+ assert_array_almost_equal(result, np.array([-1, 0]), decimal=4)
+ assert isinstance(result, list)
+
+
+def test_fliplr_joints():
+ joints = np.array([[0, 0, 0], [1, 1, 0]])
+ joints_vis = np.array([[1], [1]])
+ joints_flip, _ = fliplr_joints(joints, joints_vis, 5, [[0, 1]])
+ res = np.array([[3, 1, 0], [4, 0, 0]])
+ assert_array_almost_equal(joints_flip, res)
+
+
+def test_flip_back():
+ heatmaps = np.random.random([1, 2, 32, 32])
+ flipped_heatmaps = flip_back(heatmaps, [[0, 1]])
+ heatmaps_new = flip_back(flipped_heatmaps, [[0, 1]])
+ assert_array_almost_equal(heatmaps, heatmaps_new)
+
+ heatmaps = np.random.random([1, 2, 32, 32])
+ flipped_heatmaps = flip_back(heatmaps, [[0, 1]])
+ heatmaps_new = flipped_heatmaps[..., ::-1]
+ assert_array_almost_equal(heatmaps[:, 0], heatmaps_new[:, 1])
+ assert_array_almost_equal(heatmaps[:, 1], heatmaps_new[:, 0])
+
+ ori_heatmaps = heatmaps.copy()
+ # test in-place flip
+ heatmaps = heatmaps[:, :, :, ::-1]
+ assert_array_almost_equal(ori_heatmaps[:, :, :, ::-1], heatmaps)
+
+
+def test_transform_preds():
+ coords = np.random.random([2, 2])
+ center = np.array([50, 50])
+ scale = np.array([100 / 200.0, 100 / 200.0])
+ size = np.array([100, 100])
+ result = transform_preds(coords, center, scale, size)
+ assert_array_almost_equal(coords, result)
+
+ coords = np.random.random([2, 2])
+ center = np.array([50, 50])
+ scale = np.array([100 / 200.0, 100 / 200.0])
+ size = np.array([101, 101])
+ result = transform_preds(coords, center, scale, size, use_udp=True)
+ assert_array_almost_equal(coords, result)
+
+
+def test_get_affine_transform():
+ center = np.array([50, 50])
+ scale = np.array([100 / 200.0, 100 / 200.0])
+ size = np.array([100, 100])
+ result = get_affine_transform(center, scale, 0, size)
+ trans = np.array([[1, 0, 0], [0, 1, 0]])
+ assert_array_almost_equal(trans, result)
+
+
+def test_flip_regression():
+ coords = np.random.rand(3, 3)
+ flip_pairs = [[1, 2]]
+ root = coords[:1]
+ coords_flipped = coords.copy()
+ coords_flipped[1] = coords[2]
+ coords_flipped[2] = coords[1]
+ coords_flipped[..., 0] = 2 * root[..., 0] - coords_flipped[..., 0]
+
+ # static mode
+ res_static = fliplr_regression(
+ coords, flip_pairs, center_mode='static', center_x=root[0, 0])
+ assert_array_almost_equal(res_static, coords_flipped)
+
+ # root mode
+ res_root = fliplr_regression(
+ coords, flip_pairs, center_mode='root', center_index=0)
+ assert_array_almost_equal(res_root, coords_flipped)
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_post_processing/test_filter.py b/engine/pose_estimation/third-party/ViTPose/tests/test_post_processing/test_filter.py
new file mode 100644
index 0000000..4701697
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_post_processing/test_filter.py
@@ -0,0 +1,36 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import numpy as np
+
+from mmpose.core.post_processing.one_euro_filter import OneEuroFilter
+
+
+def test_one_euro_filter():
+ np.random.seed(1)
+
+ kpts = []
+ frames = 100
+ for i in range(frames):
+ kpts.append({
+ 'keypoints': np.tile(np.array([10, 10, 0.9]), [17, 1]),
+ 'area': 100,
+ 'score': 0.9
+ })
+ kpts.append({
+ 'keypoints': np.tile(np.array([11, 11, 0.9]), [17, 1]),
+ 'area': 100,
+ 'score': 0.8
+ })
+
+ one_euro_filter = OneEuroFilter(
+ kpts[0]['keypoints'][:, :2], min_cutoff=1.7, beta=0.3, fps=30)
+
+ for i in range(1, len(kpts)):
+ kpts[i]['keypoints'][:, :2] = one_euro_filter(
+ kpts[i]['keypoints'][:, :2])
+
+ one_euro_filter = OneEuroFilter(
+ kpts[0]['keypoints'][:, :2], min_cutoff=1.7, beta=0.3)
+
+ for i in range(1, len(kpts)):
+ kpts[i]['keypoints'][:, :2] = one_euro_filter(
+ kpts[i]['keypoints'][:, :2])
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_post_processing/test_group.py b/engine/pose_estimation/third-party/ViTPose/tests/test_post_processing/test_group.py
new file mode 100644
index 0000000..2ec66ef
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_post_processing/test_group.py
@@ -0,0 +1,72 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import torch
+
+from mmpose.core.post_processing.group import HeatmapParser
+
+
+def test_group():
+ cfg = {}
+ cfg['num_joints'] = 17
+ cfg['detection_threshold'] = 0.1
+ cfg['tag_threshold'] = 1
+ cfg['use_detection_val'] = True
+ cfg['ignore_too_much'] = False
+ cfg['nms_kernel'] = 5
+ cfg['nms_padding'] = 2
+ cfg['tag_per_joint'] = True
+ cfg['max_num_people'] = 1
+ parser = HeatmapParser(cfg)
+ fake_heatmap = torch.zeros(1, 1, 5, 5)
+ fake_heatmap[0, 0, 3, 3] = 1
+ fake_heatmap[0, 0, 3, 2] = 0.8
+ assert parser.nms(fake_heatmap)[0, 0, 3, 2] == 0
+ fake_heatmap = torch.zeros(1, 17, 32, 32)
+ fake_tag = torch.zeros(1, 17, 32, 32, 1)
+ fake_heatmap[0, 0, 10, 10] = 0.8
+ fake_heatmap[0, 1, 12, 12] = 0.9
+ fake_heatmap[0, 4, 8, 8] = 0.8
+ fake_heatmap[0, 8, 6, 6] = 0.9
+ fake_tag[0, 0, 10, 10] = 0.8
+ fake_tag[0, 1, 12, 12] = 0.9
+ fake_tag[0, 4, 8, 8] = 0.8
+ fake_tag[0, 8, 6, 6] = 0.9
+ grouped, scores = parser.parse(fake_heatmap, fake_tag, True, True)
+ assert grouped[0][0, 0, 0] == 10.25
+ assert abs(scores[0] - 0.2) < 0.001
+ cfg['tag_per_joint'] = False
+ parser = HeatmapParser(cfg)
+ grouped, scores = parser.parse(fake_heatmap, fake_tag, False, False)
+ assert grouped[0][0, 0, 0] == 10.
+ grouped, scores = parser.parse(fake_heatmap, fake_tag, False, True)
+ assert grouped[0][0, 0, 0] == 10.
+
+
+def test_group_score_per_joint():
+ cfg = {}
+ cfg['num_joints'] = 17
+ cfg['detection_threshold'] = 0.1
+ cfg['tag_threshold'] = 1
+ cfg['use_detection_val'] = True
+ cfg['ignore_too_much'] = False
+ cfg['nms_kernel'] = 5
+ cfg['nms_padding'] = 2
+ cfg['tag_per_joint'] = True
+ cfg['max_num_people'] = 1
+ cfg['score_per_joint'] = True
+ parser = HeatmapParser(cfg)
+ fake_heatmap = torch.zeros(1, 1, 5, 5)
+ fake_heatmap[0, 0, 3, 3] = 1
+ fake_heatmap[0, 0, 3, 2] = 0.8
+ assert parser.nms(fake_heatmap)[0, 0, 3, 2] == 0
+ fake_heatmap = torch.zeros(1, 17, 32, 32)
+ fake_tag = torch.zeros(1, 17, 32, 32, 1)
+ fake_heatmap[0, 0, 10, 10] = 0.8
+ fake_heatmap[0, 1, 12, 12] = 0.9
+ fake_heatmap[0, 4, 8, 8] = 0.8
+ fake_heatmap[0, 8, 6, 6] = 0.9
+ fake_tag[0, 0, 10, 10] = 0.8
+ fake_tag[0, 1, 12, 12] = 0.9
+ fake_tag[0, 4, 8, 8] = 0.8
+ fake_tag[0, 8, 6, 6] = 0.9
+ grouped, scores = parser.parse(fake_heatmap, fake_tag, True, True)
+ assert len(scores[0]) == 17
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_post_processing/test_nms.py b/engine/pose_estimation/third-party/ViTPose/tests/test_post_processing/test_nms.py
new file mode 100644
index 0000000..13d793d
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_post_processing/test_nms.py
@@ -0,0 +1,81 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import numpy as np
+
+from mmpose.core.post_processing.nms import nms, oks_iou, oks_nms, soft_oks_nms
+
+
+def test_soft_oks_nms():
+ oks_thr = 0.9
+ kpts = []
+ kpts.append({
+ 'keypoints': np.tile(np.array([10, 10, 0.9]), [17, 1]),
+ 'area': 100,
+ 'score': 0.9
+ })
+ kpts.append({
+ 'keypoints': np.tile(np.array([10, 10, 0.9]), [17, 1]),
+ 'area': 100,
+ 'score': 0.4
+ })
+ kpts.append({
+ 'keypoints': np.tile(np.array([100, 100, 0.9]), [17, 1]),
+ 'area': 100,
+ 'score': 0.7
+ })
+
+ keep = soft_oks_nms([kpts[i] for i in range(len(kpts))], oks_thr)
+ assert (keep == np.array([0, 2, 1])).all()
+
+ keep = oks_nms([kpts[i] for i in range(len(kpts))], oks_thr)
+ assert (keep == np.array([0, 2])).all()
+
+ kpts_with_score_joints = []
+ kpts_with_score_joints.append({
+ 'keypoints':
+ np.tile(np.array([10, 10, 0.9]), [17, 1]),
+ 'area':
+ 100,
+ 'score':
+ np.tile(np.array([0.9]), 17)
+ })
+ kpts_with_score_joints.append({
+ 'keypoints':
+ np.tile(np.array([10, 10, 0.9]), [17, 1]),
+ 'area':
+ 100,
+ 'score':
+ np.tile(np.array([0.4]), 17)
+ })
+ kpts_with_score_joints.append({
+ 'keypoints':
+ np.tile(np.array([100, 100, 0.9]), [17, 1]),
+ 'area':
+ 100,
+ 'score':
+ np.tile(np.array([0.7]), 17)
+ })
+ keep = soft_oks_nms([
+ kpts_with_score_joints[i] for i in range(len(kpts_with_score_joints))
+ ],
+ oks_thr,
+ score_per_joint=True)
+ assert (keep == np.array([0, 2, 1])).all()
+
+ keep = oks_nms([
+ kpts_with_score_joints[i] for i in range(len(kpts_with_score_joints))
+ ],
+ oks_thr,
+ score_per_joint=True)
+ assert (keep == np.array([0, 2])).all()
+
+
+def test_func_nms():
+ result = nms(np.array([[0, 0, 10, 10, 0.9], [0, 0, 10, 8, 0.8]]), 0.5)
+ assert result == [0]
+
+
+def test_oks_iou():
+ result = oks_iou(np.ones([17 * 3]), np.ones([1, 17 * 3]), 1, [1])
+ assert result[0] == 1.
+ result = oks_iou(np.zeros([17 * 3]), np.ones([1, 17 * 3]), 1, [1])
+ assert result[0] < 0.01
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_regularization.py b/engine/pose_estimation/third-party/ViTPose/tests/test_regularization.py
new file mode 100644
index 0000000..a93cc63
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_regularization.py
@@ -0,0 +1,19 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import numpy as np
+import torch
+
+from mmpose.core import WeightNormClipHook
+
+
+def test_weight_norm_clip():
+ torch.manual_seed(0)
+
+ module = torch.nn.Linear(2, 2, bias=False)
+ module.weight.data.fill_(2)
+ WeightNormClipHook(max_norm=1.0).register(module)
+
+ x = torch.rand(1, 2).requires_grad_()
+ _ = module(x)
+
+ weight_norm = module.weight.norm().item()
+ np.testing.assert_almost_equal(weight_norm, 1.0, decimal=6)
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_utils.py b/engine/pose_estimation/third-party/ViTPose/tests/test_utils.py
new file mode 100644
index 0000000..9b4d1c1
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_utils.py
@@ -0,0 +1,100 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import multiprocessing as mp
+import os
+import platform
+import time
+
+import cv2
+import mmcv
+import torch
+import torchvision
+from mmcv import Config
+
+import mmpose
+from mmpose.utils import StopWatch, collect_env, setup_multi_processes
+
+
+def test_collect_env():
+ env_info = collect_env()
+ assert env_info['PyTorch'] == torch.__version__
+ assert env_info['TorchVision'] == torchvision.__version__
+ assert env_info['OpenCV'] == cv2.__version__
+ assert env_info['MMCV'] == mmcv.__version__
+ assert '+' in env_info['MMPose']
+ assert mmpose.__version__ in env_info['MMPose']
+
+
+def test_stopwatch():
+ window_size = 5
+ test_loop = 10
+ outer_time = 100
+ inner_time = 100
+
+ stop_watch = StopWatch(window=window_size)
+ for _ in range(test_loop):
+ with stop_watch.timeit():
+ time.sleep(outer_time / 1000.)
+ with stop_watch.timeit('inner'):
+ time.sleep(inner_time / 1000.)
+
+ _ = stop_watch.report()
+ _ = stop_watch.report_strings()
+
+
+def test_setup_multi_processes():
+ # temp save system setting
+ sys_start_mehod = mp.get_start_method(allow_none=True)
+ sys_cv_threads = cv2.getNumThreads()
+ # pop and temp save system env vars
+ sys_omp_threads = os.environ.pop('OMP_NUM_THREADS', default=None)
+ sys_mkl_threads = os.environ.pop('MKL_NUM_THREADS', default=None)
+
+ # test config without setting env
+ config = dict(data=dict(workers_per_gpu=2))
+ cfg = Config(config)
+ setup_multi_processes(cfg)
+ assert os.getenv('OMP_NUM_THREADS') == '1'
+ assert os.getenv('MKL_NUM_THREADS') == '1'
+ # when set to 0, the num threads will be 1
+ assert cv2.getNumThreads() == 1
+ if platform.system() != 'Windows':
+ assert mp.get_start_method() == 'fork'
+
+ # test num workers <= 1
+ os.environ.pop('OMP_NUM_THREADS')
+ os.environ.pop('MKL_NUM_THREADS')
+ config = dict(data=dict(workers_per_gpu=0))
+ cfg = Config(config)
+ setup_multi_processes(cfg)
+ assert 'OMP_NUM_THREADS' not in os.environ
+ assert 'MKL_NUM_THREADS' not in os.environ
+
+ # test manually set env var
+ os.environ['OMP_NUM_THREADS'] = '4'
+ config = dict(data=dict(workers_per_gpu=2))
+ cfg = Config(config)
+ setup_multi_processes(cfg)
+ assert os.getenv('OMP_NUM_THREADS') == '4'
+
+ # test manually set opencv threads and mp start method
+ config = dict(
+ data=dict(workers_per_gpu=2),
+ opencv_num_threads=4,
+ mp_start_method='spawn')
+ cfg = Config(config)
+ setup_multi_processes(cfg)
+ assert cv2.getNumThreads() == 4
+ assert mp.get_start_method() == 'spawn'
+
+ # revert setting to avoid affecting other programs
+ if sys_start_mehod:
+ mp.set_start_method(sys_start_mehod, force=True)
+ cv2.setNumThreads(sys_cv_threads)
+ if sys_omp_threads:
+ os.environ['OMP_NUM_THREADS'] = sys_omp_threads
+ else:
+ os.environ.pop('OMP_NUM_THREADS')
+ if sys_mkl_threads:
+ os.environ['MKL_NUM_THREADS'] = sys_mkl_threads
+ else:
+ os.environ.pop('MKL_NUM_THREADS')
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_version.py b/engine/pose_estimation/third-party/ViTPose/tests/test_version.py
new file mode 100644
index 0000000..392ded4
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_version.py
@@ -0,0 +1,9 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import mmpose
+
+
+def test_version():
+ version = mmpose.__version__
+ assert isinstance(version, str)
+ assert isinstance(mmpose.short_version, str)
+ assert mmpose.short_version in version
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/test_visualization.py b/engine/pose_estimation/third-party/ViTPose/tests/test_visualization.py
new file mode 100644
index 0000000..f04dad2
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/test_visualization.py
@@ -0,0 +1,99 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import tempfile
+
+import mmcv
+import numpy as np
+import pytest
+
+from mmpose.core import (apply_bugeye_effect, apply_sunglasses_effect,
+ imshow_bboxes, imshow_keypoints, imshow_keypoints_3d)
+
+
+def test_imshow_keypoints():
+ # 2D keypoint
+ img = np.zeros((100, 100, 3), dtype=np.uint8)
+ kpts = np.array([[1, 1, 1], [10, 10, 1]], dtype=np.float32)
+ pose_result = [kpts]
+ skeleton = [[0, 1]]
+ pose_kpt_color = [(127, 127, 127)] * len(kpts)
+ pose_link_color = [(127, 127, 127)] * len(skeleton)
+ img_vis_2d = imshow_keypoints(
+ img,
+ pose_result,
+ skeleton=skeleton,
+ pose_kpt_color=pose_kpt_color,
+ pose_link_color=pose_link_color,
+ show_keypoint_weight=True)
+
+ # 3D keypoint
+ kpts_3d = np.array([[0, 0, 0, 1], [1, 1, 1, 1]], dtype=np.float32)
+ pose_result_3d = [{'keypoints_3d': kpts_3d, 'title': 'test'}]
+ _ = imshow_keypoints_3d(
+ pose_result_3d,
+ img=img_vis_2d,
+ skeleton=skeleton,
+ pose_kpt_color=pose_kpt_color,
+ pose_link_color=pose_link_color,
+ vis_height=400)
+
+
+def test_imshow_bbox():
+ img = np.zeros((100, 100, 3), dtype=np.uint8)
+ bboxes = np.array([[10, 10, 30, 30], [10, 50, 30, 80]], dtype=np.float32)
+ labels = ['label 1', 'label 2']
+ colors = ['red', 'green']
+
+ with tempfile.TemporaryDirectory() as tmpdir:
+ _ = imshow_bboxes(
+ img,
+ bboxes,
+ labels=labels,
+ colors=colors,
+ show=False,
+ out_file=f'{tmpdir}/out.png')
+
+ # test case of empty bboxes
+ _ = imshow_bboxes(
+ img,
+ np.zeros((0, 4), dtype=np.float32),
+ labels=None,
+ colors='red',
+ show=False)
+
+ # test unmatched bboxes and labels
+ with pytest.raises(AssertionError):
+ _ = imshow_bboxes(
+ img,
+ np.zeros((0, 4), dtype=np.float32),
+ labels=labels[:1],
+ colors='red',
+ show=False)
+
+
+def test_effects():
+ img = np.zeros((100, 100, 3), dtype=np.uint8)
+ kpts = np.array([[10., 10., 0.8], [20., 10., 0.8]], dtype=np.float32)
+ bbox = np.array([0, 0, 50, 50], dtype=np.float32)
+ pose_results = [dict(bbox=bbox, keypoints=kpts)]
+ # sunglasses
+ sunglasses_img = mmcv.imread('demo/resources/sunglasses.jpg')
+ _ = apply_sunglasses_effect(
+ img,
+ pose_results,
+ sunglasses_img,
+ left_eye_index=1,
+ right_eye_index=0,
+ kpt_thr=0.5)
+ _ = apply_sunglasses_effect(
+ img,
+ pose_results,
+ sunglasses_img,
+ left_eye_index=1,
+ right_eye_index=0,
+ kpt_thr=0.9)
+
+ # bug-eye
+ _ = apply_bugeye_effect(
+ img, pose_results, left_eye_index=1, right_eye_index=0, kpt_thr=0.5)
+ _ = apply_bugeye_effect(
+ img, pose_results, left_eye_index=1, right_eye_index=0, kpt_thr=0.9)
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/utils/data_utils.py b/engine/pose_estimation/third-party/ViTPose/tests/utils/data_utils.py
new file mode 100644
index 0000000..a04e2e6
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/utils/data_utils.py
@@ -0,0 +1,47 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import numpy as np
+
+
+def convert_db_to_output(db, batch_size=2, keys=None, is_3d=False):
+ outputs = []
+ len_db = len(db)
+ for i in range(0, len_db, batch_size):
+ keypoints_dim = 3 if is_3d else 2
+ keypoints = np.stack([
+ np.hstack([
+ db[j]['joints_3d'].reshape((-1, 3))[:, :keypoints_dim],
+ db[j]['joints_3d_visible'].reshape((-1, 3))[:, :1]
+ ]) for j in range(i, min(i + batch_size, len_db))
+ ])
+
+ image_paths = [
+ db[j]['image_file'] for j in range(i, min(i + batch_size, len_db))
+ ]
+ bbox_ids = [j for j in range(i, min(i + batch_size, len_db))]
+ box = np.stack([
+ np.array([
+ db[j]['center'][0], db[j]['center'][1], db[j]['scale'][0],
+ db[j]['scale'][1],
+ db[j]['scale'][0] * db[j]['scale'][1] * 200 * 200, 1.0
+ ],
+ dtype=np.float32)
+ for j in range(i, min(i + batch_size, len_db))
+ ])
+
+ output = {}
+ output['preds'] = keypoints
+ output['boxes'] = box
+ output['image_paths'] = image_paths
+ output['output_heatmap'] = None
+ output['bbox_ids'] = bbox_ids
+
+ if keys is not None:
+ keys = keys if isinstance(keys, list) else [keys]
+ for key in keys:
+ output[key] = [
+ db[j][key] for j in range(i, min(i + batch_size, len_db))
+ ]
+
+ outputs.append(output)
+
+ return outputs
diff --git a/engine/pose_estimation/third-party/ViTPose/tests/utils/mesh_utils.py b/engine/pose_estimation/third-party/ViTPose/tests/utils/mesh_utils.py
new file mode 100644
index 0000000..a03b5ab
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tests/utils/mesh_utils.py
@@ -0,0 +1,35 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os
+import pickle
+
+import numpy as np
+from scipy.sparse import csc_matrix
+
+
+def generate_smpl_weight_file(output_dir):
+ """Generate a SMPL model weight file to initialize SMPL model, and generate
+ a 3D joints regressor file."""
+
+ if not os.path.exists(output_dir):
+ os.makedirs(output_dir)
+
+ joint_regressor_file = os.path.join(output_dir, 'test_joint_regressor.npy')
+ np.save(joint_regressor_file, np.zeros([24, 6890]))
+
+ test_data = {}
+ test_data['f'] = np.zeros([1, 3], dtype=np.int32)
+ test_data['J_regressor'] = csc_matrix(np.zeros([24, 6890]))
+ test_data['kintree_table'] = np.zeros([2, 24], dtype=np.uint32)
+ test_data['J'] = np.zeros([24, 3])
+ test_data['weights'] = np.zeros([6890, 24])
+ test_data['posedirs'] = np.zeros([6890, 3, 207])
+ test_data['v_template'] = np.zeros([6890, 3])
+ test_data['shapedirs'] = np.zeros([6890, 3, 10])
+
+ with open(os.path.join(output_dir, 'SMPL_NEUTRAL.pkl'), 'wb') as out_file:
+ pickle.dump(test_data, out_file)
+ with open(os.path.join(output_dir, 'SMPL_MALE.pkl'), 'wb') as out_file:
+ pickle.dump(test_data, out_file)
+ with open(os.path.join(output_dir, 'SMPL_FEMALE.pkl'), 'wb') as out_file:
+ pickle.dump(test_data, out_file)
+ return
diff --git a/engine/pose_estimation/third-party/ViTPose/tools/analysis/analyze_logs.py b/engine/pose_estimation/third-party/ViTPose/tools/analysis/analyze_logs.py
new file mode 100644
index 0000000..d0e1a02
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tools/analysis/analyze_logs.py
@@ -0,0 +1,167 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import argparse
+import json
+from collections import defaultdict
+
+import matplotlib.pyplot as plt
+import numpy as np
+import seaborn as sns
+
+
+def cal_train_time(log_dicts, args):
+ for i, log_dict in enumerate(log_dicts):
+ print(f'{"-" * 5}Analyze train time of {args.json_logs[i]}{"-" * 5}')
+ all_times = []
+ for epoch in log_dict.keys():
+ if args.include_outliers:
+ all_times.append(log_dict[epoch]['time'])
+ else:
+ all_times.append(log_dict[epoch]['time'][1:])
+ all_times = np.array(all_times)
+ epoch_ave_time = all_times.mean(-1)
+ slowest_epoch = epoch_ave_time.argmax()
+ fastest_epoch = epoch_ave_time.argmin()
+ std_over_epoch = epoch_ave_time.std()
+ print(f'slowest epoch {slowest_epoch + 1}, '
+ f'average time is {epoch_ave_time[slowest_epoch]:.4f}')
+ print(f'fastest epoch {fastest_epoch + 1}, '
+ f'average time is {epoch_ave_time[fastest_epoch]:.4f}')
+ print(f'time std over epochs is {std_over_epoch:.4f}')
+ print(f'average iter time: {np.mean(all_times):.4f} s/iter')
+ print()
+
+
+def plot_curve(log_dicts, args):
+ if args.backend is not None:
+ plt.switch_backend(args.backend)
+ sns.set_style(args.style)
+ # if legend is None, use {filename}_{key} as legend
+ legend = args.legend
+ if legend is None:
+ legend = []
+ for json_log in args.json_logs:
+ for metric in args.keys:
+ legend.append(f'{json_log}_{metric}')
+ assert len(legend) == (len(args.json_logs) * len(args.keys))
+ metrics = args.keys
+
+ num_metrics = len(metrics)
+ for i, log_dict in enumerate(log_dicts):
+ epochs = list(log_dict.keys())
+ for j, metric in enumerate(metrics):
+ print(f'plot curve of {args.json_logs[i]}, metric is {metric}')
+ if metric not in log_dict[epochs[0]]:
+ raise KeyError(
+ f'{args.json_logs[i]} does not contain metric {metric}')
+ xs = []
+ ys = []
+ num_iters_per_epoch = log_dict[epochs[0]]['iter'][-1]
+ for epoch in epochs:
+ iters = log_dict[epoch]['iter']
+ if log_dict[epoch]['mode'][-1] == 'val':
+ iters = iters[:-1]
+ xs.append(np.array(iters) + (epoch - 1) * num_iters_per_epoch)
+ ys.append(np.array(log_dict[epoch][metric][:len(iters)]))
+ xs = np.concatenate(xs)
+ ys = np.concatenate(ys)
+ plt.xlabel('iter')
+ plt.plot(xs, ys, label=legend[i * num_metrics + j], linewidth=0.5)
+ plt.legend()
+ if args.title is not None:
+ plt.title(args.title)
+ if args.out is None:
+ plt.show()
+ else:
+ print(f'save curve to: {args.out}')
+ plt.savefig(args.out)
+ plt.cla()
+
+
+def add_plot_parser(subparsers):
+ parser_plt = subparsers.add_parser(
+ 'plot_curve', help='parser for plotting curves')
+ parser_plt.add_argument(
+ 'json_logs',
+ type=str,
+ nargs='+',
+ help='path of train log in json format')
+ parser_plt.add_argument(
+ '--keys',
+ type=str,
+ nargs='+',
+ default=['top1_acc'],
+ help='the metric that you want to plot')
+ parser_plt.add_argument('--title', type=str, help='title of figure')
+ parser_plt.add_argument(
+ '--legend',
+ type=str,
+ nargs='+',
+ default=None,
+ help='legend of each plot')
+ parser_plt.add_argument(
+ '--backend', type=str, default=None, help='backend of plt')
+ parser_plt.add_argument(
+ '--style', type=str, default='dark', help='style of plt')
+ parser_plt.add_argument('--out', type=str, default=None)
+
+
+def add_time_parser(subparsers):
+ parser_time = subparsers.add_parser(
+ 'cal_train_time',
+ help='parser for computing the average time per training iteration')
+ parser_time.add_argument(
+ 'json_logs',
+ type=str,
+ nargs='+',
+ help='path of train log in json format')
+ parser_time.add_argument(
+ '--include-outliers',
+ action='store_true',
+ help='include the first value of every epoch when computing '
+ 'the average time')
+
+
+def parse_args():
+ parser = argparse.ArgumentParser(description='Analyze Json Log')
+ # currently only support plot curve and calculate average train time
+ subparsers = parser.add_subparsers(dest='task', help='task parser')
+ add_plot_parser(subparsers)
+ add_time_parser(subparsers)
+ args = parser.parse_args()
+ return args
+
+
+def load_json_logs(json_logs):
+ # load and convert json_logs to log_dict, key is epoch, value is a sub dict
+ # keys of sub dict is different metrics, e.g. memory, top1_acc
+ # value of sub dict is a list of corresponding values of all iterations
+ log_dicts = [dict() for _ in json_logs]
+ for json_log, log_dict in zip(json_logs, log_dicts):
+ with open(json_log, 'r') as log_file:
+ for line in log_file:
+ log = json.loads(line.strip())
+ # skip lines without `epoch` field
+ if 'epoch' not in log:
+ continue
+ epoch = log.pop('epoch')
+ if epoch not in log_dict:
+ log_dict[epoch] = defaultdict(list)
+ for k, v in log.items():
+ log_dict[epoch][k].append(v)
+ return log_dicts
+
+
+def main():
+ args = parse_args()
+
+ json_logs = args.json_logs
+ for json_log in json_logs:
+ assert json_log.endswith('.json')
+
+ log_dicts = load_json_logs(json_logs)
+
+ eval(args.task)(log_dicts, args)
+
+
+if __name__ == '__main__':
+ main()
diff --git a/engine/pose_estimation/third-party/ViTPose/tools/analysis/benchmark_inference.py b/engine/pose_estimation/third-party/ViTPose/tools/analysis/benchmark_inference.py
new file mode 100644
index 0000000..14c0736
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tools/analysis/benchmark_inference.py
@@ -0,0 +1,82 @@
+#!/usr/bin/env bash
+# Copyright (c) OpenMMLab. All rights reserved.
+import argparse
+import time
+
+import torch
+from mmcv import Config
+from mmcv.cnn import fuse_conv_bn
+from mmcv.parallel import MMDataParallel
+from mmcv.runner.fp16_utils import wrap_fp16_model
+
+from mmpose.datasets import build_dataloader, build_dataset
+from mmpose.models import build_posenet
+
+
+def parse_args():
+ parser = argparse.ArgumentParser(
+ description='MMPose benchmark a recognizer')
+ parser.add_argument('config', help='test config file path')
+ parser.add_argument(
+ '--log-interval', default=10, help='interval of logging')
+ parser.add_argument(
+ '--fuse-conv-bn',
+ action='store_true',
+ help='Whether to fuse conv and bn, this will slightly increase'
+ 'the inference speed')
+ args = parser.parse_args()
+ return args
+
+
+def main():
+ args = parse_args()
+
+ cfg = Config.fromfile(args.config)
+ # set cudnn_benchmark
+ if cfg.get('cudnn_benchmark', False):
+ torch.backends.cudnn.benchmark = True
+
+ # build the dataloader
+ dataset = build_dataset(cfg.data.val)
+ data_loader = build_dataloader(
+ dataset,
+ samples_per_gpu=1,
+ workers_per_gpu=cfg.data.workers_per_gpu,
+ dist=False,
+ shuffle=False)
+
+ # build the model and load checkpoint
+ model = build_posenet(cfg.model)
+ fp16_cfg = cfg.get('fp16', None)
+ if fp16_cfg is not None:
+ wrap_fp16_model(model)
+ if args.fuse_conv_bn:
+ model = fuse_conv_bn(model)
+ model = MMDataParallel(model, device_ids=[0])
+
+ # the first several iterations may be very slow so skip them
+ num_warmup = 5
+ pure_inf_time = 0
+
+ # benchmark with total batch and take the average
+ for i, data in enumerate(data_loader):
+
+ torch.cuda.synchronize()
+ start_time = time.perf_counter()
+ with torch.no_grad():
+ model(return_loss=False, **data)
+
+ torch.cuda.synchronize()
+ elapsed = time.perf_counter() - start_time
+
+ if i >= num_warmup:
+ pure_inf_time += elapsed
+ if (i + 1) % args.log_interval == 0:
+ its = (i + 1 - num_warmup) / pure_inf_time
+ print(f'Done item [{i + 1:<3}], {its:.2f} items / s')
+ print(f'Overall average: {its:.2f} items / s')
+ print(f'Total time: {pure_inf_time:.2f} s')
+
+
+if __name__ == '__main__':
+ main()
diff --git a/engine/pose_estimation/third-party/ViTPose/tools/analysis/benchmark_processing.py b/engine/pose_estimation/third-party/ViTPose/tools/analysis/benchmark_processing.py
new file mode 100644
index 0000000..d326f3d
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tools/analysis/benchmark_processing.py
@@ -0,0 +1,58 @@
+#!/usr/bin/env bash
+# Copyright (c) OpenMMLab. All rights reserved.
+"""This file is for benchmark data loading process. It can also be used to
+refresh the memcached cache. The command line to run this file is:
+
+$ python -m cProfile -o program.prof tools/analysis/benchmark_processing.py
+configs/task/method/[config filename]
+
+Note: When debugging, the `workers_per_gpu` in the config should be set to 0
+during benchmark.
+
+It use cProfile to record cpu running time and output to program.prof
+To visualize cProfile output program.prof, use Snakeviz and run:
+$ snakeviz program.prof
+"""
+import argparse
+
+import mmcv
+from mmcv import Config
+
+from mmpose import __version__
+from mmpose.datasets import build_dataloader, build_dataset
+from mmpose.utils import get_root_logger
+
+
+def main():
+ parser = argparse.ArgumentParser(description='Benchmark data loading')
+ parser.add_argument('config', help='train config file path')
+ args = parser.parse_args()
+ cfg = Config.fromfile(args.config)
+
+ # init logger before other steps
+ logger = get_root_logger()
+ logger.info(f'MMPose Version: {__version__}')
+ logger.info(f'Config: {cfg.text}')
+
+ dataset = build_dataset(cfg.data.train)
+ data_loader = build_dataloader(
+ dataset,
+ samples_per_gpu=1,
+ workers_per_gpu=cfg.data.workers_per_gpu,
+ dist=False,
+ shuffle=False)
+
+ # Start progress bar after first 5 batches
+ prog_bar = mmcv.ProgressBar(
+ len(dataset) - 5 * cfg.data.samples_per_gpu, start=False)
+ for i, data in enumerate(data_loader):
+ if i == 5:
+ prog_bar.start()
+ for _ in data['img']:
+ if i < 5:
+ continue
+ prog_bar.update()
+
+
+if __name__ == '__main__':
+ main()
diff --git a/engine/pose_estimation/third-party/ViTPose/tools/analysis/get_flops.py b/engine/pose_estimation/third-party/ViTPose/tools/analysis/get_flops.py
new file mode 100644
index 0000000..f492a87
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tools/analysis/get_flops.py
@@ -0,0 +1,103 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import argparse
+from functools import partial
+
+import torch
+
+from mmpose.apis.inference import init_pose_model
+
+try:
+ from mmcv.cnn import get_model_complexity_info
+except ImportError:
+ raise ImportError('Please upgrade mmcv to >0.6.2')
+
+
+def parse_args():
+ parser = argparse.ArgumentParser(description='Train a recognizer')
+ parser.add_argument('config', help='train config file path')
+ parser.add_argument(
+ '--shape',
+ type=int,
+ nargs='+',
+ default=[256, 192],
+ help='input image size')
+ parser.add_argument(
+ '--input-constructor',
+ '-c',
+ type=str,
+ choices=['none', 'batch'],
+ default='none',
+ help='If specified, it takes a callable method that generates '
+ 'input. Otherwise, it will generate a random tensor with '
+ 'input shape to calculate FLOPs.')
+ parser.add_argument(
+ '--batch-size', '-b', type=int, default=1, help='input batch size')
+ parser.add_argument(
+ '--not-print-per-layer-stat',
+ '-n',
+ action='store_true',
+ help='Whether to print complexity information'
+ 'for each layer in a model')
+ args = parser.parse_args()
+ return args
+
+
+def batch_constructor(flops_model, batch_size, input_shape):
+ """Generate a batch of tensors to the model."""
+ batch = {}
+
+ img = torch.ones(()).new_empty(
+ (batch_size, *input_shape),
+ dtype=next(flops_model.parameters()).dtype,
+ device=next(flops_model.parameters()).device)
+
+ batch['img'] = img
+ return batch
+
+
+def main():
+
+ args = parse_args()
+
+ if len(args.shape) == 1:
+ input_shape = (3, args.shape[0], args.shape[0])
+ elif len(args.shape) == 2:
+ input_shape = (3, ) + tuple(args.shape)
+ else:
+ raise ValueError('invalid input shape')
+
+ model = init_pose_model(args.config)
+
+ if args.input_constructor == 'batch':
+ input_constructor = partial(batch_constructor, model, args.batch_size)
+ else:
+ input_constructor = None
+
+ if args.input_constructor == 'batch':
+ input_constructor = partial(batch_constructor, model, args.batch_size)
+ else:
+ input_constructor = None
+
+ if hasattr(model, 'forward_dummy'):
+ model.forward = model.forward_dummy
+ else:
+ raise NotImplementedError(
+ 'FLOPs counter is currently not currently supported with {}'.
+ format(model.__class__.__name__))
+
+ flops, params = get_model_complexity_info(
+ model,
+ input_shape,
+ input_constructor=input_constructor,
+ print_per_layer_stat=(not args.not_print_per_layer_stat))
+ split_line = '=' * 30
+ input_shape = (args.batch_size, ) + input_shape
+ print(f'{split_line}\nInput shape: {input_shape}\n'
+ f'Flops: {flops}\nParams: {params}\n{split_line}')
+ print('!!!Please be cautious if you use the results in papers. '
+ 'You may need to check if all ops are supported and verify that the '
+ 'flops computation is correct.')
+
+
+if __name__ == '__main__':
+ main()
diff --git a/engine/pose_estimation/third-party/ViTPose/tools/analysis/print_config.py b/engine/pose_estimation/third-party/ViTPose/tools/analysis/print_config.py
new file mode 100644
index 0000000..c3538ef
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tools/analysis/print_config.py
@@ -0,0 +1,27 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import argparse
+
+from mmcv import Config, DictAction
+
+
+def parse_args():
+ parser = argparse.ArgumentParser(description='Print the whole config')
+ parser.add_argument('config', help='config file path')
+ parser.add_argument(
+ '--options', nargs='+', action=DictAction, help='arguments in dict')
+ args = parser.parse_args()
+
+ return args
+
+
+def main():
+ args = parse_args()
+
+ cfg = Config.fromfile(args.config)
+ if args.options is not None:
+ cfg.merge_from_dict(args.options)
+ print(f'Config:\n{cfg.pretty_text}')
+
+
+if __name__ == '__main__':
+ main()
diff --git a/engine/pose_estimation/third-party/ViTPose/tools/analysis/speed_test.py b/engine/pose_estimation/third-party/ViTPose/tools/analysis/speed_test.py
new file mode 100644
index 0000000..fef9e2d
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tools/analysis/speed_test.py
@@ -0,0 +1,86 @@
+#!/usr/bin/env bash
+# Copyright (c) OpenMMLab. All rights reserved.
+import argparse
+import time
+
+import torch
+from mmcv import Config
+from mmcv.cnn import fuse_conv_bn
+from mmcv.parallel import MMDataParallel
+from mmcv.runner.fp16_utils import wrap_fp16_model
+
+from mmpose.datasets import build_dataloader, build_dataset
+from mmpose.models import build_posenet
+
+
+def parse_args():
+ parser = argparse.ArgumentParser(
+ description='MMPose benchmark a recognizer')
+ parser.add_argument('config', help='test config file path')
+ parser.add_argument('--bz', default=32, type=int, help='test config file path')
+ args = parser.parse_args()
+ return args
+
+
+def main():
+ args = parse_args()
+
+ cfg = Config.fromfile(args.config)
+
+ # Since we only care about the forward speed of the network
+ cfg.model.pretrained=None
+ cfg.model.test_cfg.flip_test=False
+ cfg.model.test_cfg.use_udp=False
+ cfg.model.test_cfg.post_process='none'
+
+ # set cudnn_benchmark
+ if cfg.get('cudnn_benchmark', False):
+ torch.backends.cudnn.benchmark = True
+
+ # build the dataloader
+ dataset = build_dataset(cfg.data.val)
+ data_loader = build_dataloader(
+ dataset,
+ samples_per_gpu=args.bz,
+ workers_per_gpu=cfg.data.workers_per_gpu,
+ dist=False,
+ shuffle=False)
+
+ # build the model and load checkpoint
+ model = build_posenet(cfg.model)
+ model = MMDataParallel(model, device_ids=[0])
+ model.eval()
+
+ # get the example data
+ for i, data in enumerate(data_loader):
+ break
+
+ # the first several iterations may be very slow so skip them
+ num_warmup = 100
+ inference_times = 100
+
+ with torch.no_grad():
+ start_time = time.perf_counter()
+
+ for i in range(num_warmup):
+ torch.cuda.synchronize()
+ model(return_loss=False, **data)
+ torch.cuda.synchronize()
+
+ elapsed = time.perf_counter() - start_time
+ print(f'warmup cost {elapsed} time')
+
+ start_time = time.perf_counter()
+
+ for i in range(inference_times):
+ torch.cuda.synchronize()
+ model(return_loss=False, **data)
+ torch.cuda.synchronize()
+
+ elapsed = time.perf_counter() - start_time
+ fps = args.bz * inference_times / elapsed
+ print(f'the fps is {fps}')
+
+
+if __name__ == '__main__':
+ main()
diff --git a/engine/pose_estimation/third-party/ViTPose/tools/deployment/mmpose2torchserve.py b/engine/pose_estimation/third-party/ViTPose/tools/deployment/mmpose2torchserve.py
new file mode 100644
index 0000000..492a45b
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tools/deployment/mmpose2torchserve.py
@@ -0,0 +1,135 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os.path as osp
+import warnings
+from argparse import ArgumentParser, Namespace
+from tempfile import TemporaryDirectory
+
+import mmcv
+import torch
+from mmcv.runner import CheckpointLoader
+
+try:
+ from model_archiver.model_packaging import package_model
+ from model_archiver.model_packaging_utils import ModelExportUtils
+except ImportError:
+ package_model = None
+
+
+def mmpose2torchserve(config_file: str,
+ checkpoint_file: str,
+ output_folder: str,
+ model_name: str,
+ model_version: str = '1.0',
+ force: bool = False):
+ """Converts MMPose model (config + checkpoint) to TorchServe `.mar`.
+
+ Args:
+ config_file:
+ In MMPose config format.
+ The contents vary for each task repository.
+ checkpoint_file:
+ In MMPose checkpoint format.
+ The contents vary for each task repository.
+ output_folder:
+ Folder where `{model_name}.mar` will be created.
+ The file created will be in TorchServe archive format.
+ model_name:
+ If not None, used for naming the `{model_name}.mar` file
+ that will be created under `output_folder`.
+ If None, `{Path(checkpoint_file).stem}` will be used.
+ model_version:
+ Model's version.
+ force:
+ If True, if there is an existing `{model_name}.mar`
+ file under `output_folder` it will be overwritten.
+ """
+
+ mmcv.mkdir_or_exist(output_folder)
+
+ config = mmcv.Config.fromfile(config_file)
+
+ with TemporaryDirectory() as tmpdir:
+ model_file = osp.join(tmpdir, 'config.py')
+ config.dump(model_file)
+ handler_path = osp.join(osp.dirname(__file__), 'mmpose_handler.py')
+ model_name = model_name or osp.splitext(
+ osp.basename(checkpoint_file))[0]
+
+ # use mmcv CheckpointLoader if checkpoint is not from a local file
+ if not osp.isfile(checkpoint_file):
+ ckpt = CheckpointLoader.load_checkpoint(checkpoint_file)
+ checkpoint_file = osp.join(tmpdir, 'checkpoint.pth')
+ with open(checkpoint_file, 'wb') as f:
+ torch.save(ckpt, f)
+
+ args = Namespace(
+ **{
+ 'model_file': model_file,
+ 'serialized_file': checkpoint_file,
+ 'handler': handler_path,
+ 'model_name': model_name,
+ 'version': model_version,
+ 'export_path': output_folder,
+ 'force': force,
+ 'requirements_file': None,
+ 'extra_files': None,
+ 'runtime': 'python',
+ 'archive_format': 'default'
+ })
+ manifest = ModelExportUtils.generate_manifest_json(args)
+ package_model(args, manifest)
+
+
+def parse_args():
+ parser = ArgumentParser(
+ description='Convert MMPose models to TorchServe `.mar` format.')
+ parser.add_argument('config', type=str, help='config file path')
+ parser.add_argument('checkpoint', type=str, help='checkpoint file path')
+ parser.add_argument(
+ '--output-folder',
+ type=str,
+ required=True,
+ help='Folder where `{model_name}.mar` will be created.')
+ parser.add_argument(
+ '--model-name',
+ type=str,
+ default=None,
+ help='If not None, used for naming the `{model_name}.mar`'
+ 'file that will be created under `output_folder`.'
+ 'If None, `{Path(checkpoint_file).stem}` will be used.')
+ parser.add_argument(
+ '--model-version',
+ type=str,
+ default='1.0',
+ help='Number used for versioning.')
+ parser.add_argument(
+ '-f',
+ '--force',
+ action='store_true',
+ help='overwrite the existing `{model_name}.mar`')
+ args = parser.parse_args()
+
+ return args
+
+
+if __name__ == '__main__':
+ args = parse_args()
+
+ # Following strings of text style are from colorama package
+ bright_style, reset_style = '\x1b[1m', '\x1b[0m'
+ red_text, blue_text = '\x1b[31m', '\x1b[34m'
+ white_background = '\x1b[107m'
+
+ msg = white_background + bright_style + red_text
+ msg += 'DeprecationWarning: This tool will be deprecated in future. '
+ msg += blue_text + 'Welcome to use the unified model deployment toolbox '
+ msg += 'MMDeploy: https://github.com/open-mmlab/mmdeploy'
+ msg += reset_style
+ warnings.warn(msg)
+
+ if package_model is None:
+ raise ImportError('`torch-model-archiver` is required.'
+ 'Try: pip install torch-model-archiver')
+
+ mmpose2torchserve(args.config, args.checkpoint, args.output_folder,
+ args.model_name, args.model_version, args.force)
diff --git a/engine/pose_estimation/third-party/ViTPose/tools/deployment/mmpose_handler.py b/engine/pose_estimation/third-party/ViTPose/tools/deployment/mmpose_handler.py
new file mode 100644
index 0000000..d7da881
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tools/deployment/mmpose_handler.py
@@ -0,0 +1,80 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import base64
+import os
+
+import mmcv
+import torch
+
+from mmpose.apis import (inference_bottom_up_pose_model,
+ inference_top_down_pose_model, init_pose_model)
+from mmpose.models.detectors import AssociativeEmbedding, TopDown
+
+try:
+ from ts.torch_handler.base_handler import BaseHandler
+except ImportError:
+ raise ImportError('Please install torchserve.')
+
+
+class MMPoseHandler(BaseHandler):
+
+ def initialize(self, context):
+ properties = context.system_properties
+ self.map_location = 'cuda' if torch.cuda.is_available() else 'cpu'
+ self.device = torch.device(self.map_location + ':' +
+ str(properties.get('gpu_id')) if torch.cuda.
+ is_available() else self.map_location)
+ self.manifest = context.manifest
+
+ model_dir = properties.get('model_dir')
+ serialized_file = self.manifest['model']['serializedFile']
+ checkpoint = os.path.join(model_dir, serialized_file)
+ self.config_file = os.path.join(model_dir, 'config.py')
+
+ self.model = init_pose_model(self.config_file, checkpoint, self.device)
+ self.initialized = True
+
+ def preprocess(self, data):
+ images = []
+
+ for row in data:
+ image = row.get('data') or row.get('body')
+ if isinstance(image, str):
+ image = base64.b64decode(image)
+ image = mmcv.imfrombytes(image)
+ images.append(image)
+
+ return images
+
+ def inference(self, data, *args, **kwargs):
+ if isinstance(self.model, TopDown):
+ results = self._inference_top_down_pose_model(data)
+ elif isinstance(self.model, (AssociativeEmbedding, )):
+ results = self._inference_bottom_up_pose_model(data)
+ else:
+ raise NotImplementedError(
+ f'Model type {type(self.model)} is not supported.')
+
+ return results
+
+ def _inference_top_down_pose_model(self, data):
+ results = []
+ for image in data:
+ # use dummy person bounding box
+ preds, _ = inference_top_down_pose_model(
+ self.model, image, person_results=None)
+ results.append(preds)
+ return results
+
+ def _inference_bottom_up_pose_model(self, data):
+ results = []
+ for image in data:
+ preds, _ = inference_bottom_up_pose_model(self.model, image)
+ results.append(preds)
+ return results
+
+ def postprocess(self, data):
+ output = [[{
+ 'keypoints': pred['keypoints'].tolist()
+ } for pred in preds] for preds in data]
+
+ return output
diff --git a/engine/pose_estimation/third-party/ViTPose/tools/deployment/pytorch2onnx.py b/engine/pose_estimation/third-party/ViTPose/tools/deployment/pytorch2onnx.py
new file mode 100644
index 0000000..5caff6e
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tools/deployment/pytorch2onnx.py
@@ -0,0 +1,165 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import argparse
+import warnings
+
+import numpy as np
+import torch
+
+from mmpose.apis import init_pose_model
+
+try:
+ import onnx
+ import onnxruntime as rt
+except ImportError as e:
+ raise ImportError(f'Please install onnx and onnxruntime first. {e}')
+
+try:
+ from mmcv.onnx.symbolic import register_extra_symbolics
+except ModuleNotFoundError:
+ raise NotImplementedError('please update mmcv to version>=1.0.4')
+
+
+def _convert_batchnorm(module):
+ """Convert the syncBNs into normal BN3ds."""
+ module_output = module
+ if isinstance(module, torch.nn.SyncBatchNorm):
+ module_output = torch.nn.BatchNorm3d(module.num_features, module.eps,
+ module.momentum, module.affine,
+ module.track_running_stats)
+ if module.affine:
+ module_output.weight.data = module.weight.data.clone().detach()
+ module_output.bias.data = module.bias.data.clone().detach()
+ # keep requires_grad unchanged
+ module_output.weight.requires_grad = module.weight.requires_grad
+ module_output.bias.requires_grad = module.bias.requires_grad
+ module_output.running_mean = module.running_mean
+ module_output.running_var = module.running_var
+ module_output.num_batches_tracked = module.num_batches_tracked
+ for name, child in module.named_children():
+ module_output.add_module(name, _convert_batchnorm(child))
+ del module
+ return module_output
+
+
+def pytorch2onnx(model,
+ input_shape,
+ opset_version=11,
+ show=False,
+ output_file='tmp.onnx',
+ verify=False):
+ """Convert pytorch model to onnx model.
+
+ Args:
+ model (:obj:`nn.Module`): The pytorch model to be exported.
+ input_shape (tuple[int]): The input tensor shape of the model.
+ opset_version (int): Opset version of onnx used. Default: 11.
+ show (bool): Determines whether to print the onnx model architecture.
+ Default: False.
+ output_file (str): Output onnx model name. Default: 'tmp.onnx'.
+ verify (bool): Determines whether to verify the onnx model.
+ Default: False.
+ """
+ model.cpu().eval()
+
+ one_img = torch.randn(input_shape)
+
+ register_extra_symbolics(opset_version)
+ torch.onnx.export(
+ model,
+ one_img,
+ output_file,
+ export_params=True,
+ keep_initializers_as_inputs=True,
+ verbose=show,
+ opset_version=opset_version)
+
+ print(f'Successfully exported ONNX model: {output_file}')
+ if verify:
+ # check by onnx
+ onnx_model = onnx.load(output_file)
+ onnx.checker.check_model(onnx_model)
+
+ # check the numerical value
+ # get pytorch output
+ pytorch_results = model(one_img)
+ if not isinstance(pytorch_results, (list, tuple)):
+ assert isinstance(pytorch_results, torch.Tensor)
+ pytorch_results = [pytorch_results]
+
+ # get onnx output
+ input_all = [node.name for node in onnx_model.graph.input]
+ input_initializer = [
+ node.name for node in onnx_model.graph.initializer
+ ]
+ net_feed_input = list(set(input_all) - set(input_initializer))
+ assert len(net_feed_input) == 1
+ sess = rt.InferenceSession(output_file)
+ onnx_results = sess.run(None,
+ {net_feed_input[0]: one_img.detach().numpy()})
+
+ # compare results
+ assert len(pytorch_results) == len(onnx_results)
+ for pt_result, onnx_result in zip(pytorch_results, onnx_results):
+ assert np.allclose(
+ pt_result.detach().cpu(), onnx_result, atol=1.e-5
+ ), 'The outputs are different between Pytorch and ONNX'
+ print('The numerical values are same between Pytorch and ONNX')
+
+
+def parse_args():
+ parser = argparse.ArgumentParser(
+ description='Convert MMPose models to ONNX')
+ parser.add_argument('config', help='test config file path')
+ parser.add_argument('checkpoint', help='checkpoint file')
+ parser.add_argument('--show', action='store_true', help='show onnx graph')
+ parser.add_argument('--output-file', type=str, default='tmp.onnx')
+ parser.add_argument('--opset-version', type=int, default=11)
+ parser.add_argument(
+ '--verify',
+ action='store_true',
+ help='verify the onnx model output against pytorch output')
+ parser.add_argument(
+ '--shape',
+ type=int,
+ nargs='+',
+ default=[1, 3, 256, 192],
+ help='input size')
+ args = parser.parse_args()
+ return args
+
+
+if __name__ == '__main__':
+ args = parse_args()
+
+ assert args.opset_version == 11, 'MMPose only supports opset 11 now'
+
+ # Following strings of text style are from colorama package
+ bright_style, reset_style = '\x1b[1m', '\x1b[0m'
+ red_text, blue_text = '\x1b[31m', '\x1b[34m'
+ white_background = '\x1b[107m'
+
+ msg = white_background + bright_style + red_text
+ msg += 'DeprecationWarning: This tool will be deprecated in future. '
+ msg += blue_text + 'Welcome to use the unified model deployment toolbox '
+ msg += 'MMDeploy: https://github.com/open-mmlab/mmdeploy'
+ msg += reset_style
+ warnings.warn(msg)
+
+ model = init_pose_model(args.config, args.checkpoint, device='cpu')
+ model = _convert_batchnorm(model)
+
+ # onnx.export does not support kwargs
+ if hasattr(model, 'forward_dummy'):
+ model.forward = model.forward_dummy
+ else:
+ raise NotImplementedError(
+ 'Please implement the forward method for exporting.')
+
+ # convert model to onnx file
+ pytorch2onnx(
+ model,
+ args.shape,
+ opset_version=args.opset_version,
+ show=args.show,
+ output_file=args.output_file,
+ verify=args.verify)
diff --git a/engine/pose_estimation/third-party/ViTPose/tools/deployment/test_torchserver.py b/engine/pose_estimation/third-party/ViTPose/tools/deployment/test_torchserver.py
new file mode 100644
index 0000000..70e27c5
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tools/deployment/test_torchserver.py
@@ -0,0 +1,79 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os
+import os.path as osp
+import warnings
+from argparse import ArgumentParser
+
+import requests
+
+from mmpose.apis import (inference_bottom_up_pose_model,
+ inference_top_down_pose_model, init_pose_model,
+ vis_pose_result)
+from mmpose.models import AssociativeEmbedding, TopDown
+
+
+def parse_args():
+ parser = ArgumentParser()
+ parser.add_argument('img', help='Image file')
+ parser.add_argument('config', help='Config file')
+ parser.add_argument('checkpoint', help='Checkpoint file')
+ parser.add_argument('model_name', help='The model name in the server')
+ parser.add_argument(
+ '--inference-addr',
+ default='127.0.0.1:8080',
+ help='Address and port of the inference server')
+ parser.add_argument(
+ '--device', default='cuda:0', help='Device used for inference')
+ parser.add_argument(
+ '--out-dir', default='vis_results', help='Visualization output path')
+ args = parser.parse_args()
+ return args
+
+
+def main(args):
+ os.makedirs(args.out_dir, exist_ok=True)
+
+ # Inference single image by native apis.
+ model = init_pose_model(args.config, args.checkpoint, device=args.device)
+ if isinstance(model, TopDown):
+ pytorch_result, _ = inference_top_down_pose_model(
+ model, args.img, person_results=None)
+ elif isinstance(model, (AssociativeEmbedding, )):
+ pytorch_result, _ = inference_bottom_up_pose_model(model, args.img)
+ else:
+ raise NotImplementedError()
+
+ vis_pose_result(
+ model,
+ args.img,
+ pytorch_result,
+ out_file=osp.join(args.out_dir, 'pytorch_result.png'))
+
+ # Inference single image by torchserve engine.
+ url = 'http://' + args.inference_addr + '/predictions/' + args.model_name
+ with open(args.img, 'rb') as image:
+ response = requests.post(url, image)
+ server_result = response.json()
+
+ vis_pose_result(
+ model,
+ args.img,
+ server_result,
+ out_file=osp.join(args.out_dir, 'torchserve_result.png'))
+
+
+if __name__ == '__main__':
+ args = parse_args()
+ main(args)
+
+ # Following strings of text style are from colorama package
+ bright_style, reset_style = '\x1b[1m', '\x1b[0m'
+ red_text, blue_text = '\x1b[31m', '\x1b[34m'
+ white_background = '\x1b[107m'
+
+ msg = white_background + bright_style + red_text
+ msg += 'DeprecationWarning: This tool will be deprecated in future. '
+ msg += blue_text + 'Welcome to use the unified model deployment toolbox '
+ msg += 'MMDeploy: https://github.com/open-mmlab/mmdeploy'
+ msg += reset_style
+ warnings.warn(msg)
diff --git a/engine/pose_estimation/third-party/ViTPose/tools/dist_test.sh b/engine/pose_estimation/third-party/ViTPose/tools/dist_test.sh
new file mode 100644
index 0000000..9dcb885
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tools/dist_test.sh
@@ -0,0 +1,11 @@
+#!/usr/bin/env bash
+# Copyright (c) OpenMMLab. All rights reserved.
+
+CONFIG=$1
+CHECKPOINT=$2
+GPUS=$3
+PORT=${PORT:-29500}
+
+PYTHONPATH="$(dirname $0)/..":$PYTHONPATH \
+python -m torch.distributed.launch --nproc_per_node=$GPUS --master_port=$PORT \
+ $(dirname "$0")/test.py $CONFIG $CHECKPOINT --launcher pytorch ${@:4}
diff --git a/engine/pose_estimation/third-party/ViTPose/tools/dist_train.sh b/engine/pose_estimation/third-party/ViTPose/tools/dist_train.sh
new file mode 100644
index 0000000..9727f53
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tools/dist_train.sh
@@ -0,0 +1,10 @@
+#!/usr/bin/env bash
+# Copyright (c) OpenMMLab. All rights reserved.
+
+CONFIG=$1
+GPUS=$2
+PORT=${PORT:-29500}
+
+PYTHONPATH="$(dirname $0)/..":$PYTHONPATH \
+python -m torch.distributed.launch --nproc_per_node=$GPUS --master_port=$PORT \
+ $(dirname "$0")/train.py $CONFIG --launcher pytorch ${@:3}
diff --git a/engine/pose_estimation/third-party/ViTPose/tools/misc/keypoints2coco_without_mmdet.py b/engine/pose_estimation/third-party/ViTPose/tools/misc/keypoints2coco_without_mmdet.py
new file mode 100644
index 0000000..63220fc
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tools/misc/keypoints2coco_without_mmdet.py
@@ -0,0 +1,146 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import json
+import os
+from argparse import ArgumentParser
+
+from mmcv import track_iter_progress
+from PIL import Image
+from xtcocotools.coco import COCO
+
+from mmpose.apis import inference_top_down_pose_model, init_pose_model
+
+
+def main():
+ """Visualize the demo images.
+
+ pose_keypoints require the json_file containing boxes.
+ """
+ parser = ArgumentParser()
+ parser.add_argument('pose_config', help='Config file for detection')
+ parser.add_argument('pose_checkpoint', help='Checkpoint file')
+ parser.add_argument('--img-root', type=str, default='', help='Image root')
+ parser.add_argument(
+ '--json-file',
+ type=str,
+ default='',
+ help='Json file containing image person bboxes in COCO format.')
+ parser.add_argument(
+ '--out-json-file',
+ type=str,
+ default='',
+ help='Output json contains pseudolabeled annotation')
+ parser.add_argument(
+ '--show',
+ action='store_true',
+ default=False,
+ help='whether to show img')
+ parser.add_argument(
+ '--device', default='cuda:0', help='Device used for inference')
+ parser.add_argument(
+ '--kpt-thr', type=float, default=0.3, help='Keypoint score threshold')
+
+ args = parser.parse_args()
+
+ coco = COCO(args.json_file)
+ # build the pose model from a config file and a checkpoint file
+ pose_model = init_pose_model(
+ args.pose_config, args.pose_checkpoint, device=args.device.lower())
+
+ dataset = pose_model.cfg.data['test']['type']
+
+ img_keys = list(coco.imgs.keys())
+
+ # optional
+ return_heatmap = False
+
+ # e.g. use ('backbone', ) to return backbone feature
+ output_layer_names = None
+
+ categories = [{'id': 1, 'name': 'person'}]
+ img_anno_dict = {'images': [], 'annotations': [], 'categories': categories}
+
+ # process each image
+ ann_uniq_id = int(0)
+ for i in track_iter_progress(range(len(img_keys))):
+ # get bounding box annotations
+ image_id = img_keys[i]
+ image = coco.loadImgs(image_id)[0]
+ image_name = os.path.join(args.img_root, image['file_name'])
+
+ width, height = Image.open(image_name).size
+ ann_ids = coco.getAnnIds(image_id)
+
+ # make person bounding boxes
+ person_results = []
+ for ann_id in ann_ids:
+ person = {}
+ ann = coco.anns[ann_id]
+ # bbox format is 'xywh'
+ person['bbox'] = ann['bbox']
+ person_results.append(person)
+
+ pose_results, returned_outputs = inference_top_down_pose_model(
+ pose_model,
+ image_name,
+ person_results,
+ bbox_thr=None,
+ format='xywh',
+ dataset=dataset,
+ return_heatmap=return_heatmap,
+ outputs=output_layer_names)
+
+ # add output of model and bboxes to dict
+ for indx, i in enumerate(pose_results):
+ pose_results[indx]['keypoints'][
+ pose_results[indx]['keypoints'][:, 2] < args.kpt_thr, :3] = 0
+ pose_results[indx]['keypoints'][
+ pose_results[indx]['keypoints'][:, 2] >= args.kpt_thr, 2] = 2
+ x = int(pose_results[indx]['bbox'][0])
+ y = int(pose_results[indx]['bbox'][1])
+ w = int(pose_results[indx]['bbox'][2] -
+ pose_results[indx]['bbox'][0])
+ h = int(pose_results[indx]['bbox'][3] -
+ pose_results[indx]['bbox'][1])
+ bbox = [x, y, w, h]
+ area = round((w * h), 0)
+
+ images = {
+ 'file_name': image_name.split('/')[-1],
+ 'height': height,
+ 'width': width,
+ 'id': int(image_id)
+ }
+
+ annotations = {
+ 'keypoints': [
+ int(i) for i in pose_results[indx]['keypoints'].reshape(
+ -1).tolist()
+ ],
+ 'num_keypoints':
+ len(pose_results[indx]['keypoints']),
+ 'area':
+ area,
+ 'iscrowd':
+ 0,
+ 'image_id':
+ int(image_id),
+ 'bbox':
+ bbox,
+ 'category_id':
+ 1,
+ 'id':
+ ann_uniq_id,
+ }
+
+ img_anno_dict['annotations'].append(annotations)
+ ann_uniq_id += 1
+
+ img_anno_dict['images'].append(images)
+
+ # create json
+ with open(args.out_json_file, 'w') as outfile:
+ json.dump(img_anno_dict, outfile, indent=2)
+
+
+if __name__ == '__main__':
+ main()
diff --git a/engine/pose_estimation/third-party/ViTPose/tools/misc/publish_model.py b/engine/pose_estimation/third-party/ViTPose/tools/misc/publish_model.py
new file mode 100644
index 0000000..393721a
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tools/misc/publish_model.py
@@ -0,0 +1,43 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import argparse
+import subprocess
+from datetime import date
+
+import torch
+
+
+def parse_args():
+ parser = argparse.ArgumentParser(
+ description='Process a checkpoint to be published')
+ parser.add_argument('in_file', help='input checkpoint filename')
+ parser.add_argument('out_file', help='output checkpoint filename')
+ args = parser.parse_args()
+ return args
+
+
+def process_checkpoint(in_file, out_file):
+ checkpoint = torch.load(in_file, map_location='cpu')
+ # remove optimizer for smaller file size
+ if 'optimizer' in checkpoint:
+ del checkpoint['optimizer']
+ # if it is necessary to remove some sensitive data in checkpoint['meta'],
+ # add the code here.
+ torch.save(checkpoint, out_file)
+ sha = subprocess.check_output(['sha256sum', out_file]).decode()
+ if out_file.endswith('.pth'):
+ out_file_name = out_file[:-4]
+ else:
+ out_file_name = out_file
+
+ date_now = date.today().strftime('%Y%m%d')
+ final_file = out_file_name + f'-{sha[:8]}_{date_now}.pth'
+ subprocess.Popen(['mv', out_file, final_file])
+
+
+def main():
+ args = parse_args()
+ process_checkpoint(args.in_file, args.out_file)
+
+
+if __name__ == '__main__':
+ main()
diff --git a/engine/pose_estimation/third-party/ViTPose/tools/model_split.py b/engine/pose_estimation/third-party/ViTPose/tools/model_split.py
new file mode 100644
index 0000000..928380a
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tools/model_split.py
@@ -0,0 +1,104 @@
+import torch
+import os
+import argparse
+import copy
+
+def parse_args():
+ parser = argparse.ArgumentParser()
+ parser.add_argument('--source', type=str)
+ parser.add_argument('--target', type=str, default=None)
+ args = parser.parse_args()
+ return args
+
+def main():
+
+ args = parse_args()
+
+ if args.target is None:
+ args.target = '/'.join(args.source.split('/')[:-1])
+
+ ckpt = torch.load(args.source, map_location='cpu')
+
+ experts = dict()
+
+ new_ckpt = copy.deepcopy(ckpt)
+
+ state_dict = new_ckpt['state_dict']
+
+ for key, value in state_dict.items():
+ if 'mlp.experts' in key:
+ experts[key] = value
+
+ keys = ckpt['state_dict'].keys()
+
+ target_expert = 0
+ new_ckpt = copy.deepcopy(ckpt)
+
+ for key in keys:
+ if 'mlp.fc2' in key:
+ value = new_ckpt['state_dict'][key]
+ value = torch.cat([value, experts[key.replace('fc2.', f'experts.{target_expert}.')]], dim=0)
+ new_ckpt['state_dict'][key] = value
+
+ torch.save(new_ckpt, os.path.join(args.target, 'coco.pth'))
+
+ names = ['aic', 'mpii', 'ap10k', 'apt36k','wholebody']
+ num_keypoints = [14, 16, 17, 17, 133]
+ weight_names = ['keypoint_head.deconv_layers.0.weight',
+ 'keypoint_head.deconv_layers.1.weight',
+ 'keypoint_head.deconv_layers.1.bias',
+ 'keypoint_head.deconv_layers.1.running_mean',
+ 'keypoint_head.deconv_layers.1.running_var',
+ 'keypoint_head.deconv_layers.1.num_batches_tracked',
+ 'keypoint_head.deconv_layers.3.weight',
+ 'keypoint_head.deconv_layers.4.weight',
+ 'keypoint_head.deconv_layers.4.bias',
+ 'keypoint_head.deconv_layers.4.running_mean',
+ 'keypoint_head.deconv_layers.4.running_var',
+ 'keypoint_head.deconv_layers.4.num_batches_tracked',
+ 'keypoint_head.final_layer.weight',
+ 'keypoint_head.final_layer.bias']
+
+ exist_range = True
+
+ for i in range(5):
+
+ new_ckpt = copy.deepcopy(ckpt)
+
+ target_expert = i + 1
+
+ for key in keys:
+ if 'mlp.fc2' in key:
+ expert_key = key.replace('fc2.', f'experts.{target_expert}.')
+ if expert_key in experts:
+ value = new_ckpt['state_dict'][key]
+ value = torch.cat([value, experts[expert_key]], dim=0)
+ else:
+ exist_range = False
+
+ new_ckpt['state_dict'][key] = value
+
+ if not exist_range:
+ break
+
+ for tensor_name in weight_names:
+ new_ckpt['state_dict'][tensor_name] = new_ckpt['state_dict'][tensor_name.replace('keypoint_head', f'associate_keypoint_heads.{i}')]
+
+ for tensor_name in ['keypoint_head.final_layer.weight', 'keypoint_head.final_layer.bias']:
+ new_ckpt['state_dict'][tensor_name] = new_ckpt['state_dict'][tensor_name][:num_keypoints[i]]
+
+ # remove unnecessary part in the state dict
+ for j in range(5):
+ # remove associate part
+ for tensor_name in weight_names:
+ new_ckpt['state_dict'].pop(tensor_name.replace('keypoint_head', f'associate_keypoint_heads.{j}'))
+ # remove expert part
+ keys = new_ckpt['state_dict'].keys()
+ for key in list(keys):
+ if 'expert' in keys:
+ new_ckpt['state_dict'].pop(key)
+
+ torch.save(new_ckpt, os.path.join(args.target, f'{names[i]}.pth'))
+
+if __name__ == '__main__':
+ main()
diff --git a/engine/pose_estimation/third-party/ViTPose/tools/slurm_test.sh b/engine/pose_estimation/third-party/ViTPose/tools/slurm_test.sh
new file mode 100644
index 0000000..c528dc9
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tools/slurm_test.sh
@@ -0,0 +1,25 @@
+#!/usr/bin/env bash
+# Copyright (c) OpenMMLab. All rights reserved.
+
+set -x
+
+PARTITION=$1
+JOB_NAME=$2
+CONFIG=$3
+CHECKPOINT=$4
+GPUS=${GPUS:-8}
+GPUS_PER_NODE=${GPUS_PER_NODE:-8}
+CPUS_PER_TASK=${CPUS_PER_TASK:-5}
+PY_ARGS=${@:5}
+SRUN_ARGS=${SRUN_ARGS:-""}
+
+PYTHONPATH="$(dirname $0)/..":$PYTHONPATH \
+srun -p ${PARTITION} \
+ --job-name=${JOB_NAME} \
+ --gres=gpu:${GPUS_PER_NODE} \
+ --ntasks=${GPUS} \
+ --ntasks-per-node=${GPUS_PER_NODE} \
+ --cpus-per-task=${CPUS_PER_TASK} \
+ --kill-on-bad-exit=1 \
+ ${SRUN_ARGS} \
+ python -u tools/test.py ${CONFIG} ${CHECKPOINT} --launcher="slurm" ${PY_ARGS}
diff --git a/engine/pose_estimation/third-party/ViTPose/tools/slurm_train.sh b/engine/pose_estimation/third-party/ViTPose/tools/slurm_train.sh
new file mode 100644
index 0000000..c3b6549
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tools/slurm_train.sh
@@ -0,0 +1,25 @@
+#!/usr/bin/env bash
+# Copyright (c) OpenMMLab. All rights reserved.
+
+set -x
+
+PARTITION=$1
+JOB_NAME=$2
+CONFIG=$3
+WORK_DIR=$4
+GPUS=${GPUS:-8}
+GPUS_PER_NODE=${GPUS_PER_NODE:-8}
+CPUS_PER_TASK=${CPUS_PER_TASK:-5}
+SRUN_ARGS=${SRUN_ARGS:-""}
+PY_ARGS=${@:5}
+
+PYTHONPATH="$(dirname $0)/..":$PYTHONPATH \
+srun -p ${PARTITION} \
+ --job-name=${JOB_NAME} \
+ --gres=gpu:${GPUS_PER_NODE} \
+ --ntasks=${GPUS} \
+ --ntasks-per-node=${GPUS_PER_NODE} \
+ --cpus-per-task=${CPUS_PER_TASK} \
+ --kill-on-bad-exit=1 \
+ ${SRUN_ARGS} \
+ python -u tools/train.py ${CONFIG} --work-dir=${WORK_DIR} --launcher="slurm" ${PY_ARGS}
diff --git a/engine/pose_estimation/third-party/ViTPose/tools/test.py b/engine/pose_estimation/third-party/ViTPose/tools/test.py
new file mode 100644
index 0000000..d153992
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tools/test.py
@@ -0,0 +1,184 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import argparse
+import os
+import os.path as osp
+import warnings
+
+import mmcv
+import torch
+from mmcv import Config, DictAction
+from mmcv.cnn import fuse_conv_bn
+from mmcv.parallel import MMDataParallel, MMDistributedDataParallel
+from mmcv.runner import get_dist_info, init_dist, load_checkpoint
+
+from mmpose.apis import multi_gpu_test, single_gpu_test
+from mmpose.datasets import build_dataloader, build_dataset
+from mmpose.models import build_posenet
+from mmpose.utils import setup_multi_processes
+
+try:
+ from mmcv.runner import wrap_fp16_model
+except ImportError:
+ warnings.warn('auto_fp16 from mmpose will be deprecated from v0.15.0'
+ 'Please install mmcv>=1.1.4')
+ from mmpose.core import wrap_fp16_model
+
+
+def parse_args():
+ parser = argparse.ArgumentParser(description='mmpose test model')
+ parser.add_argument('config', help='test config file path')
+ parser.add_argument('checkpoint', help='checkpoint file')
+ parser.add_argument('--out', help='output result file')
+ parser.add_argument(
+ '--work-dir', help='the dir to save evaluation results')
+ parser.add_argument(
+ '--fuse-conv-bn',
+ action='store_true',
+ help='Whether to fuse conv and bn, this will slightly increase'
+ 'the inference speed')
+ parser.add_argument(
+ '--gpu-id',
+ type=int,
+ default=0,
+ help='id of gpu to use '
+ '(only applicable to non-distributed testing)')
+ parser.add_argument(
+ '--eval',
+ default=None,
+ nargs='+',
+ help='evaluation metric, which depends on the dataset,'
+ ' e.g., "mAP" for MSCOCO')
+ parser.add_argument(
+ '--gpu_collect',
+ action='store_true',
+ help='whether to use gpu to collect results')
+ parser.add_argument('--tmpdir', help='tmp dir for writing some results')
+ parser.add_argument(
+ '--cfg-options',
+ nargs='+',
+ action=DictAction,
+ default={},
+ help='override some settings in the used config, the key-value pair '
+ 'in xxx=yyy format will be merged into config file. For example, '
+ "'--cfg-options model.backbone.depth=18 model.backbone.with_cp=True'")
+ parser.add_argument(
+ '--launcher',
+ choices=['none', 'pytorch', 'slurm', 'mpi'],
+ default='none',
+ help='job launcher')
+ parser.add_argument('--local_rank', type=int, default=0)
+ args = parser.parse_args()
+ if 'LOCAL_RANK' not in os.environ:
+ os.environ['LOCAL_RANK'] = str(args.local_rank)
+ return args
+
+
+def merge_configs(cfg1, cfg2):
+ # Merge cfg2 into cfg1
+ # Overwrite cfg1 if repeated, ignore if value is None.
+ cfg1 = {} if cfg1 is None else cfg1.copy()
+ cfg2 = {} if cfg2 is None else cfg2
+ for k, v in cfg2.items():
+ if v:
+ cfg1[k] = v
+ return cfg1
+
+
+def main():
+ args = parse_args()
+
+ cfg = Config.fromfile(args.config)
+
+ if args.cfg_options is not None:
+ cfg.merge_from_dict(args.cfg_options)
+
+ # set multi-process settings
+ setup_multi_processes(cfg)
+
+ # set cudnn_benchmark
+ if cfg.get('cudnn_benchmark', False):
+ torch.backends.cudnn.benchmark = True
+ cfg.model.pretrained = None
+ cfg.data.test.test_mode = True
+
+ # work_dir is determined in this priority: CLI > segment in file > filename
+ if args.work_dir is not None:
+ # update configs according to CLI args if args.work_dir is not None
+ cfg.work_dir = args.work_dir
+ elif cfg.get('work_dir', None) is None:
+ # use config filename as default work_dir if cfg.work_dir is None
+ cfg.work_dir = osp.join('./work_dirs',
+ osp.splitext(osp.basename(args.config))[0])
+
+ mmcv.mkdir_or_exist(osp.abspath(cfg.work_dir))
+
+ # init distributed env first, since logger depends on the dist info.
+ if args.launcher == 'none':
+ distributed = False
+ else:
+ distributed = True
+ init_dist(args.launcher, **cfg.dist_params)
+
+ # build the dataloader
+ dataset = build_dataset(cfg.data.test, dict(test_mode=True))
+ # step 1: give default values and override (if exist) from cfg.data
+ loader_cfg = {
+ **dict(seed=cfg.get('seed'), drop_last=False, dist=distributed),
+ **({} if torch.__version__ != 'parrots' else dict(
+ prefetch_num=2,
+ pin_memory=False,
+ )),
+ **dict((k, cfg.data[k]) for k in [
+ 'seed',
+ 'prefetch_num',
+ 'pin_memory',
+ 'persistent_workers',
+ ] if k in cfg.data)
+ }
+ # step2: cfg.data.test_dataloader has higher priority
+ test_loader_cfg = {
+ **loader_cfg,
+ **dict(shuffle=False, drop_last=False),
+ **dict(workers_per_gpu=cfg.data.get('workers_per_gpu', 1)),
+ **dict(samples_per_gpu=cfg.data.get('samples_per_gpu', 1)),
+ **cfg.data.get('test_dataloader', {})
+ }
+ data_loader = build_dataloader(dataset, **test_loader_cfg)
+
+ # build the model and load checkpoint
+ model = build_posenet(cfg.model)
+ fp16_cfg = cfg.get('fp16', None)
+ if fp16_cfg is not None:
+ wrap_fp16_model(model)
+ load_checkpoint(model, args.checkpoint, map_location='cpu')
+
+ if args.fuse_conv_bn:
+ model = fuse_conv_bn(model)
+
+ if not distributed:
+ model = MMDataParallel(model, device_ids=[args.gpu_id])
+ outputs = single_gpu_test(model, data_loader)
+ else:
+ model = MMDistributedDataParallel(
+ model.cuda(),
+ device_ids=[torch.cuda.current_device()],
+ broadcast_buffers=False)
+ outputs = multi_gpu_test(model, data_loader, args.tmpdir,
+ args.gpu_collect)
+
+ rank, _ = get_dist_info()
+ eval_config = cfg.get('evaluation', {})
+ eval_config = merge_configs(eval_config, dict(metric=args.eval))
+
+ if rank == 0:
+ if args.out:
+ print(f'\nwriting results to {args.out}')
+ mmcv.dump(outputs, args.out)
+
+ results = dataset.evaluate(outputs, cfg.work_dir, **eval_config)
+ for k, v in sorted(results.items()):
+ print(f'{k}: {v}')
+
+
+if __name__ == '__main__':
+ main()
diff --git a/engine/pose_estimation/third-party/ViTPose/tools/train.py b/engine/pose_estimation/third-party/ViTPose/tools/train.py
new file mode 100644
index 0000000..2e1f707
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tools/train.py
@@ -0,0 +1,195 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import argparse
+import copy
+import os
+import os.path as osp
+import time
+import warnings
+
+import mmcv
+import torch
+from mmcv import Config, DictAction
+from mmcv.runner import get_dist_info, init_dist, set_random_seed
+from mmcv.utils import get_git_hash
+
+from mmpose import __version__
+from mmpose.apis import init_random_seed, train_model
+from mmpose.datasets import build_dataset
+from mmpose.models import build_posenet
+from mmpose.utils import collect_env, get_root_logger, setup_multi_processes
+import mmcv_custom
+
+def parse_args():
+ parser = argparse.ArgumentParser(description='Train a pose model')
+ parser.add_argument('config', help='train config file path')
+ parser.add_argument('--work-dir', help='the dir to save logs and models')
+ parser.add_argument(
+ '--resume-from', help='the checkpoint file to resume from')
+ parser.add_argument(
+ '--no-validate',
+ action='store_true',
+ help='whether not to evaluate the checkpoint during training')
+ group_gpus = parser.add_mutually_exclusive_group()
+ group_gpus.add_argument(
+ '--gpus',
+ type=int,
+ help='(Deprecated, please use --gpu-id) number of gpus to use '
+ '(only applicable to non-distributed training)')
+ group_gpus.add_argument(
+ '--gpu-ids',
+ type=int,
+ nargs='+',
+ help='(Deprecated, please use --gpu-id) ids of gpus to use '
+ '(only applicable to non-distributed training)')
+ group_gpus.add_argument(
+ '--gpu-id',
+ type=int,
+ default=0,
+ help='id of gpu to use '
+ '(only applicable to non-distributed training)')
+ parser.add_argument('--seed', type=int, default=None, help='random seed')
+ parser.add_argument(
+ '--deterministic',
+ action='store_true',
+ help='whether to set deterministic options for CUDNN backend.')
+ parser.add_argument(
+ '--cfg-options',
+ nargs='+',
+ action=DictAction,
+ default={},
+ help='override some settings in the used config, the key-value pair '
+ 'in xxx=yyy format will be merged into config file. For example, '
+ "'--cfg-options model.backbone.depth=18 model.backbone.with_cp=True'")
+ parser.add_argument(
+ '--launcher',
+ choices=['none', 'pytorch', 'slurm', 'mpi'],
+ default='none',
+ help='job launcher')
+ parser.add_argument('--local_rank', type=int, default=0)
+ parser.add_argument(
+ '--autoscale-lr',
+ action='store_true',
+ help='automatically scale lr with the number of gpus')
+ args = parser.parse_args()
+ if 'LOCAL_RANK' not in os.environ:
+ os.environ['LOCAL_RANK'] = str(args.local_rank)
+
+ return args
+
+
+def main():
+ args = parse_args()
+
+ cfg = Config.fromfile(args.config)
+
+ if args.cfg_options is not None:
+ cfg.merge_from_dict(args.cfg_options)
+
+ # set multi-process settings
+ setup_multi_processes(cfg)
+
+ # set cudnn_benchmark
+ if cfg.get('cudnn_benchmark', False):
+ torch.backends.cudnn.benchmark = True
+
+ # work_dir is determined in this priority: CLI > segment in file > filename
+ if args.work_dir is not None:
+ # update configs according to CLI args if args.work_dir is not None
+ cfg.work_dir = args.work_dir
+ elif cfg.get('work_dir', None) is None:
+ # use config filename as default work_dir if cfg.work_dir is None
+ cfg.work_dir = osp.join('./work_dirs',
+ osp.splitext(osp.basename(args.config))[0])
+ if args.resume_from is not None:
+ cfg.resume_from = args.resume_from
+ if args.gpus is not None:
+ cfg.gpu_ids = range(1)
+ warnings.warn('`--gpus` is deprecated because we only support '
+ 'single GPU mode in non-distributed training. '
+ 'Use `gpus=1` now.')
+ if args.gpu_ids is not None:
+ cfg.gpu_ids = args.gpu_ids[0:1]
+ warnings.warn('`--gpu-ids` is deprecated, please use `--gpu-id`. '
+ 'Because we only support single GPU mode in '
+ 'non-distributed training. Use the first GPU '
+ 'in `gpu_ids` now.')
+ if args.gpus is None and args.gpu_ids is None:
+ cfg.gpu_ids = [args.gpu_id]
+
+ if args.autoscale_lr:
+ # apply the linear scaling rule (https://arxiv.org/abs/1706.02677)
+ cfg.optimizer['lr'] = cfg.optimizer['lr'] * len(cfg.gpu_ids) / 8
+
+ # init distributed env first, since logger depends on the dist info.
+ if args.launcher == 'none':
+ distributed = False
+ if len(cfg.gpu_ids) > 1:
+ warnings.warn(
+ f'We treat {cfg.gpu_ids} as gpu-ids, and reset to '
+ f'{cfg.gpu_ids[0:1]} as gpu-ids to avoid potential error in '
+ 'non-distribute training time.')
+ cfg.gpu_ids = cfg.gpu_ids[0:1]
+ else:
+ distributed = True
+ init_dist(args.launcher, **cfg.dist_params)
+ # re-set gpu_ids with distributed training mode
+ _, world_size = get_dist_info()
+ cfg.gpu_ids = range(world_size)
+
+ # create work_dir
+ mmcv.mkdir_or_exist(osp.abspath(cfg.work_dir))
+ # init the logger before other steps
+ timestamp = time.strftime('%Y%m%d_%H%M%S', time.localtime())
+ log_file = osp.join(cfg.work_dir, f'{timestamp}.log')
+ logger = get_root_logger(log_file=log_file, log_level=cfg.log_level)
+
+ # init the meta dict to record some important information such as
+ # environment info and seed, which will be logged
+ meta = dict()
+ # log env info
+ env_info_dict = collect_env()
+ env_info = '\n'.join([(f'{k}: {v}') for k, v in env_info_dict.items()])
+ dash_line = '-' * 60 + '\n'
+ logger.info('Environment info:\n' + dash_line + env_info + '\n' +
+ dash_line)
+ meta['env_info'] = env_info
+
+ # log some basic info
+ logger.info(f'Distributed training: {distributed}')
+ logger.info(f'Config:\n{cfg.pretty_text}')
+
+ # set random seeds
+ seed = init_random_seed(args.seed)
+ logger.info(f'Set random seed to {seed}, '
+ f'deterministic: {args.deterministic}')
+ set_random_seed(seed, deterministic=args.deterministic)
+ cfg.seed = seed
+ meta['seed'] = seed
+
+ model = build_posenet(cfg.model)
+ datasets = [build_dataset(cfg.data.train)]
+
+ if len(cfg.workflow) == 2:
+ val_dataset = copy.deepcopy(cfg.data.val)
+ val_dataset.pipeline = cfg.data.train.pipeline
+ datasets.append(build_dataset(val_dataset))
+
+ if cfg.checkpoint_config is not None:
+ # save mmpose version, config file content
+ # checkpoints as meta data
+ cfg.checkpoint_config.meta = dict(
+ mmpose_version=__version__ + get_git_hash(digits=7),
+ config=cfg.pretty_text,
+ )
+ train_model(
+ model,
+ datasets,
+ cfg,
+ distributed=distributed,
+ validate=(not args.no_validate),
+ timestamp=timestamp,
+ meta=meta)
+
+
+if __name__ == '__main__':
+ main()
diff --git a/engine/pose_estimation/third-party/ViTPose/tools/webcam/README.md b/engine/pose_estimation/third-party/ViTPose/tools/webcam/README.md
new file mode 100644
index 0000000..30960fd
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tools/webcam/README.md
@@ -0,0 +1,28 @@
+# MMPose Webcam API
+
+MMPose Webcam API is a handy tool to develop interactive webcam applications with MMPose functions.
+
+
+
+MMPose Webcam API Overview
+
+
+## Requirements
+
+* Python >= 3.7.0
+* MMPose >= 0.23.0
+* MMDetection >= 2.21.0
+
+## Tutorials
+
+* [Get started with MMPose Webcam API (Chinese)](/tools/webcam/docs/get_started_cn.md)
+* [Build a Webcam App: A Step-by-step Instruction (Chinese)](/tools/webcam/docs/example_cn.md)
+
+## Examples
+
+* [Pose Estimation](/tools/webcam/configs/examples/): A simple example to estimate and visualize human/animal pose.
+* [Eye Effects](/tools/webcam/configs/eyes/): Apply sunglasses and bug-eye effects.
+* [Face Swap](/tools/webcam/configs/face_swap/): Everybody gets someone else's face.
+* [Meow Dwen Dwen](/tools/webcam/configs/meow_dwen_dwen/): Dress up your cat in Bing Dwen Dwen costume.
+* [Super Saiyan](/tools/webcam/configs/supersaiyan/): Super Saiyan transformation!
+* [New Year](/tools/webcam/configs/newyear/): Set off some firecrackers to celebrate Chinese New Year.
diff --git a/engine/pose_estimation/third-party/ViTPose/tools/webcam/configs/background/README.md b/engine/pose_estimation/third-party/ViTPose/tools/webcam/configs/background/README.md
new file mode 100644
index 0000000..7be8782
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tools/webcam/configs/background/README.md
@@ -0,0 +1,73 @@
+# Matting Effects
+
+We can apply background matting to the videos.
+
+## Instruction
+
+### Get started
+
+Launch the demo from the mmpose root directory:
+
+```shell
+python tools/webcam/run_webcam.py --config tools/webcam/configs/background/background.py
+```
+
+### Hotkeys
+
+| Hotkey | Function |
+| -- | -- |
+| b | Toggle the background matting effect on/off. |
+| h | Show help information. |
+| m | Show the monitoring information. |
+| q | Exit. |
+
+Note that the demo will automatically save the output video into a file `record.mp4`.
+
+### Configuration
+
+- **Choose a detection model**
+
+Users can choose detection models from the [MMDetection Model Zoo](https://mmdetection.readthedocs.io/en/v2.20.0/model_zoo.html). Just set the `model_config` and `model_checkpoint` in the detector node accordingly, and the model will be automatically downloaded and loaded.
+Note that in order to perform background matting, the model should be able to produce segmentation masks.
+
+```python
+# 'DetectorNode':
+# This node performs object detection from the frame image using an
+# MMDetection model.
+dict(
+ type='DetectorNode',
+ name='Detector',
+ model_config='demo/mmdetection_cfg/mask_rcnn_r50_fpn_2x_coco.py',
+ model_checkpoint='https://download.openmmlab.com/'
+ 'mmdetection/v2.0/mask_rcnn/mask_rcnn_r50_fpn_2x_coco/'
+ 'mask_rcnn_r50_fpn_2x_coco_bbox_mAP-0.392'
+ '__segm_mAP-0.354_20200505_003907-3e542a40.pth',
+ input_buffer='_input_', # `_input_` is a runner-reserved buffer
+ output_buffer='det_result'),
+```
+
+- **Run the demo without GPU**
+
+If you don't have GPU and CUDA in your device, the demo can run with only CPU by setting `device='cpu'` in all model nodes. For example:
+
+```python
+dict(
+ type='DetectorNode',
+ name='Detector',
+ model_config='demo/mmdetection_cfg/mask_rcnn_r50_fpn_2x_coco.py',
+ model_checkpoint='https://download.openmmlab.com/'
+ 'mmdetection/v2.0/mask_rcnn/mask_rcnn_r50_fpn_2x_coco/'
+ 'mask_rcnn_r50_fpn_2x_coco_bbox_mAP-0.392'
+ '__segm_mAP-0.354_20200505_003907-3e542a40.pth',
+ device='cpu',
+ input_buffer='_input_', # `_input_` is a runner-reserved buffer
+ output_buffer='det_result'),
+```
+
+- **Debug webcam and display**
+
+You can launch the webcam runner with a debug config:
+
+```shell
+python tools/webcam/run_webcam.py --config tools/webcam/configs/examples/test_camera.py
+```
diff --git a/engine/pose_estimation/third-party/ViTPose/tools/webcam/configs/background/background.py b/engine/pose_estimation/third-party/ViTPose/tools/webcam/configs/background/background.py
new file mode 100644
index 0000000..fb9f4d6
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tools/webcam/configs/background/background.py
@@ -0,0 +1,93 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+runner = dict(
+ # Basic configurations of the runner
+ name='Matting Effects',
+ camera_id=0,
+ camera_fps=10,
+ synchronous=False,
+ # Define nodes.
+ # The configuration of a node usually includes:
+ # 1. 'type': Node class name
+ # 2. 'name': Node name
+ # 3. I/O buffers (e.g. 'input_buffer', 'output_buffer'): specify the
+ # input and output buffer names. This may depend on the node class.
+ # 4. 'enable_key': assign a hot-key to toggle enable/disable this node.
+ # This may depend on the node class.
+ # 5. Other class-specific arguments
+ nodes=[
+ # 'DetectorNode':
+ # This node performs object detection from the frame image using an
+ # MMDetection model.
+ dict(
+ type='DetectorNode',
+ name='Detector',
+ model_config='demo/mmdetection_cfg/mask_rcnn_r50_fpn_2x_coco.py',
+ model_checkpoint='https://download.openmmlab.com/'
+ 'mmdetection/v2.0/mask_rcnn/mask_rcnn_r50_fpn_2x_coco/'
+ 'mask_rcnn_r50_fpn_2x_coco_bbox_mAP-0.392'
+ '__segm_mAP-0.354_20200505_003907-3e542a40.pth',
+ input_buffer='_input_', # `_input_` is a runner-reserved buffer
+ output_buffer='det_result'),
+ # 'TopDownPoseEstimatorNode':
+ # This node performs keypoint detection from the frame image using an
+ # MMPose top-down model. Detection results is needed.
+ dict(
+ type='TopDownPoseEstimatorNode',
+ name='Human Pose Estimator',
+ model_config='configs/wholebody/2d_kpt_sview_rgb_img/'
+ 'topdown_heatmap/coco-wholebody/'
+ 'vipnas_mbv3_coco_wholebody_256x192_dark.py',
+ model_checkpoint='https://openmmlab-share.oss-cn-hangz'
+ 'hou.aliyuncs.com/mmpose/top_down/vipnas/vipnas_mbv3_co'
+ 'co_wholebody_256x192_dark-e2158108_20211205.pth',
+ cls_names=['person'],
+ input_buffer='det_result',
+ output_buffer='human_pose'),
+ # 'ModelResultBindingNode':
+ # This node binds the latest model inference result with the current
+ # frame. (This means the frame image and inference result may be
+ # asynchronous).
+ dict(
+ type='ModelResultBindingNode',
+ name='ResultBinder',
+ frame_buffer='_frame_', # `_frame_` is a runner-reserved buffer
+ result_buffer='human_pose',
+ output_buffer='frame'),
+ # 'MattingNode':
+ # This node draw the matting visualization result in the frame image.
+ # mask results is needed.
+ dict(
+ type='BackgroundNode',
+ name='Visualizer',
+ enable_key='b',
+ enable=True,
+ frame_buffer='frame',
+ output_buffer='vis_bg',
+ cls_names=['person']),
+ # 'NoticeBoardNode':
+ # This node show a notice board with given content, e.g. help
+ # information.
+ dict(
+ type='NoticeBoardNode',
+ name='Helper',
+ enable_key='h',
+ frame_buffer='vis_bg',
+ output_buffer='vis',
+ content_lines=[
+ 'This is a demo for background changing effects. Have fun!',
+ '', 'Hot-keys:', '"b": Change background',
+ '"h": Show help information',
+ '"m": Show diagnostic information', '"q": Exit'
+ ],
+ ),
+ # 'MonitorNode':
+ # This node show diagnostic information in the frame image. It can
+ # be used for debugging or monitoring system resource status.
+ dict(
+ type='MonitorNode',
+ name='Monitor',
+ enable_key='m',
+ enable=False,
+ frame_buffer='vis',
+ output_buffer='_display_') # `_frame_` is a runner-reserved buffer
+ ])
diff --git a/engine/pose_estimation/third-party/ViTPose/tools/webcam/configs/examples/README.md b/engine/pose_estimation/third-party/ViTPose/tools/webcam/configs/examples/README.md
new file mode 100644
index 0000000..ec9b961
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tools/webcam/configs/examples/README.md
@@ -0,0 +1,110 @@
+# Pose Estimation Demo
+
+This demo performs human bounding box and keypoint detection, and visualizes results.
+
+
+
+
+
+## Instruction
+
+### Get started
+
+Launch the demo from the mmpose root directory:
+
+```shell
+python tools/webcam/run_webcam.py --config tools/webcam/configs/examples/pose_estimation.py
+```
+
+### Hotkeys
+
+| Hotkey | Function |
+| -- | -- |
+| v | Toggle the pose visualization on/off. |
+| h | Show help information. |
+| m | Show the monitoring information. |
+| q | Exit. |
+
+Note that the demo will automatically save the output video into a file `record.mp4`.
+
+### Configuration
+
+- **Choose a detection model**
+
+Users can choose detection models from the [MMDetection Model Zoo](https://mmdetection.readthedocs.io/en/v2.20.0/model_zoo.html). Just set the `model_config` and `model_checkpoint` in the detector node accordingly, and the model will be automatically downloaded and loaded.
+
+```python
+# 'DetectorNode':
+ # This node performs object detection from the frame image using an
+ # MMDetection model.
+dict(
+ type='DetectorNode',
+ name='Detector',
+ model_config='demo/mmdetection_cfg/'
+ 'ssdlite_mobilenetv2_scratch_600e_coco.py',
+ model_checkpoint='https://download.openmmlab.com'
+ '/mmdetection/v2.0/ssd/'
+ 'ssdlite_mobilenetv2_scratch_600e_coco/ssdlite_mobilenetv2_'
+ 'scratch_600e_coco_20210629_110627-974d9307.pth',
+ input_buffer='_input_',
+ output_buffer='det_result')
+```
+
+- **Choose a or more pose models**
+
+In this demo we use two [top-down](https://github.com/open-mmlab/mmpose/tree/master/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap) pose estimation models for humans and animals respectively. Users can choose models from the [MMPose Model Zoo](https://mmpose.readthedocs.io/en/latest/modelzoo.html). To apply different pose models on different instance types, you can add multiple pose estimator nodes with `cls_names` set accordingly.
+
+```python
+# 'TopDownPoseEstimatorNode':
+# This node performs keypoint detection from the frame image using an
+# MMPose top-down model. Detection results is needed.
+dict(
+ type='TopDownPoseEstimatorNode',
+ name='Human Pose Estimator',
+ model_config='configs/wholebody/2d_kpt_sview_rgb_img/'
+ 'topdown_heatmap/coco-wholebody/'
+ 'vipnas_mbv3_coco_wholebody_256x192_dark.py',
+ model_checkpoint='https://openmmlab-share.oss-cn-hangz'
+ 'hou.aliyuncs.com/mmpose/top_down/vipnas/vipnas_mbv3_co'
+ 'co_wholebody_256x192_dark-e2158108_20211205.pth',
+ cls_names=['person'],
+ input_buffer='det_result',
+ output_buffer='human_pose'),
+dict(
+ type='TopDownPoseEstimatorNode',
+ name='Animal Pose Estimator',
+ model_config='configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap'
+ '/animalpose/hrnet_w32_animalpose_256x256.py',
+ model_checkpoint='https://download.openmmlab.com/mmpose/animal/'
+ 'hrnet/hrnet_w32_animalpose_256x256-1aa7f075_20210426.pth',
+ cls_names=['cat', 'dog', 'horse', 'sheep', 'cow'],
+ input_buffer='human_pose',
+ output_buffer='animal_pose')
+```
+
+- **Run the demo without GPU**
+
+If you don't have GPU and CUDA in your device, the demo can run with only CPU by setting `device='cpu'` in all model nodes. For example:
+
+```python
+dict(
+ type='DetectorNode',
+ name='Detector',
+ model_config='demo/mmdetection_cfg/'
+ 'ssdlite_mobilenetv2_scratch_600e_coco.py',
+ model_checkpoint='https://download.openmmlab.com'
+ '/mmdetection/v2.0/ssd/'
+ 'ssdlite_mobilenetv2_scratch_600e_coco/ssdlite_mobilenetv2_'
+ 'scratch_600e_coco_20210629_110627-974d9307.pth',
+ device='cpu',
+ input_buffer='_input_',
+ output_buffer='det_result')
+```
+
+- **Debug webcam and display**
+
+You can lanch the webcam runner with a debug config:
+
+```shell
+python tools/webcam/run_webcam.py --config tools/webcam/configs/examples/test_camera.py
+```
diff --git a/engine/pose_estimation/third-party/ViTPose/tools/webcam/configs/examples/pose_estimation.py b/engine/pose_estimation/third-party/ViTPose/tools/webcam/configs/examples/pose_estimation.py
new file mode 100644
index 0000000..471333a
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tools/webcam/configs/examples/pose_estimation.py
@@ -0,0 +1,115 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+runner = dict(
+ # Basic configurations of the runner
+ name='Pose Estimation',
+ camera_id=0,
+ camera_fps=20,
+ synchronous=False,
+ # Define nodes.
+ # The configuration of a node usually includes:
+ # 1. 'type': Node class name
+ # 2. 'name': Node name
+ # 3. I/O buffers (e.g. 'input_buffer', 'output_buffer'): specify the
+ # input and output buffer names. This may depend on the node class.
+ # 4. 'enable_key': assign a hot-key to toggle enable/disable this node.
+ # This may depend on the node class.
+ # 5. Other class-specific arguments
+ nodes=[
+ # 'DetectorNode':
+ # This node performs object detection from the frame image using an
+ # MMDetection model.
+ dict(
+ type='DetectorNode',
+ name='Detector',
+ model_config='demo/mmdetection_cfg/'
+ 'ssdlite_mobilenetv2_scratch_600e_coco.py',
+ model_checkpoint='https://download.openmmlab.com'
+ '/mmdetection/v2.0/ssd/'
+ 'ssdlite_mobilenetv2_scratch_600e_coco/ssdlite_mobilenetv2_'
+ 'scratch_600e_coco_20210629_110627-974d9307.pth',
+ input_buffer='_input_', # `_input_` is a runner-reserved buffer
+ output_buffer='det_result'),
+ # 'TopDownPoseEstimatorNode':
+ # This node performs keypoint detection from the frame image using an
+ # MMPose top-down model. Detection results is needed.
+ dict(
+ type='TopDownPoseEstimatorNode',
+ name='Human Pose Estimator',
+ model_config='configs/wholebody/2d_kpt_sview_rgb_img/'
+ 'topdown_heatmap/coco-wholebody/'
+ 'vipnas_mbv3_coco_wholebody_256x192_dark.py',
+ model_checkpoint='https://download.openmmlab.com/mmpose/top_down/'
+ 'vipnas/vipnas_mbv3_coco_wholebody_256x192_dark'
+ '-e2158108_20211205.pth',
+ cls_names=['person'],
+ input_buffer='det_result',
+ output_buffer='human_pose'),
+ dict(
+ type='TopDownPoseEstimatorNode',
+ name='Animal Pose Estimator',
+ model_config='configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap'
+ '/animalpose/hrnet_w32_animalpose_256x256.py',
+ model_checkpoint='https://download.openmmlab.com/mmpose/animal/'
+ 'hrnet/hrnet_w32_animalpose_256x256-1aa7f075_20210426.pth',
+ cls_names=['cat', 'dog', 'horse', 'sheep', 'cow'],
+ input_buffer='human_pose',
+ output_buffer='animal_pose'),
+ # 'ModelResultBindingNode':
+ # This node binds the latest model inference result with the current
+ # frame. (This means the frame image and inference result may be
+ # asynchronous).
+ dict(
+ type='ModelResultBindingNode',
+ name='ResultBinder',
+ frame_buffer='_frame_', # `_frame_` is a runner-reserved buffer
+ result_buffer='animal_pose',
+ output_buffer='frame'),
+ # 'PoseVisualizerNode':
+ # This node draw the pose visualization result in the frame image.
+ # Pose results is needed.
+ dict(
+ type='PoseVisualizerNode',
+ name='Visualizer',
+ enable_key='v',
+ frame_buffer='frame',
+ output_buffer='vis'),
+ # 'NoticeBoardNode':
+ # This node show a notice board with given content, e.g. help
+ # information.
+ dict(
+ type='NoticeBoardNode',
+ name='Helper',
+ enable_key='h',
+ enable=True,
+ frame_buffer='vis',
+ output_buffer='vis_notice',
+ content_lines=[
+ 'This is a demo for pose visualization and simple image '
+ 'effects. Have fun!', '', 'Hot-keys:',
+ '"v": Pose estimation result visualization',
+ '"s": Sunglasses effect B-)', '"b": Bug-eye effect 0_0',
+ '"h": Show help information',
+ '"m": Show diagnostic information', '"q": Exit'
+ ],
+ ),
+ # 'MonitorNode':
+ # This node show diagnostic information in the frame image. It can
+ # be used for debugging or monitoring system resource status.
+ dict(
+ type='MonitorNode',
+ name='Monitor',
+ enable_key='m',
+ enable=False,
+ frame_buffer='vis_notice',
+ output_buffer='display'),
+ # 'RecorderNode':
+ # This node save the output video into a file.
+ dict(
+ type='RecorderNode',
+ name='Recorder',
+ out_video_file='record.mp4',
+ frame_buffer='display',
+ output_buffer='_display_'
+ # `_display_` is a runner-reserved buffer
+ )
+ ])
diff --git a/engine/pose_estimation/third-party/ViTPose/tools/webcam/configs/examples/test_camera.py b/engine/pose_estimation/third-party/ViTPose/tools/webcam/configs/examples/test_camera.py
new file mode 100644
index 0000000..c0c1677
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tools/webcam/configs/examples/test_camera.py
@@ -0,0 +1,19 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+runner = dict(
+ name='Debug CamRunner',
+ camera_id=0,
+ camera_fps=20,
+ nodes=[
+ dict(
+ type='MonitorNode',
+ name='Monitor',
+ enable_key='m',
+ frame_buffer='_frame_',
+ output_buffer='display'),
+ dict(
+ type='RecorderNode',
+ name='Recorder',
+ out_video_file='webcam_output.mp4',
+ frame_buffer='display',
+ output_buffer='_display_')
+ ])
diff --git a/engine/pose_estimation/third-party/ViTPose/tools/webcam/configs/eyes/README.md b/engine/pose_estimation/third-party/ViTPose/tools/webcam/configs/eyes/README.md
new file mode 100644
index 0000000..f9c3769
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tools/webcam/configs/eyes/README.md
@@ -0,0 +1,31 @@
+# Sunglasses and Bug-eye Effects
+
+We can apply fun effects on videos with pose estimation results, like adding sunglasses on the face, or make the eyes look bigger.
+
+
+
+
+
+## Instruction
+
+### Get started
+
+Launch the demo from the mmpose root directory:
+
+```shell
+python tools/webcam/run_webcam.py --config tools/webcam/configs/examples/pose_estimation.py
+```
+
+### Hotkeys
+
+| Hotkey | Function |
+| -- | -- |
+| s | Toggle the sunglasses effect on/off. |
+| b | Toggle the bug-eye effect on/off. |
+| h | Show help information. |
+| m | Show the monitoring information. |
+| q | Exit. |
+
+### Configuration
+
+See the [README](/tools/webcam/configs/examples/README.md#configuration) of pose estimation demo for model configurations.
diff --git a/engine/pose_estimation/third-party/ViTPose/tools/webcam/configs/eyes/eyes.py b/engine/pose_estimation/third-party/ViTPose/tools/webcam/configs/eyes/eyes.py
new file mode 100644
index 0000000..91bbfba
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tools/webcam/configs/eyes/eyes.py
@@ -0,0 +1,114 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+runner = dict(
+ # Basic configurations of the runner
+ name='Eye Effects',
+ camera_id=0,
+ camera_fps=20,
+ synchronous=False,
+ # Define nodes.
+ # The configuration of a node usually includes:
+ # 1. 'type': Node class name
+ # 2. 'name': Node name
+ # 3. I/O buffers (e.g. 'input_buffer', 'output_buffer'): specify the
+ # input and output buffer names. This may depend on the node class.
+ # 4. 'enable_key': assign a hot-key to toggle enable/disable this node.
+ # This may depend on the node class.
+ # 5. Other class-specific arguments
+ nodes=[
+ # 'DetectorNode':
+ # This node performs object detection from the frame image using an
+ # MMDetection model.
+ dict(
+ type='DetectorNode',
+ name='Detector',
+ model_config='demo/mmdetection_cfg/'
+ 'ssdlite_mobilenetv2_scratch_600e_coco.py',
+ model_checkpoint='https://download.openmmlab.com'
+ '/mmdetection/v2.0/ssd/'
+ 'ssdlite_mobilenetv2_scratch_600e_coco/ssdlite_mobilenetv2_'
+ 'scratch_600e_coco_20210629_110627-974d9307.pth',
+ input_buffer='_input_', # `_input_` is a runner-reserved buffer
+ output_buffer='det_result'),
+ # 'TopDownPoseEstimatorNode':
+ # This node performs keypoint detection from the frame image using an
+ # MMPose top-down model. Detection results is needed.
+ dict(
+ type='TopDownPoseEstimatorNode',
+ name='Human Pose Estimator',
+ model_config='configs/wholebody/2d_kpt_sview_rgb_img/'
+ 'topdown_heatmap/coco-wholebody/'
+ 'vipnas_mbv3_coco_wholebody_256x192_dark.py',
+ model_checkpoint='https://openmmlab-share.oss-cn-hangz'
+ 'hou.aliyuncs.com/mmpose/top_down/vipnas/vipnas_mbv3_co'
+ 'co_wholebody_256x192_dark-e2158108_20211205.pth',
+ cls_names=['person'],
+ input_buffer='det_result',
+ output_buffer='human_pose'),
+ dict(
+ type='TopDownPoseEstimatorNode',
+ name='Animal Pose Estimator',
+ model_config='configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap'
+ '/animalpose/hrnet_w32_animalpose_256x256.py',
+ model_checkpoint='https://download.openmmlab.com/mmpose/animal/'
+ 'hrnet/hrnet_w32_animalpose_256x256-1aa7f075_20210426.pth',
+ cls_names=['cat', 'dog', 'horse', 'sheep', 'cow'],
+ input_buffer='human_pose',
+ output_buffer='animal_pose'),
+ # 'ModelResultBindingNode':
+ # This node binds the latest model inference result with the current
+ # frame. (This means the frame image and inference result may be
+ # asynchronous).
+ dict(
+ type='ModelResultBindingNode',
+ name='ResultBinder',
+ frame_buffer='_frame_', # `_frame_` is a runner-reserved buffer
+ result_buffer='animal_pose',
+ output_buffer='frame'),
+ # 'SunglassesNode':
+ # This node draw the sunglasses effect in the frame image.
+ # Pose results is needed.
+ dict(
+ type='SunglassesNode',
+ name='Visualizer',
+ enable_key='s',
+ enable=True,
+ frame_buffer='frame',
+ output_buffer='vis_sunglasses'),
+ # 'BugEyeNode':
+ # This node draw the bug-eye effetc in the frame image.
+ # Pose results is needed.
+ dict(
+ type='BugEyeNode',
+ name='Visualizer',
+ enable_key='b',
+ enable=False,
+ frame_buffer='vis_sunglasses',
+ output_buffer='vis_bugeye'),
+ # 'NoticeBoardNode':
+ # This node show a notice board with given content, e.g. help
+ # information.
+ dict(
+ type='NoticeBoardNode',
+ name='Helper',
+ enable_key='h',
+ frame_buffer='vis_bugeye',
+ output_buffer='vis',
+ content_lines=[
+ 'This is a demo for pose visualization and simple image '
+ 'effects. Have fun!', '', 'Hot-keys:',
+ '"s": Sunglasses effect B-)', '"b": Bug-eye effect 0_0',
+ '"h": Show help information',
+ '"m": Show diagnostic information', '"q": Exit'
+ ],
+ ),
+ # 'MonitorNode':
+ # This node show diagnostic information in the frame image. It can
+ # be used for debugging or monitoring system resource status.
+ dict(
+ type='MonitorNode',
+ name='Monitor',
+ enable_key='m',
+ enable=False,
+ frame_buffer='vis',
+ output_buffer='_display_') # `_frame_` is a runner-reserved buffer
+ ])
diff --git a/engine/pose_estimation/third-party/ViTPose/tools/webcam/configs/face_swap/README.md b/engine/pose_estimation/third-party/ViTPose/tools/webcam/configs/face_swap/README.md
new file mode 100644
index 0000000..02f4c8a
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tools/webcam/configs/face_swap/README.md
@@ -0,0 +1,31 @@
+# Sunglasses and Bug-eye Effects
+
+Look! Where is my face?:eyes: And whose face is it?:laughing:
+
+
+
+
+
+## Instruction
+
+### Get started
+
+Launch the demo from the mmpose root directory:
+
+```shell
+python tools/webcam/run_webcam.py --config tools/webcam/configs/face_swap/face_swap.py
+```
+
+### Hotkeys
+
+| Hotkey | Function |
+| -- | -- |
+| s | Switch between modes
Shuffle: Randomly shuffle all faces
Clone: Choose one face and clone it for everyone
None: Nothing happens and everyone is safe :)
|
+| v | Toggle the pose visualization on/off. |
+| h | Show help information. |
+| m | Show diagnostic information. |
+| q | Exit. |
+
+### Configuration
+
+See the [README](/tools/webcam/configs/examples/README.md#configuration) of pose estimation demo for model configurations.
diff --git a/engine/pose_estimation/third-party/ViTPose/tools/webcam/configs/face_swap/face_swap.py b/engine/pose_estimation/third-party/ViTPose/tools/webcam/configs/face_swap/face_swap.py
new file mode 100644
index 0000000..403eaae
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tools/webcam/configs/face_swap/face_swap.py
@@ -0,0 +1,79 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+runner = dict(
+ name='FaceSwap',
+ camera_id=0,
+ camera_fps=20,
+ synchronous=False,
+ nodes=[
+ dict(
+ type='DetectorNode',
+ name='Detector',
+ model_config='demo/mmdetection_cfg/'
+ 'ssdlite_mobilenetv2_scratch_600e_coco.py',
+ model_checkpoint='https://download.openmmlab.com'
+ '/mmdetection/v2.0/ssd/'
+ 'ssdlite_mobilenetv2_scratch_600e_coco/ssdlite_mobilenetv2_'
+ 'scratch_600e_coco_20210629_110627-974d9307.pth',
+ device='cpu',
+ input_buffer='_input_', # `_input_` is a runner-reserved buffer
+ output_buffer='det_result'),
+ dict(
+ type='TopDownPoseEstimatorNode',
+ name='TopDown Pose Estimator',
+ model_config='configs/wholebody/2d_kpt_sview_rgb_img/'
+ 'topdown_heatmap/coco-wholebody/'
+ 'vipnas_res50_coco_wholebody_256x192_dark.py',
+ model_checkpoint='https://openmmlab-share.oss-cn-hangzhou'
+ '.aliyuncs.com/mmpose/top_down/vipnas/'
+ 'vipnas_res50_wholebody_256x192_dark-67c0ce35_20211112.pth',
+ device='cpu',
+ cls_names=['person'],
+ input_buffer='det_result',
+ output_buffer='pose_result'),
+ dict(
+ type='ModelResultBindingNode',
+ name='ResultBinder',
+ frame_buffer='_frame_', # `_frame_` is a runner-reserved buffer
+ result_buffer='pose_result',
+ output_buffer='frame'),
+ dict(
+ type='FaceSwapNode',
+ name='FaceSwapper',
+ mode_key='s',
+ frame_buffer='frame',
+ output_buffer='face_swap'),
+ dict(
+ type='PoseVisualizerNode',
+ name='Visualizer',
+ enable_key='v',
+ frame_buffer='face_swap',
+ output_buffer='vis_pose'),
+ dict(
+ type='NoticeBoardNode',
+ name='Help Information',
+ enable_key='h',
+ content_lines=[
+ 'Swap your faces! ',
+ 'Hot-keys:',
+ '"v": Toggle the pose visualization on/off.',
+ '"s": Switch between modes: Shuffle, Clone and None',
+ '"h": Show help information',
+ '"m": Show diagnostic information',
+ '"q": Exit',
+ ],
+ frame_buffer='vis_pose',
+ output_buffer='vis_notice'),
+ dict(
+ type='MonitorNode',
+ name='Monitor',
+ enable_key='m',
+ enable=False,
+ frame_buffer='vis_notice',
+ output_buffer='display'),
+ dict(
+ type='RecorderNode',
+ name='Recorder',
+ out_video_file='faceswap_output.mp4',
+ frame_buffer='display',
+ output_buffer='_display_')
+ ])
diff --git a/engine/pose_estimation/third-party/ViTPose/tools/webcam/configs/meow_dwen_dwen/README.md b/engine/pose_estimation/third-party/ViTPose/tools/webcam/configs/meow_dwen_dwen/README.md
new file mode 100644
index 0000000..997ffc1
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tools/webcam/configs/meow_dwen_dwen/README.md
@@ -0,0 +1,44 @@
+# Meow Dwen Dwen
+
+Do you know [Bing DwenDwen (冰墩墩)](https://en.wikipedia.org/wiki/Bing_Dwen_Dwen_and_Shuey_Rhon_Rhon), the mascot of 2022 Beijing Olympic Games?
+
+
+
+
+
+Now you can dress your cat up in this costume and TA-DA! Be prepared for super cute **Meow Dwen Dwen**.
+
+
+
+
+
+You are a dog fan? Hold on, here comes Woof Dwen Dwen.
+
+
+
+
+
+## Instruction
+
+### Get started
+
+Launch the demo from the mmpose root directory:
+
+```shell
+python tools/webcam/run_webcam.py --config tools/webcam/configs/meow_dwen_dwen/meow_dwen_dwen.py
+```
+
+### Hotkeys
+
+| Hotkey | Function |
+| -- | -- |
+| s | Change the background. |
+| h | Show help information. |
+| m | Show diagnostic information. |
+| q | Exit. |
+
+### Configuration
+
+- **Use video input**
+
+As you can see in the config, we set `camera_id` as the path of the input image. You can also set it as a video file path (or url), or a webcam ID number (e.g. `camera_id=0`), to capture the dynamic face from the video input.
diff --git a/engine/pose_estimation/third-party/ViTPose/tools/webcam/configs/meow_dwen_dwen/meow_dwen_dwen.py b/engine/pose_estimation/third-party/ViTPose/tools/webcam/configs/meow_dwen_dwen/meow_dwen_dwen.py
new file mode 100644
index 0000000..399d01c
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tools/webcam/configs/meow_dwen_dwen/meow_dwen_dwen.py
@@ -0,0 +1,92 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+runner = dict(
+ # Basic configurations of the runner
+ name='Little fans of 2022 Beijing Winter Olympics',
+ # Cat image
+ camera_id='https://user-images.githubusercontent.com/'
+ '15977946/152932036-b5554cf8-24cf-40d6-a358-35a106013f11.jpeg',
+ # Dog image
+ # camera_id='https://user-images.githubusercontent.com/'
+ # '15977946/152932051-cd280b35-8066-45a0-8f52-657c8631aaba.jpg',
+ camera_fps=20,
+ nodes=[
+ dict(
+ type='DetectorNode',
+ name='Detector',
+ model_config='demo/mmdetection_cfg/'
+ 'ssdlite_mobilenetv2_scratch_600e_coco.py',
+ model_checkpoint='https://download.openmmlab.com'
+ '/mmdetection/v2.0/ssd/'
+ 'ssdlite_mobilenetv2_scratch_600e_coco/ssdlite_mobilenetv2_'
+ 'scratch_600e_coco_20210629_110627-974d9307.pth',
+ input_buffer='_input_', # `_input_` is a runner-reserved buffer
+ output_buffer='det_result'),
+ dict(
+ type='TopDownPoseEstimatorNode',
+ name='Animal Pose Estimator',
+ model_config='configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap'
+ '/ap10k/hrnet_w32_ap10k_256x256.py',
+ model_checkpoint='https://download.openmmlab.com/mmpose/animal/'
+ 'hrnet/hrnet_w32_ap10k_256x256-18aac840_20211029.pth',
+ cls_names=['cat', 'dog'],
+ input_buffer='det_result',
+ output_buffer='animal_pose'),
+ dict(
+ type='TopDownPoseEstimatorNode',
+ name='TopDown Pose Estimator',
+ model_config='configs/wholebody/2d_kpt_sview_rgb_img/'
+ 'topdown_heatmap/coco-wholebody/'
+ 'vipnas_res50_coco_wholebody_256x192_dark.py',
+ model_checkpoint='https://openmmlab-share.oss-cn-hangzhou'
+ '.aliyuncs.com/mmpose/top_down/vipnas/'
+ 'vipnas_res50_wholebody_256x192_dark-67c0ce35_20211112.pth',
+ device='cpu',
+ cls_names=['person'],
+ input_buffer='animal_pose',
+ output_buffer='human_pose'),
+ dict(
+ type='ModelResultBindingNode',
+ name='ResultBinder',
+ frame_buffer='_frame_', # `_frame_` is a runner-reserved buffer
+ result_buffer='human_pose',
+ output_buffer='frame'),
+ dict(
+ type='XDwenDwenNode',
+ name='XDwenDwen',
+ mode_key='s',
+ resource_file='tools/webcam/configs/meow_dwen_dwen/'
+ 'resource-info.json',
+ out_shape=(480, 480),
+ frame_buffer='frame',
+ output_buffer='vis'),
+ dict(
+ type='NoticeBoardNode',
+ name='Helper',
+ enable_key='h',
+ enable=False,
+ frame_buffer='vis',
+ output_buffer='vis_notice',
+ content_lines=[
+ 'Let your pet put on a costume of Bing-Dwen-Dwen, '
+ 'the mascot of 2022 Beijing Winter Olympics. Have fun!', '',
+ 'Hot-keys:', '"s": Change the background',
+ '"h": Show help information',
+ '"m": Show diagnostic information', '"q": Exit'
+ ],
+ ),
+ dict(
+ type='MonitorNode',
+ name='Monitor',
+ enable_key='m',
+ enable=False,
+ frame_buffer='vis_notice',
+ output_buffer='display'),
+ dict(
+ type='RecorderNode',
+ name='Recorder',
+ out_video_file='record.mp4',
+ frame_buffer='display',
+ output_buffer='_display_'
+ # `_display_` is a runner-reserved buffer
+ )
+ ])
diff --git a/engine/pose_estimation/third-party/ViTPose/tools/webcam/configs/meow_dwen_dwen/resource-info.json b/engine/pose_estimation/third-party/ViTPose/tools/webcam/configs/meow_dwen_dwen/resource-info.json
new file mode 100644
index 0000000..adb811c
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tools/webcam/configs/meow_dwen_dwen/resource-info.json
@@ -0,0 +1,26 @@
+[
+ {
+ "id": 1,
+ "result": "{\"width\":690,\"height\":713,\"valid\":true,\"rotate\":0,\"step_1\":{\"toolName\":\"pointTool\",\"result\":[{\"x\":374.86387434554973,\"y\":262.8020942408377,\"attribute\":\"\",\"valid\":true,\"id\":\"8SK9cVyu\",\"sourceID\":\"\",\"textAttribute\":\"\",\"order\":2},{\"x\":492.8261780104712,\"y\":285.2,\"attribute\":\"\",\"valid\":true,\"id\":\"qDk54WsI\",\"sourceID\":\"\",\"textAttribute\":\"\",\"order\":1},{\"x\":430.11204188481673,\"y\":318.0502617801047,\"attribute\":\"\",\"valid\":true,\"id\":\"4H80L7lL\",\"sourceID\":\"\",\"textAttribute\":\"\",\"order\":3}]},\"step_2\":{\"dataSourceStep\":0,\"toolName\":\"polygonTool\",\"result\":[{\"id\":\"pwUsrf9u\",\"sourceID\":\"\",\"valid\":true,\"textAttribute\":\"\",\"pointList\":[{\"x\":423.3926701570681,\"y\":191.87539267015708},{\"x\":488.3465968586388,\"y\":209.04712041884818},{\"x\":535.3821989528797,\"y\":248.6167539267016},{\"x\":549.5675392670157,\"y\":306.8513089005236},{\"x\":537.6219895287959,\"y\":349.407329842932},{\"x\":510.74450261780106,\"y\":381.51099476439794},{\"x\":480.1340314136126,\"y\":394.9497382198953},{\"x\":411.4471204188482,\"y\":390.47015706806286},{\"x\":355.45235602094243,\"y\":373.29842931937173},{\"x\":306.17696335078534,\"y\":327.00942408376966},{\"x\":294.97801047120424,\"y\":284.45340314136126},{\"x\":306.9235602094241,\"y\":245.6303664921466},{\"x\":333.8010471204189,\"y\":217.25968586387435},{\"x\":370.3842931937173,\"y\":196.35497382198955}],\"attribute\":\"\",\"order\":1}]}}",
+ "url": "https://user-images.githubusercontent.com/15977946/152742677-35fe8a01-bd06-4a12-a02e-949e7d71f28a.jpg",
+ "fileName": "bing_dwen_dwen1.jpg"
+ },
+ {
+ "id": 2,
+ "result": "{\"width\":690,\"height\":659,\"valid\":true,\"rotate\":0,\"step_1\":{\"dataSourceStep\":0,\"toolName\":\"pointTool\",\"result\":[{\"x\":293.2460732984293,\"y\":242.89842931937173,\"attribute\":\"\",\"valid\":true,\"id\":\"KgPs39bY\",\"sourceID\":\"\",\"textAttribute\":\"\",\"order\":1},{\"x\":170.41675392670155,\"y\":270.50052356020944,\"attribute\":\"\",\"valid\":true,\"id\":\"XwHyoBFU\",\"sourceID\":\"\",\"textAttribute\":\"\",\"order\":2},{\"x\":224.24083769633506,\"y\":308.45340314136126,\"attribute\":\"\",\"valid\":true,\"id\":\"Qfs4YfuB\",\"sourceID\":\"\",\"textAttribute\":\"\",\"order\":3}]},\"step_2\":{\"dataSourceStep\":0,\"toolName\":\"polygonTool\",\"result\":[{\"id\":\"ts5jlJxb\",\"sourceID\":\"\",\"valid\":true,\"textAttribute\":\"\",\"pointList\":[{\"x\":178.69738219895285,\"y\":184.93403141361256},{\"x\":204.91937172774865,\"y\":172.5130890052356},{\"x\":252.5329842931937,\"y\":169.0628272251309},{\"x\":295.3162303664921,\"y\":175.27329842931937},{\"x\":333.95916230366487,\"y\":195.2848167539267},{\"x\":360.18115183246067,\"y\":220.1267015706806},{\"x\":376.0523560209424,\"y\":262.909947643979},{\"x\":373.98219895287957,\"y\":296.0324607329843},{\"x\":344.99999999999994,\"y\":335.365445026178},{\"x\":322.22827225130885,\"y\":355.37696335078533},{\"x\":272.544502617801,\"y\":378.1486910994764},{\"x\":221.48062827225127,\"y\":386.42931937172773},{\"x\":187.6680628272251,\"y\":385.7392670157068},{\"x\":158.68586387434553,\"y\":369.1780104712042},{\"x\":137.98429319371724,\"y\":337.43560209424083},{\"x\":127.63350785340312,\"y\":295.34240837696336},{\"x\":131.0837696335078,\"y\":242.89842931937173},{\"x\":147.64502617801045,\"y\":208.3958115183246}],\"attribute\":\"\",\"order\":1}]}}",
+ "url": "https://user-images.githubusercontent.com/15977946/152742707-c0c51844-e1d0-42d0-9a12-e369002e082f.jpg",
+ "fileName": "bing_dwen_dwen2.jpg"
+ },
+ {
+ "id": 3,
+ "result": "{\"width\":690,\"height\":811,\"valid\":true,\"rotate\":0,\"step_1\":{\"dataSourceStep\":0,\"toolName\":\"pointTool\",\"result\":[{\"x\":361.13507853403144,\"y\":300.62198952879584,\"attribute\":\"\",\"valid\":true,\"id\":\"uAtbXtf2\",\"sourceID\":\"\",\"textAttribute\":\"\",\"order\":1},{\"x\":242.24502617801048,\"y\":317.60628272251313,\"attribute\":\"\",\"valid\":true,\"id\":\"iLtceHMA\",\"sourceID\":\"\",\"textAttribute\":\"\",\"order\":2},{\"x\":302.5392670157068,\"y\":356.67015706806285,\"attribute\":\"\",\"valid\":true,\"id\":\"n9MTlJ6A\",\"sourceID\":\"\",\"textAttribute\":\"\",\"order\":3}]},\"step_2\":{\"dataSourceStep\":0,\"toolName\":\"polygonTool\",\"result\":[{\"id\":\"5sTLU5wF\",\"sourceID\":\"\",\"valid\":true,\"textAttribute\":\"\",\"pointList\":[{\"x\":227.80837696335078,\"y\":247.12146596858642},{\"x\":248.18952879581153,\"y\":235.23246073298432},{\"x\":291.4994764397906,\"y\":225.04188481675394},{\"x\":351.7937172774869,\"y\":229.28795811518327},{\"x\":393.40523560209425,\"y\":245.42303664921468},{\"x\":424.8261780104712,\"y\":272.59790575916236},{\"x\":443.5089005235602,\"y\":298.07434554973827},{\"x\":436.7151832460733,\"y\":345.6303664921466},{\"x\":406.1434554973822,\"y\":382.9958115183247},{\"x\":355.1905759162304,\"y\":408.4722513089006},{\"x\":313.57905759162304,\"y\":419.5120418848168},{\"x\":262.6261780104712,\"y\":417.81361256544506},{\"x\":224.41151832460733,\"y\":399.9801047120419},{\"x\":201.48272251308902,\"y\":364.3130890052356},{\"x\":194.68900523560208,\"y\":315.0586387434555},{\"x\":202.33193717277487,\"y\":272.59790575916236}],\"attribute\":\"\",\"order\":1}]}}",
+ "url": "https://user-images.githubusercontent.com/15977946/152742728-99392ecf-8f5c-46cf-b5c4-fe7fb6b39976.jpg",
+ "fileName": "bing_dwen_dwen3.jpg"
+ },
+ {
+ "id": 4,
+ "result": "{\"width\":690,\"height\":690,\"valid\":true,\"rotate\":0,\"step_1\":{\"dataSourceStep\":0,\"toolName\":\"pointTool\",\"result\":[{\"x\":365.9528795811519,\"y\":464.5759162303665,\"attribute\":\"\",\"valid\":true,\"id\":\"IKprTuHS\",\"sourceID\":\"\",\"textAttribute\":\"\",\"order\":1},{\"x\":470.71727748691103,\"y\":445.06806282722516,\"attribute\":\"\",\"valid\":true,\"id\":\"Z90CWkEI\",\"sourceID\":\"\",\"textAttribute\":\"\",\"order\":2},{\"x\":410.74869109947645,\"y\":395.2146596858639,\"attribute\":\"\",\"valid\":true,\"id\":\"UWRstKZk\",\"sourceID\":\"\",\"textAttribute\":\"\",\"order\":3}]},\"step_2\":{\"dataSourceStep\":0,\"toolName\":\"polygonTool\",\"result\":[{\"id\":\"C30Pc9Ww\",\"sourceID\":\"\",\"valid\":true,\"textAttribute\":\"\",\"pointList\":[{\"x\":412.91623036649213,\"y\":325.85340314136124},{\"x\":468.5497382198953,\"y\":335.9685863874345},{\"x\":501.78534031413614,\"y\":369.2041884816754},{\"x\":514.0680628272252,\"y\":415.44502617801044},{\"x\":504.67539267015707,\"y\":472.5235602094241},{\"x\":484.44502617801044,\"y\":497.0890052356021},{\"x\":443.26178010471205,\"y\":512.9842931937172},{\"x\":389.7958115183246,\"y\":518.7643979057591},{\"x\":336.32984293193715,\"y\":504.31413612565444},{\"x\":302.3717277486911,\"y\":462.40837696335075},{\"x\":298.0366492146597,\"y\":416.89005235602093},{\"x\":318.26701570680626,\"y\":372.0942408376963},{\"x\":363.0628272251309,\"y\":341.0261780104712}],\"attribute\":\"\",\"order\":1}]}}",
+ "url": "https://user-images.githubusercontent.com/15977946/152742755-9dc75f89-4156-4103-9c6d-f35f1f409d11.jpg",
+ "fileName": "bing_dwen_dwen4.jpg"
+ }
+]
diff --git a/engine/pose_estimation/third-party/ViTPose/tools/webcam/configs/newyear/README.md b/engine/pose_estimation/third-party/ViTPose/tools/webcam/configs/newyear/README.md
new file mode 100644
index 0000000..8c655c1
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tools/webcam/configs/newyear/README.md
@@ -0,0 +1,31 @@
+# New Year Hat and Firecracker Effects
+
+This demo provides new year effects with pose estimation results, like adding hat on the head and firecracker in the hands.
+
+
+
+
+
+## Instruction
+
+### Get started
+
+Launch the demo from the mmpose root directory:
+
+```shell
+python tools/webcam/run_webcam.py --config tools/webcam/configs/newyear/new_year.py
+```
+
+### Hotkeys
+
+| Hotkey | Function |
+| -- | -- |
+| t | Toggle the hat effect on/off. |
+| f | Toggle the firecracker effect on/off. |
+| h | Show help information. |
+| m | Show the monitoring information. |
+| q | Exit. |
+
+### Configuration
+
+See the [README](/tools/webcam/configs/examples/README.md#configuration) of pose estimation demo for model configurations.
diff --git a/engine/pose_estimation/third-party/ViTPose/tools/webcam/configs/newyear/new_year.py b/engine/pose_estimation/third-party/ViTPose/tools/webcam/configs/newyear/new_year.py
new file mode 100644
index 0000000..3551184
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tools/webcam/configs/newyear/new_year.py
@@ -0,0 +1,122 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+runner = dict(
+ # Basic configurations of the runner
+ name='Pose Estimation',
+ camera_id=0,
+ camera_fps=20,
+ synchronous=False,
+ # Define nodes.
+ # The configuration of a node usually includes:
+ # 1. 'type': Node class name
+ # 2. 'name': Node name
+ # 3. I/O buffers (e.g. 'input_buffer', 'output_buffer'): specify the
+ # input and output buffer names. This may depend on the node class.
+ # 4. 'enable_key': assign a hot-key to toggle enable/disable this node.
+ # This may depend on the node class.
+ # 5. Other class-specific arguments
+ nodes=[
+ # 'DetectorNode':
+ # This node performs object detection from the frame image using an
+ # MMDetection model.
+ dict(
+ type='DetectorNode',
+ name='Detector',
+ model_config='demo/mmdetection_cfg/'
+ 'ssdlite_mobilenetv2_scratch_600e_coco.py',
+ model_checkpoint='https://download.openmmlab.com'
+ '/mmdetection/v2.0/ssd/'
+ 'ssdlite_mobilenetv2_scratch_600e_coco/ssdlite_mobilenetv2_'
+ 'scratch_600e_coco_20210629_110627-974d9307.pth',
+ input_buffer='_input_', # `_input_` is a runner-reserved buffer
+ output_buffer='det_result'),
+ # 'TopDownPoseEstimatorNode':
+ # This node performs keypoint detection from the frame image using an
+ # MMPose top-down model. Detection results is needed.
+ dict(
+ type='TopDownPoseEstimatorNode',
+ name='Human Pose Estimator',
+ model_config='configs/wholebody/2d_kpt_sview_rgb_img/'
+ 'topdown_heatmap/coco-wholebody/'
+ 'vipnas_mbv3_coco_wholebody_256x192_dark.py',
+ model_checkpoint='https://openmmlab-share.oss-cn-hangz'
+ 'hou.aliyuncs.com/mmpose/top_down/vipnas/vipnas_mbv3_co'
+ 'co_wholebody_256x192_dark-e2158108_20211205.pth',
+ cls_names=['person'],
+ input_buffer='det_result',
+ output_buffer='human_pose'),
+ dict(
+ type='TopDownPoseEstimatorNode',
+ name='Animal Pose Estimator',
+ model_config='configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap'
+ '/animalpose/hrnet_w32_animalpose_256x256.py',
+ model_checkpoint='https://download.openmmlab.com/mmpose/animal/'
+ 'hrnet/hrnet_w32_animalpose_256x256-1aa7f075_20210426.pth',
+ cls_names=['cat', 'dog', 'horse', 'sheep', 'cow'],
+ input_buffer='human_pose',
+ output_buffer='animal_pose'),
+ # 'ModelResultBindingNode':
+ # This node binds the latest model inference result with the current
+ # frame. (This means the frame image and inference result may be
+ # asynchronous).
+ dict(
+ type='ModelResultBindingNode',
+ name='ResultBinder',
+ frame_buffer='_frame_', # `_frame_` is a runner-reserved buffer
+ result_buffer='animal_pose',
+ output_buffer='frame'),
+ # 'HatNode':
+ # This node draw the hat effect in the frame image.
+ # Pose results is needed.
+ dict(
+ type='HatNode',
+ name='Visualizer',
+ enable_key='t',
+ frame_buffer='frame',
+ output_buffer='vis_hat'),
+ # 'FirecrackerNode':
+ # This node draw the firecracker effect in the frame image.
+ # Pose results is needed.
+ dict(
+ type='FirecrackerNode',
+ name='Visualizer',
+ enable_key='f',
+ frame_buffer='vis_hat',
+ output_buffer='vis_firecracker'),
+ # 'NoticeBoardNode':
+ # This node show a notice board with given content, e.g. help
+ # information.
+ dict(
+ type='NoticeBoardNode',
+ name='Helper',
+ enable_key='h',
+ enable=True,
+ frame_buffer='vis_firecracker',
+ output_buffer='vis_notice',
+ content_lines=[
+ 'This is a demo for pose visualization and simple image '
+ 'effects. Have fun!', '', 'Hot-keys:', '"t": Hat effect',
+ '"f": Firecracker effect', '"h": Show help information',
+ '"m": Show diagnostic information', '"q": Exit'
+ ],
+ ),
+ # 'MonitorNode':
+ # This node show diagnostic information in the frame image. It can
+ # be used for debugging or monitoring system resource status.
+ dict(
+ type='MonitorNode',
+ name='Monitor',
+ enable_key='m',
+ enable=False,
+ frame_buffer='vis_notice',
+ output_buffer='display'),
+ # 'RecorderNode':
+ # This node save the output video into a file.
+ dict(
+ type='RecorderNode',
+ name='Recorder',
+ out_video_file='record.mp4',
+ frame_buffer='display',
+ output_buffer='_display_'
+ # `_display_` is a runner-reserved buffer
+ )
+ ])
diff --git a/engine/pose_estimation/third-party/ViTPose/tools/webcam/configs/supersaiyan/README.md b/engine/pose_estimation/third-party/ViTPose/tools/webcam/configs/supersaiyan/README.md
new file mode 100644
index 0000000..9e9aef1
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tools/webcam/configs/supersaiyan/README.md
@@ -0,0 +1,96 @@
+# Super Saiyan Effects
+
+We can apply fun effects on videos with pose estimation results, like Super Saiyan transformation.
+
+https://user-images.githubusercontent.com/11788150/150138076-2192079f-068a-4d43-bf27-2f1fd708cabc.mp4
+
+## Instruction
+
+### Get started
+
+Launch the demo from the mmpose root directory:
+
+```shell
+python tools/webcam/run_webcam.py --config tools/webcam/configs/supersaiyan/saiyan.py
+```
+
+### Hotkeys
+
+| Hotkey | Function |
+| -- | -- |
+| s | Toggle the Super Saiyan effect on/off. |
+| h | Show help information. |
+| m | Show the monitoring information. |
+| q | Exit. |
+
+Note that the demo will automatically save the output video into a file `record.mp4`.
+
+### Configuration
+
+- **Choose a detection model**
+
+Users can choose detection models from the [MMDetection Model Zoo](https://mmdetection.readthedocs.io/en/v2.20.0/model_zoo.html). Just set the `model_config` and `model_checkpoint` in the detector node accordingly, and the model will be automatically downloaded and loaded.
+
+```python
+# 'DetectorNode':
+# This node performs object detection from the frame image using an
+# MMDetection model.
+dict(
+ type='DetectorNode',
+ name='Detector',
+ model_config='demo/mmdetection_cfg/mask_rcnn_r50_fpn_2x_coco.py',
+ model_checkpoint='https://download.openmmlab.com/'
+ 'mmdetection/v2.0/mask_rcnn/mask_rcnn_r50_fpn_2x_coco/'
+ 'mask_rcnn_r50_fpn_2x_coco_bbox_mAP-0.392'
+ '__segm_mAP-0.354_20200505_003907-3e542a40.pth',
+ input_buffer='_input_', # `_input_` is a runner-reserved buffer
+ output_buffer='det_result'),
+```
+
+- **Choose a or more pose models**
+
+In this demo we use two [top-down](https://github.com/open-mmlab/mmpose/tree/master/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap) pose estimation models for humans and animals respectively. Users can choose models from the [MMPose Model Zoo](https://mmpose.readthedocs.io/en/latest/modelzoo.html). To apply different pose models on different instance types, you can add multiple pose estimator nodes with `cls_names` set accordingly.
+
+```python
+# 'TopDownPoseEstimatorNode':
+# This node performs keypoint detection from the frame image using an
+# MMPose top-down model. Detection results is needed.
+dict(
+ type='TopDownPoseEstimatorNode',
+ name='Human Pose Estimator',
+ model_config='configs/wholebody/2d_kpt_sview_rgb_img/'
+ 'topdown_heatmap/coco-wholebody/'
+ 'vipnas_mbv3_coco_wholebody_256x192_dark.py',
+ model_checkpoint='https://openmmlab-share.oss-cn-hangz'
+ 'hou.aliyuncs.com/mmpose/top_down/vipnas/vipnas_mbv3_co'
+ 'co_wholebody_256x192_dark-e2158108_20211205.pth',
+ cls_names=['person'],
+ input_buffer='det_result',
+ output_buffer='human_pose')
+```
+
+- **Run the demo without GPU**
+
+If you don't have GPU and CUDA in your device, the demo can run with only CPU by setting `device='cpu'` in all model nodes. For example:
+
+```python
+dict(
+ type='DetectorNode',
+ name='Detector',
+ model_config='demo/mmdetection_cfg/mask_rcnn_r50_fpn_2x_coco.py',
+ model_checkpoint='https://download.openmmlab.com/'
+ 'mmdetection/v2.0/mask_rcnn/mask_rcnn_r50_fpn_2x_coco/'
+ 'mask_rcnn_r50_fpn_2x_coco_bbox_mAP-0.392'
+ '__segm_mAP-0.354_20200505_003907-3e542a40.pth',
+ device='cpu',
+ input_buffer='_input_', # `_input_` is a runner-reserved buffer
+ output_buffer='det_result'),
+```
+
+- **Debug webcam and display**
+
+You can launch the webcam runner with a debug config:
+
+```shell
+python tools/webcam/run_webcam.py --config tools/webcam/configs/examples/test_camera.py
+```
diff --git a/engine/pose_estimation/third-party/ViTPose/tools/webcam/configs/supersaiyan/saiyan.py b/engine/pose_estimation/third-party/ViTPose/tools/webcam/configs/supersaiyan/saiyan.py
new file mode 100644
index 0000000..5a8e7bc
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tools/webcam/configs/supersaiyan/saiyan.py
@@ -0,0 +1,93 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+runner = dict(
+ # Basic configurations of the runner
+ name='Super Saiyan Effects',
+ camera_id=0,
+ camera_fps=30,
+ synchronous=False,
+ # Define nodes.
+ # The configuration of a node usually includes:
+ # 1. 'type': Node class name
+ # 2. 'name': Node name
+ # 3. I/O buffers (e.g. 'input_buffer', 'output_buffer'): specify the
+ # input and output buffer names. This may depend on the node class.
+ # 4. 'enable_key': assign a hot-key to toggle enable/disable this node.
+ # This may depend on the node class.
+ # 5. Other class-specific arguments
+ nodes=[
+ # 'DetectorNode':
+ # This node performs object detection from the frame image using an
+ # MMDetection model.
+ dict(
+ type='DetectorNode',
+ name='Detector',
+ model_config='demo/mmdetection_cfg/mask_rcnn_r50_fpn_2x_coco.py',
+ model_checkpoint='https://download.openmmlab.com/'
+ 'mmdetection/v2.0/mask_rcnn/mask_rcnn_r50_fpn_2x_coco/'
+ 'mask_rcnn_r50_fpn_2x_coco_bbox_mAP-0.392'
+ '__segm_mAP-0.354_20200505_003907-3e542a40.pth',
+ input_buffer='_input_', # `_input_` is a runner-reserved buffer
+ output_buffer='det_result'),
+ # 'TopDownPoseEstimatorNode':
+ # This node performs keypoint detection from the frame image using an
+ # MMPose top-down model. Detection results is needed.
+ dict(
+ type='TopDownPoseEstimatorNode',
+ name='Human Pose Estimator',
+ model_config='configs/wholebody/2d_kpt_sview_rgb_img/'
+ 'topdown_heatmap/coco-wholebody/'
+ 'vipnas_mbv3_coco_wholebody_256x192_dark.py',
+ model_checkpoint='https://openmmlab-share.oss-cn-hangz'
+ 'hou.aliyuncs.com/mmpose/top_down/vipnas/vipnas_mbv3_co'
+ 'co_wholebody_256x192_dark-e2158108_20211205.pth',
+ cls_names=['person'],
+ input_buffer='det_result',
+ output_buffer='human_pose'),
+ # 'ModelResultBindingNode':
+ # This node binds the latest model inference result with the current
+ # frame. (This means the frame image and inference result may be
+ # asynchronous).
+ dict(
+ type='ModelResultBindingNode',
+ name='ResultBinder',
+ frame_buffer='_frame_', # `_frame_` is a runner-reserved buffer
+ result_buffer='human_pose',
+ output_buffer='frame'),
+ # 'SaiyanNode':
+ # This node draw the Super Saiyan effect in the frame image.
+ # Pose results is needed.
+ dict(
+ type='SaiyanNode',
+ name='Visualizer',
+ enable_key='s',
+ cls_names=['person'],
+ enable=True,
+ frame_buffer='frame',
+ output_buffer='vis_saiyan'),
+ # 'NoticeBoardNode':
+ # This node show a notice board with given content, e.g. help
+ # information.
+ dict(
+ type='NoticeBoardNode',
+ name='Helper',
+ enable_key='h',
+ frame_buffer='vis_saiyan',
+ output_buffer='vis',
+ content_lines=[
+ 'This is a demo for super saiyan effects. Have fun!', '',
+ 'Hot-keys:', '"s": Saiyan effect',
+ '"h": Show help information',
+ '"m": Show diagnostic information', '"q": Exit'
+ ],
+ ),
+ # 'MonitorNode':
+ # This node show diagnostic information in the frame image. It can
+ # be used for debugging or monitoring system resource status.
+ dict(
+ type='MonitorNode',
+ name='Monitor',
+ enable_key='m',
+ enable=False,
+ frame_buffer='vis',
+ output_buffer='_display_') # `_frame_` is a runner-reserved buffer
+ ])
diff --git a/engine/pose_estimation/third-party/ViTPose/tools/webcam/configs/valentinemagic/README.md b/engine/pose_estimation/third-party/ViTPose/tools/webcam/configs/valentinemagic/README.md
new file mode 100644
index 0000000..8063d2e
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tools/webcam/configs/valentinemagic/README.md
@@ -0,0 +1,35 @@
+# Valentine Magic
+
+Do you want to show your **love** to your beloved one, especially on Valentine's Day? Express it with your pose using MMPose right away and see the Valentine Magic!
+
+Try to pose a hand heart gesture, and see what will happen?
+
+Prefer a blow kiss? Here comes your flying heart~
+
+
+
+
+
+## Instruction
+
+### Get started
+
+Launch the demo from the mmpose root directory:
+
+```shell
+python tools/webcam/run_webcam.py --config tools/webcam/configs/valentinemagic/valentinemagic.py
+```
+
+### Hotkeys
+
+| Hotkey | Function |
+| -- | -- |
+| l | Toggle the Valentine Magic effect on/off. |
+| v | Toggle the pose visualization on/off. |
+| h | Show help information. |
+| m | Show diagnostic information. |
+| q | Exit. |
+
+### Configuration
+
+See the [README](/tools/webcam/configs/examples/README.md#configuration) of pose estimation demo for model configurations.
diff --git a/engine/pose_estimation/third-party/ViTPose/tools/webcam/configs/valentinemagic/valentinemagic.py b/engine/pose_estimation/third-party/ViTPose/tools/webcam/configs/valentinemagic/valentinemagic.py
new file mode 100644
index 0000000..5f921b0
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tools/webcam/configs/valentinemagic/valentinemagic.py
@@ -0,0 +1,118 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+runner = dict(
+ # Basic configurations of the runner
+ name='Human Pose and Effects',
+ camera_id=0,
+ camera_fps=30,
+
+ # Define nodes.
+ #
+ # The configuration of a node usually includes:
+ # 1. 'type': Node class name
+ # 2. 'name': Node name
+ # 3. I/O buffers (e.g. 'input_buffer', 'output_buffer'): specify the
+ # input and output buffer names. This may depend on the node class.
+ # 4. 'enable_key': assign a hot-key to toggle enable/disable this node.
+ # This may depend on the node class.
+ # 5. Other class-specific arguments
+ nodes=[
+ # 'DetectorNode':
+ # This node performs object detection from the frame image using an
+ # MMDetection model.
+ dict(
+ type='DetectorNode',
+ name='Detector',
+ model_config='demo/mmdetection_cfg/'
+ 'ssdlite_mobilenetv2_scratch_600e_coco.py',
+ model_checkpoint='https://download.openmmlab.com'
+ '/mmdetection/v2.0/ssd/'
+ 'ssdlite_mobilenetv2_scratch_600e_coco/ssdlite_mobilenetv2_'
+ 'scratch_600e_coco_20210629_110627-974d9307.pth',
+ input_buffer='_input_', # `_input_` is a runner-reserved buffer
+ output_buffer='det_result'),
+ # 'TopDownPoseEstimatorNode':
+ # This node performs keypoint detection from the frame image using an
+ # MMPose top-down model. Detection results is needed.
+ dict(
+ type='TopDownPoseEstimatorNode',
+ name='Human Pose Estimator',
+ model_config='configs/wholebody/2d_kpt_sview_rgb_img/'
+ 'topdown_heatmap/coco-wholebody/'
+ 'vipnas_mbv3_coco_wholebody_256x192_dark.py',
+ model_checkpoint='https://download.openmmlab.com/mmpose/top_down/'
+ 'vipnas/vipnas_mbv3_coco_wholebody_256x192_dark'
+ '-e2158108_20211205.pth',
+ cls_names=['person'],
+ input_buffer='det_result',
+ output_buffer='pose_result'),
+ # 'ModelResultBindingNode':
+ # This node binds the latest model inference result with the current
+ # frame. (This means the frame image and inference result may be
+ # asynchronous).
+ dict(
+ type='ModelResultBindingNode',
+ name='ResultBinder',
+ frame_buffer='_frame_', # `_frame_` is a runner-reserved buffer
+ result_buffer='pose_result',
+ output_buffer='frame'),
+ # 'PoseVisualizerNode':
+ # This node draw the pose visualization result in the frame image.
+ # Pose results is needed.
+ dict(
+ type='PoseVisualizerNode',
+ name='Visualizer',
+ enable_key='v',
+ enable=False,
+ frame_buffer='frame',
+ output_buffer='vis'),
+ # 'ValentineMagicNode':
+ # This node draw heart in the image.
+ # It can launch dynamically expanding heart from the middle of
+ # hands if the persons pose a "hand heart" gesture or blow a kiss.
+ # Only there are two persons in the image can trigger this effect.
+ # Pose results is needed.
+ dict(
+ type='ValentineMagicNode',
+ name='Visualizer',
+ enable_key='l',
+ frame_buffer='vis',
+ output_buffer='vis_heart',
+ ),
+ # 'NoticeBoardNode':
+ # This node show a notice board with given content, e.g. help
+ # information.
+ dict(
+ type='NoticeBoardNode',
+ name='Helper',
+ enable_key='h',
+ enable=False,
+ frame_buffer='vis_heart',
+ output_buffer='vis_notice',
+ content_lines=[
+ 'This is a demo for pose visualization and simple image '
+ 'effects. Have fun!', '', 'Hot-keys:',
+ '"h": Show help information', '"l": LoveHeart Effect',
+ '"v": PoseVisualizer', '"m": Show diagnostic information',
+ '"q": Exit'
+ ],
+ ),
+ # 'MonitorNode':
+ # This node show diagnostic information in the frame image. It can
+ # be used for debugging or monitoring system resource status.
+ dict(
+ type='MonitorNode',
+ name='Monitor',
+ enable_key='m',
+ enable=False,
+ frame_buffer='vis_notice',
+ output_buffer='display'), # `_frame_` is a runner-reserved buffer
+ # 'RecorderNode':
+ # This node record the frames into a local file. It can save the
+ # visualiztion results. Uncommit the following lines to turn it on.
+ dict(
+ type='RecorderNode',
+ name='Recorder',
+ out_video_file='record.mp4',
+ frame_buffer='display',
+ output_buffer='_display_')
+ ])
diff --git a/engine/pose_estimation/third-party/ViTPose/tools/webcam/docs/example_cn.md b/engine/pose_estimation/third-party/ViTPose/tools/webcam/docs/example_cn.md
new file mode 100644
index 0000000..69b9898
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/tools/webcam/docs/example_cn.md
@@ -0,0 +1,171 @@
+# 开发示例：给猫咪戴上太阳镜
+
+## 设计思路
+
+在动手之前，我们先考虑如何实现这个功能：
+
+- 首先，要做目标检测，找到图像中的猫咪
+- 接着，要估计猫咪的关键点位置，比如左右眼的位置
+- 最后，把太阳镜素材图片贴在合适的位置，TA-DA！
+
+按照这个思路，下面我们来看如何一步一步实现它。
+
+## Step 1：从一个现成的 Config 开始
+
+在 WebcamAPI 中，已经添加了一些实现常用功能的 Node，并提供了对应的 config 示例。利用这些可以减少用户的开发量。例如，我们可以以上面的姿态估计 demo 为基础。它的 config 位于 `tools/webcam/configs/example/pose_estimation.py`。为了更直观，我们把这个 config 中的功能节点表示成以下流程图：
+
+
- Official PyTorch Implementation
+
+
+ 
+
+
+ 
+
+
+
+## Web Demo
+
+- Integrated into [Huggingface Spaces 🤗](https://huggingface.co/spaces) using [Gradio](https://github.com/gradio-app/gradio). Try out the Web Demo for video: [](https://huggingface.co/spaces/hysts/ViTPose_video) and images [](https://huggingface.co/spaces/Gradio-Blocks/ViTPose)
+
+## MAE Pre-trained model
+
+- The small size MAE pre-trained model can be found in [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgccZeiFjh4DJ7gjYyg?e=iTMdMq).
+- The base, large, and huge pre-trained models using MAE can be found in the [MAE official repo](https://github.com/facebookresearch/mae).
+
+## Results from this repo on MS COCO val set (single-task training)
+
+Using detection results from a detector that obtains 56 mAP on person. The configs here are for both training and test.
+
+> With classic decoder
+
+| Model | Pretrain | Resolution | AP | AR | config | log | weight |
+| :----: | :----: | :----: | :----: | :----: | :----: | :----: | :----: |
+| ViTPose-S | MAE | 256x192 | 73.8 | 79.2 | [config](configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/ViTPose_small_coco_256x192.py) | [log](https://1drv.ms/u/s!AimBgYV7JjTlgcchdNXBAh7ClS14pA?e=dKXmJ6) | [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgccifT1XlGRatxg3vw?e=9wz7BY) |
+| ViTPose-B | MAE | 256x192 | 75.8 | 81.1 | [config](configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/ViTPose_base_coco_256x192.py) | [log](logs/vitpose-b.log.json) | [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgSMjp1_NrV3VRSmK?e=Q1uZKs) |
+| ViTPose-L | MAE | 256x192 | 78.3 | 83.5 | [config](configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/ViTPose_large_coco_256x192.py) | [log](logs/vitpose-l.log.json) | [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgSd9k_kuktPtiP4F?e=K7DGYT) |
+| ViTPose-H | MAE | 256x192 | 79.1 | 84.1 | [config](configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/ViTPose_huge_coco_256x192.py) | [log](logs/vitpose-h.log.json) | [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgShLMI-kkmvNfF_h?e=dEhGHe) |
+
+> With simple decoder
+
+| Model | Pretrain | Resolution | AP | AR | config | log | weight |
+| :----: | :----: | :----: | :----: | :----: | :----: | :----: | :----: |
+| ViTPose-S | MAE | 256x192 | 73.5 | 78.9 | [config](configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/ViTPose_small_simple_coco_256x192.py) | [log](https://1drv.ms/u/s!AimBgYV7JjTlgccfkqELJqE67kpRtw?e=InSjJP) | [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgccgb_50jIgiYkHvdw?e=D7RbH2) |
+| ViTPose-B | MAE | 256x192 | 75.5 | 80.9 | [config](configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/ViTPose_base_simple_coco_256x192.py) | [log](logs/vitpose-b-simple.log.json) | [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgSRPKrD5PmDRiv0R?e=jifvOe) |
+| ViTPose-L | MAE | 256x192 | 78.2 | 83.4 | [config](configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/ViTPose_large_simple_coco_256x192.py) | [log](logs/vitpose-l-simple.log.json) | [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgSVS6DP2LmKwZ3sm?e=MmCvDT) |
+| ViTPose-H | MAE | 256x192 | 78.9 | 84.0 | [config](configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/ViTPose_huge_simple_coco_256x192.py) | [log](logs/vitpose-h-simple.log.json) | [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgSbHyN2mjh2n2LyG?e=y0FgMK) |
+
+
+## Results with multi-task training
+
+**Note** \* There may exist duplicate images in the crowdpose training set and the validation images in other datasets, as discussed in [issue #24](https://github.com/ViTAE-Transformer/ViTPose/issues/24). Please be careful when using these models for evaluation. We provide the results without the crowpose dataset for reference.
+
+### Human datasets (MS COCO, AIC, MPII, CrowdPose)
+> Results on MS COCO val set
+
+Using detection results from a detector that obtains 56 mAP on person. Note the configs here are only for evaluation.
+
+| Model | Dataset | Resolution | AP | AR | config | weight |
+| :----: | :----: | :----: | :----: | :----: | :----: | :----: |
+| ViTPose-B | COCO+AIC+MPII | 256x192 | 77.1 | 82.2 | [config](configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/ViTPose_base_coco_256x192.py) | [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgcccwaTZ8xCFFM3Sjg?e=chmiK5) |
+| ViTPose-L | COCO+AIC+MPII | 256x192 | 78.7 | 83.8 | [config](configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/ViTPose_large_coco_256x192.py) | [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgccdOLQqSo6E87GfMw?e=TEurgW) |
+| ViTPose-H | COCO+AIC+MPII | 256x192 | 79.5 | 84.5 | [config](configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/ViTPose_huge_coco_256x192.py) | [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgccmHofkmfJDQDukVw?e=gRK224) |
+| ViTPose-G | COCO+AIC+MPII | 576x432 | 81.0 | 85.6 | | |
+| ViTPose-B* | COCO+AIC+MPII+CrowdPose | 256x192 | 77.5 | 82.6 | [config](configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/ViTPose_base_coco_256x192.py) |[Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgSrlMB093JzJtqq-?e=Jr5S3R) |
+| ViTPose-L* | COCO+AIC+MPII+CrowdPose | 256x192 | 79.1 | 84.1 | [config](configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/ViTPose_large_coco_256x192.py) | [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgTBm3dCVmBUbHYT6?e=fHUrTq) |
+| ViTPose-H* | COCO+AIC+MPII+CrowdPose | 256x192 | 79.8 | 84.8 | [config](configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/ViTPose_huge_coco_256x192.py) | [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgS5rLeRAJiWobCdh?e=41GsDd) |
+| **ViTPose+-S** | COCO+AIC+MPII+AP10K+APT36K+WholeBody | 256x192 | 75.8 | 82.6 | [config](configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/vitPose+_small_coco+aic+mpii+ap10k+apt36k+wholebody_256x192_udp.py) | [log](https://1drv.ms/u/s!AimBgYV7JjTlgccqO1JBHtBjNaeCbQ?e=ZN5NSz) \| [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgccrwORr61gT9E4n8g?e=kz9sz5) |
+| **ViTPose+-B** | COCO+AIC+MPII+AP10K+APT36K+WholeBody | 256x192 | 77.0 | 82.6 | [config](configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/vitPose+_base_coco+aic+mpii+ap10k+apt36k+wholebody_256x192_udp.py) | [log](https://1drv.ms/u/s!AimBgYV7JjTlgccjj9lgPTlkGT1HTw?e=OlS5zv) \| [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgcckRZk1bIAuRa_E1w?e=ylDB2G) |
+| **ViTPose+-L** | COCO+AIC+MPII+AP10K+APT36K+WholeBody | 256x192 | 78.6 | 84.1 | [config](configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/vitPose+_large_coco+aic+mpii+ap10k+apt36k+wholebody_256x192_udp.py) | [log](https://1drv.ms/u/s!AimBgYV7JjTlgccp7HJf4QMeQQpeyA?e=JagPNt) \| [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgccs1SNFUGSTsmRJ8w?e=a9zKwZ) |
+| **ViTPose+-H** | COCO+AIC+MPII+AP10K+APT36K+WholeBody | 256x192 | 79.4 | 84.8 | [config](configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/vitPose+_huge_coco+aic+mpii+ap10k+apt36k+wholebody_256x192_udp.py) | [log](https://1drv.ms/u/s!AimBgYV7JjTlgcclxZOlwRJdqpIIjA?e=nFQgVC) \| [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgccoXv8rCUgVe7oD9Q?e=ZBw6gR) |
+
+
+> Results on OCHuman test set
+
+Using groundtruth bounding boxes. Note the configs here are only for evaluation.
+
+| Model | Dataset | Resolution | AP | AR | config | weight |
+| :----: | :----: | :----: | :----: | :----: | :----: | :----: |
+| ViTPose-B | COCO+AIC+MPII | 256x192 | 88.0 | 89.6 | [config](configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/ochuman/ViTPose_base_ochuman_256x192.py) | [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgcccwaTZ8xCFFM3Sjg?e=chmiK5) |
+| ViTPose-L | COCO+AIC+MPII | 256x192 | 90.9 | 92.2 | [config](configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/ochuman/ViTPose_large_ochuman_256x192.py) | [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgccdOLQqSo6E87GfMw?e=TEurgW) |
+| ViTPose-H | COCO+AIC+MPII | 256x192 | 90.9 | 92.3 | [config](configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/ochuman/ViTPose_huge_ochuman_256x192.py) | [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgccmHofkmfJDQDukVw?e=gRK224) |
+| ViTPose-G | COCO+AIC+MPII | 576x432 | 93.3 | 94.3 | | |
+| ViTPose-B* | COCO+AIC+MPII+CrowdPose | 256x192 | 88.2 | 90.0 | [config](configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/ochuman/ViTPose_base_ochuman_256x192.py) |[Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgSrlMB093JzJtqq-?e=Jr5S3R) |
+| ViTPose-L* | COCO+AIC+MPII+CrowdPose | 256x192 | 91.5 | 92.8 | [config](configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/ochuman/ViTPose_large_ochuman_256x192.py) | [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgTBm3dCVmBUbHYT6?e=fHUrTq) |
+| ViTPose-H* | COCO+AIC+MPII+CrowdPose | 256x192 | 91.6 | 92.8 | [config](configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/ochuman/ViTPose_huge_ochuman_256x192.py) | [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgS5rLeRAJiWobCdh?e=41GsDd) |
+| **ViTPose+-S** | COCO+AIC+MPII+AP10K+APT36K+WholeBody | 256x192 | 78.4 | 80.6 | [config](configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/ochuman/ViTPose_small_ochuman_256x192.py) | [log](https://1drv.ms/u/s!AimBgYV7JjTlgccqO1JBHtBjNaeCbQ?e=ZN5NSz) \| [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgccrwORr61gT9E4n8g?e=kz9sz5) |
+| **ViTPose+-B** | COCO+AIC+MPII+AP10K+APT36K+WholeBody | 256x192 | 82.6 | 84.8 | [config](configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/ochuman/ViTPose_base_ochuman_256x192.py) | [log](https://1drv.ms/u/s!AimBgYV7JjTlgccjj9lgPTlkGT1HTw?e=OlS5zv) \| [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgcckRZk1bIAuRa_E1w?e=ylDB2G) |
+| **ViTPose+-L** | COCO+AIC+MPII+AP10K+APT36K+WholeBody | 256x192 | 85.7 | 87.5 | [config](configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/ochuman/ViTPose_large_ochuman_256x192.py) | [log](https://1drv.ms/u/s!AimBgYV7JjTlgccp7HJf4QMeQQpeyA?e=JagPNt) \| [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgccs1SNFUGSTsmRJ8w?e=a9zKwZ) |
+| **ViTPose+-H** | COCO+AIC+MPII+AP10K+APT36K+WholeBody | 256x192 | 85.7 | 87.4 | [config](configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/ochuman/ViTPose_huge_ochuman_256x192.py) | [log](https://1drv.ms/u/s!AimBgYV7JjTlgcclxZOlwRJdqpIIjA?e=nFQgVC) \| [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgccoXv8rCUgVe7oD9Q?e=ZBw6gR) |
+
+> Results on MPII val set
+
+Using groundtruth bounding boxes. Note the configs here are only for evaluation. The metric is PCKh.
+
+| Model | Dataset | Resolution | Mean | config | weight |
+| :----: | :----: | :----: | :----: | :----: | :----: |
+| ViTPose-B | COCO+AIC+MPII | 256x192 | 93.3 | [config](configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/mpii/ViTPose_base_mpii_256x192.py) | [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgcccwaTZ8xCFFM3Sjg?e=chmiK5) |
+| ViTPose-L | COCO+AIC+MPII | 256x192 | 94.0 | [config](configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/mpii/ViTPose_large_mpii_256x192.py) | [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgccdOLQqSo6E87GfMw?e=TEurgW) |
+| ViTPose-H | COCO+AIC+MPII | 256x192 | 94.1 | [config](configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/mpii/ViTPose_huge_mpii_256x192.py) | [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgccmHofkmfJDQDukVw?e=gRK224) |
+| ViTPose-G | COCO+AIC+MPII | 576x432 | 94.3 | | |
+| ViTPose-B* | COCO+AIC+MPII+CrowdPose | 256x192 | 93.4 | [config](configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/mpii/ViTPose_base_mpii_256x192.py) |[Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgSy_OSEm906wd2LB?e=GOSg14) |
+| ViTPose-L* | COCO+AIC+MPII+CrowdPose | 256x192 | 93.9 | [config](configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/mpii/ViTPose_large_mpii_256x192.py) | [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgTM32I6Kpjr-esl6?e=qvh0Yl) |
+| ViTPose-H* | COCO+AIC+MPII+CrowdPose | 256x192 | 94.1 | [config](configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/mpii/ViTPose_huge_mpii_256x192.py) | [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgTT90XEQBKy-scIH?e=D2WhTS) |
+| **ViTPose+-S** | COCO+AIC+MPII+AP10K+APT36K+WholeBody | 256x192 | 92.7 | [config](configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/mpii/ViTPose_small_mpii_256x192.py) | [log](https://1drv.ms/u/s!AimBgYV7JjTlgccqO1JBHtBjNaeCbQ?e=ZN5NSz) \| [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgccrwORr61gT9E4n8g?e=kz9sz5) |
+| **ViTPose+-B** | COCO+AIC+MPII+AP10K+APT36K+WholeBody | 256x192 | 92.8 | [config](configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/mpii/ViTPose_base_mpii_256x192.py) | [log](https://1drv.ms/u/s!AimBgYV7JjTlgccjj9lgPTlkGT1HTw?e=OlS5zv) \| [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgcckRZk1bIAuRa_E1w?e=ylDB2G) |
+| **ViTPose+-L** | COCO+AIC+MPII+AP10K+APT36K+WholeBody | 256x192 | 94.0 | [config](configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/mpii/ViTPose_large_mpii_256x192.py) | [log](https://1drv.ms/u/s!AimBgYV7JjTlgccp7HJf4QMeQQpeyA?e=JagPNt) \| [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgccs1SNFUGSTsmRJ8w?e=a9zKwZ) |
+| **ViTPose+-H** | COCO+AIC+MPII+AP10K+APT36K+WholeBody | 256x192 | 94.2 | [config](configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/mpii/ViTPose_huge_mpii_256x192.py) | [log](https://1drv.ms/u/s!AimBgYV7JjTlgcclxZOlwRJdqpIIjA?e=nFQgVC) \| [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgccoXv8rCUgVe7oD9Q?e=ZBw6gR) |
+
+
+> Results on AI Challenger test set
+
+Using groundtruth bounding boxes. Note the configs here are only for evaluation.
+
+| Model | Dataset | Resolution | AP | AR | config | weight |
+| :----: | :----: | :----: | :----: | :----: | :----: | :----: |
+| ViTPose-B | COCO+AIC+MPII | 256x192 | 32.0 | 36.3 | [config](configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/aic/ViTPose_base_aic_256x192.py) | [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgcccwaTZ8xCFFM3Sjg?e=chmiK5) |
+| ViTPose-L | COCO+AIC+MPII | 256x192 | 34.5 | 39.0 | [config](configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/aic/ViTPose_large_aic_256x192.py) | [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgccdOLQqSo6E87GfMw?e=TEurgW) |
+| ViTPose-H | COCO+AIC+MPII | 256x192 | 35.4 | 39.9 | [config](configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/aic/ViTPose_huge_aic_256x192.py) | [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgccmHofkmfJDQDukVw?e=gRK224) |
+| ViTPose-G | COCO+AIC+MPII | 576x432 | 43.2 | 47.1 | | |
+| ViTPose-B* | COCO+AIC+MPII+CrowdPose | 256x192 | 31.9 | 36.3 | [config](configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/aic/ViTPose_base_aic_256x192.py) |[Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgSlvdVaXTC92SHYH?e=j7iqcp) |
+| ViTPose-L* | COCO+AIC+MPII+CrowdPose | 256x192 | 34.6 | 39.0 | [config](configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/aic/ViTPose_large_aic_256x192.py) | [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgTF06FX3FSAm0MOH?e=rYts9F) |
+| ViTPose-H* | COCO+AIC+MPII+CrowdPose | 256x192 | 35.3 | 39.8 | [config](configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/aic/ViTPose_huge_aic_256x192.py) | [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgS1MRmb2mcow_K04?e=q9jPab) |
+| **ViTPose+-S** | COCO+AIC+MPII+AP10K+APT36K+WholeBody | 256x192 | 29.7 | 34.3 | [config](configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/ochuman/ViTPose_small_ochuman_256x192.py) | [log](https://1drv.ms/u/s!AimBgYV7JjTlgccqO1JBHtBjNaeCbQ?e=ZN5NSz) \| [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgccrwORr61gT9E4n8g?e=kz9sz5) |
+| **ViTPose+-B** | COCO+AIC+MPII+AP10K+APT36K+WholeBody | 256x192 | 31.8 | 36.3 | [config](configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/ochuman/ViTPose_base_ochuman_256x192.py) | [log](https://1drv.ms/u/s!AimBgYV7JjTlgccjj9lgPTlkGT1HTw?e=OlS5zv) \| [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgcckRZk1bIAuRa_E1w?e=ylDB2G) |
+| **ViTPose+-L** | COCO+AIC+MPII+AP10K+APT36K+WholeBody | 256x192 | 34.3 | 38.9 | [config](configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/ochuman/ViTPose_large_ochuman_256x192.py) | [log](https://1drv.ms/u/s!AimBgYV7JjTlgccp7HJf4QMeQQpeyA?e=JagPNt) \| [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgccs1SNFUGSTsmRJ8w?e=a9zKwZ) |
+| **ViTPose+-H** | COCO+AIC+MPII+AP10K+APT36K+WholeBody | 256x192 | 34.8 | 39.1 | [config](configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/ochuman/ViTPose_huge_ochuman_256x192.py) | [log](https://1drv.ms/u/s!AimBgYV7JjTlgcclxZOlwRJdqpIIjA?e=nFQgVC) \| [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgccoXv8rCUgVe7oD9Q?e=ZBw6gR) |
+
+> Results on CrowdPose test set
+
+Using YOLOv3 human detector. Note the configs here are only for evaluation.
+
+| Model | Dataset | Resolution | AP | AP(H) | config | weight |
+| :----: | :----: | :----: | :----: | :----: | :----: | :----: |
+| ViTPose-B* | COCO+AIC+MPII+CrowdPose | 256x192 | 74.7 | 63.3 | [config](configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/crowdpose/ViTPose_base_crowdpose_256x192.py) |[Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgStrrCb91cPlaxJx?e=6Xobo6) |
+| ViTPose-L* | COCO+AIC+MPII+CrowdPose | 256x192 | 76.6 | 65.9 | [config](configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/crowdpose/ViTPose_large_crowdpose_256x192.py) | [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgTK3dug-r7c6GFyu?e=1ZBpEG) |
+| ViTPose-H* | COCO+AIC+MPII+CrowdPose | 256x192 | 76.3 | 65.6 | [config](configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/crowdpose/ViTPose_huge_crowdpose_256x192.py) | [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgS-oAvEV4MTD--Xr?e=EeW2Fu) |
+
+### Animal datasets (AP10K, APT36K)
+
+> Results on AP-10K test set
+
+| Model | Dataset | Resolution | AP | config | weight |
+| :----: | :----: | :----: | :----: | :----: | :----: |
+| **ViTPose+-S** | COCO+AIC+MPII+AP10K+APT36K+WholeBody | 256x192 | 71.4 | [config](configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap/ap10k/ViTPose_small_ap10k_256x192.py) | [log](https://1drv.ms/u/s!AimBgYV7JjTlgccqO1JBHtBjNaeCbQ?e=ZN5NSz) \| [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgccrwORr61gT9E4n8g?e=kz9sz5) |
+| **ViTPose+-B** | COCO+AIC+MPII+AP10K+APT36K+WholeBody | 256x192 | 74.5 | [config](configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap/ap10k/ViTPose_base_ap10k_256x192.py) | [log](https://1drv.ms/u/s!AimBgYV7JjTlgccjj9lgPTlkGT1HTw?e=OlS5zv) \| [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgcckRZk1bIAuRa_E1w?e=ylDB2G) |
+| **ViTPose+-L** | COCO+AIC+MPII+AP10K+APT36K+WholeBody | 256x192 | 80.4 | [config](configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap/ap10k/ViTPose_large_ap10k_256x192.py) | [log](https://1drv.ms/u/s!AimBgYV7JjTlgccp7HJf4QMeQQpeyA?e=JagPNt) \| [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgccs1SNFUGSTsmRJ8w?e=a9zKwZ) |
+| **ViTPose+-H** | COCO+AIC+MPII+AP10K+APT36K+WholeBody | 256x192 | 82.4 | [config](configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap/ap10k/ViTPose_huge_ap10k_256x192.py) | [log](https://1drv.ms/u/s!AimBgYV7JjTlgcclxZOlwRJdqpIIjA?e=nFQgVC) \| [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgccoXv8rCUgVe7oD9Q?e=ZBw6gR) |
+
+> Results on APT-36K val set
+
+| Model | Dataset | Resolution | AP | config | weight |
+| :----: | :----: | :----: | :----: | :----: | :----: |
+| **ViTPose+-S** | COCO+AIC+MPII+AP10K+APT36K+WholeBody | 256x192 | 74.2 | [config](configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap/apt36k/ViTPose_small_apt36k_256x192.py) | [log](https://1drv.ms/u/s!AimBgYV7JjTlgccqO1JBHtBjNaeCbQ?e=ZN5NSz) \| [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgccrwORr61gT9E4n8g?e=kz9sz5) |
+| **ViTPose+-B** | COCO+AIC+MPII+AP10K+APT36K+WholeBody | 256x192 | 75.9 | [config](configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap/apt36k/ViTPose_base_apt36k_256x192.py) | [log](https://1drv.ms/u/s!AimBgYV7JjTlgccjj9lgPTlkGT1HTw?e=OlS5zv) \| [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgcckRZk1bIAuRa_E1w?e=ylDB2G) |
+| **ViTPose+-L** | COCO+AIC+MPII+AP10K+APT36K+WholeBody | 256x192 | 80.8 | [config](configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap/apt36k/ViTPose_large_apt36k_256x192.py) | [log](https://1drv.ms/u/s!AimBgYV7JjTlgccp7HJf4QMeQQpeyA?e=JagPNt) \| [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgccs1SNFUGSTsmRJ8w?e=a9zKwZ) |
+| **ViTPose+-H** | COCO+AIC+MPII+AP10K+APT36K+WholeBody | 256x192 | 82.3 | [config](configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap/apt36k/ViTPose_huge_apt36k_256x192.py) | [log](https://1drv.ms/u/s!AimBgYV7JjTlgcclxZOlwRJdqpIIjA?e=nFQgVC) \| [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgccoXv8rCUgVe7oD9Q?e=ZBw6gR) |
+
+### WholeBody dataset
+
+| Model | Dataset | Resolution | AP | config | weight |
+| :----: | :----: | :----: | :----: | :----: | :----: |
+| **ViTPose+-S** | COCO+AIC+MPII+AP10K+APT36K+WholeBody | 256x192 | 54.4 | [config](configs/wholebody/2d_kpt_sview_rgb_img/topdown_heatmap/coco-wholebody/ViTPose_small_wholebody_256x192.py) | [log](https://1drv.ms/u/s!AimBgYV7JjTlgccqO1JBHtBjNaeCbQ?e=ZN5NSz) \| [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgccrwORr61gT9E4n8g?e=kz9sz5) |
+| **ViTPose+-B** | COCO+AIC+MPII+AP10K+APT36K+WholeBody | 256x192 | 57.4 | [config](cconfigs/wholebody/2d_kpt_sview_rgb_img/topdown_heatmap/coco-wholebody/ViTPose_base_wholebody_256x192.py) | [log](https://1drv.ms/u/s!AimBgYV7JjTlgccjj9lgPTlkGT1HTw?e=OlS5zv) \| [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgcckRZk1bIAuRa_E1w?e=ylDB2G) |
+| **ViTPose+-L** | COCO+AIC+MPII+AP10K+APT36K+WholeBody | 256x192 | 60.6 | [config](configs/wholebody/2d_kpt_sview_rgb_img/topdown_heatmap/coco-wholebody/ViTPose_large_wholebody_256x192.py) | [log](https://1drv.ms/u/s!AimBgYV7JjTlgccp7HJf4QMeQQpeyA?e=JagPNt) \| [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgccs1SNFUGSTsmRJ8w?e=a9zKwZ) |
+| **ViTPose+-H** | COCO+AIC+MPII+AP10K+APT36K+WholeBody | 256x192 | 61.2 | [config](configs/wholebody/2d_kpt_sview_rgb_img/topdown_heatmap/coco-wholebody/ViTPose_huge_wholebody_256x192.py) | [log](https://1drv.ms/u/s!AimBgYV7JjTlgcclxZOlwRJdqpIIjA?e=nFQgVC) \| [Onedrive](https://1drv.ms/u/s!AimBgYV7JjTlgccoXv8rCUgVe7oD9Q?e=ZBw6gR) |
+
+### Transfer results on the hand dataset (InterHand2.6M)
+
+| Model | Dataset | Resolution | AUC | config | weight |
+| :----: | :----: | :----: | :----: | :----: | :----: |
+| **ViTPose+-S** | COCO+AIC+MPII+WholeBody | 256x192 | 86.5 | [config](configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/interhand2d/ViTPose_small_interhand2d_all_256x192.py) | Coming Soon |
+| **ViTPose+-B** | COCO+AIC+MPII+WholeBody | 256x192 | 87.0 | [config](configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/interhand2d/ViTPose_base_interhand2d_all_256x192.py) | Coming Soon |
+| **ViTPose+-L** | COCO+AIC+MPII+WholeBody | 256x192 | 87.5 | [config](configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/interhand2d/ViTPose_large_interhand2d_all_256x192.py) | Coming Soon |
+| **ViTPose+-H** | COCO+AIC+MPII+WholeBody | 256x192 | 87.6 | [config](configs/hand/2d_kpt_sview_rgb_img/topdown_heatmap/interhand2d/ViTPose_huge_interhand2d_all_256x192.py) | Coming Soon |
+
+## Updates
+
+> [2023-01-10] Update ViTPose+! It uses MoE strategies to jointly deal with human, animal, and wholebody pose estimation tasks.
+
+> [2022-05-24] Upload the single-task training code, single-task pre-trained models, and multi-task pretrained models.
+
+> [2022-05-06] Upload the logs for the base, large, and huge models!
+
+> [2022-04-27] Our ViTPose with ViTAE-G obtains 81.1 AP on COCO test-dev set!
+
+> Applications of ViTAE Transformer include: [image classification](https://github.com/ViTAE-Transformer/ViTAE-Transformer/tree/main/Image-Classification) | [object detection](https://github.com/ViTAE-Transformer/ViTAE-Transformer/tree/main/Object-Detection) | [semantic segmentation](https://github.com/ViTAE-Transformer/ViTAE-Transformer/tree/main/Semantic-Segmentation) | [animal pose segmentation](https://github.com/ViTAE-Transformer/ViTAE-Transformer/tree/main/Animal-Pose-Estimation) | [remote sensing](https://github.com/ViTAE-Transformer/ViTAE-Transformer-Remote-Sensing) | [matting](https://github.com/ViTAE-Transformer/ViTAE-Transformer-Matting) | [VSA](https://github.com/ViTAE-Transformer/ViTAE-VSA) | [ViTDet](https://github.com/ViTAE-Transformer/ViTDet)
+
+## Usage
+
+We use PyTorch 1.9.0 or NGC docker 21.06, and mmcv 1.3.9 for the experiments.
+```bash
+git clone https://github.com/open-mmlab/mmcv.git
+cd mmcv
+git checkout v1.3.9
+MMCV_WITH_OPS=1 pip install -e .
+cd ..
+git clone https://github.com/ViTAE-Transformer/ViTPose.git
+cd ViTPose
+pip install -v -e .
+```
+
+After install the two repos, install timm and einops, i.e.,
+```bash
+pip install timm==0.4.9 einops
+```
+
+After downloading the pretrained models, please conduct the experiments by running
+
+```bash
+# for single machine
+bash tools/dist_train.sh 
diff --git a/engine/pose_estimation/third-party/ViTPose/configs/body/2d_kpt_sview_rgb_img/associative_embedding/README.md b/engine/pose_estimation/third-party/ViTPose/configs/body/2d_kpt_sview_rgb_img/associative_embedding/README.md
new file mode 100644
index 0000000..2048f21
--- /dev/null
+++ b/engine/pose_estimation/third-party/ViTPose/configs/body/2d_kpt_sview_rgb_img/associative_embedding/README.md
@@ -0,0 +1,25 @@
+# Associative embedding: End-to-end learning for joint detection and grouping (AE)
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+![\"Open](\"https://colab.research.google.com/assets/colab-badge.svg\"){kind=icon}
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "9_h0e90xzw0w"
+ },
+ "source": [
+ "# MMPose Tutorial\n",
+ "\n",
+ "Welcome to MMPose colab tutorial! In this tutorial, we will show you how to\n",
+ "- perform inference with an MMPose model\n",
+ "- train a new mmpose model with your own datasets\n",
+ "\n",
+ "Let's start!"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "bMVTUneIzw0x"
+ },
+ "source": [
+ "## Install MMPose\n",
+ "\n",
+ "We recommend to use a conda environment to install mmpose and its dependencies. And compilers `nvcc` and `gcc` are required."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "9dvKWH89zw0x",
+ "outputId": "c3e29ad4-6a1b-4ef8-ec45-93196de7ffae"
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "nvcc: NVIDIA (R) Cuda compiler driver\n",
+ "Copyright (c) 2005-2020 NVIDIA Corporation\n",
+ "Built on Tue_Sep_15_19:10:02_PDT_2020\n",
+ "Cuda compilation tools, release 11.1, V11.1.74\n",
+ "Build cuda_11.1.TC455_06.29069683_0\n",
+ "gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0\n",
+ "Copyright (C) 2019 Free Software Foundation, Inc.\n",
+ "This is free software; see the source for copying conditions. There is NO\n",
+ "warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.\n",
+ "\n",
+ "/home/PJLAB/liyining/anaconda3/envs/pt1.9/bin/python\n"
+ ]
+ }
+ ],
+ "source": [
+ "# check NVCC version\n",
+ "!nvcc -V\n",
+ "\n",
+ "# check GCC version\n",
+ "!gcc --version\n",
+ "\n",
+ "# check python in conda environment\n",
+ "!which python"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "26-3yY31zw0y",
+ "outputId": "fad7fbc2-ae00-4e4b-fa80-a0d16c0a4ac3"
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Requirement already satisfied: mmcv-full in /home/SENSETIME/liyining/anaconda3/envs/colab/lib/python3.9/site-packages (1.3.9)\r\n",
+ "Requirement already satisfied: Pillow in /home/SENSETIME/liyining/anaconda3/envs/colab/lib/python3.9/site-packages (from mmcv-full) (8.3.1)\r\n",
+ "Requirement already satisfied: yapf in /home/SENSETIME/liyining/anaconda3/envs/colab/lib/python3.9/site-packages (from mmcv-full) (0.31.0)\r\n",
+ "Requirement already satisfied: pyyaml in /home/SENSETIME/liyining/anaconda3/envs/colab/lib/python3.9/site-packages (from mmcv-full) (5.4.1)\r\n",
+ "Requirement already satisfied: addict in /home/SENSETIME/liyining/anaconda3/envs/colab/lib/python3.9/site-packages (from mmcv-full) (2.4.0)\r\n",
+ "Requirement already satisfied: numpy in /home/SENSETIME/liyining/anaconda3/envs/colab/lib/python3.9/site-packages (from mmcv-full) (1.21.1)\n",
+ "Requirement already satisfied: mmdet in /home/SENSETIME/liyining/anaconda3/envs/colab/lib/python3.9/site-packages (2.15.0)\n",
+ "Requirement already satisfied: numpy in /home/SENSETIME/liyining/anaconda3/envs/colab/lib/python3.9/site-packages (from mmdet) (1.21.1)\n",
+ "Requirement already satisfied: terminaltables in /home/SENSETIME/liyining/anaconda3/envs/colab/lib/python3.9/site-packages (from mmdet) (3.1.0)\n",
+ "Requirement already satisfied: pycocotools in /home/SENSETIME/liyining/anaconda3/envs/colab/lib/python3.9/site-packages (from mmdet) (2.0.2)\n",
+ "Requirement already satisfied: six in /home/SENSETIME/liyining/anaconda3/envs/colab/lib/python3.9/site-packages (from mmdet) (1.16.0)\n",
+ "Requirement already satisfied: matplotlib in /home/SENSETIME/liyining/anaconda3/envs/colab/lib/python3.9/site-packages (from mmdet) (3.4.2)\n",
+ "Requirement already satisfied: kiwisolver>=1.0.1 in /home/SENSETIME/liyining/anaconda3/envs/colab/lib/python3.9/site-packages (from matplotlib->mmdet) (1.3.1)\n",
+ "Requirement already satisfied: cycler>=0.10 in /home/SENSETIME/liyining/anaconda3/envs/colab/lib/python3.9/site-packages (from matplotlib->mmdet) (0.10.0)\n",
+ "Requirement already satisfied: python-dateutil>=2.7 in /home/SENSETIME/liyining/anaconda3/envs/colab/lib/python3.9/site-packages (from matplotlib->mmdet) (2.8.2)\n",
+ "Requirement already satisfied: pyparsing>=2.2.1 in /home/SENSETIME/liyining/anaconda3/envs/colab/lib/python3.9/site-packages (from matplotlib->mmdet) (2.4.7)\n",
+ "Requirement already satisfied: pillow>=6.2.0 in /home/SENSETIME/liyining/anaconda3/envs/colab/lib/python3.9/site-packages (from matplotlib->mmdet) (8.3.1)\n",
+ "Requirement already satisfied: cython>=0.27.3 in /home/SENSETIME/liyining/anaconda3/envs/colab/lib/python3.9/site-packages (from pycocotools->mmdet) (0.29.24)\n",
+ "Requirement already satisfied: setuptools>=18.0 in /home/SENSETIME/liyining/anaconda3/envs/colab/lib/python3.9/site-packages (from pycocotools->mmdet) (52.0.0.post20210125)\n",
+ "Cloning into 'mmpose'...\n",
+ "remote: Enumerating objects: 12253, done.\u001b[K\n",
+ "remote: Counting objects: 100% (4193/4193), done.\u001b[K\n",
+ "remote: Compressing objects: 100% (1401/1401), done.\u001b[K\n",
+ "remote: Total 12253 (delta 3029), reused 3479 (delta 2695), pack-reused 8060\u001b[K\n",
+ "Receiving objects: 100% (12253/12253), 21.00 MiB | 2.92 MiB/s, done.\n",
+ "Resolving deltas: 100% (8230/8230), done.\n",
+ "Checking connectivity... done.\n",
+ "/home/SENSETIME/liyining/openmmlab/misc/colab/mmpose\n",
+ "Ignoring dataclasses: markers 'python_version == \"3.6\"' don't match your environment\n",
+ "Collecting poseval@ git+https://github.com/svenkreiss/poseval.git\n",
+ " Cloning https://github.com/svenkreiss/poseval.git to /tmp/pip-install-d12g7njf/poseval_66b19fe8a11a4135b1a0064566177a26\n",
+ " Running command git clone -q https://github.com/svenkreiss/poseval.git /tmp/pip-install-d12g7njf/poseval_66b19fe8a11a4135b1a0064566177a26\n",
+ " Resolved https://github.com/svenkreiss/poseval.git to commit 3128c5cbcf90946e5164ff438ad651e113e64613\n",
+ " Running command git submodule update --init --recursive -q\n",
+ "Requirement already satisfied: numpy in /home/SENSETIME/liyining/anaconda3/envs/colab/lib/python3.9/site-packages (from -r requirements/build.txt (line 2)) (1.21.1)\n",
+ "Collecting torch>=1.3\n",
+ " Using cached torch-1.9.0-cp39-cp39-manylinux1_x86_64.whl (831.4 MB)\n",
+ "Collecting chumpy\n",
+ " Using cached chumpy-0.70-py3-none-any.whl\n",
+ "Collecting json_tricks\n",
+ " Using cached json_tricks-3.15.5-py2.py3-none-any.whl (26 kB)\n",
+ "Requirement already satisfied: matplotlib in /home/SENSETIME/liyining/anaconda3/envs/colab/lib/python3.9/site-packages (from -r requirements/runtime.txt (line 4)) (3.4.2)\n",
+ "Collecting munkres\n",
+ " Using cached munkres-1.1.4-py2.py3-none-any.whl (7.0 kB)\n",
+ "Collecting opencv-python\n",
+ " Using cached opencv_python-4.5.3.56-cp39-cp39-manylinux2014_x86_64.whl (49.9 MB)\n",
+ "Requirement already satisfied: pillow in /home/SENSETIME/liyining/anaconda3/envs/colab/lib/python3.9/site-packages (from -r requirements/runtime.txt (line 8)) (8.3.1)\n",
+ "Collecting scipy\n",
+ " Using cached scipy-1.7.1-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.whl (28.5 MB)\n",
+ "Collecting torchvision\n",
+ " Using cached torchvision-0.10.0-cp39-cp39-manylinux1_x86_64.whl (22.1 MB)\n",
+ "Collecting xtcocotools>=1.8\n",
+ " Downloading xtcocotools-1.10-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.whl (303 kB)\n",
+ "\u001b[K |████████████████████████████████| 303 kB 1.1 MB/s \n",
+ "\u001b[?25hCollecting coverage\n",
+ " Using cached coverage-5.5-cp39-cp39-manylinux2010_x86_64.whl (243 kB)\n",
+ "Collecting flake8\n",
+ " Using cached flake8-3.9.2-py2.py3-none-any.whl (73 kB)\n",
+ "Collecting interrogate\n",
+ " Using cached interrogate-1.4.0-py3-none-any.whl (28 kB)\n",
+ "Collecting isort==4.3.21\n",
+ " Using cached isort-4.3.21-py2.py3-none-any.whl (42 kB)\n",
+ "Collecting pytest\n",
+ " Using cached pytest-6.2.4-py3-none-any.whl (280 kB)\n",
+ "Collecting pytest-runner\n",
+ " Using cached pytest_runner-5.3.1-py3-none-any.whl (7.1 kB)\n",
+ "Collecting smplx>=0.1.28\n",
+ " Using cached smplx-0.1.28-py3-none-any.whl (29 kB)\n",
+ "Collecting xdoctest>=0.10.0\n",
+ " Using cached xdoctest-0.15.5-py3-none-any.whl (113 kB)\n",
+ "Requirement already satisfied: yapf in /home/SENSETIME/liyining/anaconda3/envs/colab/lib/python3.9/site-packages (from -r requirements/tests.txt (line 9)) (0.31.0)\n",
+ "Collecting albumentations>=0.3.2\n",
+ " Using cached albumentations-1.0.3.tar.gz (173 kB)\n",
+ "Collecting onnx\n",
+ " Downloading onnx-1.10.1-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (12.3 MB)\n",
+ "\u001b[K |████████████████████████████████| 12.3 MB 4.1 MB/s \n",
+ "\u001b[?25hCollecting onnxruntime\n",
+ " Using cached onnxruntime-1.8.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.5 MB)\n",
+ "Collecting pyrender\n",
+ " Using cached pyrender-0.1.45-py3-none-any.whl (1.2 MB)\n",
+ "Collecting trimesh\n",
+ " Downloading trimesh-3.9.26-py3-none-any.whl (634 kB)\n",
+ "\u001b[K |████████████████████████████████| 634 kB 978 kB/s \n",
+ "\u001b[?25hCollecting typing-extensions\n",
+ " Using cached typing_extensions-3.10.0.0-py3-none-any.whl (26 kB)\n",
+ "Requirement already satisfied: six>=1.11.0 in /home/SENSETIME/liyining/anaconda3/envs/colab/lib/python3.9/site-packages (from chumpy->-r requirements/runtime.txt (line 1)) (1.16.0)\n",
+ "Requirement already satisfied: python-dateutil>=2.7 in /home/SENSETIME/liyining/anaconda3/envs/colab/lib/python3.9/site-packages (from matplotlib->-r requirements/runtime.txt (line 4)) (2.8.2)\n",
+ "Requirement already satisfied: cycler>=0.10 in /home/SENSETIME/liyining/anaconda3/envs/colab/lib/python3.9/site-packages (from matplotlib->-r requirements/runtime.txt (line 4)) (0.10.0)\n",
+ "Requirement already satisfied: pyparsing>=2.2.1 in /home/SENSETIME/liyining/anaconda3/envs/colab/lib/python3.9/site-packages (from matplotlib->-r requirements/runtime.txt (line 4)) (2.4.7)\n",
+ "Requirement already satisfied: kiwisolver>=1.0.1 in /home/SENSETIME/liyining/anaconda3/envs/colab/lib/python3.9/site-packages (from matplotlib->-r requirements/runtime.txt (line 4)) (1.3.1)\n",
+ "Requirement already satisfied: cython>=0.27.3 in /home/SENSETIME/liyining/anaconda3/envs/colab/lib/python3.9/site-packages (from xtcocotools>=1.8->-r requirements/runtime.txt (line 11)) (0.29.24)\n",
+ "Requirement already satisfied: setuptools>=18.0 in /home/SENSETIME/liyining/anaconda3/envs/colab/lib/python3.9/site-packages (from xtcocotools>=1.8->-r requirements/runtime.txt (line 11)) (52.0.0.post20210125)\n",
+ "Collecting mccabe<0.7.0,>=0.6.0\n",
+ " Using cached mccabe-0.6.1-py2.py3-none-any.whl (8.6 kB)\n",
+ "Collecting pycodestyle<2.8.0,>=2.7.0\n",
+ " Using cached pycodestyle-2.7.0-py2.py3-none-any.whl (41 kB)\n",
+ "Collecting pyflakes<2.4.0,>=2.3.0\n",
+ " Using cached pyflakes-2.3.1-py2.py3-none-any.whl (68 kB)\n",
+ "Collecting toml\n",
+ " Using cached toml-0.10.2-py2.py3-none-any.whl (16 kB)\n",
+ "Collecting colorama\n",
+ " Using cached colorama-0.4.4-py2.py3-none-any.whl (16 kB)\n",
+ "Collecting tabulate\n",
+ " Using cached tabulate-0.8.9-py3-none-any.whl (25 kB)\n",
+ "Collecting click\n",
+ " Using cached click-8.0.1-py3-none-any.whl (97 kB)\n",
+ "Collecting py\n",
+ " Using cached py-1.10.0-py2.py3-none-any.whl (97 kB)\n",
+ "Requirement already satisfied: attrs in /home/SENSETIME/liyining/anaconda3/envs/colab/lib/python3.9/site-packages (from interrogate->-r requirements/tests.txt (line 3)) (21.2.0)\n",
+ "Collecting iniconfig\n",
+ " Using cached iniconfig-1.1.1-py2.py3-none-any.whl (5.0 kB)\n",
+ "Requirement already satisfied: packaging in /home/SENSETIME/liyining/anaconda3/envs/colab/lib/python3.9/site-packages (from pytest->-r requirements/tests.txt (line 5)) (21.0)\n",
+ "Collecting pluggy<1.0.0a1,>=0.12\n",
+ " Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)\n",
+ "Collecting scikit-image>=0.16.1\n",
+ " Using cached scikit_image-0.18.2-cp39-cp39-manylinux2010_x86_64.whl (34.6 MB)\n",
+ "Requirement already satisfied: PyYAML in /home/SENSETIME/liyining/anaconda3/envs/colab/lib/python3.9/site-packages (from albumentations>=0.3.2->-r requirements/optional.txt (line 1)) (5.4.1)\n",
+ "Collecting opencv-python-headless>=4.1.1\n",
+ " Using cached opencv_python_headless-4.5.3.56-cp39-cp39-manylinux2014_x86_64.whl (37.1 MB)\n",
+ "Collecting protobuf\n",
+ " Using cached protobuf-3.17.3-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.whl (1.0 MB)\n",
+ "Collecting flatbuffers\n",
+ " Using cached flatbuffers-2.0-py2.py3-none-any.whl (26 kB)\n",
+ "Collecting motmetrics>=1.2\n",
+ " Using cached motmetrics-1.2.0-py3-none-any.whl (151 kB)\n",
+ "Collecting shapely\n",
+ " Using cached Shapely-1.7.1-1-cp39-cp39-manylinux1_x86_64.whl (1.0 MB)\n",
+ "Collecting tqdm\n",
+ " Downloading tqdm-4.62.0-py2.py3-none-any.whl (76 kB)\n",
+ "\u001b[K |████████████████████████████████| 76 kB 1.0 MB/s \n",
+ "\u001b[?25hCollecting networkx\n",
+ " Using cached networkx-2.6.2-py3-none-any.whl (1.9 MB)\n",
+ "Collecting freetype-py\n",
+ " Using cached freetype_py-2.2.0-py3-none-manylinux1_x86_64.whl (890 kB)\n",
+ "Collecting pyglet>=1.4.10\n",
+ " Using cached pyglet-1.5.18-py3-none-any.whl (1.1 MB)\n",
+ "Collecting imageio\n",
+ " Using cached imageio-2.9.0-py3-none-any.whl (3.3 MB)\n",
+ "Collecting PyOpenGL==3.1.0\n",
+ " Using cached PyOpenGL-3.1.0-py3-none-any.whl\n",
+ "Collecting pytest-benchmark\n",
+ " Using cached pytest_benchmark-3.4.1-py2.py3-none-any.whl (50 kB)\n",
+ "Collecting flake8-import-order\n",
+ " Using cached flake8_import_order-0.18.1-py2.py3-none-any.whl (15 kB)\n",
+ "Collecting pandas>=0.23.1\n",
+ " Using cached pandas-1.3.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.7 MB)\n",
+ "Collecting xmltodict>=0.12.0\n",
+ " Using cached xmltodict-0.12.0-py2.py3-none-any.whl (9.2 kB)\n",
+ "Requirement already satisfied: pytz>=2017.3 in /home/SENSETIME/liyining/anaconda3/envs/colab/lib/python3.9/site-packages (from pandas>=0.23.1->motmetrics>=1.2->poseval@ git+https://github.com/svenkreiss/poseval.git->-r requirements/optional.txt (line 4)) (2021.1)\n",
+ "Collecting tifffile>=2019.7.26\n",
+ " Using cached tifffile-2021.7.30-py3-none-any.whl (171 kB)\n",
+ "Collecting PyWavelets>=1.1.1\n",
+ " Using cached PyWavelets-1.1.1-cp39-cp39-manylinux1_x86_64.whl (4.3 MB)\n",
+ "Collecting py-cpuinfo\n",
+ " Using cached py_cpuinfo-8.0.0-py3-none-any.whl\n",
+ "Skipping wheel build for albumentations, due to binaries being disabled for it.\n",
+ "Building wheels for collected packages: poseval\n",
+ " Building wheel for poseval (setup.py) ... \u001b[?25l-\b \b\\\b \bdone\n",
+ "\u001b[?25h Created wheel for poseval: filename=poseval-0.1.0-py3-none-any.whl size=25993 sha256=412ec354869baa10f28ba8938ca6a63c0c9233d8fbb839377f201c398d1cf5a6\n",
+ " Stored in directory: /tmp/pip-ephem-wheel-cache-12d_ns95/wheels/0f/4a/c4/17e52eb6f9f3371b8cf1863940bff5118b00875b66809f9f51\n",
+ "Successfully built poseval\n",
+ "Installing collected packages: toml, py, pluggy, iniconfig, pytest, pyflakes, pycodestyle, py-cpuinfo, mccabe, xmltodict, typing-extensions, tifffile, scipy, PyWavelets, pytest-benchmark, pandas, networkx, imageio, flake8-import-order, flake8, trimesh, tqdm, torch, tabulate, shapely, scikit-image, PyOpenGL, pyglet, protobuf, opencv-python-headless, motmetrics, freetype-py, flatbuffers, colorama, click, xtcocotools, xdoctest, torchvision, smplx, pytest-runner, pyrender, poseval, opencv-python, onnxruntime, onnx, munkres, json-tricks, isort, interrogate, coverage, chumpy, albumentations\n",
+ " Running setup.py install for albumentations ... \u001b[?25l-\b \b\\\b \bdone\n",
+ "\u001b[?25hSuccessfully installed PyOpenGL-3.1.0 PyWavelets-1.1.1 albumentations-1.0.3 chumpy-0.70 click-8.0.1 colorama-0.4.4 coverage-5.5 flake8-3.9.2 flake8-import-order-0.18.1 flatbuffers-2.0 freetype-py-2.2.0 imageio-2.9.0 iniconfig-1.1.1 interrogate-1.4.0 isort-4.3.21 json-tricks-3.15.5 mccabe-0.6.1 motmetrics-1.2.0 munkres-1.1.4 networkx-2.6.2 onnx-1.10.1 onnxruntime-1.8.1 opencv-python-4.5.3.56 opencv-python-headless-4.5.3.56 pandas-1.3.1 pluggy-0.13.1 poseval-0.1.0 protobuf-3.17.3 py-1.10.0 py-cpuinfo-8.0.0 pycodestyle-2.7.0 pyflakes-2.3.1 pyglet-1.5.18 pyrender-0.1.45 pytest-6.2.4 pytest-benchmark-3.4.1 pytest-runner-5.3.1 scikit-image-0.18.2 scipy-1.7.1 shapely-1.7.1 smplx-0.1.28 tabulate-0.8.9 tifffile-2021.7.30 toml-0.10.2 torch-1.9.0 torchvision-0.10.0 tqdm-4.62.0 trimesh-3.9.26 typing-extensions-3.10.0.0 xdoctest-0.15.5 xmltodict-0.12.0 xtcocotools-1.10\n",
+ "Obtaining file:///home/SENSETIME/liyining/openmmlab/misc/colab/mmpose\n",
+ "Requirement already satisfied: chumpy in /home/SENSETIME/liyining/anaconda3/envs/colab/lib/python3.9/site-packages (from mmpose==0.16.0) (0.70)\n",
+ "Requirement already satisfied: json_tricks in /home/SENSETIME/liyining/anaconda3/envs/colab/lib/python3.9/site-packages (from mmpose==0.16.0) (3.15.5)\n",
+ "Requirement already satisfied: matplotlib in /home/SENSETIME/liyining/anaconda3/envs/colab/lib/python3.9/site-packages (from mmpose==0.16.0) (3.4.2)\n",
+ "Requirement already satisfied: munkres in /home/SENSETIME/liyining/anaconda3/envs/colab/lib/python3.9/site-packages (from mmpose==0.16.0) (1.1.4)\n",
+ "Requirement already satisfied: numpy in /home/SENSETIME/liyining/anaconda3/envs/colab/lib/python3.9/site-packages (from mmpose==0.16.0) (1.21.1)\n",
+ "Requirement already satisfied: opencv-python in /home/SENSETIME/liyining/anaconda3/envs/colab/lib/python3.9/site-packages (from mmpose==0.16.0) (4.5.3.56)\n",
+ "Requirement already satisfied: pillow in /home/SENSETIME/liyining/anaconda3/envs/colab/lib/python3.9/site-packages (from mmpose==0.16.0) (8.3.1)\n",
+ "Requirement already satisfied: scipy in /home/SENSETIME/liyining/anaconda3/envs/colab/lib/python3.9/site-packages (from mmpose==0.16.0) (1.7.1)\n",
+ "Requirement already satisfied: torchvision in /home/SENSETIME/liyining/anaconda3/envs/colab/lib/python3.9/site-packages (from mmpose==0.16.0) (0.10.0)\n",
+ "Requirement already satisfied: xtcocotools>=1.8 in /home/SENSETIME/liyining/anaconda3/envs/colab/lib/python3.9/site-packages (from mmpose==0.16.0) (1.10)\n",
+ "Requirement already satisfied: cython>=0.27.3 in /home/SENSETIME/liyining/anaconda3/envs/colab/lib/python3.9/site-packages (from xtcocotools>=1.8->mmpose==0.16.0) (0.29.24)\n",
+ "Requirement already satisfied: setuptools>=18.0 in /home/SENSETIME/liyining/anaconda3/envs/colab/lib/python3.9/site-packages (from xtcocotools>=1.8->mmpose==0.16.0) (52.0.0.post20210125)\n",
+ "Requirement already satisfied: python-dateutil>=2.7 in /home/SENSETIME/liyining/anaconda3/envs/colab/lib/python3.9/site-packages (from matplotlib->mmpose==0.16.0) (2.8.2)\n",
+ "Requirement already satisfied: cycler>=0.10 in /home/SENSETIME/liyining/anaconda3/envs/colab/lib/python3.9/site-packages (from matplotlib->mmpose==0.16.0) (0.10.0)\n",
+ "Requirement already satisfied: kiwisolver>=1.0.1 in /home/SENSETIME/liyining/anaconda3/envs/colab/lib/python3.9/site-packages (from matplotlib->mmpose==0.16.0) (1.3.1)\n",
+ "Requirement already satisfied: pyparsing>=2.2.1 in /home/SENSETIME/liyining/anaconda3/envs/colab/lib/python3.9/site-packages (from matplotlib->mmpose==0.16.0) (2.4.7)\n",
+ "Requirement already satisfied: six in /home/SENSETIME/liyining/anaconda3/envs/colab/lib/python3.9/site-packages (from cycler>=0.10->matplotlib->mmpose==0.16.0) (1.16.0)\n",
+ "Requirement already satisfied: torch==1.9.0 in /home/SENSETIME/liyining/anaconda3/envs/colab/lib/python3.9/site-packages (from torchvision->mmpose==0.16.0) (1.9.0)\n",
+ "Requirement already satisfied: typing-extensions in /home/SENSETIME/liyining/anaconda3/envs/colab/lib/python3.9/site-packages (from torch==1.9.0->torchvision->mmpose==0.16.0) (3.10.0.0)\n",
+ "Installing collected packages: mmpose\n",
+ " Running setup.py develop for mmpose\n",
+ "Successfully installed mmpose-0.16.0\n"
+ ]
+ }
+ ],
+ "source": [
+ "# install pytorch\n",
+ "!pip install torch\n",
+ "\n",
+ "# install mmcv-full\n",
+ "!pip install mmcv-full\n",
+ "\n",
+ "# install mmdet for inference demo\n",
+ "!pip install mmdet\n",
+ "\n",
+ "# clone mmpose repo\n",
+ "!rm -rf mmpose\n",
+ "!git clone https://github.com/open-mmlab/mmpose.git\n",
+ "%cd mmpose\n",
+ "\n",
+ "# install mmpose dependencies\n",
+ "!pip install -r requirements.txt\n",
+ "\n",
+ "# install mmpose in develop mode\n",
+ "!pip install -e ."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "aIEhiA44zw0y",
+ "outputId": "31e36b6e-29a7-4f21-dc47-22905c6a48ca"
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "torch version: 1.9.0+cu111 True\n",
+ "torchvision version: 0.10.0+cu111\n",
+ "mmpose version: 0.18.0\n",
+ "cuda version: 11.1\n",
+ "compiler information: GCC 9.3\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Check Pytorch installation\n",
+ "import torch, torchvision\n",
+ "print('torch version:', torch.__version__, torch.cuda.is_available())\n",
+ "print('torchvision version:', torchvision.__version__)\n",
+ "\n",
+ "# Check MMPose installation\n",
+ "import mmpose\n",
+ "print('mmpose version:', mmpose.__version__)\n",
+ "\n",
+ "# Check mmcv installation\n",
+ "from mmcv.ops import get_compiling_cuda_version, get_compiler_version\n",
+ "print('cuda version:', get_compiling_cuda_version())\n",
+ "print('compiler information:', get_compiler_version())"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "KyrovOnDzw0z"
+ },
+ "source": [
+ "## Inference with an MMPose model\n",
+ "\n",
+ "MMPose provides high level APIs for model inference and training."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 421
+ },
+ "id": "AaUNCi28zw0z",
+ "outputId": "441a8335-7795-42f8-c48c-d37149ca85a8"
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Use load_from_http loader\n"
+ ]
+ },
+ {
+ "name": "stderr",
+ "output_type": "stream",
+ "text": [
+ "/home/PJLAB/liyining/anaconda3/envs/pt1.9/lib/python3.9/site-packages/mmdet/core/anchor/builder.py:16: UserWarning: ``build_anchor_generator`` would be deprecated soon, please use ``build_prior_generator`` \n",
+ " warnings.warn(\n"
+ ]
+ },
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Use load_from_http loader\n"
+ ]
+ },
+ {
+ "name": "stderr",
+ "output_type": "stream",
+ "text": [
+ "/home/PJLAB/liyining/anaconda3/envs/pt1.9/lib/python3.9/site-packages/torch/nn/functional.py:718: UserWarning: Named tensors and all their associated APIs are an experimental feature and subject to change. Please do not use them for anything important until they are released as stable. (Triggered internally at /pytorch/c10/core/TensorImpl.h:1156.)\n",
+ " return torch.max_pool2d(input, kernel_size, stride, padding, dilation, ceil_mode)\n",
+ "/home/PJLAB/liyining/anaconda3/envs/pt1.9/lib/python3.9/site-packages/mmdet/core/anchor/anchor_generator.py:324: UserWarning: ``grid_anchors`` would be deprecated soon. Please use ``grid_priors`` \n",
+ " warnings.warn('``grid_anchors`` would be deprecated soon. '\n",
+ "/home/PJLAB/liyining/anaconda3/envs/pt1.9/lib/python3.9/site-packages/mmdet/core/anchor/anchor_generator.py:360: UserWarning: ``single_level_grid_anchors`` would be deprecated soon. Please use ``single_level_grid_priors`` \n",
+ " warnings.warn(\n"
+ ]
+ },
+ {
+ "data": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUAAAADWCAIAAAAvuswXAAAgAElEQVR4ATTBWcxtW3oe5PdrxphzrrX+brenK1eVq9zEtuIYiOQGZFkEkwShKMhICIgQN3CBiAQ3thIgDogICRTETYyDwTIxoEAgNwETWZBQQTgkohFEgJvYVafqNPuc3fzNWmvOMcbXsMuI56HHX7ikngCZohAlOxGf70e2ZJVMhzAzZUIUus8geEN0n/f14kYf7jcfVGux1cKJSd2DmeZLYWU7RuuZGcSuqiPdR2dS7yizJIcNZyuRbw3i1FKJCUBZsDzS+WoA+nA3Hj9V2sntJ5udaql88YzaaKdPKLa0rm0d0VMg05Xtbkrv3h44ELAQ1u5GjQjkFioKcmLxzADSnTR0Ec9UUndnEJIQbymxJ5KBSCG2y2u+eUdffmpSdf80BIoUMv78w3NvYKLlQprH+W4oNDnqnp9+cLm5H+/PaugeVQVK7Q69bzePHm/tOC1oI+SiLVdKdajI699Af63JNl9WhruD1QAdR47Iso+wTJOxBUW++3sqLe3ianf/8vTwoq53UVCgqZqczAWYnbiiU18bK08F28aifbe/8m2rV8tc9NNPT1/97t93d383P5zfuWzvXl3zdlI/7+d62/kv//o3EfPYLAAqoxSxRrUoyJkmiLuNabeLaT1c7Szj/Nr6aahCJt4echu9mGbJynUMc0A0yi6lTGtbo3OZlTkJ4REprNU5aT2ljsnJBOSR0+WU7JpEjPNxUGqmB4UIk5CHF2jCWTiTFTkcHknsy4UK0/FuC6vEg5nDkl3dAUZRidGtkZkxKzPniJQggYrKjgKgSHgM8otnYtbzVE8PXmTSyS3dezAV6yZKEInN0wKclCwqFqMU8ZJESUZ2hhTmKYqWseVolk4iRJoZmZ4AiZTwSApRAiOImCPCibjMJJOOPnyLUqa6ZyD7Oei7fvDpduoBGAUZMrKv0U+JtwigVFXWjKARo+502oltaS0i/fG7iw06H7v3TA8i1Glu2wD88slOJzk9rH6SzEgEEbiwCvdubuCaOmlbe3b2iDrz4TCP3t1znpcQoxrX75d5LrdvTh4hLNOSQSUJV4+mz765vv7NlQ2kU9s2BiOTq8qSkcFRGaHgiOxmbGLDmDgDoCQmKAdciKQQFfZILUTg3gYTwxFBXCIyZSZQlizj6POBSJmmpBrTMvUxxrmPu4kpI0Inchdb3Vr4MOZcHk+P3p+N21L36+rRB5LuPun9aCJKNeYLyVmmq/P10935fI7g44eyfpZMzJdeSNez7Q5lnmJ7oPWYRbTjjBByJrAoYde5ZtXFRrcz+yARrgsL0bSTrNZvdRLd1i2BecF51asnzKU303EyjXKi/id/+hf+5t/+ld/+tf/xnT2eXMy0Pixk81Jfbf2//fBVeGVw3YtnH2cb53z3vYvhw7q65/HuuNtfBdrWel1qO5sN10JOKHNa3WgUP7FkcR9Uox6EoGml91MyXT+fR2w+yF3K4jdPy7r6+XM+fbYBxEFEPF3UHqOWUhZt595OTYizRNHCxG1rU5ksBitN82TDraV1r4V4oVp1O28Z6sODQoSjO8CeQRCyRFJ44tuYBQnPSIHIXrkYk6wPWYR1byatlLm9yTSaZrJ0c1Dy6MYMSMCJQgDKyPlpcRitHJFJLiLe05sBAqAW6cPDnAgQogQBRGLhzMTEIAIQ4ct+sjQPQ4JFEkDm9XsLaLSjbcdKP/gjz9ZTvn69AsW7WzO4phOreQQ4VVUkkjgFQEKUg6OHefBsytq3yFBG50JaxBzUcneYWLmtzSKJqE7FhjlGmTQzIyjgEPbhAiVwFp/nKfpQyYvri+A4nu5yF1dX+0g7HzONDk9MZh3B+51+9g0/f7RyqlmaWy3q5hKaFElBIM8sJBlpEYWUiN0cSYFISp2ElZkSnBaW4DIl5TS6ERJOECZty+Fid1OSO4yPn25Xj1Av5vMa96/GdPDrpxfW7fWHvZ0aa4kcnDUjbQQ7EJFaLp/zdENcSwa8bSrlzYd93HkEkmO5meanwrWPRrYRnX07G5Rkx8Jg03ZE0tAdadGISAuM0lsnJwRToemCoRFJ7GhbkGsOS0GpwiVJg0KmpbStW8uE0Nze++Lh4RXVpWPlh885Lsqf/lf/o6/92l/6nV/7G88WerSb7c2rR/uik3x+3n714zceqqJlZlEZm42jHWat++l0xMPtiYLaGI+fXgyH9c3ChzkBEFw8oasvzUg5fmLnl8NjXL83Xb9XX7043b0ApcgUj97bkdj93ZqGZTfpTLd327jzfFBmzpExkgqFJLFOCzFR37pQkcK9dZUSHkTsZHWRaaf9lLaF93FxWHqCCK33MWxaJJExPAYygwThwBAVZuLejZnBQcqIhANKKF1LGavYeaiq7Hi6rLG27TQuLuY2bIzIoDAXYYgRM1IoJDzoOpf9NB5GPw8qBEBA4+w+GECd4EHpXkoZEenORJ6h87Tbl7a2GEFJRGBhSydJchk9IDntpO4P1rdC7A5670vPehvtvAnSXMmDhSGEzIgAJwAmQEGF0iNF2FJYzDwVQogAp7oFF9dJt+4IFJbCiiSSfAvfRkFjWsow8wQyI5FBZAkgCgCiARXOCq6x381Bej41LS1Cyg6HR+weo3td+HRH0tyitjvrfRBxREyluEUiE5mDyUFAEqtSRoYHk0ASTGAkQxlSCOyqhSjPRw8DUYAoOfY36kl1yelaS0VNItezj3S9//joSfsnhZFvPsxYE0xgEy8pHgQhiu4I2l3P5YLlIMM65yjT9Oabo71p2QnEWfPiuTJLDrG2mmnmKAtzhVSzk8Q2q0aPznNA2FqQkQoplXYeJMzirORBCAyHpMAskyCQIomcZ9dF3aifGTRGxMVlGfey38+p2zhO2NO//qf+g1/963/5m//L33rvWq8mztevbhYtu/L5w/mvfP1FdyGAgrmyCBfWq3r5+vR5b+jnQUK60MVTKXN9+LQn5xieg3qL6ZIefXUh9vvPRnstZmP3LN7/3uu+ndaH4KzrKYLi8ePr0baXH67Wsx5i/2g5fraNB0DYmlEIU5IQkRCNWtUiWncy1SrDBjERSZBd3Mx1j/tPW47c7eZpJjPyzG1rZZrKAX2z9uBxQsCoRHqSKyVFBJIAIsqQSIQQA0zKZRZGJrzdwgdkx7LzaKSC4UHJYUgPFgJ5SBAxh4YlBDQlC1FQIDNAHj4kzaZ56u4EVOUItJGczswQQhVWjz4qSwwHkw0CiAkAeQdp7p9UTL0ftdZZZqebR/twDgNxZiIyiUkKkMjg8CCKJBCBlSMTwswx1cl6IAK/KzMpMyh5EYgXERj1NeE8FSHmbpZEoJgP7B5IU9WtmztiJBAiBCCT3prKRBr1gDH4dD8SDTbpbjz/zoPMWO+GtVzXTXVCoh/R1+E9YfxWMpigiUEBk0SUhcgzjYgVlaI7lSRwDiQFl6x1niaXKqc3vh4714xkUuwu68iupM+/t9QbF67bRuvp9V73pzf58GZ45wIcbxmnYeZSFIhgUhZ4+sjCmnNi8sPVJFJWO3Pm6WWMewcoAsxy9e60bm59FGZUZCQ8idjh5ERgropqbhtcYigNQGJeikXP5NFTKcPEPFQ5M4nA0DrxvJQ2xnQYQvV8Cne23sMRFsJ08XQZ5lipPrv40z/97//Kf/9Ln/1v/+fTG74i0fXz/f7xottn2/hvfvtu7ZaZDN5fKXHSKBjClOfztjXXS/3SD17L4cF9bS+Xu0/76Y2T1rAWKSm4fB59RI7KvBWd9DLqBe8P/P7TRx9+8tpNSXTalYcXb85nU62l8vkep89HbuyjpStRMGmteXFVThtyG0bshGyZ6bzw5fNpa2N/SM3p/rVZs1IKyCet5aLc36+aqZfZz2kPaZtFCIHSPB0gsAAgpISDicyHCLOyHIKqVSmiut73850XXmTpEemezOxGBCCSiZxSlNwHQAIJhBYCZTLgYBaHIxIgSslAJnb7EhHnU4eBGSRSLwUZ1rMojRZplBGJCIFmJcpUL3u5eCJwvr/t82GiR0+u3NItmBgEomQhsJsbwJT0bcxECMqIEGYmEmKAPRz/PwZ5BAmViesyx8D5tMJTqXgaKRIZCa3pTnOpOtPaxxieIwgohSNCdUq4uZdF5r2ao60mggxyC6p+cTOpWDsTijHK3SvDSB+Rg9IzghKhKgIgOCNk8qtnVaieHnpvESAhJUVEuLlq0cUE1TuFNGs8WoJCg1DSS5Sp1IX2z+nwtD7cn+YdFZXsfL6N9ZWM1TCiPYQHRJQkk4zB6c6Qtg4O1UVlpt1hpiTjMU3y6pN1u9uIGEAmll01OAmKotRq7t2MSAiRhgwEh1YmAI6+hTiRMglk4jLFaKBw65Fe3Z0oRYQk94fFMzwi2Zdl31obY7ghg4REhByeSDWp7xz+rT/xS//Vr/zc6f/5u88flR3RvL04LAuJf/zQfuW3XntASwn3MnNmts3ViwonYbOxHCpfnfZP6XC1a3f+yW+d/KESA0zWPQVXT8knUymlEJM83K9aSyk47IR2PbNsZxPWeY/TcU2betNxl+e7TiEkQSNHoOzqkw9KlVy35K3evjnZcCaRRepVeeeLTz3W0W/ffOLRMNZIxzC7vNrNl+XNm/tJWarYRtvq5JQWfR2cTCBwMhOzhMM9I0aZBEhmTg1mlpRgmsrUzqNvHUzEEBUPi2AVtt6BLPPkYe7mnkiISp2VmGy4dycmVnDh9HALlYIg4sjM0ZNZhdNH7C40HL1HmZHgfia3JgJoSJBbcpmCc7eXw+V8Oq8QoyfvXaV7uGdQuLCgTBLpSSEiYYkUkEdmJEDQTCJBEDODmIDMjP9PUqaVqknsZiKSEWMNlpwOU8J9QCrcGBEkHkQZSEsRUuWM0DKBsI5NKxNHBEcESHY7JsbpoSOFFcy2XKsNHF+m9wEnBMISYKIUYQhJJKJwyeWq1EmOx963SCdSY+ZAJOc0yaOnxRvfv/KObiMyJUcIslxQCtJlvqTrd663sW7nVatHJFOJQYUgOd191r07M9D5rWmvI8N6F5LoNLbu7iKiVXUSKbi82r38dFsfzplJREBSksxMBcwQZvPwCBZxcxhAIM30BAiJDBICkoJAEtBIq0qRgfDITCZhZlDUpQSlBSKSmVjIbGQQkVCEFibo1lc2vvjikz/zM7/4n/6X/7Z//Vvv3CzXajwe9hiq+LsP+Ku/8aKWiUQjRiJBCApviSQRCc5lt+dlm64JEkSIdb799OzdIpODe/dHz2dezm7LsPHOB3j9hk5vfKpT3WW9hCq/dXw47Q7X6+lka9jG48j9PIgAzQyhuT3/8vT+96RC7+/H+q3rF79znHU5tvN7X3mnyfHczrt9AZ23u7LdOTp7wzCXgmlHESRK6TgdO4WUUtJsbK5ciAKEiMxIEMIVMrgQsxYQKAliwyxIiIV59OFO4Njt5+49IgkgZEYSs/vIRAYSKSpgmpepbSOGA8kVFkEAE6cHiFS1d0OiaAFlOJaZ3MgDjlbKpBOdb7t3BpEQMZFUcMUy68hzRqlloqvnF4Rgoirzeh4sUWZ2dwiKFhvhg7Q4kVi4eyonssDBQpRkZhEBYJ6nPsyRWhnmbzELEadlInTWRHoHxFQmhGVkAhFAJhEtSyEid7hFy1FmRXh0irRInZZhusS2IeEihUNmNot+NBqMBCW7BwEqFEgqQgkKXnayu6TudH/cRosKQQGcnIxqLnOZd0Q+nW4bJgKjbT0aL1dyeDK13rY7v7xapovLN2/ufG1lKUSpkxA7hEop68n2FwmSh0+2cYw61/lQM8xa+Mhx5uzet8FS6jV2c6X00x0f71eiJKIIu3p0kZKtj8I1aJg5g9IRSAoiQkoQKByZycJSI10ycqo8YN4V7sgQ5QSZOYMRyppZkoSJwj0yOCKEkEEEiIQW7WY0yqPveu9n/+U/9wt/4Wfjd775wdPLC42PXj084Xh2wb91b//D198oOEhIIj2pZNlTpGJgrEYMPdSrx6X5rcys81QnOd2d2eb1TZzuNowCMb2RqycQFSm99VhfKkXB3MGyv8rlEp5uJz7f5ul1kLMNjuHCnBz1cV7dLEG2u8jdExsj7n7j0D5jSqzZphtermlWbR46N2t8fMX9YUSHezAzaLBMHs6MGMiRDOIiYajKqmGBTBl9aBHW1AkpWFfjlYkInIHg4DEcrkSSsESCQUpakAHvLqzEMLeIJAjSibmUIqpb3wSc8CCjKAAyAoBMBMjoDkAzU0inGdnHSARnDKHKc/oprQXBUSSNmQuEdKGn703Ru4Lo6uZSi19dX1vT492d09hfl26gJABmxiikwz3dk0C1ElLcPdMYkiDvUcu0LLvNmvdNiQYoPJmJKBDsESwMJnhkkhRJcoKEjwykYxh0zmWZ+jlGs4DXWZPTR5IDIJ0l4DkCBFRYBoGQxAwmckdYZHeGaNGET1MZQtH77iDJ7H20VVIGoQiNIEryZV+hmR120mwml1yXejquMCpL0T3qku0s49iK1rZ2G16qTBcFk087LZNOOwbY2yiXdbvb2rppJV1qu8P5c4sGWomZw2Nbx+Pn+3e+VO/e+OuP2/nOAp21Rrbn715vaXcPD1XmtMw0ZIl0ZmLiTEQEOAHOiMIZM6VHZmotDLIRnOoWdSbV0tZGRGnsEckpmu4Mc7ylIoJMT8a8nwlxOtpM9Oir3/kn/8V/95f/kz8zPv743cvF1/OLT18/vZavXO3+71fHr33zyMGOtyhiEEOEU2RaSmstu++eLGWXIW1edNpX8MjwTB7b/OobJ28oe90/1v1VGd1aO2mZTrex2y1UB5GLCk/s6AW0PdDtR56rMUmUUXc0qT7+krTonLv7F50UOWBnbg9J4Ua4fqdYEmz0Bt2xzBwrZUPvfX8xP7w6WYM7WJnJWdhHooMyA8SFdWbVcFLbbHcjYJzucHEdSbK9stEzAdEMp2hGKeZZCyNgqRKDd0woorLs9P50sgwaCckp5pHnDplQjLsQZyQRUsASmeTGWpOpeDfNNCqgUBX3ZMp0dov0ZHDmFO7EgzSlKiXZGiH05N3ZBysH3Ty5mOe6LPtt9fP6IErznq0jPdw54TqlR1p3IfWAVskMZiJCwAFKh7sfpsoytW5mkRKEQaAcGmH4NgJIVcyclVmYCW5DtYzh7RzLXuZ5Wk+2nYcIuIAlPSg9EsnCQYlIVoLAPZmTlVglI5CEQXYeWViJlVVF1tyUS50QoO0cRNhdkUdjnzZrWqE1ItQ3YMyU2WhkRgaUhQTTnuc9nY/mZzXz9BQp4L7sK4qnBk18cTO7wc372PpqV1dXZi2c+0Nst0ENFDTNtZRyPJ7D49EHBVxvP2+tbZQOV+F88uz6bO28bexi0YnSjJmDSSMCSSAiTygnB8MhwonMpCLwyEiiEp7MVFTb1pkYhYk6EmESkYTfRWBNIYKkTjXCRyOJ+OAHf+CP/zP/xn/8y/8mvfr83ctdbW9uJgiGWP5fL+1rH96neYAAiHCpKkIEbbaBEBZlz9fPL852VyozRb0oOmmZlt7y9dcftvtBSvsnS6LBOPowzSJUJomM/SVPS9k2Dy8yBVO8ftH8DFI6PNvGqIdd3HxAD2+oZHn1DbQzcTANPT9sFDFMlxueLjyBvnJSsNTRey0C6ZePpu2eH15t7WTTVCMHi/oIOClnG6PMhRTKznXqJ5/2fLgod69GUheZYmQ/O6e6jUgwkVskExLTHLTjdjSBKM9jxOXFodmpj/7k2Qc3j3br/Zu/7wd+TC7e/+aLFy9f3xFyW8+n00M3BxxEjFKn3Qfvv3/7+rO71y+Z8/7hLpMIUuZA1HBm4cAW7hGRQZLh6UkhxNGTihErgekLX3wqPLfWiWLLPhURzb4leQYIkmBnVhgNi/31zbRcnc+ven+gyJFOICa2PsQLEWvh5MFciQeSR2NlZy7rudkIVWEhMEWmMlQ5k1q3seZhX0Xk7v7I0GmugCfCIiMDSCQinYuwcrinQyu0qke6UUb65koF1YsU75aB4bzf14sbbtGOtyYllv2BODH0bMd5X6qyBTAIXqm4186up7stHN5SJp537I29ZR/bO+++i5TPP/90t5sg0clqjcOj+fbuzFCUUcoyNrdu0RE9xYqdIy3LxPvL/RjjeP9w8WR69t6TT77xZu0d4WNLJVxe7wdHRPrqVAOI3pI4WRMgJN6KrtBkBSPDmRNvBRNnRkSCM8Hgt9ICoME2VRDIh1DCM/BWJogZycqQ9NiQM2x89Yd//x//Y3/qF37xZ6fj/fs3h3h49ZXnk/Ww3v/3r9/9zU/bUmt3Cx/hISJahHUEMoO8p0xVdrh5fnF/vGPyi8cXb+7vOIu16K9chvZwvkKdOIzYs1xOwqMUGpEZPO9ofzHf3d8dnk4y+d3r9XL3+NF7cRov19sn4wSnozcoZT+527S+sjhyOFiG0CyT7J4MFro/tmVXM8V7gmSYE/NYt3HK2CAqScHJYUnCpSYxAukR+/3kcD9RSHKCnG2kE5U50si3JCfzyDQuHJKcMe9o91ROb7IfkYZ0ziCduBBff+H3/PAPf//f+Gt/Zbl80mVaHz6PM5g4wkdvCTYbAEQUmeAQSXL34KBW5qyT8pC71y2Zpkupk8aWYwsDZfP0qFUDMXo7PNrvr+vp1OmD73g6Go/RdgdOod77cqhtJW+uU3CVdUuOzlSa57Mvfunx0+96/eo37199I4cNI2QqcWaywDZQEBeBdWKVKkmekXC2kdYjkWUSYvLISQoJzL11S6PDMrPwcT0rg1nMPDwjMziYmRygQAELpZEnREKZRwchWYWFx9aYRFTcXUUJOc87Er9fNyTfPK6n1epEN5ccNequTFxPI2K1TD48Lc5tvcv1TR6Po59WEmEB22TDk8bT58/GoNPp5bNHj9a+umQ52P5qGiF3dw+jtd182R58PFgP50x0ys5YkWxllqDsW+qsl9d6eh1tdOZIUy24uNmfx+qeEjxkJDxciFFUPDwTSEJCKoTYmiFEiJg5iYRyhGcQCU+lbutKSQAyKBOggIADKYy3PIJSiTK9zEUU5y2zj9/3D/3EP/dH/6U//x/+a8t6fHqofrz9yvv7bGznl//rN9v//KJdXezLVNNtDHsrIvTSIsjOsBVSUg8yX9c+OovLpO3UuZfmhlN4B026XM5ScT41hC+HGiOZQSX62SP0cLOvOzz9ykSEF5+8gkcQ6i7Jp4cXZGPsDnk+6XLZpqtcP+bTx7BGsvCstBmWR6QaUejm+dzb1u/K1jpDxjq2c2bnGMmMDBfoGBYUF5dFJ4mI0aMuCsnxAEtKD6EIA1eBOiXZ5kLatxAFT3DxHEhXXjZqUxJUtJ2bkEYKhn/h9/6Bn/jxH/rPfvnPFdsMKHOOkWFIz9GdAP42ZEYITXNxzxwpTMTpCJ0m8jE2lD3tn6VOdPpc7l93rpjmyU7RjiM8hHn3aL54GsOCHj9+6m4kMc2SjLEOYfF0DNaZoDki0IkEBJkfP/6BH/zRT771+rNPvuaD3T06CTOXKEvpq/WTTVpJSSYQIQzMSZxugaG9mRYiyQQINQxuRvAIBhIgt1j2FZDh5t5tJDSIiJOQGRSsTEkZSAaBo6Uo0TSkqp05V+cloVmgIgLW0d26EeVyVd79An/H+5fTzc7aiHBSbO4qRKHLrrjzpx9un316l1QcLTnXB8Y5A14nbs1F89mXlmk/nXtzg0zkCEre7lsEts3HmxibwHtAplqLwDf0tWeQwzODlHwEnAmkxMleZtWlJtzGgFOmZ5EkZCQBHOzuUoU0MziRkSZZGOCCsoh1z8yIfItFskc6wEQkuXU3zhqFhKkwi9mWlMwc4ao03Vw/3N7GOX/0j/zhf/IP/7M///P/ysVmjxaf3R5flKlwv3v1tz5q/8enzRGzTjpLKSUTp+PZyRGejjDiPS2HlIoxhIgt+v5Q+zm7GxpsS+EqB/FhBNgYy37aPSNVd51k9Iej7fc3u/1yjjui3lePGIxp2ul+xy8/7tZivtCUpsq6Mzvh4Vvqd4oaBE5O5px2lMj98ymR66vRN8u3KL2JbcFJBESmMhMSBExgjmXZZ2w2sD24yGxpiGBQILJESeUievC6mx8+3dwTbMJFNNbj4FAmmWZxeA8rUtxinP297/tDP/njP/DX/+tfujuuNEuhfvvQlBfAt7VzSSkQ5rH6XKfGY38tpzeRDZlpZqqyu6LDo/LmdahG2efxs1wfYr6Y60XOXN989DCOyuy7x/NyTb4ZXR6uwLHslNUyta8+uoMI4Ewjhii/FQkCpif77/6+H3n5Yv3k6/8TA5EeG5AkVXlxIfEeYXR1PXEhG2YtM0OqBKidR2xgATGYJTxGczcrQsGSCbMhynUq87yY2bqu7kmceCuRmVIFBDjCEelEEDA0mQsB6W4DukidSjs3hiTB3ODBFV98b/mh3/v0+TuPTuPcm3dzRzIGqwww1K7p8etX9tsffjRA64hmdryPvnadSIUU+uzJdZPj/DhWa32tHHR8GL31w6HevHNhVl799u3Dy3OR0kYs01SZpKitnk7m1r2Jqg337pRCoIDrxPOySOFu27Z1SiFhAjIDBARFhlYuu5JOfW0qCoUN4yIysSSIAhQi1FeKYRkMEgSNrTNXnZyzuFupkmmeGcFIcjdIiiAa/QM/9Uf+8X/wj/38z/2Ji/N455EeOGQcd9Pip4e//dn4O591swYHKfKtoMyEQwrXWmzk6MY1y06opgg55ePn17evT+O+w8hbZjCVCACUUsr+hg9PVMowohjYtn5xedBKrz8/te5P3qXeKboUme8+u2v3IkqHRxMvA9K3e7q42N1+vNl9LvvFPJgpYJY21zrfiA+7e2XpQBIXRpqPtBbhWYi/TTiZ6wKkW09RRmYMVi0B780YFB7uXqbClecrkolvP+npVGfqzSMMwUKMDJmYCqZdMTNNtDWffeUnf+zv+eLX/upf2lab9/tF5ZNPP4lUiCVxuZDrJ2Vdjwku3+kAACAASURBVP3EynzcxuFqTrPtIZAYYxDJ/qZc3sjrF6Ofkgv8zN4zxadlQrg1t8YALzcsi7Uj6PJwlbDDxTTvZGuxHls6g8jCM5IoVSlJSFiYl2dXX/rKD3368WdvPvo7aUYga55JRCJTiBKzhPN+ES4UHtbQugUnETIzNiaODGcqQGRQZooAzBFk1uskQShVi+jpdHYHAkCQIJJIKSPTEuAwZ8pSxTk5hJAXl7sOrOeVwNaNU5My0ih82pfv/sL1D3zfo6urfaJvI4aNSJ90t4a9uH+I2i6n+fYVjie8eHn2HigW2p++P9fd7vOPbreXush8uIjdM7x4+fDqIyLplDUzdhelXuvu4urum/fHT3upNDxuLq5y9OCMlkTCSqftvK2dkgDE4PQAp0582B/Kouf13JtZz3QXJRDlW0TJWZRoEkSkOUMyOXpEQKfiOoh9nnm3n46vPMhB5JZkPB9oPTsFpaFUsOToQSIRPHowUxXIQu1kP/FTP/WHfvQf+4Wf+5mLzb/wzuUcp+LbpJV7+2vfePj1NyNiYFCpyixm5p4clBTEFI6khFDZs0yBBieadjKaxykBSkdYgJyLyFya2/5QZMdTzUHezrbsJiksBdv5VOf5vS+XN7fnh9vkMT+8GBTJi4uKzgyJKlwmevnN4bfTNCWRBGXEkLcgXjscY/jYjFkgYOGM9BZIpgwkIrPMk1aeJXtDd0cGJcxCq+6W/bqtPjwdJDLcyyGllBwWQaARDoSGByhKES2SJaZZe+8z9q3jg+//R//hH/v+v/gX/uzp9tSGF9K+RaKMXOs0OdvukoizrRBSECdomhwDZjGGAUwqSd035FaTE4GMSEoKSiS+jcE073n/SHsDXS4HUplmmRZdx7BmlORpkYFkFpZCHlmWVCnX7335yZPvffHpr99//FuStW0D5kV1uIdTmUmr9hYMrgurynbu6xZEIPKqEknTXMPcLYcbwAkQJQtFkFkX9RRlylpKOnXz6EEEnohYe2/hQc4BrzQRMsgcXGsuu4pSHL6dmrfMICWy8IhQ4qnqvC9PnuA7P3hadbfZphqHffEipz4+uXuz4v5i/+TTb9xOupyOcbptus/5EX3wPbtGvr50e1X7w6qyt+mYwOlzWx9YKEiSa1mudP+YT5+fb79J046J5GLaxehEGpFtNBQAtB1H37pUxYCPYEXdFymcETbSu4cRMqSyc4ojhKCkki5ZKxEhM80zN+RAeFKtIGeJOsk4w6XVSRjKxtfv4e62tTcT3Hd7KaUcHwYImWTDM6OounQM/oP/xD/147//D/7Cz/30I6PveP8mH15dlKii3Nuv/s7pN++7ckqURLp7a0NVoUkRPhKpSCYZZWZPYARJjWjCEhkgApCeUpiUy6LNO3n0jGUqUBTlCAdRZJYD7S8FlNPCrYNa+fxbp8sbfuer5Xznd29s3l0E2v5Kbj8a68cUlsxaptJtq6RCesaKoHCDUUQ6BZNkGmcSBCUVXLQO85729GqxnqetA4EkgJlVRVZbk+A9GZYkPCFAMSIcWpJIEZbgCC9VDstkbEkgsN3BSL/89/7RP/Cj3/Of/9K/c7o/BXGMtOaZJXIgSCFOmYAwewqwMiawwQPgTEIysXsIk3MmVwIjwkSEEus2hAuIKSmS91eoe6LdcjEVLLsaoNY7UZqFG0BU5pBCY0RaEXVa6tXz9x4//vKbj7/1+sW3KmdrXYh2y24bbT1bZZrnuXmPgcystfbew4mYIpyZyqy7XRljtLNn5rQsREESPsTcPAwwCBOR0Le5BUDJSRyRbGYIRAQn6aThwc5lQr3Ecjmvp7Fu4T29OTwIAiSQqlK1OrIUPuwrUwrZfr9cPqnyrL4+NbSj9a33cv+6w4Nj31ub90Uqba3LMpYnZXeQCdrXfP3SpIy2SXtwEYwtq5blZjk8sbH6p7/RWIJ8nqWKOBgGj5OiDJ1hR9rOHexu7J24Rllm8hy2MbMb0kmESABCd9O5crWIzMhShQiRSQwRWAvyQoHuXiZ1DDKORvNMUkB9Nl11ke2++8oiyszxu0QkIswMwKi5c/wj/8I//2Nf/fv//J/9mZvav/zsnTi9vCzYKWXSf/f1Nx8dK0+MPspuBnA6nt0iKdzch8MjWUrRRBAQ6cwcSCJ4QCDMmXAmCaHg1CKBjmAkMmI6qAivW1OZ5oOU2VGi1Kmdhm1Y1225mp5/V+kP6+mVlFq5QKs8vDydP9McYEUEoiU7AklCFt+Gtzw5kZQJykxElqq7ZR9uiTEMIgrALYDIhHsyMRCinJzdOxmxEgm35srk7lIkMjhLll7naS6wSLAE7sQetWHo9p0/8k//5A9/93/xi//e/f1D95CktpkWTgSReHgMF2GA3JKDiCklRBBOyaELEXnRIpVQ8Oj9erxrD6+rlhVZt9cjmyAQxMi0HHVi2u/2y1xK1WYDSBBsWCSSeJqYlSOzbYNGkurhyfXV5RfvPn99evOtKhIpYRYZw8wDzBARYjBxZjKzuyMZRBEuIlqhyjbcBhFBpJBAlUS4D4scWqj1/5cnOI/9NL8Lw/4+PsdzfI/fPefOzh7e2fWJDYZAneIAImraRhVqpaqIJGrV649UaouQoialbVDvVq0pCiESSUhiFSUmohjTcpZgHDCHbdY2e3jvnZ2Z38zv+F7P83yO9/vdsdX29RIENFEAdN61bVu05DKpQK0VDREQ0QGWGJ1jYgzUCjWKDCqL7WqjqWoCrRpjqLU4x8BkCM45NK0Ki97tL2eLJc6u7t09u6cyVvVjyeMutzE4CqMWXyCG7nKa9vfc7FqbaCU2yiWvLyT6sLkoLDjr2u1mckEX12O335mt3vs65wHKZCAWvQVPLgbKHgik5HEnaSrOQ87VhCkIsENVAwMAVUMw53zXz6aUpnHwbcPOl5LQ1JwQo6phNgSPiEQa9tEFJU85gazCuKpgwuRQUXhywWmGWtFMiYwYiJx9k4ponMVqikP9N/+T//iFa8//vZ/88cMoT51cqZuHHcm8cbXKr722fmcD2NisazgEqFjGXKacTWqpKoaGyIwIACZSkZSYAYGZzLElQ0WRgo1SdAoKBiDVkWNHzAYYxYqY1Cqh75o5NXMsmsvOzODo+HjIQx53LhRQPw643HfIOK7H3QPUhDFEyYqVpGRFMDQAUDMDsKomRo6AUEEZkB2ZAhGToZjGGB6bpnHKCY2kmEMPBFULgAECkT0GyKaAqAaGTAbivLqGtHIIUXWUSs5D2XE1ItFnPvFv/cDHb3/6Z/6ncZgEULMgR7VSpSIykjISmBkAky+afUehc743zdY0bdUsqGDSzNj3EIjGQVZnOl+wSq0bP17AuM1mBMAGoCY46/rgWFDZEzh4DBEQoRZUFe9cKRUJnfipSnu498St23fvroZ3XyULiiKqZoYAgIhEgEaIBoaIAICIzpGImoGZhUhgINXAyMwASVWBlBh8iIBiVnNWJqfVANBF7rpGoJZSpIhUBQMwMNCmwb39Dki1WhLMVfq+2ds/XK/PG9euL6bt5ZYIDYyZqiIzhuBFRRW8kza4k8NWSTOk2bK5v0q7aXDeEbl2RnHP7x7sNKGbt7isDOHy0aPZzFewcZNbP0ujDReJlEste1f56Mngug50eONLmu5nreAdhQDI4ILr2E8TTkMShVqliW4aq6C5oAAsZiIKAERIwOzck7efubxcPXzwTjefBb+33Z4RVvOKCJLQpooUVDU25BcsmIhwGgUntoyIJkIEDkkRUSoIVCIA0H7W+RCmaSqlAAB5UtE6yb/3n/2Na/Ojn/upnzh09amTKzqctVj2552a/dIr5/fWAFE8uZRqmaojN2t8MSilllIBEJEQQVXMlB045xTMTDE4p06yPoattIvGwNKYsRoR+cBdHzbb4iJ5zynnDHW2F/s9V23sZ3uxCbXo2cOLfCFxr/bLcH6/EEBsG1a7vJfTSgFIkiy7HtGqiACoGQCImoiiIRKSN8HKyOQgBMfoG+zOVxeIMJ/Pcs7DNJFxzQqKQPCYAahWJEAgM3jMoBIjMhDB/jXXde7hgy1hbHpaP5o0swgAMRs+/+f+0vd++/V/8nf/1+1mzNWsWAVgBiBQBVN1RKCmYMH79ihCEAJkh1AhTUkfA4cIbe/8TLXobsgcQtu5prPpHFfvld0mBR9zVgJnqrjsl1Kr7yjOHDCVIs6RWrFEJVdGNrOSiaxKaF74M3/2Ix/+5B9/5aU//b8/jVqJWMEAEUURyBgBjFTBOQAQEedciEbEpVQECpFqUSkKRqpqBmKGBOwcO69WDYoKOvZSjZCAgViAzYysCBggoIqJYNvCcr8TsjKVaVImvHXz4MknP7TZPZwv99964+wbr76ICIgQY8iJHVbnsYI1DpvOB++XvQsNGHCuw+k0TgMaCVLju+xat1mVkupsGYVz2biaJXTWLCJYIYy1ym5Vy05C4xbXyv4N5gDDGd9/KQ33wDH3XQOYg3Ps+XgRHp3m1VRCiLVkB7bZlgqVG3UYq4oIIIJaBTMXfNO2IYSLs41vqHEn4/SAnJoTLUBTKKieAdRMGFHMCAABRJXNgFw1ZGdMiCJqRsCTmcXY9H0/pqmWUmtVMzAkNWH8j378by4ofvqn//sDlicP98v20SLgwbKvap99/cHDFU8loQQ1NAXncDZ3U8pSVUQBQBUAzDkyM0AjZjUFMEMgIzRCAgFxLQIYKIOpAgJCbDw6dI6KZAACwixDnPH+UW8NI0Jel7KxOmW/72Jf0gWkNSpaE126sOlcq5p3dP3KESNtd0OqdRwTIKhBqdUEiNB3YFyZPLE1c4diPc7HpNvtVkSZXC1aa1UzUzA1RCQiMwMCE0IDJjA25xHQkI1n1gSF6lOGsCxlzWnjkA3YoMBzn/zLn/joyWf+7qfGIU9ZSFms+sBIWKuYEBqgASAiSXe1FayyUx+aNI7BB0NAKyqMBNRWECxi3R75Fin4zf1sWzduJiTUSgRca8X5bA6K7CH0ROBUVczUjAVyEVNkxQIKytyED33/933Hx3/gn//BN772uZ9UCIxKRKoKAGpKRIigasRmQqqGqqH3IXJOCiCzWczFSlYtaFL1MUBVILLQRFWVWgGBG8MSlbNn13aR0OesuYjkHYkz0FIrMjat98HVajVlQrx6/eT6jcOTK4fHV269+NW7f/Llf4ZiBsrBS8Zlx/O+2Qyp6/GJK/OOeTlvY9+K4i6PDy92712Ml0OqNddYnPMlVRQuU+GeJSsyhI7j0gBMK0rhOhWwEns3P0K/UNfg5T1dve70ApDJsykDEvWeQ+CL89I1jXNu/JYyoqKQV/ZUihkoEioYK5jDp5566vrJnS+99DtWAkVMqzV3QEHLBl0K6Ei7DAayZi1VlIgJqagxGQGCUmUwRFYFFUACH6BpvHN+TClPyRSBmEvOzIzTj/7ET2NZ//yn/seDxp6+elh2l3uelj1U5c+8fH66MxmFzIsZESABEZATNVMxIhJBMyGHzGiqAICIAGAGqgoGwXsLQgzsXRVDVWRTNUQHiNDA0ZX91fl53hXfeWj0ys39abdenda8ASKOkd1e6ZchjbI9LTpxN+s3Z9t8ZkRsprFxHJ1KtaKgbCSqkKuqAhOSs9Czi0HSRAG7LvZN93C9rhvLg6gpqLFzqqZmqgWRiNgUwAzVELGCIYP3ohJjC9RbbE12YuZyFctYMwARIwjrBz75l7/nI9d+8ef+l2FXRAnmQgJlIFNVAHRQBwUl59ERYMB2FghtHKwU8dErFMcOfM4JoZBi9Q3HmXVz9B0NO4RNPX8IOpgiOk+gFfcWSxE0BXaIBIioqlUqk7rgEaimqtUQmdrwwic/+fHv+sHf+8PXv/7ZnwTwBoKIZgbfhI+BmQEAgoihATOSY4Nqhj7A4d4sFxiHosUQoZQiBmYGaCFGUZFSySEFcBWNJbb90cnVWnGzmfI4TbsLEFB4TB8jBO+dGaqYoTQz/8zTt9///ju3br/vxa+++cXf/xUy2m1HZJ8zeJRZ33Lwt47aJ67NHFrbeiIy4Gr14eXwtTfWG9mZSBLhiGkqaGxKbJBzcZG5Qw7mHLngSi054WJfXQD05gILC2YaT2VzN6L67mA8vh2HFWze47wrxDbru3GcUkq1VhUyUCRwgaWqqCAiOTIk5Hrl5MrHP/KDr771x47mr999NY+X7Lnrw7CqNiGAKYOZWhYBJGN4DIsBIxAgKCqJmoEZAiCAEtvBwZ6ZrcfBsoChmqJHRWLOf+2/+Nuri3d+4W//b/vRnrt6WDcXe5EWHRZ1n/n6owcbBTEEZwhECAD2GCoiAgACIamqAAERgD0GCIhIpmCmRAQArneqAoAGwKxNFxQVCRULBdc13W61LaqxjYbZBSyjjqtK4JGgaZ1f6PK4r7k+eDWnlRBT2VQwFFHnyEwUOXiHBgSETEVqrYUf8wigauJc0FqdJ0RYzOePLi/rZIGCmRRTREJDEXWI9hihgoAqIgAiEDITkklFBOVg4SiUbeVMpaoWICA1ASSK/MInf+R7PnLy2X/4qd2QhLQ/2UPNlw+mlDKxhyplMATHjggRnC4PmqbhnHQccs3K5EopzYxzNVVzUQHr4ckiNJDd1PaeM959t4znakjzfiY14eHeiVhuIi0X7Xbn1us1AJgqsYWWvXc51TGZibgufuDP/cD3fOcP/O4fvf61X/6UCQMYACAiAJjB/88QVOwx7yjEMKUBwc3mbm/RTcl221JzYYKcq4CBATl0wZsqmCEBIC4bYu+EYpwtU4acBPIQbMOGglqRStFatIqSKZhTtNlB8/ydOx/64PtPrtz4+stv/d7vfo6M1qtdEUD0Wk0kx8Y/dWV268Z83sWu856dKRjZxXr809PLlaRhO27XEwKjGSGbqiXVCuAIG2CGVKd2Hjgggnb7IMolM7nEvXVNo1u7/1KN0F/5kB3coIu34f7LVUdYLCOiu7xc5VxVFZFEhR0yoxmoKRL64A0o29S28flnPoaUXnjuO//gxd9767WvQAlICBrQFICkChKwg6JZqoI9pqqMDgABAUlR1R4DAEQFwK5v+75NNeexTGPyjcNOiBgb+Ws/9lP33n3jF3/27+x7ed/JAsfdzMFywea6f/L1Bw8uVLMBgAEQIYABKDpDQkQwVQTWx0DNFAFMEZEQGUHNABHVFBmJmPCbAMUFrlDnez0FrLUwut16UMa2CaaSxqkUtGqIGFrH3ucytX2LQKv31jYxEFDBAgaA8BgKIqgaIgIou4aIainee2RTrSVXMEZEBiylhDayo5QyGqKaMQGCqhgYGZCxmYGnfraPYOM41Jx869B5k6xJfAPdtWZa57KZavYoCACIIADNvH3hkz/8vR+/8iuf+antlBRypV7GPK0VQNXAUq0TqoCRmhGCNJ1DFjI2tZJEBRWMUIC9bzgsxUdqZqwoLrBrs+542Pm8gbzT4GPOIx7u7fvoYggN+/sX25SSY4cGj7VzL1qkGrO2wYW2eebjn/jwh7/v975295Xf+Jk8qck3IaKZIeNj9C21ioqCadvFfhZ323HYFR/qcq9PCWuhmhOZiYgiAYALjMyqyojISuivH3gmd7kzcRFd1CIeh5v7NPekVipAVVR027GcXgy7VVWDKzcPn//gsy/cecH7/stfeeX3v/DrWqRWNeTogwipKjvpWr5x0h4v+1kXFrPORNnjkGyV0ksPLu+9d4GiSJ5RmsCBaShlWBcDFLTYshKYr65DmpyfCzpC1f4Y9m96rbo9xXtfgd66/Y9AoXH1its9muZN63xT6jQMowqaYRUFFIAym3XKmHICsLZr+xAvtlOlqevinWef/1d+8Ie/+vKX/9lv/FMusdYq4qpO5HwXwXkqSlog5WSAuRTJCmyGYGIIzszwWwwKkw/BX712YlrOz7eb3Y5bJqrMbAv86z/6t157+Uu/9Pd/9rjh5446ttyS9nOnrvvF1x88OsOyqQhkgMSGVJlBEQGMCAzARAHA4JsIUaqZISIjVSQyBEAkA1MCM0S1alUt9nGx30+pIup83g3juNmsmdCh04qVChGrVW6Y2Uji5mxEdVaVzRfMDskITQEQzKopARCgOo9AoNVMARTNFFANwIwQjYzhMbbgyZByqqj4GLCBk27RStV0WVARPPUHhwg2rteWEnWz0Ha1XJRNEafNArEi+pg3pU7K5BRMpDTz9iN//q98z0f2PvvLn6rGPsg07jV5tr0YSl1NZUSNu80UYzCsaaqsiEiCAEnJmYKKITnzyCLkGnAMcYbQCDccWwLSzX0thZzheCmgxA7w+OCQfSS2YVdFBL/FzGLQ44N9ER2TeIYmAsawPHny5Ok7q7Ny+sbXJU1JhJFVhQAMkRgZCVUFDVVUgLwd7C2GXXr48GFsXOvDxbDLGaJTMC0Zcs0ASEQKZqCOHkMXG0eEVqogueh8QCOH6YkDiG3DUtmRQvGh7bq9e6fjS2+8Nt/fv3nzxhNPXTs5vpXK8OY3Tn/7879ZdhOb80HJxyoUHC56Oljw1aPZch5C8ES+WmFrN/n8dJ1ffXu9GSYqKugd1OW8Y+bL9W4aTVTJ1zALQugbbFrbroBDbpchZzm+Tf1+HIedk3j3DyXmvr+jm8u8fl27Dk2IifvGrXfjMBZQBq0KhFaPD7qEsE6DIjiz5fJ4N22aHp9/9jvnC/oX/8y/+srLX/7SF34Zq45JUsVhrMg0a/lwOc/DuFbIqVa1Iec8GXtEBFAEJUB5zAyRQcViDMu9WZ5kgh2QkWDmSuD9PPznP/ZTf/iHv/Z//P3P3Fjoh28c2XC57OHk+OCR2q/ePbv/huYxISEAEDokABBkNKiAqIqM8JiYAgASmCoCgjqggo4BEdWYqRYFQzBEtaoQe3QhemIfu37enZ2djuOOgLQaGjtP6KSYgrPYtAZ5vESsYmbo2cRYiX0zTbvYNWI47TYIDKbeMzHmVM0IDRUE/j9oBACIaGY+ICKXVIlIUNAYHR5eW2KV80cXDgIoEoOpCQJ5ZgfIkFO2rGbgIgNrDJy2midzDFrBDLkJH/4LP/KB5/r/83N/p2XCRq2G7X0rOUpNWlLNVTQzNUQqScxMxIgYyIzMeTKppgyPEapq1zvfOOUaWtcvSSVvzm3aUhqTq94EIBJeObkCqLVCKYYAZgYAiMCutm1DxDnlEBoArUhHN5984skX3njt7fW9r4EMCp6ZmLGNgRxVtVQkZ2FiB77UIqCeDRBzys5zoG7IJRf0XL2DnDUnIQAkMELVivCYIhIzmVZTj4Fi0zikcZSWNv2iVRFGBhYAQPSqNuRMvj08Pun7JvqZyphH98orXxs2A4IjL7FrpSIYBKeLNuzv8bWTfj5roIKC9r4TD2+cbV976+E4jg4cewoE874xxfVut9mWVHPoSD1DqRTB73PdqSowsUrxnbUHbK52TfPoq8xju3jCnZ1u6iU0DU2jqFrXuTGlXASNQNWAAPRg2VeTbZmUjAn7LqDj0OD3f+8PTXn77e//7nfe+so3/vg3p6kOScZCu6nscmHQvg2Nd6Io1XZTHYoCmG9QtJA5NhCTMVVVIsRalQi7PtYq1CJ6zFMCthjYzeKP/9jf+5X/6+d+7ec/e+uAP/bUDduczqItF8uHAr/xzsOHp2ZiVhyiErGBqVYiY8cGhsQqxR5DADAQYI/sPCDXMgGhAdBjTJIrVCRAU60GHBTYNbFv2jY4Pn94KmIIaChIpsZAhQKHjgAJicoWZCwUzIBrEQIycJqk6+bFbBrOTZAQiZAjShWpBgaI5JwTEVVFQwBARDNjQgNAInau1KQVHbvQOkBR0eD8NE0K0sWmbdpqmnMqUqUqmRMR8kgM874bdmMajBlKEiPk4D/6L/2V29fwV3/pH0T0GrAU1KnGuK+a8rTLaUKwmghMQNEeA3OeAbTUEgKbiGoULUjAjlwgdgiECNAfaNv7ml0VKBsrg23XCZnx6Gi5XLg0hc0wEjhRMTVA8N4RsZkSgYHGZkFNd/WJ6zdvvu+1V988f/OrdUqCwoTe86xvPWJKdcy1iB2cHJSdbdb31dgAEQEMkIgwGbqS0SwpohCA9CYjoBETgCKAmbRt6GIjIttBZsv25Oqxij16sCLdPXVtURG9EoAhWU5JmX3w/fzAMJqTGOaI+eGD3Uuvvj7tBlAMvd241rGPacqeAJhFp+hcF3tgBIDo/WVGqoXQD7u1d41jk1piaAVpHNOQCnmHDmrJLOgaV0Nl1u2QtWgfG0CnnFxnLrjLd0bI/XwR1hcbqKCgpbIhEIqoGCgDSJVSKjMDARgoqLIxQxObpp93M/+dH/v+XKdvu/PRt978yt2X/9gjTEU2u3y5GTejmEj0TKjo1AeXsg0FRPHg6g0VGdfnUgZi3o1lmsREVcE5jo1DpAoFAxqCGTq2xcnef/offuqzv/Izv/+533lin567ut+Wbed1Npu9N8mvvnphTbPYd9tH0+5CAQERHkMEMyAmdlRrMRMgMFM0MDAkNmCrFQGAiL1DZ1pEsxEggAlgM3dqSBRKSahqpQI4BDRfw8KBksDE3vvGXKA8cd6WslVmM2VRi22gBoeHA5oDj0BiCUzNEMApAhIgAkoBJBIRAED4f5kZGDITO7bHsGox5hBiUFIAjSGoSCkpOh+dL1KnktDIDJDJqDD7POY+dlPKtaBJlaoYXdc3T377v/zMdf8bv/wLHrrqVJWqTnt7V4fhEiQhF6i2vahswD4Acj/rhmlbkmitqGagRE60KqhzxIGRzJFT5dCZaqkVvceu7bXi+aM1AOHR/sHVk3Ya2rPtGlEfExEAaAIiOyL0DtSI/R43/fH1w+s3X3j91dcevfnHmiZBZCQwbGJMMmmpgM7FePO55zen46PTl0EYsBLDtzCiIvg0gUCKxNkJ83FKl07FENUqGZKzo70FEW62u1x5sT87uXLM0DxcD8HSrVkpJkJISAAguZAjNHGhYT/nkH1YNr2/vJQ/MpXaFwAAIABJREFUevG13ercSjk8ab/tuaPQtiKVQD1zxUSea9YZNyIgpi/fnxYNXtnflzrUgqI25Txkebja5ixqwM4hgWExCLUog4Kin5FIdRLY+SKZHDddL2NB2gsAm93aDGLbijXACCKiRSWZVJGChoA05CJVTQWshugUwSh0M7557blSNkf7N88u7m1P74Xo1WAYp1IqEahAE5hIXXDzWYdIu129mIbDKx+qRS4eviJ5w8wlFzNW1Zyrc9y0zoRSScoKHpAqgY/7/i/9Gz/6m7/28+986bXbx/GwwT2U6PLt20++sR1+682HNo+ugfV7MjyqRIpoZkhGRcwAulkjUlUrkKmJgZqaVZaKBIYKhoCOOVQwMgUwAFMBCC0pIhkrVCYEgVqFyVmAvZvLJuL5+nQ2W6pOIdbdKpRtTWtkAzRfanYdQUv5vGhS7tA8cCUtaoHQvokARUQzwbeYGZgiopkBALtIhGCqUoEshiBg6H30XEolYjCEmtqmAbUxT6LKzAYa54F8MvPDZZHRRCpBIyUTMra+7fjGh77/iX37/O/8qpcu2eS9Uwqz2fE0nZuOFDIJXN6b2BgdV6GjkyupjprTbr1BNTMABCIWU0AEkBAB2eVkzKBibA1CsahNEwmploIH+wc3jxfbic4vL8mzqpkpgMw6H4IHMDAVAWsPHMdbT1+/fvu73njp1bt/+vmctkaAgIQYQiM1p5oZwLXx6pPPbi6m1YM3AFlVEFFVATT6CIhTgmpp3jgKPtli2q7BikFBcggBUZrWE+EwTIR4cDQ/PLhqaNttaoOb4wWpoWN7DAAMnENABDADRaez2ZGIrSu+/Npud35Xarlxrb1z+6DtHAKaMXB1pMGHknU38GaaVN1g84O9p9PuaxGjongqqJZL3Y7DIONmqgwszmrV2Ww/+jANm1RCf6jDdJY3xuigsoL6nvJWyoqqCqgSR2ZQLQBmhm2MQzXUbOTSuDOOnqBWMlRAIwLfRBEhrleu3KiTItSuPXz73ddqCQCGCADKgEVg2XHoYuMkRpcVk7phEGhDngpNGwREkCElNQcmoBGhHB4tS4HNsAFyyMkYyPPezcO/8e/+9D/6zH/98m9/8Wju7lxfyLBpS7ly6/is2Oc3u81mgqIw+Zps1vW11m0aU1JUYMbQ+b1+NuVsBKXmYZcMxMRMycQeQyREJK9AqNUYGVBLFecdOiPmPE1d31ZQSUoO0ZtvOC6D4hhakqIAihMPFzqNgAkIXSkipAgIaqaAgKFh9coNIIoWDH1IW5X1pMCmikQIoGqEDkQkwnJ2PedtLQOiEbnZflehIKAPXEUkg05mWvquq7XUXJRI2VzsF8dSy7Q9L7ZzaVMcezMTEe+9eeW+ffbDf3H/ZPvF3/gcCuSputiwb/rZ9XG6a2OCUHXQtFMwp5jUmqMbV7gVB+ny0Wa4nKyCQyTvilQCVDNukD1KUUJCIRUgcAZG3lygXDLuHRxcOWi3ibfDjkFM0cyQtO0DsyulShVT3bv+bOO7w5PFya0Pvfnqy+9+/Z+TstSCZAbCDsBckeoAIPprt5/fXA6r+99QAFQEQBExsyawD3EYIZWpjcRNk2EvbVeoA7EZEGEE9oDGTFJrYD25st91S0DbJmwc0/BWFzg2nn0lJLDGISoqMSgKQSBPYHC2xdfu5rw5zWm6ca197tnjGIgAS1LFArUadQ838OB0XaWZcr5+531Xjj72tS//UhQPkABT7+Dm8bwPuK1y7/yiilDP5Gdhhteu3149omzmomw2d60+LGMokxhUanK+7KYzqbU4xBCdb3ypkrKYikeXyF+/9szq/J3NxUNB5xEErUI1thDdrFsAMIDeee7D4y43gRfzq1/44m9tL9eI8BgismE1O5zRcm/uvHmHBjhNRbvlzae+Lad8du81HS5mLdcybnfjZoD1phSBtsPQBNPKFFUzISfh4ztX//pf/Z//4T/+H974/B8czf3z1w+jbubE2PLZRF94VERBpBI0KWdHtLdcEOmDi3cc7AU/T3XbuLibtnEejHXYpd1mQvVtmEMtwzA6FxBJMImqSJ41bUEgbpxviVFgIgMkMoaUktTsHDhHoiNGaeYxhlDNGk/rs93qrJQNQkGtaqAIpKbI4IM3rELQzztDLaMdXmtXZ9t8WbSgiCASMwMxgGIt3WHfzJabzUpKJXPoqJk1Shqd56AItFtP28tpEXtDSLkYGLuqbFduPh2XedoMlw93ZaN5KIaAQCLWd/NSzRq99uR3td344K0v97FPYz04OFaKFCjJQ0kKhnks0bdDyjpIKnJ0bY/84F1z9nBdk41DYseqamqeWKS4hpvW5zQkAXIeEaUI5cxMDI7J4bWTKzdOZufberm5RARVNANA7UJw3qVSRKFdLLqDJ2ax9SFfffL5Bw/eefDaV8cVmwoiIFYfzESLFBQVh08888H1+e7y/mtA6JBUrRRRsb7zzodxwjGPkU18VD6wtAUZiFUNvetc3CNiRFMVptr33vuu5pyIScnlB23kpmPnpG2CVvBe1VwqhBRSmtRZ4916cPcfZRsfDsP29q35s88cEBIrprFUo1TNwt57Z3l7dupoMabxiQ89f3zwHS9++XOczOqqdVal3rh5zTOtLu+fDyOgUk+He0+I3/jGed5bHn1wvqjri3vnFy/WXC4faZ4wNJRXls5VpHq02dwt92dTrufnY67CAjaffcd3/+uvfP3XL+6/XdUeI8IKFci6Nh4dHJk6MHj6qQ+crx7O+27WHn/hD39rezYQMgAgkjAaUOdS03gRNBNmllqvPv/8zVvfzt7O7r+zfufFo/2mbSjndHFB33jrvSJxf5/u3F4c7gUtVlR3eXjwUNub1/7qv/3fffoz/+3li3+619HVed9gjqZx0d29kN9946Lv5ujh6o2nNtsxTck563u3TWdQerB2TEN0fipj6INQVaHdekRxwcfYL2ut+/sHInWzEkNUKdFD6BdEIVdTA7ORkHa7HXqH4EpOYGYqw+40Q1ausXEeIlJSgWGtBqqTSRI1dcQKpgTkCID7ftZ2nQJo0Ukua9K8K2QgKkTExMTOrMi4iXs9z/qaMypZZSzFmJGJmQoJqoFAniw4AyBRQCJT4sB7Rze5TVZ2u9VURikpIzkzZHLOBZFd4dofPBN9rpdnbdvnVPYXB9y6JBM5rpkZA5ot9/bPLi/QTMR1s0Zh14Vm2BbHjYimade0bU7p+pUrD87fLmJm1eqY1/eu3Hi2afZ36+nho7c9YwxQ0g5v3rxxctCsEzy8fIiKKghAiNbFBgiyFh/9fHmt3btONTfteOvJZ9669+bm9MHmvMs5MTiy4rjUcllEyFQd33r6zuZiXD1405BMqxnWqirQRBdiFHXbcd0xJfbqj2hamyUkUUMwH7omhEakAKkCEVbCoFKqBYQO87lIVcyeXdtA14HnbjtM27EoMPkGfdsEEnBpk72sdrv1U0/O3/fsvgOGCpLlcoVvrRN2y3G7rtsdQyhan/y2j145/tiLf/LrvmJ028P50enZaQiuCX57eYYhMuqg27abx25etRKWbu+DzfydNA6r1XsIZVzFNAB7q+uSLlRQguO438dIaUrD1iRP3gAOZt/9ff/+S1/8hdX9NxSoiiQAICSzNjYnV/ZzBjB4+vYHTi/uH+0fNuHgi1/6re35xsxUwXuvgOSCx4wOyKyIGoCU+syHP7Z/9HSF9eWj7b3X3wquOJZcMxQ43+xm/XXCs4++7/rNa7HUtCvTlPIwULh29Ud+6L/69D/+b8ZXX22jLWNcaHKa28PFvXP97TfOJYsRNN0Be19MiQVhqEI1p7aZpWyOAJHQOQAWMzLwpFJH8weiNURXa6nTZSmmIgQlxJhSQU9Vs0OHgEhc1YpkVSNEJkoF21mMDSPWWqgCqJrkChWgchoLICAiEauhIYBBYEcOqykBAKFW1joqICEiEZjVKmhF0xRmndsL3hEbQMVhtS2GvmlNqpnTPJkIsUOPCGQCCKyU2HE/u7IbH9VpMkUCLCUBEAAgovdei2CYLa8/X/JZOn0NvZuG4XC5p42ACSK7YGMSNmRyRWxal6J2cn2vXdTtJWilccgl5RBajn6cJkeMLLU6BAQZIKubLRWdZ8jbnXfRVNs24O0nbx/uMXSLDNGyDmNWkRDdrGnEoIp2bV/NcSCrEONw49az79y9W4ZHlvZrxrHsas0BsOp5KRZIfd/PjvZkKKvzB6A05WLiax40ZwGbLWcAIU0jiCQTCAdld0E2+hgRnIpg7IOfAUcFICREIQrOQTvrzjYXXkBqAauIHkCRCwhq3YqqaARP+1ePY9vvNjldXnC+ABlnh0fP3V70fWW0cYfvvJfvnQ4htABIxkgZjE+ef/bmUx8/v//2cHZhMjSdn3appEyorUNxroql3alYkcgoaqqza88b3dfh/rSpVR1x61vw7bR6N4+PCij4JT3z/J06Xd1sTzcXj7YX73mKfNx+x3f9B1/94s/WTeaWPMapslAya72Dw8P9aRyR6NqtJ3aX27Zz8/7w9ddfHNYyTKsYFxxltR4dhrZDA8cmxAXZDyPsXT2++eTzTPHB3Xdff+UPEBfOdcRkRXbrt+P8KccP9g5vPX1jedjnWlbn49bRbHHrqT//iX/nH/zvf7O8eTpvpAfYD4oMbYhv7vTFR7re7JZ9p25m7ONiybUQPNqVdT7nKzfi9iLtNqXrmlowlQk0iyoSN02/HdaQzbEXBhDNqRI52l8cBDi7f8aUk6PgW0ektRQpRcERGdXQcalQR3AsITiONBZliuPlRtQAMScN1EMdqqIqILJBJWN2AuSQMPh+msZSBgQGIwAwEDQAIh89eWr60O2B5TnV7epSmz4sF5qy7jZ12CQRZUfORREDQCavpNDIvD2c1hfTuANFAlQRQDQzVei6LtXR98fzqy9Ivru790YIuKnwxN7Berrw3iG7OCdWIwex49W6nr2xMehv3DnwB7luwrBZT5uUBiVlP/PjRSVn1FQ1lKEqMFYLPgCBEaWpqFZmmi06fPLJ9wfeQNtXaxAxTblvWu9IsRqCqJiBI0fsiFzf2XLv2v1H921aW21NsaZUSgaPLcJYldB8v3d8fKRJh2HFLgBV03h++t756akPeHT1JDR7U0oXp+9tp8F3R7o7RyzsPRiVVEXENTMXekNWqABE6J+9c2V/8YE/eenFtL5wHtWI0AMgkho5sK2nUSSB882safpZTjCcb2V7b3/RNMuDOO8P5yU6fPfB8OjdVBQN0wc/8P7X3n7QxdZTs//U4bN3/uxbL3/p0d13ahkUimeHaojmHaRcNVugVG2o3dyDlvX9Wx/8ofPNS9PmVXQ2jo0BLPYjhXH1Ttm8JwiuOYCnX3gSdLFZPXr43umwmggonOw9875/7fSdX2nR+VYcUHGY61QLBd8S9LVU73nvcEkKolm1P109SKtkiKGZX7t6/M79e8NlaluOzcLFMt+jWsr9dzeHT9yZ7x+Y6fri4YM33zBzzNEcyZTG9dvN/Kk2ni1vfoi8m0XbcynITk36mze/71/44X/06Z+Ae5eLOfhSFkFj8K3zZ9S9er584+GDg3l05pRid9gzWKl2sX5ZNvnKzeO6Wwy7bTuf+ebAhQh5EilVhV1cX5xtL+5p3hJxqlbSWI3f/4kfyo9efuVPvugjcZwjAYKqllxzEYghANZ+0YmltNbAhOi8p4v1VAuR4pRSv+QitFjcJrmYhmHcTSKmoATYdD16JkAzGMeRWGvJZqhijyETgItNxx7beXBdJTtCS+t13j/oD+dpmtL9dy+GXQlNS8FpZQBEZEJGisplPuvW5/emYVIxUkAAJLJvwhCCIIXZlebgFuh5fvQmedrh8qPPPfvKN36PDAicbw1Ju6Vf7If1Znr3K1u19uozR2EP0m6XBwXhmnMe0VgtSaljCK2B6ZirCSgTICJRcEkYKLcNIigeX3um95v+8HDMXLVatejDrG9Z0QhVINdaLSO3IcDx0aEP86/8xT8q487UvPO1yGMhcDasmhsPojqbLUoq47gFxNhEEx63a/dfMhGGWRvifs5pWJ0LmmsP6+YRs5IjRKoJtGTywTWzqqTqEZE55nrZLvusYKMxeuNKyACISNS0BNY4C1jI94CCjou6cXUmu/cO99r54bWB9+axdsGtNnW1Gp3PXdfced8nvv7yHzngPszCtf7mjW97/eu/m7crIjAjrZnA9pbLKY+ryxVWbiMCibR9wzCu75088z2r9du71SvcmyI6JnJWZNi+Gzf30//DFpxAW3qWBaJ+32/6xz2efeZzakxVksqcCgkJSQiERIXABQWujV4RxMZr220jDd3QIIOICoIgcG3RbkUmQRkUAgECJGYCQhKKmpIaTtWZ5z3vf/im99K5y7X6rtXPw0DIKh+bugI5htKsL5zOehoYYb0xMX0Lc8dSoXhkZyYnPRvlWeksFJnyvrBWjrWm4ijW+bC9s9nuQd8VpdHAAslxfmpmY6fNQMVJ6LxA5pkssl5/e3VXyCCupd5ZW+bZyCOgiqKgWtcjM2wv1CcuS8O2rE0WIMDbuoTYd4LQzl1x061Hf+nvPvPusFs2qozrsqJASAy5WCv5ma7MGYLuBmQr0WQ/l4PBatyQFoeDLZ+OVZwtRx3Oomhm/kgYJLkZeaOdJyZDIdjuxoW8u868yQvjy8JzedWdv5mvP/HMEw+lzZaUIXlHpAGo0CYvckYQhipKg15v5HJKExQCnQ36w0LKhAs56HeFsiDi6tglo87TYI01znougoChjcKmQ+1L6z15Ai6FNYWzxmjLkLFIBCp23soAkrTl+VDycSFtt82np5tF7/ywNxx0cwCRVpskBTlOBIwx7z3n3HsvVDka7pIh5zwZT94LIYjIe4iiyPEobe6lMImk1e1FlMKn+6+/cs+xp/4ZrSAdMYGEEDck8lxndvNs20MwuX88rAXemLIgb5h3wKzkUeCKQZbnHAQxT2XGBMuz0ltLnoChVw2lCgYZEuDYzGXNJD905EgQjTlrS2M77U6p82YtSNO682w4LAjZcKRFaOdn5onCYy/7wcTH5oti0KjU1tptsPryPTNjcW1ztJvUZGGwADbqZXnWNcY1m1VT4PGX/UC9W2AE+/bv1ZpdvHDeZiWGSqVTRX+bkVEh51zkQ0tao5QqqRovwRMwjRAAMBFHBkA4Ag/OATLAZ2lGoYq9HfliBxDDSIog4GFS9i3TG0nIo9oExVVtIWCiv7Wpy6ISp+MTVWJidW2bYRmGldkrLxtrXnPu+PcD7zzj5E1ZDsHbyfGpUme9Xs8YHkSScZXWKkmodgft2sReMqUebLIIHRiGzHsGJDqrmztLq4LzysR4bfogckV5vnbuSVd464tgcm5m5qbO7vfGqo3aRHV8YkqGdadHo9Hy+lZfeNnP1IFLbqhVx8+ffXRndWmUqyLgFjRnQS2Bueb4iTNLzvK9h+biakta6wlcNupvrfV7/cwY5zwasoAcPAui+tSePMv7O+dm9lwrYMWKcQtOW4uOQX93aoauv/3O51x516c++/64oyuhk1pXQ1SBSKJoMaczPZHWWqOdJW8HnIelUxzN9NwVeZFtbSywmLy3wx1nOZ+YvKQY6SxvK+TAAlAhhoEEVvZ30lD3R/3+es9Le+0v/HZ/8ZELTz4ZTx3wPifjEKxUQaFh0N3y2sVREERy+3fazhCBQwT3fGQPchUEyJjRpXxYyh/UZNrKBytgDGOhSsZ4EJZFe6K+Z1TsjLqbxnohY0BB4LzNbVkwxlVtvFqpaTuQQaHUuIddyesO+3o0liThzsoJXZBzHkEiUw6Rc+mdY4wREQpACpTKS51zBKut1VYg45w754Ig4pwPLKs2D7BKWAmk7qxqX9Rmrh9vFItnTwrGrQamHHMcQh+EsL3czrtllNSShuKJ4kxx7oqCsj4DR1GlAaZflkQoZCjcaChCPuztgvPkyZO3bCwKcqs7giSOzx6pJ/rQJfurzb0WwBSlMbooR81KGNXHybitncFg1LGccRbOjEdWVI7d/fDYx8dCIQWvbLR3xl6Qv3Rwbe/mzP5w7qf7n5hZ2DPQNgAkl1nPVMDJJ9+4/sv8PTg9Xbnm6I3r68VTP37EFiUPIxa1it4GZ14FLAgCXYK3JjcuiaqFJyRuSQTcE2Nchhgp1M44A947zqUUsRBl4Q2CECHoAcdho9qScSTSse21izTcUFIfvvbGsHFomGWDTtbdWulunCOytWrVGcrzImCchLv0ttur8YFzj/8TYMLQCiEYAQC2JiYFyzc2t7KyVCpwKMdak2Otqd12L242i8HKaLBRuswXBnmsggTB93f6w+3tQKqkMRZP7IuS2mB3fevicactgO+/zVXG9nmzFkV1T2C9JuJCckQoikwK4RwjL6SUZZ45XWpjkQseJQQukCpR0W5nhzMuVSijWJRFZnJQrKIiAc4ZKj0UoyIDm5ki5dwJAMO5sWmSOrDgU8+cFkCZB1ZKZg4eOTw9Nn/y+MPSek4ajYlDJYSQUmxmbuikipJ8uK0z8uSIszgQxljjPQqTxBWOYb06U6nV82K4u7PsISDk5AzjgjwXoIl0JEW/n21dsTn2yvjKu/5DsfLQ+VPH6+Ozo9yDzaXiKAKmqtvLZ9AaJTlINvi9DH9f5t8e2Nsd/Kv6q2bl98rybSb+cL2gEH2GXpR+UG0etmXfDTrVfdcKglH3QjZoM2QEShBZl1urEZAHSRAlQqk4jZBzRsbqUT7sOzbZqMvNxfOKMwPSe3DOcy48AhEhACMABEIVRpEuBkJwq0dOl5JJFF5rL6QCcEbbsNZi9QPz++eHG0+Pet1o+ob9U3Tq+MlIhkU2EAocYCS59rwcDPLBsD4ZWtQMpJDCM8r7zGfOe5BhJAQfDbqCSWuMkoy8Mw4QiSEB+DKaq8jd0U4fGeDU3mtikc3MzPBonEtjS6uk0qaIJKq0TtZ2e6W2eVBNK+lYxA2PKk/e/XDyp2kk1eDq/uLrzw1v6MD/onl84qb33DZVayGUnkRJzpvgvhu+wN5F49PNub2HV5c2Vy48zVBgEPKopQfbiKWUyLkMVBpEtd3tbc4w3TPdOrB36fii2dp1XKooAsnAeGAciROnKOb1atRpZ8M8lzwlmyeBb9bHBkVXJfV+pwP5BvnywJErJ/ZfGaX17m6xs76a7Z6zpiCisjDGlrEMueCHnnf7dHJg++S93gcjk1uHA+cybRIVpYmwVhXaofAe8iAIwjDO86I2e4ke7BSDgSNn8iEwBt6URX/ULbnTSZqOPCoRhnElH3az9pp1kjE/eLu+4r6XM30silsjbXI9RMc8HyEI4cdUpOK43u9n2ujS9ECbbDQCLkkFRBQH8XhtfOHCmVZ1IgyCXrbbqE4K4C7LrNJBvRJjMsmDfZ4dmGh1u/35Ky7z/XK9t3Vi/eLJwWBt2Gk1jvAESw2U5Z576fStL77tir2HvvjpP0qtDyGjYTYxlkRhnCTRowudLV+Z23fpxtJPhru6KEsvUCKIEArQQaQlVFFXJ2am663ZRr2xurIwNn85E1G/sx1EFUS3uXCh7P1kXzx1fn359GuX0w8nl7/odcOLj24tr9RbM/3ByOqRFEIGqYiqGysnXKE548b7/C05PiCL7wzgfxE8GqQvqeq3m+hDdctiwQwC5xKi6kx/2A6QwvosOluMBqYcABVaF2AIwJF3AIikgzAVUhGCI80Y894ToMFWs1Udjoo4SghcWZTekZSC0BND7zxZR+itcVGgdDFAY8t8AM5wjuio1JZx4Qhl2qg1m0Y06rWITC/vdsKpaxvh7uLZJcHQ6IKYF1x4F6q0YYvd0bA9fckV1meUrxnnsXT5QFiQzuhKreGc1cXIO+IIiMgQrXMAHsGTp8bBOyeru6cef0wqh625yydrojlWc0FNsKAsCgY2K/rNtIJhyogcCeeLEtzM1AFmc8/EqZf9OPiTkCM89aNH4X/nOe+8be+ZPd7q0gAoXub46B1fV3/AKq1qo7F3Z3Nt0N4kjzyMWNQq+5uc2zAUQsrRUAMEThspWTLTqszt3zi7Yrs7pIRQdSaQOfBMMqadR/IQBoq88F6D597mQuo0iEZ5T6VVVxIUGwisOlGfP3CtAdvv9xoV3lBVxulnrKVR3lck+oNB67Irr05asPR9EtXlQX6x7dZ6XUKB1hg3ErKOIuKBRB8DA6kUgZs4sK+/tT3Y6QEyzhgwQjBlPgQ9DFkRRqiByXgCGG6vr+TdNgAnMKO325se+RXTOd3rZ4Xx1ltU40FSRc6VJOeBIcf/iYgUEDjzM5poVBQjZF5y3+n0904fatXGsqzfK/oHCrkc8Nlm8ovp1BEzHAsgWxm4IzOCi6DVgNzCU2clcFGbXA3pU53F3ZkrG1Py/vu/6vIgltFdr7x7X2vsK5/9aGop4gVmebMSRHGowuiBs50hr+3Zf2Tt4lNF3zIeadKCJEWjAtelG2OmDlQyVfHg01oYxWJq+tq5fQfAZK3J6VGvvP/ej7VsbR47T+y2n/nNneTPaje85LdWj38l65tKY6Lb2SozHUexlEFhqbd10RrHmTDaPudv8V1PTr3zP64+MD24Y70CAHesVT7wzT6+j+n/qqMPNUTctKN2ECpkTIUpgRdScSGQvDYZYwqJ6Tz3LnfOOGsRGTLPWMh4IJTyOKaiRAaJJTYqtsYmJjwLmVDEGUMo8xwAVBCFcUxAkotSl4NOB4lq1aR0VPTbfrhrs67O+vmwIwXzwFtzV4dJMNQqkOj0dsgxmLqurnbOn17wJremBATODPKaqozl/VVbQGvflVEa6cHiqLvhR4UupUYuGHAZSMFHwz5jSATIGBECAaIH763xU1fdWlPrJx57vFmr4uyBy2ebwdh4VQeRUmhLRFJbW7vVxMi0zhEYD8lhib6aTilyxOj0y481Pl7vXLlz6qPH4H/n7le9EgqfDwfWi7DCvAl/dOereuhwAAAgAElEQVS3gj+Q1fGk1bp0fX1x2N1B50EqDMf0YFsIHcUqjpL27tCVpQMUwIhxZDGGjSBMeFjx4ID5kCkvIgQQnFtXeDJoDdHI28L7EQ+EQqXLQoRVKrEcXUSv4prcd/Ca3qg37Pb3zzXnpvcC84BAxLQpmabtnc3qgUNXpeNi+YHM159Y7ZwdZdWoRs52u1sB5iBFVqCQDQ+o4ihIo8LqfYeu2rywtLW8wQVwKbhSHqFebZBXWKwL6lpy1easN/n64gU7GmoUCJS/01/2jZevn/2W1tY7LHLjgVTaiiqznKUkmRDMWYsA4L0jJPAAFogb44SQyCjvb8Uhb1TTVqMeaha0Lx49dOT5AxLFus+yeDOvHNzHLRRFFgHZinRjY8mFzbJVh34RTI6vz85/oHex6JrTC6etNr/826+bqYx96TMfrniIZYlZMVEJ4kripXxksdgt6Iqrn7t28aliYC0ioUenWLVv/TaUUShmslGpRByEYRCnKkwAWbVZr8RsfHrSajj544dalu8p2X1LP1n4d7vpR6s3vOi3Vk9/NRLNoNJcWXpG56WSKgwDQzbPcuest64sy533zwPAe46uA8C7nph+z9H1dz0x/Uff2X3/13f12436YJVHDeqtezCIksArEF6qICDkEIjYeuGBnM2c4d6X5AwiEoJzQsiIkBNo4JJQyTBJ6o1qtdbuDJkQjIGSwhsD5IGh8d55xxkTTJqi0FrLKGLgvOWeGHkvAMxwzeRdD1RtTnvuKZqcn50RthdI0eXzzaTX3WoP29u9nW0AYAiolBchZT1yLqq0RFINwmC4dnE4bDMWEBBnIk5T750ucyLvPQEKQiSQQBoA0YuxwzePxRunfvyEZBxnD1/aqiR7ZmOoVpnktkQGwerSZqNapmNVD4yzAGxUWgQWRJg3aunJV5w8+o3rB/3R1970z50ru/CvLjs7MXIGHm9de++13hW5cSbP0zjNBuHjL/pa8Idha6remJ5ZX+v1ttehNJ5LmU6W/Q3OXRiLNE17nWExKhGR/j8IxHiQTAVpQ4gkoyKxUiPwMGaxV6aq9ueKketWB9sd73LmCcFb57lk5PSovxQy12xUZ+YPt3udbDSqxMFEqyk4YwDgPaIiU7b727NX3344qmSnH326PzzT7kXNyTQMuztb/UG3OVYhx0adIaiG8RjEMt47ax3OTsxtLy1uLa0hYBCzZrNV+LzWnCpK6HX7wvah2KimqXF2a20NSg0ADuzw3WzvZ27p7R5nSprMD7tlrc4piAYdzUWVqwAEMiG8R0+ODJB2ZA1yRoESImAOrN7GqBFYziO8NB68+oUvunzvQb6y7v/yMzEZJhUPBbcqrFddIGQlYfUJbktQkWWIy2tOxcX+Q29cfbyXZWWB/+aNr4s9ff2Lf1EFH6Bh5Wi6liahWB8MTw4bQ+DXP+fWrLNARYgS0DmOfLO93C9WBU9b9UuNzYl8vzeyflRNZ9rdRaRARr6RBrE3U9HE2U5vspcf315ZenOWfiB67qt+d/Gpfyj7UaUVb65tki6kEJX6mOMBkZAMvDHXTdi/fnn2wMzgjrXKT7vVq+v9B2YGd6xVfrSMr/mi2XrDsvijmlMBjTaRSRUGcRpaU4LxzppA1TRlDkphhQUvvNCS66xAZ8h55xwiMsY8AoIgEDwMJw/dwN3O+uIyETJER8CYIARkgiGS98iYB8Y4krXgHAAnQCaEJ/BMRWEIWbssN+N4vKLMKNx3+PIjZbEo89yPXS5tG4GNBfKxJ76n+yaQMUeXk/fliEQUyMCDUHGq+5tFr8tYwCLBuEyqLWOGNs8YKusMcu4dAgdvLQJHHiTjh2u1fPnM0wHTuO/QYSXkwUvqvJZYsM6gYMnmejdNBs2JhvPESJHDskysV81IN8aSY/ecuP2hW/qDbHXpmYffe2z/344t/vLmVc80f/7bV9ohfvrVa5f986VSUGk7hZWhkMPd4IfP/3r4/ri1tzo1v3fx7FZnbdUbDypQyWQ53ORowkTGUTzsF/kwBwB8VokQEbNWN6any2i+goWcDN/wuld+4Ztnp+TU+JXln77lrU7bD37i8YeffKRsb/aHI1d2Oc8Z43v31a6YPnLft/+qnqQTM3uMNaPh0BpbTVOGjNADkUUtStbv9+ae8yJpooUffHWt1+NSqiixZVlmGeOsMTkz7A/KbpeFDU7cI1NjdZRq7yVXbC1e3FndQEQe2rm5CY82L4PRsGSKhYGyRbm3IYpseO7M005bQvDkh++G/V+8ddg944wddUegKd07zy1117ZVtaZak46DVMpbcqO8zLTONXoLTKGSyAR6omwbo7oMUovuhftqr3nRrdH6lvvq96KNFdGoRWHCnAzLXKiQapVgfjK+/gZ321WpDwcP/8hu78DT50jDU+O1/3T2PATRb77xl0eD1e/8/acn4gDNMEY31QgDlZ7b6C+a+ZGQV1xzw/ryE2bEkopqNpKJsclTp09srC0w4ZWsV5OJ+flpQugNesbK/u4aY2pkctL9u/ZPIckfnV2YHhRPlaONN+fJB6PnvPzfLx/7PBU1EcP22jZ5iONQSpaXZW+wG4dhWZib54Mv/cY4PPFv4egnP9B9+T3ikSPPvAKOfvLxdfGyv1wJ3gfBn9V6OxZ9DpyntbGkMmacBWP7/W3voRWkz5s9cM6OEksDm60P+oP+ABh121vWWgBgjBFD8sB4AEKNH7ya9G5vuw/IiAEgY8ic9yQDzpm3ThA6RjJQThurNfMcELlSwH4GiSumS6KCc1WNL+vR+Zpg3uymcbN+6a3RaHF9tzPXHDt24kd+ZFGysSTuDLPcjAQXDAQQKqn6nQ3tDJdBEjeAkQxrQsGo1+eMARFjwjoASd5YRG481KYvE7DbWd2IJMMrrprXRs7NJ1EjtDJwlgJR2VzvpVHZmEjzvAhEhQte5kmWYYIDluqzr1o69OV9ZamLYnj+Vau1P43mMy2luuRQPCGO7MSXf+Xn7t/zldlY9C1kUtYH6+zxO78j/zCePlydnJlbOLXeW93yRKhCkUyY4ZZgOkwUZyIbGp2XjDFEpJ9Bj6TCitx/2bUgogH521/8/Bdf/XMfufc+vbIop+ldbzx6amnz7/5heXeTxLBnHVGZz+8Z52q2NnH2xTf90he/8fn+xvH56f2BRGtNURqpmHeWnEOE0mpmoq3d7tx1d7hRfOzRLzptIyELp3WeA5GUEmRMznKbo6p4VEyE3pLj7oobX7i2cLa7tc0FcxCOj0VpA6qV5sZm3hsNVJQIGQu/m7fX+jsbAEjeE0HxXtz7uRs7m6ezQWEKg4QkvS6MxMhHaaU5RgHjXJqhtsY6C+SJyHEvkDNPhIDg+yqq8UqNMfb6PePj3c3w1LmGLyr1aTU3Hp88w1SCkYh5yGUIlSS5+Yb4zufKqansvgeyEwtsY4WVBoLKWwZLD3dHv/eO/7y6fPL7n//M3qmWzbtjoZTeFJZtZ2Yhq2vJrj56+9lTj2ddjoKnFbzs8Pzm1jNrS4tjY5eoUGo3bNVnKvWYyYRhMOpvI/nSimK4sz/hu53udmGqw/zU7k7397L0o5Vbf/HNF576/Fh6MDO9xXOLjFshwiSqKhnstheF4NmoWPz9SXjWvY999rvXnllfW/58V8DRTwLABx8cfvvVtcaHG4+cXwa3vnf/obRSE0qS4DvrO1uby1IGqMKakCWytBaYUQlhbPJyaXnJdDPvPXuW9Q4JPPAgrkUTe9H2y8w79CAEACCR916l1aCSmFK7QWbznBgSgQDmAIEhcoGcR0nI04rJrNelt5ikk7pYaSZ5Oeo5UrNH756wawurq3lvMCg6NDQkeTUIR6UryyyQknPJkTlTDkaj+bnZ/mjHmVSTHmvNeCrz0UBrH0qOgIWx5BxnjAgBeeuSW3y+uLm04OwQn3Pn4a0d36wyEXrt0DsDjg96uiJ4YyrJjQYKrXVEoRS1hAznZuHXVmb/dvJI3R5+SeG0+5O3m6vHYM847m8efHBbTjavO/UbJ/76tD/eK/9uoQhB7Sz70694TP1JOnswbU3PXjy90VnddWQxiGQy4fJdBnkYKyIshs4ZAwD0LIboUAZB8ebffvPCdly5tHfPof/jZG/1bz577+7KaTSy6FlM25hNWHsBES0wb4rxPfXW3HPvvr75qlf8yle+t/Lwfe+anL0kDpExW2irrfXWSsBIKafNescurG1cedPdw3b59BP3BZiQlF4bcMY7Y0mTJiEYI00qrR88RBj2FlcMZlcefdH64vnBbhuQvADmeRj4MFIlhs6SCKQIkfmgffE4Zm0iLJnnxIe/78Y/cclw65wuyYIkyVmQMgWuN5RC5d5HzZrgPGuPEAVjHJETEnngnCFDriSHfG56fqcsUhncuLk5vrUxD4wJNxWnUgvpuyptKa0RWaACwYWcHhfVlGlu7UhamzvHhqVTYitJf3nr6fe84w/OnfrR9/7xCxP1RHLbCkQz8pkNljr54rA+YOLm2+8+9/QP9UiwSDEmdNnz9qIdZLPzBxrjiXOSGPeUR+F4IANLw1CC5K3O5lK+ssodaRmOdrcv9LvZm8voo8lzf+F3Fk5+fv/0td3B9oUzC4oLEYaOnBJgCu7JHJ3xn39NAs869dB//eT1dq8++6aFFhz9JAA8ciF/7cHB+CfGFoe9ieleGuyPgolKJYhric3Y9sZWWmm2t3eHxWhqz57xamiYYSyAzHzv/u8OOn1EZIwBgLWGyCOTXFXD1l4wfa09CSLkgjOG4KxlKpCNitEGs9JluXUeOWfEEJFxDpwDMlmv1aamsPSdrZ0Isd6Mt1dPjTXmiJEgL6ePXFIx27ubcRh3dbsRpVk+6o36a5td2+kTQFJpOGcZaBY2Xv/KXzp16qH7Hz2pKnGj0nrujVedP/f0+YtbScAb1bQ77PUy660DQE88Gj8QQnt3ZdO5HK+/+YpO3zlL3gzzohAcAYAsNKphUg2M9Z6Mc2RAVSuTZEasNMtv3K59ODj71un3HF1/1xPT77938N8eHh4eq65jtDKcrdWnln73qfccXX/XE9OfeDz9q2NbpshWfrM99rEkagRptbJxob+73OUcnVBxbQZMG3zJOHGOeZbbAojAGAfAJAPvGEl/+Kq9GFzS2D+648bXrqwvPf7osX73maPXP6+Xb184cd6Uhjl0bmS88GaU1qd4bX7PnujK667+waMPss7i2Oy8jBxjjpPURhMhOWIO8tytnFvb7nZueskrdzfLi0983RNjwBEVMkTAn2FeFW4UgHXg5dh0Wk3KntHkDl17c6+/Uew4AssUB+MACgDmBeeI5DzwHErW2Tzjy4IQiMi4Mntn3PhIyw06I+/IoWech01eSdz2EgqFSlxy4wt6nf7W6WPeWxEmwDghkCPJOCBTUVTm7fHJ2Qz97KiYWllsmXKW06T1IVcgsB5GclQqrWUYMM4VFxzJtGphX0PIWNLoQNYclF4wVp946fbpd7znIydOPPDovd8MQadK1oWbqWLfxAu9/sZgLA/lTc9/6fnVn9DQEbmwNTno9fLNY2VvMLFvql4Zd4SVSlUbG4UpETNuaHND3JD2abvDCTVjuzv9C+3t3n/qV/68edfL//PTxz93yb7buu21Uyd+CChDpbzTHMF6RkTO+c+9JnruXgEAn7j/S/q6v/9pmf7e6bGr7vxTeNarFzYe+rTyMSTjuSShO8nE3laUCO+tt17wgAx2O/0DB2eaYwqAe6NXF9d+8OAzpAk4k0Gowog8Q9Cj4RBZqJqz4DPjUXoynIdhrMkIY0U9SuPQOABVAWHz3V7WHjBiiKB+Jgy00bJarY01O7t9xUKvS4S+GWxHiI5CrMrLr7qnps/l2gatpKpiUkGpC5NTu7Nz7viPmWzs2buPkICRlHDD0evbK88cv9CphbJWr8zPT5R5/5nzi0GAkUgvP3zpT356IrMiDgKdDZeK6esPse7WztlnfoqXHJjPbQQoTdnxhhNYIs+ESEJWqSSeXJg4T36YsWo6I0Az6F54Xfvn/r71qdckD8wM7lir/Ms5+8iS4mEFIQbGCNlbb1p+YGZwx1rl0UX49XuLtDpa/Y3d8f+ekkUAHGza3uZAylR7UOkks12gIk2FCuRw4Kz2ZalNoRHQMcc8RwQLZZjMk3B6tCFFEAb1SlK95xdfnI0uX23flyQtU5rdnZ3lC4t6uMuEhHgsL7o42BXCXnJwLJycRlEyZhkxba1nyBnTRem6bOncSmnc8172mq2NwYXHv+Y8gEdOnp7FGNOWyQADImLIGLIgCeLpMlBHrn3hYGtB5yz3GecpOiYVMC44Q2OGrsy03i6GvWF3hXuwYMExi2TeU2n++fRg/UwJgJ5ZAERrUUkNICKuZFCPHQpfci4sjxLkCXoJLmdxQETc2MFoI03nQcCe4erk2oowWEGaY1TzFKNIGBNEAZL0IJSEQMSO60QmO4OiqWBkWCUQpbXNak2Hv6OGb3j3R//l0a+ce+jRNCBvi4TMeAjbOtgq7KaeMjE/+ty71rdP6M4QIc6YFtwVu8vD7W7SqEahUGmcNmPOfCzHyPOyyPLBKKpJ9MA4Cz1mo3xnbXd1daP/lmH04eod97zpzMnPHTn8gu2t1dMnj0nEIFDGlowBgrTWeu8R2Wv/cfj8x+558LG/fPdhmIatN56ZePdh+P7YG+9QT5zYkv/mM7CWL6fTrhJUBmt6z/55CII8c1Y7LlytVu9tda697NLLD80LGRbl8MGHH3zwgeNACMjDKE3SihRVycr19UUmJNTned51HHzA0EjBhGWAzgoVWatZEM4dumzPtXtXn1lYO3WeFZYki+I4jCKtNQaVMBadXpsjRhjasix6W7q/wYGbMP25X/q/5ysbjz31g2oUGgW8MNoXuWGJjC4+/TSq5t4Dc1yC9z7Pgnq1no+6GiuzUw3jTBwGAmmojbclgmi1GtubC4vr7tL55igrf3hm8wU37osD8cNHv4O/+uoXrW4Vg+FophUyBqW22mA+sr18WE2UUsyxoSXW7mGSTHFWCtxee0MWvy/876+s37JfAMBL/ir80fac4Bxl1VFsqf1Pr3z61t3XwdFP3v0JfdEF0xPm6dcsNT7eKIqciGzGi2GhVFQ6IlHz+XYU0b49lVLr9XXtDSOiPM+DINDkfWkQABFEo+m1onwTSBAWRLFlpjXvXvq8l87tuVoqvrvV//J9924unQcGcXOS2YhJbWlwYErxWtNBKRgwQm/QIwguyVE+shvLG2Vpbn/Fr62tts//4KtECJ4BAgLSsxAd51wCijSs16e2thfQMaykl9348u2LJwYZQ+ct6ZCnzpVCSiAFzHOQ0tnBaGE03GbIATwnrxHNe2vNj7YG64sFOebAAYIHAucdEvdehSoMVDyeVPYHTGEUE6dQOF1aFsbgwQ77W7unW60jCMXh4Znm0gXrJQDUyDUdVZyvMsE5KaDQoxScJAvnZ4Knl/NWEO4Oqdlw1qAlX61UyP7FnrGXveVD93/3Uys/PlaNfJkPQmsasVroWVDJpp8qJD73phddXP5RymS3n++MdozbQV1ChgxrDKA5e9BAJ4iKqcYhzni/2y3zImpKcjbiIIlttPudnf7O8urwrXnlzxu3v+R3Vs9+/ZKDN66tLZ4+eYqTV0pqUwSBdBaNMUSAiL/+oRvefeJz37/4xIP6KAA8fwdecAvcoZ74/tgbv3y28of350vbHVGRaaXhB1YEoYiSJG5666zLVczKQTfvbr76F3/+miuuEcr/9d/81Xe/84REBGAIDAANMwxZEteqtYmh43rUiQ7un7v52pWHH+tcWJPAuHceiDEW1KrVqQkISUqFnkxegPmfAMBZ65ApFQoZIQtj6bKsHGytlt21RIZ9TF/1hrfc/dzp/3Hv3/VXN6Vze2XSJ72p7XAr213bChvN8bk6484byja6IcrNdoeJaHb/uJPoGQIyrrjXZSAT7QvmpbVJHBVliWcvmquPTCcxnnryx3jPy1+8O4DRoHdwOgziKufAEI22K6vr9Ypv1quOk/K4O0ASVW06oeue+bWNqz637/Gf9F9/vRYnf/+D+p8QBfgOifHpSbjpsn2rW1tvy6/5l8qffeC7jIfVetRvv2kj/XCCXgB4a9CjjYIk09ZB6rPtWg3374+yLFtctDYDj2C8S6sVZ3k+7AI5AhY0prw2kHWJAQL3YL2jKMLb77x1bOqKTPf1CH74yP39nS1CHrWmyhJksVWdUNU5xQNunZVcggMPXsrQaUALozbvbm07Z5/30tesrXYuPP41AE4OgBxjjLwnIsY4MkbO7Lv6kqNHXnXfd/6CnGnOXjp7yS2glwpwv/aSX/jB06e/982HB51dIFBc8YjpwqLV3vRKOwISBFYCLxnCH43VPpx2li9o9GDJOm8QBRnCmHErAJwIHAipAhYlqjLfHG+myXBlPZcUM+J5kblyI6nuHfZWbws74xe3yTsnODmsOBcTVpEiZLGHgDPhbABomODMhZZI8i+9TR9/Ib36Q5Ujj0Wtsvjy1Qdv/N33f+Nbf7H507PjVVb0u2NSMWRnulkow6WyAXF63bW3Hjtxv7DgOcu1dbhpTZ/lYSWc7XS2mnsOu6LfmCzHxw44rzubHSE5r5KzJiJWEenqWmdno9/tbAzfkiUfbTz/nt9ef/pbrfH9WzsrSxdWGMcoUEWeSY6EMs9zRBRCXvah7jePb8L/37srnwwf+8Bf/jwEH26tbi+y3DimGDrPAi4YIkiuyBEwyQVH8FKym2+7rlFPvvPNezeWM4ZA5IGc954BEGKY1lvTc7sjgHwHQxmNJ4OdNvRKYJwYIZEQUsRR3KyBBhFIQADrnS298wzRaGPJMEAmFAviUMUO7GBrPdAm4HYoKr/4ujf9n3ffcv+PP//Vr38jauetMJpuNZ7c2thd6gz7eW22WZ8Mgbly4H3BK0KtbLUV4mwqVStt+5LFEdjSZ8bmGKaKsN6q8KVsg+Wq0w+nZ5gKzfqZDbzznpcWhd3YbE9WZFqvEgNE4TVtbm7EiZ1oTRkqAsG0CZhKskJjf23h9Ys333vl48c2f/2I5iff+oHiK4zXEHmeDd78hqO33fHy7Q068/bvfGDrT955z/h7vxXt3ef9W3dm/2bSGae1L4xp7+aFcc6Bg0QPN8Za7OBc7KF97qIaDbl3OaBMqqk2WA7a4LlnEFTHqBBluYrIJaFhIIF78iC8EOicQ+DkiXkFzIrWnM8t5OuTByoTByNkzjFmNJSZBmSC8SIzZUHllqXCGw+3/9LrlhcWV576NiBDL423iEDkiYgTMA5Gm3hq8tB1N492cqv7t9w8e8vlK/UGHxX1sZlbL+7Y7Q21sXt2MLSRkkBVbduF9tYzIsmQa2djzj2Dn77k+KVfu4myAgCRu1I7Dl7KkDNpjFEBEjBkvChzxQLGVFivp7HPhqUhnI7t8lCUZR6kUX5qFy/cH5w7w40RgpeISMAIU4TEu8hhjTxH5jwqriPLFBf/8M7iH99h4VnfqTU8wIO3XHbNG973j/f9P92T5xsxz/ud8RCHxg01E6FaKeZcJbnqyHU/PfaQN+C9sZbIZ85tSxFIHiPxosiielip1idakwLEqBwFQgLYXA+c9hWJW7vZ6sqaHg77bxlV/tv0bXe9cemZL+6du3prc/XcmQsMeRgEJusCDzw657x3JITsvalb+dKLl+781NTJjzftxbV9rzdn7hMPvt9Zx/6QxR+ZHQ22bD4CAEQEAA4IAIhIRA48EQAgMuY9MMYAgIg4SgIicvAzzhNjHCwEkaxMgx6SQ+CM8YAxBMYQJTJkDDkXyAUT4CwILpwz1oIUDKREQCLPpTKm5IyAMQH9fNipOFmCzll618+9+tY7rqCi/5WvfXbz5EWOwb59yfndQbaWtwfD6f1T8YSTNh3ZvuwnlVa13xPDbLWZqLgejJBAodfD7WfaFYyiGGVtphH5vh6u93OeXpsmK+QHm2c28Xl3Pc97vrE9alWTWj0hACnCYmR3NlekMFNTU8Y7BmQoiqv1Trdv+2urb9i4+h+uPH5m67euGcGx3/3j0ReCaIIr1e8O3/Yfr7ntxteeuNjpvvdHf7T5x297ibrr8vGz/eDDdy7e8tVDEFCMwjr+5PGNE2fXHHEQVTPaGJuQB+YCgB6xWcbqoyzb2s6y0jptimGPEfNgVG3C5Q7cAIijB8uJeXKIAsh7h4iMMcMY9wKgDFpz3hLPt8anK63pBDnlhc0LTQ6JyGibjQprAC0oYIW1L3jV6y6eX9w88S/ee/LcescYMobee04MkZz1rFlP6tNCWFLp3Zcv33T12ckWqRBUIjlrCDzgUDobIx8hRQ5KZ4k0t1QyDrqANOFOuI/PLd/96NzCphgNPeMESAjee2EclNr1Btjtm1JzrckSS6rx9J4bc3N2TMzdOv+Tu256xZe+/FhzcPh//OCsPXeGFeuzpocClEBFiETGO4ZMcYwsJOAlYgE4Dl6UXkj+unIE/+pDdwWX/VA9efTSQ//ug1+69yOd0xfHU5H12rO1ZLk3CIT0Il4Pr/EhzM5OrV7sAmltjbXW6a7ON1jRBx4GcWJIM8+BRwQ2in0UqEqYFqPCOOs91UKmHbtwYdHmw8Fb8ujjk7e88LU7C9+emrhsbW1x9eIqSJRS6WGbqYhLnucFZ8IY2/7wFfnT9wJAsvWjK07+vin0+spCPijAef2OMvjQNLihzvsAQEQAwIHhs7z3jgiREQABABEiAoD3hOAAEQgRGYJzHhRnEFbDtJX1VskTMEAWI+PIAyZiriKhQuQhl7HgnAsBCJ48moEBjDlt/M2T/MctIvLeMQDJJZIrR1tSRUwYD8ns/Pzm1ja4Pnnwhba6f2hiajQslrIiBZYxSqJQVNAMsqKwYZD6QF3x2JWp3F+TsSvVvon4a9/9hytL3najXLDVIjg6EfVtZzFjYXClqi6hG66f3sTD185IEfQzlgRJnHzRaJMAACAASURBVBJnoFSATHU7A4G+NV61xGNkmRZh2uhnhe2vrL9h+zXfu+SPbxvBzzzxb5v/9GkRzRAPEWxFlfv34WsHv/o7T79fD3fzj9bhWX97pvrp06h4KqiUil1Y7F0484wnxsKmzTYnJ9n0BNdmNDlz6XhzLxP8wsLuyRPPFNrasvDWNZqBrE2i0/3dnSK3CAqlnWzVSDBLgIwbrREBHOt2M2eLysSBsigx24xSFYWKkLwj54CAOW/BM++BIUPmBQMHdNs9r71w7sz68YeZ5N6RcwRAjKH3nhPjHInAp0kyOe43s6zcmZ0qD4xDIgdjKUnlRAhETEVeSB4JAHRpBaQC61kYA5Lsd3wtpalp+Oz17reeVlqX5EUYcOcsFwSAXDBgXpKzjhnDrQPtCZCDuIRoPW3aluZB9dqF9025R8+umpUHDL9AwxKoATjOKASLnIEnToAeJIICFxM79Xy/dQdx75HBwQfhI98xAHDNY8GvvgsOP1k7e9nczFs+/oV/fG95cbkZwqjfH0+TxZ2yEmBhg+XosrgST01NXFjYsS4HLxlyTkUx2rXZ2aCyl4fjXASOIXqJJvNlm8JuLAQZ64xTUVgPyYG6sLBM1nbf3Is+tve2u39lY+Fbjeb+8+dPdzbaQnAVyLy3gyIkBCGk0ZYxfvE9fwUX74Bn7X3o39c6T3Xa6/nQOGPz/zIMPjiFfqSLAT4LABghPMs55wGIABnjQjpTIiIAEBGQA0AA/BkCQkAluJW1KG3m/WUyxiMgeQDwwBGVR8mEBBYyEQUyYVJyFSITUaK0y698wfxjNz6UfuwKAoEeGAIxJpmnxR9C7RDRGsqx59xw68LCqmMKHEcq+htPHpqqypLyLHjN0ev+ZenxR3b5Lxw5sLh54qcLG2ky3n1XZ8+jR7JnrB+aq46++NBU/Mi9Xz7i8m6qf7y2UsD87ZI3fedepytTt6pkId9eXbnYx6l9TfTeQiIhDCojJM0YAmfMyjQO44qzTlSjaFSo+vhcPyvznbObv9X++tq+q1scfuaJf/vyk194bH0PD5jTAarMDvL/krzifd1PPX9q559+m+BZxzbxN+6NLBSeCYaQd8rt5YuGUEQtV2yN12StDtoQD+sqavEARz29uriki8J7zwj27q2xZH8k9fbaxvZW2yFMTyc3XXc4aaToOGfSGMM45MYfP3Xx3LkL1dZhnQ0l6+0/vD8KU2tLUxoPrDS2KIfOoDOewDHyBDbX9sY7X7N64ez6iYeJMfwZYN47xhAA0HshmHfk0vjSq27vZX23vYmVqFmb2Fk/lmAuoQTQSjLwJXjkzEjp4ijyrgSPXACQdcYFDJMqPPV/+ed9mRnLuHBKkTccGXHu05iHAaV1kgIRKIq5lMQ5FA4qIaukTMVYrer+f7iZzpwBB00clhiuAC2T//YLvEc49H1wZIAhIX/iHYaBDwmnHuKXPyTqzgn0yFgI7PxtdNUP1PHnFadfiC/80r7K2z72xS+8rVjrVSSNhqUrTaHteLPS7poLrFGf2rN378GLK6eDMJKIo0ybwnhdFO2Vyvhhz2vOWf7/sgXf8Z7fZYHon+f5lG/5lXPO7/Q5c6ZlWibJpEwCaZAEMEGuslJkKWYXAQGRXVhw0V33LnB3FaWoqKtukKbLFaXIgkqHEAwhgfTMJFPP1NPLr3+/3095npvNfXn/2ft+AzKJC5U1OqVY9dfBd4FDnqezrXxtc3DpwnqSmPV3r2cf333Ti1+5eOzr9ZH5S5cWBhtdNNpYW3TWdZJFAWZBJKPtqU++Fu59Pzznyr+7MU302soFV2KIYfDve+lHp8X3qnKAzwEAEiAiAAjPEhYRRCKlOEQiQkQRQSZABhAAYCCFYgilNt4YnXWDxUG7C6iFEoQoElFQACIHAETUTEjaiM7S2tiBG1/66++7/RXX3vCG9d94+ePvXlrd8lFzZLRSHj95z5c+eOP171zaeKTd9UeuPTwsaehFg+0XvcHSU8snj2osj2y/6pduPDgxOnz3396/bf+24uKl0+sbrcmZ7gc3p75x+caDG9qXB69/5c/ddMPf/cPnVo8+MD2RrQe33Et3MPyCLx/S/NT01RO18xMxPHD0PF55+EqIqihdFd2uOTs22hCMHmBtZRMw1hvKmrohtVX1m6P72hvd/urpwXuKDzy27Y1X5fCsh986+tUv6KRGlNusyTH6sv/ryb94oPUX3znR638ownNet7xx7GsaRLvAMUj0Ug76gQ2aJsa1TAdFKkQoApaDPjMiau+9EsUiiKit1o1xI6Ec9KqyVNrWG7Y13sjyNEmsTSxpzLI0t3Z5uX3s6OmstSs4t73lrjywQ+VZYMkUG20GJVTRGaTI4iMyCFdy6sL5+cvvuHjm1NqTDwXyjMRCBIGQQiBtxCABgJkc2X/4heWgX3a6upaMtHZfOPlI2e0IRCIMkZmFmSfGcxbrqi5KYJYIQDE69ll9VEux9o5ix5+HWDlAtloxe1QKADWyBp9Y0IQoURFqEsJo0pHUdpoGI0k+Dtlmct2PG9Mr0cZSkB+6HX7luwP4Z3d/0AwQPdFdHxSAxJMXBBFUIYJGg2BBxT128pwDY2oq/OnvpD9/001HFr5JGHSi21uoMA4QjI9OMpBMZ2mWNyJkCNNCI6684Iplrft+ENZ6rUubdujKdj/th7HJqStuuuquge+trK0/9vgPL156pJnNhsBrK521jTY4334XNz4xd+ttr7p48m92zM5vrF8cDPudEiDU2t0NkSmtS+9AUS1J5MJnFwbD34CztyX2d/POY4lK3LAQcS4EJG0+1grlUIqCGQBQJIrihJVTSocAabp9rnnuVJe1A2YUUgqYAUUEhTBBjMylgoQ06mwq5plsLTpfCQIxIBpQKYIGkzE7RBEWlAgiAD6bmn3Ra9933VXx3a983l2n3jr/5RetLq+Vw9L7KjKj71185LHa6I5q2LbN+bkJ6boqeKy8i46qd3cv7jyWmKT50bHZkzsmEnOpuxVFFeXm+Pj4RGvX2Xc+OfU/jmyeXVOSX3bNta9/+S88+dNv/dOPHt23Y/vJ848ubhT16eblp843K3niyluS6lRZrJ07u4FzB3YhUnyWhL3b6hOtcUXGee53e6Uf1EaMUpmrwsD7JJka9ML66rniXZ3ZT4392vWjh8fl3pP4H8ff8aH2Zz/0A2tsTQRvni/+/qrXwJF7AGDXfzh3035979mi/z5f/3hOLEGEGYBRaTEmN9noaH1YRAfirdEC4gYQBRA1KiAAYRQx3kvAmgGW4ABAK2tSrbUiZRKrRIBB1LOQUGlXul5IsgQalrPEggWIkBhQiJ7BUEgSo0gZmyiDBpMLq6u6vm/hxLHNpx9hctu2z09MWKuiglhVVa8TnYd2r8CJ8d0Hbu1321W3r3Izuf2q1QvHim4PCQFU8A4AXFXt3jWlaOz8paPISekGXpgrJwpmZnZttVc2f7XT+EPrhwjCIAwILEJKCXsEJjIIwjFohUSBDB44/Pzl5Sd9NZxolIaZ+hgazpY86m3m4Pt/3jlzc4B/Nv8Des2dWQm8/EL/wG/FmQfojg/kgD5jMIh1VClX5sBEba1oVN6QXr/ywBNv/snMtJn/Kg2/Yic2JtBmEzduuV/uFFyEoYkhAAEZKUsELY26wihpgo2RGAUUQWJAR2QtUY9ru49j0ACD9gUl64JSFgGEELBk9Yn9/AsPpePjO+r2RJqg1j4GAmd9dAFxeRUePzuyugoVxRDrflgmSDaxg2pIqtEbxrUttdVW6+1B2QUXaxIqLvsIxBwFomUFBAVVgNnktstfdNsLvvr1T1QdD9EGrrQ2KFogCrMIAXkiYyGCIkinTKM1WD1GIhJNJAERABZCBSljrtI6GAFNjEhuODU2tevgvpfcvHt549w/3nB871/v67S7IiAhJKFaL123szw12gxBrRV++ziQHgfOWWDxTccW33Ic/pn9oa4/Ptn8vYnp2bmlzslts7OX7Tr02M98P3x4mr31xK9+/Zunxie3Z83HTx/fOn/8B49/A2LdtNK5or9rDX+ya3+x/ORwsDkY9vHGV+0JMRAhCiIphaYcVozYtIkySsjFaKMrkaw2aVXi1ma//aaNy764Axk2lsKxdxh4zu98iz/8XY1kv/bWcOvmm+DIPQDwmk9f+OE58D4U7xvWP5oCcYSIBInRZKOxjYhZq+FAlSyoUBtSACHNEm2U1toqAsTh0A/6vN61WaIkOB+8NsnIaMOmidLGUCKAiJqUBgYBcFVY3grjOSUaWVSUwEAIHEJQRNqqGFlECEEpAKYy+rHpa048+k+9s8d0rg8cONSantLAVgFg5HIQQa+sd1aqam7nNYN+1w+Hqp7s2X1ld+1cZ32j8oGjbzSSWp7EwIKIiQz7+WC4ESsfkar+oJJqfv7wxeXTnXe06x/LORRV5UQAkRQQCAeOSilSCph9CIpIEFVCl193y/kLD1fD9vj4zt5GZ6vbHjXby/5SltbzmoVfOfbYO/vwz7b/k5q6H7MS7n9/gOfc+F/VHR/QhBCIU8DRHXVtdK093OWaP0qTL2Xlz0/I5py/dIPs/S80Ec1uq2drLcjTv993dIujxMpoLRJjoCiSZaxBjAFTI2U5zTHPFaYhUybPqPQVIfqAQJLnqFEIJM2g0VCo4TOXxzu/CxKAPdZHZXIGEXCjE3XQraZpjPiIQWFqTBUr7UMUEaVFW+KApKPV4Aq1scWb7fSZpebx0/21Teq0XdmH4DGgbG5Rr228cLptYn4kP3DlzY/85JvdrbZOWKGxanhgl0qysLIEK2t5t+IomTITlcRaY2649nTleiAggiSCohiN0X60NWlSmt0+Nj02OwTgUC5eWqJ89M4X3bXuiwdu+ofZT13lPfd7fYXoQsihWl5ZaNanvOtxbW/KqyJl4BC8Pv8Xj7mbPPxv9vzVYfptn+Wjt99263cOf5F/Ox0UfTt58Bd/9m002v3xD7+8vLzu1zYDF80d+5PoNnpLcUN23nDnpRP3dbe6mhy+/NeuLVyR2rxmaigcI3t2FReECoFYeNCVfjkMjgVKDtY7bv/rQeNTdQS4Nq19/pW1ex/477ff9LYfnSlf+vHlKHjzS1787bkXwZF7AGD3R9cDsiHT+7ed2U9MJMaijqS91eLjcFipjS7maYnggkiSKIVeJTpNlELJk1QRRXHMyKxXV+oQq+BLUlQfobyZJCkCCiERGQGFqH30WmWhUGcXY6uhTOZRFIHlABxRkarlqUJ0AaoyKJTonffBgd+248jSmaP9S2cKYq3zSBUBElKIEaMEFgTJJkZ37rq+LIZlv6fr6ezsnv7mpWF3UFQOmKemG2PNXKHWOlaBV9aLonBV5ZkIhFnJ9u1Xnbv45OZbVq770l4E6fT6g2FFpEphxRx8LB0jxxBjVTkQEACTqYPXPP/S0iPcrSa2H1k4fh9XLaPJuVXTmBkBve19x79x9zo854UfMrf+tpHIC7fz57/u4Dnb7qNX35mLMCpIgjTreryUqVrjk65YovCLd1xd3PvM6YIYq4v/wR+8T9/8YPPI5SPDs9IYmf5PeKI7O58mycryGa2t99HqUrFLtUyMp1NjY6lFkWJrZdlLQPEWiYRYkDmmBoJmYFKASjMC/eQNfOCexCofgqCmxigYJX6IEoJNoNbEILrW8NoYYR+HGhUggQD7KOMTqjEqtbqMjtHsuHiOCrQxHKNwRBFxJT5xHKPAd79rjm3OvvjqciNKryhUEEG2gnt3x+ddUZQDOr+oF9uycFY9da6xNZhM7cb2naCcqxzEEEG4bgVRCh8m61Ib39b2GSSTc7Pb19f6KoRzi4s8tAPd9X3a+tWV+h81ETShqqrCotisFVRpu11Kxjq6sX3cD7b6QVKWyDdVC589Cf9/Wv+tdeXfXDExMXb/XY+O//50r1iFgFfd9i8Ga09sbS4NN3pDwFbeTMenvOt1u+saJibmr105953uVpkZwje+55ZWfcba+sBXGxvLDjukWEParfpabKJMv+xzjDapu0o6vW7RC4tv6KR/UouOpd3Y97bP3f7UkQ/sh1c89YbHHnzw4kv/8rcGN39gP3y6/p8fefzpz93/fWEkxPLdRfNPmqSVMVokEhFIRLHeeU0iGHbPzR64bHeWJE6cr4pe0XXinS96g34VQuXc+mJWdAcxVqaOtToqAwiglE0zRNQSBUGzEBJWlQz7tSwTUe3orTExePTeKaXTNLVGERkEo3WK2taNLSrfmj0Yu+tFe7X0Ym0aXEFEibXCEpidDyLsLI5P79BBhVhErZsj84P2RQgBCTkEF7lyDiSgQcXkvXgWEgSIgkpnODd36NSZ41tvPnvn/bcoFUOQfq8g0omFLNG9vju7uNXtDZhZBEKIMyMNqYXW7KHjx+4FwLntV5185j4ux9Naveis5bUaZa3zf/6T4bWb8Jw3vKK2517kZwF94R+qU7d6ALj7TtO617CCGH2D9JzKHkvhgTgonHrv3Xe87nUvWe7wfZ/6RnNtc/eJwUM39/ffmb/MNgDg2D2x3x9933yeNZNTjz48YGNQWAgT08rTsRFz5cGrpsdmy77/zv3/sLa2lhghCsAIgIQyNt5E8igQyyGCB5SFt1azn5xuNmba7QUWjexzHQCi81SrU5YIqZAqqGkCHwE5BkaFRBB9VABKsdWQ1Gmshj5ykkiSAZKgQqVxUEjhTObjA0/ons7nJ/tVFQMQcYyoJIaRTDVrEryUASrB02dhvbd9oKcvaz0zP1OQQgEhgmaC0+NQN7B9O89vU+Oj0VrpDs1mp3ZmaeTBJ7MT54vG2BX7999Ylfre275y4G9uFBUqJ8FFrUK7vbm8+vT2ke3dol1le+YmB1S0lAHP8eTrH9/7Fzu+f98/wv9m5GjzBe89hHnzkbt+uu3TM9Wwb2y2tdnZMTebWi5i7C73qaZ0fTY4h6E96Mn0vhdtLj2wurLqBh5/8yOvPzB/+LK5K7bNzhcMTzz96IOPfe/8xeMuVKOj9eFgGIIwq3Loh4Xv9fpVAetv6qQfyzECT9618tLPfOAEPCtfebjVPXZx3923rcMdN8N8PHfFUx/66dGvcBQRLv7NMPn9lAhEhJmN0RwCsxJmkSCK63k6MlJP80yZjEMQYW0MOxCGoqiCD+3+0BfOWoNGFEAIgZlFAJARSVgQSSSAiIBSMM5QRO4rTBEjihJgpVFrEmABiEEQEHTMjdEm3b3/+Z32pcHaYsWSWIOitLWoCRA1pkiEACE1s1O7ErBRqkpwZGQa/CaylK6qhmFQVWVVIYhCEhaODIpCZAIBUiqDmenLzp4/t3L3qZ2f2+19hfS/ICCEaLUIQOGhKp13Icty53xrJK+30pGJQ4/++Osmz6654dpLF368tTSqbS1WPWMspSPtvaeO/8lDADD3cPPXP7h96wdnFREPAdA9fVtcvhWu+6BJFDiONk1WFJ9SMj5qOOQ33bzjDXe/LFE5Y2rE29HtX/rN3z9cDXv/sfrBbP/9D892zsKJz8i53XP/bVotPHj/oLKIHpQemZkfqdmyv2iULYclgmp3e84JgjbGsutpbfJaliQarYpRqrLk6Gv1/MIvLzc/sfd5N9yxdO57ValXlzsoEDS2JtLRMYEYtTZaQWKVK6ogvvLRRa+sZicokSCiuKb3CZQataEii4ZISCISF0SDCCtLfLHbzMfsbGsg4iWEIBSZEJBCsBaUQkBhhKVF6rrJyjSnG+fmWpVWACDGomFMDKdGjY7I1ChOT2F9NCR1FSKfPIfHTqh+ZYmCAAilx14Xtv9lTaBGVCPM2t3MlH3RbecJVN6av0788PTihedN0COL+fo7Lo3/5ZGw/+TyGx82P84nf5ht/OfhxpEuANzyf82NPpRRok6+emvXX00qEyvPRAaYczBkBTyUVQl2MnDs+0urm7Ot1mWDjQc314ehRPyN//PuvfP75lqzE6MjU/O7Wq3tTtSl1Ut//92///JXvzQoBj644L1zrMkwMwFuvb2T/X7dICXcXHjj0/CcK9a/OX38M9+75a/hOQeXP79r8VuPn/mO8DBG6f3qIPmYAdAAKCKECoRjZABAEgnGaBWFGYMCAkQQQIFoIgoIsyJFQMJACpGQkZkBEUgBoVUKI4dnAWsQJcIkNeQyhgCMClU0rDUqLUTAwt77GCMRaUw0CWl98IpbNlcvdJYvRkZSHEVYxMUAhCTAECXK6Ozc7PaDGqB0XZs1m42ZQWcJIrsYXREjchRWQIoVk7AIIggASFAm1TlOT+++cP7C4utOjH9qFh1ro0UiAHiPSgkSAingyMyKlAggB9swO3bfePSR76QjzT0H9l08/0B/o6UQhJ3SOquPlSVY7g5f0N3x+NSB8w/tmG+sPLxabLaxApTaA+/v3/Q7qjOmbZIODIYqprkOPuw+NPOLr35R2tiW19MIQrpVb7W+/zufOoAL+1+cwsNvhSP3XPx+XLkPEtC/vSe/f3GBXBq5igpm9lytqdxcPWpUvd9lEKuiYwYgIQ0smOU1Mtr5YtRoZqycJwU2kYV/dWnsLw5cdcX1C8d/GBz2+wWQQDIyNzuCPIBgPPTyLNOaOHoEYVAucpA4gtwrq61+2S+Dc6IiJ7pFflCETUIhYQLRKInJbdLv9M0wmahbiMGX7b7nAEAgRAgACogVsUa2Ono1PnRWYW+kWaH3xAGVWCN1A82EcyO5RaNJEysFFdiB904IVBSFSgMLH7+brvyciChQsrmFqxv6X70yHNytssT3fJO0NjR/uDs2bzsfPLK0dM9v3beQ4OBkcvbvhmX1c9dcuXtmfPnV859vfmXnn6FmD54X7u5e/sW0TjUBzRGAY6RQFcHoZgawFZopcGK7Cxdivu0GEx576vFL1RDwNW+849DBa/bM7Tlw2Z7J0cnEJlUIQeGFlY0fP/LkPZ++R6CbWFsMPTIgQvTcfkcv+3jdkMo4r6Zv3Tj8trH2o5dvfoPQrKSXXbjs9RPD0+NP/f7M7MGHT3/PD7scpP/OYfYHGYOAILOIgEICEAYBRIFAQIgACixgJVFQEEDpRCliDkTAACysFCCBIs3ChJLXTZKiSaxNNCpxVYgBhL3GzHlxJSOLwgTBAgpzAJDUGudCDFEEuVIsjgzMzR3sdDb6qyuAVtgVMYIIACAii0R2FKkxve3AwVs4DHuDNW2zNG0Nuosk5FggihALAoHGKJHAMytUhJGQySRo4+zM/gtnz1587TO7/nZHYCGlEBCRFEURQNSZrWmlY4zGWGbBgFGXY9OXb648EQimp6/dXHtqc43rNgHxPsQoalAUFquRkTGr7ZF8Nd98xjD6bjXoFcHJP72ruPaPa9bxQKSMUqIMnViFv/q6G9PZsbkdB7LGDtBIZOr9x5685+8aSZYyjmz96sbYny7fBwjgBTuN+vtqnl0o+oMKhgcO36Gwu7JyvF4bA0fOsa8K7wVFaW1Eit2796b1mo8u+FIENtfWWXxZVYu/vDj/xRuuuOLwuWM/6G0Oet0iCuRT28cb6cVzT0EUQQORtEKQgJa0Tq3Ji7Jk0CFWRVGIKOQIAGgaivu+csIcfUAAUKJAXEQCsWNjWiWCfrixGV1EBEQEJAQWRAFUAkhkm80QAa3RGAbrmxACEhESAAiUSZLVckhMGKlhM0tAygAShYhYqcghGApn3857P2kQBQyfPoUqo9uO0IF91dSoLlmP5GHPWO2a/t57t/UAoHns0C9/+yXcP0fnPz9upm++an5q7zVv+S/v+0j3Nb/41eqzR1ePtzcefmnv4Oe3FdWwXq+HshytZYUvtVgvbse4XLmbR2fiY8fU9x6ZmBg/rN0Dxx5fHRaM//KXbx9JmrOTszv37Jmbnhsba6U2c5VzRAsr69/7wbdXl05WAQrvEtIDX7Gvzr9udfzTo86zX03Ahfa179x5+q9m53bm2UiSJE/P/vwL4UFrk6WNzncf+uvgh+hl+K5h+sc5giAyEgKAImUSBQqCRA5ACtJMK4WoEFVQlkQkT1SSJKiUDwGgVGQIlAhorRAsCGoDuVECEZVomyhCAAne5bXcqtQ7RwAxciDvIwtQcMEAI+WDoS9coaIOEYw1SWb7g3KwWSVYI9IqIBmxVjezRqKTysUspYWlwtaTuanJYeF6JVvT6HbWRWJgZpEk1cIRkDgyKQCFRAZ1RBAVqUKcnd59afHC2Vc8deireypXiZBSpBSAIZCIiIqsVqRTLaQkUqyKwNnUxO7N9omy4JnZK4ad5ZWlzUYzN4hFr+j23TC4xKjJiboGU+f+wfXHk5y43+91Yhn9g+90z/+juucKnO6LdxE2y/jzt+y//IqRqfkDI1O7TL0+2jnaXVz/9h/89Orr0ye/2a+xXD3zG0+v/0EESLUV4TrZD10/P4i2GA76ZW/bnmsEur21c5fNTU6MjA2L+NSp8+123zvWWmeJOnhoV61Z6w5cWboEqovLG8MiclBLbzm9/YvPP3LN1VtrC+uLG+dPn4mxnNx7xUSrsXr2DGPlHThXxRhESCcWUQETCEZB4Rh8EGEQVmQjKinXg+cQYuQoEhURCIQYRcQ2m2RSLdjbWImeiRgAiAwiklYAzCEqEDsyjoCsU4FquLkB0SttFEiIEiKTwjTLEUBr3RwZQa01kiCDUblOlVJpZk6/4emJPxvT2mXEvbW2NnF6vBgxLtfAosQCjb/4L25dg6UjMPvw4535h9anpayg91hTZ3Wi617+iln/zA/UfQBw+2Ljqcvyj8/88GU/qq32Qq+nSIMC7HraNxmFxUo4eBm3plAYf/jo+Ncev3ZjZeGRHy+VVYLves/PGouJTkbyyVa90WpNNRqtwOAldAIvLC6sb5zrFYMgXLOpB47eP/ySZ675zp72Vjj+cLvacO1r3zl19OOtyen57XuyWvPoxM/d4u6bnN6+0ll78vyXGyotxJ959eK+v5sBrRGiiDgfjBGWABQjeIgJgiiN1uqxywAAIABJREFUSqHSSkgQwQf27LIksdbys0KFAAjyLCCIAbQyLJ5ZobDRRiurIFptkSGxSaophMjPilGwCkQlc4icEqe2WbogIEASg7AIkqSUuSIIkI/Bu14IwhEyWxttNEmZLEnPXAyZmZ6baSmtVja3OEDNJggIoAQdKXDei0DJA1AYIRau0qAdiR+6yDQ7efmFi2fPv+rp/V/Z4X0QJhHUWqFiAdZaIxLqQIqEBRgARaS2bXrv5uaJytmpbXvaa8fXluLImJXgfYW9dnAR01Rv29ZQCgXSA/3jAaqgahAG6wU88fqLN346cxc3CzcQT4FtPkZve80VtelDrdSN6mgov/9bl8JDD+6+svX4NweDqpxRyeWT7znd/qNBcF441ylDfOiyPd9uKmIcVr1tl11bVmtuc3W0Pppmic7N4oW1zfUOM4rw/PzY/r3zWkunOxiUokJ5YWmz160i88rblvb9/e3PO3K5Gi4sXmr/9LETvaLcd/iW2e2Z62x5QYWKOZZlZU0OkvQGHc/OuRID+BiFJXKMrkzS5jCwFJvVwHHwCAyCSquyKp1zEqNp1BkVRim7G+BZJAIAogKiNEtj9BKicExGxpGIMUVyw611jKyMAUStUyQS4STLSCkQwf+FIYrW1mSptjg+PsOoF15/dP7TB3zwyKXGIrBsrSzUoBxvBDR+JKmHPa//xK9csfU/Ln9k9zc+u3TrYKOjTC3tnWzp6j0f+rWrZpv622/XK4/eu613+2Lj5FWv/XDy7+8+pYGJJeQJchBmKYZ45rxZWfS75mjb9rBtCustAJ90OuX6pu1UHl/+5ht2KpprpPWRmtZZGaSKRCZhQKqNnNtc7pWdkVpTFBnBKBFQfvzCp6//3u5eG578ab9YK5cPvanx4O8F7E1NzzWbE6uH3n6XfTjLR70Ka/4HmaFO2X/iZ85e892dSAlz0NqUpQMSkeh8iYTGWO8doCBC5EiEiU2i59L3RUQrTQCBAUEUoSIEQgAEIEL0EENVpTa1Kq3YsQ8oiIK5sUlSi8JF0RNkrTQJZsYCCkQAASKMAijkvY8SRmp1Qip96cERWBAsyn4VBxC0IBrKVjYbY7WdWY20hcAydMNEJValSukMdL2eN/JantSRgiBVPlRVWXk3CG7YKze7w6mZPQtnT534+ceu/sf9QydInKSkdGRWMcbSV0qDgBUWEY4cIUZrW3NT+1fXjxWOJiZ3b64/2lnPExsZvERd9jAyp7mZ2pai8iQSRJu87quBsAKJzlenXvnMtZ8YT4+fGQy9lvjWm/bv+9mD9aK7uhwuLWYb37x/ai5NRTYWuA5aKTUUv2f0XY+vfiRIdBIV2GZCdnzXRw6Pp1Ft9bemd15VFKudzolWbTaFhkLyyN4zoVKalLV5ojRBfzAsS+eGvdWNQXCsKSy/bWn3V19w9aHdOS+ur5VPHF3YGpRXHnnB/r01M9yQEPI8IyXDYYGoPWK/V5Ay3kUvugrgQ4y+Yh+7paxuDacatj8YOF8AQ79Xsq/KsvCu9GXZHB1lMsH5WPZcVTGLUkqElTbG2ugdAQTvdaOFzzI1lGHRaRMAkAKl0yQzNhFgREqSRCnFzGXR90WplFbGalKk0LPvv3er9YeTDIooNhtYDv1Gp19VYHXQ1MjHmu/8zc9ePvjc8rff9OVdZwMELhxrbubJi2/e85rbdimUrYUHdz30Xlg6Eg6HZ679r+98aNvzv2J15ApDBaqIMKUBQlSV2ujiZI3SCVebtFOtmIxwPbdax7Qm+Kp/+8LooYU0RTxV12l9ZKsoO4OiP5Sz6+21yhUc6nmSN+rkIwPYTJ9++dK+/7lj0POrRwc8rJaueFt6/+8F72ySjk/OdK9/9+Gtr2zbPp+ONNvyxCAupqSP3rW4/1uzwAAgihQzla7vQxSRNEkZsSiGSikRAWSASALWJilZeBZi6SrRURGBAMcoAlprEGSWIIFjCN5bkwIQc2COSquasVpnIcYgVYLKcYzMWZoyCwdPggopMYpQS2StsWlHFVFk56QiUIlq+hDL2GdyIQaF6blLeV6f1omQijGy50qJYcHSV8ySGTuS1UeyJhif2gayVkSpSQ3q4dCfXFvbNr5tYeHUiVc8eddDt2YmqeWNNMu0Jsdl5fywHHiuSu+q0kcJfdcvS+e8bbV2rG8841zSmji0sfaT9qrFEFmVWZ6XfQ5V1FaNTJLNBEVFxFQ4oOqX1bYcb5htHf+5Bz/y+Pwmtd73sfveU2/UdqvFM/3144VBbZWdPFBfdb3Vk37Nu4rYIIjwS6b/00Orv5cAEgKjHtE8qSc+/5oXlmVc3VqpTezu9Rf7KxfHJybHW9OEltmFwISkNLFA9E6R6g+KkTTbWF9e2eihD2WEtXdc2Pv126/ct6/fvrhyaWNzo73Z3rjqxheMtxLf60SJGZE2ACiEGJmBo0LJsiRFGHisQrSEiYFzK+2zS+1r9u3KUx3Yd3rlY48d73XKctiX6L2rsta4KCPMo420t7nZ6wwEnhUSnZJS0TtiCRx1o4WEqHOS4WBryyitbKqIgLTWBlAERCmltRaRwaAXipIQGFgkKoUxsvsPLvndHAAIwGoirYoYULLx5kjMs2z71f/3e+/+6wceufbJn/vcjhOpaBerZmP0wN7mL73oQH2EFzddGfHKr3+u/zOH3cS1rj/8he9P3fYFhRIZ46BUjiJ6MgSdrj661TSJ3DAyHGsWympFLCiomEjhXb92JQgKgwgq4XFNTWvBJP2A7f7QC8QoqU6nG9Mb5ZB9yUjHX3Z2x1f2FN3e6tPD0Kk2jvxacv9HfKjyrDY1M9u57t9csfn1epqntYaZXLNZp4r8yIueft4PLicJmc28c4GrisA7jxFynSAhR8zSHCCCmDKUlRtqpayoiBIkSAig0BpQOh1G76qhDx4oMutUmRBD5SokVGSssY5DxGAYUakowuiR0fHAR9EKfAjBgSAbkxkIASUxNtdaRVBECAAiWlQkiI6bqaTU7Fb9NJrjq0mjtjPNgdFHFhDwoXJeex9j8JEhsDAoiAFAee8VoVGEJit63aw5Njm68+zZo4uvP/HSH92eZrnnGCWkibVpTSFppQgVM4sICJaF67uq58osaXU2Tg28qo1uv3T2qWoQKZJkUVDQGfZBawu5WFMKaDdkxuqm7VMv3Dm6dyJxcfjtW775/odnAWD9lH/4m71zZ/rwLEYEuuquRnEO+meCZdEEfe9KEAdy1fh7Hlv7SETwwIhsYjJi6KFbn/90PesN1vOZQ1xubC6fbaQjU5MzDiMBxBC10tYmkTk4T0Q+hpRwbWV5a7MtpHx0K29ZPPSNO7fvnHbtpZXVjXa73+v1L3/ezWPNpNfeEAEFCEBKKe8rQAHAEEJi0xjKKJjlaQwlChb9cHFl48BlM2PNlGMhDD95+Onl5U4sq1CWIfp8fEKliUV39d5tzZphVxbOb/YriOhD5Z0vSl/5UKEZlGBVzbkhD9s+YpKkkSFJElLEIpoQEAizCK4Y9mNVADOiEogEGIWr3yqz380RlSbSGjRL19SzULRmDjQmGi+4811v3fPgex+84VeX9vzt9DG2Sd6AF1x/4AXXzRiM/YpJOM/T1t/c03n9m1OL7c3y+n/Yuf+vJg06Zi8chKMfhComkuhub8yFYSP3c41OjmasRqlsCoBCwJveuJcIAcBaq1ELBIWCEcEwKU2khSFGl6maBwbkUVN/4mfOH/jWPIZ48mG3enK9d8O7mw/+waAcpGk2NTPTve7dV659DRNbHx0bmd5MTE9QP/iip25/4Ig2ylASvTjvohTOhxglOicEKAQImgBIu+hD8MG7RBvnPSJmWeqdOCgUmswSo3bO2ZQjQx3zzbJbumHTPiu32lS+9OAQgZTxkYflUEiClImtKcQyVsWg8K6wlDIoJrBGJ4oSYwCgKEuOrHXC6MQrzTp66PdDinqrqtVrswLBBy9MGHWQisF6DhKEhUEEgFEJiPZOkAgAUJmq121NTjcbExcvnlz8pZOzn9oRSAkCAWkg0PIsiKK1UVYREgoCc4yx1hppjc4uX3wirY3njfkLC0+EAnRijUVSejAMzrN2XZNpx5RyuPPysTsuH1ODzY6vole3zWz+yS0L7394FgAWTxbf/7ONms4BhCUeurO28K0KUFiEhZ1EECAiENg3/u6n1j7mJXrhQJJEbU1c2LHri1P1bnv5ssMv9uV6v302wRRYuxAVaqWVCAOiUoJCpHSMLBF8WZRVBYqYw+qbFw9+4yVz2yd7K2c7nUGn0+90O4dvfMlYQ7fXlxSqACFJMu89c7Aqda5CBETwQYBIKSUcIsqg5za7xZ4dk9pmwQ0U0MKZ8+dPnhIf2HsWSUebYCyi7N05Mz1VN8jeuSBkkFNDiVWuchniwmp7tQeDksphezTVC+dXtEkRME3TyFFpbVQSoyMwPvS9G1hjTZLavGGtIVRa6eW3n9/5md0oYJRSBD4UC0tdLnrN6R1vfuntd155OAwfO59fa7+4+3t7FvKR5m13HNq9u5kIdHqxG2F2Op84fk/25Fv9W3izixnR/q/s3v/JPSiBmUkp732SQGAMICtrMVb9vbsm04QX1tXs1NzSyX8Kg25KEZ93914i1EYphQq0ECitgMHHipCstjEKSzDGJkqJwoTU0Rdf2Ped7Rj98jNw4fGN3rX/rvHAh6sYktROz8x2rn3384t7mURlqRlbBrusKP/JC49dc+8eg4LPApWnNQu2DF4r453ToLO0BixakVKqdM57b8kgktWGkCrvi6pw4MrSJ4qjCIImFYmIQ6jEe64UR6sbCinEENHnpOv1utZmMCiGXCIFQ0kzryllVtqdtd7aYMD9rX5RVkBEQIgaEMvKN0aaVk0Nq+VyUATnvXfsjEioNbdNzWyHKCik0FYxMFRISeVdqFgrDRhrdasU9XtVDEioGAMq09tYndk2X29MX7jwzOIbTk9/Zi+DI1IKFQqgUgAQfSQkIFZkg/fWUoxxbHayWZ9ZOPHg2MRko7Xj/OknxJOQFRCrNKFwUW0Nedd0/qqrZw5vg2I4yOJWE4utXjncGD7yoL/wvtU/6ewCgPv+eH3lPCsGQbn8ztrT3x4AAQqiCAmSYAB2wq3seYi0OvhRQAkgIkgMSYI8Ov7hPSNFe/3QdS8d9Je31s/kaRZdDD46J4gUQxRhBBOFEYEASGlFEJlZMLhq4+2re7/2gpG6Wb90GkBFlk63c+VNP6ugv3jhtNaJRlOv5wCglCGmyAEAvHcxCGpCRE2aY+h1B5vdYtvUiM1SYZbIG2trCydO+KIMPhCpdLSJJhGQep7aJDOaYvCEiKS1ZqMgeo9KdQfVsAwsWlxvfnL06RPnCLWxCRGFGJI0JSJElgjT083r99UJpTHS7JXVMNqidAh09NXP7Pr8ocQYiV4YxFdPnVrqdYpX3brnT1+8Af+fh9/6P6cXG1e9emT7hBt4Uz1+JuzYe2Dnztm8/vQn+aFfGd4NgFGlcft/n5v/8+39YZeIhAUROKIi8lEoHS3K7mSOjVyv+cm5bdOXTv+k1y0VId74r/elaYJEAJAoxT5qrSKhAUWAghhBNOiI3ErzUgIhH33J+X3f2ZlrvXk+3tv4SDH1PACo/+0r6usP9V/z1f7EEQB4yfmPztHFXu1UBecSahz/Py4e+PYcRKx4CBIsaYVaAJMkEc9Ks6Yksbl3rpYm1iZVGVNtrU5TZbwLgpSaNDFWwGwM1oJrZ7ZljTEqE47iPAt3XRGj06gJdeUrYVYamEOvNyglAFaNrF5PLGrVLQoA4MIurS8VhUeyRek5Os9cucBAZSEcSoxaKUKN7CKmkpqpZqsJETOd5kmGiJogtfUYmHRQKqlCCSp4EefEVUEp5aOUATqbGzNzO7PG2NryqYv/8sTkZ3drQCEM7BEACEEAAREw06byDISoQyq6Nt6qZdMXTj/SmNyWj2xbOPkTCKQ0Vgjoq8taY8/bPX3TtlozKSbiGR9psNU7uU6PHu3jhfbOjTBahZO3x8kDtelPxGAVi0xepmd2J499u4fPYmQOgMgcDVoBBpArJ997dPMPYghBOCCIkEepoU5Hm79zxXhn5dKV176sGK6tLp1LM0U6MkfmRDyLRw4cGEAxUrCGEE300VWRQVMMy29b2v+1W0dytbZ8LkYAxK325vW3vHwwuHRm4RkWMkjaECJqZVEBs8TIhAq9YYwApMlWroQoRcBURcQYRVeVD65qry3GKoIAKW1G6oQEgct+n8RHZgBQiNqkSByiYxYAyRstzwptFnw/46rbGRIS2RQRldbGGgSNJIJmbn6q1TAKXZ4QKjCkATVHf/L1p676m92NRkYIvgwxyv3Hzq9tVt966+ThiQL+X0tHYPbhdhi9f+O6JLfX1X/awnUAWKi/apZOpt+/sfeHb1TMF1fLT33981+q/XHrU9PDXjXs9YmDK8uIAMIuWtUYVcWW+FIrCMn89OzE+sUniqFPNOEtb7pCK0+og7BCBQgCHDgqRGOsQs0c6kk6qFxik1TbAPzMz5zb/Y2ZsbS57K//Xv3D8Bxz8Uf1Bz+69aovw3O2uVOvDV8453/UqS5ojcfvvHDgOzMGjWNPWgGjQgkSBCDRFiKLiFKamROV1rN68KGKlSYS9ihQy+qZSY22oKgKPngwxKm1taQZghcWRIwxuBgIWAA8AzAoFI1ZcA6EnCuTxNbyugtxs71pNWgEQGU0VS5GMlvdTWYclMPIVTlEjdr7KEiIrl92gWwYtCZHx0TyXmdVNVXZdVyqJBUBpTQqHY2y9XquVGItxRC0yphtCKHXW6lN7Gg26hA2H33Z43u/sCdqsioJ3gvEKFL50GzU2DsNenPQB9JG+cxkNp0mxKWLT2a1Ha3xvWvLD5RObcvya+Zbtx6c2WG3sFzs9wadrf4jZ2FrbbDSGa5dbO9dwSt0GtlrpRTSF36z+8rfazLCzG4LCGtnPACHGEMEBwyAlpRBIESNdGjs3z2+9pESI7JmFRSSE0yUyRP14RvmNzcXDx54aSy3lpdOo5KsZrRFdqKV7vecqyB41hrSTGe5GRYxSgguckUM5dqvrO3560N5nnS6a94xs3S7W8+/47X9/smzZ84rxUpRXrNCDEAUpQpBWxMiI6DWKICKkujK4cCGSo2PS4wogiH4ypWD9hYEKIugjUVti9JJ8H7YBw4xRg4RCIwyRORcKc+KnI9OBRJjRjj0hpsbFDUoUVppY22aCZDSBIAxuuiGiPT/kAXf8Z6eVaHo11pPed/31/bsvWfv6b1lkky6aTQhoBCaB4TrRZHiAQufIyoHBNEjRwW8xwKKFUQQUQFBI4Qu0gmQhIQ0kplMZjJl9/r7/d72PM9a60bOP/d+zverCAAGEBDE514Vxm/YmPmrHYX3RGQNOoKLG21s6t973uzPXVHB/zZ/Ley4+4E5eWweL9mFR7fD/8/dr/3cgS9ufeH/6rotf/vpD92+/0Mn/vX6uq01amjbuhwvrayYNpVJo9qsQ3lslKSS7oFDxy6e/mq5KQYiPu21lxMmQm8cGeORoKoqBXHGqgAnyb1Ti21sY4zWGgN07vkr+z49YxgsdD6z/TvwQ/6bvz+4610rr1+EH7q++eJNfMec3tXKnLXu4WfNX/alfWRsiK01ZNU2WicOqNov+qMmAkpoSoVknENBeIJRESAQS2idMTZDoKYN5By3yWDyliyYxCyigGgMkTOejFFUUBUwRIR5TA0l8t4hIoHNrA8SWVNKwZLJvI8hdbu9qmkR1BmvYoehnMg6VskiqaFhU6+ujU/9YLVjs4mp/YuLj0vR5NqPjQqzqFFV0Wgt5nle1RvG9tqWjUHjRBhiU+4+eLX3NlRLp15y8tA/H8zyrJtloglIe32DHltJgmiimqxQsDu2HoIQN6rlPbD03N78eXvwt2+rbz0WnvuMm47L48zt/MLF8xtw8kJcWt1Y36xSUg2QNVHPhEOGCtGC0AHuOFJ89W/aF/9+nwSWTof5MyEJiz5BgoCAEqAHMgQWiVgvn3nDfWt/2AJ21ZXSGDLkiQEGrv8nz5pdW13Zs+uZbbW8NHdBmFEBVEXVWqtKwoQSAdU6QlRNTimhNbFJzvHCq+cOf/S487Cx2SoIEjdNfeW1z2maxxYXFpQaIoeEiRMgGafMao1NKVmLeeGITFuJsTDcoGrM27dbU4QUAcE1dVM3I01KaPK8u7YY1paTCE9Po8u8pKQskkQBVTVGjjEqBzIT47Z1rg+x1DgiNUnUWPJ5x7lMES0qkImhaatSmVUFCZMwMAKqsVS9cTj4oy2ERAjGWos4UqsKh3bvfPWR+Wv3db53IV0191/vnnzXn319jIZ6xnzz17fB/9fdr71n8i9OXvM3xy/f+d5PvPfr1/7bjZ+5DlsGAW8pcTO3UY5WVzZqXVrly/dvzYqIbbr/Ypjdtm1t7r6yVOGEz3zdlWigqdl7BLBt04qmLMvIYOYLZSUDyswptinGFNGahRdu7rhtEpi9y88uXH1h2y8IS/djLyCy+fFnNze94ZjfeIp+e7NZXbLftZ7KanjmefNXfv1A3WJq68neIM+7EiWm1hINisFm28YYYqhApeh1mrZNKWbetPyExJIQBBxoYhE13qekBtUaVBEnGBOTsQpEEnq+yK0HUAl1ZnxoNWLjLCCIcy5GViAFcblPyJTA+zwznhDKUIEqKKbAdc1IBp1vlFMZm4iLS6uLc+OZrNud3LW5vpKgnZ2eBYFypJEBQGKQrJsGg8HmcC3PJzc3GiLYus3GRJur64eO3FSV7eriufOvOLf1fdsIrCfp9zs+61x6qDezbSCobYDFJjz20CTXE96HfAKKrP+R6/8Nfuiu82nXgaNry6vfX+pcPHdhY31UVwlILEHuLSiXYMcPba6u6TbSfcb1AAswr/rjHf/z2vnfvnvH3Z/b/Nbn1wlMUkEQUkAiUEBRAnCEGRpSODH7xntX3uW8b/Pqxo/8y+b8wr1vfsfMyiJP2Hc/bVcYl9u330xmbXN1TgWEQRhiBBFu26iCkAwChhDxCWyiiqhaY0HT+i/M7/zAQWOk5XJ2+8DYtLa6dujSF7bh9MbaMus4MakiGSPKmiClBPqfrPNogICkEcwhVnlT6sQ0kyVRJpIQy7ppNKlBU3R75Xq7PCeqYWZHohyK3EtgVGuoEBUAUFVhGa65cRkEMqthx+4sL7yqsd4TelUSjZ2sj4aqcVkOS2a1ZMkYUFBmn3lEePzlp3Z/6AARgqiCZOjOzC2VZX306CVt2/a7Ew7si5af8Z7Ru2NKLnMT3h6cXv/lJ3fuPZuu2+uv3ufg7te+fv59t7zmozuOZL/7x+98+DnfvvxjP1JrS2SspcStqculublS3Nq6XHOk2+kDBXhwUfud7vrcSes7oY341F88jsamiATRkG/qJobWGtIMJIGyWGc8msznStikRAQXn7e255MzIhENjk/j0kPN2onXZN/6IzIuy3140hufCl+enT3WwmjR3MWmTJHPPm/14Ge2h9hkhvI8I+ONGHRQVVVmc8aEgpODKVRi4KptEMGStBpFsGkaVTbGESiiAIEnZBFhNYSFz1mxjdy0wRJP9LZIiAwpF+q5wlMhtvFAMdV5bkEhRc68t9aMm6pj0bgspeRIjVoRl7i1eZtBEZNUUTaqelw3w2HInHOYM8iwboer69YMBKIlMLRNybdlqEthu1IUvVG5um12FtiGFlVW6kSItGfPNWU9Xr54Ye7Vy1N/MY0WvU29btfa/swud3D3dNf6zA4+/tmNalRCMtZyEvcj25b+9ic24Ye+cbL8zoVeOR62wzrLsNfPDCE4MOTAZpTlFx/ZvOuOlQAbW8DvJJoie+igf+Uv7/wKHP1ROPnYo/UH/vQiKSlqhpQxGBRCBERV9YSZoDPmipk3Prr+bjM1uPa2j3R37/G2CFLe9tSf2D5c/eNnbQnDOLv1yazLa2uPIpGCATApxRACojXkrdUYuW0SAGZkY8K2Yk0ggsuvPr/tvfsAkkW86prDE5PF8tKa23K5Mwvj9bU808BaN43PXEhVU0tSiRxZEhkjxCBKLUaUZtPGhrbMAJiQZZ2macpy1FTjzHhQ8Fm2si71ulpq9xx0aBRQmqZV8QRgLSGxggim0XJ/PFLRjHm8Y78OJnMAZ7xH8dZ6Y4BZkqRq3MSaxTAISEJrPKHz1hPacy977NA/XmYQiBBEjaNTj50bDcvjx6/GwdZetzNcw1cOn/VPk1+/8dprsS2/+LXbR0sLxm5kBlHgNTuvOruow6sOvOzXfqEaL73rve+5/7l3XPrRaxSQjFVVQK3KjbWFjY26ddbPTucFAFi3OKKpgR0unCt62XBtiDe96pDLO6yIQim2hoyoWGsTc2zqxAEMGEXjjBIqAqLMv3Bj7+1TAJaAxgs4f2ezecXr8m//iXW8ZWpndd2vHJr7SL/f0Y6mLefV1k2Mcy9cOfGlHaOmVTVFZq3kYxlbm8U2IoJg69UPulvKNjGOnMlYWK10TGGdTyIqmmIIKVpLrJIZrwSEZBStgRijsTYyC0ci3zbBOmI1krhX5MpBrM0QOtT1hAPXdUaEcBxGW2yGaNq2NgaREMU4wswCqEtSCQRRLWhQNRVaA4aYsa6bJiYkKstGWlHMNuuWSx1X7aisQoqENOjbouiEaNuqTDHmnU5/asLZqVFZnfmpx7f/9b5SSuvG22YnvO2jVyykn01tnI2BB6cfXiVMmcubEJfO/+Df//vOa3cmALjlPadx3T35Sju1fddg1x7bn8yKjmZP6AZvuor/9I6/Wz21yIIRdQDYBxMhfeQ9x+Du18K17/2DPzl/8nRpAXOgjkIGmBssEgWDwmqt7WikBE/d9z8eaf78qi/d1p8+SHmyhhjhi8/5KXf67Aee0w8l2O41/YnNOL7I0GEqOWR13QxHpXMFKiUWUY6JEZ0BRQQAqsqgwc2//NTM+/aqElint52TAAAgAElEQVSZ6HVnpjubw/HeYz9aDh9fX1jduXMCkQzQjpmZIvML6ytnzi+OqiZxo8DGqHdWBdpUh6YIDWyZFpdZQFTmarSemgiUEL2oDjehGan1PL1N8tykGIUBATmJKltnOamqjEs/WkfjexLX825yGRhTeJ+TtZ1u1um5umFJWo6GINFYV9U1ixhjUBFAMu9XXrOy7QO7idD7HNVplMWFklkOHToa+lt3dCYurk/8zcFf+uDsl7/98Jl2PRm0oVw3NPZx5AZTP7P6ox9v/+jd7//w1j0Tn/32dz7yDx8+85J7dn700i5mbQredVMMEsuL5+fGMev2zNaOR+N7g/zieuyCbKydNWCahvGGn9lH1gOQCoER731KCRHJIoqoauAIACoCAM4aInP+BauzH+uoQFYUPMzPfmM4OvHL9rt/okL7981uuepFedHbzY/XKAv472pC5HTxBatHPru1jShIimw0F2l9lqUQnDNAXkJLJGDRkjNkiEitpsCE4K1TVVYJoXHOkcEQVZQRxFtrTd40taoAojFKZFNiACBUADREZLgwZHweUt7UNbKAJu+KEOvpzpZBfwKSgmgbG2NIoVFoCukaA8psyVpriMAQoWpUVmVnDRpEQYsoAE2KMQDUPByNN2JtkTRijKJIMaa6gTqyuuTYjcd48VVrx/5xdxPN1u1HR6EpeoM9Ww9BwuHGRrfnvvm1L+7fd/jhh5bzghjceP2i993LJ5e+8sBiQbDjxOxl1x2d3la4TI3ARDYBnJxqlNoW5pG/f3j00IWV1OynYoPTOoZXPXf2E59e+ekdv/mJpXcaBVI1BF6ho9BB0wfNwQqCglhCK37CxhumfuvOVy7/+OvfWhCjzwXYttVtb/6Nwe3f/L0f6WijM7uuJ5xfW11gtowBJbNG2xhVidsUoiJhkicAsAIpIYYQnfXLrzq79f17GciQ5TbGdmRscd1Tn784f//FM3OAtfWFIWMd5rkx0a+uj1QJQRAJQFBBFRURgVScYrDWgCKnth4NRSMgG1sAiCKm6FTbqWlED6ENhig0gaNhCTHEGFkFKZuQxpqio20T06ZBsTZHZ53PkIgMIgEo1OVYUhQGVWUWYwhQrDEEUP7qsP+nfQXNMqeCIsoxR2sP7DsMvamiu6OkI69dve6vtn6HKIMklFkMjffGJCi6vV8NN9Ob9LoDk9XY/M77//K+r3177mceOvDRK5iTkhrqGMK6LRcXVpQG/Y4MPOWdXtuO5jdkz6R5/OxjoLmS4vWv3o+KeVbExAZRCWOMDok8OrIxJrQGDBkyHJOyJOHHb13a/om+NeSyTCp/9hujqn9jAt3ZnkXEXq9Dt7zl6fT9Td64wN8CqeumfPz5S8e/sEc1JQAkzbM+kMbUtk0DCtYSoiFyzArAKslYS97FUCOQJZMSx5QAFFGttWhsChERjLWGnLOWJYIKEgFAjAnRFNbk+aCNbZJ6a9au100reRsbVCcSc981wILWe5fbTJK2cRSjIBlRIBAEcmQzY7uYO2Mza7y1SKASnVFCVVAASCkaaxg1Z2OQKo1OJXcdUBQU1ZTAl01Y21whzi5eKO99/uNX/uvuteGW+Ytd73xmM8gzbwDUZJ0t9fiREycuv+fuM3VdO+ckDEPkWM6FdvW5z7hKd+6sDTsvLE3TVrnLI0XDENar0bmV9q75tkn7sFgxVaPwtAP9QPDQw6UQ/uz2t/7Twu+RQheNJSwAO0CTBo1IV6CTZf2E4LkSf3jitTs++7xte45bC5C7sLosot/5w/dV//SPr5vgrYPpQ5c/pRyemZs7Z8mziHUeWJMKKIAIkqoCGRtiVAAEUJAYGwAc/vzc1Pv3oHGcQqjaph0VnS03Pf0n5y/cM39+gUxy6FUBCYxBEQVWToyGUDUGMWhFJEUhy5wIEQEMAoa6qocbLAZIvSsYgnVIthtCndkoCTlFFVUVtRZAmFkVVTDv9kPDvujGNMIUEcD4zGUeCQERySIKSGrqWpglsrAQEQKQQQRQ5fqNZf/dAwAUYRFGBZYs73R37zk0Nb1zcu/V0e+59ZH9750+aVIEk3o9j2icN9aZp91w8FVnL9GfZSjaXItX/uHvnP7aPRd++qFtHz5MDE1opqd3NHUlqItLw+07DzteLjdWyRBzg8Wu6Yzn5i8QFmIUn/aLhx0SIUUWZkgEKSYJ0ebQzbuEDq1NBIaoadoYkwhfeO7q7k9NGSOIyLWduzOMLtbh5l87Ov+PU1NXRVgeX/XLN8evNlk8XX8+ptYaOHvr8t5Pz1gCNGSsknpABZDYRmOyruugNSxcVqWI5pkF1CDqjThXiBpATDGBaJ5nztoUW2TwLq9DBCvem5RCkXlrOmU1JjLdYuAgOd8fjsdBygzyWpiMgzT2vlOFMbOANIJPUCRg1sgtKBAaFWXVGIQQ89wzW0nJIhqijs8NqSNwhAWCsV4ZCC0QW3Ko6i31sCAICgGNyakI0CRhAp8bw2w/c+ODz/naFV/71sI99zUB6tx0EBnMhM992y4YYm875Wg1s5Nos/W185Hr4wcHajr97V3qFIrOupyg4MRFx8hoozl7ceL78+e1TQA1ope4D+zlR3sN8PdOjQ2QVXrZ9rd8ePHtAJCDZRAL2kGzE8GKHMn6BeC8pEdSNWv9rZf/wWVfeZlqEpP9x+tf870Hz7zpK1+8568+cfadb/mDQxMTnemt+68Ybs6vLM4BM4JHS5o4SARQ4ISIxmQxcRtq5eBdDgAh1gJu+PNzU+/dReRFQls3Sapub8sNT33JhfPfXZxbRmJVVCYERARANYgsjM6SphRREqj8JyIRJjJqbR5DastROxqpOiR1vhPS2JFD32mqGkKlZIQTgIKIISeaEEGFRDTv92OT8l4/QJ2BK7o9cDYjm6RBIgRDZJqmrMdjZQ0pIACIIgAaeAKqVm8c9t/dUbGgyMKeKImzeb5///FsIh/rnunZK1+6cPX7d9y/pet8n3b0u1fs35dlWUz6gqfumv0Ytq8Ry/DAyvxr3/DLE+tw8qcfnP77A1YMGjSm0zYViayO4p5Dl44WHxyvr/jcZU4rmdo+3a2bRWezxIjPfv0lxjoFaJoW1Dlv67pJrBbRWWessRZYDBpkYTKoLI89e/HIF3fyEzSQ4vmHeG7xyuGLP9q784+PL3xs9YpfaXbc+KT0jZwfPFd/tpUSJMy/cLzrU9MIai0iAhJIFBZkAUTTzQsijLEVlY7xPi+iSEzJk08pGWPkCRzQRWddYbaUTUlKRCAUM1d0s944NLnVIISKKsl5JLUMsW6TYPLGFaZrrR22Y5bgbCc0jWoEgwkjkVg0QglYRWJiRFVEn1JSCAYKJfa258mKaJTEGhEAEAwgJ0YB4bbjckIiZ3KTOWBPaNCiRQfWKBUdY5MK6O1Xnz784cs26mJpfm1lfrUe16opz/qiMckYga7aEd7/gf91ccX+1u/+WdFbv/bawymWKoYlq0Uw1gHzwHDm7tPVqY12c/1KyjdIzrBYVYfgAa4/0ikBHj45tgheSUFfuv03PzT/doeAwFEtIg4QDiFekvUr1s+mjULxMtvd6viFv/L93q9PhhBGF5e+/JRnbl517Odu+5fV5bVPPf+Wz149y9LJeyfacHF9aW48ikDUGxQeXduOFXwUZq09FijUppo1IHgAYmFo25WfX9zy3hklw4klNSk1U9N7r7nu2Y89fuf66tCghCYAEUsiQ6SQIBkLIIxgiYyCioTETlUQlCwBAAI09aiqx1YtWex2J+p6XcVxOWCosDPU1qIKiKagACqiAGDIKkdnp0JYK/qzKQSw5DvofEbqmNlYstaypNDUsQkcQcE4ZyQlUAPKaECUq19dK/5woIK5zznGlCJQd8vMYOvsjpXlhXOnHn/5dW9GY74/sz47MfmqFz/pg5/8nHC9c3sXkrs+HAdbHH7pwa9886uPnfvBg3ff48ide9kPdnzwsAISOed8WY6E20a3Hjkwc/bkPdLWBEq5QbdvZhLLMJ8XOQrhs//bMQEQVWYh8YAChKLgM6cCMUVjwKgnQ6wRgL3vPPLMi8f+fWdKjIZBzfq8/faxu+D/8KxzT+FiwxlvUB999sLOT0wyhCzzzIkMtimAojXeO4eE1nlOnBKDKhkQEFbNKG/bNssyAECEoui0TY3MjCos3mVAgCDTg5myaZgDgyJgGxtFtuQROUYlFOty5WDICLqYNpmtKBqLzvrEQSSqqPPICQFTG5MK574DAKJRGFmazHVRBZ5AmERBFYyNITjznxiMam3JxGB9K6bvwUA/77gIG7F0Zvuhrdddefl1R/Ze/gf290++9rHRmbnRaBhC9HlhxdVhnCQYaz/z33bfvM8CwIWtN2XbTqxX8oMLD8+tny+x6hI5nHjgjnOb59eXTy7Wq7VaO2AZGNzUtlVDYBD00sNFj/D0I1UG2CCvA6vAy3b8j48uvt2JRLIJIXE44YojmN/B45MSrsMtfVtO2k6K7Qtf9Dn3envgqh/ZCOXHD1xWHd//85//1ML3Hv7w237u/sPTykWSvXk2Zi5HoxVnTb+7RZA5lkRZy61RY9CDIhplVE9ZW7cEwGB+8MJHDt92MCaVVGtKLMnY/q49N61t/qAsx0DSNNEYRIRO0SnHNRJkme0UWZQQoyIgoGqAKDHvZCG1lkxIzXC0joTC7DveUt40m+0IF89Q1kl+oo6BU0zKKEmFEUmLwgOoCoZRl7Ix0US92RRTyRbgqEsWOGnmCwBkTjHUwgGBYkzOOWVAMJFVRKyh0S+uD/5ywhhr0CKCJiw3s527pwb96VMPf3f+6R/6reapbzsK05//2cH4oXp9rkx2+86rx3OPjOuN37jq3377zh+d3bF3ecw//vSnnZtfNEQLrzg5+7f7BS2RA8CUosSGil3bpuyZU3dLajt5TplJum3/ji2Lq2dUsBrX+Pxfu1xEWDQyW5OrsvWuqSNwAhQEss57YxNr4mgzEDanfnzh0Gdn6T/ZlMKGv+HOwd/A/+Ga1V911RcU6tzZiy/c3POpaVbNfAYAMQUQ9c4hoIioaqfTs9aKSIxBUZgjAAJYZlZVY4yKENjMu8QNs4YU8rwA0RDDbH+y5gQAkWNiISJARmsIxGLWtpWiEQ7GUJ73HFdtpJAUHRprECi1ARCIJPeDKE1SDiE4460x3hlQH0M56E2G2FqiBNK2rbO2DTVzstaosMtciuCMKcebNbYoW2Z6x47tueLokWuO7Tq8d8/sfafuPnvvI48+fNe/P+mb8hvlrsv3nHt4eflCGSMhgUiDEJ98pPtvv7QbfmiTu/e0u6ZndvYmZ6ydWhtu3Lvx0PLq2r3/8E18fGxLaryV0AZjcxaLIAqAgogvunX247cvRIMRhcTkyYxJX7r9rf+4+E7kFNSwTU/Jtz4QVhdVnZp95Pva9qhDEg9PPi12s92/9KQbXv6zppt/7k1vefKb3zS5ZevZL//HX3/0LWe6TqVIvHvQa0hK0TGAFNkgsBiM3hWRoyX0WdG2LTMbY4lQYjJomxTuf+6ZK754SAAl1igIAGvrrffHp7fWAqlOtULqFF5YhUGAcmdRE6BacCKoqlnmI0PLQYDLqnImD6mKsWGFxCjKbauooRmGcz+wvclmchdYb5uqNcY656pxrcD9QdGGug2w+XjW3ynaFGvL7exRcYXJbUHwBCSynCTF2LalAoMoCxLYFJkQhVRYQLT6hbD17wZVHUARCUA4rM7s3j3RH0yfbradvPY9bzsJT7AX737XmVfFZhzKdMXNP/5AFatnffBtJ+FPDi68aueZhfkzp+7+LKUyxbD4qkd3/t0RJUIwqgCg5aic2nG068uFuUeEW4cWLbRpatCBjeFqU4rGhC9601WqQMYooFcEgCzrhqDjcpgkCGuKanJAyOq29Z4U6bFbF/d+cjrLPJGRFquyunPiA+30DQAw+/CfL13yOgDYxY9fu/qm1tzfojFAp39sfs/tU85Zax0gpMS5zcmYuqlZ2JFFRFVRFZFkjHPW1XUdgI0x3nsi4pg0tc76xGQpC1IjaeGKum0KRSp85ASAoMZbZww2sTEGve3E2HJiY3PmKNqisne9qmmtI9DUy3q5yxMHRO13tw7rjXFTghprMLOum3eVyCLnfqJuGoSEFqu6NIR1DDazdV1mmWPO19dbi7M3XvVjN5y46ejeAzNT3UfOPnLf3Xff/91vDRcfmZ3K7DRtUv3Vp5255Ev7unn22B3D73/50dxCJWLUOEzTk9mDv7kLfujUUjO/Dvev+MPHDvQmu1l3RursobnN+TKGNG6Xl9vP3TWqYkwcUBGQUBHMTzxn5l8/s0wKqMCiFUCJlEBeuv2tf7bwdlHdYuAyLM6ncUBLKFvJTIEpgDqArdVbtr71U8tv2/78//K69/3F/PLS7u27UxvY0alP3Paer/1Ri6aqFf2hohi39VKSZCwVnU4UIYiOCjLU1rXNfeTY6/QkqnUUYwRAA/T9Z5+55NO7RJEIx5sNIlYVTExf2i2WM2siMAIgchRtW3WZ80SE7Lyt6wiAqkJG26DMYsgQGUUsx2sp1Za8cQ6NCa2CtnUZ7v9W2nMYdh5xABoDkyWRdlwqolpLMYY2yPJDfuKApk27uZz2XOFNZrqZDEs2hoqiiDHWKSrEEAKRQRUV4pScI+stgUmBV396vO2ftozHNYtab7lpRhe2Hz4ym/cG9991x4WXPQ4/ZO7+2J+uTAcwrtfJekWwu5bdFAC87Si8/tj5Lemxr3zp/a5sUkznX3bqyD9fMbFlMB5VKTFLWl4ZDab2dvPR+vrjHediFTv9YnEZ+n1hgeFqU24O8Zd+9ym7d152+bFbrOktLp39weNfX1y5xwM522V2TaC19WFZtiFo3bRtLItuvvh/V1Mf7CKaENTxBA/duUfn7P7raOneyU48ftkl4eBT9tCZxfJshZveaGA6/ZzlQ5/b6Ywg+JA2CDMFUFBATZwSU+6csxZQWdkAZi6vmojIAGgMsQRruqFtokQroMaqMiJ1in5MMYSxc47IEZBBq6gCSURCG7zzhmzd1OTIEPWyvIqhqseEBtE6ss6bLMtiYI61d946X9VV7nIwzqhYgx4K1wMyRJyvj5diZJdPmqYepSKUbnU+1KVKRdt37/nxW5581fE9G+Xmww89dPHU/TpemJzCNGPGlkPQzbBRUPHA0y9c+aXDTEJg1s83K2fWD01Pbj84oz27Mr96ZGr0km30ya8sbJnoD4ft5fshh3zT9T991/ojcwzFJCIl8lDkEwh7Tz4UVudHZERQgX7y1q0fuX0REBQggLZiotKYIiD94rbf+MDCO3pgLMZEAEDEygYPGncUfKPpAofW0ku3vvXzS797yRted+MLXrT7wKVok/OZQfzeHV/66w//ehmMatEtBuVwmHU1YQSwFvqRQyfDqm7UdZu27GR5r9cl58v1KiGwRpFgyT70E+cPf3RPazFjO7e8Soa6WZbn2SBH6EBDsbBdMNg0TcfnYiyBWEMhcXoCqzU2s1ZROJF1TjEpcwrjFGNI0s2e0PF5MTd/UVu6545m1yE9eKxTt60qAqjPKEYlAucMc2xrOHmfOXiJLF+EjfV02TUTNtfIlVGraLNOZ9xWYVzXoUVERwYQ6rY1iBZNm2KR5YT0+IsWtn2kz4zOOjIQG114PD+8d1d3Qr/z9fkw84zRC/5oz8YPJi5+sQCtUho14fBlVzxWHDxrjsMP/e4l93oDn/3Cu9LaApE7++JH9370mHUuhKAsT2hbmuwfLuuHy/EiqgeAGFkVQClzHkTLcYXf/I+/n5jaPrvtssg2pPG4Xr948T5th7mhNtRVMyKrViAGiSmJtlU7/vhV3z27NBdZynEKjfOuaNuRy5PL8swZ6wtrkjEOEdvEmXNJxFh7xZcPgaGqbmOqvS/KesMYx0lQodWm8E9wiROCqgoQqZIlxyxVVZEBVm3KmhEnio4kFExNaPNOR0SbtiTCLOtwaA1alhQ1WXJZlhFi29ZJUmaMIauIqgSgiJSSOIuGMlUGCpEtCnuTkbggVdHrVmUZQqnkCzu5Z+bYtSduPnVu+FO33vzfP7n1kQcePDz/qXK0FmPpjGnrjbWlxViu7d7We8pNx7bu6q+ZzbKHsfWN1N2iY1tzcfOCJXzkx+aOfHHaYwcdo2aYoIVx4SfqFCmOcz856d36hn/kW6de8czDMD+3PL8+0+/snsprMX9/dyWYBbWGOgGxp5tXl+sbZ1Y3gH7i1q0fu30xAdUCUVHRBWpy5Gm2BvHFO976oYW3t8wlGlEh0K51N9hi1DRzyC3QtLd7VG/e9qbPLL5r5vK9t77zXYefdBWPdeGhb3zjE1+47CVP+vZ3/q5uKqQis1lsGwRV0siwujKq69DrZG0bVzebcZMG/UIhAmE1gpX1YQwyNbHF2PTw/7W8/S8nW7KFcaOmUeQt3Y7jRDls3zblHY1SSpQCJGs9YKbIaE1kbcpaFfKsEOYst8okot5bJWibkQojIqDNsy6BGQ2Hy4vlhUft1Ez7IzfvW1pfBEDmVBT55samtXbQHzRts7E2mjs9cei4WZorq5G58saJJo5D5JS43x/0im4M7bgePwERrTHe52VTq6ozNrWSZc4Snv3JxQO3bVUFEbXGtJHPnSwO7ZhtsHrwzJQAyY2vOwKPeSqr5bWkxXhc7Ttx9eF9fbv75u+v2iP44C675AG++vW/lvHYGnf2xaf2/vMlILauKhUV4XHd2bdrz9LyPePNIQmllJjFGMusCEhEoIjfu+M2tTQ1e9hmg06WofqN9QsSh6pRJabUxhCUOM86TV3FVGXirKW2rVkig13ZnN8oh+MmlWluc7S+MNyo2opcZsACgoCvU0qoKTEwFD4LqRGwgBK5cdYhGO/cxqjMnDOEIUVSmzQCKiIRKKENISSOgQMBMcBUbyCsjKmsxkoGEQGECJnZkjdoQ0wJUr/IEc1oPEJiEIcAZEhAObWGLJFR1TzrIFoVTtxatWgoCkdN3Brmrft3Xr5v14lj23ddc+Jop7Dn584b7yd6k6uV/PrvfDBf/O6ZM2fLzSECOdvMbsmOXrGzu6+3oiPsudxms26qDXXS2O8OOllvebQcojxwy9lbHnjS2spKFddjYiTMbJZZbkLo0gByJ5SyWlrnmrjZi/ZS7cips8ujwEGfc2V31Ni7FtL3zgcrut00NxzLq3byRG/4vo8ulsyggiRksC8yhW4EMgQZgXnptrf8w9w7GKEjqIQJkqqCsZYwSdoN2QD4osJP7fitB1b+n+r6E7/9qS+g0zrCJ3bv3/Gan+0899j5k/9Rx+CzTGKEmBBYRQ0ZNEZErTMhxM1hXbdKDmJqDaB1+caoSgFSiFWsvnPr/NF/mB22TApqMbRNN7OpNi3BzsFkkfmQmrybCcHaaGQ6PZu7OoWYtKxHhKYoOmVdEyVjvKgiYFWOASHLbJFlHnMEU42qTpE/fGpjdYEmJsO+A1kCBoC6rhFNAIkxImKMiUvaWNwys6NpSoLU27FfyrY01rFiv9NH0bYel00T2tYYNM50syyCsCqpNq0iCoCs/tzmvn+Ysda0bRsjW+sXF/uHZicqN1is94fLX7L1wjepsLJ5vq3LbOLAsNk4euLm3fv61155g81lOH+GQFPTfub2P62XVxFp7mce3fXhS5gTJ0aAFKP6Hbu3dx979LvlKOTetk3LLN5nMSZrLaFRRPzBnV8AL0V3e3cw6/IuaZa4irFtqzVOFWqbYhAJzmRtW6m2oIpIzGyNQwARUkmBx8K2rTbufPz+xc37Z3pbmybbaMvl8SbrOCs6mR9IS2tVFbUhGiStyVoRZklNCLFtQMQa6k8MQms2hqvWokVy3jmbqQKACiuBJlSvUGtQZSJjyIdYOedjbK01iZMzWdOGOjRbOh1RNxqXItGKrU0yDid8pgZViFkQlYBYWxW0ppe4u33y+N7ZSzzBNceu2Dp98PBhH2vz4OnHVhcfv7C04QYHRmV98dEHzp6864Fv30scmroZ9Onw0ZmJg912gAF54Ipx3XQ73bIaF/1ioxw5xI7P+p3+sBq2KT32nKXjX9heNuycpISZzyYnCqtZmdoYhgYzb1ykPMVNVqSYXOG2iT260TZL42pzA1y2o2d3TpooeHyXO7YjA4A77qv++QOPB8FWJCE1KdUEG2zWMe4DcoL/ZedbP7nwjkA6UkVWj8aCiRgUcIAuELcCU4Q/M/vWzy++Y9erXvH8V74im5j43gPfOP/qN1z9P37rkZ3h1PlvK0eWkDnjETm1fd8tXKYaugRoIaYACpYoqORZx0GWWSybYJwNsTFqPnH9Q8/66uEqxBgiI403R54wOtOqdUMplVMCicmQ3Rw3EXwrkZwLDWdW+91CAaMiw9hlWRBJrE1q6zY2dWMEukVHFEbVyGd2NO+WF4yxlBUJCY1BZmGGGCOAGoOinIQNTCued7itU3RNNmxiTFISdvqDCREeV8NQRYtPEDUAichaBNDEIcj/1vxa1X/PgDmqgrU5CEC+/ZJdg2XY29l5xfreW/L7b0vawebc/PmTs9tOrK+cnT12w81PvVq2Pfm1V1ffemxhXI1Gq6M7v/Xx8flziHjx5Y9s/+DRwMkbKyIpxHxq75acH334nsSCKgAooogoItY5QAOEePcdH3dks+50f+u+TtEntADAHEfleipXgEsGRnAiUSWyRNQEiiCKaACYJYVQJQ65y8dNNRytdzIL2NkYrSyPFpbHy0b59PLmWlXnRhPEyBqUWJEMe9sjY5G4rGsW9tb3ii5SVjVjY20ISTkypxDbbrer4BInRLWAZajbpsx8lvmM0DBqYgYFxDb3Wwh804wDN0Eis3jyLIlE+91BFLNZr3ITmNhibmjSct70n3Pj7pln3HjiyO6ZPDNrozpT843TZ++5d+mao+b0Y8tLTVofT993/wPlfZ8iGPcy5BBmZ/zEgZ7dZsSrJqhSTYvEaTAAACAASURBVMZ0bQEEQMSshc8kMQASmaLolikQ0iO3nL3k87siU+RxCFWWdazxpJDneVIJoW7rUZ5l6Ip+3g0hqCoR5RkcbmTnOA3nlpkoNnG20P/6Y7PwQxc39QMfX1u5d74iPx/aZdRNgRbxkEKNZhfgS3b8xvuX364JERIiGiRCNCyNQSMYMe0kN0X4Y9ve+t6Lv2evv+a1b/21+QvnT9z0rI/fdONrvnvH73zoV6gPk2ZQBvbObpSrK+srE4PJzJk6DHPFiW6PQ0ixNRC3Tc4UzneLjKIRETBknRXW22948AV3nkDEJlYxKQAYC9ZmIUpoEjEurY/RpuGoKSsajcpxkxzkS+trnR4azYZl3Dm5I1CFqmRMG9NmKKtxuzFs+q7njVuvyjZEiClxHhtBUcLI5PA/gaqCgAASWQAgY6zvZVkZEuW+Y6zGGIh8p9+f3TZVN5tr68uxTioCKswpsFpEBWEVUhIBAG3eMOq/Z8BJAdAYAgCne/Yc6Nf9m7ft3Pf1Ub+47y/c5KWz3WJpac7R1OrFu3pbjx+6/slw+TOet3O+UqttvLi4cu77/7J6cUEFF15xcuYDBxwUQkqa1jfb3XuPAy+cO30mxtKQV2FhNkSJ0ed5Sskg4bf/4/3OOKCiu2VbMTGb+S6RU9WmrVK1HpphjI3zpKwqLMwKCsrOGuakKSHBE4iQAENMiICg43KUQisaBFISt7o+HIfR6nhBiTfr8Wo9DKBJO1UzThzyrEhCQIgimTGdbo+RBWFc1qqJmUXAey+qVVsjAopam1fVmJC870St2hAkRW8toSMDiEpgYtLIwT/BusQinPIsDzFZQ3WMYd0WcWt/6tIJn2dXPvOtt156annz+/c9OFpb1TBeGoZRA8PGrW3C6kZ14fRjvHpft7mQOTvo+8ldWOyRNqNWoxgjQiQEFokoNx5SSspk0BCoUggxyzJjbEwRAc48Z3Hfp6ac74mGlCKhIwMGyFpbNXXbhG7u+/1+E5Ix0Ol0xuNx0zSUkVfqiFxBnenlpirHqY1vftE2+KFvPDD8xsNxLvnz91zYKO1JbXcoLCAcZEOQthh6+fbf/ODc/1RyXQVBSKAJRMBkQmOKByjvIqqkm7e9+f2L79j/9Btf8ubfOH7JZUWnu3F2vrNn5pfeeWsBJnJjfEeSNKmuebx/Zn8nGyxsrKhGCZL7rApV5Lbf6ZPqZrmZZd4C9ly+pdOb7m3/4o3f+9FvHhIJuZvIXMdRRkCENnIMIVq0rbi6LcuqtaZflml5Y9G7bH0UUhubql1Y3zBCQWoJoqpoTOKkSYImTNyGYWQFNYZNUiIl4QSQ1KqIFEXRtq0ycIrOWmVsWTq9HYMuJoqCrNAKszD1i8m2jSmlGKOKFxFUUEU0QIj6BAS0QkCKWv/Kav7HPVEgQiRAsda47buvTBNHBzPubPGsLQ99dnljfscWOf3Ifc5PcL1g+wd3n7hern7xLduWG6Uu6fmLC3MP3t6sbFhr5195ctvfHmIVBrUoQfzuXfuGG+cunpkTaVVRmAFUVVKA3qDPKRkk/Nbn/tw5730Hjbe9iX5/BqhgJdVYjdZQGpWgnAwZg0ZVRVk4ify/PMEH0K7pWRjmuzzlfd+v/e1UnS1arXZXFYQlggAhqiiimwAm0eBxwNgkY08GHFMM2EFghG2MB2YMeILiYEwodkAIFoSQkERRXUmrVdmmPWfLOef/z1++9rbnee77ztFmnOtKZoqqRAwAImo2oGEpkiU7lDSOWoQNuWESp4CjZY/xZHN6uLp1sj29urrRSV5t+uAnRoLOKUBwzqE3EEEbhpzyEEJ0PpiZQmLGfuiBEI3ZUS7JDFNKgKgi0ftJtdcNm2HsEXG+WHTbVrQQo6kROyLMaWTikvHoI0N/Cl/0upff99Kvv3b1k62rnsIXn/W8bouVMZ1dc6uNW33G2sMoJ2jjYrp7+e5dd4c7g5MBi+bIVsA0Bt9uti565zwiAthsOh1L2vYbzwDgQ4ymBoBgYqZXv+H4ytvmzJWqjsOIyMhABjlnRWjqGoUcYrFST6qcs6oOfV+FKpuBCkfYCdUDUk2ePbuD81e8cvHkc8Nffnr71B2Tpy/Vk/ecHX/85FDMgUzALbXczb4hbQS+8+JP/Mbhm8/MnAIDEiKjLqHciTWSgCoiv/Hij/7OjZ99yVd9yQ/80q8uDs57tFvdMoj70X/zpsQdqGa1nWaWSmmH7vL+gWruZdjk1mtlCtu0jZ5IOWctJpF8LynnPKkbR/SJr3nmtX/98uPjE8J+Np3OmkXgeHHnynbcnK1OEdxOvZfSGEPTD5kM1u0tj9AbQQYofLhsuzZXk0m3Wc9mfrpwfRpksKHgZtmlsQYsInl9to7Ri45EoGrNHNNYmB0iD8MQPAYX1qtBUmkWO/u7nuqhYC6jbFbSbwkyDL2lUQyUwKmpKgCSKSORGRCiCCKwD677oePm3+whOXYuVsEERIfZpdeff9G93e4LTfnGh/9TWZ1FouXRNeJQUo6zg9mF+/W1f/9V9VPVYr7r9KEPf4RWj5gAM179rofPv/U+70BUPeJmhPtffN+z1z59dnMjOpqAmYFpyclMm+lERFAN/+ZP/j2RI2LJxU9ibPbZz5vZLrnp2K9lXIq0MvaEhAigImUEMAAzUzCDzzERMQSHrpRcoAQlRSxarCg7M8KcRlJJBrmMQ+7Ech5tmdOjzzwh0DnXJMA2lzGLUCN59OykGDrKJSGA8867WAWfSu7yQGRFh1SGXAqpuVgTh6FLPoyEkyLWD2s1QaWmmSRJjmwo0rYdoXEg31Wnf70ddIiT+nVf+RWPX8v96Y1te7a8dUpmogmKqiM2evk9B3gww1kqc1vBKVpw5NCEwNp+QEP2bpv6QC6NAmbEcDDd3QytmDC74DyAIVIIsWi57YmvvX7PgxdAiZmJse/7JOqQVBUdA+SdyZ4DHKUj9imltm3xNg5ckkTW0WL06CCSm1CYm/EalgvSym9zP/+L06PPLG+OpQO9i/REgYHuBjKn33/+p/7P6z9tAA7IwEZQRK3NJQIArZGzwLdf/tF3Hr3lS7//Td/9L37Ggm/bFpQ++uEH3/qeX/RhIZKMjFNh74loPXSOOQIfdjev7NzlwB8tbxqlxs8n1W4pmsftWbupQ8xtP93f++TXPfPq97x4vd4QRWb1gVPK3mGbu2yq6iKiiC3mO8uz5UHdNFWcV25d1prYG2xSX9fVYuaceQCJET3Ovbn1mE1BzLVdaxDbrU4rZlfYA6Hv2pHIjWOuqqbv21TaW8fLvnOnxzljqpwDr8Ww7/oy2NhisU6LL0WIRZIhopgimolDdgaACiiCSGYy/mi390vT9WZQYEMKsYnV/sWXfcOd91+6Nn9d9dnfPTte3XjkY5CO8/rIxUpLjdNFNXmBf8OPfF7z2WZnUY8nf/Ynf3rXfIMWQuDHv+OjF996P4cABlhkK+6lL7ny6Cc/0i3zMK51KCJSSkIzF5oQo4GhGn74z9+KSIgIRkiMxMa+me/FamKARLGUUboupSMyRkiiI5ihkWOfSyolgSmAgo6AhBRUDUwAQFXhNkQEUdUMqCkxczEVEQNJOQGaqPQ5p5LPNqtcys32+ImTm1IsVhWSDkWyAhl58wVp2Gxi5XcWB8xhGNVwNHAUsi+x4GDsLI/MPIqVkkP03dCyo7pplsvTtm2DjxWD5eln/+QamnkXzftmsdhsWlDruhYKRtbzF3bq/Xp2weN5HHRUg7PNKkMGgypGzz4NiRERvPPGwackOUkumRgYSUquq5qJSilSVEVD9GPOJecb3766908PKl54ABf8rfWpKuYxd30XKkZ0wfsiQogxRiLati0ylTwQuCpWbbuZz2aZTcckpk0VmX0pyXn0Mfq/uLX5WHsi/SngLrk9o+cwO4D7jb/1yo/8wXM/b2hLUwFrwJgY1QxgB2llWpl+2wt+8p233lIa/K5f+5XXf/k3Pnf0dGPy1h//3k+9GL3bd6iF0wLm62Hpqni2WRLwhfm5Td/WRk0Tnz66EXx05CUVk7TWBKRNqGuuXcWPfO2zd/yXHdE08ztd3642SyHbmc2DbxCRnTlXtd1ZM5mqhJTXhKDIB5N9qmB5etLnvuZ4eW+nriZduyHK56d3dHLmwF1c3J1KTgVjrHNqc+pjrNRG5gpLYL8o1iG4YSu3Nretz9Zn627MYzk52SgyGIt1kabTyidJRWQ+4yqO67WtN+lsNYxjyj3NGzeqOYd7dd3mQQsf/aOjl/7ni0dH27bVthun88bXB7L3ild8wec9e/CGvxWvbdf9Jz7yvhsf/8s8PEt+FwF8vSNhZ+/b/vdX2iczYhXh8ff91wrFhQYMnn7TJy/++ourwEkVUofTS3e/YPHRj3w4963mPAydqUNw3juR7Jwzs1IyPvTut6qqKZopIZkVY3RVA1Q1zWI2f4GoyrBMeQUIhFa0Z3JoOI7D2G3AlAjAVA1BlRBNFaHknAGAiABItRiAc8H0c5AIEAEw5QQI7IgKFFExRaarJ9eeOLyKVIxkuV0V5eVqdDzpAMA4t0kdUoCubfttPjg3j1UzlgFHq5ugAH0eqqqaxCaN487uznJ5VtfN8fZkSEPdTNabrZYeHJy8H8uNFmptN33wFQjUHuqDZnHHxcW5iZudLUGmdePEVv0wqSer9TJpBkBmrmKTxnEsG3ZN228cBy2jD5UBjHmIHOqmKUVyKVlLGbN3zlTBo4rd/LbVpf8yC5VrYs3sl9uNyBi4GseEjspYbgMAImqaRkRyKUhUUlrMduuqOT072d85WPWtldSNg2cB45xLjIHY5p8Y8MPtDSpLM5B8xUU2PbYiRD97/p//2uFPbc1m6ityCaUHLY4X2TpKAdwF4q8++Cfvu/XzLw+zVvru677oO3/hFx75n/7hX106/fQ52PM1kOvHnoqrJ/W23wLjfDJtfN1Kb2N2jrdD66mKPjp2UjLTBBkAUUR2quYjX/nEpd9ZKAlphqpqu6FxdROc6nhbiBOwUrI202rbroBdiOFke1ZxyGNRlGW/nVbT+TSkoUynszz2B9PFWBID7i8WIN6QzbQKiAUbP6kQJ34xaueVATKEMAgxsKbhNK0aqOowRQyjKIMbbeOcD55rngwp59KWskaFzSYXcTdvHj910mu3unlrfXH/0uU918ym0eFffeOnXvv79wzZ+gxtn5+9cXymd222k7/9vd/+IX3lvfJRLe7JTz/02Pv+GLdnHBfFWYwRqvPua3783vYDmzFfPthZfua9phukqXN09Xs+cemt94YQAfHo8OjyPS87WJSPfeRjuRtT3+ehN6BY10YAGQAshJBzwo+845dVgYhvG0r2IV648qJ6egmgJUIFp5gQG8eBiMA8ECMwAqTUHT31oXHoAVRKRkIpGUxUChrfpqqIKDoSsd2mAkiI6Iidc0CcSyk5MaICmhkAGMDYLyE4Nj7drE62z9Zx1nWybodPHV2zUSNXR3l7a7WuIrO5SR2qiS/sPBqz74fRzAAxOJfMhmEkovlsBqBIXgDPlssiA7rouuH6B49tq4MOu5fnB1cOdGrqy6gynVdsExlHMxsq169vgcKQRiOZ1DN2t/lcSu7X5JuseWcyTykjuvV2UyQ3tZ9MpjmVru2UgZG9c8yYVUXk+jef3fH2fTWLoQo+jMPYDVtQkqLFimdCRACo61pKKSIGUEQck6dQiiBYHabZZD6Z3Dw5cqjjKMwOEbzT+Eyev/P0OsNSSquyILqCrOROpPz9Sz/6Bzd/NhC1lsXUGQLjXEIH1lp+wNVfNX/9HS6e5r9+chyvliE7mJf8t8Le7/69S0sVM+hKj0pD7hwGHwHAEE21zJp5zpmIQqw8VWMaUSHGmgEMUM2yFQb4xFc+fcfbdjny+bCzSluDPPEuY0j9JotgqKBIzgYoyNkRj2nE2leOU5+IqTWNrjIxh65kOXdwbmjPfGicozy2Q86KKGmso8/ZKl8tmvr8zsUph8qshoYxnvTPLapzEeqBjLTUcWpA2TLquOpPk7XBNSBQuZlZJHIoVEpywSEYGYPCY08fHp4cdd2WeHrn5enffMNnvvAPHwApDJ4tLrsbn1m/ZEiz1su5L/vBu/r3P/P0zauffmj52Gd0e0wxAjfOIdQXwtf/s7tWHzSuL+3g6Wfe2Q4bpoocPPemxw/+w10IUDl3azXe88BLWW89+sjjMuQ09LkfFQ0IgNgDw+egiOJH/+zfIiKRQ0TNkqHsXr5z58LdTDMARIwABaFokZRvgqmR81ybSs7t6uZRu10DKJiyoZgCgaqaCADC5xiYofMgaiVzjKCaUw7eKwEj5TyCASLa5ygRAQVRiUTURMvOpCTNfR5yTl6o7bp3P/aRZV7V0TkKY+qMoKeQyxY0rrv1jJqqqloYY5irZNOS81g0MQUi7x3X9aSULEA00jhkZBRnrY6oRhGCmy5Pz0C3wjSPi0m1WHcrANz23XrYNN6z8wDkY9C0xdisNyeVDwrALrRdV8qITNGFyCFypMahYRpHUx2yMNNz33x89x/tGzgEKzmDKleVJCFCI0m56PNijJ6dggGiiBBaFZuh63POTZi0/XZa1ZvUV/XMObfdbnIease2kYv/z+GJ4dLgRA0ZLoN5MDT8uxd/4peOfpoyEgEhFgMDqxnvo/gA4otc9fnnfujdhz//iGxbkBpdUfXOvbJu/usPXKlGW5Vu1uwM68HqNKxHx2FvZ3e5PjVMAd1QCnsvqSAxEuzu7Jvy2fYw8sRx6NIwpu31b9le/v0ZQPGuAjFmZ4iTWNXc9HlcDivJIzEAWowVE0mBVb+d1VElA2KvUHHtkXYXi8PDo6auOfqSs5pWVZ1TEdXoq8bHvt8Ag+CA7CNZ9HDX9P47mytd/yy4uTNnKIQ1KIulAv2ivjDKeLx6Zja5YEiNi5rVBYdO2u22rmZ1nN1qr05w14dJLjdLqYpMvFv/55e+6zV/8IrT9dHJer1aZ+fi09uXv/4NX/MYfcG337M9Tens+OyPf+etx499ZrLoXYObm8xV5Nml2bf89IWjd2NcXNlbnX7mz5LE1AszXP07j19+6/2etSG6dja+7OUvPzt+4tqTz+R+1DxoYSBFQBNCLACISACED/3JzwAAEamZiD388MfuuD9+/ud9T8aCAFqk3W6vPvleKOOL739VM38RsOe4y9zomA6vfcxSx+wyOlCg5yFizl1KOQSPSGAskg2MCE3tNhUxVQQjJhNFJAUGKAAGQCoFkJEYwYiLoStqZAogqlhEVQGxNiy5pJSLRxbQbOV0uawbHbI9fvjskyfXmGAvNJULveRN7gYq2bppWGg2EW8WhlScZUK3HrNkzaU7OLggjJ0M0id2MJlMNm1PZkPOBkkT+8o5xpRzUkNmj6oCY1KVsaoqEQEHiqgFVcbpZDGMKzV13puxM3DePfaGG3e8fWca5wowjnkY86xuJpNFztlgAGQzExEAYI8ixcDMZF6dUytqJeURDM2sbdu+73d3D+qqWa2WuQzFxFx84I+fu3mMo8gNKANCrYhkF4zfdOlHf+PwzYbIgsTyWjd7gPg8+jMbd6m+Lus79//pr97450U5OayKCOLdsdo5Fx/7B6/GMXlfEdnQJkBYtqcicsf5S9ePDxFLHedj7hkpkF+nlpnWm5MYJ1Ls4s55MTORQddPfO3xi/7kBcvVGSnOpj4Va4duXs/q2hPjdrNNxqbiHddVHNNQh7jebCgEMzQoopqzRodA4D0XlcV0BwGLmIBFz0XyMPRmtuNnvY7MhpJKEfJxVk8vTvbS9mzXLSp2XBtYjVbHsHCOAzXL7vpyOO5S3p3OK7dLWnLKrpl7pjFtx9J5msQwq+J87E+j562qcXnXqx/6loe+3KHfrE+22+2zZ3w6/YYbzzz85OLLvvFK3ow3rz/7zCPveW9ZHn/JF1/EsvmrD3VINr9wH33FD93TP8Q8OT//bHfzgyW79Tp5Hx//tsfv+U/3iZYMeHKSX/MF9x8dXn380adSmywVDoGRvPdJk+YiFkLwmgs+8uf/Dv6bLPnsJO2cxxB2GIqZqmRR2WyH6GJdL9Q51GTs1LLkjrGKjgHQ0BsAIpRSkJAARISZVZUwqGUzZUbEygANQU0pS9YO2Vdxt5iqFhUxAyhdyiuT7CgCOQNUKQRKXKkZOboNTBFBzUSVmdQUAIkdaT2Ow6pbLvuVdzSvm5xSlwbF/Ozq5OrR0zHOpxyz2NHpCQUOorP5zvXN2gC2/aaT0Tm/X++uNYNBztZvE5MCOxd03sx6yzmnorLt+qLDrNnJKTuPY8pNnBAAOigi02a+WZ5Wvh40L9dnFFkUsJQY4/VvW93/rheAWj8MTD7lQoYlw87Obtdth9wRkX9eYJfLbVlUfRUlpSqGXFIVZulzckqJHXoft9sNoJJiW/LFWzj/8+NnAVTpGMsc3QT0xMo/uPgT//rWz1QFR8TOciSrlS85vs84YT5R+95LP/m7t376uNgS5CVuspT2C6rm9HWX7UtefNqtXIya+r2dg+PlaeF8tlrtz/fPujalXrtCHgmJkYuMVZzkMizm+4dHh1MXgeDi/i5Q854v+MDXfvr1N25ex8qPwyYlzZrHYfAeY/SqKgCevedgBiqiKkgEiAGdD4QM27b1jsecQvBZyqSeolgdK3IslkXUOU/EDvym3UwnjQM861e79X4T43JzVLmqQUTbUuxFaFYdSEd7830FLrrJJQ192a+vZDiTEkKkG91VJ3FeT7QMWyyVn1S0R2AOw3I8Ehs//jUnb/ibFwXaKSkH7x+/ubBzb9zbnf7Wp6q70/uvXHyA5NaD//GX77iYX/Ly5rOPnj38ZIJc7V95AF//j+8dH4rz81X+oBy9t1jPuTnb2lPfffSi376jwbAeS9uFB+67Eyv/kYfef3zzjKV2jvKYJ7OqbtzRzdYrmilUih97x8+XUlQVEc28WqrqYOLMDNFSySISK08Y2HGWPqBXcACOkJXNIZVS0HmTQoxEqCpkYAaIWErB56kWA3XAIoJIhkAYyZsRAca6mjO7lEYANahTWknuHUYlY+dLSpIG4t7MsuRSCrMjIEQzE6SKkBAQAIVGBAiuViEVMVPRYiYIyL5SKxU1fR5SKUaw2p6JJVV94vBGSnnVrq4e3xhMp81k1ffAvh+yJTm32BkyJRvqpmqaCMRjyeM45HHj/KRrRwMxRBCLIbDHwbIkZchNPfPs191KybbdQAZmcvPbN3f/0Z4RpmF0zjE7Qqeq3ofbUk6qWlVVzomNSinIBEjFIDgPpohAiPLfMEUiVpUxtRgdjpm8u/JbV5/ZwqiwQVMpew688Q+c/8lfvfnmxFRJvpP9HPXjRQaCrDojvwP5uy/91G889y+eNpoRXmTnLL1wPz7+LS/s9mo/ylYHn2U2m42Su9Rn0wk3aUgHzey43yiC877fbgkBIIiMVZxsS9eEakgDS5nOdj71ldfuetvedFKdtevoXRoNWIKbOg8iIxFlLWgISqoAAi74MfVIqKJV5dChiDr15JjZqUGMwSNJzjH6PrWIdJt3YeIaRNlstsFVmSVqCMGthzNugg6iJU3nlYLpmEjLpIkeJFCMMJtW+/O4l1Q226OJu8A4rMez9XhaTf3Ye4PBUZCSQwzbTis//dAbPvbl73slmZvNwjgMf/XYA4fzL7s6eY2qvOzW7y52Zpvls0/8zV/8kzc94Kvt8rT+vT/92GNPJP9VP/ale08ViM25K/X2A3LzA5suUcGj9vCJ7zq+9CsXLu3trMcxxkuXL50Td+usPVot1+sz6dbqQo1s5y7Mt531qzOmenGxxg/+4c8iopkhIWQRFsfzUgZm8j6QC4CsYAigaqaIjoKvkDxRyHlQUyQjBNQCoCmPItnyyMzOOVWl54kAIomJmTp2hFhMmUI/tAgjk5NiBqJWqqphDqZAjI4qIC5qaIoYAVHUQqgRK1URGUV6QgC0IY3eO2d1KhsiJfLsXJZcSkIpgIYU1UbmYKBorAKqYmhWtABIypv+pCReb9cZV6eDXju8OchAlusY2uxP+03BYpJdVTezacmDDB16JxraVR6Ldm0/X8wwELMQ8mp9LEVdcDF65+MwjMNYHNO1bzp+4dsPhNFEvXMmImYiJaUxxojoiCjGCAZNFYkoFTGi1HVjSrPFYhgHlTKfz3POKSVPDpFyTgZC5IQ1lxQeHy78+emTmFnCFhWt7IH77ov/2+/dfMsW7ITzQn0LugMASDMlYEWj//7Cj//7wzeLSs3AoPed2x+/5/6T8eSkbS/OFjfXtxzXaMaOgGkWpyFWq3a7qGszK4pjGkseGBkssBPnnJLkJBA8pCw5H35HuveP95lsNbaWdXdxft2eTpqdzebUBzSTJImQTCHcRtUoKctIAAQgWojJDJkIEEuRLMLMwRETAZrjaGYixUy9BqNspKmYp6BUiIkgMEMeCxk67xVLykmlhBAiskea+HoSG+dL8HWF++eafdTSwo2bJ89Ucc+hH8rpmCQlPJjt9117ce++B1/z3td/6GV9v0YzyLM/eP/+J1/76/C8F+vHv3TyxDOPPPzC6cdf/dI9Jq4ni9/6o48/uPsv5fKrAeAVz/0fL9rz9y4+45ePPH1zPD69dWuZ1j+4etXvXzw56ds0rJdw6fzB3ReqTWk7GYx4XMs22why4fLuHef3n3jqUdSdqir4gbf/nJk5F5i45MF5b4AKSEQAQM8z8t4FFUBkQfGuAcdVtafg89ihjlq6XNpSBkRgZECSPKoVJiglMTMiighhgP8fCprLaURM3odSBBFyLiF6x0HVEA0oGICZ0W3smDwitt1aTJnZsfcuiBYAEBFmB2pGBMBs5GNVzAhZUlYiz6HbnKbhpC/dOrVdSg5tLd2l2UEV6mcOD8+GtXfBjFRNUnKumcdmv7lt/7lbzzpPFblb67Pnzq4teXQ9lL47ljEhTsOkLbTarIcRvU+zZoe9hWt0yAAAIABJREFUy3msQmxi00sLbGOft+OmqpunvuHkvj89MB8kZVSSzGPqmVFBHHPbj5XnMfVU154554KAYBaiD9xs2rNUMntXVZOSeyRXUtmZLFJJyZIDzqP52qeVwm8+URU7gmImBWkK+H0XfuzXDn+uEiWiTvMUeQWYGZpSgGFh+A0Xf+q3j34msyXRS+xO/sdznLLGkMc8m05zGiVr0i5UYTHbK0WKZUNx3pvQrJlv1m2xQpoXi3kV533fRReWmzN2TBysyOG3b6/84cV+2EgWcDSfTE5WZwfT3bPlyXw2bybTYRy61KODvttEX018U0qBgI54GAbneBxHdpxLTmV0gR03aWgBQASAEMxAy7Rp+jG5EJhQS2riJGsuBWNwqSQ1G4cUfAih8sGNQzLFqa824zLbGLxDYq9uLGla8aKp0Bcu/tLOeSxwa3VjQHEWz8ULFU8mdfNHr/7A57/rgKlyVo4O73jX4UueesWPwfNmw9Wvdu/Co4e/5O4j8qAsTPSnT1x68OAX4Hnx5JE3Lh7+wt33TmMOHHOybUdv/6IHv+b9L1mepeU2Hd8YNn27sxuQ0tEZnguBCIYCXTdO6np/ry7joDnECeKHHvx3opmZRQRBEVENQgiApKZgYGYuBhFVheCDgZFrfNM0k32hwFYRoUhXtm0pHWFO41Ck9+xMpZTETGAGiOMwmAkzExEzGxQEZ6q59M57QASzcRxjcIgOgdQKIKkqPo/YgaGBSclWskhRKXxbdExORFUtxGDImouUzMjmcN1uvHc3T29dPz3cUn7u9GQcXJuHqqlFsoJOfJSSkxYlKmV0zhE5B9wNGthVDPuR0HNVVWzM6tXw5np1tl55rDd5SyyLuhoF+yLjCKJD9ABMpeQ6xuxL369JNfUZiJn8o19/64F3XCAH/baXDI7i6XKczpr9/fl0VpnmaV133ZCKJXAqua5CzoMRD10Bp+2QKoYxqwsI6L1jTeLQGeJ6OA0U2dH2MA+Pn1888sGk2qojswDyDy//xG9c/4meuREuRB1AZaU21wMtEQ3gH1/44d+69QsJS7G884o7tq912jeKm+DrYeiZUUyzKhESWPA1MRIDsUulWFEtMptNmUh0rOKUHYyj5DQiIgB6X9/4luWVt58rudNc+pz3d/a2Q9/125zTZDLJuUgZFZEdD/0wiiwm8xC8IjSO1+tN2/ZpzLGpRQoyEGFopo4gOB5LNhUmllK846FvfYhM5B1H36zbFUIgEBWY1JNt18WqaodNDK6KlSkholhGhDQk9p6Uz9r1hYPd081prJrSy94sDmhVUfZckuw3OzXX07p53xd/6r97z4uiKIL7m8/Irafp5ut//bR+AQB80+ThF7gb4+HbXnMZitkAy213+uQz/W+efw8878Jz7/r6K8+8cv5hwLZPy0W8nK384avf+80ffv2YV0VcE/Nqy4s49Z4ee7r76Cevnt9Rx37aNKXvbm3Ws0ljI3hP+NA7fzl/TnHOGSohSinBe2JvZvg8DhNENDMACCGqBiVxvjYrIUyJqxArMGy3JyVvHVopo4ggmIogJhEholIK2uc450opziGTBwRmVgPnnIiYGSOZISEbiKoCADGXUlQzgIhkACyCAEagZoaqjj0S9V3vHRBXg5Tnjq4/u7w2sHvq8NYguEoDFa1DlZJWDRctisaEs2Y6pJEYFTQnaLtVjK4UY3YGTqSEiGJgRuMoCBACIDg0HlDGbRd8DOyx6KQmBYdMachVDGI6jH0TKwY0sKyy6VtRNimPft31y79/vp6oZgCFOlQlwXwx6/p1rK1LIyOaUl3PKuedak7dZFoPYA540viuy0NOy24gtjQoEt5mgMyhyDCpq6oOh09u2w/x+596+Muml8vZjRYgAzPg/3LxR952/V8nn5yKM0rgEqaFc+dLiGz3n//hd91884nh/I774lfd92zz4bprtrZxLiJzjI4BshQV9eTJ8Xq9ZEIEFjDRzEQENGl2DFIVp0UHolhuyyUEL5Ke+aaze95xPiAaaF+KB6YYdDSRhJ/DIdbLzXoYhuiCqlQxMLsikkpfiopY23bIjIhg6pyr68qzM1H2JFbybUWQKTg0saaaNJPoLN5cHSEFLMkx1LFeb7YZTEUAhJGJQog+S3boUCkEz0DLbht9SGUkcAc7Fxx3127dpIISOGlpQnCKgd2TX39079sPeOby0eyxh/Jq2DQve1O49/VfvH94drrca+Zz//5z7pRs2eNGEjz5BH5g+8bqtd8/ba/2f/0rr/3Cl33+wacFh3V7xMAxzt/xmr95wwfvMZh7ngRcjRJCtZv6dhqmhSvJnRiAlO3q9Oqt437bzyc1FsK/evAtZkjoiTyAIqLm5DwbkogQETMDOkADAAQMvgIIOW8QgXkmlJvZ7mR6pRRN4wakz6lHyyplGFrnkNEDwm1mloa1956ZRQRNVc0QDQB0VFVmRkTnHaEHQFFBJFV1jlNKaGSQS0neh3HsS5Yq1gSU8lhKrirfD12BoU14fbt94uiZZ87OptNmHPuSxkkTnNXFIHGqrVYTNUWEMZVkogCOeRQr0jEzQsiaPUczrWpuKtf3ebPuvHdo3FN3rpowNcu2K2kgAnJowOOoJa8dVHu7B0WLipjKiOiANGUAQDM0e/SNN+99x92SMyGhaVN79lEVhnGLnNpO1cTEKt+4KsxjnNVVO/ae2JVy97m92lcV0eHJmh0czM8xxdP2bJnaQYRQmoY2w+rap04/9K6NDenFL31p1W7s5GZ/2u5s2v/1jn/xb2+8JUMuhtdVzqO9xE/G3I2OnxT9wRe++cG9P9pcvnNvP+5eOPdY97ZsbKX0wzhfLHIZ63rRpZOxT+d37rxx+rRzVMeakAPXx+tjAZWswRGSxDBtu/V8NjfgVHKovYzDZ7/u1l1/ct5lE6IC6hDAc3A+54GZEV1F3PYdIpBBjFMfeL3exFCPZRBVIpTbFJho6PsqVu241gxasJ5UAVUNyftcZDqtdLTpdLrptld2Lz556+nJ7l63WXlmTapgbcmeoI4hp+I55jwCmqoy+gzJISUzUuz7QWScTXaYJAY3jggheMldERMBhGvfdPLCB8975msf3z0+u/LAK153euV12w/8y6p+4au/+Esef/y3z+19KpRu2uwO0DppPvy+7eT8l+989Q/dlT+8PN6yXr3Lva+Zekdobkhl+9BXnn7pu+88Wp3eeemF1soAZ10H22L7VTDW8/WFQTSreM9JsqEz6C1F/MCf/SKAl6JIBYqGEADAOadWzCznjGhGxFgBOGYCUuf8OBZ2biwFwbnYTBc7db3nOLCLapa7fhyOJa0dYgEahjVCIVA0VBXRQgSolEoHYCUblGSmIXoiMuRSeiQgZEdutT4LIaparCciYmbMbMYAQkTjkG/e/OxkNkX2qnzaLh87u77NKSOUTElGROzHtBk2jW+yCmmZNRVR5X1IpY/okvYJcNmNIFo386Kr4BckffFxfbqOTVxMfcPTW+uTqiYZAVHYTTZpmLHbDKVt22nFjFUvyTlCECmcJbEjD6boDYqpOReckif/8a+6+rJ3XpGSRU0YZOzO7+6v0yCGHtyYhjFJM51IGUFjCN5MmZlQnQp6aDwbMCGVNE4aX4NVXJ9bnK99vT+bX5jddbI5NKMf/on/e1HJa175stXp0EoRNWi337X6O+8+/bm9oUyyFtBT43endI3QT8LBubv+Z/d9H37Vx0yK8/T6r/nav776r6Kfbbdt1ytzNCxVVSMhGBJGAQ3eLTdrjo6djO1owC5UgDCsW2bMVByjiqBh8LHrtp9+w41733GHSKmrsN1umDlW1ZgHB94H3w7bpmpQnUA/ShaF6B2YeY5guWs7H6I5zik3dQNqJWdjlVIYyTuXVUlFVYuBFhWVIoUdn9/dP10tm6Z2SKNkMgvOr7ebwYTRM5J3aAqjpLEkQvS+JnLyORkdd/1Qh4YVVUcEH30kEoC4mO9st5snvv7ZV/3F/abuQ3+xbcc7XvW6N3yoP39w9MiFOy6/+J79p6/+6tnqYxEWrlItVrbusQ+SR3/wd//j114+eeTRJ2p+8uzkt6vKSZLonEA+/s78Re++e7PNV/ZedGH3ouSh7bboxzG3TZzu+4Ojthv17Fx9p3NNVzZJTyfxPH7oT3/JQBx7KaTQhRDGcSQiQGT2qmZq5Jxjr2q3iWlVNYjkfEhj8j4A+zidAS+IvA8Nsy/DqutOUHsrxROnsRPNOYupACg7QgQiAnOqkksLmpzzUtTUDFA0e8+q5shtt2vniYjpeWamn2OqBREQXTGfZcgyZin9kD5147PH/fqZ1ZJSDlWdUpnOFmJScWjHQdNoFACdgIypc+S55oj+yu7F4/aGYdJSqwmibdNoufgqDH26sL932i5jqAMqknRDAY8M9aYdihQGJWBFc94xQi4yjCMhgZhn14/tYneWinjwO5P5+1/7qdf85d3dmEsW8m7qHaBv09gNbXQegAVZDUvKSMafgwbG4hgJvCOFTCW4mMZRS8kqzOC9AUjtbOoXq+7IW3P8ib09Z8TV4eDU1Buw6ff03/l/xd9QgGS8c+EFX/SaL3zzT/6jKXuaXrzv/vu/7NHXwA9cGMv6JfffCxxedunR07PTEQTFoVJVR+bpWXtysr3Va9/1nVkWG7vU3dqe5THFEI2QHGsyEVns7YR6agZD1zNxKv69r37kpX9+NxhUblbEhjyqCTGWYlnFRweWcytVxewDIWQZkRDIiQmYApihlVzMtKqqrm198GZWRBipYALTMeUxF8ioJoCgIPMqppxjjITIVRj7sTxPFcBQVRFhUtdZVQGD88OwAaDnATk/jGlMmdGNKTnyTMQO2DlQ9OQPv2N91x9d7jb9w+/em+zcN6anZm/4xVft3oiVzuXwk4+9xWgoJXvy7O3WZy9e+0Rjur789371nu17XnDXi5erd1777O8g4ryZF8uGev2bz17y4IXt0NXcLJr5fD7p2q0qpLGdVq7C+WyyF6Or47mDOj789EfZze+5eAX/8g9/1nlk8oR1lpGIzExViR0iAhAAmgkRqhoCk3Nm4HxkdgjofUgF6vlBiBNgXzcLMUpDO/RrtESouduk7gxJkILpaKpICABMDsEbmOggZUAkZge3GSDgbbmMjMzO5ZRyGRHMAMCsiIBZSgMRITBiCcEPQ0YMY0nXl0fXNyefOHxak7pQiUDRMp1OKhdHKcN2W1eVAGRJQxoW9fTWdjmdz1hMqAxpc/H8i67fvD7mMuTRK8TJZByLdxhnDQrkcRQQpOAcEJCo5pzb7baKlUpBoqISK49GVayGYSQDtdwNbVXVdZyIlE9++bXXfuDeAjlwGKRUiG2fDJW9Q9O+V2AGpMCxaBGRfuiICLXXpOLjrJ5xBC0a2COwmKaURYFdsNJG34xDqYYLcXV50x7NYqllHClCsQT0rd3/8I67/vLCxXN333Hn9mw9nccf+vF/5sDqCt/4gm/dbPP3/YcfWK3S0K/7Pr147/F1d3Ru58J8umPICSxYBiWHjhD7klHZjARAUR2xmo5pyNYzejMYU6KgCNB1LZgetse/ee973/jQF8wm89XyVhbb3d8d0nbZ6qjj6fJm1q6qzzmIsSJEbnNbLFFgcj6g/d7f/si5JyZFgLAhNmZKuQzDlhiZXQyNSM5aAMmzF8m5ZDMTVQGwoo5dKYUQiwoRmZmqiRRVZedi8Gbw/xEtqmZqxOBdncbRh1BUTEVUnXdmyo6kiIptH0izR13qpv3pficbGv/p4srvhVlzfvdc130U9LDrB7MREUJotsu4vLnrfCb+sXte/mA79poflnJGjAxOzET09N7N/DM1MjOCYwcIRQqi886lsSeKjkAlu8AR/fnf8Yv55QYR3/22n45h5jiqiWNCxFIKACCCmapKLpnInGMAJoxI6H0EJEQexjyZTOvJPoddKSv2NfkaKPgwMylSxuCp71Na35B0llW09IhExMH7cdzk0jMRAKOZqiCCmYIBAJkBkWUrjitEJ1IcqZkBQCkFDc1EVMFQVFJuAZTZKYZNt37suc8+dPOq5OhizUR9aoEMiiUp86aJ0ReRAtCNI6vOp9Oub4ExF0nZ6qkFP2/H/P+SBd/hnqdXYdjPOW/5tl+9bebOnb6jnS1quyqrXlcSoBJJFAH+A5yH0A0mMbZJ7ITYOJDHDhgsHEA4dkCPEAgJCYlVR6sCSCtppe27s3XqnZnbfuXb3nZOls0/8ZPPp6nrtXJiikHb7ZXFdK+eFZp9jC49S2ti0ui9s9Y6540xwXljrUtBKUBmBDLWBmDXtyhCoqw1pMyjd156/peOG0oCFAQwxT72XdPmWUZKQdQRorG2yKvoeubUu95ak9hnlDmGwhSdeK0AWYJnwRhDQiQkndssJqdIT9yJw7Cxt9dd2nXXG4AULLpbzhx79873fm7rs967wXSaDca7F554wSverPXorx7Zu+PbZy7//Nl/eHq+u7/NYPu6v/vh35215286dPT0+tax6WFCnWWVVnmeV0rh32NWigInQVSoEIkBkBMqBYAILAlFEoKkFIXxt7c+8ovbP0yiQAmiQUhNO2v9QttShAyST7HIBotmb9k0y37G4kMKxtrSlB88e/eb/uam3eXMRVcOdeKYGLRmlsBCWg+cNAf1ou76wg5CxBjScDRxPgYmCMxR9vdnMXW2KohIIQUW59um7wRUpjFyQhQUjohKWRE2RgfnCKDIiz56ABWiN5lGBSDK+w6Jnnrr7o2fXd29Mrn4yOF8dIJf9r6b/f3jEVZxMT94oip0zQFyjuFSwKu7lzavPb3qm3P+Vf/Lj91ezuNi+/xvxnAtSVitVuswDx6f/r7dzY+NyVLyvVZ6MJyEru1Dn2WFRm2MipJElEF46K2XT/3lIYUtgMZ7PvcfQvRIoBTFIESCSEoZEMXM1loRYE6AWpmcAVPolQIi2/V1mY+KcuiC15liAWsyQW2LyhRrhEpEKVJ9t9fXS+/mNisgtgBkSHvnumbHe6+UQiRAp7UFZoDYtQsAsDbjxL3vM6uJlIBGRKWUiHgfmFkkkQKtNKEOMQmAD8GFPnFqg9ve23l679rlZmFs1nWNNgZYA3BWkNJWkiiiul1olbnogSikxF3QlUGVs+9a3xulJMVBUbSdK23uAzfsJdHa6qBpa1WWvl+E6LXJmQk5GW2rahhCXNZ7wQdCbYyphsOuWXRt61Mi0muT1fPv3D/0MQEsg48siSUWdphSCiEQUVWa3kWbZSwBI8foY0xZZpQ2imzfO0DOtQaTd32whI7DoBp45zilosyjF231elqJuycO9pYgiXT60R/7SWWUIrzxi8ceuOPJmGLdLsfTQQwIwOuH1974hZf+6jn41Rvhp2+dv2ft8Ws7102kz5//+O7sGwzaZgZTb5LOFAGmtWpwYuXwtJrkWdk0zbAsNibHjqwcAyFRgoDMTIBJmEgJCD4LUEB+5/Cf/sKVH5RnAQIAIhJR4gjPQUQQBACtdYwR0RASMwNA5P79mx/7hWvvBc6S2Obgidk8MtRbm1sgRTDY94sQ0Wjyfdd08z2/9F1XDYpls7y+3I0BDWUq4fW97Z3lvBYHCo+vby5di1AYo2KKSuKiXrYIJBDYM+jIgZiNwsDJs2RGNd7lea4TsLYiUSt48I3PvOjuFz9yn2rqO45t0YXB7YN7fjN0B8vlLHCWGWsHa2/+3re99a2vb9r2W/d967Ofu+f65SvxZf/ozpPNzbfc8NC5f1UvrhUWkRQIJ5Yn37538q9WDSIzMJAyJvkYU4oxWmt9jJnSo+HAR//YW6+e/vSGCMfI+I3P/PsUEwMoawxlIfYxxDwvfeiVUiKitUqJtbGIStscGEN0mgpSokxprFku95hDXgwzWyAZVArMiMgYU2qdMQcC6bt9H3zsFiIx0xh8F5wLwSMxcyJSzKltFkZjSlGYjdGIkFgTitHWZEWMXimVUgIQAIgxaZUpyn0/15YYAlJqm9R1XQRufH9+9/IjV6/OG2+HQySPZBlYOHR9QIa16TRy0IRIJMikAJJpQrdceKsQKfVdz4DMQoqic6vT1brvo+vYMAOMipGPjfOOtFFkA0f/rOCstcPBxGZZTH+PMLi+Q0Tnk9GlVuncW64973NHUowhRERwviUUfo4xZlCUAKooSh97FEwpOdd577MsB1DGWAFOzrHREViBGDYp+SK3ItwlTjGYPNuSm/Nm/WCxXCzD2ZtvW5kWZ87e+D8/cfuP3L/+qzfCvzj+tRvU9sHe7m3y/NMHN3zjav5wuwIAv3oj3L7W/tHbDi5cuH7l8pUzzzv+6x/+b40VUqoslO9j13LS0rbNuBqHvmEh5/2wzCd5PHvkxqOTI4U2pw6dnU4mmEArkwglsUJBEUHzO0c+/POXfwBQQJCI4DkxRREBEaRnIQCEEBCRCJRSKSXvPRG8f+uj/90zb5+MDn3t65/f2aPB9JhSobl+4dabt06dOSteC4hgDSnvoZNoMm2IMDB3Td33sRoMrFLR9cuu7WP0ISL489efWsa+T21MBAZ99Jkyg+FKCF2zbJTiJghL6mMIkRmkSwEU+q4PEpvWM6bz79wbfeB03H3xeMU9s/K20YMflNAbrUyWd3UfXDfdXO079bxbNt/9jjfsXn/iS1+4d7Ez+/bqj7xqI42Pu93zv5dZJJK67Ys8R8SH33T+zGc3MBEQhpQEBAWKoljWSyLyPgqzMbosy6e+d+fGz212Xe99xM/+xa8Ns0GeVT4ySFIKRMTaPKXog2uaZVWVRue9bxDR2hIQtEJhDQiMYrUhzBCNQNCmBMCUvC6GWpe2HCmVh1gzG0Kyuohhtpxf7+s9TZxS1Frt7++MRkMC7UPHyXMKNhsJJ5CkiBiFY9TGJEYRQQTmKMIKs97PEFPfhbq5MBoeKvLVFI0ySRL3ridr5nW30y4OmsWF2VXFyhFc2d1pm9pFWp2O8yzzIWSWkTSnlOU6sQCx0aUi8b5z3i+admV1lRyG0AMgs2JxEUVrbcmIwN5yluV5afKDfhmjF+GiyFxwxlgX/GKxUKhspgeDilA/C1U69+adGz67IRyDDyEGIslUFUKIMTZNUw4K14fVlRWtSYRiDM455igsAKbzPkZ3fHpstzloXaeJsrIklRiScwHFDIqCRU7w64uUtrcXNz3/heU4K+3wojn5f+y+6VfPwf/rjvYLuTXXj88vHbpytXjev919Mzzn9157+Y5Nzwm+891v33D0zL/84A8XZTYcDQVc03lMCIASeVSUy3aZFWXbtdPxqAtsFEXfKQVTa7emR15+4+1b062hHQBI7x1YZZD+49GP/dzlHwBABEBEABARAEBEAGBmpQgAfAhGa2ZARHkOJ/mPJ//sf9j9yd/+nd8d5fnG1oYdj568qijQscPN9sXla1970+GNiahhvHpfeeglLI4kPUvbwoXFbLGsBkOFQqCBQWuTWFg4hpa98iFe3Hvs4YtPH7imHJaGrKFkSJeFwsissAtBhFxMIYUgKaXQRdYmX9TNN155Uf2Pm8uuJZD9W37m0GN/SEjOJ2UzJIKUFZOszNQNN9z45GNfe/T+B1bHa8OV4TNH3jd6+GM//m/+3T3f/OUYHAKgJqO0tfaRNz5z+q51EjR51gdHSoVetFYiQqR833euT4nH4/ET33P5pi9scRLvGX/3//zRw6P1abWqdJGbAkkAAEEbk3nvlEJAAdGJWxEmsoBKJGlVCrAi5WMvCEU51JTpbMCJOXSBYzlcMfkYVWFNBmR7vyyMDp5SaHy3n3ztuoMQAhFZm7muJcUpRmQCq1BYggMQ0ZqDRyRlMmZAFKUwRM8hhtjEyNaMNBkkYUyAolBDSszigkOICUzv4r5vNKdHLj1zZXEwX9Z1XK5Mx4u2djGVRaWoiIGVgsSCGKtiEGInWnV9ZzOTZTYncqFPrJgzdi65XjhRoauibEKLhOxS6wOzMDMRphCq0WBez1mYIO98XZR2UBSECrV54m27z/v8RBhns4UxmkiQk7AAgta6Zwk+VkVpjeZEIfgYg9Y6OI+om64ThIEtEqXISSUKjEARSTnPnHhgMhtvev7keRKX+wfLLBuOxmtZWQ0Gxf8UfxGec6f6uzfjN3a3r5tKX81vyrLytmnzjDn58kN+OB6vrK5E7+7/zr073fbfPflnbewWs4XSsRxMBkr3LiYR1DTKsy6EkGJV5J1vCBVETIFNMfJdPbA0rLKj043jK5urxSgzeTkYfvjmr/zjKz9sVC7IiAgAzKy1AQB5DiLBc0QEkFOMLMLMRPL+rU++9asv/M1/+yff98pTW2dP7fXdtXp9b744ua7LPB9U9q/uPveu129ePPfkG1/z4pSvKooiKa+GBMrYilCRMGNSSN53XXBJOPoFgi2Hw4P9g5Sg9/2im7du2ffLYTmIoV8EnzQ03mfZwOiMY2i6mghmvg+hNXr4ofXvjP5VXncGQzN/1S9V97y/GAyVNgKyOj36mtff/OlP33vDicnTj98XXN017HxvrZ2/9J+eevr37WhoX9gURldlFlMoM1NkxYNvvHj2M5sptaQ1CwJpEjHG1E2dZ7kl3UXvXFREz7xrf/OjwywrUkT8vz7wUxjTofXReHBSUqdsJoyZUaTL4LygE1DRL63NQwhGa2SFBrUeAIlEECQWyGyByKiVoqLt5kAynmzlxSqj0roA0iG0BvLO7wpz6JaKYx9bTlETCEdO5GOTWQMMKToRAvYh1iCKISIggEKUZ8FzEBInYokEVoitNSklAEgcu64ZDaecjAgzMCjtI7f9bD5bzurl3nzvgttDa/YPOtf1WVUwgDVWEkxHGsRn2WRvvlMLxxiIKEWx7FdWVnvn9+pFxir0vhqWDt24LNBo7lPTO9CEkKUQgUJKwYCt8lIZmjvf96133aH1tYCRPT/xvTtHPznK8tFif9bF3hA67/IqTyFOBqMUIaEkjlVZdH2PiITWyaluAAAgAElEQVTKd64LCRGYRYEpLIoYIOhcrYiQNJFxbfCp1aY47d86XVPtosO+Wzl+em39kDL5oKy+8Mj8a/q1h5bffdHFPxiubqyfeP5j7WGGeFOxU1XT0cp0OBoOx5PReLy6svbNL3/5wflXLi7vq7u2b8SYLMtRY57YLZYLBrA6CogwI6JLQaGxKlMIgNb5DoRArMZsOML1ajA1+vmnXvyFVzz4S9s/CsKEAJhijMGHsipANAixJBdqa0pEQkTgBABEShgT4n85+9HzP7O7fGrxhpetzuzW+tbg0lWYNW2RZS/cMmLj9h645bWL9fhn3/fG79z9iSMr9qZXvktAur7Ncr1YHgCkNrnow7RaIcFBtYrCznVCAMIpBK0Nkkni6mWf2C2ba/ddedz1XiAl9pRriCQJV9ZWnG/3a8qV/m376dj2xo66V/6i/r+/qs58AwiZxVqLCDbLmBNIOHlodTbbO5j70di6w6879Mj4Ef9HVTEsTkI+LBSw1jlLBJDDF1c3npmIC957ACyLqgudVppQud6Ni0HdNYyQJJ57y9Uzd20AAYDgv/j1O4+ur07MxmSUDbNNm5UiCJIQDEIkbRgIhRCRSIlg3+6ioqKYIpEAaq1QwBp7sNgbjqZZviYKOUQWyQpb1/Nqcqgs1/quVoQKTOCYQuP7BXCvNYW+SzEwSwzOZloSO98abZzrRILzLYAAUGZyREFE53pSBMAiFGOndZaYrSkJM6VyZhdjF5PPbM7sU/QiQKhJ0ryf7fv+qav7Owe1J98yLtxi3RgvHEVQhDRmmXURXPKTfMXkqu06Fli0i6oonY+t6zPg2jtBXhkNEkOEZIHAUFN7rZFTMiaftTNISACj8dDVtTE2iiBhVQxS5IfecumGT68RqmaxUIXVSL3rfYwsorVShKQppggA83ljtDZGAUhkIQCtNYEWgcyWs/kcSaq89DEorcssb7u5lcmJ9OpBqZazXXaxSzEKveY1rz/7/BdNVjZ+69v26AP/+erFh1M4eOTkj5/C7VuKeUgyGg0G09XhaDyaTicrK0VpLz7xzJ9+89/t++sSQl6O2r4xOQzy4WK+yG0FoIyCvu+VUog4r/fLfJjZLIQgAsEzElhLWo17v5QEStLptfHj79z7X7tfHmdWK0tgU3IxNbPlvtZ2PJqmFF3XlsWABRFEWIXoffAhBpVln3jRV77+vd8Us/rCzfTKV938nadm5/ezoyeOjIf61JgefvqcD8NTG3LPuW5ttezj5Mfe9brJ1KUEyQcR8d6HEJzErlvG1KXktclza713QFTYaWF0bjOtrQQQ8M71CKoJPQFZTYvZ3uPXn7yyvwfE4yILYjtM9z509MlvfeeSrHfv+wsA0Oe/WvzJe0DrvKgYAJJUozzFvsyL206UO7N27tPB1eu7L/rnd+pPfveBJ69e629/101rt3TNEjj4vMi9c4CgtSp0FkJQyiilABgB+r7LMgOgtSYACC7c/6aLN376sCALML77H73k7AlzcnJic3wih7wqBkSECjvXIfaDfIMFm3aW5UYpLax87/I8U7pMKSJBlllhARGbTQQiKTtvdsui1DojzAFMNhqi5KQQUMXYZVmFqKJznGrXdQTMMTB3iBCCkxQAyLveWEukASmlJMwiCViUQkABkOB7YYqps6ZQphRJSECEQVhYEI1WJnFM0RmtOUrT7BtVJIFZvdhfbkfor9WLvTjPM3ZB1W0Ulsb74KTtGlsMRuWYNITgtTE+RRbe25+NxyPfLRMp592wyGxW9dEp4QRJosQUFFGeFS46BLJaI0ff98rayMIA65Np5/29r3nqtr87YSEXDgvfYyIgrorhfLlgjAo0CIQYTWZDkBgjIoQYAClFn+d51/QAMcvL+XxubS7c28yGGKuiQBzB9ituOGyJu6YOIvnRI1NG3bT9ydMnT586/bG9m969cfHRBx7486tHb9v5WIjz6fRINZigwsF4OhqPq9E4y/PBID9Y7PznL//a0rdGkhnkbV9PxiMJOWAMvs2MDTFmWV7XbZYVKcSiKBFQJJX5CBEAk0DgKJ3rglfG0GLh9n6ovvlTg8OTye2nX3rDsbMr1bpVJoTWOwcAxmS9OzA6U0oxp8SYJAlK07WTYfXhs3d/9wcemPQNUnH7rasPNtm8G0ymuVXq+15/9OOffbDI9G03rjx6ed7yxrVF944b4sZNLzqxtYHARDpGVspw6JwPxlrXuyRslE4puhA6XkbX5drkeVVmVUxtCMHossiHIkmiF+bg0169COzmO9udWnzgz91sd3d3p/U/+Ifd+u3wnPJP35Nd+Zb3Qdu8yIi510YTZVVVDPzBwWKx2Lqze+UvbyzvW7n/A489dX85PvSiN9xiT+4zkGGMMQIgc8qs6bqeSLneISarjTEaJEUgjn1mMteFC+9d3vL5LRccS8R3/8IrjlTVTYdHZw6fzfNRboosy130566c09CcPf4yrQwnDRCIFIJl9CxeYwaQ+n6ZF6WgItIpgjUEop1bCJCxmcqGxgxVXhJw7/v1jTMxgaAoTF09B2SOUTh612tKhCqGPiWnAX1slTKAmcnz5EOKXdfNEBSgpBREUuKgVY4khBYBfPCICAJGK0SZzffGk4GxqyH0KYUYUvAth2Xn3Ora1rLpUdzc9w9eePix/b28HDbNjLkVLChlihIn24RmPKr6rh5VQ0YJKLO61qRDjDlZRYpynaJnhNJYl5JWkpnB/t7eeFo2rtXWBucpha71JsuqQbVcLnMquhQefduVU3dNSp1nliCzvpdFvatYKaWVARIxZJTSXYqaMMTEIoklK0fBd4iiSEP0vXOAWkD5vo/CZVlJTHH31Xkbz96wBqlRVF65XleqbVza2lybHtp4/oteclfzkh8/c+0PHt24o/3SYw/fB/2SkYejqbEDm5lqMMjLam1j4/KlR2u+8qmHPgQmrZSHmtQ61w+zilAzp2Xd5MVQIFibLeaNtRlKIlLGGJE0KNdAtV3X5Nk4ivN9GFWry3rGKZx/5+7xT25FFzLLk9LcsLF1ZuvGsZmQEkQk1NNibE3OzD70SvGiblyINstyUR956d39r/C9X/3ObS/Yet9bTv+bP3mCaTIYpNe+9rYzR/AzX7mQm3DmxMrf3XP96qLePHJqQ/fT6vLb3vSWweGTIsAMRASkECGFHlAkEgH74ITYqEqhAAhq5VyfmJfLeVlm43wsmFLywoAaMSEJXt+79rV7n/nD//QFrxEC9K/+J+1Lfg6es/77N/umBRbUxGnPd2F9fZMJmeHQ2qFy9dTfvuFD8JzJQx88ceXTR45evO9re2/44ROqrOvk28YR6RhEaW6apihK55yxREgcRSstHAWCMCDYi+85uPnzR5zvtSH8/p97TaVxOFBnDq+98OjtSudMrq7nhdFlvmKtDoE1JSCNRCJABIQmJQ7B2bywtiIFwgoRlUJhqqoRq0xEnPdZkSOQzgzpVVSoFDITEYQIWsD7hYgEX3NoQt8QcEwJIQgDc4rcW13EGOU5WmvhyBxFkggICIGkFGLyMXijjVJZ37VCfZWd8jLL8xyEUkoh9IR50y2YEcEZLUqt7RxcnC33zx/MHfFeM2MQUmbu+lxZ57vkEmbkY4Fqfng8bTy0YV6VY98caD0haWLSUcWBsWJ052KULs/KpmmKMm+db1uPKERBgy6y0ktSKWllCPDBOy/ffPcxYY7e2zyPwghqMa9jdEongxSZQ0oxxqKolCEQSX3aWj+zM7/ep8ZHNy2Hrg/j8XA239M0nnV1nrHaeUu/1wzLdOzEZnfQrw4Xb3rvT3zkTz+JMVSj8e75J8pBdvmFP33y+MmfvvWAUV3avv7QPfce7G8bBStrh7u+0RZH5WC4dmjn8b/98vzze6EziabTSoHunOtDa3Pruyhocsxq146Gg65tzLOsBuCm7eq6LVQhhDFJUVXIsW3bwbD0PkSIl96xe+qTk9Z7gKx3LjMmM5Ywra+uZCrL7WBUUNcshuWQoxLux8MJe1mZrq1l1X+56Quv/cwLPvSBu//795wZrtIffynJdHzl3FPv/f5XFoYfON/tX3nijhccve/p/L7v3ts3DevsxaeO/NRPvH5z68aZX45IBRkwNAQIiBGksoOm2el9s7dcrA6mRVZZY+t+Oav3iqwyqHNb6DxXggbpWTEh+uZqcn/4+5/crcNsx/kop04NvvrF+9Mb/mk4+sri6//e7txf5IPeN6dv2Lr/G3+jiQDh0JGtmDhRNrj1zsfv+A/wHH3hq9OPfs/41DGZ46Eb1zZf3HlRCKSU6t0yBHkWJzHGEAApca5LCRBRg6Ys0wDPvP36rXcf77oOhPEdP/uyknE8LG49dfTs5nEja0XpczsxtgLBtt+3VgGbECCkJsGyUKs2K2IMPnbloFQwTskze6NzUphSQiBEBiRjcgEUJJNZkw364BHscLjS+16pXHTK9SD2sW2uQEqunRktIfqUHIIolQkbxCAiLKKIQmi860CYEACYhQgNKSOhSxBTSkopm5UASjD54Ii09wGEu742pJXVMQDH3maVIkzJ9X26Pr/qfLq8nD852/HYDIvhWGUJ4kHrfUo+2CTN4ZXNunUMTjgpLa0j5KUtpt7tjYz1zINquHCtVrn3wRg1Xy6UsUSCxMpYAmqDpxiYJLP23lc/c9vfnoKorDEhBkEcD8u6dZG5aZchyOxgXyuNQCl41GplOgm9y0wqqsnG+rELT11G49vOodLjlUnfd22zNPPXuT1J0U0HamNztTloN8bh8Hr56nf9CGLZz68fOr753dnmr39Ffu3sdxWBJjWsBrM6Pf7IA9e2L2ulxqMKEkcLK1X59NUv3j9/JIAujUrIwiQC8+XSZpqERKtJOZy1TVs3RmvvHSTRVg0GVYiRSAcfqrLs2t5mlXN9WRaLxTwf5JffuXv4I5W2mTGFAM3255p03dUxhbKqxuOJ8zWnMKiq5azNVC7ASJw4NLG58n392c+uPPHx67efzI8eHT/eTaDLbnrx8zOzT8Dbs1ynxaFR/ZV7uwsXLx49dvplr3jt6fBYbevdvHzxLcfygBf7i/WyUVpd3dlpg9tcPbx7sFe7sL07X5kWo3JIipKka4t9C7qkfFwMTx45VOZFislFv7W62tX69z78mQGsXntq++qiP/qfNh9duS//jWH6QlImEyamqGm0uj7Ny3Dp8WdSAmUzVLSyMomuCQH33vPncuJVADC96+fN+S9m8SqzWr31xOlX2KZpjbFZbnrX51YLUmRGxBQSIIBISolBICTKjFX4xPdcu/WLx+aLJSfG7/mJlyRLY5vec+sLtjZOazXcW16KWk1snmeTEPxgMEVIiOry9jOR6/F0q25mrZ+5sLjpxKsLsxZCS5hSosQhz3MinTgprVEZETCoTZYLcowyXj+OlDMiQI6Qdf3FTKHrvO93o+sN6sSIwDE1SBJjAglExM8STiEoghg8gPT1PCVnDMfotC20GSAWRTmK0SfsgMurO0+mVK+sHKnylRBFIyARQsap90kQGo6S6bJJwffu0uL6Pc88MLRlJFU3zczXENNoMFWmnC/no/EgoXSd59QjQe8h1266enS+mFGKUZJrHVDiZBSRNkA208a40Pehz43t295LrIz2DKTo0Tdfue3uE8LkUxSQsihjbJkhASqti3xY1zPv+hREEH0IZZlliiKkZdNMxtOuboOLoKhzPoRoMp6k1/fXBpRqrSlTvHnsyLWLVxFMVcahXt74sjsPFu6vH756fGPtKXvrG+JnBuOVyfrWaHqoGFRd2+9sX2kW83yQhUVTGg+8/7W9r/QaFSoUqV3vfbA2U2BZgnNuvDaxSFd2dlzXa6X5WVHyXCujiahzvTXWah1jtFkOIEqTdw7JXnjH9ROfXCvLodESkjgflrNlcF5pLSh5WSTf912bZZmIOPBKGSLjXCxzs/3ugyOfOmJ38NZWLlzZuZxtGZOdOX3i1BHp02B7N0yrOIALn/m7g3OPX/yrT/xFaLd/6/3vzydHX3PL4C+vfWOqcpeSNbnjmEBiSlpZhEQSB1nB1viuQ6LI3C/qohru7h8IqjwnZN07R1ayLKso395ZvODYytc+fX75i3D9J7fhOeorir6sj/z+ifXN4twjlw8dXi8qfvyhS0QZGQMQsyzLLbV986E/++vf+uaFy3d9pJ1d0sXYL89fO//Qy+581dotPYNxriuKPKUEEm2W+5SUUsvlgkhV1aCulyklQyoR5EY9dueV5//18eBj3dT4jp8+OypWyKhhRrdsrh9fPzzQR11aGpLxcEMEtSq86xi8NpmiglMU0DYfJAGBVqNBiEZljMLMSmkim2IfQmBhYzNO0ZihsmzNgFFYrC1KZUY2K1KMZTF2fd01e6FfIHjnOpCU58O+c4m9goiIikhEYgoAHIOL0aPOJXni4F3DTGU1SgCRg1YIXICKXRtEGmtLozNEEEnKKO+EMEjCrj+wdszSA5C1g735/NLuZUXpyeW+C2GxOOgAfceDYdm6kBOjVbv7vc2wicF7Wakgs1MfFsF3dlgoUcu2K4tx0y6znKfFIYa0bJdAWNc1sGCmMmQmBMTH3nzl5k+tsclNnrngc2O7tiEkm1kUVKiS9Maa2byOAD6kGBxKUGiysmiaxhqTkkLFIfiqGkl9tr+yrkOtMVoNkPyJG07vXds9NOm52bdV2XS1/qE/PuavPPDxP3pg+upX13dlhWVJRVGtHDuxunoEQM33dkLobanbC/d/132ztVpQXDtXujC6COxd1+c6R5IQAmY69R4VKaWstilyii4KC0vvY55bES7LMkSnlV0ul2VVhBBiSlfeMz/+l6skNBkNXYp932fWdl3vHRdFGUIAYef7xMla3XSJYyBga8hkg/Pv3L7lM89bpsXisW7UDWZNPLx+aO3Q+sZox05eun1wUCFsDnc//sWdd7/3h4XPf+0bj7/0eHFp6awKo+PXz7koDZVlLpqapjVKkSQgnedZ6v0i+iq3RLQ3W3ByVTFYLFqOgkZyM+CYyHAb44rNlmyyrH76S/sHn2iXN3bw/1H8Rgn/Eo2xVVm2/bx3TMoiMKSW0Z699bU/9A/ee9ddH9blie3ty65jDn5lc3rxO58//LzTa3eoMs+yTLdtk5IQkdIaBZnFZOz6VBSlc33veoM2IRilnnnHtZs+u8kxoSb88V95S+MaQqWYNibqlqNrp1duNSanxDbPtCXgnMUjGkIlEIWRJTEHkRRTrMoqxpTnOaFVKhMBkZSEmSMAgihtwOohgLZWeVBICAgCYvOJtZXNR6SL4EPfXk+ua5e7CN5o412rtCBSdAIYEiAKIkqMXgSib5xry7IKIVEKO3uPbRw+HqUyhEgkSCyiAZA4xgSiFvVVayqtMmbWOhMBfpZ4gKhMAaj39veXbnHQLQLj9ry9vjhYtPPVYSEkycdIGVGx6HvW4vplVRYgkKu86WufgjX5stkpBmNhIQTjIxs8tL61fXAVhGaL/UEx2FzZausDIf/Amy6f/czRZdeJ7qtyLbPUzFokICuZHTX1wphcESoOHqi0ZYh164MPPYHq2ta5hnSW5croKsPjyyfOQlhSaozG3CDH5tiNNz/11IHrXJYT+fnlG979+vZjN7/89S97/dv/4JH1O82DX7/n2+fPPbxqk824GGWTjWNHDp9YNH2m3Zcf/qNLstCkJ9Nh17cpynRc1m1b1w0J5UWeOBmTJZ8SMnCqymKxXAhziCKAQrIxGXc+MYNCtJlumz6EmBJX1eCZd1w/9omp1kpr1frQd12V54jEjNZakQQMLgZOCQViSsGlLIeQxGL2zHuub31sHPtQVRa/vBdWTzvu1od5uf6Cvb3Hz95yx/Xt6696Ufadcwc+4hP3Pf7C1736mFb79eNX2+qG0wff3q/VIFSmWiyXmsx0NG1d69sGAUISUZJSym1GSDH4wDFFPRhVkqQPntBQbJa1ngxUylRF9Mx9O+0/6869bRv+a4N8LEh5CcvZwmCB4NHYwFaYvv8fvPOzd921Ohxlo7V6vuz7kARUhiU5MsPxi9P6hhmNR23XJeY+eSXILC6GMjeMFGPMjfIeRLjIc6vtd1937vSnD8UYrbX4I//kDYvlcjAYW1NKrM8cGp5eOT4ZVON8pKhQipVVnIhIAYjWCv4epsSIyAwAjAjMDJCUsgiaSAFRSoEIlNI+9JmtiBRS8r0YY0lZRKXKDAkZMM/zsjxSL/cV6badmeh9WHKKwFqMjy4yuywf9v3M6CGhAvQxQNMurTVKmeBa72oURRo4JW20UgggIibLbAhBKQ1AhBoAYgwskVmYkzEaSIcgREordM53MbTtwbX66gOXzu+4ZfJhUORGl8vElDBHDVovlvOiqhZ1M+9nRZUnDoXNC7K7swNAGI2qauXYk08/VCmNKmqt6nqR23JjdfPafFeDPnfn9iu+fuba/MD7eOrYGYxhp94l0kZj7NVBvb+6tha975umC2EymFiDAJoRl/XSFCqEPvSeY1odntx5/KXo9yj1SpwxaFTSrI+eOtHV88b5rsNzx9699fhdortbjg8OT/Nvrv/oz9w6O3H6RlWM7v3uo5cfe/Jb93xz/9Lf3PLCmzeP37TXXf7i9oetXqOEgKyUyvOiGpQuhizLfduTFh8wxmQULOt+ZTyaz/eVpoSp71JiyDJb5YTK9i4E54zGFIETap251O3+YLv50UqTQrSkdJbpGHoBndmchUPwo+G4bpqUoiK1vz8n1F2/YMYsy698/96Rv1ghwKIY3v/xR+oFnDpy9g1ncS9b3b16rRM7HY3Xpot5e3rZX7m+PT91dFNZPaDkJSu2zNX0mBoNJHrnQpbli9k8z7TSVDf1YDSR0COgJFZKFdW0bWtEWxQZJmlCX5aj8cBu7yyIG8Ojyw9ea+aDnQ89Obt1Af8/+f9WwN0Jv6gItTGQBAQyIUmQDh0auTqO1yazg1lbdzbPTD646WVr17f3jt4wGm5ZQYnMISWraTgYcpKD2YxQkFS9rDNrBS2SBHbGqMfevH3604eqqvLe4w/90kuY2WZlYvK+n5Tq8LjaGA2Glo5Mz4yySVGMGVJKEZEAMMaQ54UIhhCLfIAIzvfeO60ppVhVQwAFiEqp+Xw2HFaE485dJ6Kq2GAMMbIyudF5PpxqVQhlRT7qun1jLILuuoN6/1LdXA2xVsqopIX7tt4fjw4nw1Wx6lxATIrKxE6Ey7Lsu0YAkosALaJiTk1TW2uUyZlFa01EKcSYvDHGe+98ByBEpJU1mSmLUYoxcWxdFxOF3l9b7OzOrzqbcUzXdq51ohp2kyIvLKEWHyOjWiydSDBZ2fc9CfQpmbJklL5vBqCauCTJl95pGKTUDyqbOFaD0rd8/r37+QeTBSyL4XQwsJRfOjjPAUaDQW6GAJ6M8j6kPjbgNGXGyNpkHSLsznYYUx9c8Ly5cur8d7dSF9j3WlhB1FoyTMqGw6fOnH9qPw2Pn6/OrN73e5tHVsOcdw/2ytJefeFPvWr+wfGoJG2q4UrUw2uXH33ZHW//xEc+/PKXv+Sc/8aji2ulJlQIgiFEY3RMkRGqatAua20EVaa1VSgiajIaLBZzUrTs5kYXgBSCLzIjCQUhSkrBA6gUxZqs9/HiO68f+4vpoKw61yNAWeSgeLnwxmrv/WQ81YZCCAiQmFPkshju7m4rXWhD5+68cOJTGyxRVLz20cXrXnz8j39UA8C3L/f/+IujZeMit10fVkYrs6uXB2vjTIzXsGXETydb1NNt02eW14Sxa70x1sc+V4UgRk6j4bBv5t6FMi+bpiWtEBOCVYbH1ViQBXVKvVKk82xxTj30pYt1d7W+o5XPMwDkf2P0V1T9Kz08R39V63+N/BpCguo3ch+CoBaByN3GxrHBMNvdmynA5EOCpHI8+4qT/eLaqZdP5k0cTyZd30VmLaiUjjFppfOchCXG1Hbep96YvOs5Rrj2gwebHx0jovcef+yfvUJSVNY2PkgAMjAc4KQsVwrYyFc2R4fWpyddiPT3jFaFSAIArbWIKKURSQRAUKkcIIgwkQ7JK2Uzm4XotdaIFoF8aIlIa+tjQqQ8H5jcMhAgZtUmApHQsr6sSHGizOSIGLrd/d0LIcysHimrAAUR82wgiULoSAEzu94roxTZ4BaMIiwKVfABtM/sgBm99yheJCKqlEArDZiU0iGA63eHw5WQxGQZ++Bil0JqIl/aOV8WE+/ctXb36kHduznIsjyccUyXtreLfGx0NSgKn1SI2PduOCgYFRPO5/NRXi26vY2VQ9d291ruXOOrahIFGHy3WOy+z53+zOEYaG9/Z2Wac5cnaI5snrh+9crRrRPXD7ana1MWqGetsuASG8Pj4bhpWh86pVWIEmNKV9+0uLZEHwlbJUCSiGKuBYGOnjp2RZ3aqTE/ePqNr7kx7y8+fP+9F67Mfcyeft4/PPvoB0yhtYbgXDWyh4/f+vB9X7/59GRyw+q3+ocpgichsctlZ03mQ39obbXtOu89CmqtWr/QylT5VLQQg1a66x1IsFlOmg4W+yujtdD7rMjrri6qgfeurmtEBKZL79zd/PhEKQKSXOfMKUqSaFLyZVlqnSnL0XlmdjGUeRY9N+1iOFoFiI+98dLzPndk0cziE+rSQ9c//Qs3vfQowXP+9f0n//Jrl06spguzer7fzq7NTj3/1IU9V0C87agSY1fWBrvF7nVn9AjqRV+UFRhp5w2ZjADWpuODxUHbtIRaGPPClqWpygFB6D2PhoXrQtstSzOAqrx0z/bs6WVi7UGmZC7feG3wLZo33PQdPKf834v0LwJwUZTY//NOKW3+1vLno4BKnFbX1713CiR5pw2VOR6+/VQ93zl8x+G+bl3fF1meYsxzQK0BECJHAIUAojofXVimkAh18PHCf3Nw8lOHUkoigr/7ez977+WLu/tXCNgrOLN26lJ3fUjN2opa0YMTa8+7Yf2kUmPnOks6FN0AACAASURBVMQ+txPhBKgSp5iCAsyyAoiYGYi0yupmNhxMiJTWtq5rYyCzYwEJ0YkIkdJa87MSsASTla3nyXQ9L9eUsc4ttEDfdxzqxfxaWVqlBhy9cEQQIe29z60VFiLFHFgic1BoEgdEYQbvO0SLihFtkqhRedel0Jo8I1TG5jFwCo1SOsaotMQg1pq2a0TYWtu0rdJZDMKQXLOIKHVMe80VydrHLj2yUy9XB5UdjKJgSIsijbqUhV5EDaYGZn1UCR+/+vTRo1vehyKvXOuSjppMihhFjEFL8O3XXbj5c0enZd555fySIy/9QhdZdHyoGl7cvb61eeryzmOsrEazUYwb30IGSpkC8kXwucZm56Z2eyLugMSDAEkiShoBMZBQ/oK3Y1ye7B9c1MGzO7r1go2TN/0/HMEHtK3nWRjo9/3qX/feZ59+7j23X+mqWZIlGffesWVcseNhDQGGgQxlwiQwCQGHEkhIZq0ZIGsWTIAheAwEG2MLFxlbtoULLpItLF1Jt+j2c0/d9W9ffUfx82weWcsy8ZGtwz+0dm1/b/6tr39lunttMXOm2dso1eWF+YXZtbxMTNdxlJ33EjEE6Fzs55lOdGcNIljvOVCq0qbulBY6SYUUs3quQCWpiBSkSBs7gyg4Ms5ZiC6wCFZUbZXr5MJbtk8+vI4BDP13MXomAAIs9fvIcdq2pSgaW7ngg3NcCwigpcoS1Vn77Bv3Tn/hiDGz0Rf2b152/+oHF3/hDX34vn/59Ivf9MbX/e+/8u92tq4M+kN38fq7N9ZPDdPf39qtyaki3Ti9OjnSZqA9tZ0nxilVhWnrrMit9TEwgkA+hkAoOIIveso6meXKN0YqZkwVnRIJ8hinI//sQ7tv/Jd3PP6xa35ifFPPqxhfDeHVwb3CiUdF8TvFdDZDYgA805KJ6F7p7UsdAOKHuNaJ+RVrX+my31Yrj2VN58/8k6PXv3bz+Os35uMGgRV5MZtOQAskjM4pJerOS+W1yhnjwbnZZLKwsOBCePZ1N45/esXaDhnDn/uN9ypWN7UeuZ22rc/01793sLVUlMfX0xP9Q1miVofHEpk3TcPQJapEIWJEIUWkoJUynfHeC8EwBACGzCU6MyFGAs4E5zxEYowHHxkTPlSMacFSqbR1XZqVziOhkAnv9VeZUByLqt7tOptoGbzB0Jqu9d4IwZECEAUfGMMIzlobY+CcM9TWtoyjFCoEQ5ETeiDR2b08LUzbcYYBU4aoOAeigITIYwyMQXCmrpuyLIGwaedJqkIE6yICJ98hB+Npb3Ywq7e2x9dAA09k3VLEOoIPBi2UGDFEKJPhvDNEMaBHkAyZ1rqumiCMQFlk/fGkMuSaqjn7phuv/8ato64JwSYiYRSWitVzo10dm9ZFpsx8DkyklsboVV/lusyQM2vb25bW5xO5MxNbl5ZD4xhUiJ5jYESMIoOIyJ87+u7be5PixjfXh1S1bYRi8VB58/pFgYN+f+UR9vK7nvu9pdXyngdefPeL3ri7685v7/71V/6d5fuLvTVPTiXqYDyq2zZPUgTuIgVnh4NBcN503bSuNg6tz+dzYwNFy5kqy37btkJiWRZEpFVWV3tcKmNdiIERBwGDdHAw23PWX3zL3pG/XV5dWp7UMykUeed91+sNkcA6W7fd8nCx7RpAdNY73yVJ3hmTF1m0/vybt488tIiRx5vu+rd22kn8+bctveaIvtot/IvPVylPdm5eWFBJtrLY395+yQtOXzv7vfNW1Bk/sZLinYvbsKd1qpggJgmdd6QTnUhpO8eYbExD0Wd5aozxPhIBB5nlwqKIvmMY2pox5NHMXcfPf2r7nh/deOazu+vl4sHl2fRFlY+kv5YgA8HZ8+q6CTEwRECmZYIInKEPoftlY19u6dURvm9zub9x70rcoAufH9/xjvX5tFFSO2OFQtsFpTQAKc0JqZo3RDzNGIvKQdRasuAv/+D+oU8uIVKWpPgzv/b2renNoe63xjEeIucDzMdAEtpT/ezOo7f3hZZ5jwIIAYJp5FyqJEYCDM47IOTIkCgi956URmuDYGit0VozJkO0jEmKyJAjEwDovWHcCZEkaQ7AI3ACTgS1qdJ8MdODfLDMmPS2nU9uchaRbHCtqeecc0BgjCED57ySqus6AOuDBYpKJww5gmi7WZ6VXTt3zgmugBjT5L0zTZNowXUpuAohdl0TvSUiIaS1DqKTShBCCNAZ07XTsiwY0xFENR/rDGtTWe87a7gc7E92d5rJgRkJpPkYJBquk7HpBr1+ytO6qZCjsVbkuUIs0oKCmEx2Txw//len/v7OL6xxW9g4zcvVeTtf1WkT26FcQUO7fuSVrbvRsLe+Ugyv3rg+Q2+9BzCsre4+9uCjfw/MzL2dxxAoohKeATGInODisfeffO4Pj916RzUzzPrh0Aq9cPeLX766cmw83W9a83n/4vcf3T/72Le+8eWHXHftzPEjL3zju/78c5/cPK1u7D2jE7U/GSFjs2amuOKMEwfNUmdt07SD/iBEMLYt+uVkNucQvYfhwtJ8PpVSOu8YwzTNgmvTPG2tMc5h5EpKN7OiYK2trr51uvyXmdCcCeFs0FzE6KXUdduVRTko+7WpbdsOFgYUcV5NfIB5VQslydrdH242Pt4b9IaL2cKNh85+91oU4P7ZO9/9U2/dOBi1X7rcfuXJnW+d/849m+sLYbfc2r3WuPN8UCjql+jvW4wUHEUueZrkWSkPxhOltURs6k7ItOu6EH2vX9TVPEQUgvUKZbrIZezpzd39ncCMp8bPQzOjnb+fDTYz7ORotOdfJHx0+GXJuCQgwQgREDHGCBA7GzhXiqP3UUrZWRu/ZN3LDXzfG37lruqy7Wh24WF/64MDFjGEiBwQY7CRc9k0NWLMClHNXfBscTm1QaCjzSOHR/O9516/v/zRBSBaGy7iO3/ufg/NsLdkO6Zk62SKjQuxs0YVmjZX+vceuV2CTJOCMSrzAUAUUnXGhOgJQqJSLVJvApMoREoUESQySeCJAkNOGBGQc4GIkaKShTEmBMsEKpWEiGmaA3LTtMCizpbSpOBZr3MhVcrYxnXTZrKdikhE1jmV6LZtOQYA8sEzRHJd19WIgIwHwhhB8hgjeO+RRak0Ap+OR71ev6prnShnOylTpTRR8D4miQYg9zzjAAMyJoSKETrXJUkWgiBqirQ/q+oAvrOTyWS2Nb5RhTZQqKHo5geDHmecz1p//mAKhkmGS8tLk9lB2S8rB5lCdM50YR7s0V7x9Nsmhx/arOLBbHztRXe9anfnoJdqGcLVav+W/PDuZDukWM3Z2nK2M6mQhciir7uN4fEr0/3q/At9O2obI9EjhBhDjCA4CUaXjr3/9MU/RrKbp2+f161zPo8SRHfHAy/b272+ceikStJPzG57MP9esbQqiH3lM58ejW/E6dX2TO1UJ3VPiKRtbNPUJnSpzpDhbD5ZXugba4XWPsQ8Lff2d3QirXGeKFUpPo/Ckc1bp9OJVNi087oORT9DhiHEEBx6PLZ64vLOpdl8Mnq/PfqJNRtba+u2s2maai2t6UzAVGcZFx14ATEEh5wlOkEQxnpE7Jr5pbePD//tMC+5Z159p75wPnndC878xDvuJ9MeybbN0ss5sE+fu/Frv/37b73lVJWbrz128XSfveXtJ/7mqdYdo5wnVTfXMltdWtk9uI6SN3UbjJVS67JPtiEGUvJ6No+KKyFT0a8PlkbPhXo2KxYGixu3Do4P6uaLdXXt/EPV4ZPDvYtb5mVykLr5w9p5YoxFCBJZCF5r1bYtAEQmXvKSF3/jq18kkJGQceVf4cMXKgA4/tzSyX+/UQVy81ZPVTiqjq0ebtrWOBPJVk0TXcwSnWi5fXCzVxbzuSNyDAh5miTamfr6u6YrHxtmSaYR8E0/fkdapLedOkM1XZ1tUz12SovG7e60q6uJTnsyU0sZHFlY3BisFdmwlJkPkVhMkx7nmiA67xGlRB6oBQhSpMhliN65mKSlQHSuYxw5F0ImGNPIGmIL5CrGIQTHGUelIqgYKEtEYCovBohZjB0A1PWeN0Zw7uwIASFIouCCxeCk4G3bMYzIorWGCCG2RMS5FFw5b4zplJIhep2WiBGIW9cK5EzK8DxnOJMhOgQkYgxDZ5o0yRiTPoSua0Cwuq4FV1mWUQQuBBDbn4w8wvboyqypQXgm/Gw6Oj+ZVXNWFpo86AE/tnhcpxtXdp6ctjZNtGubLNXB2gjsyTdcfeBLx5LeoVG1v7SU+Sm7uHflxNpiwjJS+ur1p0+v3XV9slWNRonmxjGlhEOWJ3py5czouoFQU7TkI5KXIlJk+/3bZwu333r1T2X0ENulU/fapukcUUQB7mWvfs3msc2mbpvOfKq6+10LZ10wSZr3e8Orl29878ojFV6ZtCOW+Gbenjp08sLVqz5ywZwWsu5CkuNoVtngJcc0KabN3ka2vmfmGJgjm2ZptH6wNDR1laaaCwYgYgidMSJRtvO27bgQyFlwtP3OUe/PxKm1W5Tkz1y/5NFnQvAUU5k187oY9BTypp53GBjjbVMt9FelcqbzxPiVN+8ffWjVhkqqrDu3WywsvyG/68ytG7artJT3r+2Pkru5KC5uTz/79cc/9flHZHAbJZy5I/Mrw3+wDetADhR530vL/YMDy0TBWNU1hS5sjEJFpYue4nM7D5ZAJFe+CPtXtjkqZHlAxsPUx6x3+qW3n+HdTUf55e+wby2U/eWvwZ5d8kJ01RZ65GAlkuOSgneRJfKopcsSBEEIzkeICEgckteS+OHebV877LFmxJXQSsqIljGODLRm+zsmyZNqPl3oFZOqCd5LpYIHDMQkOgtccfofQfxx8JL1VR//6a++vnMxUkSy00lz1613PHPthuvmQiZZIkJHvX4f7WyxtPccewWjarlcyvMBF4yCHM93peBZmiGJVA+dbxkD7ymGJsuXCCBSw7AACiGGLM1JMqAEudPp0HaGcWBMek8UWpUUIUQOkbjmKs3SoRAQPIUggTlrHYaZMZZzir7jPLHt1Lk5F8x3wTmjlORc1NV4NhvneYqIMQLnjDEMwQMqzkmIDMBzEJ1theAMMCIDjFIIxqQP4Gxj2oohOO+zrADGuBAIMsbgnG3bNktF2xnOtHEzDNnO7KoFN560X37iG0lvISb+ech70E3yhbwzghBMVyvGFoc9ELJpzOOvuvDKb9zhoTk0uOV72xdfePLW7116LtjGepct4EqyMp+Ns3J95+BG3da9LFeiN61u8Pal+1eQuSballEAep5nGKuFuxjG5cm3BPhASL5bOnVvO58HkIuLS/Vs7+SZ2xjXUqhyMPxse897l84nmdKJDpGETv7hyU9eHX3V+7a2nefIPc7HUydCNIznZTe/lhUrRNw776zlisgDl2q1XLo53U6yxEdq6m55cb1rqhC8UhK4z5J0NB4NhsPxaGxbt7S0cjCe6Exuv2M++KtUgF8ul/emk83NzfloOp6PhSClE2Rp240ZBM8AA8kkjZ4Z0y4uLLXd9NrbDm79/OEQYuvMsFxpmXvdlYWNo0el5N76W1dDkqkoVxRnWa+HvP+5rz31R499mp1QzPsoMQKpyExbBSHqhkpCrbMWOkdtAsKgBO8xvrZxK/vnVjljuzenvLxbTr4a/NNJe+A23+2vf1iUp2D6WZ2tvvgNP/HYGz5a/PrBXvcKxmtqL7pmt5QthdqL3EFDjgdygAKQAzmGPFIEwEix6C8AEvwW3f2Ph52tlUq6rklS7aJ7nhBcCFbNqywvGOfWOhUYISFXgDJP9N7+TowCgA7eO136b31KxEKe4rt+/m7k0kSjMjRtDG2XyMLYpugNyPPQtf281yAWIltI906u3Hp8aVPylHMcLqyGKIzpEp1olcbnUWjbRgiJXOp0McTo3VQAMA7BR0TufS1kAYiMkUBZt02aFQSCcaazzBjDgfK8x5OEi57tzM7e5ZXVDQJhnC9SbbvYdftSyuBbck5y2TRWss66FhEAGEbpQ4csciZCCIDUNLXWOkaLjKRI2q5uq0YIKMu8M07IHJAoRs5V0RvW1dy0c4rueZwL5EJpHWMMwSNCCJ4DAwjTyTjESa76lY0gWWvnu+PuenXlym6bFNyE6XJx6KBq2q4N4JQQ0btB2ZvWNUP1zFu23vzd+w8ObrIMUzFkRX7x8veybFDXEYUzIWmanSPLg6Ls3bh59fUP/Mjjlx7dvlGaK0fATEUwMXbkPWcAMY4GdxKG1ckTkgWKDjFy8sMT95qqXt44HANNxzubx0/0euuck0j4w929712+CkDIWVkWxOBb5z7/vcsfX+j3D6aTlkI/KWeTmc7SCG45PXTh+pOtDYPBkDMRoktVZrxbLtfH0+3Kzziilql1ZJ0DjIlO5tNZUupUaSklIe6OD1KVUsTW2s2llatvGy1/vDRu6kxkjOdZaTrjpHDt/PDy2kKyeG77UqrE3mSEnkQmnTW2i1rqLNPn37R14lNrgrOZrdb7azK2Sz4/fL0/WO5zDuur6b3D8fnZwuJwWC4MUqmAeKvFH3zt0e9Ob4DragwcxGQ2K7MlRmYS5szOS3VrzEdb3z6+c07J5QfZJITBetx9gpbO8NmELxxvISjLWZG5/a+JtZfZG3/D7Xc426Bf+usXfOYDXXzJzXE6a/NI+1hdYzIPMsrRn8h4w0TGKAA0FCUyJABEBowdOXIkML5+H1cL5uzrdo8+tKqU7vVL5zvjOowhxkCco+uCj4wLF0IqmXFdkuU+QJkNmraOQVpnLr99+9jfrLBEKAb4vp97kaeWayGzvMd6TbfDQFqhBYEzvJrfPLyy4h3zpn7hidObxRoKWBiscKQ8K1BmRIEAEGUMpJRCROcsR0CdAnIJQEwb2+VZESNyDklaxsDms20INslSFzwyKUUSISDD4BwDIimkWpBSUZCd3yqTE0KLzk5TOezsPoOkmm9F21JwnPMYo7UdUZRSOTsn8AigtI5eOme9d1qreTXlHKTMGIu2bSh0IXiVFj54oiClROA+Bs4UZ9L5DoFi9FzJSJGhBAJrHRE07ags+t4hFxFBebDeu64xteme29s+COf6xcbVrW3FyWGR53Jsp854Flk9b+Zk66q7+uDkZV89gwBHV3o7XWedIcI06yV8cGh5aTI3F7f/4fDiLdcne9O2PZqTtXfvnFuK3QEPDTkfgIA8h3Du8PuXJ08OZ9+WGBhFQqRoEwHlkbtc05WLA4bSd205XCp6SVkOLsbjeZ7f1RsJkUilfPSa8W8+95VvXPz/YmB1O6MAZMFyPlCKZ/o9J37wY898OkDLmYjgDVny7lW3v+5bF79OiJ3FQqnDKxvTyWxiW6Fk13SK8TZ2WZI467gQo9G4yFLGmI+hqpz5MVj5eOZ98L5LVRJisEQ8ME9Y5uX+7rbSejjobR3sZColYFy4ahYiecH5lQenm3+ztLysinJY103rzHw8/YA7vTOugPTxQ8tLA3XPxuRKvXjk6GFR9KUP3mOINpF55UmB5lp++JvZb37iJPpp1n2yS99q9x7LhuvNyCJE4T/ncUMd/mB387vS36DhiylZJTEC6vPO+Dzj4wuwclvcv4j/y++y//zTEcar4tH+8DXTSo+7VRsZCIb1k2z5hX7yCA/7NP1iwjsp2hgUQbDBE9Ly6tq0rk+/YxAC7L17/9QjR0xj+2VJ0VsPoWu9D0wnKKCZVabr8qLw3guJWqkkSZqqWlpeHI3asizOvfXGrZ9cqkwTDOLP/pvXXK9rY2ktFVm+euXgylp/vTJjLYGz5fFsl3NUREWe3LJ+iDxl0W+sbvTLxURlKimb+X6eZwEYAUqpKVAMkQvmXVSKW19rVRhjkiSz1gGELM0BGRG1XV230165AeCFSBljQogYoyOWZTlyQcTyfDmClILNprtCIHASLDNmRDFigHo24twwnWIgBEAEYzyA45wDCeQGSHgfYvSIjjPtfQAg7zohBGfCee+jpQhKcO8NQ+6Dbdtpr7fsYlQqQWLBBR+CUNGaCMA4c4z1CGz0nbWdtYYLlSSFc2Hc7OyMbgZmKottNJWrrZHzZqr6spo1xMBZVhvz3Fv27//SMUCeSZMmRRBBBiWVbuumQ59ItVNFZ8a3nbxvXoUroxvmwp3U1jw25D3GDghZdM8c+cAtlz/MMTJoGRdAXjJiELWw+eoD43raHw6lUE3TrKysbWyeaE33BX//O5eezdOcfx9jTOr00XOfvLr9FRuCjaGuGzDBM1BKlCphAPN2muYpCTZIDk3qa5In6yuHLlx8mikhtUzlUmv3O4tANmfFPFZdY1vfrZV9zkIL6BrPFOqEYUgO9g9uvqc+9pnl2biDaFWqOYNSJ47xEDud5G1bpzqVGhmnejqLAQkFUYTopy3MPtAMPyIFZ6uLg91xxYViLN6pl4881+w5tlgslBJefKI9v5/de/vAyWFWFJ2x0XofRYAIBAv95E3/erjlFUuOmMk59F9iyQd99XmRnqJ2K6qcmbOIQzj0T2n7K3GwzrNbvOzEZO4WFlTNX7757a/s3WF/5i/Z//Xj3O1jU4X+4V79u8uDU61Yb9jhJq4GXoTtTwu1aQWV4grff0S045r2kTmKbH2t5wIeellRFZh37uapm4InydkkVQkxVuZiMp1wrnzApZWlg91d01E5KIPxnTELCyWBreaRM9c1Zrg4uPL2/WOfWnZEhWb44D+/88TSXVvbT5EuZQz5QMz2mqIskGRe9GvThMgZa4apPqLKE2uHhU4BRJ4WWZ4Jntb1WApBwLJ8wVorBQs+EDgirKp53UyytBwOF611RCAld84BgNZJ27RJXjCWCc6J0LqOiLTWXCgpMx+8kKTyBS51CEKyLIYGFUBU5NF0o6YZS07BecEJKDhrmWCSZ7P5SEoeAhH5NM2tdQBxPt9jyLXWAOQ9NW1d9koA8pY4E5wz79qIMUZgjDgrfGgRnhcYUjA0ne2PppcWhytpuiEUAQnTdkReSB4ixcjqen9r68baxlKiV7YOtrgu9mY7ztmDuFtbYRuhS9YZMGAfe/Gzt3xueTnvF33hrDd1E4INGPKkZ4IapMXUzYpkQNHtt9X02Vfz5oCogughOgqeU3j68A/feu3PIAYlAAgiRA5BC0o5KzJqijNu3op+qaSOMaZpvrS6lqTZZ9oXvnflohBcKSW+Twn9ra1PP3v58zyTRZAddYfXTj9z5dyR4Ymr9U07nUUGnMXG1WuDzfFkj1zIi6zIV3f3bwZojxw6SWj3RjNrrCMnOlW7uVJpa2YL/VWteQx+OmuEJMmFI3rq1Vdve2StrZzz1BsutG0DNjj0g0HW1rYsc/LEOEdkbWeVBudD/O88Mbr85oPbv3CkmjdMuKoyeVZa23nF3utW/E69W4VTm5tH+uZtL5gBwNSlT82OMsERGTAWgiDgXNVv/OXjbcL55DNx42fj+AlovkAL72PtWbSKYoccI10gOinKH+CAba/PaSmGy68/PPrcj38UAH7tvpu/9o7/yGQJniJERMJiadD+p6Vsg4UjUQixcnxrvxukTX8+NwEMtQ5NNck65gv69KnNuq5m49X3bxz6ZEvWNePRB/3Rz6woxhrbtVMTotdJwrnQueCI1Twwwb01jAmlEu99CB7JWxPTPB39k+rUZzYa5wZ5gj/3y6/olLpw47LmKnQ8UUYm6aAvo8tVwqvaSZWQMyuL/dUkHZb9zf5qmS8DIONB8bTpas45AoBAzrhgPEYigOCDEBIZASEia9sOABJdSCk5Z7PZLE2ZVH3nvVZJ27ZSSa2kc5YLgSikVPN6omQ6b+dpnhdZCVGbGJp2vr52uGpskS3YrjFuFK2H0HEGkVByYW0HSNYazgXn2LaN91EKQTEQEABwLpzzSiljjFQyTbNqNpOKOd9yniBGKTIE8MFbUwfvmYoutIlYqmuTSBdBBg8AteAZQeRCNK1hPAOsBKpI1hsDoqi6CTL6zpWb4/Z6V3dNCCH2Sbqn3vDc6YfXh0lgEm1XB4NMydZbRQp4OsyX5mE/kZIxvPDU3fHgQLIaYnxy9d17+ZnF6ZOChduufZiAFAeMFpAjRC0oEXG5r/ma3jpYdnOPaVrkJec8TfNjJ493nf2seeDB/jNJorTWnHMppWDZ3z/3YZ4eXL5x8WAyUSkvsM/Qe6xlSO84dtt3Dy6b8VykkltIk2LWVY0xbbe/ubo5r1sOTCWJcV6JvqUDO/Inj99zMNsSqrh+82IqsrkZJ2kRouGgW2N33lUf/cSgqhrGAbVyLpILQrKFYd51LlJAwZlnFLC3OGybadu1IXjT2eDh5nsnJz8zdM4AJV1jtE6RMcUAhfhAO9i6MZs23b/4oeGhBQ/fd/6gZBwJkDFEpgAZcPexR0sLMoTrFIktvgX2HyZooPfK2PyjDNyBYxAAbMRllqxHApQLxNxrVp965en9X7vv5oceW3/9H7/9kec2pF52wkmTurx71b3+9G39yYd/6+v0dsmql7z8zseeeKYZ7zb+1tr2nXg6tgkEr/FTD74hu3KJb03u9SdXVxb/C/A4P9kIjcNnkqpt66lTiRaCMYHOU6JkXXmCGELs93Pk0XsLgDH4PBva0DU/6oYfETZQmkp83y++tDG1jx6jDrY+tLi8X3uguVSJ4DxajcwomSeCbllZlchOLC0e2bgNAgPokAli3ForGVamFlymuqCIREFIboxhjCuliAABARkCtW2TpLppasGiUEPCjiFTqiCKRD5Ex0C23VSpJEsXA5FMhjrLrWlkqoriJHCOxNv2ADHGaBHA2MY1EwkRiMfQOOc4FwDkvIlkheCCy+DIhy5ETxGNmQuuGJOJzgMQYzyGEKMlikols2pPSWGdS5NUcg0ESLwzNRcBUVP0jenSVCApAO5cRwwZEwwAQU6ne4zFpnUiSafVRMn83O6l2lTOmnkY8YRLHR6977n7vrqiYCGJQqpwc79BhGnrVwaLk/lOKlTVPAAAIABJREFUXi5Y4hKjt2sXHy90aEnxpxffevHwe+D73vm1tyAAImH0qQCGJARo5g+vDfcG0/NwcGR8dzfDKGWeFUIIzuXxkycA2OfcA+9eOs+4UEohopRSpclnvvv7zm8bX3kn9+sxBuoNykQvUOuzLJnM9qxrpC6Xyx5HsV8fTJtpocv1fOHSZCvLZV3ZgAje8ZCjYCUWEzfVXFR1IwoGNe8N8+l0dnT97u3ZuYtvPlj6i0TqNJWhjdQZl/CkrWoTmqQoO9tp1JvD1X5RjqrR9f2dtEh0kpCNAeDcm24ceWjZW7+yPNjf3VdpZkNE8BDSEwv8/i184ontt96XvOMlPfi+/+PjTjJSOpUq0SnTeT8q/v98fP/LO6+BgmD618yTP/qLyl6z2x/h6avCyrvZ7qdD/QTjgWIGvQcozpg+geLOf/P6hz708i/DYz8J9/3hb3zlVeSbR58uvvxcD8vjooib4ouvef8P/+Rf/+u/O/TC3z+7gkbc+dLTX3/0YkUstDMIKc97qAqIXc/+e5UsKH36xv49YuHo8LaLq/2Pbr1755ZPrY8mlYteysR5LxVDYAi+npv+sKyrmdbK+8CYYsp2dVuUC9kgufDa7eWPJMQlsoAP/sz9hNjPQ8R4auWlN8ZnEzkYt3vNvF3qKVN7QrY3M3ka7zp86J5Dp9NiIecJUyBJRCDGhI9RSEnAiCKD55FzhkvkTGvVM6ai6AVn0UcGHoUEphCVEBqRS8m7rhFCEnIAcqaOSM7CcLjsvBNcc4EuBMYEk1rpTKokEqpkAKQIOu87yXVnm2B822wJJUTMZ9MLHJAJ5r1nwJ5HRNPpQZ71QjRc5gypbWaJloEEURSCA4JgqfM1IkqROd84b4Mz3lvGgIg4U0QOmMqykv47j0wBSR86Zx1HEkwgw8hjsBaBW8/m1X6hyo7mF/bPndt/QkBTlOtfevHOHQ8nCooWuFDoa6vTfNYBBJZoGaxTnJGG5sLJ0VbFBAMUj578xZ38Dvi+W7/96xuTr65lMHLtiuYqhZyxzc1iZ/FgYfGOUTiIz5TTSSTivX5fai2U3jx6HBE/Vd/znqVnpdRCSkAmlUxV8Rdf/1WP865xy4uDEmExvzfPY+tnF3auWz83ITRtC+QSLbVQXQwUXdcYFikqaYxHJiIQBlpZHEyaSjNZm7abz7hOGffkmEx7XMmTSb4XzXdeeXHzoSEBusZvbm7MDsaHVzcu7V5hLPFtgDxBF+rZAedqWjsGFoDpNA+RVIKTD7SbnzwcQgvMRp8c3Tx2sH811UXrQx3bjda94Gx9YWf8jhcNTh/RX/p2d27bYIxMSUSRZbmQEoW4sj/5w2+/Cxb6sX6M2ScipbT0k7xUdOl3A23g6v8E3YRmf87IYXF/NNfRG1x9d4CFf/vav/uV7PbXnmu+fOUQGPuqW7Zff3LuJxcwnP/NL8r3fvCfKbXy0utfftxc+9tnlqnVY37S+B54wuiC22cBGHydVV+N2Q+43gcxSxm1OL6kN0T+od87/BBLhZ7Xc2SUZilAM5lZRVzlRYi1dY4TcwCF0FG6am5joIVB9tybd1c+VgBXHjp89/96Sy56Jw+funEQnbmoi0Shrn2tVC4h86GO2K+qvSRNVOzuOHrozPL9pVRMoLWNEAoRuVT0PAApZIwUIwkuve9CiIkuqmaqJI8xxBCdazlXWVYKpQkpeGKMdabmgEJJIUprGi4QURDwoijqugkh5EUJxBljkXHkiU4zpTLOMiHBOQ+gfGiQwJtWl0OMbHRwMTrjzCh4x5DSRIaA1rVCJNY2SungPQAwZM5bgBhCSNKEonCu01oHj5E6Iojeem8DC4kcMh4FT51vgChRInrrQ2BMhWgYikjMmooxTjyF2FpjmciEjIxwOh213s/dlDHcGT32uZftvuVb9z97cAmtLoqitb7qYG//YNLMhWKyKHuaN9PZpnnpaDYPBjnSXnnb5078WwDYqJ6457FfCRCDmZSJ1tJtLiTJkH3LX3Dd9NDGnaPm4FB7JjaFi9Qf9FWStsacOHWrEOJhc/97lp6VUnMhCJALnublZ//xd1KVVrODjvtVdTTvpU9f+kqNRrJ+N68wS5gDLUUIbtAfTOcTR87YWSpTa22W5BAxBi91nqf65niPIQvGy1RxkYbQztuYAhNSRoiM2IW37q9/TMfI2slYZH0bg4+tUrxIiuWl1c3e8tV5dfXqpaXlxfls5kPwzkklkMUy7X/vFZcPf6pI0x4hcvIQnc5TJkrRdTVSIJmQfe15NzrYK1VCMQKxaHxUkgnOGAuAKIUg9adP3fbMfImxNfDfEPYCsqwbvJctviA+9zuMiGdvCfwUTv8M1AL0HqDJJxh5WPmpCP0PZcu/Of2eTZc5VlRNMR3wcqAm7Bv/6dtf+ub31tY2m+V7Lvs7P/fxj11/Aq/HW33QICJYq2JBzETaF3A+tk9R/0cd9biysVjiXZXk7fJv//jxR07UBxPAkOeLAbtm3grCgEKrSJB2bSVz5Zu6NV5yyTn2B8UTr7i09tEyBMxyhe//5w94xcoUQ0s6W5jU2ymsJf2ANnZmxtjAuLrUpZQqVYzs/MyhQ7duHCNfcBm10CEEYhwBKQQuhJQqEijBvW8jRSkSQtk08zzPnA+cAUXmQ7S2TZI8SfOuNUkiq4lRqeGstLFVMpFSaZ0ZG7uullIqKUMArUUkBC4DgOA8kkoyzVhibQDGESNG2ZibiwsrrjMQwAVHPlSzA6UigtSJ8I66rkJ0nXFZVsRAbTNnHIgoxsg5AgDngjFhbQ3AyYcQbVYsEdjgEVgTA5cMQnD1bCa1Hk9u+NAdWruDkMbjnV6/L3RPcUYUjQXG43y2BxR7+YoPTioRQ/Gfj/7Ve5+4u6oET6oLN88fdOrsM8+UQm7Xs6XFVeCOy2A8rNV3dtPOet066ELXmXD5yPtOn/u/h6W/tt+slTLJNinunb5t8R/9zZnrbBCK2zzPlu1Rv5e6CGWvnxWF9f6WM3daaz/bvfA9S+c450wIRM44Fwn/7Hf+o/dTAUvLuoylv7rz3Gtv+8GPP/7fcr2UMimKcnf/Rpnk/V7vyo0rWTkIzi/q3gOnXvQ33/loW5ssSXxwTRVOHd28vrdNiNPZLCsL3oqF9WHo5jWFTCqlwDTx/Jv3h3/O0rSwnqTOMEaM3rrOR8+zNOdJ1e63lmW6XF8azubb0YkiGeRZ/tz2WftjZf6ntLRwZDwfN90IWWQiid5b8Av9BTuyHrsjK8PDW9WpPYjSRBCuaogTkwkDJoQMRA74d7YOf/zcIEQe9N1oPomu0Yxa9SY6/qC88F9dfILES1HencRrrTvLll/Hdj7ukNjyj/zq4BW/Pv1SlLkk6bFTtslA/tmf3v/aVx366f/5D8eRv/CweEF6MUvjY1/99q9+/hgLqeCJVUshH3BcAUwhUCyWMVrRUiAniRwACYw/95f3/MNZKUcMLQBasiyoXpk1xvfLpO7mvbJoXDQz52LsmnmqtfF+5z3V2l8VTe0YAj74s3dGylmsgUfv0qQM0gsmUXjWWc+4y9K8JTUd7Z88epR7krF98ZkzCQ64rFKxqHXiQkySlAU0ziBHnWjyUNczrXWW9ogJxNjZjnMhGDKmjTE+tJKh9xRCjDFwETuzs7b6wogQPHGOBFyrIsbgfMcZMWRSF5wjUYzBM5lbG1vX9fvLXLIsPxTBRkcRkKHz1lI0PnaSyei6utrDGEMwnGvraohSaoWME4FA5r1zzgJA0+2naU4RY6SmnaZJqaWM0bbdWKtejFFw5YON0QEBIOdcaF3GaIiAMYERIwXPvKlbZJFAC4FKpJG8EHo+Hwdnslz/wfG/++mrb6mrZlR1T114ane0d7O1h3LepW5hMJi5dtqMOe8N9zbnBzHEYJyPnjiYbyz/yNpTH6mqakHXWZ6y0By5c3nSs5ea/WVUPNVMCQi0QOt+W3cd5Hmh04xJcfTYaSJ62Nz/zoWzUkpAZFwwLlq39+1rfzKtKp0vVG7aTmdMZtbMTq7e2Ql388YlrYcuVCLy5eXlrf1tCOioWS2XTywde/L62Swvy37v0uULxrOlPJt1TSBIEhVr+65XvO2Rs9+8fnDljiP39qV6fOfxaOHp1948/ZkjPnSClMigNd2gt1rXkxhdKRRmGXlXO+QYDy8mcx9Hu3NJzLR1Q3L8vvkdXzhj23keyeTy6vbNxbQvMh6BB2c76jjXijhJKiFmB6bX663YuOKd7IJinEcWYzQBfYz/22fv5Wwc2Cao20L7J4IioGDhfnvyR+T2P7jmc5zfFdZfI+rHw+wyLb0SR59k3vzq8d/6kL3JqIsICFLag+98/u2HjpZK4l88/I3f+KXfX1rZvOcFy7ecfNXP/9T94sk/evr8/KN/8YkvP8e/+nQW6ACgjMkhkR+K+nDIj2LMsSUETwJi4Oxf/b/3fX0vTa57OzdeBnBCkM57oa27Lh46vDyuZxr1wbzpJcpa66O88eDu2sdKiqIzFn/hN9/8gvW7v/jMN5aWMsXWdupLrhulqgBiSLHtLHFJQXDmkmRjZnZ6Sp9ZXjm5tjbr/OZwhZMQTAjFIjJGQAQqSQBAcjadTjkyJrgQIhB471OtQggAwBhTUtfNjHMhRNK1zvlq0F+L1BIBEA8xhFCXZQ8g6azrzDRRCWMyksjynidKiqHWuTMtV5pxTsiBKUW6NrtKlk2zAxEQKBgrOCfk88kNwQhZiowpGWNIpGZtWyOCMVZKGb0jIOdcmibGdgCBMya5YFwDoTGNzqRpWyFyZI6CIuqcs1JK7z0wiQw55955JXjb1ohEFBkiZ/pgfL3XW4jEJNf/5/pf/+zWmwRLO2M7767v7JzdO08CW79bFurC1SvDsje2dmN2upokruKAtfdsrxLnD73zB7b/QEjWdNnINHceLqZnWCadN2Icq5whBfJpyCc9vrfR1I3IB3mvH1137PSdiPSwue+HBs8wIRkDJEh0dnn2zX+8/nn0c5QFhZhnCREejKfry2Wh1h2553afilETA+dcsNbGrq+GgWh1WIxcJ5n0JgLy21duf/zydxq3e2JwYu5mkxj6TrE+N85mJLxIuzBt6vbSWw5WPi5jrddXl67e3DpzZM0EN2miVMCDPHn0+M3d7cBjoFhkZde0XVd3noOvOs+2fujgnkdvmVazQCwVvOs6E6hMc+OaQmuep4wAOTsY7Q/7/f39+frahnM2zXQ/6W/fvCTAa2BHyoXBfP5Hf75xfT4P2HG6zcvD2H6Kk0NAI+4Qw/dF6tj+p33/VWLldBifZZMnYfAA1t9+FU9o9YOPtJdFdEHhirLPPPH+6DqJ4qCZ//CP/Yenv3wxXbjjT/7Le9/wmjsJIyH80oceP/voMy87+RR6pG7661//AXRTZvZCs0VK8fIwS+8nJp1F+oX/qn73fzj+Ax+7Zfmb47a1nohAcs6Zd56VpRwdtCoRdT13xmo9SPp05U17xz+zYmzHUeKP/8Yr+wRzQtZhpoXjguKk6YBJnngMnqbWHlo4srp6aOvmgfXew2SQSu/gpSdXBr1jjIKzXqk8LZJgXZIkyAURRe8RgQFGICmlj8Q5pxCaplFKSaGQsxiDFDoECiEqJQRPCXyg5zEhhA/W2bG1Ns+HSuYhEhcqycsQAahTOg8Ezlguc6lSIXKhcqCKsBfBIHlvmhCcD0YIFpy3TasUj9F3beVdraQmIB+aGAmRUyRnR4JL76MQKgQicolWwcfW1KnWTTPtmllR5ERKCJ0Xi4QMkTsXEBhBaLsR56BVyXlKFENw1hpEEly3XeVczHtaiOz31j/5o8++Fpjyzjnye+PR9uzadDYZDvmli8+QhGJ1dc9TMVr21/i8kYEqNBbAf/PQT6w9/Wd9NIy3quSollZf6ubzdj6bW7JlNrCmBTCvuP2VTzwytQaLxZW0KCVnq+sbIbiHzX0/NHiGCQlAHFEI+cTOX96cPMt4rFsqdGqNWVhc3t4faRlM0yhZ1G0XmeecayWjdXPjN1bW9mcTBWLQy8ZNVdW1BHbyljNXrl6tm246PTh0aOX6zesvOHz7sY3bz974/zmCD0DbzrJA2O/Xv9V2OXvv0++5/Sa56QncBBJCEnogdILlBxXBggyiM8woghRFVNR/FB0RgQGliCBJaBoCoYQE0nu7vZ+62+rrqxN9nocvPe/AeJydv+2ibz5y6yPPP3jdQ1dx709sHhtlGQUjw2g8qVsdKahg2NV5HcZBXdfgMGXcOm15kE0zhMnGm8Y7vzNPKPLWgzEYs1qbJAxG6URXCkvJAZgQWZaGQRDGEhOaplMELkzarSQ0jUWOSYbX1fqJB5PDdzQYSfBKk13YzFr3A+aQx8qTXV683AUdxqSZ3g3zrwEz9qPHSLzDjR/6cP+qD6s1F17A7fS8/vjuH99kkdTOSabf9sHP3/YP32m1L/n9j7z2V37xEgwGIaFB/+7vP/bpr+UYGkX8R665w0zqXpvseeGL7zl+zne+++hDD/wUL2w3wTno8se89XD3FYNdD+/a8emmSTsdajU2zjQNDgLiHCIMO6OEoEWuEKGrrx/PfS1gDCEE6NXvvlSyKCuqXqL7yaBSFQHmOZ7UeZuEGJHCad+MnY9nBt0s1QIL74wy7tq9g22z+5BDrbilLZRV3m21jbGEMeecqmtrDcUEMJJSWg/PwggRQsBjTKi1BGNf1SlhnmDpPbZWBTLx4DEG8p+4t1LrjBAEHmuViTDCmCrjMKZCcIQxohwQAfCT6SgMBcLdMJ4Jw7mt4fFQsLqpuCBa187UUgTWKG+UtUaXpdK5lBEgarTBhFDK8ukwCEKMqVLamNpaJRhjlCur82kWSKqakgUxJl5b5712tkGICR54cBgJ6yrnNKWSIOkBqqrEGBOMATlChNY1wlxI8dfzX/214y+1HhulHUKVasqicNrVaowJHMsmo3xcW9XxIjs0UHgyWouCcFLX8MT8TZeu/d8hRM3EUaij/vw51+rK+kabQLBTq1uyw2yVznfmp0/3bGWS7kyr2/MeOjMLAO52feANvYOAMUKIYkQpvv3Q3zpTIMYQws7U3lmEKROBtrVqKrA0zaogZmBsK4mtNXWtGEbKO4Y5I5HyBhMUcFGpzFVxnJBjq2tY2mxknShD4pDrFtN8+56e3cxrHwa/tyP9y8cxJZRxq7RnLp/UUdLVqkYOcQFKO8aw0bpWthOK6aj0IZVBxFn46DXPbP/WoN/v1EXtrGWUj6fj5bnZg8dPtaKWBs8FL/NcSmG9Fx6FrZb1rqmLTqfnoLTeOACtncSo8fKOTwjiLQEwiFoceR15OEwsotgC7Tl8QC+9jNdrbusrvvdGiLfj8aqN0B+66I9O/SnuXAXtV84l6m1vSSgxmPjltji+eu+ffuyL3eW3pNV5b7zeffZT1wJy4KkGfcmL7ji6JUWt3vWLC+99175khqPVB9Hqg4DN3fdPP/CNlbvuegyWXnjVRz/2wQfm/uS+lz+kFmdn391PPPK2qMEj6HbaCNk0LcOA5UUpOHOabPx8NvuvIQYmZYJ+4bdfkPmJgZYgdmGmm6dpK8KVY90gyMrKekuQA8yrhvKQWoWdK0EBM02rJZ6/e2+/vc1qL2NKCeOEOuet98YYyTmAB+cbrRhjhHFrLefcOUCIGO3KeisMOtZAp9NGWBASVPUQASeY5vkQkGu3+qVKkScIkODcYgEAFNG6bgCcNWW70/VEkmdhrrXGGGHGgGALQSglgLBaMUys0Xm+BsSBd65yeT0MWaTNVPAWIFTVBWO4rktAGACstZxza1TTFFIKZxxlVLDAOmedtqpSugSIgoh7C1LECKFG5YhwRgNjrfcN8g5j7JwjhBiHPNQYBcpkgjJr7Ce33/bu1Tfk5SYGgrGoGh0Goij0tJ5uTafDydlY8NFkEnbIk/dyWyqP/OqWnZHZg0u/svvwZ2KOCdFMdLqLaG12uBy0Tp6dLM/O5libRpdVanQzN31uVVR10zhEFpe3Le7cb4y6rXnOG3oHESEIYQwOE/e1x/5ssd3FktfpsMHEqzqSMi8KixBn3a3ROqYNQUEUBBgBD2Ve5NQz7R0YY5CJMI/CqLSmm7TytJxm06Sd5E25rb1ycP3U1XtfduehW0xhDIm0SqusWn9TevFPdpVV1RgcEWK5kj5EgTeVNooM5meyNIsjab1Pq9rV+c75S08Oj2OuAx49/qJje29bEpQ22m5uboZR4L1KqGwwMY1tjAqDwGqFEap10w0Tx2mUxGU2cZZWVWrB5bXuR5y1lruk+MY/epsjY0MKBXjpAWmwYB2BylEXuG6VXIbbL4cmh9HfedRHvZd6suP90eJHh/eg8ecd6eGFdxjHsCs9pdAwHBbo+B/MzP7ykF14fq+8555XY+K9wwjc5df/+EguXryz+dqXrvPUYkwQEO89OGUB0Qc/99gza+NH74QXngKAa88m7G/+rNd5en7nx6lVRCZxBMhRRHSWGSmpVpJxHXJx+IbNpVtalBBCOXrT77wgiFlWmPmIF6rOs2mSMOIY4lKZpirzwUw35hwTIgImSdgYfmY8xFBsi9v7l/b0oqAdLZW6kEIoCwEVniBsnXPaGEMpx5gBss45SplSJaUMIWytwxgbY52rpQw94gRTQrhSGhFDaahNiXyAkUEYaWOklIAAecEY9143pvEWjFGMggz7iAGhSZFvciaFjBxwQKSuK+tLShFxjIi2M9oZ7H2GHZTVJiGeIKaUERL+kxPWaGOUtbYxY05ChKxSNghiDw4hcM4zJqt8bIxlnGtrCTIEGKFUWyV4pJzCiCJfIRJa6ygjnBNvUNNUAB6eRfCz/nbpG+88daOpNUKIc660bpqyqvI4icuiKmyjdIkhAjq+70fZdFLVdQEeVof86W2vu2Tts4hg72PCTWe2S3ZOqjpvsmphbqG2djgaB2HL6nG/PCcdg7PGeSSiZO95lwBC3zNXvKb7lJBSa408KJ7ffv/f8wh1gs7pjdMGoU4QNFol4Wxtp97bvCiZDCOKnUGVVTsWFzfSDJxy1mJKi3EqZVBr5bxb3rY8Gk4xokZnXEiKvKpdGEGauThO0nTLWQwEP/mSjX3fib3COS2b3DMeJpRXqpbtmFFqyzoMg4DLaVEUzoAxF3Xnj023fBwv9lo/veLw7n+fS/NhiNhWnjMqKJMeubJMQ5ZkWi/NJNN0GnY6+TSn3DcaDbq9rcmGlMg21kNDINLWRUlQ5KWe9h64eUQdeBIZSzHC2AqLCu0Rdp5Q7pFxbC/M/Jz3BKX3ouagh/oDK7/zEd1gMzTT77HyKTP3y5htRzZzHAAHS+Jbo1PBJz/7rosuCM7b1QECHoiz8PqbfvSDw/5l55Gvf/Eqh7DDgL1HQI1XFJhChjiPv/NbZP3hHy5m155NXnLLO+44c/H2XZ9bbn1fidI7jF0cBWaUFYzgtPKLnTiS4vCrs+VvRLYhyFn02ndfNj/THmd6ph8YJb2tttbLbjfECClXA9IE+4CQWhshmaBRZXWeafA6DsI9i4MEs0EkVmb3l03R7w7qxhitueAA1jnnPXaulKKltS6rcRIPhOBlWRCCMaZ1pSkDgjkQa61DiID3WnkLacAWjB8J0cEYW2uMNZQhjBjnoqxyShnnsq40IwQTbAFz3nZGOd/IsMWDDhDhXUFpZA22SolIlkXBA++tyIozEnedMt7nWTol1DSqaiW9Ip8wJpKon5fjIIirqsAYPQvAYuyrqkQIW1sJEUqZeEcaVagm9c6GcQ8cEwGra11XKWMeIWKMUarCCISQ6lm6Rp4IKf5+5Tu/dfpVWusgCKy1dVODo94bbTRjvGqstYrRULvpz+7aOHt6Uqu6rmqPqkdXfq3z8JeEB4trg8J2rxPsOCWCRlkbhpGrlSXEuPGO2b3uRFhMcFPXxvmZwcK5F12pVPPvzaVv6B8ijICHqqhJ2//ssb9yZcu2KHdumk1nuz1DgjQfUs61qoeToYwT7FUYhIgzXRQE0WlTdTqdgMrxdMNZo7WmmMooQoC914C0wK00H29b3jMergVJGwFYW2eT0qDm9OvKuS+ByusgiapCaQzYu36nPy5yKjh4wIzqspYymJYlNGrfth2r+YQSLiW57/mHz/nuovVGAM5NykWwfX7vtBjFvHNw41AsALu2sqUu6+X+yoNHH56bn3OFpSwxJi+AMjWS8QxoTbm5YO9LTq/ee+QhcuJgvrCbiG6mTTReJdMToVonGufgGfIBp04DN/03ILTDV8c88R+KL/rjs193/aspbavmMN74V9S6GpIXIVRaype6dzdH8fHV91HnrSWe1IxQgsx//4NH/vctepZM3/L6mec/L3nFVTtQGH75m49/9rPrb3vr7JtfuZcFAVm7H337XfDAr/1w8UcvuuUdAEzg8LKrf5c1I0TLQMZpMa4LLEOq0wxHbSbR8A11/6u81eqn0xTd9N4Ds6241LQwU+IEaI5JgTlph3MWlNJFJJnXkJU1lyyUXa2KSHY82DBIsmxKnZ0RzZ7BLuPNrsGK54JT5izBxKdpGkUto0tGI0C+Ual3mHNO/hMihGLMvNcIiHXOGO0cOOsow4LNFvUJihMqqLWGEqyVoiyihHnvALy2inPBaGAsStPTUdQnlDEiKXMeMSpjzAJCe4gI712jUuotoWySjVqtJd2MRxuHQkEZS7L0FEJY8BaGwLuKUGytHU/PJEmPYGKMBuydAUqRtYYQ6h1x3jinvTOIBFZlQjCPBQIKoDBwwI5iDB4557VpABxCVDXKWBVKWVbF/1n59i899YI46tL/kuc54wjAWmeLvOx255qm9t41qjpyNH/4weNl3RhrmxqdoeeRyLGNJwsFO2Td7rXXlkqGSoe8tQYpayjSKl9YCLvNjvRYoJoGU9FYdM1LXg6Abk0vfG33GUp5qx0vzM6mNv/nb7x16qpe0BlN9crcSjZurGBFnQsDvX53WmSekHE+munCmehyAAAgAElEQVR2GmdN0XTa3a3xBCiydUMD6py2jfYeB0FkG6fdmAWhrpyMcSeZB21G5bjX64+21q3BGMtDrzh74K5zJlsbzjtB6Ho6IgS3A6Gso1IqY7pJVGQlJRT/F0dwWuTtIOahfOgFx5a/njTKypA2eRnKmRsueekzZx8+lm3sXdorxfYTG/cd3zpjshIhrGo1M7O02OliVp4djQjpZ5OnDA28C5QaM8KjKA5CHPCwdEU5jjmOgo6u8/zkI9HJnwpva02FM4ohpDHCwQHovMw30w/Eez7UPEAz5cIO4glyyI++QKBj59+MoTi3fyvKz7v3od9h2DXWgKYa3L0PH3rXH548dDKkjAMQqwqUNC/YYXr95FsPcZXh7TT98feeuzBIyKfI+NJP3fTeyQ/1y5xOSZmHfXHdC/6X93xrc1O0u4vJ4onRiRaFxkA6Ga++MZ/5l7DVSYpSoevfsdBKQiZnnMoF6/ZbcTs878FnfijD0GhFvNs2P4eR1w7VpqlKJwNcac+ZjzALwxghII2ca7FBN9jT2Ta1thPEVmlAXkpJCDMaIWybpuZcIuSdAyllXTcYAyFMm1zKUDXWe08p9d4DIt47VSPClZRtjMBZ5YwFAs4apRrKGMHcA3igCEuCC8o6gHWRqjhJirLqzAwAIQs+iBMPWNfWgw6iThAsNLbOt07U2brkkvBOPjljnYnCEBAC3zRNTamsmwkjHCNmrCMYCKPeO+c8xsy5ChwQTL3z2iqrjAeLGaTpBIOKgjZQbC0AEM6F0bqqs057lhDmQYOj3rtPLN/67jOvMqbxAFopxth0WrbbcdPUAFiIAGOkVOkdWT2bff/7jyjta6W8tRbh+2Z+/qIzn+XUFaiNaE/MbqR4oxslhKBJka5sW0nLIaKt0HJ9JLTWOaDb95wjEyZEdLu++g39o5dcev5wtOaMTe3kM9/477u3XTYaHz9w7mUn1p7sJzuObz0j+dLG1rF2pzVM01avN9ocB5KVRttCFd7NJTMAuvK107hq0pl+XzWAtD9n1/mHTzxYO8JIWOpNiePzd557aP2YaowURPKW1eqnVz567u07BIfRtAgwAUbqql7odTwgj+nm1tD7JgjDulGUUuYhaHeR9YzTvBweuWFr17dn6tIRbAkWmWs6xvAgGemp8ng5SmJPT1nrlFaiikjoQLREMBxvZLX6zRf/xr/c+enG0c3ReiSipjS1rnjYQk5FScwQGFGGuJ2nDRHyzFOt4T28QdwacGAxFBS0Ydt896YPd675YPpT8BY553hImdAkFvltaLrpFt54ycpP7fipu+/5AgaEMThntGanzo6uf/1PN63EDWtcQ2XH8UYk1hpsU4tDAmP3xHees7Kdo/8buHf4P3/Du//k+Gs0DXxdMSe3XfK9XUu3ZKOCxDMLhHdneidGZ5WaFLU7+fLx3NdmMG2ayqBX/cYFLBDDzSoJnHPBYH5gynIwNwAtFEmjMHF1ignDrimVq6pK8EiDBeSYS7Jyq92Ns7EW0uyeGax02p1kfjAzzwFrp5uiQoGVLLHGGGsYY94jjLCzSpuGsyAKu3VjCEcEiNKVFIFSHiOrLLKuEpQIKQHAGIcRds7mTSZ5K52cHSzsQ4ABkPeoqjIpQgBw3hDClKl7vYG2GJMAQaO0C5NZY3JtTCATgmVdDj3ouirDMKqySaNTo413OmDdstnyYEPRtd4QxBDy4FXRNIwwxpA1jbeorqsoipTWCIBxSZlodMMwbWpFGWAstCowFQgBQS4vKkqJ0ZoQan1ljP/8uXe+/fDLlFKtVgsjgjAmlCplGRNa1d7VlFJtIC8mqiE3f+NBUyllbd14Y9WT296y7+jnlWcEEA3IYFdXiVNhkmQbZ9vdXml1mtugRdtRB54Kxkp1O0vDM4+fc9m1swu774muubH92Imja1EUUyk7/f7PDn4SNza15b5tFz70xDcJTc6srba7gmk+M9OrSZFNikm+OmjP18h3UIyQW2xt+8nhe0bT4SCZV6iUYRIG4TTbWO7MH1sft9qJqioIkdNIabOytHJ6da3X7xIwysKT153e9e3+bNTayKfO2uFwU3LGKWWCZ2nRnZmvVSMpKovSA7UEBKEYgYyCKIruec7je7+7tGtl19mzhxAJ63KzhijglGPBcTVVQlc5wiRitIGi0uiSwU7Vqg4fPuuxbHXjYpLt33n+sdXVcTWsy8xa35tZmG5uKqi1pcgrzjBmyXy7v9wLbv7SdOsIx4QjF7ru9Rphuvk9a077zks/uPDrH84f9giLuvrVt+0qMjVcx9PhkbOr951aFy1q9l54zYfef9n1V80DFtYZa+GG13/7pwe91cbzEJoIsEcD+pxZ8e5f6n3i82dOH9t6/J5XUmzRP2H0K/yWv/y9t37lRYgFti4ZpsrTF173gVE9vmR54YnTp3YtzJ9dXVMaxe3gmetP9f81qPLCGYp+7rcPhGGELBp0orQqK1XFnIetOGJYE1U2CjlvHQspcp7quiy97LWSyWSjqKEVkLSiTLh+MLOWjXbOdJZmFpa7/V7UrpBSadWOpQxnzq6ekgF2TgdBCyEGFpVlGUSYYGaM1VYD+CTuKKW5oONx3u23COpU5YhgKMuy2+0Yo6oyk2GIEKcEYUIZTZxz1pcAgbUOAFFKMdaUyrKsg5BpS8Ab+iweN6WWUYgYq5XBpmGcAYAHcFo7W2DPPIBqKkywtYpQ7B1yoOqqIoghAuCJs54yYqxhjGIMdVVhsMZ6hBBlBHmPMfVeO4u1bijnzjnJuTYOYwDvqirnQgLAJ5a++Z6111IijTUI4bIsA+k8EG1trWvCnHcuoHFRTLQKvnrzfbaxylpnSKOahxd+8dyTX9TaAydBKJaWOrv2NcO0oMyHPHrs+DHMwsrWeVHsdQcmVcXrjHdXzrv66n3b9338Pvv2/eOVbYtzg7nReDwcDf/j/s+tj892CF4fnkFhrPKKc7E+Gc52Wm3p4nZ3Zemi1ZPHjuSrBFmJG+CztNl0np+78PxnsidPHXlyeXF5KxvNdWebXDsWKpNS4GuTzVYiHGsCP6OcMbYOBc2Vevq6M8/92d6k007T1BjrjLfOI2cwQkWWdzozxpmtzQmlzGELBjjnzluHIOT4+KvGS7cmnIgiN1GLcLn4vHOv/Pe7P494TwqvrZEkyMqcY5E1XjJErYIkoNpYUGErZJRNh+n2uZVeb+nQ0YMAxVpZcMNrm4HX/f7CZLQpwm6T5o67ykxP3HHp+ETmGOcu9RDp7kuIW3bFnX84+LmPurGLOhe2Jj/50WudJh57jNmhQ4+97MXvdfjqYvbq97y+9ZEPXIQAA8K19te86rZHT1LmhaaNh03k+75h1+4W377tSuZ0oVlIjEYgPsU3btr4pV/8gm0NfnCfxqTnk8RndueBB3cMvoIAWVtRiow1TeM7M62DL97sfTUIpKREoHd+7JXZJJMMRzIoyhJT0g6jBrnIKs/p2mhjYbBtPD4dhHFpuG50yGTEheC+Vt5jq1GSl+MQiSCOJTRtGe4e9DyWx04fPLD/cuwQJdQ7ZK1HCHHBOGNGV8ZWjCZllTFGq7KO4r5qnNYqDDmWRJXAhPEuxNA4i4Tkx44dWlqcQ55wGRAaaV0onRGCVYNlGAdBoBqNENVap+lGFAWh7GIqnLMA3jvrQWHKgzjxCAkeKOWMQ0JI74iuh9hjYy1C3mgH4Lmg3oFzhjNUFoX1SoYtaxDnXGujmoZz3qimaQpOeVnlUSR00zAq82ISxy1tvWAUADBlWjlAFpwuitRYnSTxXy9849ePvYRTaYwhBHPOVaVkGKV5hihqNHDKQp4IQSpl//HTt3sDjbFG18bAg3Nv3n/2S7pxwKkMxLaF9nOehx88dGhlfqC0O3Jmap3TAKXOV9S+tLB5qS6+5nUtVt1Jr3l4Ovj/tp26chkXZTHT7RKEv/Wzr50pHjfTrTDuZ8Wmqd00yzuDzvrZM7uXtuf1JB1POOMv2Ht+JsnJ9aMMFGEdjJqEiCNbpzsz21xDuctWq4JylLBQq0JjWFm84PAzD9YYJUJmSkkpdJ42zh+9YfPaRy4qrXOmokCd9Y33QCAKo9FwmERhXU7zXBFKEfGuIR57hHFRZAizs6+dLNyS1I3u9xbzdBLH5LmLlzy8fshgUGVpLPLeUEG9UjjgXdGvHGhshEFlOfVIA9aUYuq7/TY/NVqbieaG5dmuWEyLvNdqH1s7tnNpn3Ho7NmnYt6tzPToPWJ0aofJ1w2ZEza1qLK04cEFvz/3vo+u3w7t/isum/nyl26sXY00Zszc8h/f+6P3fdPzlSPlhb/9xj1//qcXIOe8AevRhVf/+4mCXrx05OmHnqg1II/d7EWR724der3xmgDzXnvn7ef4V9oP/uafTD/wnB9++I4Vw+aBe9SEeIFee967Mh7smFmc5mdjyUejQoTBE9cf2/PdxbpRWmn0K39wHUG0qkfgWcRZp92psrwhIClPKx22mK5wpx0zBNaCy8c4iFgQWl1J4NOq4JJmmcWk6iWLxlZVU/YkneusLPaixXgOy0g1DSGQ5xMRcE5j7xCjTOsGE9Q0aZ5Ou92+8QGAIQR7j7QxTTWVMtJOU8wFj631ZZlavTXTnbPQBK0FQdsIeedtkecADSBHMMeIIUqR92WRC86MQyIItKq9qTzgQLYIkY1RNEwIjQkNCOWMJdn4KAXrsbe68t5gjPOswlByGkwmm9ptzc5eLETkAZdVqVURhhFG1FrvvQUA1ZTO14wgjIKqTjkXyiCCHaW0biyl1FqNkSvyPIoCjMnfLn3jN0/eiLxFCNVNRSkhRDDGjdFKVVwIQgh4pnSZtAd/9/ffLvNaW9DWaeUemXvT+We+oA3CCPNAdAbB4nn1qeEmp64TJJO8jiKCiazsiG/sbTYbvrinn6Avnlp5bOlX4b/8+EWPR+0EE4TAC5kcO3HwB0/923Tzqfn+zImNOmy1TbNV5nXVIBIBM7hyRTXJB4PW8TOnrr30hY+e/tEg3H7O0t6n10/oZiIpT/Ppdc951Ze+97nlxcWNSR7Tlmvyi/ddcmLz6DAdNQaSuC0I7STy3qsOnntbjyARBmErjMqqRlKMRltx3HHe52XqPZ5kI2ddxBIFpqgrBI5jCp4cf+X6/NfDUZq96qpXPnTkoU68uDo54sC0xEwv7J2ZHBVhlE3N8mBxY3om8sDbYjyy8wutja2xoHEr6rVaM6PyROTYli57rajf2ZZP8kkxXJ5dPHhmVaJ6a5qVeUWFYAX58VfPEH45xtscBVs8DRBiZxzYD86/8SOTm0n0qve889L3/Y8rKGb3P3LPQ/cc+etP/iyJYSbZfWp98zfefvk73vHSKI69Aeftrku+PUz9d754zZVX9O68+8QdP3rqK1+49czpU//tt170O//jl5fmZhyyxDH3GfQ3/sn3flZ/YN9H/+SBX4DhKdve42xJG3/Jy2/Gzb37z9nvudo4fXK2v5yp+qEXPDP7lZAiTglDv/QHL+qEop5O5+bm1yZr3nqSRAFwaycgFuuttRp8wANBsbPWA4hAUMcFN4QkAdXgKSEwKQnFlbGQVls7B8vP37U7ae8tq7HXmYxaBDGtNHjlQFRNzhgiKJlOH+30LqgL582ESm6M5lQ6C40ugyBAiGDEy2qz1Z3X2lEMCALja2e1INJB6QAojoB6TkMAVBQpZZgQjjHHGDunKBF5njrnojAmtKVNw6WQMlLWAlhEKBWxradM9p01ZXqqKbRxJWdMGw2WAc51bQkzBIWt1kzZZMZq78A6rXXeCgfW47opGSPOIYyBUsZYiIBbV3vvsNfGKms9QkipCpAGJ5Vv/uXCe375yctlMFNWuWAxJrSuU8J4lk6cruM4IJgDuDwby9bg1lsObYxyo0rjsNb6kbmb9p/+ojEYMcw4Wxh0r3i+PDWZPH36BDajA+df+szq2kW7z93IismxerrOvOhunXzsyA3fuX8k4b98/JKD1+8O0nzabkcUce98mIQnV88+dvDuBw7+qKzPJIGIwzCvm6aqwvbMsROHO+3ucr8/nNQNq+tCO13MDQZtGeRFUdUNpczreiaZ6c7NPnz//agVqXydYGgUCZLu6ujMDc+98fTomNPp3VcfvfKuvVIyxmgM4agZG+84SybVNMJsWpeVapKoO003VkTrcLHOPfWMl1nFJX/qJZtLX25Xutm+e25t7eRSb/8l511y233/Vmamy/lqVa7Mrii2ZRqbbamgK2IZZekQIOEJgboJwx5GanFh/+FTj21bOMfUa6c2snZbdMKoalwgFjbTDWlV05Seuru/sqlHzrMLQBcmmGXxa1x5ypVHvHcfWvj5D2/eQvGhmItXvmbHPQ8dOnV4zAkHZ4AWnfZ5ZX3cNCgQC296y4W/997fCTvR0rm3V5ZuPfnSSAQKKwFYo2qw/QdVfS9Up9/7ztf+0cde7ZBi/yjfevAnX7sret+Br33o68sEYvDUGE2zNN6PL37ep3rJTNmM6goSiZT1mz+nLrxt55Hhaow4eucHriMEVapmgjNgsSSjsuoGs5kaz4RotRCmHFVVubJtyVpQDgJhOGlbg6qmcKrUyjOOelFnczie77cxEfOthCO8c3FB1VWYDDgEQtKyyJ3zIiTeCYSslIFXwtFSm1qXRrZiXTsE0DQ5IRQj4IJVVU2oc0AplYwgybnx1HkMTlmtEEHesVpXQnCtbRTFxjSMxnk+EVJSIrRpMEFaafwsKhjj1nplnAwiziimAWaRaTwRjmLkSlvoLd3UjCCrLSbUQgoWkJdKTayvBe9a4wjDGLhSOWOMIlTVBRcMPFDGOYvGkxHnIERcVqnVLi/GrVYPALx3dV3EYd9Y/w+7bv6NwzdkZSlDYDTRuqSUVLWmhDCCjQVCkW4azjkg+/0fnDx0bGh0rbSz1j4y9+ZzT/6Tc9gTJCVfWZy54VV7H3nq0Tvu+tnSzMIVB87ZPrc7Jsn9Z352+rgbrvZtfkZ2l8+2dn01fg8AHJgtLuxWGLE3zB1NkqRuqigMuOCY4FYUSUofPvLUl+/4l43Nu7pJEiAyKobZtCGUtyIuoo6xU+zjzdHpHdv2pfWYYeqsHY/HvU5Ag+jIsePPv+CiVNtssskQASw98crV9cao0489wvdcc/zC2xcIwduXd5/YGmKvPGfMamUVA9QYZ5SZSXqZmnoPoMzqZLijOzs2dTku13++WvxqUlSuN99fPX0WmuzAhVc+NXpyx8yuRw4f3LPzvKVOb7a1/Mjpe09vncSeUeaNDZ0Ycd9VZUrD2EMZQLQ6LHhUJ3Qujkhe1KZuDHG7ZldG2XTn4sqhbG1RnfOlv7rVW4zYbkB97AtjC9K6yrV3c73xv9iFH9/6jMYo1EWhTzLcvfTFr3PF6pnjDw031qmXsr3tvIu3PfLQfb5aUPjgb73zLX/z1Quo664fehEBjzQC5sDIj/zlPR//26fCHrn+wvov/uKGbtALvkz+cbD1P//8YKOOfPDcBz76s8tsVjujPeog2r72Fz6h9XTvQufgiTPOOYNw97fbMzfHja7yukHv+rNXZ5O0HSW2qcuGS9w4prtRsl5OEqDaq5i1EFKcYcajrLCeei5kXSnAecBniCsVYKp8K4x1NSZRW0hmrO5jubu/EgQsiHpKuThJnK8bVWMIEXZNUwvBKYmttZgYD5FSI0IArLSupoSUZd6oqjsz7wERQsssX986uLi0z3nEqMeOAfacxdpZBDaO23mWO6/rRmNinHNJ1LOOcC4Yo957AEIo18pa8AimCKQMZmnQAaYAs7qqOaV1MaIoVPUYHLagYtlNsyECa4zGGBs78ZZR6ggWCGnvPKJMKSNl4L0zqvQOlVWOifUOMSo8MgRzRlhVl4wyxnhZF9qs/ePO+1/3wLbdO640tplOx5TyQLbSrCQYh4JXuiKYemc5JxjQzx44c+/9p72zznmt9cNzN51/+ovGOiAkCNieHYMbXnxBVY2083PdLhHh2ngrrzdUBcPK3PeTkjSjGset3pJX41NsrjM+uOeiK3v93h3mSkrFpT1zYF4hBIGUsbAIhUSydoeunRh+9nufW5/e51VVK2Mc8qZ2lFIrtJtWDe72ozQtWmEI3iqrnbVh0h5NpwMealJEIp4MJ0LKpCWRM9wJQdnYFg9ce/zC2+bjdosoP6qKVktmtQLvAs4cAEfMM0a1H7TiE+ubSRxtZOOFoJUhHRJ8+6VPX3fvBaNssx0vT+qzxoZFtTlob1O5LnCts2KpP7sc40fGGcWECONqK4UJyM7VySpYNWmaoh4eOOe5Zzae3Lf8XKgmT62f5DIq0jzXzaAtFtvnnkhPtkJ86j5+6K7DjROdmQH124bZKoaKgtCoDXby/rm3fvTsXyDENPTw3NugeODG11198fPOfeLQWoSrteNnz7/kOaNi7fjRUycf/149nB+XgsIzEL3y/HPP/fTfHNh17hKAweAaS/7p5mc+/fGHv3XbK86emS5+s+N+zfzvTz72V7c6ojbfv+Mzf/zka/SZxyneb7kkvLXnsofn9v4H6EqDLTaK3Ss7n7hx7bIfLJ+dbEoWog/+xZuYDKuqpsgXrpgNZ2rbeFUsJ3NPb64eWNh2+8F79y+uxKFYGGyva7Y2WeUByiY18+h0Pbxgcfn4uLBeEws7BwtK551k3ro6kni5v7Mq0kCIsmoI5Vrb7sxMWVRCUEoC42rvLABY5xllRisAL1gA2E2neRBI5wwgjBnFCLnGYo6s96rRDGFtSo9cEs15TDki3oPz2lpNZcta45xFiDT1xFogz6JAsGBMaGMAWWSY89Zir5FlEBLKo3YPkTAfbkYtg+3cMH1aV9Z7ILTkLADkORO68QCqzkeY+OHW2UFvZTw90koGBAuEMJPtuslaSSdPNSYOYQSIWuucrY1RjAmMOOFelfVn9v7g14++hgi8uXV60F/KsilCmDIuODWqJDTMi0lZ5O24Y3QzKfkt33zEO7DWPOuh2Zv2n/pnQGABooBdfvGOa66ds5VHNG6KTFEfebGRT+555q753v7HfjZdG6FtfWKaEfbI4oX2ynJZ6B27lhGC8TQ7CYujzkWzs4tv37eOSMSYwxRFXHiC4pCfWd/65M0fq7ODM70l4qxB2FiTFlNGO8agSbHGMUpawWY6lDKs88ZibFUdRqypHEWYM5daNIgTjGpmqcb28Zee2flv8fzyytlTp5e6PQhgnFZUcJsXLI6EY6UtFAKqlaGCgQdt406HIFzV+aPXn7rwtsHmcLywbXk8SYWIJ8PJ8tLg5GgqARMZdgV2jFKnR0VDMEfejCclkcAR80ppFEzzTVP6Bqta204YxkEoA9Zk9TBTywu7EzxcnQzXs2r4wAVueHhc54uD/mgcM++UNd4r5Kbew1XJjRa5O2uEe5e46YOw7RfJ4d8l8WtF5+JrzjPbz0nOP7+9/4LetJ6+9W3fyjdubslZmVyXDx8hzLa7/q47/yFqh4wIDADEbYzKQZKkSuFPu9VXrb7wxn/PlIDF7R/ov+vDj/5PdOrrDu/wbDuSu+PFjT2X/5XTzgvDdbQ427/3isd33zXPlEeihf7u73+hsWarrLtJoJRWGo9Ho3YccWg4Y5PxWEqxsrir0FuShJiLJ46duXz3XCxmj26eWuhGZyaNxWU3iGsjOBP7+jsD4kKWTLLNVrtFaOR0hRByzpVVyQUJg25RFlx4AIkxppQZYwj1GEJtlDZTAhFjqKpLSigi1FtXViOKKRWEYNaomuLQeYy8FoI5EMaOAQmK5ThLCfKUAmOBYLFHDjwCD4QyhIWxlfcuCtva4qZJKeFRay7LRpwHgGkr6fIoOvj0Y3t37xtOzhhlg0AKGkwnBaUlWDmePtIKVypVCN7WujE2A0e1bgh2ZZlHMUdAKGUIEYJZVWdNo8oqHfS2B2FQVSV4b5321n1q921vP3QVFUEo2wBUqRpjXtclAHAunEecY2uVtZp4m5Xoc1++zzunDGitHpl744Wnv+jBOMvjRF558cKBKxYwJpTIskqjpDOZTKUMCRaUmc/8yxGkTqCyqXxkHdKeGgce4x07dxiPZBC3e7NChM672/XlQcB+dc9mfzArROidCQKhrY2j+JY7v56t/rAuTlnWKbQfpdOQBZnJuq2dTx55eq7f84zYahq2YiYYdqH1pSoq7ezp4RCcaseDLN/0ns1E8ZMvOXHRj3dHDBVVo40tiqw901JQgYmVyjkTglljJCINAVo2Zr6fKFV5ypAjj7742EU/2HHo6Il9K4vTppBAx00eIlx6nAhcaZQkSTqeYABtTZx04lhsnlrv9Jank/WgHSqDtRtPx0hGiFFkeQCVwVU9Q8XI1byhJ9SGRIFNV9y0vStWjz3xZF0l07Rj9WnjG+ekwxGNXqqzZz60+48/nD2sAXh9xNiSzl+Fj/y5CvfT7nVIzJqaYWUs0T6KWP6o2fqaxFvJ4qtNCkofCcLpf/vdm258xXWLi71W0kLEYM+ds/bv4TW3f/zHPynI7IoOIvTE+9//tjd/+J9SCn2FJQKJ2/uvf/MnGrU225FnNvVs29994Jnz79gTRSyzDL3nfVdcfs7+MtdG1VOLinJtsd9J2Nx8b/t4OkzLcV6lxpSzcTLb66W4O55MEGxKlpSF2ZhMo7gxRvU6u6um6nTtvv753uZzs/uNdlVda1NHUYtRluc5wggsItQbYwgW2pYI4aqqO52OswRhj7DXugHnhRDOIUqCSb4qeATeMiqVqtMsa7Wk0ahSE8lbSk9muivOOm0c56IxOpaDqh5ShhBwrVMPCBEmZWSMQQjAY8YCGnCjjHeWCQAXI4woh7xMMYnzfBwEPRFxW6jJdMgoYowaraRgGEiWrmlr47hNCCeYG2cxQgCAADmlMXFlNQFkESLWaopDxkjVZBgxrR0iBKO6KLLPnfPjd564EVOLPOOcY+Kn6TCKutbYvJgIHoIHDGCN1cbyIPz05+9yTlqn6/bphQYAACAASURBVBwenn/d/tV/ps4BMjKc2bmtfuPrX661d1A463XtlK7iKMKEeF8OU/ztW+90PnIOjHUWsEfEae/BY0KdR0G3052Zbff6nV4viYJbx5en6eSqZfXCnag/GHAe17ViCKcWnn760bWz3x3lTyLBNoajCsdziVwbbYByncFsltfD8dZMVzIsR3neiRNA0GgFLtCN6nZaqvIImidfcXrum30Z8Fo1aFzjJNraWI8kEkHAOBlupcuzCwZVeQ6N2Zif3SEwH402kfTGo6dfunrx97cpYxH2iAgwNRCOjdKAVV0ILhoN/ZmZ8WSSVeVMt98000ESF1VTVKkMB8pneVZxBlgmAklVVyKIa7VZj6lv0Xw6GQQDmuTTg5ecfvyuzdVpo2Guv7NWk/GUeCeQm3XUkWCHbb/+A3Lhw9n9WIcuJmz9Ftu+AoIVf/b/R3YRWN+2ziHROYhFzLhGN4QnRm/Q9F8DmIbhC201VfUpltjrr1l83Ruf/7IXvzZKHCaAPyPYe76AxWFsx2hupzv+l3/4puv/8JYFhCpwbQ8dL+cue+ndDf72ztn5pCuPHTub/tK0+zVvqw7jgP7PP7y9LcUoV5VqJuOjnZZc7u/rt4OFznbrATA7u7YxVScDH3KCj48zyYo4HvQ6C9PhKIhbIaN5OplfXE4ziNq1nciozWuTBqLlnI1jCUhaawkhzjpnvAMVRZFqfF2PnPOEEGscoZHzjfcOY2qtR8ha13DOrXGMhc5ZjCVGHiGkdGEN5EVGKOnNzClVNKUOYqlU7TE2eiT5rNbOQ85wSCgFwhqlojDyHjB2WT6WPKhrk06H4Mu55XM5i7ynCHsqOAKNUEJFYGxFqEBAjbGm2uKkVZbr4D1nSVVPijKNopgSwIDqusEIE04wJnmeh2HU1IXzNXiUTTcIsmHYNRac91WtuGCf2fOD3934OWsUJdJ5a23DmCBEOAvem6oZeufDIHLGOty04+VPfvbOremwzj0h+b2dX9l7+lMcWQwCM7PY6/zqO66s6sLpsK7r02ce2b59x2SSt+K+amwrFpOs8x/fvVNrizHV2nnA3j7LYIKtQ8Y5TyiV0dzStt5gOenGQRA+o5afLvr7+Prls3kcy/nl+UQkxqO1UXX22Ppm8blmWpdoWmhaNLkpKh6EG6OtUMYz3WS4lXV6PePd2ubm4vzCmdUTjIkwaOf5lIrg3iseP//7e5KwFcdEG/fEiaPP233e8c1VS00UJ3VRSE43Nif9wWC4MZKRaCXdNB2HkSgrdezG1X3fWB7MDraGG8Y0jSXI/z+W4ANgs7I8EPZdnnLOecvX5ps+DE1BEEtEjQhBbGjsJWrURFNcjbuaqLuuDUEwrrElxpLERH81Go0mixqxxUQURCV2UARmYIbpM1992zlPu+8l5L+u4kxeXVvftef0Iwf27z79bJU0a2ebk+li1SPPbN3KtDUYrR/Ownpl+5trk+Xdp60fPcoQ1qaht9A7eeDundvvt7G5Nrfdk+688W9+smfeHjqVAKGulrsAEfukc+LOwnhC5Rid8edXRHdV90NjlrJuot3Oxz4sW1/C9ZZy5FrIDvCg8G5YuJAGu7U04AJnw97ktZ/T5DtD1we7w4XRZgnbBivPeMHep//mcx7+iIfd/ebbn3fg2B375njjVionw9pnr7p8+e03npWngDhQ3Un9xfkd6496ype7yYxkbX1CK7+1sftf9jKdnKwT/vwbHxTMrr9YZjEWPX7yxHCeUKCuqrpuYuxKyaWDKY4k4LDfJ9c3QLPpxsLSNsmzJBVgW2K0hpSVtZcM1carFhCJIbMpMUZrLREBlpzFOde2U8MkIswmxlQ1FYEz1paSCL21JufC5NpuI4QMnI11DBhCrGqWYqRkVy2sbRzpVz5Fsp6NMchGohOdOMegTkBns8lw0A+pK1IG/WUVk3KwxmcVV7l2Miu5bephVfdCmhg7P50c87YnWpCcAsTU2QoNDUuQLEch9wFbYxoRRSxZlYlKyQSoAKUUACHCzY2pM0KKm5unQo47tp9O1gmWydq69/XHzv3m7952SWjXl5Z2tG1nDKSkzoNhD+qzZGaSkkXEUgU0+s4PVr///XuYQ5zK8fm9k+nc4voP+zUY65bnw4tf8KgY49zcQNW2HVa1MYaIeNqOPPtsnU5Hv9p36tZf3BGCgHKIqiAihcmQ2AKSVVNC11RL205b2r5rsDjX1P6OsGd/Oc1Y/6zlu3PZrKv+1m3zs7Dy5je+8hnPPG81TBqcS2VWUSaGY5NJN+t8IxuT0dnD7SfbCdbNwYOHB/1l60G034bpoD/88SV3nnndnpQIpN0zv7Xp1UfbtX7lJnEWW9m25Jrezo3NdUOBFZPpeU+bm5uV77mG/v2CHz/8hnOm7Xh5sAQZVnNb2XpzsmKETm5EV1qoquWlwcb6SSBbtLIeKwNqcOPEZNuueVDfhc3JrLNEZ+4+bRrjkjnz+l9+VTrt1f1Tp45gzfH2C48dWtXxsVayhUmGrahatEewg3b+vq79o8YTsPjMty4/922HPiODXSiWZKaUcPULsus1WLKOD1Dx2n0XSpDhw8zieWrnQNpCBmMk60xeoemBuj3CdSU5pvZw1f+lwqlXuf/1f05+Cuxi2vVb2JwDxz5Swv2uuOQ711w3r7RV8Eyut4gfPv65f9FK2wAA011PO7L9H+sFv30zdHjn9z7NjDHNum5We9e2ser5GMk5IGRr/XTaMlHWNoS2NnWIwVZ9gQzFFJn2Blu6MEPJpSRjuAvBWSsiddUXFdFkyMcYq6pCxKIAUFQVgAC0bTsAZQZD9l5Ziogaw6oqIpubm4N+j6yzxpUURbMzlAsImBAmddWActsGZjXGEQESxSwImcEg5HuO//L0PQ+NHQhMnatB2Xkb4mZBk2NBwMoZ53tZCUBjlxXCYLArY1fXO4REczbWsfFdtFhWppOAZd0w55JXj+5f2rosJa+uHfPeN/V8aKXopOn1Y0xYFE07m+pg0EeumaALm5bmY25F5CNnfeN/nnx+O1vr97eKQpGYulkpCQ2xYUSDiExGFVSwC5s//unGDT+8J02idZLC6JYdLzjn8Gc2ZKGq+lv86pWve8ok5sm0HdQ1Gw4xICgxWWOyojE0Wh9XzheF1bXJz35+x8nDxy1jFhV1CpALIJmUiooWUTRmuGVpbsu24ZY9i0tLzPb/27+tqgZ//JAwN+y99MXPWFsJs3bt7AcMHvDwZZXxloVtO7bv6mU6tHrcVNlTvzdsjq8cu+CMC9ZOjA7OjlVVb9y2XnDXzjOvfdB3T//awpZmeaE3d2rtpPW2nU4Tux07tt1158/37N476VoV3pzmSR7bpu5m4fzT739kbU1mJ295/OHzrj9z5+JehPWDxze2LO2oUKK07Wa2W6qThzfQM5c2dtIBLAz7jjB0Y7G+Aj4RVrfXvUmIYtiAyTJF9jAp05hns4BFtDfANf3eFzedn5duEtK60TbhFtAOaVdRg0u/RbAqazciV1ec87fXHP4c2IG4hgSBGcc/kriKO16i3SkqQUwNa7do93NozqedFwnUisWylkwEU82iKs3qDRYzOpvCMe5OvHbLc69Z/5CVSdGgOoLBU2R84Ionz139lSXFnsIimC3kdz306ddVzc/rmCbo9j3h9jO/cc6oW11u5vDO7/39dDrOJTZNPZ2MFMh5VGUGjilUVUVESKDApeQTRw9u2z6POABUZz0Td2G2sjJZWKgUXCkJEURLiqXf7ysAIBgyAICIzFy0qJau63q9AQCvrZ8whvu9+RyKonhfqSoiTiYT5xwStrO1xaUd7SzXvskyDiGoAiBb60rJoGCdBXKIgIAKQGhH45V+3d/YWKncFnYxp2R5SI6Kzsaj2ZbF00qOQF3OrRYCNE2/33VpNLlnYW4Pke31d4QyISggRRSMdcVGjBU6wwVCt+Ga4WxjbTIN3eTw4sJWBDMajabtiYX5HWxURAj7Cp13Vdu2TV3FlEOMztsUo6p8YNeX/nDf5QBl0DOra8eqylqzgEB11ZuMJ/XAhpCqqgdKBaPl5vCx2T9ee+t627WzpGju3Pmcc+/+rDdrvcU9Z53uXvz0h0y61lhDKgC9kDZKKb1mURR95cbjqbMU89SansHKOXfgwJGDBw+MJ936RhvbtgioUhEIgJAUhIoocjPctZNj1Lq54KEPHm2af4cHI8BX/uK1w5Xvi5AWnzAtL7unPWPXGbv37txz1ujEKTD5yPp0ue4Lyo4ttbMNQdWmyYEThxaahZPrh258wolf/855MYbKw8amLixtKzpLURfmtx8+ftcsjKLIcOimk2jY7W0W7mpXpnFmPE9T2Pe4kw/91tlA1drmiZ7tLS0tHFs9NGi2DBq3vtkaTHXdrMzG27fuqFI7nkQ1rl9jTXhs2p2czihK0/dt2FwcbptsrjbN4ups0vfNoWP3qIM9/cWbvrxx977YX3jA6Mi3iVWTFt6Dsol4WtExNI+THRfj3e+2MHnzGZ+4Ot6h47vQ71ELWsRZV45/DoZP1Lkzy3hMVQAa0OYh2PhG7j+ctz208IAhSEzonR2tZeOxN4/heH/1exjk1xa31u2hr4f9Zu7StHETxxHpASjmjU9/4BVfrdAMoSwZXpTU337RwbmtHz9nz+ljkNufdM8jvvYQW6/dfrTD/Td/opRijAkhWFt1bQRMIIoEqhJjUC29pingRcpo86R3Utl563jadkQF1AEG53pEDu4TY3S+EikKoACWDSLmnEWkCDIzABjDpWguYX5uoRTIOakKEZSciG1KyRgjIgxQVFxlpuP12EVjfdPv5ZykcM6prr2CSkEkMOxiCs73kELsMiLEONvYPLFlyzZvlxIky83axkljO0e2bbFXD8nAdLoynNsd4gyEmBObOaCkSqCQQ1tV1SxGhlzZfiutt3Y6iv2F5dxNmSh2436/2RitdGHT2UbFO29U1fleyTmETe98iqmul2Iez9rirKjCX5/xtT868EQEOxlvzA3mpbDveSkJNK2unVSg4XCO0FnrsoAlW0x53we/nKeznDkL/2z78x9y8jPSHfvlnSf/5NXPufBB51hvRPJ4Y2NheUsMUtc+5VaVpKSqqjdGa/1mUQVEYwijqtpinK+afgx5stkduPvArbfc1rVJRXKSIBCKsqIKFEcLC7vHwTzwIRecd/75/+v1r9o33Rp3XeqPfNcf/RaBR6ZZqBd3zF7xgksvfsR5Z5324BPrx0ApSTDaU6MGdd/Bu6nfSMqzHP/+rG89/ocXHT92y+a03bLUm/P9nXNnHDl+296dZ9d9PnLy8KBePnTyQDGVrerStrPGzsbtoBkcG6/d+viDl9xwv7uO7uf5waKdG48m2cDiYB50DmXd+eFo1pY0MmR6vYYINMKp7uRu16xpWhmFpu5VFoyvuFiSUPnBgdX9FVanTq71hv25uf4XP3IsbpgkLUQoMNUSC++EkgkHAqK0A/a+Eje/Buvfv2Lby64Zni2rPyFaAm+wYIbCxuLxj8muVwsJTzuxpLaC0UE69QXY8kJc2KXG2CLRGpyui+lM2IK2A+8xrl3TDq/Z/LxN12+Y0/gB76pW3tQe+jFoH/jJVz7ltqu+sktxt/ISVkt2mB9x+Z/Fme0vyuozN3Z/9dKlfnvn/lvx1m9/mJkBSBWAhMCV3Fk2SUWkEJGI1MZHEed96DomE9sxmmLdApJqjlIgac5xYo0jsnXdCykzo7FGFBkxpYSIqsqGmV2MMedAJjM1JQtxVq2YMHQzRHBVLSLW2LZr28mK9XVV1ZozISK7kAKjIPl7dV03nU0bZ9ggKBNjKqXITMUyuZhz7ZeQyqQ9zLhknDZ1b7TZpnYymOsjIZApIU7CZtPM9aphDDCLxxiHxNgMFnNsZ7OJr3tSTC6nhnOnd9NJ1Z/nupHZak4jKtV4cspX1nA9m03ZZOd6hNyGKYMDmHazaJpGSgpdt7y4ZxZGoUsfO+eb/+3ux4VpnFtYKgKgkMuY2SJ7JouqgMjMMSaRwCDke3/1sa995COfq1xd12by6Dc0N7x9Y8ZFuw++9/cuvejRBJJTsrbKmQRijKFphpPRar/qh9DaqpESAYFMZV0fpFUmQNBSRNXZiskcOnD4jjsPHrzrntm4UzWpQFSQzNTfnfP0tLN3Hjm+75MfvZbRMlM67dF5z0WWzeAH7xVI3vYnXWhjPO/8+T983mOe/NhLuqQE2ip89z9++otfHoNqcPYZO48fP/STZ5143M2XP+bBp9myikZnk01rdNTRtuFCCWMkH/N4NOsGg23tbP0/7vj5cGHYjjdLz87h4NMPvvmFv7qsDePj6xuLg8FssmE9r05GYvsaaaYRsizOL6yurqDriHtiyXTTKHTs5Mnh3HwGWe7PjQtJFyvTGuLD69PN0eisPXtsSb3h8G//7GfaQc6gWkRbTEl4RxZLMAXaI2UDtr+Omoz7/64YeOu5n7xm7QdFJoYbYSC1pFjkMG7cpac9W0engBXAgLV47GuSNnjrb+bhLiOlGKLY8rSTHhIMZXQPnP7At5TT3vert5VyWOKPyFDME0ZIUqN/3lue+PO3XadI5yjtQrdE1fbzH3dlNeQF4+54+oHzrjttW10dZ4O/+u5HEdFam1IqAv9FRAyr3IeZY8hFOmK0plJkBgIgsgZBVDXGFPOmp7rXazY314BSf7BlNs4xnzRUW1MpWuM8I86mI+MMs51MOqVoyVXe59QZ70sB57wASYmOOYYoiMbaUjL+J0JFADXGhNC5qhIBVBXJigEKpNiWEmy1wIAlp3E7GQyXDLGI5BhM1Q9xIlC8a2pjUgFje0jmyOHbtm7ds7JyuK59r95a9wa5ZIVMhNbUKWURdd7NQiDTW1zei24R0YTpRp6eUqwq12ys35m6Y0USAEsOBoGNCVmZDYGilqLMjG0cV26rtfj+Hf/8mqPPKVQM17l0SC1AI6IgRaGUklDJGktAYCSEBJK//+Nj//3173XIIBouvcJ9509FVSG/620vfdyljzDMOQFgtN6XTMTC5LOAr6uSO5CAhKUIkylFFFlEVBXvRcYaK1JEhQWN8ceOrv7qtn137j+0sjYB01vYMjfaHAHwpz/zT0kto5IoMRUEgzR99OuZTXXTuwHA3Kdt29Pvt7dqlh958QNvueVwNzqKih35hcW9Naz/9Fm3vGfxlWcs7Ri6CGoB8nQaqppQiQByijEGayh0LZE65yazzhjXdmFlfeWvz/nGC2+/ZMk1425axMwNl4b94b5Dt9928u75heVt9dzq+MTmZGx9peS8ARRZG21OpYBUvrKHjh3vzc9XbEfrR62t6rm58TT0+vO2gtE4LHv81F/eEccS8gyVQTXlKctcoL0gx5i2io5x/rdg6UJd+SRt7LvqgZ+7YnwHpntIvbBTUQBkY2Dt27BwiRJrjoimoFKc6anPqL8IdzwMjAW0XKYSOwRSDKaDPLf01vp+78DD7p4vx/EXEb2ku0mLwEL2L73yyfuu/uJJMFtA76/cQ5570LO/t9T8cDBwt//m6vJnFwb9RssI7/z+x/N9rLXEVu7DzAgCACIC/4kQy72YLLBIESJVuBeVUgz7Ii0qq4L31XgyqlwznkxL2ejVA1FwvoeIIqKlWGdTLsRWpAXFe8UYRJGQjHOgaNjkHBGBLaWoROC9z7nEWIgASVOKbbfi3aDfDENIxHWv1wtdK1qm3dgSO+dDKpWrckne2rZtCaP3A1Fc31ypK1c3fVFUYKZC0JNS6rqfZdy20dgKEbuua5oeoCJqzoKkxvmcEGHiewvON+ONjSBTpbjQu58jiLFru84zra3+jMzWpq672dQgkDUpB1/1UX0pm10If3X611978rkhBOecZFOysQ3lVHpVVSSR8SV3iHljY8XbKsbOWz66Ep/621dYIBDtfuMt7oa3S1Fieuvrn/uUJ15svfFukNKU2IBS242qqiFjFECKxHbG7KqqUlVEKjJj9ikKghGYAiAAIppYIiHFmJqmDq1Mx/HOOw7cceeds/Hk5lsP/OiW/YaMiKiiIgGjAxJUAGwveiMi2sM3uSM3dY/+33rGJbv2//3TH0EvesHLXvzil3LX9pd2PvSiJx3d9/0bHv/TB3/pzFe+5EkuHj/3nAd2YdLvzUnBnMV6XzSjYtdODZNIUVHnPCCnnInNB3f/86uOPFeKxNkkSUYUlZAEQ87j9pRkiqWM2lGbwtpoZGtnkArgqY3NfjNXN+7k2jGBZt7bcczj2cpcz55qY7/Xi100pGyqm76ysu+nY1UpkLUkgaDZZH6k6jGDs1L66hZp9ysgz/DQe96y9wPXVFE2TgBmQCY2InovJtTVH9Ouy8p4RYkNBnFLcvIrPDusW56J8ztFBa2XdgyzqTehmL5gdeXcBW+Dw9hbdrc+X7o7MdcFSaigvxp985ZL/+mqLwGYhwA4h374gMXLnvjuA4fD2gvS/b+xHYgtEt5+098BQCnFWivKdJ+UEqECQCmFiJAI76OKWkpRYUaRoqCqBcGKJCnFOZezEpqi2TuXc0AAIXRsSxEkkJjZMhIhsmoCBGe9KpaUiggZjikhJBG0tjbkAXPXtczkfa2gRBhCh4ggqAJAIpqIfMpRVZyrqnq4sXaqrnvIHiBKLkQgpaAhAF8EiAFytNZOpzPnq6IzFVfXdWhLypv94VCVS0ZrLTGoltlsVlV1CF2/3xuPJ5gyubruzauyxIJ+gtKsrx+pK+71lscbxzZWV7btvV/qpqlriRmxKdqOxutNf2AB2boP7PqXVx97JsnwxKn91svc3BzQ0FmvOcfUERdmC8CALLHNECdrI/LVo5/8P40iIbaXvMnd8HZVROTXvOIpL3jO49kRcWWdzaljcm036vUGMQEzExIrZp3mnK21IgXAxdAiqXNGCquK9xWoVaNMJFJEivX92EXQBCoH9q//6La7DhxdvfHbN92+7x4mIgXSJECCDIqKioB6+qXh4jfk0y6G+zz5J8/40v/9wrNf9NJDv7i1LeGyJ19+209uPv0T5z3x5sePjnzvUeefOze/1HVj72rRkFJxVZVyVIWq8rHtEIGQkAgAsgiAfHD7ta888JSEYAgUDbMHEUSTwgTBkc8WmqzpyImjXYpLvblhv5m1rbXVPaeOp7yZs0y6qXd6alSmYbrY79/TBTRhso6Vm66trUrsf/7jd2m2KkmkU1UpLbqnxnw3lnuYzxY5CXvfjQjlxKeumn/cNW6XaNI8BSRAAgFABLVmdFtaPItaUVKUVu085mNw4lptLoTlS6CucDZmS5BRwkhQDffeMv+gK/M9RhW4hs0P89HrCgWTh2nbXwiOrvyNW6/+YgdQqc6Zarv6Xb/zin/aMZ++cM71v/at0zYm09p7vP2mvyulEBEzI7lSCgAwc8kdM6sqAMScrLEiQESMIAhIzIpFkY3EIM7zbLQOoFVVg7IaBJAUlAjRWRaSUrImC5RKNNamnEkUCJxvUlaJbV3XgtC2M2cqIVXNhhDAlpK6LgwGwxQmRCyCTDbEdWvrELNxzpsGSABRlTfH6/3GEiObqohBEC0imlGYLSggoRdJgGqYJ9NJf7AMikU7EZAUYgr94TyTE4GUOkR0rpKcilLK0u8PYpoS55QCsSF0IpY5pbZl8JMwklJSDL2mJzmoStXrx9gO+ouATEZjF7TAB3Zf+/L9T+DKNPUwdEmxEJEqhNnMWNQsImCdY2tLQjWFs1WW8x/1e0aRiWYXv9F+5xoiq4qvftlv/sHvPrVL0+H8tpSLSrSmFo0pgbVGIYMKFiImRFAQFUlxJgKGnTE2IQFEACD0RQIogrK11V37bt61+yzr+gBV0WRZALBuFtdObXzr+7/4xGe++JOf/cIrCBhVAFIQJKLZi64rey+B+1x91g+u+O0H/+N13/jfr76iAO7atfAbj7p44037Hvud35yX8TlnbitFATsiV3JmtKiaUsiSvXOT8dRZqwiISMwCqkk+uPfaPzn+HAJIMfqmJ1IkR8PSJVJVzDgY9NowU9UYixIRK2hB0axk2bRhfM/Ru3Ztud9oulFIT6wcBk1B6mMrd25f2gainY7f+fafT2csolKCFAKYqL0kamvD0cI91Q4XXwALF/L44JsIr54e1eULcTZBS1IUkUQERS1NyohgqdFpVONNtwq9ZTn2BYDVMvfbtHUnJrji17/8mD0Hrv72hdfffaYSXTn3kKvzAXQNHbtVtnRu359nGYsxpgxz9VtXXn70rf/+EDO+W/t7rK1DveVZT7/W9w7c+oTjZ35hj7HoXcE7v/9xUFBQQhIVZs45IQIAMXOMHTOLFETMOfN/MqVkYkIiEFOkI1KmShVEpG3bptcDUJFMiMYYS348PVayBewE2NpG1brKScwK2bBTzDklY2sAzpoMkEqZTdr5+SZKsqYCoJQ6UcOMiEBksyYUiWGWNffqZZGkWlRRIdd1r2QtWQ0XJKOEMRfNAQGYOedMZNuwAQD93mJISXOuG59KMVhba6bTTeuMcXURBUBR1dxZV01nE+PIgrfsutBZ3xDZNsXKiLLNnVQNiRCghtmGMTbnbK3t2rFzPgt6X0HGlKcf2HXtq488ezLdcK6q6wYASwqCIAKeDdqq5BJTRGtsUWQYTdeW5nad86gXea2DhnTJm6vvvEOQHcAf/MHlr3rli8J0U0q23rOtUkqISESQRSE5O9ycHrfsfNUAUMrRIM7aqfcVoQPDxnAMnUhGVe8cAKSUgHshjPq9JhUCFWt9KWLYZgkljet64fCx9j0f+uTXv3IDAeaSCwoTyd7HzF74VQCYH93yzZfO9Yep168ee8mziqS9e3e95MVPv+7C6x7zzV2PfvBFvhKLWFVNAUbIAGSdkyKCKLmkELSIsaCqzLYUYcb3bv3MHx14WlU1Iinl5FxVctnYXGcy/f6AiEVlOt20TIgGkKvK55xKyZqjYZdEcyqxzHr1cGXlRFVhr56bRgxGeQAAIABJREFUTaYh5+Orq5JHm6PpyqZc/Rc/MLkTYFUUmYmcLfNn2NFagLtQtqNdojNeq5Mjb+2fd/Wdr8b7vUfGdwsTACCwFgFNjKaMj9rF0/J0go5KOzHVsEzuxMnX1V8OOx5+1WNvfOvDvw73oXe+RWN79eChV5VDaNmcOCrLlR56E7drgqxKqnDpefcH88Dr9yPnxVxvufTcG/H8X+VZPPW82ZnX7ZpBYkG8/aa/VQUAFVHnqlKK905VVBEARLKIGGNEBBFLKTknIlZQBTBEIc4MOxFUlV6vyUVyyYarlEKKXcrdgbv3n//AMzRvBQyWKwUxFosGBEvos0wJHQIZhzFGEQbNxlSIwM5KTqoAoAqFwCJJKcUYXwRKzr26F2Io2hIxAKpAKcWwYcNt2yIAINa9XhFFLYQmZylFVENKUFWNMawIBLq2tra0dTm0EyZWBURU1ZQTWweABNY6NxqPqtqX0jqaE+ysW2jTZHnb6e1oXVhZzWS80lQ9gKJiSkkhttZaLQGJ265z1rPBXMIHdvzf15564cbmaq/XpBibpr+2ura8vCOm1hkPBN10pgjGWlCRxHXtRfkBj36+iaZQDo9+o7/h/wiAI/jtFz7mja97eddOENRYq/chopxz7Foy7GzfWExdADTOV4haNBFCSsmwFwUp2TmTU2bCruuMMSklIo5higi+7pecvK+6LhBR1mTAMnPG4HvDo8e71772PT+/5VYCV1QVysLccLr7sS++/LTLLnvGgTu+fb/zLv3bj3z4xOETtccXveRpb+uufkd66QUPPndhuKQ5IIkQgCKizmYz5ywpllKcd4gYuui9b9uWiEX0I2de94oDT0FkY5HJTKezuu6pdCFEIlJVIm+dnU4m3vuSpymnUnJKsakGqZSqahSQUGLIvdofPnIwR1lcXCyqaJiAiF1/UD/3d67edyilnFUSCQQ9yzVnFZjo7OaCZ6lMze6rMvPVuPS2w2+Q5T8wzuVCKqAKqIAoKkrdSJp5jK0wMZNO1qFGPfU5or2y+IRvvexfLt2zH+5z/aHT3/bZPwaB62GEUnxvLkmLB99s4lEVZWUEU7D3hqfsufrfLhj6n5+J+z76jst/99ovVQInXpbO/cbOo8eme7fO4f6b/56Q7pVLLiUDACKIFGaL/0nLfYwxIgIAqoJIxEYVVUNKydmKmXNOzCwCyGS4LjmUEhWKt/2cCHkK2lOYIWBOQsDAkkrr7AAgqdhpu9Hr1ZZ9G7ra94vOgGrJGUkRAREJOJcopaiiUnHWTSaTpmoUidnGUIyxMY4r70LsEICMiTEOBnMpZ1HJKVd1DQoIhdAoppW1w9u33L9tp0WFTFV7G0MktiHEytmUE7GNKTnLXQi+qqytUkwldchU9Vw7adkOB5WPmnNRwwJqYheQCBGs5Rg7AIsIKsV7N51F7937lv/hdadehESqOYSAyN4bApq1Y181G+trg+EQEWPXKelcfy7kaW9+4awHPk+mKpTTxW+ubvwzgeKYLnviBX/1rreMphNEIIQco3UWAFSUDZQCxATgUFsRBKJSMqBRzZKzZPVNnXOGoogIGJl5NpvNz8+L4GRzk5AEyRgbQqjrajweNc3AmCIFtDS+5lCmvfn5b3z9F697059NO2HmhWHtvP0fr3kTVZsf/PNPvONdf/mFf/ncXbffpTGcWjkw987m6tmzktgHPfgcyGJtnQWlROcsMaUUPFMXAhAa5ySJSLHWIqKqvnf586879XyRXKSbTtuFhYWccwq5lEJE1nJK0ThXirTtrOf7pRTnbIhdTNlY67yToohYcgGRlII1PJ5Oe/2BFCE2CrSxerhN9lm/9w7N/ZRbVpvo1yxRcMnEtVQMlGAWnh2XLn5sOzOy/q8b/0Sn/TGMT+l/QkBEFcFsQs7EzFTijJshjE+I97D5b5RWSv3Yyx7W//cXfQoAvrNv8TGfevG3nnzo+rCCTN/df843Tm130hueekVoNypnZrFVZI/VFU9uPnn9PXPQe87lj/r1B23/t9nh75xKR55z95N+9MjV8epkknHfDz5eSiYiVUVkAFAVZhIBZs45ppS8r0WEmREx52CMFUUAQlURUSiqoiqqWNf9IkoEMQZCSCmZe1mKUb3zZFxKU0JIIbdh3GuGSKpC3lcp5Xayab0C1l2bxpN7dm47MyuIZhVAZAAQzaUUa7wo2soaRhIRBAQLgMSQQkSkUjKzKYLMthQxxrQh9AdNKTGmQGKIiNmmqCKBScg5dr3paMNai0jGmK5rna9E1Tqf2pn1XFVNTLqxfqBfLQJK0QqwDOYXZqM143thujptx4sLOwEZFbquIyIRcU2tmgk1paDCxvBf7vjnVx19OmKlGgDVmSamhJq7NlZ9XxIKqIpaJGQuOXbCj33Sf5uFTMlkSvHiN7sb3smIDHTO+ctf+vS7N8YTMsYxShFVBQQETHFG7FKZGjNESNY1RZQIEHE6GXlrQRS9Q9WSlZCIkYhCCCKChAZZFdGYHEC1qIp1RjAhGNWY4oZ3W1WE2UynG/XyuY+57LkqRCU89JEPeeJlj/38v3yhmX/Yp//h/de8833f+9710OGBO+7c+/G5d+Y/3LNza0GQFBDBVpVjyDmXIgCI5EUVAHLOhpAIRQoRTKeTj97vm6+85xnMICrWOBFNKVvjSymAMplMvCMARmLvXc4l52StadtZ5Zos+V7GOSIrUgyzFk2pDSX1mj4pFoACVJObdKvXfevnb7/mH5AwpVLsRVTWC2Ts7ZbpT6mckZ3nXa/TcPitgwv/9MCbZdcflvEqEgkgIKAomMwtFsxke9ytpGqRcytpDOWQzr5H7hHIc1nrJzxw819/VtBvv3LH46/p9ktpH7v9jkvPmwngsP369249enCNucSSCyE+7NzehadVX/1heuFTLxgMekdcd7TBf79s/2U3nHHL3af83ABvu/HvSimIKFKMsSl13vsYi3MOAHIpRISgIqL/BVAhVb4qmZn1XkQIgIa9QhYpqkUAiDjG5F2FSkSIBKqCCCklVQUAAWVCQspJJtNRScEaWxL0hq6qKmtcjHFz/Tgzk/VkawFChW42syym6mthgXY8WV1c2J1SdI5i1yJgDAEpE1W1d77XF2AERqbYRgVy3qcwMbbJuSiRs54IFaWUYsiKFDYUQufMIOaRM8OjJ3+5c/lM0RS61NTDJIXZMJsQ0qwbLS5uKUVTLkyOCKQEKZkMlxLruhZBlShFQszOOVUATe9Z/IfXr7xw1K4MB1sJnUJBzTlRTNFXJCUTW2ISTSpQIn3txu+/7erPhTAREUSMl7zJfeedSAXVb9lO3/3qR6ftCJEtUwHJJVrvEZ2WwkTtdGKMibGr66ECAmJOmY0rpTCDahYBa61IIi0pi6hUdU9LLkWziDFWpCCiChCypMQGcy7WVBEFs1rOh4/uv/qdf3fb7ZsM1SR2f/x7j4/O/NMXD3z9W1/91Kffs2Xrme96z8fPPvuM3Ussrz/++s3fcVzFMiViAEYgwwUUS4GStWi21jJzzpkMkeaSM7EBMO/d+tnXnXyeahEtiFCKWlMnLd7XIAJQhMjlMu2mor6qTEbv8iRTlWXmuIlphmAAojEmxmLIZ80qaJhiaJGxlMw8YCN17S558mtWNifQovBpEZc4TbXZJrPbAHcCTnH7y7U++0rtv2P0tRTWYOGR0HWFiLMAa8kARrjN2htA3MSI0m9wuoJpo8TrkR6MblnAGbY5HkZortz+tGvCASkzP4thcasxZql88QnD6y4+p29tLsV84t82/+IPtzzsjB4A/Out9cjO9+b636N7vvbrh+5/7Y754fywQdz3g4+LCACur68vLM7FDpxzgJGIYozGmiKiBYiImYlICrTduGkaUFZVJCgl5ZyqqpdzNmxFFaioACASEiESIYCW+6iqtRYAYkhsiNkqEFPJpRDTeDLrNwNQBaSuDa62miITZ1FjMKXMxhjmLowNG8J+2428RWRbRFUANIcu9Aa+ZDSmiinVddVOZkHayjk2BolIbYFkrWPbB0m5FFFwroopel+nmJhZsiKJ4R5QWwqWkkLsDNuUusrXIlJKMc4Z44gNIlk3jGGmGkuJzjoABQARlRJyzqUkkVzV/RTbD+/+4sv2Pd5wg8hFAhtB9IY9IrbdxPuGkFKOIEXVV1X5kzf+9U3/cdd4tAH/P0y/8SbznbcS1ojp2998+/LcnjAxwCOuKm/rMJsCZQROORvnU0qIjMjO113orBFnfdcFYihZrfU5J0QwyLkUYspFSAsRJxFmIyXnGGrvS86zkJumEi2gUEIX0mwwv/vDf/WFz/zzlx501s7FLQs/+PHtr/0fl3/4Uz9481Xvbwar3/rXE5c+8dw/+qPXPvsZz1o5dQTecPCa8MfG5JzFGFNKAQQpuWSp617JooQAYIzpus67WnJiAyklYPOh3de+8vAzCZGUiCSljEjKFlIGByQxFqhbmMQDO+ozpgPTguZuY319fOYZe5l708kYybmI0zAuLBujzc1xmE6mC/MLSIBoLAPk2O/TcNgfdTvfeNVf/tsN3wXpZ7xQyxGAWsQgr3O2MDgfFp/yZjn3HfHWsvK3fPpb46nbrLEJGUARCaBAO6HBbulOQAzQ3w7dGuXNEm5AOgfsomLNaLWcggJv3fWMq0a/YNtHlQyelnxP13cefLktvG1LfxbTh35/x0PPqOA+R9fx23fN2eXFXzSzmx5zz2XfPW3QH07WBO/8/keJGIByEmN8kRkz58jGCSKmnJEQCqoqM6uqSLSW19dXh8MhoFNVADWGSxFVJDIli3EsUowxqgqqREiEqgLAKSVmRkTJYr0FJBGMMTjnU8m+sjmLCogIsxVQyQlVjXUltcYaACAy7WxC6KftqbpqGFEU2VgiE+PMmjqXlpiNr8NsxgTMAFC1s5GquqqxbNs4dc4rWgYtpfi6yVmQ8F5FlJARMwKVoqKCZK0xIQYAISBjXAihrl0XEwA0vZ6qlpKM8USe2ae0SWRSSgCCalIOWnIpqd8f5pzft/wPrz76TPKoiqgWkTTPCDnnzIa6IMawMfbE8ZMnN3/1oAdcfPnTXr026WbjIAIqkC59Sznt0faGa/jgjQgWOH/sI3/ysAsuEIxWK1AI3TSG2WC4oKqxJGstm0pUiBiQUDTETkSYrEK2xqWcrWUR8s7NZtOq9ppT1wVbVQBoSGazmYqWlKumr6oihQjAVCQS2qmr/KFT5W1XvOvSc/WmWzb3nLn3xHjuqve84ao3v+8Tn/7Mf3/N826+4fhTn3rxnXcddm8d/SX/QVGr0jJzKYWZc0JjTClCRAIJEVWViFAopGgdt7NJ1eu/f/vnX3vqt0spjnwpqe0mzJadtKH0Tb+NIfXi/k98+Yq/+tZlVf3ic8/2k1M7Dx9411i/tGPn4pbhy1/5u/Pzu1cCIDU33vjTLPVwqdeO4/59h4+d3NyYjUHUYmynY2A9//Ty6+efdt79lw8dnX3y8zcfPMQqnkWirjJu4R5BXn7T/T/w9vUbZfajYsjmMxQ2i9QACpqBPLYrNHe6xE192AN17/3xphvw4K3S/RB5j/IWoMayT/EYZrly97Ou2vwJUkO+hlhwsS+9HecdfQl1G3Mc61onrX73HfeD+/zgNv3ZwcqcuWfLwx7w5Ufe/JwfXhxT9J7wthv/RhWM8VKUGRAZiXKOCEZEkBCJCICIcs50L4CcknUcYqcE1jgAnk6mTc+3bVtVNREBGgDoulnd1AxGpIgWEQEAVbXWEhEglJJV1LJPoESGEVWiqBIxIqlqQXXMoiCKpQhiQila1Fhuu2ld9VNKuYgx5IzJKQhICqAQm6YG00MVBA05kuaSg3cV235KUdUDQsHOgitSjDUCkkO01nahZcaqme/a5CsTQ7EOVRgAAcQYRLQ5p1Rm3s/hvQhEBQuzYdGChIhMaIoUgJJFDQEjjUcjYvGu976tn33NyeeHtjPWEFEpuaRIRACASGiwiISMt/5yf9NbKvmul738Q8geAVVx88LXtRf9b7jP4B0DQHX9uXe8+RlPuvSJYFUTpLJRpPhqAYqqJkJJoWsG8zF1yFQKGONEMiLmLHXtc1YpopCJXUqJCUBVcjbGZhXv6jBbF0BfD3PRFKdM3jkfU1dZn0JXRKx3IOKb6h+/cPM7/vT9Z565+KiLLrr78Mob3vKhT33mTx9+4dPf/vY/u/jih5xYmc2/a3bN2gu4gRQ6QiYyzFa1u5f3PsZIaEQE/4sW8lURYBBRff+Oa191+FnG2JwESbyvRCDE5K0tEkVC/91/M/nh/vZBF81Nj/dO3fOtA6sf2/XorY961Et+77nHj8/a0GVNqDGn0PiqxFJYLGqv34tAUvjOO44dPHI3U+/b3/zJYFk2130M6xrb8fqvEPu5NZ30DXIu6xadYnnTjvde0x7iwdZ86GN85u/ne36IfhFLEY3cbC2zo+AW4KJH6YUPgvvge67S9sfIy2j2/D+S4ANQs6o8FPZb1lp7f/srp82ZM32GzqACShMRbKAhsWPEqDGaZoEYr5rk3msjRhNJ+zW2JJaoiS1qRAVrRDA0pUkfZobpfc6cOd85X9l7rfW+7x35n8eoRCDVw6j4gZUv/aul+9QSSmElOj8n6+YuO/A2xCNBRUGc82etLt74gqlt++JjB0LM/rI/fNPcuon/b9V/vm3vS533Tcr4+O1fdM6JiJmJJGZPRAZCZCmlVqsYjYZk7Lw3JGK21GRVds4QTMy7oom1c2jKAExEZrnVqmKMiJhzZu9zzs4xESBpTsrsUq49ukaSdwUZGBuYM0MiUxMmZwZmgCg5Z1DxzMCOiJqmCSHElBHRzJjZbIhaHju6f1jvX7v2XOe8KSNysujYmymSovnReFSUhaqCQSh8k9GZsCtiSmVZjIejqluNlobjYao6PpQh5kzMiMiIhphVicmUTLMPQckVSKlJZdmp4ygEiireoMm1dx12cAICqzYK3jRDNi4LJv+R3ufeffx1qCyqIXgziSmpKZgxEwBbHP/F+z40rtMD2/aOR4VHZpOJFROH9h88fvX349rL4EnVV36zffhuoPie97776ivOPj5e7lY9RASwnJOoMiETgmlSAcAQiqaOLjgGjSmzb3EaRUByQUUNpGBqmki+5ZizJB+cmYJS04wZDdHIt/UEiURmQiKpKIrxKG3Z/lhO9U233Hnjt3+VwJ+ypnfRZa+64OKZfftXnv/s2Xf/6d+vX9/dcOrT59/04N/FVzWWu9VM04wRTTWPx7UvTihBDCCpopoRI6FHxKZpEJGRP7rm63+y72XeVTFnIgaw4bDf6c5gHuTFw8vv+7fJpeGtU3OTw7RxsPzQeS/tX/6S1dOwZFYPlkvfjTIm5wvI5EeSWgaJEeM49qZmBs147+49T9l8+oGj+1wzefcDjy03o8e3HETQqhju3HMwcLU4/8BU5+RDRwqgPikQ+f8z+4a/scksEXFREkE5TceekN4aPz4moaM5YmZ9w+tg1Up4En75s/DEd5mmpHWmY5fAuDlqGP569or/vfAgsUI8qFR66urkzB9f+JNm50ODo/uSUNsJkChiFr9qqjd7xsWvf/fLl8bNp9f+55v3vGw8Gk31JnDHXV+yJ6lqVgshZInMgOhUc0qp3a40a8wJkXIWQvChkKwiCpgB0HuPaIQBgM0MUYnRzFTVzAgAELwP9Tg2uamqtqqp5uAQmHPOIIAOzbAIpapEMQAgZCJSUWYkohQTEakqEZmZSPTeO+fMLMUi60IRmKHKkBFRVOHX0PuQcyaGNBp6XxC7LOJ8u6kHRadbj/qFD9kUkQJzHWsRaHeLnFsgY0BAQNATCEGZQDWz60oeIxkw13UKZUCOztpRVSwFhSSZfSvlSISq5gMjelNRGaKrGN0/TP/Hn+x9VavXzblRVTOQPKxa3RQtZ2CMzleHRuO77rlneX7hk5/5YbdNz7j4xasm2zJa+NXiylue8mEAcHtun/721URxbtXk71/7ljPW4FmbTlZTIlIVIlQjZkopOT7BUk5ggESShZnx1yiNllwRzBCQyLGpAoAZSR6xL8zY+zKmOniXUhSRVA/anWkDNrMsKRQuxehOCOXj2+/7yX/v6S8uHlmUW2/9/kc//Pef+dYN//b5T7z6d16z3C9Wzkxc8apXP/HSn73/6MuILSUgRlUBsJySLwswyk083j8wNTFTVq0YI3MpIkTEzDmlT6z/r7cfeAUYj+t+8CUi+MCALni84SXXTv32Ow/d+E0e7b/14t89v+v4rLlde+pVk+3hYClUodMK0yt6okVVhW7R5NRejARI7Tbu2rV7/boztz764LnnXLB935atDx34zZdftPXRRycmTvrRrfc98djDdULUdGjvLzetOz14e/RX1vhlB/DeFW97/+I3ic/DcpMevxXXvIaG+8RQ05JL2SbmrH9In/JMeMnlAID7D8DXPoP1L8BmsHO2NdFaXRjuvm7liz44f7PQLLUnbOkJgKngMFd7r7mK3/OnLzx+ePno0qJmrpcG+7buvvu2e4tu59q/+IMl2V25VZ/c+F9v2/sqZkpNxO13/JuZ5ZwBANmHELLEGMfBVyLiPBNhXTfOsUgui8KAASClGJwDQlVBRAAyyI69mYkkVQIARDQzMHXOAaAp+rIYDAZFEYg4joccvCoUzkcRZjZD5zwQEpGqmBkCmgESqZojUVV4khr+/5qmCaWTzGAGmDABMSHCCea8qnjvmyYWrRYixSYRUVItMAM5x4IafVGB2XDQr0cjwCFhOxsYU1G2QmjhCYYh+JQiExrCcHnQqaaQCsNRFHBcHTu2fbK7SoU01vsXdqfBoPBF0QrdbhVaK4NzhZ8RbcDGQP76ma/+8a6rfPBMAqqSgUKhKj6wY0yoZvKjnzz0P3f87Gc/29rqtfJStqIMrn/2GRtXzxbjoT4qT6GddxDS9EznVa9/3TdvummmEz7wjtepIgA0TQ1gZavy3kvKdd0AEDsysxACgKhaSqkInpwbj5dbVVXXWWJk51UVQYMr63rIAQ0VzDOzc94Ah4tHfKi8rwCwySMAcs4BgMBo587hL3512xuueuN/ff8n+w4u3HnLj9/w5utu/p+v33n7jqnp2enJ2d9+1bMeesW97zp4FTIzMv4amBkaoKOmiZaFHRORiZkhsJ6AiABgOX90zTfecfC3HbvReFyElog0zbjodO78xw/vuVv2XdhevXT2def9KwA89ejtFy18b8XpU6N+TnnUKlegxYnJQi2Y1m3f9CY63Jkqy1Kk2vLYtksuecbunXv27z2+9rSp//jsjX/5wXd//ctfuuSyy+aXdHmw7xtfvceX/eVjY9R05slTy4P57Ttsvl8HoPeuetOHDn1a+PQUJkIs89T5sHSQeifrwt3UPTcPttPkGRqP4Xlnw0lnwH/+OwxuBpzB6ulQz2N7lY0PfGDlCz9w5McEPZ4+OfUPA9TkHPGxf/w/Zz7/0uQV0hCtUERi9p2qJ6yHjyx0Wh3LS5/Y8MN3Hr465QyEuO3OL5oZIqopM0u2E7xnEQUEMFMTUWUiRhDJ5Nsm8fjxI3Mrp9VaWZIZEAYiVQNEBFAEYmYRQUT2XkQQAUARSS0753JSMgImAHJA0YYpN8zsqCAKOSdmirEOIagaIJFzpuScizE651KKRVEAQEopOEJEM0o5A2UmBsCcM6gRQ/CFZFSTwrNzDlQOL+xDbn/si9/bc7RPSlk5pVQW9NzzzprodFqlm5mdm6SmqLpZnS8qojpmffjxbciBrXnKWWe0Cl5aHky1p7bt2Ieom9ZtDIUVbX/ocP3Rz3y3P1IzFEs5j9tufM45aw4f6HOAYQNr53p3X/nwpzvvzEkXjh2aWznbKbvkGZHASBUo9G686du3Pbj9v2/e2qbAzp1+1voXXvnGX9yz/Uc3fj6P56dmJtZMlzMrZqpOb3rFqu0Hdv/qnt3Puvjc977rpVXohhBUxUwRUk7Z1Bx7LkoRUc1lWapAVnDe1aN+4ZggLi8tT0zNjhsDdkXhUz0kBGJWQ2IH2qQkWcH70jtMMSIaWEIKYD5ndc4/sfOxM572jM987htvft1vLY+laJeXv+Rt73zz6//yHz6D3D719FPLkpAKfP/ip8q/YGxEjZkBQFWZXZOaEHyOEcnFWHtkBO9KFhFEhCd9dNXXr9n7SmaWVBehUrWUmuHxffdd/aGbN26YlXzjRZ/ZvulSeNJL7n3nmu5IS1rujyZXr7Kk01OlL71ixexKDzmmwo/Ql9Mzax5+YMsFz37elz7/1T+69lUf+t+f/cuP/Mntt9+3/+COFz7vymFc+sH3H96x5+GUbLy8NDfVWTHhII2PHRs8tqf/Z703f/jw59TU8UW1Pcju8szsaU50CVzL4hL7deZGNjhmv/v7dufd9NgXDVvUuhCarcpzjuQ9M5ded+QHzlrWPQWbsUKtxAXvvv2G83u+LTYajY8wFoZEzECGqfCloHZEhp8+6aa37niJmSkCPvo//wpGLngXCCVIbNQyB0fo9ARTJhZRVTBovCsMjdlLspzFB6+aiBGRJDeEbEBETs2YkNBiM1K0VjGZZVkysSNVzTkzM6EDADMTEVMzY3aU88j74EKJAI6giSPQBCIq5soqxjopTkytsTT0HhVZRMgiJFlcXmiw8RaC9z6UoeyMa0lRXADPne37tt17/0M+dA8cPjZqJtvt9qZ1K5966tqxWc46OVU4R52itWfv/qYZxvHi7fdv9541D1bP9kJob9mxb++hptXGdZNVKF2rKshsottat6p72qknLffrURyCsTi87a5DWw40DE7VFDQwiAIBOpVeFzqdzh1X/PL8m84ejAw0IYw1L5uhqoGlldO93bse2XcEDh9WR/7k02Zf8JJrbvufex6/74ZLn7f5tFPO//ktj9519y0A8LyLn1p4vO2eh0zc7IqJFXOd97z79etm55xzIuDZZ8pg2dQIAwB9LqhfAAAgAElEQVQSkcQkIs6xqiKhmQIVZJGIkpDjmJoaiShUaAwSTQWcRxVVYYcpNY69ZA2hHA7H2ZQZ21W7CNV3f/LDF7/kFT/57k0XXXJ+EQKSfPQf/33PgQP3PbjfnC/bvanp6Q9+6K/+ofrr59+EV730jUYOAMwyM5sxgJ0gamhABISGRuDwBFUlIsnxo3PfeNf8a3MWTdkXRU7RdcINb/3z4U562lT40Z7Fbee97ceXvgeedNEtfzlYt1FbbRaYHDy4evzE3KoVE5OEvpuy86EoC6zcqIEqZb779rt+982/++9f+v5rXvuKT3/sX5921tlnPP30b33t63/4+2/cs39H4Wb+7ZvfJivSeDk3S6tXTvcKJk0e6Lf6V/35juvHIzTrAc5mMPVPQRIuztbh41jNQuJclBSX9PzNEGbdzR/W3IHqHIgHkJ26ieumnnnd4RsMO1xsVDQitrjk3MK939tsJsRAyJoRyURyWbYAAVAla4z6qfXfunb/KwFQBfCx2z5L5BAAyLwrEMAMYoqIYmaqambsmCnk3CB6QouxCcF575omh+BURUTHdRNC4ZxXg+BLlUyESKCqpigaQ2ipqIgAgHMuSWZmAGBmA0bKzrmcrARJZkB5OOpP9VZn1TplQM5xmNJIRA4ePXbzL7dCys+9+JwWV+JbO7dt974SDBvWttaunnNEQL4MpIJIGSx4X/rAoonI1GKT7fhi/+jBA7fe+eDicGGhH3sTq6a6dPjIce+LENzx4cTyqF/4wnJI0KAVKiOPTFU5auoQvOYUlYLzKTeq2YAKKExGiFWWJSZvlkwbZEfIhDg9OTGzYkpV7nzR3c+95VmDZjzoD5s6N3UKYLVFwZBzsf2BhyXr2lNmnnP5635192N7nvjei150yeTM+T+86Se7dv/iihc+sxkXy8ePHVs4vvXxbYSuNzW5bt30vv3Da/74xS969rqYsTexAjFIimbO+aCQYs4I6OjXAENRhqZpck7e0WjQJyJftIpQxVibAbIzM9CMaHVMjvgERHTsxnHgXTAwRJCkJxStMsb4nRt/mqx++ZVXtqouYR6N9Zf3P/D+D36qYG6UBf2qVavf93/f9IWTv3J9/Rfso6iYmYoxO9GUciJ2RAGVAeJo3C/KwOSZua7rTqctqf7Euu++de8rnfMqiRyiJFf4D739by9vZHbb7r8/4zmr53rt5VM+sfoFJ+/6+QTJ7D3X6xlnC2C98uL9U5uRcGrFTChoTdqzJu723k1VI+aiP6QtO44+7fRTHnjk4XWbZh99eNfg+OJfXf++v/rLj5+0Yfbq3776O//90zQa3r9lfx4cHTYyN9ELvtWphMy9Zvzaz+LXt+9+ZOHY0PJqo6HBaYAT4leT9LSaweXD0O2ZFbYiQ7kepufxf27i8uk5LxHNKBz5wIrnfPDojQoT4Fc79JmBvTt1eunmb521vLwEgCkKOwDQlASBx/WwLAv6NfeJtd98x+FXj8cjVcNtd35BFcHMB1aVummc82DA7MzMOaeqyEkFEQ3RaVZiS7lBNMYOgBgok1PQEEITIyKZgHOOiBRUkhhkplI1IjlmVtWcs/OuruulpaWU0pYd+2MaVFVLBOvE42g7duyfnp6Z6LInPHXTWrLofadTtTrddqvT7vkq2zKRV6sjtggzIogoqGlsvHdITlHAHJISeU3RcUB0CLy0dLTdmwJySF6TBycHDw/v+MWv2iumnMqamcl2VSI1ew4MduzrLywtTXjnivr4Ah0fzCNyMx6tnZ3cfOo6TaOf3/Xg0eV6LDkYIZBgzpbboVIum6YpAgT2kxPdbrfT7nRUovf+zivvu/S/zz968GiSCAgpZ+UwM726LHwzPL7r6NJj9z1UL41WrysvueJZ05Pn/fynD+zb+/3nPOeynOCeO7d0J2X/roP9YTLwALp27YaJyRVnP+ucycnW1c9c119ebHVavcmepahq5LyBAgYmVhEiMiLn3GCwVFUVKJqkxf7xVrsqiyrm1CrLWI+9bykYAIJmM+73+9PT08Ph0JdtZkaEpmk0DouyhcSK5rlKmVGzwIBRQcujSwuv/t33VKEcjIWd37BxzcuuPOsnz3/s4+GdqmKAqmoGTB4wD4ZLIRQiwKhEmJJ2OlOAGUCZKaYmMP/Dym/82bHX1XUTQplUnOioHkElD7/8uqn+6Nib/jDHnRd99YYJnPz3l16zeNpJ350/vf9f1y8sD1h2bJgoNq6cRuJQhoVVFx7onhVK1263g6c1eXtv+fG7bt9y+eWX3vfLOyvqGNQbN87uPdIPeuz5v/lb37vxlnWb1vYXy/sefnjlzPTeA4dOWr9CJKyfXH5h/5pHX3DvP/31V8GIqdGMFnoqQflMDuvFraRR36opcFM4fBye90pNx+nmT4s7jzwikOZ9H1jxgg8e+YFBC11XsUXel3rk85/cfO5pLURIKTMXCKmJTatsOVeoZQBsmgbQ/nn9t9+652XD4YDZ4ba7vsjgTK2uB84F9KSWGUABzEBEnHOIhMApjwCoKDopRwQBNELfxHFZFs55xqAqappFPLKYAQISkTFQNC0BooLSk0REkZg551wURUBMCZyjJg46VdFEZQ6Wc8wD76vxuK46VZNFk4ZQiqqRAGDWCOADK4Crh02nXY5j8o5zUkGWXHvfdg4JWcF8UYzGdafblSxmGUzNlMDqcUaSsihSnRGsyVEISh8AUiDwhsOmn+tFzakoe6qWxCmwQW61ppF8irnlPbZDSnUztMHS+KEtu3/xwOODRgvPk9Pd6ckOMUbJLXYn/OjZt138g6eNB6luxslsamplKLtR6nE9LkOrGact2448/vhDDrgZ9ntTcsFFFzG2b7rhu0WL6zoZekeOALksZ9euXbVhTa9o6lF9+WWbn/30p5ZFqWJqwOwNhinm0k2aEzAwVQPTbM6xmSJCUnWIxKiQQUnMEAwlAyKFIsam9KTizMRAmdHIGSAAMfk0PJ6EyqqHhKg5WnaajUGzWs6t3sQrf+fd4xzqRjasn1u7euJpT1l765VPXDd8/VR3JuaMCGaaUgSkWA8cmSRZWqpbrdCb7BE7cqVzZCBF4YmK6ye/+Kf7X4FmasKtljXSalVf/tRnmh88ElcVW1avPb3C/v39lS96ud+4enZi5cPbtn7pvpM33XfTVumN5DgOHzhzdmHzionIIXlv7JSprMJw5YWHq83DYSw6nX5/cNrer7e9pmHOVdmDONlt71uETSs8dGd3zS8vLAylrjU17d5Ex4//KL3h2p3/Z/HIBGAD0kEeio4CSANtcy3C0yx3pb2eBAnLfNF5dOe39MLftHvu8zbOzRZE//65133wyHfMhFCZo9jx0m/76r+8f/NpE6qZCAAIABFVTcGwaZIqlGUR0/ifN3z7mr0vTymbIW67/fPOOVUwMzFBDDENvPdExOxSysGH0XCByEzV+xKQstTOcavoiViM0TkHABRKM5Ns3vmk2VQIDBHMDMxUxcxCaNXNyDuXxZwriAAIRcyBGIgZmAIwm5lk896bgZkhWs7Zh4KIVA2AgdUxA2BU9aApCRGKRuIuO4cAxCdYzoIIIkrOckxkUtcD70oGQHLGhWlOKSGAYyb2BkKEZqqiiMDMOSdBY/AmSISaFUkBU0y1phGCR1IRbJVtQwAEYgfSuFAQdY4eHe88uLC8POj3l4idYxTT71388+ffcv7yYlxaXlg5u74/OlJVE4jKLtQR+uNm0B/ffdddYDE2UbLGJhGTREEHZtkTa2Zf6cz03MREfMYzzvjxd355zjlPff4l51xy2TkCldiIqeCipXUe9A95pN5Eb2m82O31lo4e8yFAMVkwkAOnjXFLY4o6cB6dToyGIwpsvtRm6AljkzMxshAGZ8W4OeqRUblshWSoljk4EQQ0zSZZ2lWRm3FRTSZLb7rmuqXjeRyH3needfHZl1709K+e89O/WLhqxcy0CJiJqoqI5UETo2Tz3qsVDNE7Llo9XxUA6j2bwOL8oc+fdcs1e14JVKgMAzr23Jpsf+RN750+kM+R5Y8dy3t8+6lPPWvtyetnnnru7M++tOln93aQP/Gs971s8cEVj9w+5urxOP9ZG520fk233R0Hb0TE5IuglnN2dVLvi52nvB4REAwA6yTn7vzCms2n79v2hOTmRVe9/qd3PXF417ajgzTXhdAq/2D0orc9/CkVTBqZCzMh7Jh44pRtD8Bphl2jjUrirJ03rAGNtm4W736AmwNoexR771/1ex88/MnCBmKJXLbUee5vPv/3XnLKBU+fFdWs4pwzzSKaU/ahYGJmUpUTPrnuW+84eJWpptjg1ts+h4hmlrOEQKoOMSGiKZipc04kOy44uOXhoFW16+HAe8w5m1JRFCklRGy324PRkJkRgdknSc55MD0BiUzBABCJgAEUEXNW1UZBqqoyIFTKOYqIGZatUkTAyMxUBQnBDAmQMTZqZmVF2iQwJOcUzPtCBQHAORYQRByNRlVVScxZhBCrqho3wsiSEzvLOQWmumlcERi9qKiYcw4hE2Fd10TkfCvlCKbMiI5NwbEzyz6UKTaMOBoOnS/BWDUTuiYOEU00k6OSJ9hblMjOx7rxofRlq65jE5N31V91Pvcbtz57x/59zXg8P79U55pc7La8Su73h7HJORdHDi+lJIOcCVFzLnxwxMPxGA07rapGUUGN3O3GTWtmLn3W5h27Hr7i0rMj6Lp16w7sW0p5qTfZspEUFe04mI7NH3jehWd1qtajuw7tP3iwKMpmpL/41f5j4zi3stUpaeceaxaPXvqcM71fLsK0M1s1N7FmzWyr055oVQd27Tv55M2RnCzGarbK6NIQh+PjTMBkknLwhRqyL5omujDxwN03sDY/uu3QbffvSk2am5258spnblw/+5Wn/fwj8feG46WirBy7wWDcNE1KVpZlq9VCJCKTPFaJ7U6HYSLq4eFwefWqzcbw9zNfvnbfKz0XGYjMELWaKD/w6j8/RZbTwdTb0P36ss1t2tSaPHNuy4827jl2ChRa4sxg8OCpl7WSf86WH7Losaq8vhk+3p5sr1oZQsHM6BwyGyGQE1EmV3gPEhExg99+8usE0JEHyO1O74qpHavnZvr98TDCvT/+VnvD3/ZHevvW77nde5PuR+yCjQyIyHtal/LDSUemq9gz2IQRwbNeQLd8M132an/bp0iW/u+q37/+6MczDjEbcAXcW7dmzWvfctkalOddsjqmVFWtJjaOAxGJ2AkxJTMLoWByH537yp/suypJcs7ho7f8S1EEM0OElBNhoVaLaBkKRFSVnBMiKQAwEbNFdY5EBJFPQEQAyDmbZeedmSKiWShCkURUFRCYWUSdC2B1Tuq9T7nx3Eq5YedUERF8IBEBo5xzXdc+cFmWYO6EpmkQyTlLUYoyxDhCYjU2MNVGkhFyVbVGo0FRODOLMXrvR6Nxr9drmoaI1ICdJwRVkSjARGg5Nt6FJqVQtlTBsxcRImJ2KUdENckpNUAYipZIdp4HS8u93kRKGZGcp5wwlKwZEQBBU4pquQhlytmQAMi5AokQMaYIIoD4sTXfevfRq1PW8TCqyfF+vXPPodNOWRsYFxeX5gcpZXf3fY8+9MjjK6Zm62YwHvW7vRaC701N13FsGtMggsvGHlwbE6c0ylT054+Lodjy3NzGpcFCExMgrl655viRvTFXrSJNT7Znp9YPMu7aufv009bOjxabo0tnbTrp5S/cvPH03g9+vvfQwWPnPuOcvTv2HVkaBs+HDx/fvvNAoPHk1Fynlc49c2bdaevvv2/31i0PT03GczefOtHrzM3Olp7m5jYY6UJ/ngP3unOH9xz854//x5Eadx0crZicvuiCMzdv3tjqtr9z/h3XbrvEFQEgmAEABl9ka/hJiMbkxSw2I5XGdFD6daHo+SKGsvjHlf/56l+ev3r1GiUAZQKPTN/8u3/pPHbk0v78l6W82RdnnXvOKI1PuvX2lUG7rVVHR8PzBosdrG88/YWLiq9fvv9rsXOHXfTEwi8aGK+bgumpTlFQ4QO5wEQpZw7BAXvnTJWdqyV7CjFrq3Cs8sDJv79y7brF/vLZ5576rXvjaPap122F606HTV/4jSNHT0pchyRiY2SfZQRMgeemJgBgeOjQMbPVdtll7raP48mvUtmHO+543+o3fNq+fXx+lcndAN3Q7f3GFZc9+4VPa/W3PvfZm1UzAgDq8mDZ++Ccd+yapi7Kcjyugw+f3vjta/ddpQBmhk/c+QUAM1NAc6EVGw0FACBITilJVuecgZkxAqRU++BUDQCIUBWZGRHNDIkBIOeoKgTog09ZyTlCSSkiGjNL0rKoUkoGAsCj8XJRlmrofUc0MjGRIwREVMsnOAcpZyYnYqQyrvtFUXjXlRRd0VbLANGgJCLArBYdBHsSAAiYcw4AzIwIVdE5pyogihxySmwiFg1IAH0omF3O4n2o66ZVhJTGoDk4zmZgTgyJyZmKCjhGdjnX3pUGGdGZqKQMZmiQUU4IzjGhoVOJmhrT5FulD+XfznzzHftfbqBkbjBaqLqzhXPDpfmUaueLdtlTAFd45z1aNgjLAz06v5SsfuixPQ9v2YPoji8PMOlJG9cNR/WBpYWpcsVZZ8085aTZU9b0kLA7PfX1/7zj5E2Tq1ZMh5IGDUxOdya6XdQ6NmMyW25yrygsK7a6GRqLzWAwbgUVLefn91cFdSfmVBvnW2VnGtFGy0P25MoWptSMD7mys/eQ++R/3Lxt63YTU41FKzvvF44PvWvPtMYp11Ou65zuX5KTN8ydd/6mVas2+mr2xgt//JZHnqbcQ0hENB6Ph8MlBNfttpmJCJGKqjvV1HWql5qxrl69NqtV3baZXD/zlXcefOV43EzMzEkWQlGpH7rr4S1/88XfKiZX/NPbf7A/3/3QY/vvvPuSPbslpmMXXrmL5Gm33LSvLK+lcM/6c3+67gVvuOsjg+nT3t48Y+nIT6M1bR9XTxTTnXZRFZ6MnVNk5kBEwXlQUeeKjA2bI0MwM26vWr3u5E07ntj1sw3vO9A+/bqtcMLEsYfax+66cd/+7/f3MHqzMWBC8GD9Fzxv1eWXnxGqtXf8Yuuu5qQHH7pnqn94fNFrXzzxwFvwlXc8/cb3vnuvcJ0p/OHrzpa8fcMp65+3uTu3pmt6QnaMgE6yxJSCD5ZzWRYpRST46Jr/esfh1xCQpIzb7/wcIZuBmYoIM3vvY4zI5NipGQKaKgCqAhGBWd0MvCMzBlJTLIrWeDxkghB800TmkDUSkoi0O53RcBRCkbM65xABAM0QgaMtEwbvCjMRU8csWVQ0pdoxA+hgOKiK0gAQHRISMSKYJiKom4YZQmjFxpKMyqJDSCk3ZVEZZsUWs08xEyIxEANYBmBVA1AQFQDN1ipKcjpYPlq2qlHNZiMiKkKZmjgYLk3NrCZwjNqkzJ6IXBbTHNmziCJ5UXLIoEkkKiuDEjpGLwYAhgSEoAhg4JhSSqbKzv3Tmhv+dN8riahpaiBAQDNlRCJWwBiPqVBZtgGMfAlg+mvZZcTAVbstKTsOTaxTasqyoECqOhw0mp0r0ZIiUmi1RCk2tUgmANPMRQBE71hyzjEF75qmBnRFEXLOiB7ZQLKaGlJgqIeL3ntFb5ZD6CGxQUzNcDxKznnneH5J7n1g2xO752+/+yEPHGNUM/YeWKypW+RO2TQ7rtOFzzzTu8HUZLVy1an/uvG77zrwUmYmhOXlPiJ634oSEbKKeK6AvC8YwJwPOdrkVJcIERkMP77xpj898DI2B8yaRUUkZwrV5159TVgcrxk2Z5Skp69+7wK/MPXbzA+UtublLz5668NX7HxiOmePOpXpXy553+U/++u1E/x5ufiLx7Y6iiajiU5eOdle0a08I7jgzVHhkdHTryE4BAA0I/Rm4CFMzHamJrY263548vsB4NRw5E3+e4Pj8+1fwWnzq442cmxpEhXN4p31ry54y+r9D/7cQidxnJxs3z/zqkua7z20ODUcjF+x8NoHpz751R91jOiK51YXnTd77/1bzzlr9RtffMooGQIGF3LK6MhMVcy5IJqYXE7RNP3zyTe9aesLY4xV1cYdd31eBJwLKTXee30SIqokAMg5hxDUMASvmvHXXM5AoMh5NK6LUMbUhMAEtP/A3unpGceBqVJL3rOqOu9NtW7GzrlRPQi+YPZgBsjOBTM8wbGpmACKaPDFYDCoqhYz5V+TUBSiCtqYKCKC4Xgcy5ZrYt1pd41MsjrvU4ps3ORxUXYUlNEhYUqJyRFhSsLMBslpISAZEjvIUQpfahZiBFMVa2LUrEbW6ky5UKTcgLIPDoicK4W9cwxIzN6AiBwYmIlGR5iTjGKsGRqVlJsxIUgeIoCZeeccV+N6/LH1N1xz4CXeddqtYjxcIlARySmmXIvG0k8hUMrRe2dgItLtdlUtiybNVVWlGNm3VTMiOE8ghVlWi4hgBmAAQE1MS4MFz1h4Do6r9pSouuCbJhJ7VfXeSU4oFFNTFiHlZGiaopllBecdoqSUy1ZbUvYeTVVyHo1rMCrLVlF4VRmOYqc7dfjo/Jq1K5eX4/zCeH5+8Ngju8oWxXx8dsXKicoDJOdwsjfZm5n9zCnf/aMtz22aGlFibLw7wauG4XCxKJz3RZSkkp0LTK7VarMDAEV0YOmfNvzw2j0vqloTauydQ8ThcFiU/MXrv7xu+5aFHeP9RSxD1VmOC8GFZri1cevOWnXytn2V91PMnrWTsevgcxe8e9O+ey/Y9dPYO/V/zcvuNDRrGAYrJ93amUlPofCeHHnPhESEjhwjMZkBeANfyDijLzuNKnt/sPWU5523GinkKHnUHNizY37fnuXR8Z37e/uPdt/+rHNeMHFB3fRllDg4k/z+U+Qp+740P3neTzdff91W+Mymne2v/d25F/o1q7q+6m558IE3vuqKp55eGmpwPsYGQAQ8MzE7Qk9sks1UmPEjU1961/xriejY/CJuve2zRC6l7D0jEgAgopkhgKogARGqqKoAgveslgG4CO6BB+854/RzcxZminEUXHcwXPLeF0VJiGoRyQBQMjJjjNF5BlQENkNiAPMq4pwzg/mjh7u9nvOemBGU2ImIqamImISicN6BRMkKSHUTq3ICUHMWAHM+iCRmNgACjKn2rkQSACYiREw5qQGCY3bLy4sLxx/YuOlClSpLraqtqpeSEJkSETsADqE0Me8DIAoaI4hmJFZFRDAwJAJgQhNFBSMCwGRGAI6dUyU084ygCurMbLi8bGoGg1zX/zT3tbfveinREvkghkkVcmxXk8zFYHmJiioUjAiEqMr2a1oUIefGVLznumnAwDmnKqoKKAguxhx84TxnUe8DGqoRmKVmnFITSibi8bhGdinnqakpyQnAHGrTREBLsY4ZC8d2ArE1GciYuQgleE8oREZIZihiZiqawCClRAgpJ6zHolZWbSACk/7S0rgZzx8djpt+b2Jy1aoNwVccyk9u/Pbbdl2Z6oiUYxNTbBjMoEnJvCsRgwKoZDUBslar6wOJZCLn0f75tJuv3XOlc92l5cWJycnhYFB12pjK4fLgh7/3Z52gu8ZF11G/VXeWdD6v1HywNd2dXGq6HiqiDmBHtCvonD2x9tmp6q5/5PsrGT4FnX/vtwBBbKkqmg1zEy1feubg2DtHjJ68J3RkQEUgIqzZF1k5MSgAixnTxjOe2riSWi1Ero/OLxxZ6Ne2egWOj++anDxlFBcG/XkuShk1r05/cHTyvCO9s+FJ150OvzP6SjhyVzXV0+GxvTv2f+zDVyMzAqRGiuCHo8W6gd5EL8ZE6IhMDQAIUD+x7ptvP3C1qTnncOttn0V0hGygZoaIIuK9N2NmFI05J0dmBmbmfSEq7ECzIgbEJAIniEhKqSi8SPI+xDob5KV+f+XsWsUEAGY4HtfE0K4mU2wAMwDFNHbOEfqUmhBCU49UEvugamVZOXZxPELGGGtkKooeAIjJCcxKFAiDKjomteQc5aR1HIcQUgMhgPPFYLCMBHU96kxMVK1eToaITGEwPs6OWqEyKkNVoiuL0FPIWQ3Rq0K2TABgJgAe2VBVAZDRRE2IGIARlIgAFUkxe3TBFAwVICFSarJzXjACAAGKClGhKv84+Zk4Gi73F5jQOzZT7zyyz0mKwE2OhGBgYOhcMFNEEM0pZiZ0johZxRDAAMwMUYaDcafTVRVR9UWhoiaKrETOjMwISSyrGviiMM2qyoQ5Z2ZQRQBDREPTnJ1zYkbcJsdmplkYkwoAApISoBmcIBKJ0AwQGAzFmqIo6lFDzCJRNJMVas3Cwjz7oqraIZRmdM/Uti889C42V8flHDOqpDhCTS64ovTjZqR56PxEqz1j6C0rO2ia8dTUTF03fzP11bfvf6lhu1sV7B0SDcejdtUajdMT99y//1+/cWBhXF15Nvz4kQzQuuyk/k+eaIudXZSDMvRHw15ROMgtxbaj0uTRmWdNBL/ywC/cUn/37Iq37k0NtKMlxuXZHky1OwWRD945Ds4XTMFBQVRQKEpQQ3FIwMnMiAGSmnOtTlFVRXuyPbdhODyc5vcuHtnnOdQZPWYwqtWcIrI9tOEPiPCRjX8AT3rZ3VfNTM8s9fdSzidtPvva37tgeTmOm1i1emjoPRBiylGyhlA2zdiAiF2W9C+bvvOWXa90jrJE3HnXV1KOTCyCELQAn3OKKN65nGLhmNByFlVBBDNF9JItFN4URccAHHwAUjBMqQG0EyRblpqZy6IzHC0FX5kBgHpfICZRRQiquWka70PO4oNTBcnK7MzETERSVbXAfMoNE5oJgxkSIIsYOWJ2OWcAy9IUoeOcWzh+uConEQFAiIg55JRD8IgoUJsVhC7lUaxj2SrVVFRjBCAyzPVIXOFFxMy8984ZACERsjEAIITQAvAIDqR2DpFdU9doWQ2S8qiJZXAILEAGiNoUHtSUqVRTJMfOiUnhPaA3dCk3iL5dVs1weCAlGSMAACAASURBVOzIrrnVq1LKlrVVtkWymmRJni2rEhZZFDEzc87K5AxiSqkoChHRnJ2jnJPzjsCP4qjdqVRs1B8QEXuXc65aPQBIKTnn1ZJkIyIf2CTWzbKpIFIoppzjGCMicggpKiACSVmUOUcAY2YxVWVElFoUEgKlGFtVCwzNNOVsCip1VbWzRAAJVIlG0UzsATw71zTj44tHJ7uruHTOkaUU01jUNaPlTru344ltU52iM9Epez1HZROboggpNgDw8Q3f/18HX2qWlEowMLXgi5ybAnnk5IEv3rDzhruW8ritPGp5GEcCD00eSrPO/NPnpg9adqPoy9AeJ8I40/ADGy64Z8Oz3/nwl7ZJc2ww+NwgH0i5IDyebLHIvWISyxCcb3nqEoeymCANhQuBvSNgAnYZwJDAQE0UBNAEyYfWcNSMxo3H7ByzAzN1gIaogKK25eS3nn3wK/evvProxDmn7vnKGeHQsWM718+deWBp38c//Cf10kJ2RaugpsmI5L1LOVet1nA08N6NB7EsAwAShY+u/vI7Dr3GDPCE3b/8WpbGOQbDJkawsXNErlRzYAqmKllyZEYAIEJ0maC9Y9dDmzaeOR4qkjGLQUQsTDFnLYrATk0dAAFIjIKIRMgMAASQETE2/48l+AD/Na8KA3/O+ba3/Mq/3v/tM3OZ6lBnEDAPIhoFRERs2FZ4dlU0ESMb8ph1jfts1iTg6tqFREGTqMGNoamgIE0GpAwdmV7vvXPbv/7a+77fcs7ZYZ79fJghE1HKhYiMQimlbVtVAO9AUAuUnI0lIlBhACGyCqAqSCiFiPBJAMgihaNzrmQAyNa6UgoRARkAFWHmklNChLYdxaGgFDQ2DknRtGPrzNpiNWtHjargUwCAkIogAAIRFwRNBpUIRUU4GyIBcMbmPCiAcbXzIQ8dgApiCLWWpMyApKLWORHth1j5QAYLZ+sciypQZkYAK8yoQBi8yTEba5mZiETgSYRIQEVL13XGWADwzopIVdUl51Kys977UAorGs7RWcMsxoCqGmtVNeWeiEQYCZeL1Xg8FeFSsnMGVFSUswhhXbc5ZQCsmjYNCVFARaGIIJFh5tbXVClQIXB9n4TL5Scunjl9am81u+feB1ixG+LJkzvjyp677rq0ysaNENFam5M0NcRoRRFtJmFhIKQiUSDnTM4RZ0zloA113wtDWJ82/RBZwVX1tSce/s+3f/xfXnjZ7HB3snbC+iBAy2U3WWuNkqLRtvofr/lF2/eH7fb88cubJxq7v2RAGTQ977n6D//wzLXNs+fOHl7b910Xik5x5aMG79769J9/zZU/HR9ca1L91ln/GWOu5K4Tc6RQPAZbkbXrWkJe6HS6s7E5CqGtgnGKhhQBiARA9EmiAIpShBWBRUWsKKsKgKqggoqC9e7a6BmFZevw8wjgAUtdto7fcd+D9/zBr73R2d6hSzwTrK31oACoy8VeCBV+HYVqnUhTjjGmt9/0wX/22Kuc87kwPnDXH4hmRBVBUBM8DbEjYwCJS2Fh75wUdd6VkkvJoWoR1VAQZWetIZtiWi5mVTsSEedoSHPmIkzWWpYczMgHm1Ky1qSs1ponqQKAE2HmYgyVkhFBVMxTrHFEWnjISayznHMpxVqXclLQEELJxVhMKVVV422bcgcAxgR9ioh471gYAcmYknnoZ1XlAUAFQePewdHW1jFVUETv7WJ5uLl+hkVSTEgIACqxMCKZUFWsgzdV7JMjG/PS+SrGVNc1AQ45IiEIkFEuSYSHmNa2dlAlxWxcAMkKKAoKAAVccKxFtBgm56wCDJkdDdaPhY0wA2QRzSx1PRJFVQFVb20pAwB474Y4AIg11cHB4cbGGktCpBgTAvq6RuXVapFTWd/YiTGCKiIScSnZWEopjkcbOTOzIClAEVbvqpwTWYoxWWvbpi0sKQ6GSEUV+ImLV86de1rKwxPXDv/oT/+iL1WMOGe1xmgpyqwpNO0oFbXOdmkBjI7S+lRDRSdPHiNKk0n15S9ePXUifOMdN643bT1t1qo69XE0XnfWFvHGQoncDcuqcoRa1dW9X/mH0yevc9WIrW/bzd8++c6fffSlKQ6rVa7q2odgrXGsK4K8TG3T7l+69N5/84fVRr5yz0HdrqfUHyPM2kvr7REOrCdK/JYXPuPq0XJy8SqiGbMFOnznc1/9xf7cL3/6zY/WddreuKHZ+LlHLt175BOB6Mrr4IzulO4UaYDwRAWTM8fXyY5GtQ+ODCCiECqAPgmUWRBRVPRJwKAKoAqgQIgAoKIqgF+7/qdve+St8CQcLu+PF8PezdefetN/+AmERnWZVsZ4ZNaqCqrsrNnd3RuNxgCYUlRVkdKO6t/YeefrL7yqappSGB/4+NusNSLarZL1iai21ooOnME6Z4hKKWTJOQsIqpzToOIAk4oVZS7ZYFEdQju1NOr7xJxBbeGuaUOK7B3mHEvhuh6DrYhM3/feOYeFOYkOMfZaFEC7boUIVePbdhuoiikaA8xi0FhrF7PDyXStsObMhvxsvn/s2LF+lcBmQs/MCrmqmtVywVq8MwRsTSAKhC7lmTMNcykylOKMNbkUIkLjxu0UMAkzoCOilJJ3PqdkXbDOCTKKDn0ko2SERMAEEeSSDRrylgwN3cqHUR4GfJIJhCkNfdXUuUiOgzHW+ScFVckCSIZAC3OwJuVsQ5PjAEAEeVgdGB+adrxYDc5XFgUMMQsgSRJrrUhBwvl8f9RseF8xJ0BjrYlxcM4huKKZLKmiKkhhBDBEqMCcFUrOyTjyPjCztQaUchZQtZZilFD5XCIRghQ0lDP70HJOhiokIFOQ9K/+9ssXruhjly4ezvthSNb6KjRdXq26ZV37UFXcDdZ6kWItlsKglPqeUJRC1bjVahFT3ppsv+F1L3/a2ZGqme2dJ9e0Y28tAdRcikoRKWBK7ti61o+nPPS/c+bdP3f+lSlxFRxzSqlTKSlnR7ZeX0MFY+w9X7zvM2/5b91De9MzO7v7Oad4ulJmEuyE7SpJm+jb7nha3Tp/zwWH/MCNNy5fcOYvHjj+hH/e68uXb508NvncA2/rr77lPDldVywqrHg40lVQqIAj0Gi63jTu2KgajeoqGCIAJCBUFFUWUPg6VFEheAoCACmyFGNQRRjx3hv++a0P/A5aWzLff3l26Up0ZfYNz3jaH/3BL3BPUbP3rSqUUozVwgAqztuUotVibS0sovy7p9/9s+dfdTibjadTfPhT/wVBEaVIMcYacoVZOKuiaHYWCa2qwtcRMxtjEJEISilKhrQpPDfGcZpJyWQNudq5Og2c0qL2bhnZGqOqKUXh5Cy0tSup75aDtQYASimr/ghBmqY9OJhvbx7PJQFZcjRu15Q8C2opAJJLZOa6apnUuUBoVQFMJvA5iXBhztZYMoSIOWUAFi1NE2JKAGQwODf6yj/+/XVnbkS0RAQGiSCEqmQwBpl1NFrLqYhmIotgUh6QFEGFkzNG0YqocWJsZQ2w2JwEFVgzIuRUAKBuGi5ChGQAaTQ/uuYdDqvBVNI2YwQbWSwBqOlWK+Mw+Bqe4pyLXR/qaogDooHCqcTFamYcTMaTuh7FWAANmuCtEUk5JyIrDNY6AEkFvQPVwkWUAUyuqq0hLTh2KccQgooRkfF4UgoLawFBLd3ioKm8rccGbYpJQUbjSUpDKVJV7Wq1rNoKVIeui4XWpg2oNE2zXA5HR8No5IQjl26Z197xl588f/VouViWpGgKi2KM0wa+6zu/lbM9d93WsWP17rUe4+K6p53uumWokIwaqBDl8uXzwrEab25sbCMYEQQtKSZmts6C8O+fe/8/e+Q7XXAIljmpiCoM/QoBmUvdVMpxsnnqU+//2Bd+779D8BvWXNkfih+ZuFwzJpWhiNQ/9sryZ+//5uPbNzzjGw7nh/94Yf+9YecVzzn2vIfv+fKZH581sx964iM/ebf9mL3RdVeOh92bm2oK3be046P+4FHJf33pwvaJnW6+KJVZH4f1xlXOoLHWEhGIsDXKIkAIAKqOEayIEImAM2QIshZg2D32POCS7v/AxYdnmzs7V3aXfUkW9NnPufF3fv+NZYjADAhgbE5Q1x4AcxYEQpKco4IC6Ntv/Lv/+d5/GuqGjMWHPvW2+fzqdDphtsYEYwwAGwPCFlBzjEQkwohIRMwCoNZaUVYVUEKMzrRSylCSJSJjU2aRXgVVe1S13hJZUBJAAF4sliQGwSBEa62oMJdudcgZQIEsbmxsr7psvIJW1qL1YblaTafTIao1aK11vooxOmdjjFVVgXqFHNOq62dp6JpmXIWRtVVVt4XZEB4eHVokMmgMpVTIGAC01jpnhYkli7D3oRRGhJyHqnZclMgxi3MWMWvJXIY4dEWlZGCOwdZDSS400+kasHTdYK0xBq1FdTWRHYaBiLytABUQVSHHbuh6JAJCg0ZEq6piLqpgjCmleO+QiBldCCl3oEhI3gYV7fpF1y0Rs7MYc2LG8WRzPFpLaXAulMLMxZBFBCTTx1QHk0tCsEBgTcg5ERlVVE39MEzGk67vQuUNqHBeLuehDtb4oeuqyg+xIAEAOhucp64fvKtKzvVoRKAKwEUN2piTtTI7WITK9v1s3FTK8sY3vUugZlBr/LLrT0zt//bzr2wUtQ4cC3i1ahjZGENk57NF04ZSuAqtMoTal5z7fqibpggQIZEppVTe/PrOO99w6VX6JDEAknPy1g99LJy9d6XkqqnzakmTyaf+5P13//kHT55am11YtGujh3d5x8egQ6X1HBhzan17o8GzZzf2Hr68d9PJ71p1lzZP777mex78q4fH3L3jg5/98RNwA9ujtbC08q16LC+G1tWPnMgmTv5EZu+++ODRan9q7LHpqG5CsGBIDalziOgYxDknKsqI0lMYaWExOaAjQEOmyOrBS/19Z3/u2N//nw7sydNbT1xerG2cXuV+Pr/43a995S+87hX781XbjrrFKrhareSU67qdzRaT0aQfeuecIfytU+/6l1d/YLWY1VWDD971ZyJ9Sqlp15gLIBOpqogwIjlrAbTrVsYYZlaVEGpEUBAiTGm2PEoxXdoYnwRvWVRFEdFbs1p2VUWohozLWdAEBRKRqvYIAigGa1Wx1rCU/KQ+EtJQMgBbhyq1r0gVEEBUQ9WUUqwzXAqRYy7MpW1bVS05K6gqel/nlJ23qiWloQg0Ta2q3jtlAtAYB8DChUQzIZAhETRkAZVIRcSQm89XVWgUB2FsmhZACYvkpFK61YKcb+uN5WKuolXbZFFmzkO/sbVVSp4d7lfBi4Iqe+9F1BAAmsxK1oYwNYjOuSKcc7LWAIAq8FNCcDEO3eLSeLxdilFBDVKFZujjcr7aOjbtVoMzru/TeDSJaRjSsLm1iYAAoArMGoIZ+gKIaCwRgwCSpFSQSES89zEOzgVC1CeBogIXAQAyhiWDqPeOS0SDwkqELEUZcoG2HaWhEyjOBkVSVYScmWNcWggoHRkY+szsf/ft7zlYmoIuF6BguXSl62++pXnjT/wAL0oxpW4n1k6QytFsr21Hhpq+Xxojw7DyLgAAGUoxjptRiklVmRmd/d2z7//5C69QFUUbgkkxGmNVEAmZC4BWpulNrtSIr//yjz/2yAffuwmeu8GBuTqXAUxLeYNkQANDbtpT4czmK8x8I4dLbVo1px7/hvb2j9/7GXPDe7a+cZbD/KaXvOHKO56u9+zMtyY762l4/AIZO7u4/7IXPPu7v+/BL37qrq/c+5WvPX71sb2hX6mwtZYQghvVbd2OmrqtH3n0S1UC2tnprl1rnA4se126slgeHLmx5v61fwcA2x/7v87MPz+kQPWaD9XBaq/rj977t79pQRAtKThrAaXrOu/9atWFYPq+t1/nfve6D7zx6g+uFkejdowPfeo/aSEiUzQShZwGBXW2UiigZI0tJTkbmIuxBABcBFCJqO87Y21l691rjxMeLbu8vXNidjTXlIa4bNtx33fTyTEbgvWBrEMyqCSi1lgkQoMiLJpLSda2CHG17MaTYyVHY5xK6ftIdkAV7xu01eHe3nRt4qxl1m5YjccTUFIFKWAskDGApuQB0SBaLoxorCMA7volohoKVVUzZ04FkImwW/XkidCJqLXEbI1B1YyogMoFQqgABYBRQXKRLB3308nx2fxQQUjBOFdK6rvFxrGzwQfJRQoDkHUyxBUi5pxCNUIkVTEhaOKjw8PD2fzcuRtLiaUUQqsgIficU8rD7uVDX+nWsQ1ED2xYhSzVdZ2iOO8UNOUMkIW1Dk3JkvLgvLXGMOOyX0xGU1BlTmitso3pAIqt6mDIEUHhpIqIgAgxDgRFFX1oiggAeu+FSy69MFVVLSIxdZY0MxpDBgswzGZL61zmpKDr66eMkTJ0QFaUDRFg+eKXdz/95ccfunDl8HCRmUMYN23JK/fi57WvefWLWCd9BIUlABM5g4210Pe9cJkdHkw3d5ilaVpr3NAfImJKyRjjLfzO2b997ZeeL8yTY2cQpJRc+0oAkUBVc44EihREmLlvRvXBVfnr33vH8NDD4+n4ifsunjzztNXATxxcPeWUWOYeT6yNxxdmL/kntx0RfOS+KzedPnmD3X3r3btf/rZf+swL/jU85VMfeeV1G6O8t5eMi7NlC063Th6+bPOSlc0T20DFGw6jSV1PXBgtVnHaChmMsZ8tjv764w/88Zv/5J/+5P+ke+e/8OEv7y36A9ZI3irDi/+P/lt+EZ5yx6+MTLO+1LJ52832cP7QY/f88r/9qec998bJZL2LA0sKxuecnXegaozlIqronPut0+/+qQdeYoMXRnzwrj/KZahCs1jtO9dYY0SLAjs7UcguuGFIJRZrmTA4Y8kiAPT9oCK+qpy1h/sHXMrm1vVf/MJdt9x8QtQY51XUGJdyAQBEg7gyNCpSQk2pNxZWtq3yUCl2hoKxdVzN6srOVwtLFlCMtT5MRfhJ3rsYI0AWRuc8gECJLGR8YAHfVFoiczE+GDXCsXA0atBVAIhIAEiEAMpcUopGGCyIKAkmECkCXECzaxtUW1IuORGS9VZVXXCpX9Z1PcRIREg1kDg7Ye1jPyCDMPS57Jw+XlJmLobI2CC8KhFsxZJ84h4NOd8gueXsymi0AUahBCIULcxMpi75wEgZVrve26i1YiNFxVhPQAQs2kwbT6EURkc5CoGCFC5J0VrrVKGu61xWKmhtsCbEsjLGimBObB0QoEguZbC2yqUggLGWxYIIaFRJomBtAFUyMJ/vP/HE5bNnrq/rNpZF7PLW5lbXLZwP3dDVdUtgF/MV6ZBjn4pMN0YqxrumZBWK3mKopoulbSr9+Gce+a/vumtV2MAimLI1hhc//6Y773j2+ugk2hX5CiIpcSrZoCdnlou90Wgjq3rAlIszlplA5bdv+It/cf7HKXS2QJGiqEPsqzAGSWKDctFUUhnGo41+WJah1Osjq3LXuz//8Ac/Mr+2PLm9nmW4jcf3Xjo4MKuqGCY5JqYSf+vx9qMuXKeHh8tYxqff8W1v/+rO8+Ap7/ryL91x8bOQ+ke7dO74zaNzk8UqWF4tZ1f71z+rmU4UWldVfb8wpKisoF03r5v66HDRrk9f/vJfQt/+xpt/9t/9m7fIUbyAuQVDyLPXfGRx3TfDU8791avHX/3Iqq52bj8Zu7X84D2v+vEX/fCrX7BIYgRGbciFvQ+FhYwFgaJKxiDQ75165xsu/0CMCQ3hfZ94e+xXKFTy0geDZHw1BbAiR9aMSwHnqRRFEjIVgICWnBMZZM5IaMhJkRhTN1xqw5Y1Jg1c1b4flovlrG2rg70L15+7fTZH12ilbtkvq2ZiDEERVfa+zmVVNc3Q9ypobUADiI6FY1wGb1QBwSAatBT7ZIhIS8oz5/180W9tH0vZOoOIgMZx4ZQ6721KyftGVY0xIqrAiKgqiAgpgUEAJIVixICLXeRSVrOLoR7V4ykDEjrm4qxBRIkDADCzIQOEoalZXNHsXJX6wRAsF4fTtU0AJGcLM4KU2HtbDbmva5+LRfQKQmhEEpFFhD4tQmj6LoXQzHbvr1w9XxwMHS671anrbyTjgjfGT1Sys1YAY+qC88Y4IDSIqvCkIUbnAwAgIAAMQ1Rg51Ak12HKzIhgjIk5c+EQPECJg/jgVRQJnQ8pFlRRKbl0gM44X6S0vs45G0s5564brPXOefMka1NaLuZ73pLFIDIgKAuUUgCwrhsRdXVjAIukxMnSFCmrRE79Pz6If/Gev7t0cNWEegPLq3/o3DNuunGxN7eT8dqoFrGj9RPdbIWa+2XnAxYdhtLHpFXjj2bd+7758Z999OUqFFw2hg6PDg/29wrh5njUjjbWm7YHS4aHlSDYg34IJloxkk1f+r/4t3+0euLyzsjpMt5g12e5XEpFu5hqcyJsXX36Of+1z5tusT65bbs62n/2q//F098MAM+/8sX/9oEfl6EaRthrgHM3Tl767cdf81J2M824/7m7F8Pnw2QzUJ1SElIFLkNZm64dHc2IXD0q3/nSNy2hfPCDb/6dN/3ZJz/4QG/mBGFdUM6+6Cuv/QAAjK984sVfeu0j+5KuDKiwfec5uVx+5nUvesatpzQ4zYN1IsWEEAozKCA5NEZKMQT/6Wl/+xP3fzuZr8OHPvVfQLIld3R4laMxXsmZqm6vXd71lfrgyATNfd2EUpCsxp6ds8ZQSrEO7vBoTuSeVLVjLhjz4XK529Zb3tVEDtSw9AbM4eHXps1mrDbX2/HB/gPTZgc9EIaUssERAyMhGVDJ1jiDNsYBsYC1cch13QJQ4lSH1pCJsQfBwgJAIuprY0ABUICILBcmksV8NpqMiTDGWFVVyoOIcBEyBLkU1VBV3WJugzXkg2+YS0oJNO9fvdzWVbu+nmMctU1OMbMKs7U2l2IMiVI7aWIEsrRcrLyzklZtO3XeKWEpxTqvWVUyKzKLdaHrl3VjcuwXq2E0rqWoQQ6+ttYPQ2KlfrX0tSw7OLF9mlWIIKeenEMQIgNoALCkzIUFiVBjTE07UiSDTIQ5J1Wo6jYnJvLCoBLJgEgREedaERBR54lQnkREpeQsR9Z4EUAVLiZUbUylnbQcEyIMQ5dLJnLB+8IFVJ2FkpMWlSTocNmv5vP5xvpGLjxqW0AQKcyRTGXIWoK+u9aM1qwd5cEyH7gwfvz88OjFSx/7woWrl1cpL2ZHu+r4zM6xYXm0vlm/4DnHUuw21zbPnNqc1NPRaFSYNtYm0MU3nXnPj3zpef/vX370wsIv5+nwoJ9Ot48WM08p1GFj7NAP0MOZ7fHLX3b7se3x8bWbF3HvKw89sD0+d/b46D1/+IFLf3d3s0Yh+gt78x989h3Li1c/efWKHdVpJN1ufyls/8d733fPn3908e53ndwY3XP5+AseuIt8Ay5+9fjTbvrnP3rspd+QVg90//W9a7d8x/C1r8bLX4s/8j2VXJnHw36gdrw9Gk9zWRFa56r5fP6FBz79b//1+8nZv3nnv5/1137w+39tguaayAh5S8kRHZ795vbiJ/K6PXPz8f2792vCiNvT6WTntubn/9W32arBglmU0K5Wy4319eVy0bajfijOWZXyBzd/6Ic+9/z19fWhH/Crf/eWdlT1XQRMWvrdvUvT6TqqZxlGow1Al3IEskPfO1OLDs61zOycBYAch9F4rKjDMKxNxn0sQArF+oqYC0vJOUnSYZCU4vaxNYfw2GOf2968pU9dF6Fuy8729fPlQeurrIZMQEVX2dXysApVGlgJzFMAEDWRrXJR630pnTGGkESULGnOCgTkQCMClZJ8IJBKhAEAEXKOznlEAqCUY2ENwSuzihASEABmEF11q6oe+TBWYBWVnLgUsEiIRGSsJSLJcm338e3NU6EyfQQuEvvVZLoGqCX3B/u7442po5rLgkwN6HMeECxhJaUDbF0QzYiW9g92x+PaeTqalUvn73v67Xeq9WRRSjJIoiKlADM5n1lDFY4O9zbWtxRMLplFfHClFE0KqNYa4VKk0JOMUQEyBgmFmQxJjszgvAMCkNx1nfceAQ1Za0LJRUmYkwpWVZgvjtBUdV0zl+C9Akjp++WR5DLeuK5wRIJSBDmHpo25OGOZi4IaMoULZl4MR3VluS/G+fmyW99oY7dgBYPWVS1DUzvwDSH6Ydnv7w0f/vTjH/v0V1erg65Pvg61n6R+seJoUXeOYePgxnMb7/umB170oVu+9I89UUGyimbVDSilcq6arHMZQLVINFTFlIcuTRre2arHddN1R0ddWc2Gn3ntd3z2t97bHB7sqdme+huTfm0R50xx56bvPb5z+YXb/SOPrD393G13XV1eeOj6Wb3C3Ie8uvOFN/zaTzs6vPbv33TpS6Z6zXeMbj8xPTZ9/H98qDA/6/ueNz+4EiPUbYvILtQpMREZC7tX9MMfu5td+yOvujGE9Vf/5C+l+xePQN4QXFe0BBmgAV+qLGrXSyCEIwo42TSj7d/7zW/PWmpjTeVSwSeBSsmJOKtt6nZSOL3l+ve//vFXlJSJCO/7xB+noQvOMwOANk2bS4kph8rnnEDYEKEBLtk5q4wsS9CqcBEYQIsPnhmInLVu6JYA4KrGkPG2zaVbzldVO2mbermcCRc0ZMintAKkKlRcFiVFZOPGbYyxbaeo3lggNCmXnLP3yEIKfe6o5JWrq6ZZT7Enb1Axp14hIYE1zexoNl2vENxyudhc23n88YePnzg2X6zW1rZKURQEA4UjEcYUQwigLUMMXgnMarVwxhnbWhtW3R4gc7ajtlnNj0BLaEcIOPQFKIMM9Wh9tRpA9PxDXzp99kwBnK5vdUczY4OoeD+Zzx5bWz85DKlPy/XpxjD0wlrXI/LW+AClADZS5iI2l+R864NVhZSidQRCxqKI5CQ+hJyzc857P/Q9oFofmJHTvFst21HjnMlsrdUUI6onK6UUSzTEhXUNIuacAaCkFEKFYI1xomgs5pwAhFmN9yFUwArAHjX9rQAAIABJREFUIuKc46/LxhhmBoCUe2udMKScq8qnoVSVU04KgSyIqvMVKJaSkVCVUSwZ5JyJaOiu5gQh1NaSCU3fRQCsW69iVQVJDo+ujaqxMUUhXLzUdSx3332hgDl/6eoTVy4CkXDljfRxOP9TF06+dYtjDzYok2IRRRUkFBVGtUAgIlwYEZlj1VSKyMwj8iuJFbi92SKstc/eXZTV5Wes3/C9zzprl3v3Ts/K93+X35qOfuNP7fJwDON7rz703NnGrN7PB93sp3/07I/+k+Ujn776xv+8980v33vmtb333H3mm67vhn08bA/uza/51ZdH3jAmsBY0JISYYlG0VpEmPnhyslwtjvb7T37hq3/06x/ch8ELjoEqACJYU2oIDblBjLnuutFtZ1Jt+qPFz33vc64/E4Yc10dbi8XlbpWrpl3MVyfOni0pc/YY9G03f+J1D34bAKfI+NUP/YfUxfniaG1j5P1m13fe11VdqRhrTY6F0PRlsEb6fkGopNTH+fraNnPDeQmGnPUi5IM3hF2/tM4hWlVOg1gn9agpuQCAinIpIjydbBlDsXR5yMLFkPqwGdPSWBFNKWNV1SWXum44FzJjhayaRC0QFM7WOORMBhUxRfGOFotZ04ys8UV6EL9cXWvrLURDBowlRZWShphCaOq6iXGwlmKfQqCDw6tNMw3VpBRUvrxY4Gi8qdoY1z366H03XH82JzbGMeecIcZlU0/I2pRLCK4Mg6o9ONwfTRxKRSTWNtYhEXXdqhmNDbnFqkfIOa+auvI0ZpD77/ncmRuur+s1Zqwbp2pW3bKpWyLbdcu6bodh5YMzxgAgApTCROScE+V+iM5XpVPnCIgPD/fqOqiYqqkFMiksl6u2bfZ2rzbtJISgqgBA6gsPPtCqm9fNRIWRQJRRnHWeRVBBVEspItI0Tcqdc05Vh2Foq3HJkaUggnFOBY2hUjKCAZL8dWKtH4/Hq9XKGHTOp5RUiqqikrXOEKUSrdGcsqpaSzEPKXEVamMMqFNhROmHVU2VwN7u3t7+3vDZ+5qvPvDYYb/PSoaq8//LI6fecpwjowMyPpWO0HHJoGKISmbnHDOLiDGmqBJoHHpjIKqGPkeLDqyrzQDVayenf2xz94P78PAy/OB1pz475fb0aOeuB49XowfOX3j6s58lX/781cPi3/yvRrdUD/39n8//4OH5G37gOc9a/7NffOtqRe5ob9rjEz3f+doffMddH//Gm7eec+fWtBl/47OemcET7GdwkLcWw25dV8NSqirUVXU1X/ux7/51IVywjhHHiA5krFQZdBvjzWfeUAXef2R374kjwXDyObf+4a/95JVL94fWzGci0ned1O24sm7WH9WjDYvhj5/54Z+6/8UpQb9ivPtd/06LIoKtDZBjkfFkWkohg4YcIQ59NAGkoArWVfXIg/ccP34Crcay8DQRUCLvXOMdzxfzpmmYczesnDV1GAOkg8PZxsZGYWEBbynlWIXxlSuX1ta3vK+sMUdH+2CGcbsJEkSM9SoixlDfd7Gf2cqNR9s5gaIaNGiGnAxBFBFjXc5iLS5Xi7XpOqIBkWW/6906kYpmaz0zGeOQvCEtPIgkFW8MGeKhz4eHi9NnTh7NjnzlsbgQQuIlWg+5Go2rrl+CkAIPcdE2a8Owcr5C0Fyyq4LkZIxnzqIR1CJIKcw8jNrNmLIxJpZchUY1AxRnXR9j1aw5kEWfHBVrfD/MCR1ZBCUR9d7HVKqqWq7m3lsEYGZng4gMQ9+0NYsoGKKYUmmqCaJFyQompRRqKykhWQAqnPQpIoKIQ1w1dSOCItA0bSk5p4yIpaRQB1WxZMhYZi6lOOdEc7fq8EmEpMYZZE5x6EM7EhFrPTMQGkBJMTV1y1CYmb7OpjQYYwjBGBIA5qyqAKhFvDecU06ZmBBBNIsO4MjaRooa4IHA0RSxK8kQdu3G8T/5k89n0b7v//udH9n6vZ0Mw7SqYwEg5lSUS2FRVWtNTskYA4hcCiAKCxGAaOFcAECUjawNk1+4Y71+7PyHlpOX3HqdXy3qed4+eRppdX65ynZy7trFI3OUmxcd+7Ufgqq/97Nv++qvfuXSM7fOXdfO77/34mfFPb9aXRY8v3rG//79o2H33e/Zf+Tossnexv77vueGO24/c/b05mzFw/xoMtnyTb2+c6ovmZL57H2f+uVffJ8IrhSccAtoQaZkNk83t1y/nXf39h5dPcFm11ipqmOnT7/911+FKcViLWmOKVTmaDW0I+9pIxd2Vv/jjR/+mYe+vXABQ3jvh39TmQ+P5hvHTlmLqkBkcuHCbK0RLipinZcioXIl9cFPYj5kEZWa3GCMVbUspp8/PmQ1xrW+8nUNGoLXxbwjRONtKWy9Ew6E1MfZ+vq6sBnSvK5rgCBSEE0pEUmkQNs2i8XcOUsYFLVk2b12/87JYwfXDpmPxuPTSmbUbqbY9cPuZHKdpWqxnLGshk5D8Jvba1xcKVFRVcUGq0L6pCIpJ6JMVGfuQtgwWETYGltytnUbh2gsgPocr3k3KiWlsmrqLdGYk1Z1VRIP/SpUVVFqfJ25I3LCQXFRkubceWeBFSiJKmLgtGhGE2PblMm5zOo49uTtajlDsJYAwdraAqAIGwtItXMhxeSD5RQRTSlqjXMOu35lnLe2YhFQARQAGbrBWQUALSaXpXe1sSHlaC0CABHN5/O6bq21iEiEjz9+fnt7q64bESlFyRlrDYHGIZZSrLVEpKoioqrGWOchxYgiyqwIaAyRA/TGFEItMRHZLEJkAFAFfRX6vhfORCTKxtBq2a9Nt1Ip3qPkVHIpGgl9VVUxRguGgZFgWK7Ga5vD0BmDoL6qatFCRFw6G6r/Z+ddP/PAdz7y6MFffvTez3/5QYZUW1OEhUlAkQREU0oiAgCWBABzURZFi7xMA5Xrinvd08dXDs3zn/tN9Xzv6IkZ5VhP2iEMe5eu3rof5zdupv7c8f/1h0d39vfe/ZnLD931uT++uDiztnxod+2GanEl1b1qGawLy5UdTsCdz3za7d9607kTp3ZX4/f+zRc++uEvBAcnjrv5XF74gpNGh8loctONZ557xx2zofvoRz/9K//3ByJibSyX4pAM4tbYHtv2y905Lv28ob1FFFTr8Nabb/vtX/0BKNh1e9PNnbzqma7d9clH/vYTj7zshbe9/Duefbgob7vlrp975CWJ1TmPX3rfr9hqygAGCJSFhbnkkr1l68bWGtWl9aMYWSQ6gqI+c9SiIPs5UUq5xKx6yVPLBKP14+34tAgDEBkjCiiACDENVVVp0cK5rqcshQKhIKrGuKrbYyo9qKBaRYNIAJhSUlw5FxBxWC0XB0cmjNvxuq+r3auX1tfXgqtyAuM150iEITQCeTHrjBHmMhqvpZQQjDAoDiHUpUTvRqUUYRUtSOxcLQWMNUTaxTgej4c+eu9UUikZpFKVvj8MoTLWiwAY5JTrxpfcA1RAlag4KyUNRBTjsmqMYiOFSKXr9j2G/cNLVe2dGQ2xC75CxZjmCKJofGgyq0I9nbaIbjFfNqOWLChQzuA9OeP6fhn7o6reCBUxM2EQyURYuAAos9bViDkrJATXx2Xlx85j7Doi6LouBIe2VpFSOFS1MYYLiwAoAAKAqgoZVCFrjYIIl5y55NQ0oetWq1W3trFpyElBwAwAiGCtzTEDQAiBmVNJACAiIYTl4mA8Wu/6FXM/Hm+oEJFBBAUgKH3fV81EuACKgHgfDq5cHY8nqoiAWQWwqBRvwqJbNKORMQYVhr7/w1s++vpHXqKa6qpdLPr7H969+0sXPvrJB2Je1pWBslKgUgQUSxEAYlAi5dxzkaw+WPvDysdOXb95vFk9+GBMjqzalg7mMi1+vLGx9uKX3fKj32LG57/4sXdde+ThC/dfKbPh4QtoNpr+C9fk1toNtFjxlXCs9TgvMbMsF2BtWfXD+nR89my1Xo++8tCuCgQXVCBzHgZA1hvO8be+8LavPLj3l39zn4hxhoHw1OkzSNDYyfe8/M4TO9Xxk6cef/jy409c2D0sB/PF9Seb1373jffcfw10OFgMj1994q8+dGBNOxnXixTPbtvXff+dH37l+Tecfyln1oJ4/8d+N3MqEZyDxawn21e1Ozrsxi0j1vvXDiR1bgonjt+ahkpk11VbuaxAjBYRXAqPqsZomqguWHPTjli89ZozO+8BkEgBCAC5iPc+lc6YKvOAqtba2eFeXbk+Fh98O1pDrABlOV/VdQWghGCsSykTiGLlQ+TiUiqhGafUEyqCEnnRAgCHB/Pjx3YUYkpZYLY2qq5d66ZrO0eLRXBhGKKvFNSAEiKqCqACgCGbcyECa2zKKQRXShIJ3lPOaAz03cJaqwBN05Si1ro49NeuXTm+Peoj58JcBkJt27G1tuuXKG7V7bV1BezCZJyTppSH2I3atcPZw+trx0Dqo72rLFHBnDh+PXhfN0GEq6qOMRuqcik+mJyUeajrKiUlhFwGVTXkD4/2t7a2hqG31hq0gGW1WlZh3MVrk/H6Y4890rbTtelWjNE5S0TO16UIKIkyUSnM1jh+kiRrgqrxLuSySilaa51zIEYRFotF3TTBmpiTc76k7LyB/58qACIYg8PQGQrGGCJi5hhXzOS9J4KUBwSjCtYalrxazLc2dkR9kV6kOB+OjpYb25vMhaUAgAVgFgJSRW/NkCMRSWbr3W+f/qvXP/qKlHoTsAyZSvEObDv5yteuffLTD69t7uxeO3jiwu49D9xjgwBKSXT25Knbbzl7+erhRz71xZdVeMcttzfXLh0cdNQdgM1Xkdbc+LS402dvci+5c9deu3rxU/vdflpKAv7Hr/B1227/Yp5+Q/W1i1CtUs+r6tjOWb88Wtv+4t274JE8pAQMztgYaG1ct6PaHz95rKTVxub4mbfftn0Kdi8XKTAZjU/uLI5v7rhgPv+1i7fdesZh7wzW03Dx4asnd67LKjEuhYYHHzt/ba//yCfOnzt96pnPHH30769evLJcSRN01hVvfS6lAcx5KFd+5vHnve9Wwlwy430f//1hOXDphz5P1htAztFYWlMzVHWd+kXq57NZt7G1NaS5Zj/a0MpulpJm8wODdrQ2KuwBUl2txWERYxyNpsyiCj4EZgVwAFkhhcoStH1cqho0Wtn6woXHd44f41JAHaCgga7vgiNr6zhkMuA97R8enTlzZrWYm8pjkcptJe1YBcFKUS7FB4qxR0QAs1jsTqcbKfEDXzs6/+jDz7rz+jM3bMcsCCn4aUrRmrqUYiyVkuuq6YaZtQQAIsC5qEpVeVFmJueRC1r3/3UEJ7+aZvdBgH/TOecdvu8O3eWqag9tA0mIUAiOGBaRQEJixZIdOzawCTv+AiTWIJaRQWLPCi/CCsRkyRBAYYhFbCdOd7un6qq6937D+57hN1D28yA4wTuIYwyiBGGn0+n+7q5vJ85lqIZGHw0REJkpgfhoJMx5sr214/r+GLqsuTf16AATS3z11cPzF/fMogrj+nC+nA6H+eHt49e/82dzmlWHas15VetmltIy+mZuKSUAhCBmrrWJSBKo7UpEvQUzQnCeuFeTBIjo7maeSmLKEajamCczq7WmJERJtTMHCyKkMUZKpda6rnkMXeZ5DG2tpZwBYvQ+TcXMI0JEPERVUxL/heoeEc7MhETEtbaSi9ouUlSt1o2o9P707P5wrZUoRcT5sj179oF2GKMzk5oSiwhs10dk8yHLYXX3MP/o04+//9t/+A9+8reWdRI+amyIAJ6HdWtxWCeA5qjrcf3446c//dn18bH/1ne/8bX3MVzpsDw8nt/8n/29W1Hbf/z/vjw9nAJsvcDrn316/u7Nex++/NVvP/vo44//w7/8/YfrE/XH3/zz/POv8E3Uk8rPP9+4LK0H4JzS6XGj57fPjzdTKXB/c/dX/vI3fvs3fmM+5KUsLz7AU2sYCEB1r4zZRgy/1tppwokOgNR8LF4cdDnOXQ2Jr9c+LcJZC6b//aM3v/fvfvTR55eLd29DIQlHFCwOF6irikkBuyQuHvXTv//6m//iGwjgRviTH3wPw1pTlvx0+mKej8t03LbGjIGRhMNDdQdKxAuhmdbC5bK/Tvm2nffqDwF2Mz8vh/V0fro53iJlDDUPZgFE1yqSetfPPvvcxvXbf+bbCAsiergk0uHC5en6+Twdwqhu7fTwo1zubu5fSBHQQpIlJffR9nh8+6etbx/+yq+BFVXt45xzoB/c1cNNgwWIhTA326a8Xq+nnN4p7uhG6hfEmKdjgPbWU5oQCdBarTkvTDzGaK3Py4RAHko47ft5nhmRr9teygzugN1suIWOmA6zB05prXUH0AAn5CQS0NVUB0/zygx73UsuScTczAkQXHciRoTWrpJWMyulCE+qjqhu6oYpSx8OiCRGAWZOzBGeEm/bPk8LAPReEbKZp0xuxAIiPLTq8G3bDoebCEAaOijnYtaQ4J15nrbtKiK1tsPhaBpDt5yLeyBgG5u2erMur7/88v7Fh8NUJPXaiAgRS5laayxh7qaR8wQYIuLuiBiOaoOZ4BfwetnneRbh2jRsvzw+3t7e130z12U9mmPX7ebm9nrdALBMZXRn5GWZI6KO7u+opTL905f/+h99/neYYx8dKAmSgHUq6AO8RyRwEvFhvVUlJsJQ9SC+K/R4jamQATDxOk2Xdsp0wKQWmGOFrIlImGPCn/7Rw3/89/9X6+Ors60399/9Cy9++IP/8daxsK/Hw6+//NW/+def1/rqkKaH8wk43d7dvz29Blpv75bCN4hatw0gIHwPbZd+czwA07hWmddA8+4yoxkjgumeoHTfci42ZNtOKdtheb6u5eHNw8/fPL4n6+//wR/83g+//Oq0T0kws9ZukAKibu2rf/jzD373WykngI4/+cHvjuZTWR4fXq8rpem2a4VAJhljlFIiArmEq/uAcLPKxBGo6u5dRM6PMeL1e++9B8hAs8gaugmnCCUJD+4VWIAZAdAj1LRM8753JkgMdb+u89f62AONBM9PQbxlmXRoWo4l4fn0lpBubt93HwCwXStiZ5nUSCRdL2+X5RiOxEAsQvb09pSmIimVMrvJ0GvEYBZVLWXW0LqrSBbBGI4ZVbUk7M2r7kkKh0nKvUXKSMQA5g7msByO/XICwqENgGSaMAgcmHKgegAxisDYryJTAGk4ertetsPhKJLMRoR7EAkRUGs1l4wIYdL65eZ4V9u5DV3WQx8mIuGYGC/nh3WegnC77iIlpQxItfZlmdx7OJn3qZQxVCNyWvZ9pCmFhukZQ5KkPMm+7ymlfd+BNEsZrR+WtUcTXsxMrRJxTll7d/cAZE5qQMhtnMMsSVLVlFM4ICGRq7GImA0AYGS1AYhlnkdrRGRmOWfDCXzTtqc8qVuSogp9nNF8Pt5qU5kygqgqAEQEERCCm7d9pEJmloQAAoP++Yf/5nd+9rcfH08vXnyw13POmYgDkmpFRAgm0VZ1ng+9VwBE9N7HPN2a7kTYemXmCGtNmYQYiGN0Zc4lT8O6JGptrMtN02tOiRB77b1BmTl8uHmt1ySTRXgYQrpcTi9fvtj3bhYAkVNSHe4BCCmJu7lhLrn1ChB12+epqFuZZ7cEuNdq0zTv18tUVkQP7B40z3nf2+W8Hw6rmeWcp2kae/9ff/jZH330+OOPPnu6+CeffkbZpnn55O/97Jvf+1arXd3xv33/H6c892EvX37w1SdfDnp7e/sBwYKESDBG37br8ebe3VNiBOjDxmhCimREN5Kj7RzQxOHaLmUu63qwQbU+5alQOoxdU4Es81AN7/vebu/eu+5ViEXw6em1YCzrDSL10QERIw3bU1rUNKeMGGaaS1GtCGSKIotqExG1DmjCqdaec06JRm85HSxaH8ICAdH6hTBPGU09giEoqBOKmXl4aEWiqUzCcq1XN3bfl3yjfgUoJcsY3sEzCwCI8H56lab08Nifv3juRqOPVFhjWPfD4SYietvH2GvtkpIIihRmiQgz7/0yT4uZDnUEzDkTMiACMbGbhcfw4SWXYRYApq3k+fS0HW/uet+maXIHU5c5hQczA4ZwMhuEHEEBhoFhSgTDuluUfOx2Hg1E0LznzKbuDrXu67ru9Rwu63ozRkeIMcY0zyxiZkzcWgUIxgQAJIxEEKrq4eDhFI4k5qbmxHJY5zFa3bdlPbi7qiKi5FK3Uxba9nZ7+7XaniIkpSwwu5y++PiPX7z8DqYFCXvrxETBxGTuQyNAl2XprRGAavzdX/8nf6P/pZzLdr0yU+8DAFl4mqbz6WTmuQgRA5AwBzgAQgRgIIi7m2lO2UPHGMLJw1T9nVIKAiKju9ZaS5mIUMcgwghHIrPRWy/v5EmkDPW6V0SQxGZKRAA+hopIRJQyuXutdZqKqosIAKg5IZkpAKibh6XEETRN2YdBIDH1vqtFhJdSEDnczH+BiQA4Qs38eLgH8f/6wz/54qvL29Pj7fH4V7//W3/8Jz99ug786X/6HosAUu8W+Djnb1z3c0ibeDZTgChlil/qQ3Mup/OXN8dno1mEa5gwEwoS7petTDy0EvFezwQpsfTox/XZ0J0wexhEIBCSAHLK5oYIiShv21cekPNEnBJjHw2hSCY3J6Jad0AgyPOSrtt5KjMyaHem0voOoMwJAM/npykdDOu0HPf6xvrjYfpzAKh+Yll6rykTouuQnEWEW+sBoW13V+ayTul8+sKG1fE437xIfFBtzLmst2EeZgBAIWOo4jUCxni6u3n++HT95re+dT6fRaZ976Xk0/l0e3PfapunrOG995QkIlzNHRBdOJn3Usp12yNiLvTJJz97/70PljURL/u+R/jhcFAV8xGh05wjYvRBRCmla1V3PxwOrbWSSx+7cFYFIiAKMN3rnnNiFgBp/eSu83SIYJFiViMCMRDRbRqjXa6PpcgyH3pvkrPk1FoQRmJ089ouKeWU01dv3twdb80852wGTEiErTcRUYuc2LQzU20KANM0nc/nacrb9QIBx9tboamOJ6HJPUZ0SbcTpmt9BNB3UkpEFA4shMQpz+ZDVXPKYY4kvVcAiMC5UITX2glToLXWj8fjGD0CAUJEAGjfL9N0cB8W+37Gw+Ewemeh6/a0rosOj6BpSuYK4O8gJUSMCLOODimn1joSiqw2Rs4Zwbf9nCQZqMcIT4juMZildySinEpEpES9t957SrLXCyKVaUUg8+bqkpITpbBXbz7qLb7z4a8hyb7VXJK7D3UziwgzPRyO+77P89xaEyzDH0kyxCo0euuStKTMcme2IZVXryv+z3/7z0SACMGgb/bJp//9L/7mX1O/d9gQCYGJ5HJ9XNZ12xpJvp4+e+/+6xacpgS+aS/MiDT2vQMEITFlxNrr2173JMu5Xtb5eSlsvk/T+w7Rejscjq2zh6WciAjDUipdDQDPTw/CsExHRKdSeu1hza1JmU11ng/73h2dAHLC3i+AMpXVnd4J9KEsTF999mo5rCkHkbRWU0J3TDJFEDO4d0C/XvY032ZS0ytS2mq9WW7d1IAimhvngqZuNso0jWHEaVgTJlcfo09TJpj27fr5Fx8/f/n1m7v3Ja/7aFOO0TxJ0TEACIkAHCACUARb7eBjWo/btgFALqnv/XizbtvIidSMERDi9Vevbt5/nkXAXccgYQDY9x0AlvVQ61bKnGQy62NUZhTJw9B9ELMFCqaIi6sxoqSl1urhEQHMOaXtfBLmNKE7IAoEc042ho6RU6Ii+k6tOSUNcXfrYyoFKIgMf4FDlvAR2oTR1AFCUt7qniUzs6qmlCygbS2lgsIIHu84s4xx7ZEGmWNehPIYiggRMaIzglsQCguZeU7ZNIACwCICkSH0fH46HldVn5e11mFmKTN46mNf11U1RtdccIxe8nI+PZVSiNFd3YgYEMkUIpyIVHWe59PlYZrW1jqAC5NIHhosOWU0tdPT28Mh910JExAT8TRNaq33wZQsjAgRYYwWRtfr5YMPXrZWmQsg9q4554iH09N1Xe8DmYndVAdJEuK27zsz55Qd3kFEQPLRAwBExMzcQ7gw4RgXRyEU4tAOfbR1ntxs7w1/8p+/1/dxvb4+rAfJN2maX7364mvv36gRsyBCwHAv7oqMACBYhg5mUOvgI5iQJvA257LvVwBwoP10yXO+u3+x1SuQiI2PP/vy5ji/+vzjl9/85rrm7eHp+P7zy6Ueb9fLeV/KpIMQHpe7r79+9ek83YhIbdfe9Pb+aMYidD2/8UCrr7c6Divf3v/K4+lhmW4C/eHN2/u7OxEJNCTpvY1RAXVZZ+sLYkheLufHsHF7f3QAiBht5CTmrXct06rD3IZ6INcEy/CY15vWRmJKEo+nUyklc+lOwuFmJc/79mRQUzrkvKgBOgy9TOt6Pe1PT0/f+OYHZuaRkJyZxnDAYMqjj1yK6RARQHC3VObRGyFQRDeN8JwzM6sOdyAkQlKtajrNs0PsT9vx2bPMcd1b4hS+9330WjFBpkmyOBXtGwITsXsg+eg6zcV9IDIzuao7IkigO7ik3HubyhzmqnuYqY4+OkDc3L7vAKkkIehtRyczdwhCfXh8+7VnH277NeVMJKfTSYQIlVlKmc7np2W+8V+KCORE5OHJw0QoHN19aAMA+SXTcK8OPC9Jqw/dcpn2vR2Px66QBd2HMA71xKW1Shz73g7rzV7P83Rn4SIIbnvdRLhVI3YE7n2s62pmqoosTMltl1QA3GwIZwCmBK4pYg/nCFWzaV5M3VzxF0BV57K0sYnI08NlPRyJw93MoiTcO5UJrELK2NU9GkValvl0uaQk4cqIQ5u7I1JOqamZ9yyT0HTd35R85BRjjP30qtWWpnsd2zrfNo3D7W3tJ4osEgAekMHgHUdIFGbm7qqKP/4v/4pAAHR04MySkrszsY2dfslsIE9mbbs+uO7HuxcR4e4AGOa5yNPjPi95jFZSQaQ67O52fnp6RCjznBViZhqOo28Pbx9rNTX/AAACAUlEQVSePX92vUC9nMsBQgHB7u/e7/xO8v2x2b5ML66XjgiHYxnv6DYvN6rWmx1vb/r5rYYnSg+X13e3L1lYgdZ1vW6bCONQM3c3DyWaiEE4D23hTug6WipZhwc4ACBQEty2HQhzLgRhgaYjXMqSzYwJiWLfGiAhEgFypndaq+HRu97fL+G5qyIChgT0AGCa1QYTsdDj0+X29kBEYwxEUMOcCkQEqLszAyKO2pCIUwYk8G7qOWcz88jhDVFbuwKk0UeZ8xh9ShKSBb1rTNPd6NfEk2mXkgjydXss04EQxxgiiYgdgpDcnQW6OkKEu2sQsVkHiAAkkXBz05LFPSMEANS6hXeZppTz49vX84zXp93M15t1u/YXzz+8bF8c1mfmBkC9t1yEMQ3thGDmzNh7jwgRCQAWAk+1Xad51hEpJUADgNEVCYmAMAfQXk9zXva255ymkgPQTInQVEtOqiMczN+xVmvKk2QMozwlfafrNE3bdnXn9ZB68zJRbyMCiUSj57S4NlVMSVq7eGhrNedjSktEZ2IAZ5GhBoBEtG815+wepSQ3HWOkPI0xWECEEXm0vUyT2SAoAbrtIxcMQ3cjESKEMLOYipyenryrTEy8eJjwlHPZ6yNhivDhnjGN0YFxNCMYzBAUX35+vbu9z9ncw1x9tDEG5zzN85s3b+Z5zjn/fw9kHoeHyfXhAAAAAElFTkSuQmCC",
+ "text/plain": [
+ "
+
+
+
+
+
+
+