【25-Q4-生态建设】模型迁移-研发效能部-模型训练-在PyTorch框架上支持 SKNet 在Cifar100上的训练

PyTorch/build-in/Classification/SKNet/readme.md

@@ -0,0 +1,65 @@
+```markdown
+## 1. 模型链接
+- 原始仓库链接：
+https://github.com/huggingface/pytorch-image-models?tab=readme-ov-file#models
+
+## 2. 快速开始
+
+使用本模型执行训练的主要流程如下：
+
+1. **基础环境安装**：介绍训练前需要完成的基础环境检查和安装。
+2. **获取数据集**：介绍如何获取训练所需的数据集。
+3. **构建环境**：介绍如何构建模型运行所需要的环境。
+4. **启动训练**：介绍如何运行训练。
+
+### 2.1 基础环境安装
+
+请参考主仓库的基础环境安装章节，完成训练前的基础环境检查和安装（如驱动、固件等）。
+
+### 2.2 准备数据集
+
+#### 2.2.1 获取数据集
+
+Res2Net 训练使用 **CIFAR-100** 数据集。该数据集为开源数据集，包含 100 个类别的 60000 张彩色图像。
+
+#### 2.2.2 处理数据集
+
+请确保数据集已下载并解压。根据训练脚本的默认配置，建议将数据集存放在模型目录的上级 `data` 目录中（即 `../data`），或者根据实际路径修改训练命令中的 `--datapath` 参数。
+
+### 2.3 构建环境
+
+所使用的环境下需包含 PyTorch 框架虚拟环境。
+
+1. 执行以下命令，启动虚拟环境（根据实际环境名称修改）：
+
+   ```bash
+   conda activate torch_env_py310
+
+```
+
+2. 安装 Python 依赖。确保已安装项目所需的依赖包：
+```bash
+pip install -r requirements.txt
+
+```
+
+
+
+### 2.4 启动训练
+
+1. 在构建好的环境中，进入模型训练脚本所在目录。
+
+2. 运行训练。该模型支持单机单卡训练。
+执行以下命令启动训练（使用 CIFAR-100 数据集，Batch Size 为 128）：
+```bash
+python weloTrainStep.py \
+        --name train \
+        --arch sknet \
+        --print_freq 1 \
+        --steps 100 \
+        --dataset cifar100 \
+        --datapath ../data \
+        --batch_size 8 \
+        --epochs 100
+
+```

PyTorch/build-in/Classification/SKNet/requirement.txt

@@ -0,0 +1,89 @@
+addict==2.4.0
+aliyun-python-sdk-core==2.16.0
+aliyun-python-sdk-kms==2.16.5
+anyio==4.11.0
+astunparse==1.6.3
+certifi==2024.12.14
+cffi==2.0.0
+charset-normalizer==3.4.1
+click==8.3.1
+colorama==0.4.6
+contourpy==1.3.2
+crcmod==1.7
+cryptography==46.0.3
+cycler==0.12.1
+einops==0.8.1
+exceptiongroup==1.3.1
+filelock==3.14.0
+fonttools==4.60.1
+fsspec==2024.12.0
+future @ file:///croot/future_1730902796226/work
+git-filter-repo==2.47.0
+h11==0.16.0
+hf-xet==1.2.0
+httpcore==1.0.9
+httpx==0.28.1
+huggingface_hub==1.1.5
+idna==3.10
+inplace-abn @ git+https://github.com/mapillary/inplace_abn.git@b50bfe9c7cd7116a3ab091a352b48d6ba5ee701c
+Jinja2==3.1.5
+jmespath==0.10.0
+joblib==1.5.2
+kiwisolver==1.4.9
+Markdown==3.10
+markdown-it-py==4.0.0
+MarkupSafe==3.0.2
+matplotlib==3.10.7
+mdurl==0.1.2
+mmdet==3.3.0
+mmengine==0.10.7
+model-index==0.1.11
+mpmath==1.3.0
+networkx==3.4.2
+numpy==1.23.5
+opencv-python==4.12.0.88
+opendatalab==0.0.10
+openmim==0.3.9
+openxlab==0.1.3
+ordered-set==4.1.0
+oss2==2.17.0
+packaging @ file:///croot/packaging_1734472117206/work
+pandas==2.3.3
+pillow==11.1.0
+platformdirs==4.5.1
+pycocotools==2.0.11
+pycparser @ file:///tmp/build/80754af9/pycparser_1636541352034/work
+pycryptodome==3.23.0
+Pygments==2.19.2
+pyparsing==3.2.5
+python-dateutil==2.9.0.post0
+pytz==2023.4
+PyYAML @ file:///croot/pyyaml_1728657952215/work
+requests==2.28.2
+rich==13.4.2
+safetensors==0.7.0
+scikit-learn==1.7.2
+scipy==1.15.3
+shapely==2.1.2
+shellingham==1.5.4
+six @ file:///tmp/build/80754af9/six_1644875935023/work
+sniffio==1.3.1
+sympy==1.13.3
+tabulate==0.9.0
+termcolor==3.2.0
+terminaltables==3.1.10
+threadpoolctl==3.6.0
+timm==1.0.22
+tomli==2.3.0
+torch @ file:///apps/torch-2.4.0a0%2Bgit4451b0e-cp310-cp310-linux_x86_64.whl#sha256=2e472c916044cac5a1a0e0d8b0e12bb943d8522b24ff826c8014dd444dccd378
+torch_sdaa @ file:///apps/torch_sdaa-2.0.0-cp310-cp310-linux_x86_64.whl#sha256=5aa57889b002e1231fbf806642e1353bfa016297bc25178396e89adc2b1f92e7
+torchaudio @ file:///apps/torchaudio-2.0.2%2Bda3eb8d-cp310-cp310-linux_x86_64.whl#sha256=46525c02fb7eaa8dafea860428de3d01e437ba8d6ff2cc228d7c71975ac4054b
+torchdata @ file:///apps/torchdata-0.6.1%2Be1feeb2-py3-none-any.whl#sha256=aa2dc1a7732ea68adfad186978049bf68cc1afdbbdd1e17a8024227ab770e433
+torchtext @ file:///apps/torchtext-0.15.2a0%2B4571036-cp310-cp310-linux_x86_64.whl#sha256=7e42c684ba366f97b59ec37488bf95e416cce3892b6589200d2b3ad159ee5788
+torchvision @ file:///apps/torchvision-0.15.1a0%2B42759b1-cp310-cp310-linux_x86_64.whl#sha256=4b904db2d50102415536bc764bbc31c669b90b1b014f90964e9eccaadb2fd9eb
+tqdm==4.65.2
+typer-slim==0.20.0
+typing_extensions==4.15.0
+tzdata==2025.2
+urllib3==1.26.20
+yapf==0.43.0

PyTorch/build-in/Classification/SKNet/sknet.py

@@ -0,0 +1,242 @@
+import torch
+import torch.nn as nn
+import math
+
+class SafeGroupConv2d(nn.Module):
+    """
+    修改版：在初始化时强制对齐原生卷积的随机权重。
+    """
+    def __init__(self, in_channels, out_channels, kernel_size, stride=1,
+                 padding=0, dilation=1, groups=1, bias=False):
+        super(SafeGroupConv2d, self).__init__()
+
+        assert in_channels % groups == 0
+        assert out_channels % groups == 0
+
+        self.groups = groups
+        self.convs = nn.ModuleList()
+
+        in_channels_per_group = in_channels // groups
+        out_channels_per_group = out_channels // groups
+
+        # =======================================================
+        # 【核心魔法】：影子层策略 (Shadow Layer Strategy)
+        # 1. 创建一个临时的、原生的、带 groups 的卷积层
+        #    它的唯一作用就是按照 PyTorch 标准逻辑消耗随机种子
+        # =======================================================
+        shadow_conv = nn.Conv2d(
+            in_channels, 
+            out_channels, 
+            kernel_size=kernel_size, 
+            stride=stride, 
+            padding=padding, 
+            dilation=dilation, 
+            groups=groups,  # 这里用原始的 groups
+            bias=bias
+        )
+
+        # 2. 获取这个影子层初始化好的权重 (Shape: [Out, In/G, K, K])
+        #    因为此时全局 Seed 是固定的，所以这里生成的权重和 CUDA 上的一模一样
+        master_weight = shadow_conv.weight.data
+
+        # 3. 将权重在输出通道维度(dim=0)切分成 groups 份
+        #    每一份 Shape: [Out/G, In/G, K, K]
+        split_weights = torch.chunk(master_weight, groups, dim=0)
+
+        if bias:
+            master_bias = shadow_conv.bias.data
+            split_bias = torch.chunk(master_bias, groups, dim=0)
+
+        # =======================================================
+        # 4. 创建实际运行的小卷积，并将权重“塞”进去
+        # =======================================================
+        for i in range(groups):
+            # 创建小卷积 (groups=1, SDAA 支持)
+            mini_conv = nn.Conv2d(
+                in_channels_per_group, 
+                out_channels_per_group, 
+                kernel_size=kernel_size, 
+                stride=stride, 
+                padding=padding, 
+                dilation=dilation, 
+                groups=1, 
+                bias=bias
+            )
+
+            # 【强制覆盖权重】
+            mini_conv.weight.data = split_weights[i].clone()
+
+            if bias:
+                mini_conv.bias.data = split_bias[i].clone()
+
+            self.convs.append(mini_conv)
+
+        # 5. 影子层完成了它的使命，在此处会被自动回收，不占用训练显存
+
+    def forward(self, x):
+        x_splits = torch.chunk(x, self.groups, dim=1)
+        results = []
+        for i, conv in enumerate(self.convs):
+            out = conv(x_splits[i])
+            results.append(out)
+        return torch.cat(results, dim=1)
+
+
+class SKConv(nn.Module):
+    def __init__(self, channels, branches=2, groups=32, reduce=16, stride=1, len=32):
+        super(SKConv, self).__init__()
+        len = max(channels // reduce, len)
+        self.convs = nn.ModuleList([])
+
+        for i in range(branches):
+            # SKNet 的核心机制：不同分支使用不同的 dilation 和 padding
+            dilation = 1 + i
+            padding = 1 + i
+
+            # === 修改逻辑开始 ===
+            # 检测是否会触发 TecoDNN 的不支持配置 (groups > 1 且 dilation > 1)
+            if groups > 1 and dilation > 1:
+                # 使用自定义的拆解版卷积，避开报错
+                conv_layer = SafeGroupConv2d(
+                    channels, channels, kernel_size=3, stride=stride,
+                    padding=padding, dilation=dilation, groups=groups, bias=False
+                )
+            else:
+                # 正常情况（如 dilation=1）继续使用原生算子，保持最高性能
+                conv_layer = nn.Conv2d(
+                    channels, channels, kernel_size=3, stride=stride,
+                    padding=padding, dilation=dilation, groups=groups, bias=False
+                )
+            # === 修改逻辑结束 ===
+
+            self.convs.append(nn.Sequential(
+                conv_layer,
+                nn.BatchNorm2d(channels),
+                nn.ReLU(inplace=True)
+            ))
+
+        self.gap = nn.AdaptiveAvgPool2d((1, 1))
+        self.fc = nn.Sequential(
+            nn.Conv2d(channels, len, kernel_size=1, stride=1, bias=False),
+            nn.BatchNorm2d(len),
+            nn.ReLU(inplace=True)
+        )
+        self.fcs = nn.ModuleList([])
+        for i in range(branches):
+            self.fcs.append(
+                nn.Conv2d(len, channels, kernel_size=1, stride=1)
+            )
+        self.softmax = nn.Softmax(dim=1)
+
+    def forward(self, x):
+        x = [conv(x) for conv in self.convs]
+        x = torch.stack(x, dim=1)
+        attention = torch.sum(x, dim=1)
+        attention = self.gap(attention)
+        attention = self.fc(attention)
+        attention = [fc(attention) for fc in self.fcs]
+        attention = torch.stack(attention, dim=1)
+        attention = self.softmax(attention)
+        x = torch.sum(x * attention, dim=1)
+        return x
+
+
+class SKUnit(nn.Module):
+    def __init__(self, in_channels, mid_channels, out_channels, branches=2, group=32, reduce=16, stride=1, len=32):
+        super(SKUnit, self).__init__()
+
+        self.conv1 = nn.Sequential(
+            nn.Conv2d(in_channels, mid_channels, kernel_size=1, stride=1, bias=False),
+            nn.BatchNorm2d(mid_channels),
+            nn.ReLU(inplace=True)
+        )
+
+        self.conv2 = SKConv(mid_channels, branches=branches, groups=group, reduce=reduce, stride=stride, len=len)
+
+        self.conv3 = nn.Sequential(
+            nn.Conv2d(mid_channels, out_channels, kernel_size=1, stride=1, bias=False),
+            nn.BatchNorm2d(out_channels)
+        )
+
+        if in_channels == out_channels:
+            self.shortcut = nn.Sequential()
+        else:
+            self.shortcut = nn.Sequential(
+                nn.Conv2d(in_channels, out_channels, kernel_size=1, stride=stride, bias=False),
+                nn.BatchNorm2d(out_channels)
+            )
+
+        self.relu = nn.ReLU(inplace=True)
+
+    def forward(self, x):
+        residual = x
+        residual = self.shortcut(residual)
+
+        x = self.conv1(x)
+        x = self.conv2(x)
+        x = self.conv3(x)
+        x += residual
+        return self.relu(x)
+
+
+class sknet(nn.Module):
+    def __init__(self, num_classes, num_block_lists=[3, 4, 6, 3]):
+        super(sknet, self).__init__()
+        self.basic_conv = nn.Sequential(
+            nn.Conv2d(3, 64, kernel_size=7, stride=2, padding=3, bias=False),
+            nn.BatchNorm2d(64),
+            nn.ReLU(inplace=True),
+            nn.MaxPool2d(kernel_size=3, stride=2, padding=1)
+        )
+
+        self.stage_1 = self._make_layer(64, 128, 256, nums_block=num_block_lists[0], stride=1)
+        self.stage_2 = self._make_layer(256, 256, 512, nums_block=num_block_lists[1], stride=2)
+        self.stage_3 = self._make_layer(512, 512, 1024, nums_block=num_block_lists[2], stride=2)
+        self.stage_4 = self._make_layer(1024, 1024, 2048, nums_block=num_block_lists[3], stride=2)
+
+        self.gap = nn.AdaptiveAvgPool2d(1)
+        self.classifier = nn.Linear(2048, num_classes)
+
+        for m in self.modules():
+            if isinstance(m, (nn.Conv2d, nn.Linear)):
+                nn.init.kaiming_normal_(m.weight, mode='fan_in')
+                if m.bias is not None:
+                    nn.init.zeros_(m.bias)
+            elif isinstance(m, nn.BatchNorm2d):
+                nn.init.ones_(m.weight)
+                nn.init.zeros_(m.bias)
+
+    def _make_layer(self, in_channels, mid_channels, out_channels, nums_block, stride=1):
+        layers = [SKUnit(in_channels, mid_channels, out_channels, stride=stride)]
+        for _ in range(1, nums_block):
+            layers.append(SKUnit(out_channels, mid_channels, out_channels))
+        return nn.Sequential(*layers)
+
+    def forward(self, x):
+        x = self.basic_conv(x)
+        x = self.stage_1(x)
+        x = self.stage_2(x)
+        x = self.stage_3(x)
+        x = self.stage_4(x)
+        x = self.gap(x)
+        x = x.view(x.size(0), -1)
+
+        x = self.classifier(x)
+        return x
+
+
+def SKNet(num_classes=1000, depth=50):
+    assert depth in [50, 101], 'depth invalid'
+    key2blocks = {
+        50: [3, 4, 6, 3],
+        101: [3, 4, 23, 3],
+    }
+    model = sknet(num_classes, key2blocks[depth])
+    return model
+
+def Model(num_classes): # welo
+    r"""Return your custom model
+    """
+    return SKNet(num_classes=num_classes)
+
+