1. Overall process

Example of conversion according to official model: use-ncnn-with-pytorch-or-onnx First, convert pytorch model to onnx model, then use onnx simplifier tool to simplify onnx model, and finally convert onnx model to ncnn model

2 environment configuration

2.1 software installation

1. Visual Studio 2019

1. CMake3.21.3

Add environment variables to user variables

1. OpenCV3.4.10

Configure environment variables in user variables

2.2 protobuf compilation

protobuf3.4.0 After downloading, unzip it to the specified folder: D:\ncnnby

Open the local tool command prompt x64 Native Tools Command Prompt for VS 2019 as an administrator to build protobuf

Enter the following commands in sequence

cd <protobuf-root-dir>
mkdir build-vs2019
cd build-vs2019
cmake -G"NMake Makefiles" -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=%cd%/install -Dprotobuf_BUILD_TESTS=OFF -Dprotobuf_MSVC_STATIC_RUNTIME=OFF ../cmake
nmake
nmake install

Start compilation

nmake

nmake install

Get the required documents

2.3 ncnn compilation

Skip downloading Vulkan SDK and do not use GPU reasoning; Download using Git Bash ncnn

Open the local tool command prompt x64 Native Tools Command Prompt for VS 2019 as an administrator to build ncnn

Enter the following commands in sequence

cd <ncnn-root-dir>
mkdir build
cd build
cmake -G"NMake Makefiles" -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=%cd%/install -DProtobuf_INCLUDE_DIR=D:/ncnnby/protobuf-3.4.0/build-vs2019/install/include -DProtobuf_LIBRARIES=D:/ncnnby/protobuf-3.4.0/build-vs2019/install/lib/libprotobuf.lib -DProtobuf_PROTOC_EXECUTABLE=D:/ncnnby/protobuf-3.4.0/build-vs2019/install/bin/protoc.exe -DNCNN_VULKAN=off -DOpenCV_DIR=D:/ncnnby/opencv/build ..
nmake
nmake install

Note: change the path of the command beginning with DProtobuf under the cmake command to the path of your protobuf

nmake

nmake install

Get the required documents

So far, the software has been compiled

2.3 VS2019 configuration

New VS2019 project, view → \to → other windows → \to → Property Manager

Right click relax x64 and select add new item property sheet

Name the property sheet and save it

Double click to open the property page and start editing. The following page appears

Select the VC + + directory and add it to the included directory

<opencv-root-dir>/build/include 
<opencv-root-dir>/build/include/opencv 
<opencv-root-dir>/build/include/opencv2 
<ncnn-root-dir>/build/install/include/ncnn
<protobuf-root-dir>/build-vs2019/install/include

Add Library Directory

<opencv-root-dir>/build-vs2019/x64/vc15/lib
<ncnn-root-dir>/build/install/lib
<protobuf-root-dir>/build/install/lib

Add additional dependencies

ncnn.lib
opencv_world3410.lib
libprotobuf.lib
libprotobuf-lite.lib
libprotoc.lib

After testing, the environment construction is OK (Note: the property page should be added during testing, and release x64 should be selected for VS Project debugging)

3 model transformation

3.1 transformation from pytorch model to onnx model

Add the code of converting to onnx model in the training pytorch model project

import torch
from mtcnn.core.detect import create_mtcnn_net

if __name__ == '__main__':
    pnet, rnet, onet = create_mtcnn_net(p_model_path="./original_model/pnet_epoch_7.pt",
                                        r_model_path="./original_model/rnet_epoch_6.pt",
                                        o_model_path="./original_model/onet_epoch_10.pt",
                                        use_cuda=False)          # Load your own pt file
    out_onnx_pnet = './modelconvert/pnet_epoch_7.onnx'           # Save the generated onnx file path
    out_onnx_rnet = './modelconvert/rnet_epoch_6.onnx'
    out_onnx_onet = './modelconvert/onet_epoch_10.onnx'

    x = torch.randn(1, 3, 640, 480)
    y = torch.randn(1, 3, 24, 24)
    z = torch.randn(1, 3, 48, 48)
    # define input and output nodes, can be customized
    input_names = ["input"]
    output_names = ["output"]
    # convert pytorch to onnx
    torch_out_pnet = torch.onnx.export(pnet, x, out_onnx_pnet, input_names=input_names, output_names=output_names)
    torch_out_rnet = torch.onnx.export(rnet, y, out_onnx_rnet, input_names=input_names, output_names=output_names)
    torch_out_onet = torch.onnx.export(onet, z, out_onnx_onet, input_names=input_names, output_names=output_names)

The onnx model is obtained

3.2 simplified onnx model

First, install the simplification tool and enter the following instructions in the Anaconda virtual environment

pip install onnx-simplifier

It can be seen that several additional packages are installed, such as onnx, onnxruntime, etc

Then simplify the onnx model, open the command prompt as an administrator, cd to the file where the model is located, and enter the instructions

python3 -m onnxsim resnet18.onnx resnet18-sim.onnx

The simplified model is obtained

3.3 onnx model to ncnn model

Move the simplified file to the D:\ncnnby\ncnn\tools\onnx folder

Open Anaconda Prompt, cd to the specified directory, and enter the command

onnx2ncnn resnet18-sim.onnx resnet18.param resnet18.bin

Get ncnn model file

ok, it's done!

reference

(6 messages) process of converting PyTorch model to ncnn model under Windows system_ Qin Qilu's blog - CSDN blog
(6 messages) convert the model trained by pytorch to ncnn model_ Fancheng blog - CSDN blog_ ncnn pytorch
(7 messages) (I) ncnn | Windows10 + VS2019 environment configuration_ zhangts20 blog - CSDN blog_ ncnn vs2019