introduction

When using MMDetection 1.0 originally, in order to avoid the trouble of modifying various configuration files every time when training a new model, colleagues wrote a config.xml file and modified each file of MMDetection to read the corresponding parameters in the configuration XML. In this way, when training, if you want to modify the parameters such as label, data set size, use model, training times, etc., you can modify the corresponding parameters in the XML file instead of entering each configuration file of MMDetection to modify manually, which increases the convenience of model training and testing, and saves research and development time.

Recently, the Department has switched to MMDetection2.11, and found that there is a slight difference between MMDetection1.0 and MMDetection2.11. I did not explore the deep-seated difference. The most obvious thing is that version 2.0 splits the model configuration files, but each split file is still familiar. Here, referring to my colleagues' ideas of writing MMDetection 1.0 configuration xml file and modifying MMDetection configuration file, I modify MMDetection 2.11 configuration file to realize the above functions.

1, config.xml file settings

Firstly, all the parameters that are often modified during training are predefined in the config.xml file

Decompress MMDetection2.11 in the folder mmdetection-master-v211, and compile in this folder. After installation, create the project. The folder name can be defined arbitrarily, but it will be used as an anchor point in the later configuration file writing, which is used to find and determine the location between different files. Therefore, once it is determined, it should not be modified.

Create a program folder under this folder to store the config.xml file and run the program.

The contents of the config.xml file are as follows:

<config>
	<module_type>faster_rcnn/faster_rcnn_r50_fpn_1x_coco.py<module_type>
	<image_size>256,128</trainimage_size>
	<traindata_lable>suokou,guzhangac</traindata_lable>
	<train_batch>4</train_batch>
	<train_epochs>12</trainepochs>
	<train_percent>0.95</train_percent>
	<save_path_name>Faster-RCNN</save_path_name>
	<predict_imagePath>/home/work2/023_X05_P01/400AF/fortest/</predict_imagePath>
<config>

Parameter Description:
--module_type: define the model used for training and testing, and the path shown in the example is in the configs folder.
--image_size: defines the size (w, h) of the image in the dataset during training and testing. There is no need to add brackets in the configuration file.
--traindata_ Label: label name, which can not be defined temporarily when writing configuration xml. Later, a program can read the xml file in the dataset, determine all the labels in the dataset, and write them into the xml automatically. If there are requirements for the order of signature, you can customize it.
--train_batch: defines the batch size for training.
--train_epochs: define epoch s during training.
--train_ Percentage: defines the proportion of data set and verification set when dividing data set.
--save_path_name: define the name of the folder where the training results are saved. The folder will be created under the tools folder.
--predict_imagePath: define the test folder path. If it is defined as 0, the verification set will be automatically tested.

Note: only the parameters that I usually modify are given here. If there are other parameters that are frequently modified, you can continue to add parameters in the config.xml file and modify them in the program.

2, Read xml file statement

After configuring the config.xml file, in order to read the parameters in the configuration XML, the corresponding parameters should be set in the user-defined program and the modified MMDetection configuration file, and the functions should be used to read the XML file and output the parameters. The function is defined as follows:

def config_parse(path, input_list):
	tree = ET.parse(path)
    root = tree.getroot()
    output = []
    for input_i in input_list:
    	output.append(root.find(input_i).text)
   	return output

Function function: given the parameter tag name you want to read from the config.xml file, output the search result. The input parameters are given in the form of a list.
For example: [pathdirmdet, save]_ path_ name] = config_ parse(path_ config, ['pathDirMmdet','save_ path_ name'])

3, Definition label: edit_label.py

Create edit in program folder_ Label.py program, because it is in the same folder as the config.xml file, does not need to determine the location of the configuration file, and can be read directly.
Program function: automatically read the image XML in the data set, determine all tags, and write them into the traindata in config.xml_ Label parameter.
Remove the previous config_ In addition to the parse() function, the file also includes two main functions: get_ The labels() function is used to read the image XML and determine the label; And set_ The labels() function is used to write tag names to the config.xml file.
The codes are as follows:

def get_labels(xml_path):
	xml_path_list = os.listdir(xml_path)
    labelList = []
    for xml_i in xml_path_list:
        if 'xml' in xml_i:
        file_path = os.path.join(xml_path, xml_i)
        tree = ET.parse(file_path)
        root = tree.getroot()
        for obj in root.findall('object'):
        	name = obj.find('name').text
            if name not in labelList:
            	labelList.append(name)
   	return labelList

def set_labels(path, labelList):
    tree = ET.parse(path)
    root = tree.getroot()
    root.find('traindata_lable').text = (',').join(labelList)
    tree.write(path)
    print('Set the label to:', (',').join(labelList))

Thus, the training and test tags are customized. However, MMDetection program can't read the custom label at this time, and still uses the default label in coco or voc dataset. Therefore, some dataset configuration files in MMDetection need to be modified to make the label consistent with the custom label.

4, Modifying the MMDetection dataset configuration file

The documents to be modified mainly include:
~/mmdet/datasets/voc.py
~/mmdet/datasets/coco.py
~/mmdet/core/evaluation/class_names.py
Where, ~ represents the home folder directory.

Since these files are not in the same level directory as the config.xml file, you should first determine the location of the config.xml file when reading the contents of the file
path_config = os.path.join(sys.argv[0].split('mmdetection-master-v211')[0], 'mmdetection-master-v211', 'programs', 'config.xml')
Statement.

Read the label content as a list:

[label_str] = config_parse(path_config, ['traindata_lable'])
label_list = label_str.split(',')

Modify class_names.py

For class_ Name. Py file, modified voc_classes() is as follows:

def voc_classes():
	print('\n[class_names.py -> voc_classes()]', '\npathPyClassVoc:', pathPyClassVoc)
    [label_str] = config_parse(path_config, ['traindata_lable'])
    label_list = label_str.split(',')
    print('voc list_class:', label_list)
    print('len(list_class):', len(label_list))
    return label_list

Coco below_ Classes () is the same.

In class_ In names.py, label_ The category of list itself is a list, so there is no need for further modification. In voc.py and coco.py, further modification is needed.

Modify voc.py and coco.py

Since the winning signature of MMDetection is stored in tuple form, the statement CLASSES = tuple (label) is used_ List) converts the tag list into tuples, and comments out the original definition of CLASSES.

Special note: in the previous tutorial I referred to, if there is only one label, such as "suokou" in the dataset, when writing CLASSES, it should be written as CLASSES=('suokou '), that is, a comma should be added after the unique tag name, otherwise an error will be reported. I tried and found that this is true. I didn't know the reason at first. Later, I accidentally saw that in Python, if a tuple is defined and there is only one element in the binary, the tuple tuple will be converted to the string str. if a comma is added after the only element, this situation can be avoided. The details are as follows:

>>>a = ('x')
>>> type(a)
<class 'str'>

>>> b = ('x',)
>>> type(b)
<class 'tuple'>

In MMDetection, the type of CLASSES must be tuple, otherwise an error will be reported. Therefore, as mentioned earlier, the extra comma added to the single label is the reason. While CLASSES = tuple (label) is used_ List). Even if there is only a single element in the converted tag, the converted tag is also a tuple and will not be changed into a string. It is not necessary to manually add a comma at the end.

The modified class vocdata set () is as follows:

class VOCDataset(XMLDataset):
	path_config = os.path.join(sys.argv[0].split('mmdetection-master-v211')[0]
	, 'mmdetection-master-v211','programs', 'config.xml')
	[label_str] = config_parse(path_config, ['traindata_lable'])
    label_list = label_str.split(',')
    CLASSES = tuple(label_list)

CocoDataset() in coco.py is the same.

5, Data set preprocessing: preprocess_data.py

Create Preprocess in the program folder_ Data. Py program.
Program function: press the train in config.xml file_ The percentage parameter is used to divide the training set and the verification set; The voc data set is transformed into coco data set, and instances is transformed into coco data set_ train2017.json,instances_val2017.json,instances_ The pictures of test2017.json, training set and verification set are saved in the config.xml file_ path_ In the corresponding coco folder under the save path set by the name parameter.

This part of the code online reference a lot, can be used directly, do not repeat.

6, Modify MMDetection model configuration file

Take fast as an example_ rcnn_ r50_ fpn_ 1x_ Coco.py as an example, open the file and find that compared with mmdetection 1.0, version 2.0 splits the model into four parts

_base_ = ['../_base_/models/faster_rcnn_r50_fpn.py',
		  '../_base_/datasets/coco_detection.py',
		  '../_base_/schedules/schedule_1x.py',
		  '../_base_/default_runtime.py'
          ]

These four documents are modified accordingly.

faster_rcnn_r50_fpn.py

In this file, the network structure is defined, and the main variable to be modified is the number of label categories numClass.

The way to get numClass: read the tag of config.xml file and find the length of the tag list. It is worth noting that in mmdetection 2.0, the background is no longer a separate class, so numClass does not need to add 1 to the number of tags. numClass will write as many tags as there are.

Other parameters are sometimes modified, such as backbone type and RPN_ Anchor under head_ Generator's scales and ratios, etc., but my modification frequency is not high, so I don't take them as variables that can be directly modified in config.xml. If I need to modify these parameters, I need to go to fast_ rcnn_ r50_ FPN. Py file.

coco_detection.py

This file mainly defines training and test set path, image scaling size, training batch size, etc.

The main path of data storage is save in config.xml file_ path_ The path defined by the name parameter is stored in COCO format without additional modification.

Image scaling size img_ The scale parameter, training batch size and batchSize are defined by config.xml.

Among them, MMDetection requires img_scale is in tuple format, so it is read into the custom trainimage in config.xml_ After the size parameter, the scaleimgsize = tuple ([int (I) for I in imgsize. Split (',')) statement is used to convert it.

The definition of batchSize is very simple, just read in the train of config.xml_ Batch parameter, and then use int() to convert it to an integer.

schedule_1x.py

This file mainly defines learning strategy, modifies initial learning rate, training times, optimizer type, etc.

Among them, the number of training times is set through the config.xml file, and other parameters are not changed temporarily, using the default.

default_runtime.py

This file mainly defines the pre training weight, whether to continue training the model from the breakpoint, etc., without additional modification.

By the way, in this file, resume_from and load_ The difference between from and
resume_from not only needs to read the weights from the checkpoint file, but also needs to get the specific optimizer status and epoch number, which can be used to continue training after interruption in the process of program running;
And load_from is only used to load the model and fine tune it.

7, Training model: my_train.py

Create my in the program folder_ Train. Py program.

The main function of the program is to call the train.py training program of the system, and input the user-defined training parameters, including the model used and the weight saving path. The above parameters are obtained from the config.xml file.

Run the program, find the program error. Since the error report involves the code of MMDetection2.0, it is not consistent with my_ The train. Py program has nothing to do with itself, so it will be solved in the next part.

8, Solving program error

Error 1

After the above changes, run my_train.py program, you will find the following error:

Traceback (most recent call last):
  File "/home/work1/mmdetection-master-v211/tools/train.py", line 187, in <module>
    main()
  File "/home/work1/mmdetection-master-v211/tools/train.py", line 89, in main
    cfg = Config.fromfile(args.config)
  File "/home/work1/mmdetection-master-v211/mmcv/utils/config.py", line 257, in fromfile
    use_predefined_variables)
  File "/home/work1/mmdetection-master-v211/mmcv/utils/config.py", line 183, in _file2dict
    raise KeyError('Duplicate key is not allowed among bases')
KeyError: 'Duplicate key is not allowed among bases'

At this time, you need to locate the error location through debug, add breakpoints in the line cfg = Config.fromfile(args.config) of train.py, enter the ~ / mmcv/utils/config.py program, and then enter CFG_ dict, cfg_ text = Config._ file2dict(filename,use_ predefined_ It is found that the error is caused by this part of the function statement:

for c in cfg_dict_list:
	print(base_cfg_dict.keys())
	print(c.keys())
	if len(base_cfg_dict.keys() & c.keys()) > 0:
		raise KeyError('Duplicate key is not allowed among bases')
	base_cfg_dict.update(c)

Check cfg_dict_list. It is found that the variable is a list with length of 4. After checking the contents of the list, it is found that each element in the list is a dictionary with different length, and the key of the dictionary is familiar, including 'ET', 'os',' sys' and other packages imported to read config.xml imgSize','save_path_name','train_epoch 'and other intermediate variables defined when reading config. XML; And 'numClass',' batchSize ',' maxEpoch 'and other model configuration files.

From this, it can be determined that the four elements in the list correspond to foster respectively_ rcnn_ r50_ fpn_ 1x_ Coco.py file_ base_ The four model configuration files in are the following four files.

_base_ = ['../_base_/models/faster_rcnn_r50_fpn.py',
		  '../_base_/datasets/coco_detection.py',
		  '../_base_/schedules/schedule_1x.py',
		  '../_base_/default_runtime.py'
          ]

By running the program in one step, we can find that the program is running through for C in CFG_ dict_ The list statement traverses each element in the list, that is, when traversing the parameters introduced in the above four configuration files, some parameters appear in multiple configuration files at the same time, resulting in duplicate key s in the four dictionaries stored in the list, triggering the alarm program. Under normal circumstances, because these four configuration files set different parameters, there is no repetition between them, so there will be no repetition of parameters, so there will be no alarm.

Repeated parameters include:

• import packages: 'ET', 'os' and' sys'. These three packages are imported in the four configuration files
• Intermediate variable when reading config.xml: path_config, etc

At this time, I have a bold idea: anyway, if there are more variables, the impact is not big, and it doesn't affect who. I'll try to comment out this line of alarm?

(I'll change it first. As long as the program can run, it's not the flood behind me.)

So I commented out these two lines of programs:

if len(base_cfg_dict.keys() & c.keys()) > 0:
	raise KeyError('Duplicate key is not allowed among bases')

Very good. I really don't report this mistake( It would be strange to return the report.)

However, there are new mistakes——

Error 2

......
  File "/home/work1/mmdetection-master-v211/mmcv/utils/config.py", line 418, in pretty_text
    text, _ = FormatCode(text, style_config=yapf_style, verify=True)
......
 ParseCodeToTree
    ast.parse(code)
  File "/home/ty/anaconda3/envs/mmdet2/lib/python3.7/ast.py", line 35, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 1
    ET=<module 'xml.etree.ElementTree' from '/home/ty/anaconda3/envs/mmdet2/lib/python3.7/xml/etree/ElementTree.py'>
       ^
SyntaxError: invalid syntax

It seems that it's the package imported by ourselves.

But this time, the report of errors is a mess. When you enter into all kinds of messy programs, you can't understand them.

Considering that the alarm involves the package I imported, the reason for the alarm must be my modification of the model configuration file, so I analyze that the problem still occurs when the program is running. The package I imported and the user-defined parameters have some impact on some running processes of the following programs. So you can also step through the config.py program in debug, and then enter the_ file2dict function, after the program runs smoothly, check the base_cfg_dict is a variable. By comparing with the variable before modification, the problem is found

My base_cfg_dict is better than base when the program is not modified_ cfg_ There are more key s in dict, just like the previous repeated variables. The main contents are my import packages: 'ET', 'os',' sys', and various intermediate variables when reading config.xml in the four configuration files.

This is also very easy to understand, because I just annotated out the duplicate variables, and all of these my custom content, through base_ cfg_ Add the dict. Update (c) line to the base_cfg_dict dictionary, it's obvious that the contents of this dictionary are used in the following program. The program reads my custom variables, and there is no corresponding processing method, so it reports an error.

So I have a bold idea again: I simply and rudely delete all my custom variables from this dictionary, so that the dictionary can be restored to its original state?

So I carefully compared the changes before and after the base_ cfg_ To delete the newly added key values violently:

del base_cfg_dict['ET'], base_cfg_dict['os'], base_cfg_dict['sys']\
    , base_cfg_dict['config_parse'], base_cfg_dict['path_config']\
    , base_cfg_dict['label_str'], base_cfg_dict['label_list'], base_cfg_dict['save_path_name']\
    , base_cfg_dict['img_size'], base_cfg_dict['train_batch'], base_cfg_dict['train_epochs']

"Pa", very fast! The problem is solved!

The program is running!

I love it! this cheers the people greatly! the whole world joins in the jubilation! rush about telling the news around spreading!

At this moment, I can't help but have this picture in my mind:

So I quickly backup the project, wrote this article, and vowed not to touch it.

But geese can't, I have to make a program to test.

9, Test program: infor_ test.py

Create infer in program folder_ Test. Py program.

Implementation function: set the batch size of the test, according to the test path set in config.xml, test the pictures by batch; The test results (including pictures, labels and confidence) are saved in the ~ / result folder.

There are many codes to refer to in TensorFlow related articles. After all, Google API has supported batch testing for a long time

• Calculate the remainder of the total number of pictures divided by batchsize, that is, the number of pictures to be filled, generate the corresponding pure black pictures, and save them in the folder.

  list_name_test_img = os.listdir(pathDirTestImg)  # pathDirTestImg is the test path, and all images are in the default path
  num_add = len(list_name_test_img) % batchSize
  for i in range(num_add):
  	img_ = np.zeros(img_size, dtype=np.uint8)
  	path_img_ = os.path.join(pathDirTestImg, '{a}.jpg'.format(a=i))
  	cv2.imwrite(path_img_, img_)
  	list_name_test_img.append('{a}.jpg'.format(a=i))
  

• Read the picture by sheet, gather up a batch and throw it to MMDetection for testing

  list_test_img_batch = []
  list_test_img_name_batch = []
  for i in range(len(list_name_test_img)):
  	path_test_img = os.path.join(pathDirTestImg, list_name_test_img[i])
  	img = cv2.imread(path_test_img, cv2.IMREAD_COLOR)
  	list_test_img_batch.append(img)
  	list_test_img_name_batch.append(list_name_test_img[i])
  	if len(list_test_img_batch) == batchSize:
  		list_result_inf = inference_detector(model, list_test_img_batch)
  		for j in range(len(list_result_inf)):  # Single picture test results
  			img_src = list_test_img_batch[j]  # Single picture content
  			for k in range(len(class_dic)):  # Single label results in a single picture
  				scores = list_result_inf[j][k][:, -1] # All confidence levels of single label detection in a single image
                 	for jj in range(len(scores)):  # A certain confidence of single label detection in a single picture
                  	if scores[jj] > thersMinConfidence:  # thersMinConfidence: minimum alarm threshold
                  		# cv2.rectangle, cv2.putText and other image processing code
  			cv2.imwrite(os.path.join(result_path, list_test_img_name_batch[j]), list_test_img_batch[j])
  		list_test_img_batch.clear()
  		list_test_img_name_batch.clear()
  

I've written a lot of for loops. I'm not very used to it, but anyway, the amount of data is not large, so I can use it

In this way, the training and testing function of MMDetection2.0 which can quickly set some parameters through xml file is basically realized.