Fine, click on the " code farmer's back garden " above, select the " star " public account
Featured articles, delivered immediately
Last issue explained the basic structure of the semantic segmentation model and commonly used data sets. This issue will explain the production of semantic segmentation data sets.
Overall production steps:
1. Use labelme to annotate the image data and generate the x.json file corresponding to the image.
2. Execute the built-in function labelme_json_to_dataset under labelme, and manually generate the x_json file corresponding to the picture in turn (or use the code to process and generate it at one time).
3. Process the files in the folder generated in the second step to generate the semantic image label.png.
4. Convert the semantic image into a category grayscale image-the final training label file.
1. File directory structure:
2. Officially start production
The first step: the installation of the marking software
1. Create an environment in Anaconda Prompt
conda create --name=labelImg python=3.6
2. Activate and enter the new environment just established,
conda activate labelImg
3. The installation interface supports the pyqt5 package
pip install pyqt5 -i https://pypi.douban.com/simple/
4. Download and install labelme
pip install labelme -i https://pypi.douban.com/simple/
5. Enter the command labelme to start the program for data labeling
Step 2: Make an annotation
A. Single category annotation-that is, there is only one target on each picture
[1] Enter the command labelme in the command line, open the labeling interface, and then open the folder where the picture to be labeled is located for labeling
Opendir “”Test_Image“” ->Create polygons ->Save->Next Image
[2] After all the pictures are marked, the marked file is saved as an x.json file and placed in the before folder of the directory after the production is completed.
Use VS2017 to view the content of the Json file, including the information for each pixel in the area we marked
[3] Use labelme's own function labelme_json_to_dataset to manually convert each json file format into semantic image data.
1. cd to the place where the json file is located: cmd D:Test_Image
2. View the label of the environment where the current labelme is installed and activate it
3. Use labelme_json_to_dataset to manually process the annotation file x.json format of each picture in turn, and generate the x_json file, which is the corresponding semantic picture of each picture-partial category label (this process requires manual correction of the x. .json file processing, more troublesome, you can write code to convert directly)
Command: labelme_json_to_data [x.json] -> json file name generated by labelme
The partial category label corresponding to the x.json file of each picture is saved in each x_json file, as shown below:
The content of the local category tag content of each picture x_json file:
annotation:
img.png: The corresponding partial original jpg file, which should be used during training
label.png: Annotate semantic images, used during training
label_names.txt: target category name on this image
label_viz.png: label visualization, used to confirm whether the label is correct
One-time code conversion: json_to_dataset.py
Will generate the same file content as the next x_json file above
import argparseimport jsonimport osimport os.path as ospimport warnings
import PIL.Imageimport yaml
from labelme import utilsimport base64
def main(): count = os.listdir("./before/") for i in range(0, len(count)): path = os.path.join("./before", count[i])
# 找到before文件中以json为结尾的文件并且打开if os.path.isfile(path) and path.endswith('json'): data = json.load(open(path))
if data['imageData']: imageData = data['imageData']else: imagePath = os.path.join(os.path.dirname(path), data['imagePath'])with open(imagePath, 'rb') as f: imageData = f.read() imageData = base64.b64encode(imageData).decode('utf-8') img = utils.img_b64_to_arr(imageData) label_name_to_value = {'_background_': 0}for shape in data['shapes']: label_name = shape['label']if label_name in label_name_to_value: label_value = label_name_to_value[label_name]else: label_value = len(label_name_to_value) label_name_to_value[label_name] = label_value
# label_values must be dense label_values, label_names = [], []for ln, lv in sorted(label_name_to_value.items(), key=lambda x: x[1]): label_values.append(lv) label_names.append(ln)assert label_values == list(range(len(label_values)))
lbl = utils.shapes_to_label(img.shape, data['shapes'], label_name_to_value)
captions = ['{}: {}'.format(lv, ln)for ln, lv in label_name_to_value.items()] lbl_viz = utils.draw_label(lbl, img, captions) out_dir = osp.basename(count[i]).replace('.', '_') out_dir = osp.join(osp.dirname(count[i]), out_dir) out_dir = osp.join("output",out_dir)
if not osp.exists(out_dir): os.mkdir(out_dir)
PIL.Image.fromarray(img).save(osp.join(out_dir, 'img.png'))
utils.lblsave(osp.join(out_dir, 'label.png'), lbl) PIL.Image.fromarray(lbl_viz).save(osp.join(out_dir, 'label_viz.png'))
with open(osp.join(out_dir, 'label_names.txt'), 'w') as f:for lbl_name in label_names: f.write(lbl_name + '\n')
warnings.warn('info.yaml is being replaced by label_names.txt') info = dict(label_names=label_names)with open(osp.join(out_dir, 'info.yaml'), 'w') as f: yaml.safe_dump(info, f, default_flow_style=False)
print('Saved to: %s' % out_dir)
if __name__ == '__main__': main()
After we get the local semantic category label corresponding to each picture (for each picture, after the grayscale category is converted, it belongs to the marking target, such as a cat, the value of the pixel in the area is 1, which is not the picture of the marking target The pixel value of the area (background) is 0).
If a new image is added to the data set, there is only one target dog on this image. In order to add the target dog category to the training set, the local category label of dog needs to be converted into a global category label, that is, during training, the background The pixel value is 0, the pixel value of the cat is 1, the pixel value of the dog is 2, and so on)
[4] Local category label—>Global category label (category gray scale conversion)
Convert the obtained local training label file label.png corresponding to each picture into a global training label file and place it in the png folder. This process also involves category grayscale operations.
Perform the get_jpg_and_png.py operation to get the global category label file and category grayscale conversion of each picture, that is, the value of the background pixel in a picture is 0, if the target is a cat, the pixel value is 1. If the target is a dog, its pixel value is 2.
The final global training label here is a grayscale image, which is a single channel. The pixel value range is 0~255. Here, it is all black because the pixel value is 1 and 2. In the grayscale image, the value is relatively small. , The difference is relatively small, so the display is all black.
Category grayscale image:
We can see that the value of the pixels in the area where the category grayscale image corresponding to the cat picture is located are all 1, while the values of the pixels corresponding to the background are all 0
1. Here we need to first create a new class_name.txt containing the global class name
2. With the help of the class_name.txt file, convert the image local category label file into a global category label file, the conversion code: get_jpg_and_png.py
import os
from PIL import Image
import numpy as np
def main():
# 读取原文件夹
count = os.listdir("./before/")
for i in range(0, len(count)):
# 如果里的文件以jpg结尾
# 则寻找它对应的png
if count[i].endswith("jpg"):
path = os.path.join("./before", count[i])
img = Image.open(path)
img.save(os.path.join("./jpg", count[i]))
# 找到对应的png
path = "./output/" + count[i].split(".")[0] + "_json/label.png"
img = Image.open(path)
# 找到全局的类
class_txt = open("./before/class_name.txt","r")
class_name = class_txt.read().splitlines()
# ["bk","cat","dog"] 全局的类
# 打开x_json文件里面存在的类，称其为局部类
with open("./output/" + count[i].split(".")[0] + "_json/label_names.txt","r") as f:
names = f.read().splitlines()
# ["bk","dog"] 局部的类
# 新建一张空白图片
new = Image.new("RGB",[np.shape(img)[1],np.shape(img)[0]])
# 找到局部的类在全局中的类的序号
for name in names:
# index_json是x_json文件里存在的类label_names.txt，局部类
index_json = names.index(name)
# index_all是全局的类,
index_all = class_name.index(name)
# 将局部类转换成为全局类
# 将原图img中像素点的值为index_json的像素点乘以其在全局中的像素点的所对应的类的序号 得到 其实际在数据集中像素点的值
# 比如dog,在局部类（output/x_json/label_names）中它的序号为1,dog在原图中的像素点的值也为1.
# 但是在全局的类（before/classes.txt）中其对应的序号为2，所以在新的图片中要将局部类的像素点的值*全局类的序号，从而得到标签文件
new = new + np.expand_dims(index_all*(np.array(img) == index_json),-1)
new = Image.fromarray(np.uint8(new))
# 将转变后的得到的新的最终的标签图片保存到make_dataset/png文件夹下
new.save(os.path.join("./png", count[i].replace("jpg","png")))
# 找到新的标签文件中像素点值的最大值和最小值，最大值为像素点对应的类在class_name.txt中的序号，最小值为背景，即0
print(np.max(new),np.min(new))
if __name__ == '__main__':
main()
3. Finally, we use get_train_txt.py the contents of the above documents obtained under jpg and png files to one correspondence between data sets and original label, in its train_data.txt
[5] The style of the final training model data set: jpg folder (original image) + png folder (global gray scale category label) + train_data.txt (correspondence between the original image and the label), put it into the model to train Now, the next issue will explain the semantic segmentation model of SegNet based on MobileNet.
B. Multi-category annotation-that is, there are multiple targets on a picture
The marking process is similar to the above. The multiple targets on the picture are marked in sequence, and the content of the generated x_json file is the same as the single target marking.
Create a new class_name.txt containing the global class name , and then convert the gray scale of the category, use get_jpg_and_png.py to convert, we can see that the pixel value of the area corresponding to the cat is 1, corresponding to category 1, and the same is true for the pixels of the corresponding area of the dog The value of the point is 2, corresponding to category 2, and the value of the pixel in the background area is 0
After class gray conversion, the model can be trained.
The style of the final training model data set: jpg folder (original image) + png folder (global grayscale category label) + train_data.txt (correspondence between the original image and the label), put it into the model to train, the next issue Explain the semantic segmentation model of SegNet based on MobileNet.
Alright, the data labeling part is over here, all processing file codes, reply keyword: project actual combat , you can get it.
Wonderful recommendation:
New tutorial image segmentation series
Image segmentation series <-> semantic segmentation
More premium content? Waiting for you to watch