Seg4Reg Networks for Automated Spinal Curvature Estimation

Seg4Reg network for automatic spine curvature estimation

【Paper reading】

Abstract

Propose a pipeline for accurate scoliosis assessment

framework：

Seg4Reg, first obtains the mask through the segmentation network, and then the regression network directly predicts the cobb angle based on this.

+a domain adaptation module to alleviate domain shift issues

Output: Integrate the prediction results of different models in the ensemble

Intro

There are two ways to evaluate bobb:

① Predict landmarks and then calculate angles: high accuracy but too dependent on landmarks, small errors lead to big errors

② Direct regression cobb: The method is stable but the prediction is not accurate enough (this article proves that this method is better)

Text: MICCAI AASCE 2019 challenge

Method

seg: similar to PSPNet, reg: traditional classification model

Preprocessing:

The train/test areas are different ⇒ Histogram equalization makes them visually similar (because there are not many tests, manual cropping and data enhancement)
Adding an additional "gap~bones" before seg can improve the performance of the segmentation model (regularization training process)

Using ImageNet pre-trained classification network (? Who to use it on) can improve limited training samples;

Use [3] to add a discriminator branch to the domain gap between train/test, and reverse the gradient during backpropagation (the loss is as follows, this article λ =1)

Network training parameters and configuration:

Experimental results

Local Validation

L1 loss：

Table 1: Ablation experiments for input types and sizes

⇒segmentation mask is the best input type and (512, 256) is the best input size

Table 2: Network performance for different partitions

⇒adding a dilation pyramid thus improves the performance of previous PSPNet

Purpose: Choose the most appropriate one (it also further explains the rationality of our choice of image size and network)

Online test

related information

domain shift

dilated convolution action

Adam optimizer

Vocabulary accumulation

alleviate, alleviate, mitigate

【Experiment Reproduction】

1、mat2csv QA

import os.path

import pandas as pd
import scipy
from scipy import io
# 'data/labels/train/sunhl-1th-02-Jan-2017-162 A AP.jpg.mat'

def mat2csv(mat_file_path):
    features_struct = scipy.io.loadmat(mat_file_path)
    # print(features_struct)
    features = features_struct['p2']
    dfdata = pd.DataFrame(features)
    datapath1 = mat_file_path[:-4]+'.csv'
    dfdata.to_csv(datapath1, index=False, header=False)

if __name__ == '__main__':
    # data_root = 'data/labels/val'
    # mat_files =[os.path.join(data_root,filename) for filename in os.listdir(data_root) if filename.endswith('.mat')]
    # print(mat_files)
    # for item in mat_files:
        # mat2csv(item)

    labels = pd.read_csv('data/labels/train/sunhl-1th-02-Jan-2017-162 A AP.jpg.csv', header=None).values
    print(labels)

(1) refer: python reads mat and converts it to csv

(2) The first line in the output csv file is 0,1, resulting in 69 lines > 68 points

Pandas DataFrame DataFrame.to_csv() function header=False comment, line index=False

(3) It is found that the copied data does not have landmark.csv and angle.csv, and the data obtained is incorrect => This step is not necessary.

Like yolo before, the os.py system file is not modified.

os.mkdir() method

2. The file visual_dir gives an output file location, and so does data_dir.

3. ModuleNotFoundError: No module named 'libs.black_list' There is nothing written in it, just comment it directly and a new error will be reported

A: The folder "You can only use the ultimate move" is marked as [Source Root] Done

4. IndexError: index 131 is out of bounds for axis 0 with size 128 index exceeds the length

Finally, the reason was found to be that one of the points had a value > 1, resulting in 131; try changing it to 0.99999.

5、TypeError: can't convert cuda:0 device type tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first. 原因

Using the elimination method, the problem can only be caused by this loss.list. Adding float will report an error.

Refer to the original parts in train.py, which should be: tensor to numpy-√ , detach() ,

a.detach().numpy()
a.cpu().detach().numpy()

6、IndexError: too many indices for array: array is 1-dimensional, but 2 were indexed

reason , dimension

Solution: After debugging, I found that the test_results_list in this step is empty (click on the test function, this parameter is invalid)

[Reproduction | Paper] Seg4Reg Networks for Automated Spinal Curvature Estimation