目录
数据特点
标准平面(SP)定位在常规临床超声(US)诊断中是必不可少的。与二维超声相比，三维超声可以一次扫描获得多个视图平面，并添加冠状面提供完整的解剖结构。
三维空间定位标准平面
| Papers & Code |
Short for Schemes |
Keywords |
| Automated Selection of Standardized Planes from Ultrasound Volume [MICCAI-MLMI 2011] |
2STC |
Sliding Window, Haar, AdaBoost, AS Detection, SP Classification |
| Learning-based scan plane identification from fetal head ultrasound images [Medical Imaging 2012] |
- |
Template Matching, Active Appearance Models, AS Detection, LDA, SP Classification |
| Intelligent Scanning: Automated Standard Plane Selection and Biometric Measurement of Early Gestational Sac in Routine Ultrasound Examination [Medical Physics 2012] |
IS |
Sliding Window, Haar, Cascade AdaBoost, AS Localization, Relative Position, Local Context Information, SP Classification |
| Selective Search and Sequential Detection for Standard Plane Localization in Ultrasound [MICCAI-CCCAI 2013] |
SSSD |
Haar, AdaBoost, Segmentation, Accumulative Vessel Probability Map, Selective Search, Geometric Relationship, Sequence AS Detection, SP Localization |
| Standard Plane Localization in Ultrasound by Radial Component [ISBI 2014] |
RCD |
Random Forest, Geometric Constrain, Radial Component, AS Detection, SVM, SP Localization |
| Automatic Recognition of Fetal Standard Plane in Ultrasound Image [ISBI 2014] |
FV-Aug |
AdaBoost, Dense Sampling Feature Transform Descriptor, Fish Vector, Spatial Pyramid Coding, Gaussian Mixture Model, SVM, SP Classification |
| Standard Plane Localization in Ultrasound by Radial Component Model and Selective Search [Ultrasound in Medicine and Biology 2014] |
RVD |
Random Forest, Geometric Constrain, Radial Component, Vessel Probability Map, Selective Search, AS Detection, SVM, SP Localization |
| Diagnostic Plane Extraction from 3D Parametric Surface of the Fetal Cranium [MIUA 2014] |
- |
Topological Manifold Representation, Landmark Alignment, 3D Parametric Surface Model, SP Localization |
| A Constrained Regression Forests Solution to 3D Fetal Ultrasound Plane Localization for Longitudinal Analysis of Brain Growth and Maturation [MICCAI-MLMI 2014] |
CRF-FA-Dist |
Informative Voxels, Reference Plane, Constrained Regression Forest, SP Localization |
| Automatic Recognition of Fetal Facial Standard Plane in Ultrasound Image via Fisher Vector [PLOS ONE 2015] |
FV-Chi2-SDCA |
Spatial Stacking, Densely Sampled Root Scale Invariant Feature Transform, Gaussian Mixture Model, Fisher Vector, Multilayer Fisher Network, SVM, SP Classification |
| Plane Localization in 3-D Fetal Neurosonography for Longitudinal Analysis of the Developing Brain [JBHI 2015] |
CRF-FA-Dist-M |
Informative Voxels, Manual Reference Plane, Constrained Regression Forest, SP Localization |
| Standard Plane Localization in Fetal Ultrasound via Domain Transferred Deep Neural Networks [JBHI 2015] |
T-CNN |
Knowledge Transfer, CNN, SP Localization |
| Automatic Fetal Ultrasound Standard Plane Detection Using Knowledge Transferred Recurrent Neural Networks [MICCAI 2015] |
T-RNN |
CNN, Knowledge Transfer, Joint Learning, Spatio-temporal Feature, RNN, SP Classification |
| Fetal Facial Standard Plane Recognition via Very Deep Convolutional Networks [EMBC 2016] |
- |
DCNN, SP Classification |
| Real-Time Standard Scan Plane Detection and Localisation in Fetal Ultrasound Using Fully Convolutional Neural Networks [MICCAI 2016] |
- |
CNN, Unsupervision, Saliency Maps, AS Localization, SP Classification |
| Ultrasound Standard Plane Detection Using a Composite Neural Network Framework [Transactions on Cybernetics 2017] |
T-RNN |
CNN, RNN, Composite Framework, SP Classification |
| SonoNet: Real-Time Detection and Localisation of Fetal Standard Scan Planes in Freehand Ultrasound [TMI 2017] [Official Code] [Third-Party Code] |
SonoNet |
CNN, SP Classification, Weakly Supervision, AS Localization |
| Automatic Detection of Standard Sagittal Plane in the First Trimester of Pregnancy Using 3-D Ultrasound Data [Ultrasound in Medicine and Biology 2017] |
- |
Deep Belief Network, Circle Detection, SP Classification |
| Attention-Gated Networks for Improving Ultrasound Scan Plane Detection [MIDL 2018] [Official Code] |
AG-SonoNet |
CNN, Attention, SP Classification, Weakly Supervision, AS Localization |
| Standard Plane Localisation in 3D Fetal Ultrasound Using Network with Geometric and Image Loss [MIDL 2018] |
- |
CNN, Rigid Transformation, Geometric Loss, Image Loss, SP Localization |
| Standard Plane Detection in 3D Fetal Ultrasound Using an Iterative Transformation Network [MICCAI 2018] [Official Code] |
ITN |
CNN, Rigid Transformation, SP Localization |
| Automatic and Efficient Standard Plane Recognition in Fetal Ultrasound Images via Multi-scale Dense Networks [MICCAI-DATRA/PIPPI 2018] |
MSDNet |
Multi-scale, Cascade, Dense Connection, CNN, SP Classification |
| SonoEyeNet: Standardized fetal ultrasound plane detection informed by eye tracking [ISBI 2018] |
SonoEyeNet |
CNN, Eye Tracking, Visual Heatmap, Information Fusion, SP Classification |
| Multi-task SonoEyeNet: Detection of Fetal Standardized Planes Assisted by Generated Sonographer Attention Maps [MICCAI 2018] |
M-SonoEyeNet |
Multi-task, CNN, Eye Tracking, GAN, Generator, Sonographer Attention, Discriminator, Predicted Attention, SP Classification |
| Agent with Warm Start and Active Termination for Plane Localization in 3D Ultrasound [MICCAI 2019] |
DDQN-AT |
Landmark Alignment, Reinforcement Learning, CNN, RNN, SP Localization |
| SPRNet: Automatic Fetal Standard Plane Recognition Network for Ultrasound Images [MICCAI-PIPPI/SUSI 2019] |
SPRNet |
CNN, Weight-share, Transfer Learning, SP Classification |
| Deep Learning-Based Methodology for Recognition of Fetal Brain Standard Scan Planes in 2D Ultrasound Images [IEEE Access 2019] |
- |
Data Augmentation, DCNN, Domain Transfer, SP Classification |
| Standard Plane Identification in Fetal Brain Ultrasound Scans Using A Differential Convolutional Neural Network [IEEE Access 2020] |
Different-CNN |
Differential Operator, Differential CNN, SP Classification |
| Evaluation of Deep Convolutional Neural Networks for Automatic Classification of Common Maternal Fetal Ultrasound Planes [Scientific Reports 2020] [Third-Party Code] |
- |
Data Augmentation, PCA, Hog, Boosting, VGG, MobileNet, Inception-v3, ResNet, SENet, SE-ResNet, DenseNet, SP Classification |
| Automatic Fetal Middle Sagittal Plane Detection in Ultrasound Using Generative Adversarial Network [Diagnostics 2020] |
- |
Segmentation, Object Detection, Seed Point, GAN, SP Localization |
| Recognition of Fetal Facial Ultrasound Standard Plane Based on Texture Feature Fusion [CMMM 2021] |
LH-SVM |
Local Binary Pattern, Histogram of Oriented Gradient, Feature Fusion, SVM, SP Classification |
| Principled Ultrasound Data Augmentation for Classification of Standard Planes [IPMI 2021] |
- |
Data Augmentation, Augmentation Policy Search, CNN, SP Classification |
| Generative Adversarial Networks to Improve Fetal Brain Fine-Grained Plane Classification [Sensors 2021] |
- |
GAN, Data Augmentation, CNN, SP Classification |
| Agent with Warm Start and Adaptive Dynamic Termination for Plane Localization in 3D Ultrasound [TMI 2021] [Official Code] |
AgentSPL |
Landmark Alignment, Reinforcement Learning, CNN, RNN, SP Localization |
| Autonomous Navigation of An Ultrasound Probe Towards Standard Scan Planes with Deep Reinforcement Learning [ICRA 2021] |
SonoRL |
Reinforcement Learning, Probe Navigation, Confidence-aware Agent, CNN, SP Localization |
| Searching Collaborative Agents for Multi-plane Localization in 3D Ultrasound [MIA 2021] |
MARL |
Multi-agent, Reinforcement Learning, RNN, NAS, SP Localization |
| Automatic Fetal Ultrasound Standard Plane Recognition Based on Deep Learning and IIoT [Transactions on Industrial Informatics 2021] |
FUSPR |
CNN, RNN, Spatial-temporal Feature, SP Classification |
| Automatic quality assessment for 2D fetal sonographic standard plane based on multitask learning [Medicine 2021] |
- |
CNN, AS Classification, Object Detection, AS Localization, SP Quality Control |
| Statistical Dependency Guided Contrastive Learning for Multiple Labeling in Prenatal Ultrasound [MICCAI-MLMI 2021] |
MLL-GCN-CRC |
Word Embedding, GCN, CNN, Cluster Relabeled Contrastive Learning, Multi-label, AS Classification, SP Classification |
| Agent with Tangent-based Formulation and Anatomical Perception for Standard Plane Localization in 3D Ultrasound [MICCAI 2022] |
- |
Reinforcement Learning, Restructure the Action Space, Content-aware Regression Auxiliary Task, Spatial-anatomical Reward, CNN, Landmark Heatmap, SP Localization |
Tags: Standard Plane --> SP | Anatomical Structure --> AS
基于监督学习方法
大多采用分类平面图像和回归平面参数的方式来得到标准平面，但三维搜索空间巨大，分类和回归没有充分利用环境的信息，只是学习了单一的映射关系，这种学习方式不够有效。
同时网络难以有效学习到高维到低维图像的映射。
论文+代码见 超声标准平面检测论文+代码+数据集+练习合集
基于强化学习方法
临床中，医生手动定位标准切面时会根据经验对当前视图平面进行判断进而调整探头的角度和方向，以靠近目标平面。
而强化学习的一个搜索过程，在该任务中就体现了天然的优势，智能体可以通过与环境进行交互，在交互的过程中，智能体可以获得当前时刻的状态，根据当前时刻的状态输出动作，并且获得奖励，进而到达下一个状态，整个过程可以整个过程的目标可以被建模成通过选择合适的动作来最大化奖励的过程，这种方式与医生手动定位标准切面的方式是非常近似的。
[1,2]首先提出了基于强化学习的三维超声标准切面定位框架。
他们设计了一种registration-based warm-up，以解决超声体积的大方向变异性，并为强化学习中的智能体提供有效的初始化。
在他们的后续工作[3]中，他们在强化学习优化中嵌入了神经网络搜索，并设计了一个用于多标准切面导航的多智能体协作系统。
目前的研究[1,2,3]依赖于初始配准（registration）来确保数据方向的一致性。尽管[1,2,3,5,6]在标准切面本地化中实现了高性能，但仍有一些问题需要解决。
1.当registration-based失败时，它们很容易被捕获。
2.大多数研究[1,2,3]根据角度和距离设计了一个八维动作空间，其中公式中方向余弦之间的耦合（即不能唯一确定一个平面）和搜索空间巨大的，使得优化变得困难。
3.现有的强化学习系统[1,2,3]仅由基于平面运动的奖赏功能驱动，缺乏对解剖结构的感知和引导。
[4]解决了这三个问题。
【1】Dou, H., Yang, X., Qian, J., Xue, W., Qin, H., Wang, X., Yu, L., Wang, S., Xiong,
Y., Heng, P.A., et al.: Agent with warm start and active termination for plane local
ization in 3d ultrasound. In: International Conference on Medical Image Computing
and Computer-Assisted Intervention. pp. 290–298. Springer (2019)
| 用于 3D 超声平面定位的热启动和主动终止智能体[MICCAI 2019] | DDQN-AT | 特征点对齐、强化学习、CNN、RNN、SP 定位 |
【2】Yang, X., Dou, H., Huang, R., Xue, W., Huang, Y., Qian, J., Zhang, Y., Luo,
H., Guo, H., Wang, T., et al.: Agent with warm start and adaptive dynamic ter
mination for plane localization in 3d ultrasound. IEEE Transactions on Medical
Imaging (2021)
| 具有热启动和自适应动态终止的智能体，用于 3D 超声中的平面定位[TMI 2021] [官方代码] | 智能体声压级 | 特征点对齐、强化学习、CNN、RNN、SP 定位 |
| Agent with Warm Start and Adaptive Dynamic Termination for Plane Localization in 3D Ultrasound [TMI 2021] [Official Code] | AgentSPL | Landmark Alignment, Reinforcement Learning, CNN, RNN, SP Localization |
【3】Yang, X., Huang, Y., Huang, R., Dou, H., Li, R., Qian, J., Huang, X., Shi, W.,
Chen, C., Zhang, Y., et al.: Searching collaborative agents for multi-plane local
ization in 3d ultrasound. Medical Image Analysis p. 102119 (2021)
| 在 3D 超声中搜索多平面定位的协作智能体[MIA 2021] | MARL | 多智能体、强化学习、RNN、NAS、SP 本地化 |
【4】Agent with Tangent-based Formulation and Anatomical Perception for Standard Plane Localization in 3D Ultrasound
| 用于 3D 超声标准平面定位的基于切线公式和解剖感知的智能体[MICCAI 2022] | 核心：强化学习（空间解剖奖励） 提升：重构动作空间、内容感知回归辅助任务（状态内容相似度预测(SCSP)）、模仿学习的初始化（强化学习的预训练） 其他：CNN、特征点热图、SP 定位 |
在[1,2]的激励下，[5,6]利用强化学习实现了超声探测器对标准切面的自主导航。
【5】. Li, K., Wang, J., Xu, Y., Qin, H., Liu, D., Liu, L., Meng, M.Q.H.: Autonomous
navigation of an ultrasound probe towards standard scan planes with deep re
inforcement learning. In: 2021 IEEE International Conference on Robotics and
Automation (ICRA). pp. 8302–8308. IEEE (2021)
| 通过深度强化学习实现超声探头向标准扫描平面的自主导航[ICRA 2021] | SonoRL | 强化学习、探针导航、信心感知智能体、CNN、SP 定位 |
【6】Li, K., Xu, Y., Wang, J., Ni, D., Liu, L., Meng, M.Q.H.: Image-guided navigation
of a robotic ultrasound probe for autonomous spinal sonography using a shadow
aware dual-agent framework. IEEE Transactions on Medical Robotics and Bionics
(2021)
| 基于阴影感知双agent框架的自主脊柱超声成像的机器人超声探针的图像引导导航[TMI 2021] | 双agent | 强化学习、双agent、探针导航、信心感知智能体、CNN、SP 定位、SonoQNet |
配准registration
在做医学图像分析时 ,经常要将同一患者几幅图像放在一起分析 ,从而得到该患者的多方面的综合信息 。
对几幅不同的图像作定量分析 ,首先要解决这几幅图像的严格对齐问题 ,这就是我们所说的图像的配准。
医学图像配准 是指对于一幅医学图像寻求一种 (或一系列 )空间变换 ,使它与另一幅医学图像上的对应点达到空间上的一致。 这种一致是指人体上的同一解剖点在两张匹配图像上有相同的空间位置。 配准的结果应使两幅图像上所有的解剖点 ,或至少是所有具有诊断意义的点及手术感兴趣的点都达到匹配。
基本变换举例：
投影变换: 与仿射变换相似 ,投影变换 [6] 将直线映射为直线 ,但不再保持平行性质。投影变换主要用于二维投影图像与三维体积图像的配准。