Embedded physics (PINN) or physics-based (AD)?

For the content of PINN, you can directly google PINN (Physical-informed neural network). Its main purpose is to use a neural network to fit the physical process, and achieve the purpose of model driving by integrating physical equations. But for many problems, blindly using end-to-end neural network mapping is not the best approach. If we already have a very clear forward representation, according to the concept of deep learning calculation graph, the gradient can be automatically solved by automatic differentiation; at the same time, because the deep learning framework (Pytorch/tensorflow) provides us with a large number of The optimal update algorithm (eg Adam, LBFGS, SGD), we can integrate these optimization algorithms and update strategies (eg StepLR) into the self-weight inversion problem; its core advantages are:

1. The gradient of the automatic differential solution is completely based on the chain rule (Chain rule), so the obtained gradient is an analytical solution without any approximation.
2. The convenient GPU acceleration provided by the deep learning framework makes the inversion problem very efficient . Uncertainty estimation of inversion results can be done very simply by using methods such as Montecarlo .

Wall Crack ~ Recommended

[1] A seismic surface wave inversion tool based on the Pytorch automatic differential framework (a little simpler than the full waveform inversion) ADsurf

[2] ADseismic, a seismic full waveform inversion tool based on julia and tensorflow

1. The traditional "inversion problem"

First of all, it needs to be explained that the following introductions are all personal understanding.

1.1 What is the inversion problem

Inversion problem-baidubaike : The inversion problem refers to determining the parameters (or model parameters) that characterize the characteristics of the problem based on the results and some general principles (or models). In engineering applications, inversion problems widely appear in the fields of geophysics, biology, medicine and construction. The inverse problem exists relative to the forward problem, and its general working procedure is data-forward model-estimation of model parameters. Therefore, the forward calculation model is the basis of the inverse problem. Only when the numerical solution of the forward problem is obtained, can it be considered to solve the inverse problem

Inversion problem-ChatGPT:

1.2 Common inversion problems

1. Seismic Velocity Inversion: Use seismic wave data to estimate the velocity structure of the Earth's interior to understand the material composition and structure of the Earth's interior.

2. Medical Imaging Inversion: Using imaging techniques such as X-rays, CT scans, or MRI to create images of the inside of the body to understand the location and morphology of organs, bones, and other structures.

3. Non-destructive testing inversion: Using acoustic, electromagnetic, or optical techniques to measure specific properties of matter (such as density, electrical conductivity, or refractive index) to understand the internal composition and structure of a material.

4. Atmospheric Remote Sensing Retrieval: Using sensors on satellites or aircraft to measure atmospheric parameters such as temperature, humidity, and air pressure to understand chemical and physical processes in the atmosphere.

5. Power system inversion: Use current and voltage data of a power network to estimate unknown parameters in the network (such as line impedance and generator capacity) to understand the operating state and stability of the power system.

6. Gravity Inversion: Use gravity data to estimate the density structure of the Earth's interior to understand the material distribution and structure of the Earth's interior.

7. Magnetic Inversion: Use magnetic data to estimate the magnetic structure of the Earth's interior to understand the distribution and structure of the Earth's interior.

8. Hydrogeological inversion: The use of groundwater table, groundwater flow velocity, and groundwater chemistry data to estimate the nature and distribution of groundwater reservoirs to understand the use and management of groundwater resources.

9. Petrophysical Inversion: Uses seismic, electromagnetic, gravity, and magnetic data to estimate physical properties of rocks such as density, velocity, electrical conductivity, and magnetic susceptibility to understand geological structure and prospect for mineral deposits.

10. Meteorological inversion: Use meteorological data to estimate parameters such as temperature, humidity, air pressure, and wind speed in the atmosphere to understand weather and climate change.

1.3 The dilemma of the traditional inversion problem

1. Inaccurate models: Traditional inversion methods typically rely on building a mathematical model to describe the relationship between observed data and system properties. However, the models are often simplified, ignoring some complex physical phenomena, and thus may be inaccurate.

2. Incomplete data: Sometimes, we only have part of the observed data, and the data may contain noise or errors. This makes the inversion process more difficult as we need to infer unknown parameters without complete information.

3. High computational complexity of the inversion process: Inversion problems usually involve the processing and calculation of large amounts of data. Traditional inversion methods may require hours or days of computation time, thus limiting the scope of application of inversion problems.

4. Unstable results (non-uniqueness): Sometimes, there may be more than one possible solution to an inversion problem, which makes the results unstable. Furthermore, the results of an inversion problem can also be sensitive to choices such as initial guesses or regularization parameters.

5. Overfitting: In some cases, the inversion process can overfit the data, resulting in inaccurate estimated parameter values.

6. Difficulty dealing with nonlinear problems: Many inversion problems involve systems of nonlinear equations, which makes it difficult for traditional linear inversion methods to handle these problems.

2. Advantages of deep learning

Using automatic differentiation techniques brings some of the following advantages, which can be applied to traditional inversion methods:

1. High precision: The automatic differentiation technique can calculate the gradient accurately, so the precision and accuracy of the inversion results can be improved.

2. High computational efficiency: Compared with the traditional numerical differentiation method, the automatic differentiation technique has higher computational efficiency because it can automatically calculate the gradient and avoid the problems of repeated calculation and storage of derivatives.

3. More stable inversion process: automatic differential technology can avoid overfitting problems and unstable results in traditional inversion methods, thereby improving the stability of inversion results.

4. More robust to nonlinear problems: The automatic differentiation technique can effectively handle nonlinear problems because it can automatically calculate higher-order derivatives and does not need to differentiate analytical functions.

5. Ease of implementation and application: Automatic differentiation techniques have been widely used in many scientific fields, and there are many mature software libraries that can be easily integrated into inversion algorithms.

Overall, using automatic differentiation techniques can improve the efficiency and accuracy of inversion algorithms, thereby helping us better understand and solve complex inversion problems.

3. Example of AD + inversion

3.1 ADsurf

    ADsurf is a seismic surface wave inversion tool based on Pytorch, modified according to the Fortrun program of Hermann, 2002. It automatically calculates the gradient of the loss function with respect to the input parameters through the Automatic Differentiation framework, and then uses the optimization algorithm in the deep learning framework such as Adam to perform automatic model optimization. At the same time, it uses GPU acceleration and matrix calculation to improve the inversion performance Efficiency can be used for efficient subsurface shear wave velocity inversion and uncertainty estimation of inversion results. The program has been open sourced in Github: https://github.com/liufeng2317/ADsurf .

3.2 ADseismic

    ADseismic is a tool for full waveform inversion (FWI) in seismic imaging. It uses automatic differentiation techniques to efficiently compute the sensitivity of the seismic waveform to subsurface model parameters, and then iteratively updates the model parameters to minimize the difference between the observed and simulated waveforms. The goal of FWI is to obtain high-resolution images of subsurface structures, which can be used to identify geological features such as faults, fractures, and oil and gas reservoirs. ADseismic has proven to be a powerful FWI tool and has been used in various seismic imaging studies.