ChatGPT exploded in popularity at the beginning of the year, and the American "Times" published an article exposing the "sweatshop" behind it, casting a huge shadow on the dazzling aura of the "AIGC leader".
According to reports, in order to train ChatGPT to identify harmful content, a group of outsourced employees from Kenya, Uganda and India were responsible for manually labeling the database. They spent 9 hours a day
identifying various offensive words, labeling up to 20,000 words per hour, and the harvest was 1.32 -$2 an hour and lasting psychological trauma.
As soon as the report came out, public opinion was in an uproar. This can be called the most unintelligent operation in the era of artificial intelligence. Putting aside labor issues, this incident also reflects a major pain point in supervised learning:
preparing large-scale, pre-labeled training data for AI models, especially large models, is quite labor-intensive.
Author | Iron Tower
Editor | Sanyang
This article was first published on the HyperAI Super Neural WeChat public platform~
In the field of holography microscopic reconstruction , this problem has also brought protracted torture to scientific research work.
Existing applications of deep learning in computational imaging and microscopy mainly rely on supervised learning and require large-scale, diverse and pre-annotated training data, but the acquisition and preparation of such training image datasets are often laborious and costly are expensive, and these methods often have limited generalization to new sample types.
On August 7 this year, the UCLA research team reported a self-supervised model called GedankenNet in the magazine "Nature Machine Intelligence". Its outstanding feature is that it does not require real data or experimental subjects to be fed, and can be directly derived from thought experiments ( Thought experiment) and physical laws, and has excellent external generalization.
Paper link:
https://www.nature.com/articles/s42256-023-00704-7
GedankenNet comes from the German Gedankenexperiment, which means "thought experiment". It tells you clearly:
I, GedankenNet, am different from the AI models outside that learn from real data and experimental subjects. My learning subjects are thought experiments used by scientists such as Einstein !
This model is expected to eliminate the shortcomings of deep learning in the field of holographic micrograph reconstruction and create new opportunities for solving inverse problems in holography, microscopy and computational imaging.
Holographic micrograph reconstruction
First, let’s take a brief look at the application field of GedankenNet—holographic micrograph reconstruction.
Digital holographic microscopy is a label-free imaging technology widely used in biomedical and physical sciences and engineering. Compared with traditional two-dimensional microscopy, holographic microscopy provides a more comprehensive , a non-destructive, high-resolution microscopic observation and analysis method that uses optical interference technology to restore the three-dimensional shape of the original object from the collected hologram data, which can help scientists and researchers better understand and study the microscopic world.
How to perform microscopic hologram reconstruction? Traditional methods are mainly divided into two categories:
Iterative phase recovery algorithm based on physical forward model and iterative error reduction ;
Inference methods based on supervised deep learning .
The first type of methods gradually improves the estimate of the complex field through iterative optimization . A physical model is first used to calculate the error between the theoretical hologram caused by the complex field and the actual measured hologram, and then adjustments are made based on this error, and the process is repeated until a certain accuracy is achieved.
The second type of method trains a deep neural network to learn the mapping from input holograms to complex fields . Usually pairs of holograms and corresponding complex fields are used as training data, and the neural network learns the relationship between these data pairs, thereby Prediction and reconstruction of complex number fields.
Note: The complex field describes the distribution of optical properties of the object, including the amplitude and phase information of the light field.
However, these traditional methods usually require multiple iterations to adjust and optimize the predicted hologram, which is slow. GedankenNet adopts a completely different idea, which not only avoids the iterative process, but also achieves better reconstruction results and faster speed.
Approach GedankenNet
Model training
1. Method
Unlike existing learning-based methods, GedankenNet does not directly compare the difference between the output complex field and the real complex field. Instead, it learns from the physical consistency by observing the input hologram and the corresponding target output (such as a clear image). Complex field reconstruction patterns are learned from the constrained data and corresponding hologram predictions are generated without the need for stepwise iterative adjustments.
The physical consistency loss is a core component of the GedankenNet training method , which measures the accuracy of the reconstruction results based on the difference between the conditions of the wave equation (Wave equation) and the actual observation data.
By minimizing the loss of physical consistency, the model is able to match the observed data to the expected behavior of the wave equation, resulting in a more physically consistent hologram reconstruction.
2. Dataset
The training dataset for GedankenNet consists of artificial holograms generated from random images or simulated from natural images (COCO dataset) using the Python random image package (with no connection or similarity to real-world samples). ).
a. Classic iterative hologram reconstruction algorithm, self-supervised deep neural network (GedankenNet) and existing supervised deep neural network
b. Self-supervised training process of GedankenNet
Model validation
1. Reconstruct image quality
According to the training process in Figure 1, the researchers trained a series of self-supervised network models that accepted multiple input holograms (M ranged from 2 to 7), and used the MHPR (Multi-height phase retrieval) multi-height phase retrieval algorithm to extract data from each hologram. Real object pictures were extracted from 8 original holograms in each field of view, and based on this, the performance of different GedankenNet models in terms of reconstructed image quality was comparatively evaluated.
The results show that all GedankenNet models reconstruct the sample field with high fidelity even without using experimental data for training.
2. External generalization ability
In order to verify the external generalization ability of GedankenNet, the researchers compared the GedankenNet model with other supervised learning models ( trained based on the same artificial image training set) and iterative phase recovery algorithms . The test data included human tissue sections and cervical smear. Experimental hologram included.
As shown in the figure below, compared to these supervised learning methods, GedankenNet shows better external generalization on all 4 samples (lung, salivary gland and prostate tissue sections and Pap smear) , getting Higher Enhanced Correlation Coefficient (ECC) value.
In addition, the researchers also conducted a comparative analysis on the classic iterative phase recovery algorithm, namely MHPR. The results show that the complex fields inferred by GedankenNet have less noise and higher image fidelity than MHPR(M=2) using the same input hologram .
a. External generalization of GedankenNet on human lung, salivary gland, prostate, and Pap smear holograms, and comparison with existing supervised learning models and MHPR
b. External generalization results of the supervised learning method on the same test data set. These supervised learning models are trained using the same simulated hologram dataset as GedankenNet
c. MHPR reconstruction results using the same M = 2 input hologram.
d. Results of ground truth complex fields acquired using 8 raw holograms per field-of-view (FOV, fields-of-view). Scale bar: 50 μm
As shown in Table 1, compared with MHPR (M = 2), GedankenNet accelerates the image reconstruction process by about 128 times.
In summary, these holographic imaging experiments and result analysis successfully prove GedankenNet's superior generalization ability to unknown new samples and achieve excellent image reconstruction performance.
3. Generalization to other training data
To prove that GedankenNet trained on other data sets also has the same performance, the researchers used the following data sets to separately train three GedankenNet models:
(1) Artificial hologram dataset generated from random images, same as before
(2) New artificial hologram data set generated from natural image data set (COCO)
(3) Experimental hologram data set of human lung tissue sections
The three separately trained GedankenNet models were tested on four test data sets , including artificial holograms of randomly synthesized images, artificial holograms of natural images, and experimental holograms of human lung tissue sections and cervical smears.
The results show that all self-supervised GedankenNet models exhibit excellent image reconstruction quality in terms of internal and external generalization (Fig. 4a, b).
It is worth noting that, as shown by the red bars in Figure 4b, the performance gap between the internal and external generalization performance of the supervised model is large, indicating its overfitting phenomenon. In contrast, the GedankenNet model (blue bars) demonstrates very good generalization performance across test datasets of natural macroscopic images and microscopic tissue images.
4. Compatibility with wave equation
The same model as in Figure 3 was used to analyze whether GedankenNet is compatible with the wave equation by blind testing on lung tissue sections.
The results show that when tested with defocused holograms, GedankenNet outputs a correct (physically consistent) defocused complex field, rather than an illusory and non-physical random light field.
In this sense, GedankenNet not only demonstrates superior external generalization capabilities (from training without experiments and data to experimental holograms), but also adapts well to working with off-focus experimental holograms . In the previous literature, no hologram reconstruction neural network has shown these properties.
Does AI have "intelligence"?
As a self-supervised artificial intelligence model, GedankenNet eliminates the need for large-scale, pre-annotated training data , demonstrating superior external generalization capabilities and high-quality hologram reconstruction results. As Aydogan Ozcan, one of the authors of the paper and a professor in the Department of Electrical and Computer Engineering and Bioengineering at UCLA, said:
"These findings illustrate the potential for self-supervised artificial intelligence to learn from thought experiments, just like scientists do. It opens new opportunities to develop physically compatible, easily trainable, and broadly general neural network models that can serve as alternatives to the current applications in a variety of computational imaging Standard, supervised deep learning methods used in the task.”
People have always been arguing about whether AI has true intelligence. After all, even if it is as powerful as AlphaGo, which has defeated multiple world Go champions and captured the fortress of human wisdom, its essence is that it does not need to understand the rules and relies on computing power to continuously repeat mathematical formulas. That’s all.
But now that the advent of the GedankenNet model, which can learn from thought experiments like scientists, does it mean that AI has the unique "wisdom" of humans to some extent? Everyone is welcome to speak freely in the message area.
Reference links:
https://www.sciencedaily.com/releases/2023/08/230807122001.htm
This article was first published on the HyperAI Super Neural WeChat public platform~