Advantages and disadvantages of neural networks,
Advantages: (1) With self-learning function. For example, when realizing image recognition, we only need to input many different image templates and corresponding recognition results into the artificial neural network, and the network will gradually learn to recognize similar images through self-learning function. The self-learning function is particularly important for forecasting.
It is expected that artificial neural network computers in the future will provide human beings with economic forecasts, market forecasts, and benefit forecasts, and their application prospects are very promising. (2) With association storage function. This kind of association can be realized with the feedback network of artificial neural network. (3) It has the ability to find the optimal solution at high speed.
Finding an optimal solution to a complex problem often requires a large amount of calculations. Using a feedback-type artificial neural network designed for a certain problem and utilizing the high-speed computing power of a computer may quickly find an optimal solution.
Disadvantages: (1) The most serious problem is the inability to explain one's own reasoning process and reasoning basis. (2) Necessary inquiries cannot be made to the user, and when the data is insufficient, the neural network cannot work.
(3) Turn all the characteristics of the problem into numbers, and turn all reasoning into numerical calculations, and the result will inevitably be loss of information. (4) The theory and learning algorithm still need to be further perfected and improved.
Extended information: Neural network development trend The unique nonlinear adaptive information processing ability of artificial neural network overcomes the shortcomings of traditional artificial intelligence methods for intuition, such as patterns, speech recognition, and unstructured information processing, making it useful in neural expert systems. , pattern recognition, intelligent control, combination optimization, forecasting and other fields have been successfully applied.
The combination of artificial neural network and other traditional methods will promote the continuous development of artificial intelligence and information processing technology.
In recent years, the artificial neural network is developing more deeply on the road of simulating human cognition, combining with fuzzy systems, genetic algorithms, evolutionary mechanisms, etc. to form computational intelligence, which has become an important direction of artificial intelligence and will be developed in practical applications .
The application of information geometry to the research of artificial neural networks has opened up a new way for the theoretical research of artificial neural networks. Research on neural computers has developed rapidly, and products have entered the market. The neural computer combined with photoelectricity provides good conditions for the development of artificial neural network.
Neural networks have been well applied in many fields, but there are still many aspects to be studied.
Among them, the combination of neural network with the advantages of distributed storage, parallel processing, self-learning, self-organization and nonlinear mapping and other technologies, as well as the resulting hybrid method and hybrid system, has become a major research hotspot.
Since other methods also have their own advantages, the combination of neural network and other methods can learn from each other's strengths, and then get better application results.
At present, the work in this area includes the integration of neural network and fuzzy logic, expert system, genetic algorithm, wavelet analysis, chaos, rough set theory, fractal theory, evidence theory and gray system. Reference: Baidu Encyclopedia - Artificial Neural Network.
Google AI Writing Project: Neural Network Pseudo-Original
Disadvantages of RBF neural network!
1. The generalization ability of RBF is better than that of BP network in many aspects, but when solving problems with the same precision requirements, the structure of BP network is simpler than that of RBF network .
2. The approximation accuracy of the RBF network is significantly higher than that of the BP network. It can almost achieve complete approximation, and it is extremely convenient to design. The network can automatically increase neurons until the accuracy requirements are met.
However, when the number of training samples increases, the number of neurons in the hidden layer of the RBF network is much higher than that of the former, which greatly increases the complexity of the RBF network, the structure is too large, and the amount of calculation also increases.
3. The RBF neural network is a feed-forward neural network with excellent performance. The RBF network can approximate any nonlinear function with arbitrary precision, and has the ability of global approximation, which fundamentally solves the local optimal problem of the BP network, and the topology The structure is compact, the structure parameters can realize separate learning, and the convergence speed is fast.
4. Their structures are completely different. BP approaches the minimum error by continuously adjusting the weights of neurons. The method is generally gradient descent.
RBF is a feed-forward neural network, that is to say, it does not approach the minimum error by continuously adjusting the weights. The excitation function is generally a Gaussian function, which is different from the S-type function of BP. The Gaussian function is passed. The weight is calculated for the distance between the input and the center point of the function.
5. The learning rate of the bp neural network is fixed, so the convergence speed of the network is slow and requires a long training time. For some complex problems, the training time required by the BP algorithm may be very long, which is mainly caused by the learning rate being too small.
The rbf neural network is an efficient feedforward network, which has the best approximation performance and global optimal characteristics that other forward networks do not have, and has a simple structure and fast training speed.
What is the core problem of BP neural network? What are its advantages and disadvantages?
Artificial neural network is an information processing system designed to imitate the structure and function of the human brain. It uses artificial neural network methods to realize pattern recognition. It can handle some problems where the environmental information is very complicated, the background knowledge is unclear, and the reasoning rules are not clear. , the neural network method allows samples to have large defects and distortions. There are many types of neural networks. When establishing neural network models, different neural network models can be considered according to the characteristics of the research object. Feedforward BP network, that is, error backpropagation Neural network is the most commonly used and popular neural network. The input and output relationship of BP network can be regarded as a mapping relationship, that is, each set of input corresponds to a set of output. BP algorithm is the most famous multi-layer forward network training Although the algorithm has shortcomings such as slow convergence speed and local extremum, various improvement measures can be used to improve its convergence speed and overcome the phenomenon of local extremum, and it has the characteristics of simplicity, ease of operation, small amount of calculation, and strong parallelism. , is still the preferred algorithm for multi-layer forward network. The advantages of multi-layer forward BP network: the network essentially realizes a mapping function from input to output, and mathematical theory has proved that it has the ability to realize any complex nonlinear mapping. Function.
This makes it particularly suitable for solving problems with complex internal mechanisms; the network can automatically extract "reasonable" solving rules by learning the example set with the correct answer, that is, it has self-learning ability; the network has certain promotion and generalization capabilities.
The problem of multi-layer forward BP network: From a mathematical point of view, the BP algorithm is an optimization method for local search, but the problem it needs to solve is to solve the global extremum of complex nonlinear functions, so the algorithm is likely to fall into local The extreme value makes the training fail; the approximation and generalization ability of the network is closely related to the typicality of the learning samples, and it is a very difficult problem to select typical sample instances from the problem to form the training set.
It is difficult to solve the contradiction between the instance scale and the network scale of the application problem. This involves the relationship between the possibility and feasibility of network capacity, that is, the problem of learning complexity; there is no unified and complete theoretical guidance for the selection of network structure, and generally it can only be selected by experience.
For this reason, some people call the structure selection of neural networks an art. The structure of the network directly affects the approximation ability and generalization properties of the network.
Therefore, how to choose an appropriate network structure in the application is an important issue; newly added samples will affect the network that has been successfully learned, and the number of features describing each input sample must also be the same; the predictive ability of the network (also called general The contradiction between the ability of transformation and extension) and the ability of training (also known as approaching ability and learning ability).
In general, when the training ability is poor, the predictive ability is also poor, and to a certain extent, with the improvement of the training ability, the predictive ability also improves. But this trend has a limit. When this limit is reached, as the training ability increases, the prediction ability decreases instead, that is, the so-called "over-fitting" phenomenon occurs.
At this time, the network has learned too many sample details, but cannot reflect the law contained in the sample. Since the BP algorithm is essentially a gradient descent method, and the objective function it needs to optimize is very complex, the "sawtooth phenomenon" will inevitably appear. ", which makes the BP algorithm inefficient; there is paralysis, because the optimized objective function is very complex, it will inevitably appear some flat areas when the neuron output is close to 0 or 1, in these areas, the weight error changes It is very small, which makes the training process almost stop; in order to make the network execute the BP algorithm, the traditional one-dimensional search method cannot be used to find the step size of each iteration, but the update rule of the step size must be given to the network in advance, this method will cause the algorithm inefficient.
What are the application ranges and advantages and disadvantages of the least squares method, regression analysis method, gray prediction method, decision theory, and neural network?
Least Squares: Finds the best function fit to the data by minimizing the sum of squared errors. The unknown data can be easily obtained by using the least square method, and the sum of squares of the errors between the obtained data and the actual data can be minimized. Least squares can also be used for curve fitting.
Some other optimization problems can also be formulated by least squares by minimizing energy or maximizing entropy. Advantages: Simple to implement and simple to calculate. Disadvantages: Cannot fit nonlinear data. Regression analysis method: refers to a statistical analysis method that determines the quantitative relationship between two or more variables that depend on each other.
In big data analysis, regression analysis is a predictive modeling technique that studies the relationship between dependent variables (targets) and independent variables (predictors). This technique is commonly used in predictive analytics, time series modeling, and discovering causal relationships between variables.
Advantages: When analyzing a multi-factor model, it is simpler and more convenient. Not only can it predict and find the function, but it can also test the residuals of the results by itself to test the accuracy of the model.
Disadvantages: The regression equation is only a guess, which affects the diversity of factors and the unmeasurability of some factors, making regression analysis limited in some cases. Gray prediction method: The color prediction method is a method for predicting systems with uncertain factors.
It identifies the degree of difference between the development trends of system factors, that is, conducts correlation analysis, and generates and processes the original data to find the law of system changes, generate a data sequence with strong regularity, and then establish a corresponding differential equation model , so as to predict the future development trend of things.
It constructs a gray prediction model with a series of quantitative values that reflect the characteristics of the predicted object observed at equal time distances, and predicts the characteristic quantity at a certain moment in the future, or the time to reach a certain characteristic quantity. Advantages: For complex systems with uncertain factors, the prediction effect is better, and the required sample data is smaller.
Disadvantages: The forecast based on the exponential rate does not take into account the randomness of the system, and the medium and long-term forecast accuracy is poor.
Decision tree: On the basis of knowing the probability of occurrence of various situations, by forming a decision tree to obtain the probability that the expected value of the net present value is greater than or equal to zero, the decision analysis method for evaluating project risk and judging its feasibility is an intuitive use of probability analysis A graphical method of .
Because this decision-making branch is drawn in a graph that resembles the branches of a tree, it is called a decision tree. In machine learning, a decision tree is a predictive model that represents a mapping relationship between object attributes and object values.
Advantages: It can handle irrelevant features; it can make feasible and effective analysis of large data sources in a relatively short period of time; it is simple to calculate, easy to understand, and strong in interpretability; it is more suitable for processing samples with missing attributes.
Disadvantages: The correlation between data is ignored; overfitting is prone to occur (random forest can greatly reduce overfitting); in decision trees, for data with inconsistent numbers of samples in each category, the result of information gain is biased towards Those features with more numerical values.
Neural network: advantages: high classification accuracy; strong parallel distributed processing ability, strong distributed storage and learning ability, strong robustness and fault tolerance to noise nerves, and can fully approach complex nonlinear relationships; with associative memory function.
Disadvantages: The neural network requires a large number of parameters, such as the initial value of the network topology, weights and thresholds; the learning process between them cannot be observed, and the output results are difficult to interpret, which will affect the credibility and acceptability of the results; learning time If it is too long, it may even fail to achieve the purpose of learning.
Do neural networks generalize poorly?
Generalization ability, the English full name generalization ability, refers to the adaptability of machine learning algorithms to fresh samples, an ability to predict new input categories.
By learning to find the law hidden behind the data, and for data other than the learning set with the same law, this trained network can give an appropriate output. This ability is called generalization ability.
For the neural network, generally the more complex it is, the higher the complexity the neural network bears, the greater the complexity capacity of describing the law, of course the better, of course it is not absolute, but this can explain the problem of a container capacity, At this time, the generalization ability of the neural network is also stronger.
We need to know that structural complexity and sample complexity, sample quality, initial weights, learning time and other factors will affect the generalization ability of neural networks.
In order to ensure that the neural network has a strong generalization ability, people have done a lot of research and obtained many generalization methods, commonly used include pruning algorithm, construction algorithm and evolutionary algorithm. The generalization ability of artificial neural networks is mainly due to the fact that efficient feature sets can be derived from the training set through unsupervised pre-learning.
Once complex problems are converted into the form expressed by these features, they will naturally become simpler. Conceptually this is a bit like doing an intelligent coordinate transformation suitable for the training set.
For example, if the training set is a picture of many faces, then if the pre-training is done well, features such as nose, eyes, mouth, and various basic face shapes can be derived. If the classification is done using these features instead of based on pixels, the result will naturally be much better.
Although a large neural network has a large number of parameters, because the classification is actually based on a small number of features, it is less prone to overfitting.
At the same time, aiming at the shortcomings of the neural network that is easy to fall into local extremum, difficult to determine the structure and poor generalization ability, a support vector regression machine that can well solve small sample, nonlinear and high-dimensional problems is introduced to carry out the evaluation of oil and gas field development indicators. predict.
Disadvantages of SOFM Neural Networks
The neural network has strong parallelism and adaptability, and can be applied to many fields such as control, information, and prediction. The ant colony algorithm was first successfully applied to the traveling salesman problem, and then widely used in various combinatorial optimization problems. But the theoretical basis of the algorithm is relatively weak, and the convergence of the algorithm has not been proved.
Many parameters are only set by experience, the actual effect is average, and it is often immature to use. Genetic algorithm is a mature algorithm with strong global optimization ability and can quickly approach the optimal solution. It is mainly used to solve the NP problem of combinatorial optimization. These three algorithms can be integrated with each other.
For example, the genetic algorithm can optimize the initial weight of the neural network, prevent the training of the neural network from falling into a local minimum, and speed up the convergence. Ant colony algorithm can also be used to train neural network, but it must use optimized ant colony algorithm, such as max-min ant colony algorithm and elite retention strategy.
What are the advantages and disadvantages of TensorFlow
The predecessor of the TensorFlow framework is Google's DistBelief V2, which is the deep network tool library of the Google Brain project. Some people think that TensorFlow is refactored from Theano.
Once Tensorflow was open sourced, it immediately attracted a large number of developers to follow up. Tensorflow supports a wide range of functions including images, handwriting, speech recognition, prediction, and natural language processing. TensorFlow follows the Apache 2.0 open source agreement.
TensorFlow released its version 1.0 on February 15, 2017. This version is an integration of the previous eight incomplete versions.
The following are some of the reasons for TensorFlow's success: TensorFlow provides these tools: TensorBroad is a well-designed visual network construction and display tool; TensorFlow Serving can easily configure new algorithms and environments by maintaining the same server architecture and API.
TensorFlow Serving also provides out-of-the-box models and can be easily extended to support other models and data.
TensorFlow programming interfaces include Python and C++, Java, Go, R and Haskell language interfaces are also supported in the alpha version. In addition, TensorFlow also supports Google and Amazon cloud environments.
TensorFlow version 0.12 supports Windows 7, 8, and Server 2016 systems. Due to the use of the C++ Eigen library, the TensorFlow class library can be compiled and optimized on the ARM architecture platform.
This means that you can deploy trained models on various servers and mobile devices without additional implementation of model decoders or Python interpreters.
TensorFlow provides granular network layers that allow users to build new and complex layer structures without implementing them from the ground up. Subgraphs allow users to view and restore data from any edge of the graph. This is very useful for debugging complex calculations.
Distributed TensorFlow was launched in version 0.8, which provides parallel computing support, allowing different parts of the model to be trained in parallel on different devices.
TensorFlow is taught at Stanford University, Berkeley College, University of Toronto, and Udacity (an online school launched in March 2016).
The disadvantages of TensorFlow are: each calculation flow must be constructed into a graph, there is no symbol cycle, which makes some calculations difficult; there is no three-dimensional convolution, so video recognition cannot be done; even though it is 58 times faster than the original version (0.5) , but still performs worse than its competitors.
BP neural network has disadvantages such as slow learning convergence speed, easy to fall into local minimum point and unable to obtain global optimal solution. What does slow convergence mean?