concept
What is machine learning?
Machine learning is the literal translation of the English name Machine Learning (ML for short). Machine learning involves many disciplines such as probability theory, statistics, approximation theory, convex analysis, and algorithmic complexity theory. It specializes in how computers simulate or realize human learning behaviors to acquire new knowledge or skills, and to reorganize existing knowledge structures to continuously improve their performance. It is the core of artificial intelligence and the fundamental way to make computers intelligent. It is applied in all fields of artificial intelligence. It mainly uses induction and synthesis instead of deduction.
Compared with the traditional computer work, we give it a series of instructions, and then it can follow the instructions step by step. Machine learning does not accept the instructions you enter at all, on the contrary, it only accepts the data you enter! That is to say, it has the ability of us humans to process things in a sense.
The history of machine learning
Machine learning is a relatively young branch of artificial intelligence research, and its development process can be roughly divided into four periods.
The first stage was from the mid-1950s to the mid-1960s, which belonged to the warm period.
The second phase was from the mid-1960s to the mid-1970s, known as the cool-off period of machine learning.
The third phase, from the mid-1970s to the mid-1980s, is called the Renaissance.
The latest phase of machine learning started in 1986. The important manifestations of machine learning entering a new stage are as follows:
(1) Machine learning has become a new marginal subject and has formed a course in colleges and universities. It integrates the application of psychology, biology and neurophysiology as well as mathematics, automation and computer science to form the theoretical foundation of machine learning.
(2) Combined with various learning methods, the research of various forms of integrated learning systems is emerging. In particular, the coupling of connection learning and symbolic learning can better solve the problem of acquisition and refinement of knowledge and skills in continuous signal processing.
(3) A unified view of various fundamental issues of machine learning and artificial intelligence is taking shape. For example, the view that learning is combined with problem solving and knowledge expression facilitates learning has resulted in the block learning of the general intelligence system SOAR. The case-based method combining analogical learning and problem solving has become an important direction of empirical learning.
(4) The scope of application of various learning methods has been expanding, and some of them have become commodities. Knowledge acquisition tools for inductive learning have been widely used in diagnostic classification expert systems. Connection learning is dominant in audiovisual recognition. Analytical learning has been used to design comprehensive expert systems. Genetic algorithm and reinforcement learning have good application prospects in engineering control. The neural network connection learning coupled with the symbolic system will play a role in the intelligent management of enterprises and the motion planning of intelligent robots.
(5) Academic activities related to machine learning are unprecedentedly active. In addition to the annual machine learning seminar, there are also computer learning theory conferences and genetic algorithm conferences.
The scope of machine learning
Machine learning is deeply related to pattern recognition, statistical learning, data mining, computer vision, speech recognition, natural language processing and other fields. In terms of scope, machine learning is similar to pattern recognition, statistical learning, and data mining. At the same time, the combination of machine learning and processing technologies in other fields has formed interdisciplinary subjects such as computer vision, speech recognition, and natural language processing. Therefore, when speaking of data mining in general, it can be equivalent to speaking of machine learning. At the same time, what we usually call machine learning applications should be general, not only limited to structured data, but also images, audio and other applications.
pattern recognition
Pattern Recognition = Machine Learning. The main difference between the two is that the former is a concept developed from industry, while the latter is mainly derived from computer science. In the famous book "Pattern Recognition And Machine Learning", Christopher M. Bishop begins by saying that "pattern recognition comes from industry, and machine learning comes from computer science. However, the activities in them can be viewed as Both sides of the same field, and both have grown significantly over the past 10 years."
data mining
Data mining = machine learning + database. I remember that there was a data mining course in the last semester of college. What is data mining? It is to mine useful data from massive data. In a sense, it is related to big data. Analysis is similar. Data mining is generally related to computer science and achieves these goals through a number of methods such as statistics, online analytical processing, intelligence retrieval, machine learning, expert systems (relying on past rules of thumb), and pattern recognition.
Statistical Learning
Statistical learning is approximately equal to machine learning. Statistical learning is a highly overlapping discipline with machine learning. Because most of the methods in machine learning come from statistics, it can even be argued that the development of statistics promotes the prosperity of machine learning. For example, the famous support vector machine algorithm is derived from statistics. But to a certain extent, the two are different. The difference is that statistical learners focus on the development and optimization of statistical models, which is partial to mathematics, while machine learners focus more on being able to solve problems and partial to practice. Therefore, Machine learning researchers focus on improving the efficiency and accuracy of learning algorithms executed on computers.
computer vision
Computer Vision = Image Processing + Machine Learning. Image processing techniques are used to process images into suitable inputs into machine learning models, which are responsible for identifying relevant patterns from images. With the development of deep learning, a new field of machine learning, the effect of computer image recognition has been greatly promoted, so the future development prospects of the computer vision field are immeasurable.
Speech Recognition
Speech Recognition = Speech Processing + Machine Learning. Speech recognition is the combination of audio processing technology and machine learning. Speech recognition technology is generally not used alone, and is generally combined with related technologies of natural language processing. The current related applications include Apple's voice assistant siri, iFLYTEK and many other domestic technology companies and institutions.
natural language processing
Natural Language Processing = Text Processing + Machine Learning. Natural language processing technology is mainly a field that allows machines to understand human language. In the natural language processing technology, a large number of technologies related to compilation principles are used, such as lexical analysis, syntax analysis, etc. In addition, at the level of understanding, technologies such as semantic understanding and machine learning are used.
Machine Learning Algorithms
supervised learning
A supervised learning algorithm consists of a target variable (dependent variable) and predictor variables (independent variables) that predict the target variable. From these variables we can build a model so that for a known predictor variable value, we can get the corresponding target variable value. Train the model repeatedly until it achieves a predetermined accuracy on the training dataset. Algorithms belonging to supervised learning include: regression model, decision tree, random forest, K-neighbor algorithm, logistic regression, etc.
unsupervised learning
Unlike supervised learning, in unsupervised learning we have no target variable to predict or estimate. Unsupervised learning is used to classify the population of objects. It is widely used in classifying customers according to a certain indicator. Algorithms belonging to unsupervised learning include: association rules, K-means clustering algorithm, etc.
reinforcement learning
This algorithm trains the program to make a certain decision. The program tries all possible actions in a given situation, records the results of the different actions and tries to find the best one to make a decision. Algorithms that belong to this class are Markov decision processes.
Common Algorithms
Common Machine Learning Algorithms
The following are the most commonly used machine learning algorithms, most data problems can be solved by them:
1. Linear Regression
2. Logistic Regression
3. Decision Tree
4. Support Vector Machine (SVM)
5. Naive Bayes (Naive Bayes) 6. K-
Neighbor Algorithm (KNN)
7. K-means (K-means)
8. Random Forest (Random Forest)
9. DimensionalityReduction Algorithms)
10. GradientBoost and Adaboost algorithms
machine learning classification
Classification based on learning strategies
Learning strategy refers to the reasoning strategy adopted by the system during the learning process. A learning system always consists of two parts, learning and environment. Information is provided by the environment (such as books or teachers), and the learning part realizes the transformation of information, memorizes it in an understandable form, and obtains useful information from it. In the learning process, the less reasoning the student (learning part) uses, the more dependent he is on the teacher (the environment) and the heavier the teacher's burden. The classification standards of learning strategies are classified according to the amount and difficulty of reasoning required by students to realize information conversion. They are divided into the following six basic types in order from simple to complex and from less to more:
1) Rote learning
The learner directly absorbs the information provided by the environment without any reasoning or other knowledge transformation. Such as Samuel's checkers program, Newell and Simon's LT system. This type of learning system mainly considers how to index the stored knowledge and make use of it. The systematic learning method is to learn directly through pre-programmed and constructed programs. The learner does not do any work, or learns by directly receiving the established facts and data, and does not make any reasoning about the input information.
2）示教学习 (Learning from instruction或Learning by being told)
Students obtain information from the environment (teachers or other sources of information such as textbooks, etc.), convert knowledge into an internally usable representation, and organically combine new knowledge with existing knowledge. So students are required to have a certain level of reasoning ability, but the environment still has a lot of work to do. The teacher proposes and organizes knowledge in some form so that the knowledge possessed by the students can be continuously increased. This learning method is similar to the school teaching method in human society. The task of learning is to establish a system that enables it to receive instruction and advice, and to effectively store and apply the knowledge learned. Many expert systems use this method to achieve knowledge acquisition when building a knowledge base. A typical application example of teach-by-learn is the FOO program.
3) Learning by deduction
The form of reasoning used by students is deductive reasoning. Reasoning starts from axioms and deduces conclusions through logical transformations. This kind of reasoning is a process of "fidelity" transformation and specialization, so that students can acquire useful knowledge in the reasoning process. This learning method includes macro-operation learning, knowledge editing and chunking techniques. The inverse process of deductive reasoning is inductive reasoning.
4) Learning by analogy
Using the similarity of knowledge in two different domains (source domain and target domain), the corresponding knowledge of the target domain can be deduced from the knowledge of the source domain (including similar features and other properties) by analogy, so as to realize learning. The analog learning system can transform an existing computer application system into a new field to accomplish similar functions that were not originally designed.
Learning by analogy requires more reasoning than the three learning styles above. It generally requires retrieving available knowledge from a knowledge source (source domain) and transforming it into a new form for use in a new situation (target domain). Learning by analogy plays an important role in the history of human science and technology development, and many scientific discoveries are obtained through analogy. For example, the famous Rutherford analogy revealed the mystery of atomic structure by comparing the atomic structure (target domain) with the solar system (source domain).
5) Explanation-based learning (EBL)
Based on the target concept provided by the teacher, an example of the concept, domain theory, and operational criteria, students first construct an explanation to explain why the example satisfies the target concept, and then generalize the explanation to a sufficient level of the target concept that satisfies the operational criteria. condition. EBL has been widely used for knowledge base refinement and improving system performance.
Famous EBL systems include G.DeJong's GENESIS, T.Mitchell's LEXII and LEAP, and S.Minton's PRODIGY.
6) Learning from induction
Inductive learning is provided by teachers or the environment with some examples or counter-examples of a concept, allowing students to derive a general description of the concept through inductive reasoning. This kind of learning involves far more reasoning than taught and deductive learning because the environment does not provide general conceptual descriptions (such as axioms). To a certain extent, inductive learning also has a larger amount of reasoning than analogical learning, because no similar concept can be accessed as a "source concept". Inductive learning is the most basic and relatively mature learning method, which has been widely studied and applied in the field of artificial intelligence.
Representation classification based on acquired knowledge
The knowledge acquired by the learning system may include: behavior rules, descriptions of physical objects, problem solving strategies, various classifications, and other types of knowledge used for task realization.
For the knowledge acquired in learning, there are mainly the following representations:
1) Algebraic expression parameters
The goal of learning is to adjust the parameters or coefficients of an algebraic expression in the form of a fixed function to achieve a desired performance.
2) Decision tree
A decision tree is used to divide the categories of objects. Each internal node in the tree corresponds to an object attribute, and each edge corresponds to the optional values of these attributes. The leaf nodes of the tree correspond to each basic classification of the object.
3) Formal grammar
In the study of identifying a particular language, the formal grammar of the language is formed by induction of a series of expressions in the language.
4) Production rules
Production rules, expressed as condition-action pairs, have been widely used. The learning behaviors in the learning system are mainly: generating, generalizing, specializing or synthesizing production rules.
5) Formal logic expressions
The basic components of formal logical expressions are propositions, predicates, variables, statements that constrain the scope of variables, and embedded logical expressions.
6) Graphs and Networks
Some systems employ graph matching and graph transformation schemes to efficiently compare and index knowledge.
7) Framework and schema (schema)
Each frame contains a set of slots that describe various aspects of things (concepts and individuals).
8) Computer programs and other procedural codes
The purpose of acquiring this form of knowledge is to acquire an ability to implement a particular process, not to infer the internal structure of that process.
9) Neural Networks
This is mainly used in join learning. The knowledge acquired by learning is finally summarized into a neural network.
10) Combination of multiple representations
According to the fineness of representation, knowledge representation can be divided into two categories: coarse-grained symbolic representation with high generalization degree, and fine-grained sub-symbolic representation with low generalization degree. Decision trees, formal grammars, production rules, formal logic expressions, frames and patterns belong to the class of symbolic representation; while parameters of algebraic expressions, graphs and networks, and neural networks belong to the class of subsymbolic representation.
Classification by application area
The main application areas are: expert systems, cognitive simulation, planning and problem solving, data mining, network information services, image recognition, fault diagnosis, natural language understanding, robotics and games and other fields.
From the perspective of the types of tasks reflected in the execution part of machine learning, most of the applied research areas are basically concentrated in the following two categories: classification and problem solving.
(1) The classification task requires the system to analyze the input unknown pattern (the description of the pattern) according to the known classification knowledge to determine the category of the input pattern. The corresponding learning goal is to learn the criteria for classification (such as classification rules).
(2) The problem solving task requires that for a given target state, find an action sequence that converts the current state into the target state; most of the research work of machine learning in this field focuses on learning to obtain information that can improve problem solving. Knowledge of efficiency (eg, search control knowledge, heuristic knowledge, etc.).
Comprehensive classification
1) Empirical inductive learning
Experiential inductive learning adopts some data-intensive empirical methods (such as version space method, ID3 method, law discovery method) to perform inductive learning on examples. The examples and learning results are generally represented by symbols such as attributes, predicates, and relationships. It is equivalent to inductive learning in learning policy-based classification, but deducts the parts of join learning, genetic algorithm, and reinforcement learning.
2) Analytical learning
Analytical learning methods start from one or a few instances and apply domain knowledge for analysis. Its main characteristics are:
·The reasoning strategy is mainly deduction, not induction;
·Use past problem solving experience (examples) to guide new problem solving, or generate search control rules that can more effectively use domain knowledge.
The goal of analytical learning is to improve the performance of the system, not to describe new concepts. Analytical learning includes applied interpretive learning, deductive learning, multi-level structural chunking, and macro-operation learning techniques.
3) Learning by analogy
It is equivalent to learning by analogy in learning policy-based classification. The more compelling research in this type of learning is learning by analogy with specific examples of past experiences, known as case-based learning, or simply example learning.
Machine Learning Code Examples
Machine learning speech design to the latest languages like python and R.
//在gcc-4.7.2下编译通过。
//命令行：g++-Wall-ansi-O2test.cpp-otest
#include<iostream>
usingnamespacestd;
voidinput(int&oper,constboolmeth)
{
//meth为true则只判断1，为false则判断1或0
while(true)
{
cin>>oper;
if(meth&&oper==1)
break;
elseif(oper==0||oper==1)
break;
cout<<"输入错误，请重新输入。"<<endl;//判断参数
cin.sync();//避免极端输入导致死循环
cin.clear();
}
}
intmain(void)
{
cout<<"1+1=2吗？那要看您怎么教我了，不要惊讶我会学习的"<<endl;
intladd,radd,aprs,rcnt(0),wcnt(0);//定义输入与结果，正确次数与错误次数
cout<<"开始学习……"<<endl;
for(inti(0);i!=10;++i)
{
cout<<"参数1（必须是1）："<<flush;//提示输入参数
input(ladd,true);
cout<<"参数2（必须是1）："<<flush;
input(radd,true);
cout<<"结果："<<(ladd+radd)<<endl;//输出结果
cout<<"您对这满意吗（满意输入1，不满意输入0）："<<flush;//评价等级
input(aprs,false);
if(aprs)//判断用户评价
++rcnt;
else
++wcnt;
cout<<"正确次数："<<rcnt<<"错误次数："<<wcnt<<endl;//错误次数
}
if(rcnt>wcnt)//判断学习结果
cout<<"主人告诉我1+1=2。"<<endl;
else
if(rcnt<wcnt)
cout<<"主人告诉我1+1!=2。"<<endl;
else
cout<<"我不明白主人是什么意思。"<<endl;
intterm;//退出部分
cout<<"您对我的表现满意吗？满意请输入1不满意请输入0："<<flush;
input(term,false);
if(term)
cout<<"谢谢我会继续努力学习"<<endl;
else
cout<<"谢谢我会继续努力学习D"<<endl;
//cin>>term;//在Windows上测试时启用
return0;
}