【Artificial Intelligence】Machine Learning

Definition of Machine Learning

Definition of machine learning: The study of how to use machines to simulate human learning activities.

A slightly stricter formulation is: machine learning is a study of machines acquiring new knowledge and skills, and identifying existing knowledge.

Machine learning is the automatic acquisition of knowledge by computers, and it is one of the three branches of knowledge engineering (using knowledge, knowledge representation, and acquiring knowledge).

Main Strategies for Machine Learning

Mechanical learning: memory, without any reasoning;

Teaching and learning: when accepting external knowledge, a little reasoning, translation, and transformation are required;

Analogy learning: only relevant knowledge for completing similar tasks can be obtained, and requires more reasoning than the previous two;

Learning by example: There is no regular information about the task at all in advance, and the most reasoning is required;

The more reasoning used in learning, the more capable the system is.

machine learning

Directly memorize or store new knowledge provided by the environment, and then directly use this knowledge by retrieving the knowledge base, without any calculation and derivation.

Storage is necessary and fundamental to any intelligent program .

Simple machine learning model:

Machine learning needs to pay attention to three important issues:

1. Store organizational information: Appropriate storage methods make retrieval as fast as possible;

2. The stability of the environment and the applicability of the stored information: the stored information must adapt to changes in the external environment;

3. The trade-off between storage and computing

Basic Structure of a Machine Learning System

The process of learning is building theories, forming hypotheses, and making inductive reasoning.

Environment and knowledge base: A collection of information expressed in a certain knowledge representation, representing external information sources and knowledge possessed by the system, respectively.

"Environment": to provide certain information to the "learning segment" of the system;

"Learning link": Use this information to improve the system's "knowledge base" to enhance the effectiveness of the system's "execution link" in completing tasks;

"Execution link": complete a certain task according to the knowledge in the knowledge base, and feed back the obtained information to the "learning link".

Important factors affecting the design of learning systems: information provided by the environment to the system, knowledge base

The expression of knowledge should take into account: strong expressive ability, easy reasoning, easy modification of knowledge base, and easy expansion of knowledge representation.

Classification of Machine Learning Methods

Classification based on learning method:

Inductive learning: symbolic induction (learning by example, decision tree...), functional induction (neural network, learning by example, statistical learning)

Deductive learning: deduction from basic principles.

Learning by analogy: Learning by comparing similar things.

Analytical Learning: Using prior knowledge to deductively derive general hypotheses.

inductive learning

The act of reasoning from the particular to the general, from the part to the whole.

It can be divided into example learning, observation and discovery learning. The former belongs to learning with a teacher, and the latter belongs to learning without a teacher.

The general pattern of inductive learning is:

Given the following:

(1) Observation statements (facts) F, used to express specific knowledge about some objects, states, processes, etc.;

(2) the assumed initial inductive assertion (possibly empty);

(3) Background knowledge, which is used to define knowledge, assumptions, and constraints about observation statements, candidate inductive assertions, and any related problem domains, including priority criteria that can characterize the nature of the sought inductive assertions.

example learning

Also known as case-by-case learning, it is a learning method in which a general concept is obtained by induction through a number of examples related to a certain concept in the environment.

What the external environment provides is a set of examples (positive examples and negative examples), general knowledge applicable to a wider range should be induced from these special knowledge, so as to cover all positive examples and exclude all negative examples.

Observation and Discovery Learning

Also known as a descriptive summary. The goal is to determine a general description of a law or theory, characterize a set of observations, and specify properties of certain classes of objects.

It can be divided into observation learning and machine discovery. The former is used to cluster cases to form a conceptual description; the latter is used to discover laws and generate laws or rules.

analogy learning

Analogy learning is to use the similarity of knowledge in two different domains ( source domain S, target domain T ), through analogy, the corresponding knowledge of the target domain can be deduced from the knowledge of the source domain (including similar features and other properties), so that Achieve learning.

Analogy reasoning process:

memories and associations

Want to find situations in S that are similar to the current situation, which have been dealt with in the past, have ready-made solutions and related knowledge.
choose

Select the situation most similar to the current situation and its related knowledge from the similar situations found.
Create corresponding mapping

Establish the corresponding relationship of similar elements between the similar situations of S and T, and establish the corresponding mapping.
convert

Under the mapping established in the previous step, the relevant knowledge in S is introduced to T, so as to establish a method to solve the current problem or learn new knowledge about T.

Analogy learning process:

Enter a set of known conditions (solved problem) and a set of conditions not fully determined (new problem)
For the two sets of input conditions, according to their description, according to the definition of a certain similarity, find the analogous correspondence between the two.
According to the method of similar transformation , the concepts, characteristics, methods, relations, etc. of the existing problems are mapped to the new problems, so as to obtain the new knowledge needed to solve the new problems.
Check the knowledge of new problems obtained by analogy. The verified correct knowledge is stored in the knowledge base, and the knowledge that cannot be verified for the time being can only be placed in the database as reference knowledge .

Types of Analogical Learning Research:

Problem-solving type: When solving a new problem, first recall whether a similar problem has been solved before, and if so, use this basis to solve a new problem.

Predictive reasoning type:

Traditional analogy: It is used to judge that an incompletely certain thing may have other attributes.
Causal relationship type: from the causal relationship s1 (A→B), now there is A' similar to A, then there may be B', satisfying A'→B'

explain learning

According to the domain knowledge of the task and the conceptual knowledge being learned, the current instance is analyzed and solved, and a causal interpretation tree representing the solution process is obtained to obtain new knowledge.

For example, based on the target concept provided by the teacher, an example of the concept, domain theory, and operational criteria, students first construct an explanation to explain why the example satisfies the target concept, and then extend the explanation to a target concept that satisfies the operational criteria Sufficient condition .

A general description of interpretive learning

EBG：

Algorithm flow:

1. Structural explanation

Use the domain knowledge DT to perform deduction to prove why the training instance TE provided to the system is an instance that satisfies the target concept DC.

For example: Suppose the target concept to be learned is "an object (Obj1) can be safely placed on another object (Obj2)", that is, Safe-To-Stack(Obj1, obj2)

2. Acquiring general knowledge

Task: Generalize the explanation structure obtained in the previous step to obtain general knowledge about the target concept.

Method: replace constants with variables , remove some unimportant information, and only keep the key information necessary to solve the problem. The following general knowledge can be obtained:
$V o l u m e (O 1, v1)_\land De n s i t y (O 1, d 1) \land * (v 1, d 1, w 1) \land I s a (O 2, t ab l e) \land S ma ll er (w 1, 15) \to S a f e - T o - St a c k (O 1, Or 2)$

The completeness of domain knowledge plays an important role in the formation of proofs, and imperfection is unavoidable. At this time, the following two extreme situations may occur:

Constructing no explanation, constructing multiple explanations

Solution:

Provide sound domain knowledge;

The learning system should also have the ability to test and correct imperfect knowledge, so that problems can be discovered and corrected as soon as possible.

Examples of some of the main content of machine learning

decision tree

Alias: decision tree, multi-level classifier.

The data is first processed, using inductive algorithms to generate readable rules and decision trees , and the new data is then analyzed using the decisions. In essence, a decision tree is the process of classifying data through a series of rules.

It is very intuitive and natural to use a series of query answers to judge the category of a pattern. The formulation of the latter question depends on the answer to the former question. This style of question answering is particularly effective on non-metric data, where no distance metric is involved in answering the question.

Bayesian classifier

Bayesian classifier is a general term for a class of classification algorithms. Bayesian theorem is the core of this type of algorithm, so it is collectively called Bayesian classification.

Bayesian decision theory uses the misjudgment loss (minimum error rate criterion, minimum risk criterion, Neyman-Pearson criterion, minimum-maximum decision criterion) to select the optimal category classification when the relevant probability is known .

SVM

Introduction to Machine Learning Methods

Ensemble learning

Combine multiple weakly supervised models in order to obtain a better and more comprehensive strong supervised model . The underlying idea of ensemble learning is that even if a certain weak classifier gets a wrong prediction, other weak classifiers can correct the error back .

Sequential ensemble methods , where the base learners involved in training are generated sequentially (e.g. AdaBoost). The principle of sequential methods is to exploit the dependencies among the base learners. By assigning higher weights to samples that were mislabeled in previous training, the overall prediction performance can be improved.
Parallel ensemble method , the basic learners participating in the training are generated in parallel (such as Random Forest). The principle is to use the independence between the basic learners, and the error can be significantly reduced by averaging.