[Artificial Intelligence] - Learning and Machine Learning, Unsupervised Learning, Reinforcement Learning, Learning Representation

previous chapter

Bayesian Networks

Bayesian networks provide a natural representation for describing (causally induced) conditional independence.

• Topology + conditional probability table = compact representation of the joint distribution.
• Usually easy for domain experts to construct.
• Exact inference via variable elimination:
  • The time complexity is polynomial on directed acyclic graphs, but NP-hard on general graphs.
  • Space complexity is the same as time complexity, very sensitive to topology.
• The Naive Bayes model is a special kind of Bayesian network.

This Chapter: Observational Learning

Observational learning is a method of machine learning in which the goal is to infer an unknown model or probability distribution from data. Observational learning usually includes steps such as data collection, selection of a hypothesis space, selection of an optimal model using a learning algorithm, evaluation, and adjustment. It can be applied to a variety of tasks, including classification, regression, clustering, dimensionality reduction, and anomaly detection, among others. The advantage of observational learning is that the model can be learned automatically without manually specifying the form or parameters of the model. Common observational learning algorithms include decision trees, neural networks, support vector machines, naive Bayesian, maximum entropy models, and hidden Markov models.

study

Learning plays an important role in artificial intelligence and machine learning, allowing agents to extract knowledge and information from experience and apply it to new tasks and environments. Learning can be divided into different types such as supervised learning, unsupervised learning, and reinforcement learning, each with its own unique applications and algorithms. The importance of learning is that it allows agents to adapt to changing environments and tasks without reprogramming or system redesign. Although learning has many advantages, there are also some challenges and limitations, such as the need for a large amount of data and computing resources to train the model, the need to consider issues such as data bias and overfitting, and the need to balance exploration and utilization.

learning elements

Designing learning elements is influenced by:

• What components of performance elements need to be learned.
• What is the feedback that can be used to learn these components.
• What is the notation used for these components.

The learning element is the core component of a machine learning system, including components such as algorithms, models, and data. The purpose of the learning element is to enable the agent to learn knowledge and information from experience to optimize its performance and decision-making. Learning elements need to be designed with the task and context in mind, and adapted and optimized on a case-by-case basis. Feedback information, selection of algorithms and models, and representation methods are also important factors in the design of learning elements.

Introduction to Machine Learning

What is machine learning useful for

automatic speech recognition

Most speech recognizers or translators these days learn - the more you use them, the smarter they get

computer vision

such as object, face and handwriting recognition

Information retrieval—information retrieval

Reading, digesting, and categorizing huge text databases is too much for humans to retrieve the web
(retrieval) classify (category) cluster (cluster) relationships between pages

machine learning

Machine learning is an interdisciplinary field covering mathematics, computer science, engineering, statistics, cognitive science, psychology, computational neuroscience, economics, and more.

Its purpose is to use data to train models or algorithms so that certain tasks can be done automatically, and its performance can be continuously improved from experience.

The models and algorithms involved in machine learning include linear regression, logistic regression, decision trees, support vector machines, neural networks, Bayesian networks, hidden Markov models, clustering, dimensionality reduction, etc. By learning machine learning, you can master some important concepts, techniques and tools, laying the foundation for future learning and practice.

Machine Learning: Definition

Tom Mitchell (1998) proposed the definition of machine learning: If the performance of a computer program on a certain task T, through some performance measure P, increases with the increase of experience E, then it can be said that it learns from experience E The task T is learned in .

This definition emphasizes the essence of machine learning: improving the performance of computer programs on certain tasks by gaining knowledge and experience from data. Among them, the task T can be various, including classification, regression, clustering, identification, etc.; the performance measure P can be accuracy rate, precision rate, recall rate, F1 value, AUC value, etc.; experience E can be training data , validation data, test data, etc.

Email Filtering Issues

Suppose your email program observes the messages you mark as spam or not spam, and based on that learns how to filter spam better. In this case, what is task T?

A. Classify email as spam or not spam.
B. Observe the messages you mark as spam or not spam.
C. Number (or proportion) of emails correctly classified as spam/not spam.
D. None of the above - this is not a machine learning problem.

Answer: A. Classify email as spam or not spam.

learning type

Suppose there is an agent or machine that receives a sequence of perceptual inputs: $x_1,x_2,x_3,x_4,...$

supervised learning

In supervised learning, the machine also receives the desired outputs $y_1,y_2,...$ , whose goal is to learn to generate the correct output given a new input. This is called supervised learning. In this type of learning, the machine is presented with labeled examples and the goal is to learn a function that maps inputs to outputs. Common examples include image classification and speech recognition.

unsupervised learning

In unsupervised learning, the desired outputs $y_1,y_2,...$ , but the agent still hopes to build a$x$ -model. The goal is to discover patterns, structures, and relationships in the input data. Examples include clustering, dimensionality reduction, and anomaly detection.

semi-supervised learning

Semi-supervised learning is a combination of supervised and unsupervised learning. In this type of learning, the machine is presented with some labeled examples and some unlabeled examples. The goal is to learn a function that generalizes to new, unseen examples. This type of learning is useful when labeled data is scarce or expensive.

reinforcement learning

Reinforcement learning is a type of learning in which an agent learns as it interacts with the environment and receives feedback in the form of rewards or punishments. The goal is to learn a policy to maximize the expected cumulative reward in a complex, dynamic environment. This type of learning is used in robotics, games, and other applications that need to learn to make decisions in complex, dynamic environments.

Representing "objects" in machine learning

• An example or example $x$ represents a specific object
• usually with a $d$ -dimensional feature vector$x=(x_1,...,x_d)\in {\mathbb{R}}^d$ means$x$
• Each dimension is called a feature or attribute
• Features can be continuous or discrete
• $x$ is a point in feature space with dimension$d$
• An abstract representation of an object. ignore any other aspects (for example, two people with the same weight and height may be considered the same)

eigenvector representation

• For text documents:

  • The size of the vocabulary is $d$ (about 100,000 words)
  • "Bag of Words": Count the number of occurrences of each vocabulary item
  • Stop words are usually removed: the, of, at, in, etc.
  • Special "Unknown Vocabulary" (OOV) entry to catch all unknown vocabulary

• For images:

  • Pixels, color histograms, etc.
  • Feature Extraction via Convolution

• For software:

  • Execution Profile: Number of times each line of code was executed

• For bank accounts:

  • Credit rating, balance, the number of deposits and withdrawals in the last day, week, month, year, etc.

• For you and me:

  • Medical testing 1, testing 2, testing 3, etc.

Key elements

• Data
  Dataset $D$ contains$N$ data points:
  D = { x 1 , x 2 , . . . , x N } D = \{x_1, x_2, ..., x_N\}D={ x1​,x2​,...,xN​}

• Prediction
  We often want to make predictions based on observed data sets.
  Given $D$ , can we predict$x_{N+1}$？

• Models
  In order to make predictions, we need to make some assumptions. We can usually express these assumptions in the form of a model, which contains some parameters.
  given data $D$ , from which we learn model parameters$\theta$ , so that new data points can be predicted.