Python quickly builds neural networks

Python quickly builds neural networks

I. Introduction

Machine learning has always been a hot direction for Python, and deep learning derived from neural network algorithms shines in many aspects. So what exactly is a neural network?

Speaking of neural networks, it is easy for people to think of neural networks in biology, and many times the neural networks of machine learning and biological neural networks are also connected. But in fact, human beings have not fully understood the operation of biological neural networks, let alone using computers to implement biological neural networks.

In contrast, the neural network in machine learning is more like a mathematical function. We input a set of data, and the neural network will return a result to us. A simple function like the following:
$$and=wx+b\left(Wandbareconstantnumbers\right)$$
we given a x, can be obtained a y. It's just that the function of the neural network is much more complicated than the above function.

But in fact, the basis of neural network is the above function. Let's take you to quickly build a neural network.

Second, machine learning

Before learning neural networks, we need to understand some machine learning knowledge.

2.1. What is machine learning?

Suppose I have the following set of data:

1, 3, 5, 7, 9

Now let you say what the next number might be.

As far as human intelligence is concerned, we can say it very quickly 11. But it's not that simple for computers, because computers can't think. How do computers learn? This requires human guidance.

In machine learning, humans need to tell the machine how to learn. Then learn through the learning methods told by humans and get a model.

Of course, there are other forms of machine learning, and we will not continue to discuss them.

2.2. How to learn?

For machine learning, how to learn is a very important issue. Many excellent algorithms have emerged among them, and the role of these algorithms is to tell the machine how to learn. Such as linear regression, logistic regression, K nearest neighbors, decision trees, neural networks, etc.

Machine learning algorithms can be said to be the soul of machine learning. The neural network we are going to implement today is also a machine learning algorithm. It is based on logistic regression, which is based on linear regression. Therefore, linear regression and logistic regression are also things to learn today.

2.3 Problems in machine learning

Machine learning problems are usually divided into two categories, one is classification, and the other is regression.

The difference between the two is whether the results are discrete. For example, for an animal classification problem, the result we get can only be a certain animal. You won’t get an animal between cats and dogs.

The result of a regression problem is usually a numerical value, such as a housing price prediction problem. We may get any value between 0 and 1 million, or a decimal like 40.023242.

Among them, linear regression is a great tool for solving regression problems, while logistic regression is used for classification problems. Let's take a look at these two algorithms.

Three, linear regression and logistic regression

You may be wondering why there is a regression in the name of logistic regression, but it is not the solution to the regression problem. I believe there will be no doubt after reading the content below.

3.1, linear regression

In the preface, we introduced a simple function:
$$and=wx+b$$
In fact, it is the basis of linear regression. Linear regression algorithm is to find a set of optimalw b, so that we can get the closest to the real result. We still use the above data:

1, 3, 5, 7, 9

For the above set of data, we are looking for the relationship between the serial number and the value. We can understand the above data as:

x, y
1, 1
2, 3
3, 5,
4, 7,
5, 9

Where x represents the need, and y represents the specific value. We can get the following function with a little calculation:
$$and=2x-1$$
We have obtained the optimal set of parametersw=2, b = -1, through this function we can predict the next one thousand or ten thousand numbers.

But sometimes we have multiple x, then we can generalize the above function to:
$$and=w_1{x}_1+w_2{x}_2+...+w_n{x}_n+b\; At$$
this time, we need more parameters. Let's actually write a linear regression program.

3.2, linear regression actual combat

Here we need to use the scikit-learnmodule, install it as follows:

pip install scikit-learn

Then we can start writing code. Achieve a linear regression algorithm is encapsulated in the sklearn.linear_modelmiddle of LinearRegression, we can directly use:

import numpy as np
from sklearn.linear_model import LinearRegression
# 准备x的数据
X = np.array([
    [1],
    [2],
    [3],
    [4],
    [5]
])
# 准备y的数据
y = np.array([1, 3, 5, 7, 9])
# 创建线性回归模块
lr = LinearRegression()
# 填充数据并训练
lr.fit(X, y)
# 输出参数
print("w=", lr.coef_, "b=", lr.intercept_)

First of all, we need to prepare the data for the Xsum y, here we are using an ndarray array. It should be noted here that our ydata length is 5, then Xthe data needs to be 5*n.

After preparing the data, we need to create a linear regression model, then call the fit method to fill the data we prepared and train.

After the training is completed, we can check the parameters of the module, which coef_means w, and intercept_means b. Because there wcan be more than one, it should be an array, the following is the output:

w= [2.] b= -1.0

The result is the same as our artificial intelligence. We can also call the predict method to predict the following data:

import numpy as np
from sklearn.linear_model import LinearRegression
X = np.array([
    [1],
    [2],
    [3],
    [4],
    [5]
])
y = np.array([1, 3, 5, 7, 9])
lr = LinearRegression()
lr.fit(X, y)
y_predict = lr.predict(np.array([[100]]))
print(y_predict)

It is also important to note that the data of X is two-dimensional.

3.3, logistic regression

Logistic regression can be understood as linear regression + special function. We can think about the following question.

Now we need to write a program to judge whether each person's score is passing. The scoring standard is: total score = 40% mathematics + 30% language + 30% English. The total score is greater than or equal to 60 as passing, and the rest as failing.

Although it is a very simple question, we still need to discuss it. First, we can write the formula for calculating the total score in the following form:
$$and=0.4x_1+0.3x_2+0.3x_3+b$$

For this formula, we can get any number between 0-100. But there are only two results I want to get, pass or fail. We can simply understand it as -1 and 1.

So how do we map the above result to -1 and 1? This requires the use of a special function, we call this function the activation function. We can use the following function as the activation function:
$$f(x)=\frac{and-60}{|y-60|}$$
In this way, all scores can be mapped to -1 and 1. (The above function is not defined at y=60. Strictly speaking, the above activation function is not applicable) The logistic regression diagram is as follows:

First obtain a result through a linear model, and then map the result to the specified range through the activation function.

However, in practical applications, we usually use the Sigmoid、Tanhand ReLUfunction. Below are images of several activation functions:

Let's write an example of logistic regression.

3.4. Logistic regression in practice

We use logistic regression to solve several problems. The realization of logistic regression is encapsulated in linear_model.LogisticRegressionit, and it can also be used directly. We directly upload the code:

import numpy as np
from sklearn.linear_model import LogisticRegression
# 准备X的数据
X = np.array([
    [60],
    [20],
    [30],
    [80],
    [59],
    [90]
])
# 准备y的数据
y = np.array([1, 0, 0, 1, 0, 1])
# 创建逻辑回归模型
lr = LogisticRegression()
# 填充数据并训练
lr.fit(X, y)
# 准备用于测试的数据
X_test = np.array([
    [62],
    [87],
    [39],
    [45]
])
# 判断测试数据是否及格
y_predict = lr.predict(X_test)
print(y_predict)

There are only minor differences between the code and linear regression. In the code, we use 0 for failing and 1 for passing. The following is the result of our test data output:

[1 1 0 0]

You can see that all the results are predicted correctly.

With the above knowledge, we can start to implement a neural network.

Fourth, the neural network

Neural network is built on logistic regression. It can be said that neural network is a collection of logistic regression.

4.1, neural network

Everyone must have heard that a neural network is composed of a large number of neurons. But you may not know that the neuron in machine learning is the logistic regression we learned earlier. We can look at the picture below:

It can be seen that it is very similar to the previous logistic regression, but a lot of activation functions are used here, and the number of parameters is much larger.

As for why so many activation functions are used, it can be said that the function obtained is very complicated. If our function is very simple, such as the following set of data:

If we use the following function as our model:
$$and=wx+b$$
will get the following image:

It can be seen that many points are not on a straight line, so the predicted data will have a lot of errors. At this time, we can consider two times, such as:
$$and=w{x}^2+b$$
But sometimes the model we want cannot be obtained twice, three times, or even more than a dozen times (too many times often lead to over-fitting). At this time, the neural network is a good choice.

The interpretability of neural networks is much worse than the previous two algorithms, because neural networks usually have hundreds or thousands of parameters, and we will get a very complex model. Although we cannot understand the meaning of the parameters, these parameters usually give us a good result. But this is really the magic of neural networks.

4.2, input layer, hidden layer, output layer

A neural network usually has three parts, and the input layer is determined by the number of our features. The output layer is determined by the number of our classifications. As shown in the figure, the x part is the input layer, the y part is the output layer, and the middle part is the hidden layer:

Hidden layers are usually very complicated. We can adjust the complexity of the model by adjusting the number of hidden layers and the number of nodes.

4.3, neural network combat

Using it scikit-learn, we can quickly build a neural network. Next, we use scikit-learnthe data set that comes with it to implement a neural network:

from sklearn.datasets import load_iris
from sklearn.neural_network import MLPClassifier
from sklearn.model_selection import train_test_split

# 加载数据集
iris_data = load_iris()
# 拆分数据集
X_train, X_test, y_train, y_test = train_test_split(iris_data['data'], iris_data['target'], test_size=0.25, random_state=1)
# 创建神经网络模型
mlp = MLPClassifier(solver='lbfgs', hidden_layer_sizes=[4, 2], random_state=0)
# 填充数据并训练
mlp.fit(X_train, y_train)
# 评估模型
score = mlp.score(X_test, y_test)
print(score)

Here we use the scikit-learnbuilt-in iris data, we use to train_test_splitdivide the data set into two parts, which are the feature and target value of the training set, and the feature and target value of the test set.

Then we create MLPClassifieran instance of the class, which is actually a multi-perceptron for classification. We only need to pay attention to the hidden_layer_sizesparameters, which are the number of layers and nodes of our neural network. Because it is a binary classification problem, the output layer here has two nodes.

The following output:

0.9210526315789473

We call mlp.score to evaluate the quality of the model, and the accuracy of 92% is considered a very good result.