Mathematical Modeling Basic Algorithm Model

Algorithmic models commonly used in the National Undergraduate Mathematical Contest in Modeling include but are not limited to the following:

1. Linear regression model: A model used to establish a linear relationship between variables, often used to predict and analyze data.

2. Logistic regression model: Used to establish nonlinear relationships between variables, often used in classification problems and probability predictions.

3. Decision tree model: The process of decomposing a data set into smaller data sets and classifying the subsets, often used in classification and prediction problems.

4. Support Vector Machine Model: A model for classification and regression analysis that maps data to a high-dimensional space and finds a hyperplane to separate the data.

5. Clustering model: A model that divides data into several categories, often used in data analysis, data mining, and image processing.

6. Neural network model: A model that simulates the neural network of the human brain, which can be used for problems such as classification, regression analysis, and pattern recognition.

7. Genetic Algorithm Model: An optimization algorithm that simulates natural selection and genetic mechanisms, often used to solve complex problems.

8. Particle swarm optimization model: an optimization algorithm that simulates the behavior of groups such as birds and fish, and is often used to solve nonlinear optimization problems.

9. Ant colony algorithm model: simulates the behavior of ants when they are looking for food, and is often used to solve combinatorial optimization problems.

10. Simulated annealing algorithm model: an optimization algorithm that finds the optimal solution by simulating the annealing process of a substance, and is often used to solve combinatorial optimization problems and nonlinear optimization problems.

The above algorithm models are commonly used in mathematical modeling competitions. Each model has its advantages and limitations. It is necessary to choose the appropriate algorithm model according to the specific problem. At the same time, the application of these algorithm models also requires certain mathematical knowledge and programming skills to achieve effective solutions to problems.

30+ mathematical modeling algorithm models + case code sharing:

Link: https://pan.baidu.com/s/1Pg_PgPJ8-EJ0RMjZ6_dF3Q?pwd=fid3 
Extraction code: fid3 
 

1. Linear regression model

Linear regression is one of the most basic statistical analysis methods used to analyze the strength and direction of linear relationships between variables. Linear regression can be used to predict and analyze data, and is widely used in natural science, social science, engineering technology and other fields. The following is a detailed introduction to linear regression.

The linear regression model assumes that there is a linear relationship between the dependent variable $y$ and the independent variables $x_1,x_2,\cdots,x_p$, namely:

�=�0+�1�1+�2�2+⋯+����+�y=β0​+β1​x1​+β2​x2​+⋯+βp​xp​+ϵ

Among them, $\beta_0,\beta_1,\beta_2,\cdots,\beta_p$ are linear regression coefficients, and $\epsilon$ is an error term, which is used to describe the degree of change of the dependent variable $y$ under the same independent variable value. Suppose the error term $\epsilon$ satisfies the following assumptions:

1. $\epsilon$ is a random variable.
2. $\epsilon$ follows a normal distribution with mean $0$.
3. The variance of $\epsilon$ is constant $\sigma^2$.

According to the principle of least squares method, the linear regression coefficients can be estimated using sample data. Specifically, the sample data $(x_{1i},x_{2i},\cdots,x_{pi},y_i)$ can be used to estimate the linear regression coefficient, so that the residual sum of squares of the sample data is minimized. Right now:

min⁡�0,�1,�2,⋯ ,��∑�=1�(��−�0−�1�1�−�2�2�−⋯−�����)2minβ0​, β1​,β2​,⋯,βp​​∑i=1n​(yi​−β0​−β1​x1i​−β2​x2i​−⋯−βp​xpi​)2

This process can be solved using optimization algorithms such as gradient descent to obtain the optimal linear regression coefficient.

The advantages of the linear regression model are that it is easy to understand, fast in calculation speed, and can better solve linear relationship problems. However, the linear regression model also has some limitations. For example, it can only deal with linear relationship problems, and its fitting effect on nonlinear data is poor. In addition, the linear regression model is also sensitive to problems such as outliers and multicollinearity, which need to be dealt with in the data preprocessing stage.

Here is a code example for a linear regression model using Python:

import numpy as np
import matplotlib.pyplot as plt

# 生成样本数据
x = np.array([1, 2, 3, 4, 5, 6])
y = np.array([1.5, 3.2, 4.5, 6.7, 8.2, 9.3])

# 拟合线性回归模型
coefficients = np.polyfit(x, y, 1)
a, b = coefficients

#

2. Logistic regression

Logistic Regression (Logistic Regression) is a classic classification model for solving binary classification problems. Similar to linear regression models, logistic regression models are also based on modeling a linear relationship between input features and output labels. However, the output of a logistic regression model is a probability value that represents the probability that the sample belongs to the positive class. Therefore, the logistic regression model also needs to convert the linear predictive value into a probability value. This process is completed by performing the sigmoid function on the linear predictive value.

The sigmoid function is a commonly used activation function, which can map real numbers to the interval from 0 to 1. The formula is: $g(z)=\frac{1}{1+e^{-z}}$.

In logistic regression, we pass the linear prediction value $z$ into the sigmoid function to get the predicted probability $p$, the formula is: $p = g(z) = \frac{1}{1+e^{-z}} $.

When training a logistic regression model, we need to estimate the model parameters by maximizing the log-likelihood function, where the log-likelihood function is: $J(\theta) = \frac{1}{m}\sum_{i= 1}^{m}\left(-y^{(i)}\log(h_\theta(x^{(i)})) - (1-y^{(i)})\log(1- h_\theta(x^{(i)}))\right)$, where $m$ represents the number of samples, $x^{(i)}$ represents the feature vector of the $i$th sample, $y^ {(i)}$ represents the output label of the $i$th sample, and $h_\theta(x^{(i)})$ represents the predicted probability of the model for the $i$th sample. The estimated value of the model parameters $\theta$ can be achieved by optimization algorithms such as gradient descent.

The logistic regression model can be used for binary classification problems, and can also deal with multi-classification problems by extending the output, such as one-to-many (OvR) methods and softmax methods.

Below is a simple Python code example for implementing a binary logistic regression model:

import numpy as np
from scipy.special import expit

class LogisticRegression:
    def __init__(self, learning_rate=0.01, num_iterations=10000):
        self.learning_rate = learning_rate
        self.num_iterations = num_iterations
        self.weights = None
        self.bias = None
    
    def fit(self, X, y):
        num_samples, num_features = X.shape
        self.weights = np.zeros(num_features)
        self.bias = 0
        
        for i in range(self.num_iterations):
            linear_model = np.dot(X, self.weights) + self.bias
            y_pred = expit(linear_model)
            
            dw = (1 / num_samples) * np.dot(X.T, (y_pred - y))
            db = (1 / num_samples) * np.sum(y_pred - y)
            
            self.weights -= self.learning_rate * dw
            self.bias -= self.learning_rate * db
    
    def predict(self, X

3. Decision tree

Decision Tree is a classification and regression method based on tree structure. Its main idea is to divide the data set into small subsets. Each subset can be regarded as a decision node, and each decision node has a corresponding conditions and decision rules. By building a decision tree model, we can divide a data set into different categories or predict continuous values.

The process of building a decision tree is mainly divided into two steps, namely tree construction and tree pruning. The construction of the tree refers to the process of finding the optimal partition attribute from the training data set, and the pruning of the tree is to avoid the phenomenon of over-fitting, and improve the generalization ability of the model by pruning the decision tree.

In the process of building a decision tree, we usually use information gain (ID3 algorithm), gain rate (C4.5 algorithm), Gini index (CART algorithm) and other methods to evaluate the importance of each division attribute. In the tree pruning process, we usually use two methods: pre-pruning and post-pruning. Pre-pruning is to limit the depth of the tree or the number of leaf nodes and other parameters in the process of building a tree, and post-pruning is to The tree is pruned after the tree is built.

The decision tree model has many advantages, such as easy to understand and explain, can handle multiple data types, has high accuracy and stability, etc., so it is widely used in data mining, machine learning, pattern recognition and other fields. At the same time, the decision tree model also has some disadvantages, such as prone to overfitting and sensitivity to certain data. Therefore, when using the decision tree model, it is necessary to select the appropriate algorithm and parameters according to the specific application scenario.

The following is a simple code example of a decision tree classification model implemented in Python

# 导入相关库
from sklearn.datasets import load_iris
from sklearn.tree import DecisionTreeClassifier
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score

# 加载数据集
iris = load_iris()
X = iris.data
y = iris.target

# 划分训练集和测试集
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)

# 构建决策树模型
dtc = DecisionTreeClassifier()
dtc.fit(X_train, y_train)

# 预测结果并计算准确率
y_pred = dtc.predict(X_test)
accuracy = accuracy_score(y_test, y_pred)
print("准确率：", accuracy)

4. Support Vector Machine Model

A Support Vector Machine (SVM) is a common supervised learning model that can be used for both classification and regression problems. Its basic idea is to divide the data set into different categories by finding the optimal hyperplane (or multiple hyperplanes), so that the interval between each category is maximized.

In SVM, a hyperplane is composed of some key sample points (support vectors) and a decision boundary. These support vectors are the closest data points to the hyperplane, they are used to define the classification boundary, and determine the classification decision of the SVM.

SVM has a variety of kernel functions, such as linear kernels, polynomial kernels, and radial basis function (RBF) kernels, which are used to solve linearly separable, linearly inseparable, and nonlinearly separable problems.

Advantages of SVMs include:

• Works well in high-dimensional spaces.
• Excellent performance when dealing with small samples.
• Can handle nonlinear classification problems.
• Robust to outliers.
• Different kernel functions can be used to suit different data distributions.

Here is a simple Python example showing how to use SVM for binary classification:

from sklearn import svm, datasets
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score

# 加载数据集
iris = datasets.load_iris()
X = iris.data[:, :2]
y = iris.target

# 划分训练集和测试集
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)

# 定义SVM分类器
clf = svm.SVC(kernel='linear')

# 训练模型
clf.fit(X_train, y_train)

# 预测测试集
y_pred = clf.predict(X_test)

# 计算准确率
accuracy = accuracy_score(y_test, y_pred)
print("Accuracy:", accuracy)

5. Clustering model

Clustering is an unsupervised learning method, its goal is to divide the samples in the data set into several different groups, the similarity between each group is as small as possible, and the similarity within the group is as large as possible. Clustering is often used in data mining, image analysis, natural language processing and other fields.

Clustering algorithms can be divided into the following categories:

1. Partition-based clustering algorithms, such as the k-means algorithm.
2. Hierarchical clustering algorithms such as agglomerative hierarchical clustering and divisive hierarchical clustering.
3. Density-based clustering algorithms, such as DBSCAN algorithm and OPTICS algorithm.
4. Model-based clustering algorithms, such as the Gaussian mixture model clustering (GMM) algorithm.

Among them, the k-means algorithm is one of the most common clustering algorithms. Its basic idea is to divide the samples in the data set into k different groups, and the sum of the distances between the center of each group and all sample points in the group is the smallest. The steps of the k-means algorithm are as follows:

1. Select k initial cluster centers (can be selected randomly or according to specific rules).
2. Assign all data points to their nearest cluster centers.
3. Compute new centers for each cluster.
4. Repeat steps 2 and 3 until the cluster centers no longer change or reach the preset number of iterations.

Here is an example implementation of the k-means algorithm using Python and the scikit-learn library:

from sklearn.cluster import KMeans
from sklearn.datasets import make_blobs

# 生成随机数据
X, y = make_blobs(n_samples=1000, centers=4, random_state=42)

# 创建k-means模型
kmeans = KMeans(n_clusters=4)

# 训练模型
kmeans.fit(X)

# 预测类别
y_pred = kmeans.predict(X)

# 打印聚类结果
print("Cluster centers:", kmeans.cluster_centers_)
print("Cluster labels:", kmeans.labels_)

6. Neural Network Model

A neural network is a computational model consisting of a large number of interconnected simple processing units capable of performing various tasks such as classification, regression, clustering, and generation by learning patterns in data.

In a neural network, the processing units are called neurons, and they are usually arranged in a hierarchical structure, with each layer consisting of several neurons. The input layer accepts the input of raw data, the output layer outputs the prediction results of the model, and the middle layer is responsible for feature extraction and conversion of the input data.

The basic component of a neural network is an artificial neuron (Artificial Neuron), which simulates the structure and function of a biological neuron. A neuron takes multiple input signals, weights them, and converts the weighted sum into an output signal through an activation function. Commonly used activation functions include sigmoid function, ReLU function, and tanh function.

The training of neural networks usually uses the backpropagation algorithm (Backpropagation). The basic idea is to adjust the connection weights between neurons by calculating the error between the model output and the real label, thereby minimizing the prediction error of the model. The backpropagation algorithm requires two processes of forward propagation and backpropagation, where forward propagation is used to calculate the output of the model, and backpropagation is used to calculate the error and update the weights.

In addition to the basic forward and backpropagation algorithm, there are many improved neural network models, such as Convolutional Neural Network, Recurrent Neural Network and Generative Adversarial Network, etc., which It has been widely used in different fields and tasks.

Here is an example of a simple neural network implementation:

import numpy as np

class NeuralNetwork:
    def __init__(self, input_size, hidden_size, output_size):
        self.weights1 = np.random.randn(input_size, hidden_size)
        self.bias1 = np.zeros((1, hidden_size))
        self.weights2 = np.random.randn(hidden_size, output_size)
        self.bias2 = np.zeros((1, output_size))
    
    def sigmoid(self, x):
        return 1 / (1 + np.exp(-x))
    
    def forward(self, x):
        z1 = np.dot(x, self.weights1) + self.bias1
        a1 = self.sigmoid(z1)
        z2 = np.dot(a1, self.weights2) + self.bias2
        a2 = self.sigmoid(z2)
        return a2
    
    def loss(self, x, y):
        y_pred = self.forward(x)
        return np.mean(np.square(y_pred - y))
    
    def train(self, x, y, learning_rate=0.1):
        z1 = np.dot(x, self.weights1) + self.bias1
        a1 = self.sigmoid(z1)
        z2 = np.dot(a1, self.weights2) + self.bias2
       

7. Genetic Algorithm Model

Genetic Algorithm (GA) is an optimization algorithm that simulates the evolution process of natural biological genetics. probability to generate a new solution in order to obtain a better solution. The genetic algorithm mainly consists of the following three operations:

1. Selection: In the optimization process of the genetic algorithm, a certain selection strategy is used to select the appropriate solution for the next step, so as to retain good genetic characteristics.

2. Crossover: By exchanging the gene segments of the chromosomes of two individuals, new individuals are generated, thereby generating more solutions, which is helpful for the exploration of the solution space.

3. Mutation: Randomly change the value of some genes in order to find more solutions in the search space.

Genetic algorithm is widely used in optimization problems, especially when the search space is complex and the optimal solution cannot be determined, it has good adaptability. In practical applications, genetic algorithms can be used in solving optimization problems, machine learning, combinatorial optimization and other fields.

The implementation steps of the genetic algorithm are as follows:

1. Determine the fitness function of the problem, also known as the objective function.

2. Determine the initial population, that is, some initial solutions are given.

3. Calculate the fitness value of each individual in the population.

4. According to the fitness value, a new population is generated through genetic operations such as selection, crossover, and mutation.

5. Repeat steps 3 and 4 until a predetermined termination condition is reached.

The following is a Python implementation example of a genetic algorithm:

import random

# 定义适应度函数
def fitness_func(x):
    return x[0] ** 2 + x[1] ** 2

# 初始化种群
def init_population(pop_size, chromo_size):
    pop = []
    for i in range(pop_size):
        chromo = [random.randint(0, 1) for j in range(chromo_size)]
        pop.append(chromo)
    return pop

# 计算适应度值
def cal_fitness_value(pop):
    fitness_value = []
    for chromo in pop:
        x = [0, 0]
        for i in range(len(chromo)):
            if chromo[i] == 0:
                x[0] += 2 ** i
            else:
                x[1] += 2 ** i
        x[0] = x[0] / (2 ** len(chromo) - 1) * 10 - 5
        x[1] = x[1] / (2 ** len(chromo) - 1) * 10 - 5
        fitness_value.append(fitness_func(x))
    return fitness_value

# 选择操作