There are many types of algorithms involved in the National Undergraduate Mathematical Contest in Modeling, but the following are ten commonly used algorithms in the competition :
1. Linear regression : used to predict a continuous output variable.
Linear regression is a basic statistical method used to model the linear relationship between an independent variable (or independent variables) and a dependent variable to predict a continuous output variable. The form of this model can be expressed as:
y = β0 + β1x1 + β2x2 + ... + βpxp + ε
where y is the dependent variable (also known as the response variable), x1, x2, ..., xp are the independent variables (also known as the characteristic variables), β0, β1, β2, ..., βp are the linear regression model coefficient, ε is the error term
The goal of linear regression is to find the optimal coefficients β0, β1, β2, ..., βp, so that the error between the value predicted by the model and the true value is minimized. This error is usually expressed as a residual sum of squares:
RSS = Σ (yi - ŷi)^2
where yi is the true dependent variable value and ŷi is the dependent variable value predicted by the linear regression model. The least squares estimation method of the linear regression model is to find a set of coefficients that minimize the sum of squared residuals.
Linear regression can be solved by a variety of methods, the most common of which is the method of least squares. The least squares method is to find a set of coefficients that minimize the sum of squared residuals. The least squares method can be implemented by matrix operations. Specifically, the solution of the coefficients can be expressed as:
β = （X'X）^（-1）X'y
where X is a matrix of independent variables, including an intercept term and values for all independent variables, and y is a vector of dependent variables.
Linear regression is widely used in practice. For example, in fields such as finance, medicine, engineering, and social sciences, linear regression can be used to predict and analyze data.
The following is a simple Python code to implement linear regression
import numpy as np
from sklearn.linear_model import LinearRegression
# 创建一个随机数据集
np.random.seed(0)
X = np.random.rand(100, 1)
y = 2 + 3 * X + np.random.rand(100, 1)
# 创建线性回归模型并拟合数据
model = LinearRegression()
model.fit(X, y)
# 打印模型的系数和截距项
print('Coefficients:', model.coef_)
print('Intercept:', model.intercept_)
# 预测新数据
X_new = np.array([[0.5], [1.0]])
y_new = model.predict(X_new)
# 打印预测结果
print('Predictions:', y_new)
This code uses the Numpy library to generate a random dataset of 100 samples and creates a linear regression model using the LinearRegression class of the Scikit-learn library. The model is fitted to the data via the fit() method, and the coefficient and intercept terms of the model are accessed via the coef_ and intercept_ attributes. Finally, the code predicts the outcome of the two new data points using the predict() method and prints out the predictions.
2. Logistic regression: used to predict a discrete output variable, such as a binary classification problem.
Logistic regression is a common classification algorithm used to model the relationship between one or more independent variables and a binary or multivariate discrete dependent variable. Its name "logic" comes from the fact that its model is essentially a logistic function used to transform an input value into a probability value. Logistic regression is commonly used for binary classification problems, but can also be extended to multivariate classification problems.
The basic form of a logistic regression model is as follows:
p（y=1|x） = 1 / （1 + exp（-（b0 + b1x1 + b2x2 + ... + bpxp）））
Among them, p(y=1|x) is the probability that the dependent variable y takes the value of 1 given the independent variable x, exp() is the exponential function, and b0, b1, b2, ..., bp are the coefficients of the model.
The goal of logistic regression is to find the optimal coefficients b0, b1, b2, ..., bp to maximize the likelihood function, so that the results predicted by the model are as close to the real value as possible. Usually, we use maximum likelihood estimation to estimate the coefficients of the model.
During training, a logistic regression model uses a cost function called the logistic loss function to measure the error between the prediction and the true value. The logistic loss function is as follows:
J（b） = （-1/m） * Σ[yi*log（p（xi）） + （1-yi）*log（1-p（xi））]
where m is the number of samples, yi is the true class label (0 or 1), and p(xi) is the class probability predicted by the model.
Logistic regression can use optimization algorithms such as gradient descent or Newton's method to minimize the logistic loss function to obtain optimal model parameters. Finally, the model inputs the independent variable into the logistic function to obtain the classification probability, and uses a threshold to convert the probability into a classification label, usually with a threshold of 0.5.
Logistic regression is widely used in practice. For example, in finance, medicine, social science and other fields, logistic regression can be used to predict and analyze data.
Here is a simple Python code to implement logistic regression:
import numpy as np
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score
# 创建一个随机数据集
np.random.seed(0)
X = np.random.rand(100, 3)
y = np.random.randint(0, 2, 100)
# 划分数据集为训练集和测试集
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=0)
# 创建逻辑回归模型并拟合数据
model = LogisticRegression()
model.fit(X_train, y_train)
# 预测测试集的结果
y_pred = model.predict(X_test)
# 计算准确率
accuracy = accuracy_score(y_test, y_pred)
print('Accuracy:', accuracy)
This code uses the Numpy library to generate a random dataset of 100 samples and creates a logistic regression model using the LogisticRegression class of the Scikit-learn library. The model fits the data through the fit() method and predicts the results on the test set through the predict() method. Finally, the code calculates the accuracy of the model using the accuracy_score() method and prints out the result.
3. Decision tree: used for classification and regression problems, making decisions by building a tree structure.
Decision trees are a common machine learning algorithm used to solve classification and regression problems. Its basic idea is to divide the data set into multiple subsets, each subset corresponds to a decision tree node, and finally forms a tree structure. Each node of the decision tree represents a feature, the branch represents the value of the feature, and the leaf node represents the result of classification or regression.
The construction process of decision tree is generally divided into two stages: tree generation and pruning. The tree generation process starts from the root node, and selects the optimal features for division in turn until all leaf nodes belong to the same category or meet a certain stop condition. The most commonly used feature selection method is information gain or information gain ratio. Information gain refers to the degree of uncertainty reduction in the data set before and after division. The greater the information gain, the greater the impact of features on classification.
The pruning process is to avoid overfitting, which is the situation where the training set performs well but the test set performs poorly. The purpose of pruning is to remove some decision tree nodes, so that the decision tree is simpler and the generalization ability is stronger. Pruning methods usually include pre-pruning and post-pruning. Pre-pruning is to stop dividing when a node cannot continue to divide during the tree generation process. Post-pruning is to prune the generated tree after the tree generation process ends. Specific methods of pruning include cross-validation pruning and error rate reduction pruning.
Decision trees are widely used in both classification and regression problems. Its advantages include ease of understanding and interpretation, handling missing data, insensitivity to outliers, suitable for multi-classification and regression problems, etc. However, decision trees also have some disadvantages, such as easy overfitting and sensitivity to subtle changes in input data.
The following is a sample code to build and train a decision tree classifier using the DecisionTreeClassifier class in the Scikit-learn library:
from sklearn.datasets import load_iris
from sklearn.tree import DecisionTreeClassifier
from sklearn.model_selection import train_test_split
# 载入数据集
iris = load_iris()
# 划分数据集为训练集和测试集
X_train, X_test, y_train, y_test = train_test_split(iris.data, iris.target, test_size=0.2, random_state=42)
# 创建决策树分类器并拟合数据
clf = DecisionTreeClassifier(random_state=42)
clf.fit(X_train, y_train)
# 预测测试集的结果
y_pred =
In this example, we use the Iris dataset and split the dataset into train and test sets
Fourth, support vector machine: used for classification and regression problems, by finding an optimal separation hyperplane for classification.
Support Vector Machine (SVM for short) is a commonly used supervised learning algorithm, often used in classification and regression problems. SVMs are based on mapping data into a high-dimensional space and finding a maximal margin hyperplane in that space for classification or regression.
The goal of SVM is to find a maximum margin hyperplane that separates data of different classes such that data points of the same class are as close as possible to this hyperplane. Specifically, for binary classification problems, SVM maps the data into a high-dimensional space and finds a hyperplane that separates the two classes of data and has the largest distance from the points closest to the two classes of data points.
When implementing SVM, it is necessary to select a kernel function to map the data. The commonly used kernel functions include linear kernel, polynomial kernel and radial basis function (Radial Basis Function, RBF) kernel.
Here is an example code for a classification problem using the SVM class (SVC) from the Scikit-learn library:
from sklearn.datasets import load_iris
from sklearn.svm import SVC
from sklearn.model_selection import train_test_split
# 载入数据集
iris = load_iris()
# 划分数据集为训练集和测试集
X_train, X_test, y_train, y_test = train_test_split(iris.data, iris.target, test_size=0.2, random_state=42)
# 创建 SVM 分类器并拟合数据
clf = SVC(kernel='linear', C=1.0, random_state=42)
clf.fit(X_train, y_train)
# 预测测试集的结果
y_pred = clf.predict(X_test)
# 计算分类器在测试集上的准确率
accuracy = clf.score(X_test, y_test)
print("Accuracy: {:.2f}%".format(accuracy*100))
5. Clustering: It is used to divide the data in the data set into different groups.
Clustering is an unsupervised learning algorithm used to divide objects in a dataset into several similar groups or categories. The goal of a clustering algorithm is to find some similar data points and divide them into different categories or clusters, so that the data points of the same category are as similar as possible, while the data points of different categories are as different as possible.
Common clustering algorithms include K-Means algorithm, hierarchical clustering algorithm and DBSCAN algorithm, etc. Among them, the K-Means algorithm is one of the most common clustering algorithms, which divides data points into K clusters and assigns each data point to the nearest cluster,
from sklearn.datasets import make_blobs
from sklearn.cluster import KMeans
# 生成模拟数据
X, y_true = make_blobs(n_samples=300, centers=4, cluster_std=0.60, random_state=0)
# 创建 KMeans 聚类器并拟合数据
kmeans = KMeans(n_clusters=4, random_state=0)
kmeans.fit(X)
# 预测数据的簇标签
y_pred = kmeans.predict(X)
# 绘制聚类结果
plt.scatter(X[:, 0], X[:, 1], c=y_pred, s=50, cmap='viridis')
centers = kmeans.cluster_centers_
plt.scatter(centers[:, 0], centers[:, 1], c='black', s=200, alpha=0.5);
6. Neural network: It is used for classification and regression problems, and is calculated by constructing a multi-layer neural network.
A neural network is an algorithmic model that imitates the structure and working methods of the neural network of the human brain. It consists of many simple units, or neurons, each of which receives inputs from other neurons and combines these inputs into an output. A neural network usually consists of multiple layers, including an input layer, a hidden layer, and an output layer, and each layer consists of several neurons.
Neural networks can be used for tasks such as classification, regression, and clustering, the most common of which is classification. The training of neural network classifiers usually adopts the backpropagation algorithm, which updates the weights of the neural network by calculating the error gradient, so that the output of the neural network is as close as possible to the real label.
Here is an example code for a classification problem using the MLPClassifier class from the Scikit-learn library:
from sklearn.datasets import make_classification
from sklearn.neural_network import MLPClassifier
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score
# 生成模拟数据
X, y = make_classification(n_samples=1000, n_features=10, n_classes=2, random_state=1)
# 将数据集划分为训练集和测试集
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=1)
# 创建 MLPClassifier 分类器并拟合数据
mlp = MLPClassifier(hidden_layer_sizes=(10, 5), max_iter=1000, random_state=1)
mlp.fit(X_train, y_train)
# 预测测试集的标签
y_pred = mlp.predict(X_test)
# 计算分类器的准确率
acc = accuracy_score(y_test, y_pred)
print(f"Accuracy: {acc}")
Seven, genetic algorithm: used to find the optimal solution to the optimization problem.
Genetic algorithm is an optimization algorithm that imitates natural selection and genetic mechanism, and is mainly used to solve optimization problems. It simulates the process of inheritance, crossover and mutation in the process of biological evolution, and gradually searches for the global optimal solution by continuously evolving excellent individuals.
The basic process of genetic algorithm is as follows:
- Initial population: Randomly generate a group of individuals as the population.
- Evaluate fitness: evaluate the fitness of each individual, and usually use the objective function to calculate the fitness of the individual.
- Selection operation: According to the fitness of each individual, select some individuals as parents to generate the next generation.
- Crossover operation: Perform crossover operation on parent individuals to generate new individuals.
- Mutation operation: Perform mutation operation on new individuals to generate more diversity.
- Evaluate new individuals: Evaluate the fitness of new individuals.
- Judging the termination condition: if the termination condition is satisfied, output the optimal solution; otherwise, return to step 3.
The following is a sample code that uses Python to implement a genetic algorithm to solve the problem of the minimum value of a function of one variable:
import random
# 目标函数：f(x) = x^2
def objective_function(x):
return x ** 2
# 生成随机个体
def generate_individual():
return random.uniform(-10, 10)
# 计算个体适应度
def calculate_fitness(individual):
return 1 / (1 + objective_function(individual))
# 选择操作
def selection(population):
fitnesses = [calculate_fitness(individual) for individual in population]
total_fitness = sum(fitnesses)
probabilities = [fitness / total_fitness for fitness in fitnesses]
selected = random.choices(population, weights=probabilities, k=len(population))
return selected
# 交叉操作
def crossover(individual1, individual2):
alpha = random.uniform(0, 1)
new_individual1 = alpha * individual1 + (1 - alpha) * individual2
new_individual2 = alpha * individual2 + (1 - alpha) * individual1
return new_individual1, new_individual2
# 变异操作
def mutation(individual):
new_individual = individual + random.uniform(-1, 1)
return new_individual
# 遗传算法求解最小值问题
population_size = 100
population = [generate_individual() for i in range(population_size)]
num_generations = 1000
for generation in range(num_generations):
# 选择操作
selected_population = selection(population)
# 交叉操作
offspring_population = []
for i in range(population_size):
offspring1, offspring2 = crossover(selected_population[i], selected_population[(i+1) % population_size])
offspring_population.append(offspring1)
offspring_population.append(offspring2)
# 变异操作
for i in range(population_size):
if random.uniform(0, 1) < 0.1:
offspring_population[i] = mutation(offspring_population[i])
#
Eight, particle swarm algorithm: used to find the optimal solution to the optimization problem.
Particle Swarm Optimization (PSO) is an optimization algorithm based on swarm intelligence, which realizes the search for optimization goals by simulating the movement of particles in the swarm and the information exchange of the swarm. Each particle moves in the search space, and records its own optimal position and the optimal position of the group, and gradually approaches the optimal solution by constantly adjusting its position and speed.
The basic process of particle swarm optimization algorithm is as follows:
- Initialize particle swarm: Randomly generate a set of particle positions and velocities.
- Calculate the fitness value: Calculate the fitness of each particle, usually using the objective function to calculate the fitness of the particle.
- Update the individual optimal value: take the current position of each particle as the individual optimal position, if the position is better than the individual historical optimal position, then update the individual historical optimal position.
- Update the optimal value of the group: take the individual optimal position of all particles as the optimal position of the group.
- Update velocity and position: Calculate the new velocity and position based on the particle's current position, velocity and group optimal position.
- Judging the termination condition: if the termination condition is satisfied, output the optimal solution; otherwise, return to step 2.
The following is a sample code that uses Python to implement the particle swarm algorithm to solve the problem of the minimum value of a function of one variable:
import random
# 目标函数：f(x) = x^2
def objective_function(x):
return x ** 2
# 生成随机粒子
def generate_particle():
position = random.uniform(-10, 10)
velocity = random.uniform(-1, 1)
return {'position': position, 'velocity': velocity, 'personal_best_position': position, 'personal_best_fitness': objective_function(position)}
# 更新个体最优值
def update_personal_best(particle):
fitness = objective_function(particle['position'])
if fitness < particle['personal_best_fitness']:
particle['personal_best_position'] = particle['position']
particle['personal_best_fitness'] = fitness
# 更新群体最优值
def update_global_best(particles):
global_best_position = particles[0]['personal_best_position']
global_best_fitness = particles[0]['personal_best_fitness']
for particle in particles:
if particle['personal_best_fitness'] < global_best_fitness:
global_best_position = particle['personal_best_position']
global_best_fitness = particle['personal_best_fitness']
return global_best_position, global_best_fitness
# 更新速度和位置
def update_velocity_and_position(particle, global_best_position):
w = 0.5 # 惯性权重
c1 = 0.5 # 个体学习因子
c2 = 0.5 # 群体学习因子
r1 = random.uniform(0, 1)
r2 = random.uniform(0, 1)
new_velocity = w * particle['velocity'] + c1 * r1 * (particle['personal_best_position'] - particle['position']) + c2 * r2 * (global_best_position - particle['position'])
new
Nine, ant colony algorithm: used to solve combinatorial optimization problems.
Ant Colony Optimization (ACO) is a heuristic optimization algorithm that simulates the behavior and information exchange of ants when they are looking for food. The algorithm simulates the foraging behavior of ants, uses pheromone as guiding information, and realizes the search for the optimal path by accumulating pheromone on the search path.
The basic process of ant colony algorithm is as follows:
- Initialize pheromone: initialize a certain amount of pheromone for each path.
- Initialize the ant position: randomly assign the starting position of the ants.
- Select the next location: Select the next location according to the current location and pheromone distribution.
- Update pheromone: update the pheromone according to the path the ants have passed.
- Judging the termination condition: if the termination condition is satisfied, output the optimal solution; otherwise, return to step 3.
The following is a sample code that uses Python to implement the ant colony algorithm to solve the traveling salesman problem (TSP)
import random
# 旅行商问题：求解城市之间的最短路径
class TSP:
def __init__(self, num_cities, distance_matrix):
self.num_cities = num_cities
self.distance_matrix = distance_matrix
# 计算路径长度
def path_length(self, path):
length = 0
for i in range(len(path)-1):
length += self.distance_matrix[path[i]][path[i+1]]
length += self.distance_matrix[path[-1]][path[0]]
return length
# 生成随机解
def random_solution(self):
path = list(range(self.num_cities))
random.shuffle(path)
return path
# 蚂蚁类
class Ant:
def __init__(self, tsp, alpha, beta, rho):
self.tsp = tsp
self.alpha = alpha # 信息素重要程度因子
self.beta = beta # 启发式因子
self.rho = rho # 信息素挥发因子
self.current_city = random.randint(0, tsp.num_cities-1) # 当前所在城市
self.visited_cities = [self.current_city] # 已访问过的城市
self.path_length = 0 # 路径长度
# 选择下一步城市
def choose_next_city(self, pheromone_matrix):
unvisited_cities = list(set(range(self.tsp.num_cities)) - set(self.visited_cities))
probabilities = [0] * len(unvisited_cities)
total_pheromone = 0
for i, city in enumerate(unvisited_cities):
probabilities[i] = pheromone_matrix[self.current_city][city] ** self.alpha * ((1 / self.tsp.distance_matrix[self.current_city][city]) ** self.beta)
total_pheromone += probabilities[i]
if total_pheromone == 0:
return random.choice(unvisited_cities)
probabilities = [p / total_pheromone for p in probabilities]
next_city = random.choices(unvisited_cities, weights=probabilities)[0]
10. Simulated annealing algorithm: used to find an optimal solution in a large search space.
The simulated annealing algorithm (Simulation Annealing, SA) is a probability-based global optimization algorithm, which is inspired by the microscopic state change process of solid materials during annealing. The algorithm accepts an inferior solution with a certain probability to avoid falling into the local optimal solution, and gradually reduces the probability in the iterative process, and finally reaches the goal of the global optimal solution.
The basic flow of the simulated annealing algorithm is as follows:
- Initialization temperature T, initial solution x, termination temperature Tmin and cooling rate α.
- Iterate until the temperature drops to Tmin: Randomly generate a new solution y in the neighborhood of the current solution x.
- Judging acceptance probability: calculate the difference ΔE between the current solution x and the new solution y, if ΔE<0, accept the new solution y; otherwise accept the new solution y with a certain probability, the probability is e^(-ΔE/T).
- Cooling: Gradually lower the temperature T through the cooling rate α.
- Return to step 2.
The following is a sample code that uses Python to implement the simulated annealing algorithm to solve the traveling salesman problem (TSP):
import math
import random
# 旅行商问题：求解城市之间的最短路径
class TSP:
def __init__(self, num_cities, distance_matrix):
self.num_cities = num_cities
self.distance_matrix = distance_matrix
# 计算路径长度
def path_length(self, path):
length = 0
for i in range(len(path)-1):
length += self.distance_matrix[path[i]][path[i+1]]
length += self.distance_matrix[path[-1]][path[0]]
return length
# 生成随机解
def random_solution(self):
path = list(range(self.num_cities))
random.shuffle(path)
return path
# 模拟退火类
class SimulatedAnnealing:
def __init__(self, tsp, T, Tmin, alpha):
self.tsp = tsp
self.T = T # 初始温度
self.Tmin = Tmin # 终止温度
self.alpha = alpha # 降温速率
# 计算接受概率
def acceptance_probability(self, old_cost, new_cost, T):
if new_cost < old_cost:
return 1
else:
return math.exp(-(new_cost - old_cost) / T)
# 迭代求解
def solve(self):
current_solution = self.tsp.random_solution()
current_cost = self.tsp.path_length(current_solution)
while self.T > self.Tmin:
new_solution = self.tsp.random_solution()
new_cost = self.tsp.path_length(new_solution)
if self.acceptance_probability(current_cost, new_cost, self.T) > random.random():
current_solution = new_solution
current_cost = new_cost
self.T *= self.alpha
return current_solution, current_cost
The Mathematical Contest in Modeling is one of the most important competitions for undergraduates and postgraduates, including the National Undergraduate Mathematical Contest in Modeling (commonly known as the "National Competition") and the American Undergraduate Mathematical Contest in Modeling (commonly known as the "American Competition"). Achieving good results in these competitions is not only helpful for maintaining research and finding a job, but more importantly, forming a scientific mode of thinking. The change of thinking, from high school to university study, you will find that the learning method will change a lot, more importantly, learn to learn by yourself, the mathematical modeling competition will bring you a change, you will find that the mathematical modeling problem should be in the Solve it within a few days, forcing you to solve it. You may learn how to use a piece of software in a few days, and solve problems that you think may take ten and a half months to solve in a few days, which will greatly exercise your self-learning ability. .
Finally, let’s encourage each other with a sentence I heard from Dong Yuhui during my postgraduate entrance examination:
When you recite words, the cod in Alaska is jumping out of the water. When you are doing math, the seagulls on the other side of the Pacific Ocean flap their wings and fly over the city. When you are doing your evening self-study, the night sky in the polar circle is scattered and colorful, but boy, don’t worry. When you work hard, the scenery you feel you will never see, and the people you think you will never meet in your life are coming to you step by step.
Finally, the software installation package required during mathematical modeling is attached for free sharing:
Link: https://pan.baidu.com/s/18sFOxOUG9ikTlOSbxjA27w?pwd=8v6n
Extraction code: 8v6n
30+ commonly used model algorithm models + practical code cases
Link: https://pan.baidu.com/s/1Pg_PgPJ8-EJ0RMjZ6_dF3Q?pwd=fid3
Extraction code: fid3
Mathematical modeling from entry to actual combat pure dry goods experience sharing
Link: https://pan.baidu.com/s/19ZqTFYQZH5zUIpZowcR5Bg
Extraction code: gopt
Mathematics Modeling Writing and Typesetting Experience Sharing
Link: https://pan.baidu.com/s/10hJMmLw8pQ5r3csnnySH-g?pwd=msy3
Extraction code: msy3