机器学习基础（九）之PCA&梯度上升1

Jupyter Notebook
之前我们用梯度下降是求损失函数的最小值，现在我们用梯度上升是求目标函数的最大值，自己可以想一下啦

import numpy as np
from matplotlib import pyplot as plt

X = np.empty((100, 2))
X[:,0] = np.random.uniform(1.0, 100.0, size = 100)
X[:,1] = 0.75 * X[:,0] + 3.0 + np.random.uniform(1.0, 100.0, size = 100)

def demean(x):
    return x - np.mean(x, axis=0)

X_demean = demean(X)

np.mean(X_demean[:,0])
1.4921397450962103e-15
基本上是等于零啦，兄弟们
现在进行我们的梯度上升发的封装

def J(w, X):
    return np.sum(X.dot(w) ** 2)
用倒数的定义来求倒数，大学求导函数是否还记得？？？
def dJ_debug(w, X,epsilon = 0.0001):
    res = np.empty(len(w))
    for i in range(len(w)):
        w_1 = w.copy()
        w_1[i] += epsilon
        w_2 = w.copy()
        w_2[i] -= epsilon
        res[i] = (J(w_1[i], X) - J(w_2[i], X)) / 2 * epsilon
    return res

梯度上升主方法

def direction(w):
    return w / np.linalg.norm(w)

def gradient_ascent(dJ, X, init_w, step = 0.01, n_iters = 1e4, epsilon = 1e8):
    i_iters = 0
    w = direction(init_w)  #是个单位向量哦
    while i_iters < n_iters:
        gradient = dJ(w, X)
        last_w = w
        w = w + step * gradient
        w = direction(w)
        if (abs(J(w, X) - J(last_w, X)) < epsilon):
            break
        i_iters += 1
    return w
进行测试一下吧，兄弟们呀

init_w = np.random.random(X.shape[1])  #注意此时不能是零向量呀

gradient_ascent_w1 = gradient_ascent(dJ_debug, X, init_w)

gradient_ascent_w1
array([0.99853244, 0.05415694])
PCA不能进行数据归一化

gradient_ascent_w2 = gradient_ascent(dJ_math, X, init_w)

gradient_ascent_w2
array([0.53178727, 0.84687797])```
居然不一样。。。。




​