版权声明：本文为博主原创文章，未经博主允许不得转载。 https://blog.csdn.net/winycg/article/details/83064112
在线性分类任务中，我们通常使用线性分类函数来分类：
d
(
x
)
=
w
1
x
1
+
w
2
x
2
+
.
.
.
+
w
n
x
n
+
w
0
d(\mathbf{x})=w_{1}x_{1}+w_{2}x_{2}+...+w_{n}x_{n}+w_{0}
d ( x ) = w 1 x 1 + w 2 x 2 + . . . + w n x n + w 0
w
=
(
w
1
,
w
2
,
.
.
.
,
w
n
,
w
0
)
T
\mathbf{w}=(w_{1},w_{2},...,w_{n},w_{0})^{T}
w = ( w 1 , w 2 , . . . , w n , w 0 ) T
x
=
(
x
1
,
x
2
,
.
.
.
,
x
n
,
1
)
T
\mathbf{x}=(x_{1},x_{2},...,x_{n},1)^{T}
x = ( x 1 , x 2 , . . . , x n , 1 ) T
已知训练集可分为
ω
1
\omega_{1}
ω 1
ω
2
\omega_{2}
ω 2
{
w
T
x
i
>
0
,
x
i
∈
ω
1
w
T
x
i
⩽
0
,
x
i
∈
ω
2
\left\{\begin{matrix} \mathbf w^{T} \mathbf x_{i} >0, \mathbf x_{i} \in \omega_{1} \\ \mathbf w^{T} \mathbf x_{i} \leqslant 0, \mathbf x_{i} \in \omega_{2} \end{matrix}\right.
{ w T x i > 0 , x i ∈ ω 1 w T x i ⩽ 0 , x i ∈ ω 2
w
\mathbf w
w
x
∈
ω
1
,
w
T
x
i
⩽
0
\mathbf{x \in}\omega_{1},\mathbf w^{T} \mathbf x_{i} \leqslant 0
x ∈ ω 1 , w T x i ⩽ 0
x
i
\mathbf x_{i}
x i
w
(
t
+
1
)
=
w
(
t
)
+
η
x
i
\mathbf{w}(t+1)=\mathbf w (t)+\eta\mathbf x_{i}
w ( t + 1 ) = w ( t ) + η x i
η
\eta
η
w
(
t
)
−
(
−
η
x
i
)
=
w
(
t
+
1
)
\mathbf w (t)-(-\eta\mathbf x_{i})=\mathbf{w}(t+1)
w ( t ) − ( − η x i ) = w ( t + 1 )
x
\mathbf x
x
x
∈
ω
2
,
w
T
x
i
>
0
\mathbf{x \in}\omega_{2},\mathbf w^{T} \mathbf x_{i} > 0
x ∈ ω 2 , w T x i > 0
x
i
\mathbf x_{i}
x i
w
(
t
+
1
)
=
w
(
t
)
−
η
x
i
\mathbf{w}(t+1)=\mathbf w (t)-\eta\mathbf x_{i}
w ( t + 1 ) = w ( t ) − η x i
w
(
t
)
−
η
x
i
=
w
(
t
+
1
)
\mathbf w (t)-\eta\mathbf x_{i}=\mathbf{w}(t+1)
w ( t ) − η x i = w ( t + 1 )
x
\mathbf x
x
w
(
t
+
1
)
=
w
(
t
)
\mathbf w(t+1)=\mathbf w(t)
w ( t + 1 ) = w ( t ) 结论： 只要模式类别是可分的，那么该算法必定会收敛，即在有限的步数内得到权向量。
w
(
t
+
1
)
=
{
w
(
t
)
+
η
x
i
,
i
f
x
∈
ω
1
,
w
T
(
t
)
x
i
⩽
0
w
(
t
)
−
η
x
i
,
i
f
x
∈
ω
2
,
w
T
(
t
)
x
i
>
0
w
(
t
)
,
i
f
o
t
h
e
r
\mathbf w(t+1)=\left\{\begin{matrix} \mathbf w(t)+\eta \mathbf x_{i},if\ \mathbf{x \in}\omega_{1},\mathbf w^{T}(t) \mathbf x_{i} \leqslant 0\\ \mathbf w (t)-\eta\mathbf x_{i},if\ \mathbf{x \in}\omega_{2},\mathbf w^{T}(t) \mathbf x_{i} > 0\\ \mathbf w(t), if \ other \end{matrix}\right.
w ( t + 1 ) = ⎩ ⎨ ⎧ w ( t ) + η x i , i f x ∈ ω 1 , w T ( t ) x i ⩽ 0 w ( t ) − η x i , i f x ∈ ω 2 , w T ( t ) x i > 0 w ( t ) , i f o t h e r
x
i
∈
ω
2
\mathbf x_{i}\in \omega_{2}
x i ∈ ω 2
−
1
-1
− 1
w
T
(
−
x
i
)
<
0
\mathbf w^{T} (-\mathbf x_{i}) <0
w T ( − x i ) < 0
w
T
(
−
x
i
)
≥
0
\mathbf w^{T} (-\mathbf x_{i}) \geq 0
w T ( − x i ) ≥ 0
w
T
(
−
x
i
)
≤
0
\mathbf w^{T} (-\mathbf x_{i}) \leq 0
w T ( − x i ) ≤ 0
w
T
(
−
x
i
)
>
0
\mathbf w^{T} (-\mathbf x_{i}) >0
w T ( − x i ) > 0
w
T
x
≤
0
,
x
i
\mathbf {w^{T}x}\leq0,\mathbf x_{i}
w T x ≤ 0 , x i
ω
2
\omega_{2}
ω 2
w
T
x
<
0
,
x
i
\mathbf {w^{T}x}<0,\mathbf x_{i}
w T x < 0 , x i
ω
2
\omega_{2}
ω 2
w
(
t
+
1
)
=
w
(
t
)
+
η
(
−
x
i
)
\mathbf{w}(t+1)=\mathbf w (t)+\eta(-\mathbf x_{i})
w ( t + 1 ) = w ( t ) + η ( − x i )
∈
ω
2
\in \omega_{2}
∈ ω 2
−
1
-1
− 1
w
(
t
+
1
)
{
w
(
t
)
,
i
f
w
T
(
t
)
x
i
>
0
w
(
t
)
+
η
x
i
,
i
f
w
T
(
t
)
x
i
≤
0
\mathbf w(t+1)\left\{\begin{matrix} \mathbf w(t),\ if\ \mathbf w^{T}(t)\mathbf x_{i}>0\\ \mathbf w (t)+\eta\mathbf x_{i},\ if \ \mathbf w^{T}(t)\mathbf x_{i}\leq0 \end{matrix}\right.
w ( t + 1 ) { w ( t ) , i f w T ( t ) x i > 0 w ( t ) + η x i , i f w T ( t ) x i ≤ 0 例子：
(
0
,
0
)
,
(
0
,
1
)
∈
ω
1
(0,0),(0,1)\in \omega_{1}
( 0 , 0 ) , ( 0 , 1 ) ∈ ω 1
(
1
,
0
)
,
(
1
,
1
)
∈
ω
2
(1,0),(1,1)\in \omega_{2}
( 1 , 0 ) , ( 1 , 1 ) ∈ ω 2
d
(
x
)
=
−
2
x
1
+
1
d(\mathbf x)=-2x_{1}+1
d ( x ) = − 2 x 1 + 1
import numpy as np
x1 = [ np. array( [ 0 , 0 ] ) , np. array( [ 0 , 1 ] ) ]
x2 = [ np. array( [ 1 , 0 ] ) , np. array( [ 1 , 1 ] ) ]
for i in range ( len ( x1) ) :
x1[ i] = np. hstack( ( x1[ i] , [ 1 ] ) )
for i in range ( len ( x2) ) :
x2[ i] = np. hstack( ( x2[ i] , [ 1 ] ) )
x2[ i] = - x2[ i]
x = x1 + x2
w = [ 0 , 0 , 0 ]
count = 1
step = 1
while True :
print ( 'The step' + str ( step) )
flag = True
for i in range ( len ( x) ) :
if np. dot( w, x[ i] ) <= 0 :
flag = False
tmp_w = w
w = w + x[ i]
print ( 'w' + str ( count) + '^T*x' + str ( i) + '=' , str ( tmp_w) , str ( x[ i] ) + '^T<=0,' ,
'w' + str ( count+ 1 ) + '=w' + str ( count) + '+x' + str ( i) + '=' , str ( w) + '^T' )
else :
print ( 'w' + str ( count) + '^T*x' + str ( i) + '=' , w, str ( x[ i] ) + '^T>0,' ,
'w' + str ( count+ 1 ) + '=w' + str ( count) + '=' , str ( w) + '^T' )
count += 1
step = step + 1
if flag is True :
break
print ( 'Perception weight vector:' , w)
对
M
M
M
M
M
M
d
k
,
k
=
1
,
2
,
.
.
.
,
M
d_{k},k=1,2,...,M
d k , k = 1 , 2 , . . . , M
x
∈
ω
k
x\in \omega_{k}
x ∈ ω k
d
k
(
x
)
>
d
j
(
x
)
,
∀
j
≠
k
d_{k}(\mathbf x)>d_{j}(\mathbf x), \forall j\neq k
d k ( x ) > d j ( x ) , ∀ j ̸ = k
t
t
t
ω
k
\omega_{k}
ω k
d
j
(
x
)
=
w
j
T
(
t
)
x
,
j
=
1
,
2
,
.
.
.
,
M
d_{j}(\mathbf x)=\mathbf w_{j}^{T}(t)\mathbf x,j=1,2,...,M
d j ( x ) = w j T ( t ) x , j = 1 , 2 , . . . , M
d
k
(
x
)
>
d
j
(
x
)
,
∀
j
≠
k
d_{k}(\mathbf x)>d_{j}(\mathbf x), \forall j\neq k
d k ( x ) > d j ( x ) , ∀ j ̸ = k
w
j
(
t
+
1
)
=
w
j
(
t
)
,
j
=
1
,
2
,
.
.
.
,
M
\mathbf w_{j}(t+1)=\mathbf w_{j}(t),j=1,2,...,M
w j ( t + 1 ) = w j ( t ) , j = 1 , 2 , . . . , M
∃
j
≠
k
\exists j\neq k
∃ j ̸ = k
d
k
(
x
)
≤
d
j
(
x
)
d_{k}(\mathbf x)\leq d_{j}(\mathbf x)
d k ( x ) ≤ d j ( x )
{
w
k
(
t
+
1
)
=
w
k
(
t
)
+
η
x
i
,
此
时
可
以
使
得
w
k
(
t
)
向
w
k
(
t
)
x
i
增
大
的
方
向
调
整
w
j
(
t
+
1
)
=
w
j
(
t
)
−
η
x
i
,
此
时
可
以
使
得
w
j
(
t
)
向
w
j
(
t
)
x
i
减
小
的
方
向
调
整
\left\{\begin{matrix} \mathbf w_{k}(t+1)=\mathbf w_{k}(t)+\eta\mathbf x_{i},此时可以使得\mathbf w_{k}(t)向\mathbf w_{k}(t)\mathbf x_{i}增大的方向调整\\ \mathbf w_{j}(t+1)=\mathbf w_{j}(t)-\eta\mathbf x_{i},此时可以使得\mathbf w_{j}(t)向\mathbf w_{j}(t)\mathbf x_{i}减小的方向调整 \end{matrix}\right.
{ w k ( t + 1 ) = w k ( t ) + η x i , 此 时 可 以 使 得 w k ( t ) 向 w k ( t ) x i 增 大 的 方 向 调 整 w j ( t + 1 ) = w j ( t ) − η x i , 此 时 可 以 使 得 w j ( t ) 向 w j ( t ) x i 减 小 的 方 向 调 整
