Network strategy and calico
1. Network Strategy
Network Strategy Official Website
-
Preconditions The
network strategy is implemented through network plug-ins. To use network policy, you must use a network solution that supports NetworkPolicy. Creating a NetworkPolicy resource object without a controller to make it effective will have no effect. (Flannel does not support NetworkPolicy, so using the Flannei network plugin will not isolate pods) -
Isolated and non-isolated Pods
By default, Pods are non-isolated, and they accept traffic from any source.
Pod enters the quarantined state when selected by a NetworkPolicy. Once a NetworkPolicy selects a specific Pod in the namespace, the Pod will reject connections that are not allowed by the NetworkPolicy. (Other Pods in the namespace that are not selected by NetworkPolicy will continue to accept all traffic)
Network policies will not conflict, they are cumulative. If any one or more policies select a Pod, the Pod is subject to the union of the Ingress/Egress rules of these policies. Therefore, the order of evaluation does not affect the outcome of the strategy.
In order to allow network data flow between two Pods, both the Egress rule on the source Pod and the Ingress rule on the target Pod need to allow the traffic. If the egress rule at the source end or the ingress rule at the target end rejects the traffic, the traffic will be rejected.
2. Environmental settings
2.1 Remove the original flannel network plug-in to ensure the accuracy of the experiment (it does not need to be removed)
[root@server2 ~]# kubectl delete -f kube-flannel.yml ##删除fannel网络插件配置
[root@server2 ~]# kubectl -n kube-system get pod ##查看是否删除
[root@server2 ~]# kubectl get pod
NAME READY STATUS RESTARTS AGE
demo 1/1 Running 10 3d1h
deployment-6456d7c676-2s77n 1/1 Running 0 67m
deployment-6456d7c676-wx9tk 1/1 Running 0 67m
[root@server2 ~]# kubectl delete deployments.apps deployment
[root@server2 ~]# kubectl get svc
[root@server2 ~]# kubectl delete svc nginx-svc
[root@server2 ~]# kubectl get all
NAME READY STATUS RESTARTS AGE
pod/demo 1/1 Running 10 3d1h
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 4d21h
##移除fannel生成的配置文件
[root@server2 ~]# cd /etc/cni/
[root@server2 cni]# cd net.d/
[root@server2 net.d]# ls
10-flannel.conflist
[root@server2 net.d]# mv 10-flannel.conflist /mnt/ ##每一个节点都需要移除
[root@server3 net.d]# mv 10-flannel.conflist /mnt/
[root@server4 net.d]# mv 10-flannel.conflist /mnt/
[root@server2 net.d]# ps ax | grep gannel
2.2 Pull the required image in advance
##1. 拉取
[root@server1 harbor]# docker pull calico/kube-controllers:v3.16.1 ##拉取的镜像可以通过官网下载的calico.yaml文件来查看
[root@server1 harbor]# docker pull calico/cni:v3.16.1
[root@server1 harbor]# docker pull calico/pod2daemon-flexvol:v3.16.1
[root@server1 harbor]# docker pull calico/node:v3.16.1
## 2. 上传
[root@server1 harbor]# docker tag calico/node:v3.16.1 reg.westos.org/calico/node:v3.16.1
[root@server1 harbor]# docker push reg.westos.org/calico/node:v3.16.1
[root@server1 harbor]# docker tag calico/pod2daemon-flexvol:v3.16.1 reg.westos.org/calico/pod2daemon-flexvol:v3.16.1
[root@server1 harbor]# docker push reg.westos.org/calico/pod2daemon-flexvol:v3.16.1
[root@server1 harbor]# docker tag calico/cni:v3.16.1 reg.westos.org/calico/cni:v3.16.1
[root@server1 harbor]# docker push reg.westos.org/calico/cni:v3.16.1
[root@server1 harbor]# docker tag calico/kube-controllers:v3.16.1 reg.westos.org/calico/kube-controllers:v3.16.1
[root@server1 harbor]# docker push reg.westos.org/calico/kube-controllers:v3.16.1
3. Configure Calico network plug-in
3.1 Configure calico
- calico简介:
flannel实现的是网络通信,calico的特性是在pod之间的隔离(网络策略)。
通过BGP路由,但大规模端点的拓扑计算和收敛往往需要一定的时间和计算资源。
纯三层的转发,中间没有任何的NAT和overlay,转发效率最好。
Calico 仅依赖三层路由可达。Calico 较少的依赖性使它能适配所有 VM、Container、白盒或者混合环境场景。
- 安装calico:
# wget https://docs.projectcalico.org/manifests/calico.yaml ##通过官网可以查看(最新版本下载比较慢,所以我使用的是)
# vim calico.yml
- name: CALICO_IPV4POOL_IPIP ##IPIP表示隧道
value: "off"
# kubectl apply -f calico.yaml ##创建
[root@server2 ~]# mkdir calico
[root@server2 ~]# cd calico/
[root@server2 calico]# ls
calico.yaml deny-nginx.yaml
[root@server2 calico]# kubectl apply -f calico.yaml ##应用calico网络插件
[root@server2 calico]# kubectl -n kube-system get pod ##查看calico节点是否运行
[root@server2 calico]# cd /etc/cni/net.d/ ##查看calico插件配置,每个结点都有
[root@server2 net.d]# ls
10-calico.conflist calico-kubeconfig
3.2 Test
## 1.先生成两个标签为nginx的pod
[root@server2 calico]# cd ../ingress/
[root@server2 ingress]# ls
auth demo.yml deploy.yaml nginx-svc.yml nginx.yml tls.crt tls.key
[root@server2 ingress]# kubectl apply -f nginx-svc.yml
[root@server2 ingress]# kubectl get pod -L app
NAME READY STATUS RESTARTS AGE APP
demo 1/1 Running 10 3d1h
deployment-6456d7c676-plxjz 1/1 Running 0 17s nginx
deployment-6456d7c676-rfr85 1/1 Running 0 17s nginx
[root@server2 ingress]# kubectl get svc
[root@server2 ingress]# curl 10.111.89.37
## 2. 运行测试文件,查看是否隔离标签为nginx的pod(网络策略,隔离所有ngix标签的pod)
[root@server2 ~]# cd calico/
[root@server2 calico]# vim deny-nginx.yaml
[root@server2 calico]# cat deny-nginx.yaml # NetworkPolicy模式
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: deny-nginx
spec:
podSelector:
matchLabels:
app: nginx
[root@server2 calico]# kubectl apply -f deny-nginx.yaml ##运行
networkpolicy.networking.k8s.io/deny-nginx created
[root@server2 calico]# kubectl get networkpolicies.networking.k8s.io ##查看限制
NAME POD-SELECTOR AGE
deny-nginx app=nginx 20s
[root@server2 calico]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 4d21h
nginx-svc ClusterIP 10.111.89.37 <none> 80/TCP 5m1s
[root@server2 calico]# curl 10.111.89.37
## 1. First generate two pods labeled nginx
2. Run the test file to see if the pod labeled nginx is isolated
4. Examples of network policies
[root@server2 calico]# kubectl -n kube-system describe pod calico-node-6k72w ##出错可以查看详细信息
[root@server2 ~]# kubectl delete -f kube-flannel.yml ##或者出错查看日志
4.1 Restrict access to designated services
- 3.2 中的测试就是这个示例
4.2 Allow designated pods to access services
[root@server2 calico]# kubectl run nginx --image=myapp:v2 -it
[root@server2 calico]# kubectl get pod
NAME READY STATUS RESTARTS AGE
demo 1/1 Running 10 3d3h
deployment-6456d7c676-plxjz 1/1 Running 0 90m
deployment-6456d7c676-rfr85 1/1 Running 0 90m
nginx 1/1 Running 1 28s
[root@server2 calico]# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
demo 1/1 Running 10 3d3h 10.244.1.28 server3 <none> <none>
deployment-6456d7c676-plxjz 1/1 Running 0 90m 10.244.141.192 server3 <none> <none>
deployment-6456d7c676-rfr85 1/1 Running 0 90m 10.244.22.0 server4 <none> <none> ##标签为nginx的ip
nginx 1/1 Running 1 35s 10.244.141.193 server3 <none> <none>
[root@server2 calico]# vim access-demo.yaml
[root@server2 calico]# cat access-demo.yaml
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
name: access-nginx
spec:
podSelector:
matchLabels:
app: nginx
ingress:
- from:
- podSelector:
matchLabels:
app: demo
[root@server2 calico]# kubectl apply -f access-demo.yaml
networkpolicy.networking.k8s.io/access-nginx created
[root@server2 calico]# kubectl get networkpolicies.
NAME POD-SELECTOR AGE
access-nginx app=nginx 20s
deny-nginx app=nginx 87m
[root@server2 calico]# kubectl label pod demo app=demo ##设置标签
pod/demo labeled
[root@server2 calico]# kubectl get pod -L app
NAME READY STATUS RESTARTS AGE APP
demo 1/1 Running 11 3d3h demo
deployment-6456d7c676-plxjz 1/1 Running 0 93m nginx
deployment-6456d7c676-rfr85 1/1 Running 0 93m nginx
nginx 1/1 Running 1 3m37s
[root@server2 calico]# kubectl attach demo -it
Defaulting container name to demo.
Use 'kubectl describe pod/demo -n default' to see all of the containers in this pod.
If you don't see a command prompt, try pressing enter.
/ # ping 10.244.22.0
PING 10.244.22.0 (10.244.22.0): 56 data bytes
64 bytes from 10.244.22.0: seq=0 ttl=62 time=0.812 ms
^C
--- 10.244.22.0 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 0.812/0.812/0.812 ms
/ # curl 10.244.22.0
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
4.3 Prohibit mutual access between all Pods in the namespace
[root@server2 calico]# kubectl create namespace demo ##创建新的namespace实验
[root@server2 calico]# kubectl get ns
[root@server2 calico]# kubectl run demo1 --image=busyboxplus -it -n demo ##创建demo1
[root@server2 calico]# kubectl run demo2 --image=busyboxplus -it -n demo ##创建demo2
[root@server2 calico]# kubectl -n demo get pod
[root@server2 calico]# kubectl -n demo get pod --show-labels
[root@server2 calico]# vim deny-pod.yaml
[root@server2 calico]# cat deny-pod.yaml
[root@server2 calico]# cat deny-pod.yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: default-deny
namespace: demo
spec:
podSelector: {
}
[root@server2 calico]# kubectl apply -f deny-pod.yaml
[root@server2 calico]# kubectl get networkpolicies. -n demo
## 测试
[root@server2 calico]# kubectl get pod -o wide -n demo ##查看ip
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
demo1 0/1 Error 1 9m36s 10.244.22.2 server4 <none> <none>
demo2 1/1 Running 1 8m32s 10.244.141.194 server3 <none> <none>
[root@server2 calico]# kubectl attach demo1 -it -n demo ##测试demo1和demo2之间是否可以相互访问
Defaulting container name to demo1.
Use 'kubectl describe pod/demo1 -n demo' to see all of the containers in this pod.
If you don't see a command prompt, try pressing enter.
/ # curl 10.244.141.194
4.4 Forbid other namespaces to access the service
[root@server2 calico]# vim deny-ns.yaml
[root@server2 calico]# cat deny-ns.yaml
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
name: deny-namespace
spec:
podSelector:
matchLabels:
ingress:
- from:
- podSelector: {
}
[root@server2 calico]# kubectl apply -f deny-ns.yaml
[root@server2 calico]# kubectl attach demo2 -it -n demo
Defaulting container name to demo2.
Use 'kubectl describe pod/demo2 -n demo' to see all of the containers in this pod.
If you don't see a command prompt, try pressing enter.
/ # curl 10.244.22.0
4.5 Only allow the specified namespace to access the service
[root@server2 calico]# kubectl create namespace test #创建新的ns
[root@server2 calico]# kubectl run demo3 --image=busyboxplus -it -n test ##创建测试pod
[root@server2 calico]# vim access-ns.yaml
[root@server2 calico]# cat access-ns.yaml
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
name: access-namespace
spec:
podSelector:
matchLabels:
run: nginx ##指定标签
ingress:
- from:
- namespaceSelector:
matchLabels:
role: prod ##指定ns的角色名字
[root@server2 calico]# kubectl label ns test role=prod ##设置namespace标签
[root@server2 calico]# kubectl get ns --show-labels
[root@server2 calico]# kubectl apply -f access-ns.yaml ##访问指定标签的pod
[root@server2 calico]# kubectl attach demo3 -it -n test
/ # curl 10.244.141.193 ##标签为run=nginx的pod节点,且访问成功
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
4.6 Allow Internet access to services
[root@server2 ingress]# pwd
/root/ingress
[root@server2 ingress]# cat demo.yml
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ingress-demo
spec:
#tls:
# - hosts:
# - www1.westos.org
# secretName: tls-secret
rules:
- host: www1.westos.org
http:
paths:
- path: /
backend:
serviceName: nginx-svc
servicePort: 80
[root@server2 ingress]# kubectl apply -f demo.yml
[root@server2 ingress]# kubectl get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
ingress-demo <none> www1.westos.org 172.25.13.3,172.25.13.4 80 4h6m
[root@server2 calico]# vim access-ex.yaml
[root@server2 calico]# cat access-ex.yaml
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
name: web-allow-external
spec:
podSelector:
matchLabels:
app: nginx
ingress:
- ports:
- port: 80
from: []