版权声明：本文为博主原创文章，未经博主允许不得转载。 https://blog.csdn.net/java_zyq/article/details/82179107
Ingress是个什么鬼，网上资料很多（推荐官方），大家自行研究。简单来讲，就是一个负载均衡的玩意，其主要用来解决使用NodePort暴露Service的端口时Node IP会漂移的问题。同时，若大量使用NodePort暴露主机端口，管理会非常混乱。
好的解决方案就是让外界通过域名去访问Service，而无需关心其Node IP及Port。那为什么不直接使用Nginx？这是因为在K8S集群中，如果每加入一个服务，我们都在Nginx中添加一个配置，其实是一个重复性的体力活，只要是重复性的体力活，我们都应该通过技术将它干掉。
Ingress就可以解决上面的问题，其包含两个组件Ingress Controller和Ingress：
- Ingress
将Nginx的配置抽象成一个Ingress对象，每添加一个新的服务只需写一个新的Ingress的yaml文件即可 - Ingress Controller
将新加入的Ingress转化成Nginx的配置文件并使之生效
好了，废话不多，走你~
准备操作
官方文档
人生苦短，不造轮子，本文将以官方的标准脚本为基础进行搭建，参考请戳官方文档。官方文档中要求依次执行如下命令：
curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/namespace.yaml \
| kubectl apply -f -
curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/default-backend.yaml \
| kubectl apply -f -
curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/configmap.yaml \
| kubectl apply -f -
curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/tcp-services-configmap.yaml \
| kubectl apply -f -
curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/udp-services-configmap.yaml \
| kubectl apply -f -
以上yaml文件创建Ingress用到的Namespace、ConfigMap，以及默认的后端default-backend。最关键的一点是，由于之前我们基于Kubeadm创建了K8S集群，则还必须执行：
curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/rbac.yaml \
| kubectl apply -f -
curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/with-rbac.yaml \
| kubectl apply -f -
这是由于Kubeadm创建的集群默认开启了RABC，因此Ingress也必须创建相应的RABC权限控制。
导入镜像
但是，直接按照上述方式执行，我们的Ingress很可能会无法使用。所以，我们需要将上述Yaml文件全部wget下来，经过一些修改后才能执行kubectl apply -f创建。另外需要注意的是，这些yaml文件中提到的一些镜像，国内目前无法下载，如：
gcr.io/google_containers/defaultbackend:1.4
quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.14.0
本人已经提前下载好，大家请戳：
地址：https://pan.baidu.com/s/1N-bK9hI7JTZZB6AzmaT8PA
密码：1a8a
拿到镜像后，在每个节点上执行如下命令导入镜像：
docker load < quay.io#kubernetes-ingress-controller#nginx-ingress-controller_0.14.0.tar
docker tag 452a96d81c30 quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.14.0
docker load < gcr.io#google_containers#defaultbackend.tar
docker tag 452a96d81c30 gcr.io/google_containers/defaultbackend
如上所示，导入镜像后，别忘记给打tag，否则镜像名称为<none>：
image.png
主要文件介绍
这里，我们先对一些重要的文件进行简单介绍。
default-backend.yaml
default-backend的作用是，如果外界访问的域名不存在的话，则默认转发到default-http-backend这个Service，其会直接返回404：
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: default-http-backend
labels:
app: default-http-backend
namespace: ingress-nginx
spec:
replicas: 1
selector:
matchLabels:
app: default-http-backend
template:
metadata:
labels:
app: default-http-backend
spec:
terminationGracePeriodSeconds: 60
containers:
- name: default-http-backend
# Any image is permissible as long as:
# 1. It serves a 404 page at /
# 2. It serves 200 on a /healthz endpoint
image: gcr.io/google_containers/defaultbackend:1.4
livenessProbe:
httpGet:
path: /healthz
port: 8080
scheme: HTTP
initialDelaySeconds: 30
timeoutSeconds: 5
ports:
- containerPort: 8080
resources:
limits:
cpu: 10m
memory: 20Mi
requests:
cpu: 10m
memory: 20Mi
---
apiVersion: v1
kind: Service
metadata:
name: default-http-backend
namespace: ingress-nginx
labels:
app: default-http-backend
spec:
ports:
- port: 80
targetPort: 8080
selector:
app: default-http-backend
rbac.yaml
rbac.yaml负责Ingress的RBAC授权的控制，其创建了Ingress用到的ServiceAccount、ClusterRole、Role、RoleBinding、ClusterRoleBinding。在上文《从零开始搭建Kubernetes集群（四、搭建K8S Dashboard）》中，我们已对这些概念进行了简单介绍。
apiVersion: v1
kind: ServiceAccount
metadata:
name: nginx-ingress-serviceaccount
namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
name: nginx-ingress-clusterrole
rules:
- apiGroups:
- ""
resources:
- configmaps
- endpoints
- nodes
- pods
- secrets
verbs:
- list
- watch
- apiGroups:
- ""
resources:
- nodes
verbs:
- get
- apiGroups:
- ""
resources:
- services
verbs:
- get
- list
- watch
- apiGroups:
- "extensions"
resources:
- ingresses
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- events
verbs:
- create
- patch
- apiGroups:
- "extensions"
resources:
- ingresses/status
verbs:
- update
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: Role
metadata:
name: nginx-ingress-role
namespace: ingress-nginx
rules:
- apiGroups:
- ""
resources:
- configmaps
- pods
- secrets
- namespaces
verbs:
- get
- apiGroups:
- ""
resources:
- configmaps
resourceNames:
# Defaults to "<election-id>-<ingress-class>"
# Here: "<ingress-controller-leader>-<nginx>"
# This has to be adapted if you change either parameter
# when launching the nginx-ingress-controller.
- "ingress-controller-leader-nginx"
verbs:
- get
- update
- apiGroups:
- ""
resources:
- configmaps
verbs:
- create
- apiGroups:
- ""
resources:
- endpoints
verbs:
- get
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
name: nginx-ingress-role-nisa-binding
namespace: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: nginx-ingress-role
subjects:
- kind: ServiceAccount
name: nginx-ingress-serviceaccount
namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: nginx-ingress-clusterrole-nisa-binding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: nginx-ingress-clusterrole
subjects:
- kind: ServiceAccount
name: nginx-ingress-serviceaccount
namespace: ingress-nginx
with-rbac.yaml
with-rbac.yaml是Ingress的核心，用于创建ingress-controller。前面提到过，ingress-controller的作用是将新加入的Ingress进行转化为Nginx的配置。
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: nginx-ingress-controller
namespace: ingress-nginx
spec:
replicas: 1
selector:
matchLabels:
app: ingress-nginx
template:
metadata:
labels:
app: ingress-nginx
annotations:
prometheus.io/port: '10254'
prometheus.io/scrape: 'true'
spec:
serviceAccountName: nginx-ingress-serviceaccount
containers:
- name: nginx-ingress-controller
image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.14.0
args:
- /nginx-ingress-controller
- --default-backend-service=$(POD_NAMESPACE)/default-http-backend
- --configmap=$(POD_NAMESPACE)/nginx-configuration
- --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
- --udp-services-configmap=$(POD_NAMESPACE)/udp-services
- --annotations-prefix=nginx.ingress.kubernetes.io
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
ports:
- name: http
containerPort: 80
- name: https
containerPort: 443
livenessProbe:
failureThreshold: 3
httpGet:
path: /healthz
port: 10254
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
readinessProbe:
failureThreshold: 3
httpGet:
path: /healthz
port: 10254
scheme: HTTP
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
securityContext:
runAsNonRoot: false
如上，可以看到nginx-ingress-controller启动时传入了参数，分别为前面创建的default-backend-service以及configmap。
创建Ingress
1.创建Ingress-controller
需要注意的是，官方提供的with-rbac.yaml文件不能直接使用，我们必须修改两处：
加入hostNetwork配置
如下，在serviceAccountName上方添加hostNetwork: true:
spec:
hostNetwork: true
serviceAccountName: nginx-ingress-serviceaccount
containers:
- name: nginx-ingress-controller
image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.14.0
args:
- /nginx-ingress-controller
- --default-backend-service=$(POD_NAMESPACE)/default-http-backend
- --configmap=$(POD_NAMESPACE)/nginx-configuration
- --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
- --udp-services-configmap=$(POD_NAMESPACE)/udp-services
- --annotations-prefix=nginx.ingress.kubernetes.io
配置hostNetwork: true是一种直接定义Pod网络的方式。定义后，Ingress-controller的IP就与宿主机k8s-node1一样(192.168.56.101)，并且端口80也是宿主机上的端口。这样，我们通过该192.168.56.101:80，就可以直接访问到Ingress-controller（实际上就是nginx），然后Ingress-controller则会转发我们的请求到相应后端。
加入环境变量
在其env部分加入如下环境变量：
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: KUBERNETES_MASTER
value: http://192.168.56.101:8080
否则，创建后会提示如下错误：
[root@k8s-node1 ingress]# kubectl describe pod nginx-ingress-controller-9fbd7596d-rt9sf -n ingress-nginx
省略前面...
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 30s default-scheduler Successfully assigned nginx-ingress-controller-9fbd7596d-rt9sf to k8s-node1
Normal SuccessfulMountVolume 30s kubelet, k8s-node1 MountVolume.SetUp succeeded for volume "nginx-ingress-serviceaccount-token-lq2dt"
Warning BackOff 21s kubelet, k8s-node1 Back-off restarting failed container
Normal Pulled 11s (x3 over 29s) kubelet, k8s-node1 Container image "quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.14.0" already present on machine
Normal Created 11s (x3 over 29s) kubelet, k8s-node1 Created container
Warning Failed 10s (x3 over 28s) kubelet, k8s-node1 Error: failed to start container "nginx-ingress-controller": Error response from daemon: OCI runtime create failed: container_linux.go:348: starting container process caused "exec: \"/nginx-ingress-controller\": stat /nginx-ingress-controller: no such file or directory": unknown
修改with-rbac.yaml后，使用kubectl -f create命令分别执行如下yaml文件，即可创建Ingress-controller：
创建成功后如下所示：
[root@k8s-node1 ingress]# kubectl get pod -n ingress-nginx -o wide
NAME READY STATUS RESTARTS AGE IP NODE
default-http-backend-5c6d95c48-pdjn9 1/1 Running 0 23s 192.168.36.81 k8s-node1
nginx-ingress-controller-547cd7d9cb-jmvpn 1/1 Running 0 8s 192.168.36.82 k8s-node1
2.创建自定义Ingress
有了ingress-controller，我们就可以创建自定义的Ingress了。这里已提前搭建好了Kibana服务，我们针对Kibana创建一个Ingress：
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: kibana-ingress
namespace: default
spec:
rules:
- host: myk8s.com
http:
paths:
- path: /
backend:
serviceName: kibana
servicePort: 5601
其中：
- rules中的host必须为域名，不能为IP，表示Ingress-controller的Pod所在主机域名，也就是Ingress-controller的IP对应的域名。
- paths中的path则表示映射的路径。如映射
/表示若访问myk8s.com，则会将请求转发至Kibana的service，端口为5601。
创建成功后，查看：
[root@k8s-node1 ingress]# kubectl get ingress -o wide
NAME HOSTS ADDRESS PORTS AGE
kibana-ingress myk8s.com 80 6s
我们再执行kubectl exec nginx-ingress-controller-5b79cbb5c6-2zr7f -it cat /etc/nginx/nginx.conf -n ingress-nginx，可以看到生成nginx配置，篇幅较长，各位自行筛选：
## start server myk8s.com
server {
server_name myk8s.com ;
listen 80;
listen [::]:80;
set $proxy_upstream_name "-";
location /kibana {
log_by_lua_block {
}
port_in_redirect off;
set $proxy_upstream_name "";
set $namespace "kube-system";
set $ingress_name "dashboard-ingress";
set $service_name "kibana";
client_max_body_size "1m";
proxy_set_header Host $best_http_host;
# Pass the extracted client certificate to the backend
# Allow websocket connections
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
proxy_set_header X-Real-IP $the_real_ip;
proxy_set_header X-Forwarded-For $the_real_ip;
proxy_set_header X-Forwarded-Host $best_http_host;
proxy_set_header X-Forwarded-Port $pass_port;
proxy_set_header X-Forwarded-Proto $pass_access_scheme;
proxy_set_header X-Original-URI $request_uri;
proxy_set_header X-Scheme $pass_access_scheme;
# Pass the original X-Forwarded-For
proxy_set_header X-Original-Forwarded-For $http_x_forwarded_for;
# mitigate HTTPoxy Vulnerability
# https://www.nginx.com/blog/mitigating-the-httpoxy-vulnerability-with-nginx/
proxy_set_header Proxy "";
# Custom headers to proxied server
proxy_connect_timeout 5s;
proxy_send_timeout 60s;
proxy_read_timeout 60s;
proxy_buffering "off";
proxy_buffer_size "4k";
proxy_buffers 4 "4k";
proxy_request_buffering "on";
proxy_http_version 1.1;
proxy_cookie_domain off;
proxy_cookie_path off;
# In case of errors try the next upstream server before returning an error
proxy_next_upstream error timeout invalid_header http_502 http_503 http_504;
proxy_next_upstream_tries 0;
# No endpoints available for the request
return 503;
}
location / {
log_by_lua_block {
}
port_in_redirect off;
set $proxy_upstream_name "";
set $namespace "default";
set $ingress_name "kibana-ingress";
set $service_name "kibana";
client_max_body_size "1m";
proxy_set_header Host $best_http_host;
# Pass the extracted client certificate to the backend
# Allow websocket connections
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
proxy_set_header X-Real-IP $the_real_ip;
proxy_set_header X-Forwarded-For $the_real_ip;
proxy_set_header X-Forwarded-Host $best_http_host;
proxy_set_header X-Forwarded-Port $pass_port;
proxy_set_header X-Forwarded-Proto $pass_access_scheme;
proxy_set_header X-Original-URI $request_uri;
proxy_set_header X-Scheme $pass_access_scheme;
# Pass the original X-Forwarded-For
proxy_set_header X-Original-Forwarded-For $http_x_forwarded_for;
# mitigate HTTPoxy Vulnerability
# https://www.nginx.com/blog/mitigating-the-httpoxy-vulnerability-with-nginx/
proxy_set_header Proxy "";
# Custom headers to proxied server
proxy_connect_timeout 5s;
proxy_send_timeout 60s;
proxy_read_timeout 60s;
proxy_buffering "off";
proxy_buffer_size "4k";
proxy_buffers 4 "4k";
proxy_request_buffering "on";
proxy_http_version 1.1;
proxy_cookie_domain off;
proxy_cookie_path off;
# In case of errors try the next upstream server before returning an error
proxy_next_upstream error timeout invalid_header http_502 http_503 http_504;
proxy_next_upstream_tries 0;
# No endpoints available for the request
return 503;
}
}
## end server myk8s.com
3.设置host
首先，我们需要在Ingress-controller的Pod所在主机上（这里为k8s-node1），将上面提到的域名myk8s.com追加入/etc/hosts文件：
192.168.56.101 myk8s.com
除此之外，如果想在自己的Windows物理机上使用浏览器访问kibana，也需要在C:\Windows\System32\drivers\etc\hosts文件内加入上述内容。设置后，分别在k8s-node1和物理机上测试无误即可：
测试
在Windows物理机上，使用Chrome访问myk8s.com，也就是相当于访问了192.168.56.101:80：
随意访问一个错误的地址myk8s.com/abc，返回预期的404：
