二进制安装k8s - 0.7 node安装 kubelet、kube-proxy 、cni plugins

二进制安装k8s - 0.7 node安装 kubelet、kube-proxy 、cni plugins

---

创建 node 相关目录

mkdir -p /data/k8s/{
    
    kubelet,kube-proxy,cni,bin,cert}
mkdir -p /data/k8s/cni/net.d/

下载 kubelet,kube-proxy 二进制和基础 cni plugins

[root@node bin]# ls
bridge  host-local  kubelet  kube-proxy  loopback

从master拉取 ca 文件。

[root@node cert]# scp 192.168.100.59:/data/k8s/cert/{
    
    ca.pem,ca-key.pem,ca-config.json} /data/k8s/cert/
[root@node cert]# ls
ca-config.json  ca-key.pem  ca.pem

---

---

---

---

准备 cni配置文件

vim /data/k8s/cni/net.d/10-default.conf

{
    
    
	"name": "mynet",
   "cniVersion": "0.3.1",
	"type": "bridge",
	"bridge": "mynet0",
	"isDefaultGateway": true,
	"ipMasq": true,
	"hairpinMode": true,
	"ipam": {
    
    
		"type": "host-local",
		"subnet": "{
    
    { pod_cni }}"
	}
}

注： { { pod_cni }} 为pod所能用的网段 我这里设置为 10.244.0.0/16

---

---

开启 ipvs

cat > /etc/sysconfig/modules/ipvs.modules <<EOF
#!/bin/bash
modprobe -- ip_vs
modprobe -- ip_vs_rr
modprobe -- ip_vs_wrr
modprobe -- ip_vs_sh
modprobe -- nf_conntrack_ipv4
EOF
chmod 755 /etc/sysconfig/modules/ipvs.modules && bash /etc/sysconfig/modules/ipvs.modules && lsmod | grep -e ip_vs -e nf_conntrack_ipv4

---

---

---

---

kubelet 配置部分

下面直接在master生成 kubelet.kubeconfig 文件，再传到对应的 node 上。

准备kubelet 证书签名请求

master上操作
mkdir -p /data/k8s/node/100.60
vim /data/k8s/node/100.60/kubelet-csr.json

{
    
    
  "CN": "system:node:192.168.100.60",
  "hosts": [
    "127.0.0.1",
    "192.168.100.60",
    "node01"
  ],
  "key": {
    
    
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
    
    
      "C": "CN",
      "ST": "SiChuan",
      "L": "ChengDu",
      "O": "system:nodes",
      "OU": "Lswzw"
    }
  ]
}

注：

• 上面的ip须要换为node主机host。
• 证书里面须要添加 hostname 不然在调用 kubectl log 命令时提示，证书不允许节点。

创建 kubelet 证书与私钥

cd /data/k8s/node/100.60

cfssl gencert \
  -ca=/data/k8s/cert/ca.pem \
  -ca-key=/data/k8s/cert/ca-key.pem \
  -config=/data/k8s/cert/ca-config.json \
  -profile=kubernetes kubelet-csr.json | cfssljson -bare kubelet

创建kubelet.kubeconfig

kubectl config set-cluster kubernetes \
  --certificate-authority=/data/k8s/cert/ca.pem \
  --embed-certs=true \
  --server={
    
    {
    
     KUBE_APISERVER }} \
  --kubeconfig=kubelet.kubeconfig

注： { { KUBE_APISERVER }} 我这为： https://192.168.100.59:6443

设置客户端认证参数

kubectl config set-credentials system:node:{
    
    {
    
     node_ip }} \
  --client-certificate=/data/k8s/node/100.60/kubelet.pem \
  --embed-certs=true \
  --client-key=/data/k8s/node/100.60/kubelet-key.pem \
  --kubeconfig=kubelet.kubeconfig

注： { { node_ip }} 这里为要生成node的ip。 我这为 192.168.100.60

设置上下文参数

kubectl config set-context default \
  --cluster=kubernetes \
  --user=system:node:{
    
    {
    
     node_ip }} \
  --kubeconfig=kubelet.kubeconfig

注： { { node_ip }} 这里为要生成node的ip。 我这为 192.168.100.60

选择默认上下文

kubectl config use-context default \
  --kubeconfig=kubelet.kubeconfig

拷贝CA && kubelet.kubeconfig 到对应节点

scp kubelet.pem 192.168.100.60:/data/k8s/kubelet/
scp kubelet-key.pem 192.168.100.60:/data/k8s/kubelet/
scp kubelet.kubeconfig 192.168.100.60:/data/k8s/kubelet/

创建对应节点用户权限

这个很重要，不然无法创建pod
最后name 是根据 “设置客户端认证参数” 里面设置的参数
vim node60.yaml

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: basic-auth-binding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- apiGroup: rbac.authorization.k8s.io
  kind: User
  name: system:node:192.168.100.60

。。。下面为node节点操作。。。

创建kubelet的配置文件

vim /data/k8s/kubelet/kubelet-config.yaml

kind: KubeletConfiguration
apiVersion: kubelet.config.k8s.io/v1beta1
address: {
    
    {
    
     node_ip }}
authentication:
  anonymous:
    enabled: false
  webhook:
    cacheTTL: 2m0s
    enabled: true
  x509:
    clientCAFile: /data/k8s/cert/ca.pem
authorization:
  mode: Webhook
  webhook:
    cacheAuthorizedTTL: 5m0s
    cacheUnauthorizedTTL: 30s
cgroupDriver: cgroupfs
cgroupsPerQOS: true
clusterDNS:
- 10.44.0.2
clusterDomain: cluster.local.
configMapAndSecretChangeDetectionStrategy: Watch
containerLogMaxFiles: 3 
containerLogMaxSize: 10Mi
enforceNodeAllocatable:
- pods
- kube-reserved
eventBurst: 10
eventRecordQPS: 5
evictionHard:
  imagefs.available: 15%
  memory.available: 200Mi
  nodefs.available: 10%
  nodefs.inodesFree: 5%
evictionPressureTransitionPeriod: 5m0s
failSwapOn: true
fileCheckFrequency: 40s
hairpinMode: hairpin-veth 
healthzBindAddress: {
    
    {
    
     node_ip }}
healthzPort: 10248
httpCheckFrequency: 40s
imageGCHighThresholdPercent: 85
imageGCLowThresholdPercent: 80
imageMinimumGCAge: 2m0s
kubeReservedCgroup: /system.slice/kubelet.service
kubeReserved: {
    
    'cpu':'200m','memory':'500Mi','ephemeral-storage':'1Gi'}
kubeAPIBurst: 100
kubeAPIQPS: 50
makeIPTablesUtilChains: true
maxOpenFiles: 1000000
maxPods: 110
nodeLeaseDurationSeconds: 40
nodeStatusReportFrequency: 1m0s
nodeStatusUpdateFrequency: 10s
oomScoreAdj: -999
podPidsLimit: -1
port: 10250
# disable readOnlyPort 
readOnlyPort: 0
resolvConf: /etc/resolv.conf
runtimeRequestTimeout: 2m0s
serializeImagePulls: true
streamingConnectionIdleTimeout: 4h0m0s
syncFrequency: 1m0s
tlsCertFile: /data/k8s/kubelet/kubelet.pem
tlsPrivateKeyFile: /data/k8s/kubelet/kubelet-key.pem

注： { { node_ip }} 为当前节点的ip。 这里为 192.168.100.60

创建kubelet的 systemd 文件

vim /etc/systemd/system/kubelet.service

[Unit]
Description=Kubernetes Kubelet
Documentation=https://github.com/GoogleCloudPlatform/kubernetes

[Service]
WorkingDirectory=/data/k8s/kubelet
ExecStartPre=/bin/mount -o remount,rw '/sys/fs/cgroup'
ExecStartPre=/bin/mkdir -p /sys/fs/cgroup/cpuset/system.slice/kubelet.service
ExecStartPre=/bin/mkdir -p /sys/fs/cgroup/hugetlb/system.slice/kubelet.service
ExecStartPre=/bin/mkdir -p /sys/fs/cgroup/memory/system.slice/kubelet.service
ExecStartPre=/bin/mkdir -p /sys/fs/cgroup/pids/system.slice/kubelet.service
ExecStart=/data/k8s/bin/kubelet \
  --config=/data/k8s/kubelet/kubelet-config.yaml \
  --cni-bin-dir=/data/k8s/bin \
  --cni-conf-dir=/data/k8s/cni/net.d \
  --hostname-override={
    
    {
    
     node_name }} \
  --kubeconfig=/data/k8s/kubelet/kubelet.kubeconfig \
  --network-plugin=cni \
  --pod-infra-container-image=registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.2 \
  --root-dir=/data/k8s/kubelet \
  --v=2
Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target

注： { { node_name }} 为当get node 时显示的name。 这里为 node01

启用kubelet 服务

systemctl start kubelet
systemctl status kubelet
systemctl enable kubelet

---

---

---

---

kube-proxy配置部分

从master上拉取kube-proxy.kubeconfig 文件

这个文件在03里面已经生成

scp 192.168.100.59:/data/k8s/conf/kube-proxy.kubeconfig /data/k8s/kube-proxy/

创建kube-proxy 的systemd 文件

vim /etc/systemd/system/kube-proxy.service

[Unit]
Description=Kubernetes Kube-Proxy Server
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=network.target

[Service]
WorkingDirectory=/data/k8s/kube-proxy
ExecStart=/data/k8s/bin/kube-proxy \
  --bind-address={
    
    {
    
     node_ip }} \
  --cluster-cidr=10.244.0.0/16 \
  --hostname-override={
    
    {
    
     node_name }} \
  --kubeconfig=/data/k8s/kube-proxy/kube-proxy.kubeconfig \
  --logtostderr=true \
  --proxy-mode=ipvs
Restart=always
RestartSec=5
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

注：

• kube-proxy 根据 --cluster-cidr 判断集群内部和外部流量，指定 --cluster-cidr 或 --masquerade-all 选项后，kube-proxy 会对访问 Service IP 的请求做 SNAT
• { { node_ip }} 为node主机ip 这里我的是 192.168.100.60
• { { node_name }} 为显示的node节点名 我这里是 node01

启用kube-proxy 服务

systemctl start kube-proxy
systemctl status kube-proxy
systemctl enable kube-proxy

---

---

---

---

master 上验证服务

[root@master conf]# kubectl get node
NAME     STATUS   ROLES    AGE   VERSION
node01   Ready    <none>   24m   v1.15.6