kubernetes分布式安装部署-简介

其他 2018-11-16 13:43:26 阅读次数: 0
版权声明：原创作品转载请经过作者的准许！ https://blog.csdn.net/qq_28470425/article/details/82842989

环境说明：3台1C1G（我自己这样的配置，有条件的可以调整大点）虚拟机且操作是系统centos7系统，三台虚拟机IP地址为：

1）192.168.56.11 主机名字：linux-node1.example.com

2）192.168.56.12 主机名字：linux-node2.example.com

3）192.168.56.13 主机名字：linux-node3.example.com

开始安装操作系统前：net.ifnames=0 biosdevname=0（目的是安装好的操作系统网卡名字为：eth0）

设置网络设置：
 vi/etc/sysconfig/network-scripts/ifcfg-eth0

Systemctl disable firewalld

Systemctl disable networkmanager

Vi /etc/hostnames名字改成：linux-node1.example.com 、linux-node2.example.com、linux-node3.example.com

Vi/etc/selinux/config  内容改成：disable

更改dns：vi/etc/resol

扫描二维码关注公众号，回复： 4098546 查看本文章

nameserver 192.168.56.2 

免密钥登录（三台虚拟机做互信）：

ssh-keygen –t rsa

Ssh-copy-id linux-node1

 

系统环境初始化

1.安装Docker

第一步：使用国内Docker源

[root@linux-node1 ~]# cd /etc/yum.repos.d/

[root@linux-node1 yum.repos.d]# wget \

 https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

第二步：Docker安装：

[root@linux-node1 ~]# yum install -y docker-ce

第三步：启动后台进程：

[root@linux-node1 ~]# systemctl start docker

2.准备部署目录

    mkdir -p /opt/kubernetes/{cfg,bin,ssl,log}

3.准备软件包

百度网盘下载地址：

[https://pan.baidu.com/s/1zs8sCouDeCQJ9lghH1BPiw](https://pan.baidu.com/s/1zs8sCouDeCQJ9lghH1BPiw)

4.解压软件包

 # tar zxf kubernetes.tar.gz

 # tar zxf kubernetes-server-linux-amd64.tar.gz

 # tar zxf kubernetes-client-linux-amd64.tar.gz

 # tar zxf kubernetes-node-linux-amd64.tar.gz

手动制作CA证书

1.安装 CFSSL

[root@linux-node1 ~]# cd /usr/local/src

[root@linux-node1 src]# wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64

[root@linux-node1 src]# wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64

[root@linux-node1 src]# wget https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64

[root@linux-node1 src]# chmod +x cfssl*

[root@linux-node1 src]# mv cfssl-certinfo_linux-amd64 /opt/kubernetes/bin/cfssl-certinfo

[root@linux-node1 src]# mv cfssljson_linux-amd64  /opt/kubernetes/bin/cfssljson

[root@linux-node1 src]# mv cfssl_linux-amd64  /opt/kubernetes/bin/cfssl

复制cfssl命令文件到k8s-node1和k8s-node2节点。如果实际中多个节点，就都需要同步复制。

[root@linux-node1 ~]# scp /opt/kubernetes/bin/cfssl* 192.168.56.12: /opt/kubernetes/bin

[root@linux-node1 ~]# scp /opt/kubernetes/bin/cfssl* 192.168.56.13: /opt/kubernetes/bin

2.初始化cfssl

[root@linux-node1 src]# mkdir ssl && cd ssl

[root@linux-node1 ssl]# cfssl print-defaults config > config.json

[root@linux-node1 ssl]# cfssl print-defaults csr > csr.json

3.创建用来生成 CA 文件的 JSON 配置文件

[root@linux-node1 ssl]# vim ca-config.json

{
  "signing": {
    "default": {
      "expiry": "8760h"
    },
    "profiles": {
      "kubernetes": {
        "usages": [
            "signing",
            "key encipherment",
            "server auth",
            "client auth"
        ],

        "expiry": "8760h"
      }
    }
  }
}

4.创建用来生成 CA 证书签名请求（CSR）的 JSON 配置文件

[root@linux-node1 ssl]# vim ca-csr.json

{"CN": "kubernetes",
  "key": { "algo": "rsa",
    "size": 2048 },
"names": [ {
      "C": "CN",
      "ST": "BeiJing",
      "L": "BeiJing",
      "O": "k8s",
      "OU": "System"
    }
  ]

}

5.生成CA证书（ca.pem）和密钥（ca-key.pem）

[root@ linux-node1 ssl]# cfssl gencert -initca ca-csr.json | cfssljson -bare ca

[root@ linux-node1 ssl]# ls -l ca*

-rw-r--r-- 1 root root  290 Mar  4 13:45 ca-config.json

-rw-r--r-- 1 root root 1001 Mar  4 14:09 ca.csr

-rw-r--r-- 1 root root  208 Mar  4 13:51 ca-csr.json

-rw------- 1 root root 1679 Mar  4 14:09 ca-key.pem

-rw-r--r-- 1 root root 1359 Mar  4 14:09 ca.pem

6.分发证书

# cp ca.csr ca.pem ca-key.pem ca-config.json /opt/kubernetes/ssl

SCP证书到k8s-node1和k8s-node2节点

# scp ca.csr ca.pem ca-key.pem ca-config.json 192.168.56.12:/opt/kubernetes/ssl

# scp ca.csr ca.pem ca-key.pem ca-config.json 192.168.56.13:/opt/kubernetes/ssl

手动部署ETCD集群

0.准备etcd软件包

wget https://github.com/coreos/etcd/releases/download/v3.2.18/etcd-v3.2.18-linux-amd64.tar.gz

[root@linux-node1 src]# tar zxf etcd-v3.2.18-linux-amd64.tar.gz

[root@linux-node1 src]# cd etcd-v3.2.18-linux-amd64

[root@linux-node1 etcd-v3.2.18-linux-amd64]# cp etcd etcdctl /opt/kubernetes/bin/

[root@linux-node1 etcd-v3.2.18-linux-amd64]# scp etcd etcdctl 192.168.56.12:/opt/kubernetes/bin/

[root@linux-node1 etcd-v3.2.18-linux-amd64]# scp etcd etcdctl 192.168.56.13:/opt/kubernetes/bin/

1.创建 etcd 证书签名请求：

 [root@linux-node1 ~]# vim etcd-csr.json

{
  "CN": "etcd",
  "hosts": [
    "127.0.0.1",
"192.168.56.11",
"192.168.56.12",
"192.168.56.13"
  ],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "BeiJing",
      "L": "BeiJing",
      "O": "k8s",
      "OU": "System"
    }
  ]
}

2.生成 etcd 证书和私钥：

[root@linux-node1 ~]# cfssl gencert -ca=/opt/kubernetes/ssl/ca.pem \

  -ca-key=/opt/kubernetes/ssl/ca-key.pem \

  -config=/opt/kubernetes/ssl/ca-config.json \

  -profile=kubernetes etcd-csr.json | cfssljson -bare etcd会生成以下证书文件

[root@k8s-master ~]# ls -l etcd*

-rw-r--r-- 1 root root 1045 Mar  5 11:27 etcd.csr

-rw-r--r-- 1 root root  257 Mar  5 11:25 etcd-csr.json

-rw------- 1 root root 1679 Mar  5 11:27 etcd-key.pem

-rw-r--r-- 1 root root 1419 Mar  5 11:27 etcd.pem

3.将证书移动到/opt/kubernetes/ssl目录下

[root@k8s-master ~]# cp etcd*.pem /opt/kubernetes/ssl

[root@linux-node1 ~]# scp etcd*.pem 192.168.56.12:/opt/kubernetes/ssl

[root@linux-node1 ~]# scp etcd*.pem 192.168.56.13:/opt/kubernetes/ssl

[root@k8s-master ~]# rm -f etcd.csr etcd-csr.json

4.设置ETCD配置文件

[root@linux-node1 ~]# vim /opt/kubernetes/cfg/etcd.conf

#[member]
ETCD_NAME="etcd-node1"
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
#ETCD_SNAPSHOT_COUNTER="10000"
#ETCD_HEARTBEAT_INTERVAL="100"
#ETCD_ELECTION_TIMEOUT="1000"
ETCD_LISTEN_PEER_URLS="https://192.168.56.11:2380"
ETCD_LISTEN_CLIENT_URLS="https://192.168.56.11:2379,https://127.0.0.1:2379"
#ETCD_MAX_SNAPSHOTS="5"
#ETCD_MAX_WALS="5"
#ETCD_CORS=""
#[cluster]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.56.11:2380"
# if you use different ETCD_NAME (e.g. test),
# set ETCD_INITIAL_CLUSTER value for this name, i.e. "test=http://..."
ETCD_INITIAL_CLUSTER="etcd-node1=https://192.168.56.11:2380,etcd-node2=https://192.168.56.12:2380,etcd-node3=https://192.168.56.13:2380"
ETCD_INITIAL_CLUSTER_STATE="new"
ETCD_INITIAL_CLUSTER_TOKEN="k8s-etcd-cluster"
ETCD_ADVERTISE_CLIENT_URLS="https://192.168.56.11:2379"
#[security]
CLIENT_CERT_AUTH="true"
ETCD_CA_FILE="/opt/kubernetes/ssl/ca.pem"
ETCD_CERT_FILE="/opt/kubernetes/ssl/etcd.pem"
ETCD_KEY_FILE="/opt/kubernetes/ssl/etcd-key.pem"
PEER_CLIENT_CERT_AUTH="true"
ETCD_PEER_CA_FILE="/opt/kubernetes/ssl/ca.pem"
ETCD_PEER_CERT_FILE="/opt/kubernetes/ssl/etcd.pem"
ETCD_PEER_KEY_FILE="/opt/kubernetes/ssl/etcd-key.pem"

5.创建ETCD系统服务

[root@linux-node1 ~]# vim /etc/systemd/system/etcd.service

[Unit]
Description=Etcd Server
After=network.target
[Service]
Type=simple
WorkingDirectory=/var/lib/etcd
EnvironmentFile=-/opt/kubernetes/cfg/etcd.conf
# set GOMAXPROCS to number of processors
ExecStart=/bin/bash -c "GOMAXPROCS=$(nproc) /opt/kubernetes/bin/etcd"
Type=notify
[Install]
WantedBy=multi-user.target

6.重新加载系统服务

[root@linux-node1 ~]# systemctl daemon-reload

[root@linux-node1 ~]# systemctl enable etcd

# scp /opt/kubernetes/cfg/etcd.conf 192.168.56.12:/opt/kubernetes/cfg/

# scp /etc/systemd/system/etcd.service 192.168.56.12:/etc/systemd/system/

# scp /opt/kubernetes/cfg/etcd.conf 192.168.56.13:/opt/kubernetes/cfg/

# scp /etc/systemd/system/etcd.service 192.168.56.13:/etc/systemd/system/

在所有节点上创建etcd

存储目录并启动etcd

[root@linux-node1 ~]# mkdir /var/lib/etcd

[root@linux-node1 ~]# systemctl start etcd

[root@linux-node1 ~]# systemctl status etcd

下面需要大家在所有的 etcd 节点重复上面的步骤，直到所有机器的 etcd 服务都已启动。

7.验证集群

[root@linux-node1 ~]# etcdctl --endpoints=https://192.168.56.11:2379 \

  --ca-file=/opt/kubernetes/ssl/ca.pem \

  --cert-file=/opt/kubernetes/ssl/etcd.pem \

  --key-file=/opt/kubernetes/ssl/etcd-key.pem cluster-health

member 435fb0a8da627a4c is healthy: got healthy result from https://192.168.56.12:2379

member 6566e06d7343e1bb is healthy: got healthy result from https://192.168.56.11:2379

member ce7b884e428b6c8c is healthy: got healthy result from https://192.168.56.13:2379

cluster is healthy

部署Kubernetes API服务部署

0.准备软件包

[root@linux-node1 ~]# cd /usr/local/src/kubernetes

[root@linux-node1 kubernetes]# cp server/bin/kube-apiserver /opt/kubernetes/bin/

[root@linux-node1 kubernetes]# cp server/bin/kube-controller-manager /opt/kubernetes/bin/

[root@linux-node1 kubernetes]# cp server/bin/kube-scheduler /opt/kubernetes/bin/

1.创建生成CSR的 JSON 配置文件

[root@linux-node1 src]# vim kubernetes-csr.json

{
  "CN": "kubernetes",
  "hosts": [
    "127.0.0.1",
    "192.168.56.11",
    "10.1.0.1",
    "kubernetes",
    "kubernetes.default",
    "kubernetes.default.svc",
    "kubernetes.default.svc.cluster",
    "kubernetes.default.svc.cluster.local"
  ],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "BeiJing",
      "L": "BeiJing",
      "O": "k8s",
      "OU": "System"
    }
  ]
}

2.生成 kubernetes 证书和私钥

 [root@linux-node1 src]# cfssl gencert -ca=/opt/kubernetes/ssl/ca.pem \

   -ca-key=/opt/kubernetes/ssl/ca-key.pem \

   -config=/opt/kubernetes/ssl/ca-config.json \

   -profile=kubernetes kubernetes-csr.json | cfssljson -bare kubernetes

[root@linux-node1 src]# cp kubernetes*.pem /opt/kubernetes/ssl/

[root@linux-node1 ~]# scp kubernetes*.pem 192.168.56.12:/opt/kubernetes/ssl/

[root@linux-node1 ~]# scp kubernetes*.pem 192.168.56.13:/opt/kubernetes/ssl/

3.创建 kube-apiserver 使用的客户端 token 文件

[root@linux-node1 ~]#  head -c 16 /dev/urandom | od -An -t x | tr -d ' '

ad6d5bb607a186796d8861557df0d17f

[root@linux-node1 ~]# vim /opt/kubernetes/ssl/ bootstrap-token.csv

ad6d5bb607a186796d8861557df0d17f,kubelet-bootstrap,10001,"system:kubelet-bootstrap"

4.创建基础用户名/密码认证配置

[root@linux-node1 ~]# vim /opt/kubernetes/ssl/basic-auth.csv

admin,admin,1

readonly,readonly,2

5.部署Kubernetes API Server

[root@linux-node1 ~]# vim /usr/lib/systemd/system/kube-apiserver.service

[Unit]
Description=Kubernetes API Server
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=network.target
[Service]
ExecStart=/opt/kubernetes/bin/kube-apiserver \
  --admission-control=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,ResourceQuota,NodeRestriction \  --bind-address=192.168.56.11 \
  --insecure-bind-address=127.0.0.1 \
  --authorization-mode=Node,RBAC \
  --runtime-config=rbac.authorization.k8s.io/v1 \
  --kubelet-https=true \
  --anonymous-auth=false \
  --basic-auth-file=/opt/kubernetes/ssl/basic-auth.csv \
  --enable-bootstrap-token-auth \
  --token-auth-file=/opt/kubernetes/ssl/bootstrap-token.csv \
  --service-cluster-ip-range=10.1.0.0/16 \
  --service-node-port-range=20000-40000 \
  --tls-cert-file=/opt/kubernetes/ssl/kubernetes.pem \
  --tls-private-key-file=/opt/kubernetes/ssl/kubernetes-key.pem \
  --client-ca-file=/opt/kubernetes/ssl/ca.pem \
  --service-account-key-file=/opt/kubernetes/ssl/ca-key.pem \
  --etcd-cafile=/opt/kubernetes/ssl/ca.pem \
  --etcd-certfile=/opt/kubernetes/ssl/kubernetes.pem \
  --etcd-keyfile=/opt/kubernetes/ssl/kubernetes-key.pem \
  --etcd-servers=https://192.168.56.11:2379,https://192.168.56.12:2379,https://192.168.56.13:2379 \
  --enable-swagger-ui=true \
  --allow-privileged=true \
  --audit-log-maxage=30 \
  --audit-log-maxbackup=3 \
  --audit-log-maxsize=100 \
  --audit-log-path=/opt/kubernetes/log/api-audit.log \
  --event-ttl=1h \
  --v=2 \
  --logtostderr=false \
  --log-dir=/opt/kubernetes/log
Restart=on-failure
RestartSec=5
Type=notify
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target

6.启动API Server服务

[root@linux-node1 ~]# systemctl daemon-reload

[root@linux-node1 ~]# systemctl enable kube-apiserver

[root@linux-node1 ~]# systemctl start kube-apiserver

查看API Server服务状态

[root@linux-node1 ~]# systemctl status kube-apiserver

部署Controller Manager服务

[root@linux-node1 ~]# vim /usr/lib/systemd/system/kube-controller-manager.service

[Unit]

Description=Kubernetes Controller Manager

Documentation=https://github.com/GoogleCloudPlatform/kubernetes

[Service]

ExecStart=/opt/kubernetes/bin/kube-controller-manager \

  --address=127.0.0.1 \

  --master=http://127.0.0.1:8080 \

  --allocate-node-cidrs=true \

  --service-cluster-ip-range=10.1.0.0/16 \

  --cluster-cidr=10.2.0.0/16 \

  --cluster-name=kubernetes \

  --cluster-signing-cert-file=/opt/kubernetes/ssl/ca.pem \

  --cluster-signing-key-file=/opt/kubernetes/ssl/ca-key.pem \

  --service-account-private-key-file=/opt/kubernetes/ssl/ca-key.pem \

  --root-ca-file=/opt/kubernetes/ssl/ca.pem \

  --leader-elect=true \

  --v=2 \

  --logtostderr=false \

  --log-dir=/opt/kubernetes/log

Restart=on-failure

RestartSec=5

[Install]

WantedBy=multi-user.target

3.启动Controller Manager

[root@linux-node1 ~]# systemctl daemon-reload

[root@linux-node1 scripts]# systemctl enable kube-controller-manager

[root@linux-node1 scripts]# systemctl start kube-controller-manager

4.查看服务状态

[root@linux-node1 scripts]# systemctl status kube-controller-manager

部署Kubernetes Scheduler

[root@linux-node1 ~]# vim /usr/lib/systemd/system/kube-scheduler.service

[Unit]

Description=Kubernetes Scheduler

Documentation=https://github.com/GoogleCloudPlatform/kubernetes

[Service]

ExecStart=/opt/kubernetes/bin/kube-scheduler \

  --address=127.0.0.1 \

  --master=http://127.0.0.1:8080 \

  --leader-elect=true \

  --v=2 \

  --logtostderr=false \

  --log-dir=/opt/kubernetes/log

Restart=on-failure

RestartSec=5

[Install]

WantedBy=multi-user.target

2.部署服务

[root@linux-node1 ~]# systemctl daemon-reload

[root@linux-node1 scripts]# systemctl enable kube-scheduler

[root@linux-node1 scripts]# systemctl start kube-scheduler

[root@linux-node1 scripts]# systemctl status kube-scheduler

部署kubectl 命令行工具

1.准备二进制命令包

[root@linux-node1 ~]# cd /usr/local/src/kubernetes/client/bin

[root@linux-node1 bin]# cp kubectl /opt/kubernetes/bin/

2.创建 admin 证书签名请求

[root@linux-node1 ~]# cd /usr/local/src/ssl/

[root@linux-node1 ssl]# vim admin-csr.json

{
  "CN": "admin",
  "hosts": [],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "BeiJing",
      "L": "BeiJing",
      "O": "system:masters",
      "OU": "System"
    }
  ]
}

3.生成 admin 证书和私钥：

[root@linux-node1 ssl]# cfssl gencert -ca=/opt/kubernetes/ssl/ca.pem \

   -ca-key=/opt/kubernetes/ssl/ca-key.pem \

   -config=/opt/kubernetes/ssl/ca-config.json \

   -profile=kubernetes admin-csr.json | cfssljson -bare admin

[root@linux-node1 ssl]# ls -l admin*

-rw-r--r-- 1 root root 1009 Mar  5 12:29 admin.csr

-rw-r--r-- 1 root root  229 Mar  5 12:28 admin-csr.json

-rw------- 1 root root 1675 Mar  5 12:29 admin-key.pem

-rw-r--r-- 1 root root 1399 Mar  5 12:29 admin.pem

[root@linux-node1 src]# mv admin*.pem /opt/kubernetes/ssl/

4.设置集群参数

[root@linux-node1 src]# kubectl config set-cluster kubernetes \

   --certificate-authority=/opt/kubernetes/ssl/ca.pem \

   --embed-certs=true \

   --server=https://192.168.56.11:6443

Cluster "kubernetes" set.

5.设置客户端认证参数

[root@linux-node1 src]# kubectl config set-credentials admin \

   --client-certificate=/opt/kubernetes/ssl/admin.pem \

   --embed-certs=true \

   --client-key=/opt/kubernetes/ssl/admin-key.pem

User "admin" set.

6.设置上下文参数

[root@linux-node1 src]# kubectl config set-context kubernetes \

   --cluster=kubernetes \

   --user=admin

Context "kubernetes" created.

7.设置默认上下文

[root@linux-node1 src]# kubectl config use-context kubernetes

Switched to context "kubernetes".

8.使用kubectl工具

[root@linux-node1 ~]# kubectl get cs

NAME                 STATUS    MESSAGE             ERROR

controller-manager   Healthy   ok                 

scheduler            Healthy   ok                 

etcd-1               Healthy   {"health":"true"}  

etcd-2               Healthy   {"health":"true"}  

etcd-0               Healthy   {"health":"true"}  

部署kubelet

1.二进制包准备 将软件包从linux-node1复制到linux-node2中去。

[root@linux-node1 ~]# cd /usr/local/src/kubernetes/server/bin/

[root@linux-node1 bin]# cp kubelet kube-proxy /opt/kubernetes/bin/

[root@linux-node1 bin]# scp kubelet kube-proxy 192.168.56.12:/opt/kubernetes/bin/

[root@linux-node1 bin]# scp kubelet kube-proxy 192.168.56.13:/opt/kubernetes/bin/

2.创建角色绑定

[root@linux-node1 ~]# kubectl create clusterrolebinding kubelet-bootstrap --clusterrole=system:node-bootstrapper --user=kubelet-bootstrap

clusterrolebinding "kubelet-bootstrap" created

3.创建 kubelet bootstrapping kubeconfig 文件 设置集群参数

[root@linux-node1 ~]# kubectl config set-cluster kubernetes \

   --certificate-authority=/opt/kubernetes/ssl/ca.pem \

   --embed-certs=true \

   --server=https://192.168.56.11:6443 \

   --kubeconfig=bootstrap.kubeconfig

Cluster "kubernetes" set.

设置客户端认证参数

[root@linux-node1 ~]# kubectl config set-credentials kubelet-bootstrap \

   --token=ad6d5bb607a186796d8861557df0d17f \

   --kubeconfig=bootstrap.kubeconfig  

User "kubelet-bootstrap" set.

设置上下文参数

[root@linux-node1 ~]# kubectl config set-context default \

   --cluster=kubernetes \

   --user=kubelet-bootstrap \

   --kubeconfig=bootstrap.kubeconfig

Context "default" created.

选择默认上下文

[root@linux-node1 ~]# kubectl config use-context default --kubeconfig=bootstrap.kubeconfig

Switched to context "default".

[root@linux-node1 kubernetes]# cp bootstrap.kubeconfig /opt/kubernetes/cfg

[root@linux-node1 kubernetes]# scp bootstrap.kubeconfig 192.168.56.12:/opt/kubernetes/cfg

[root@linux-node1 kubernetes]# scp bootstrap.kubeconfig 192.168.56.13:/opt/kubernetes/cfg

部署kubelet 1.设置CNI支持

[root@linux-node2 ~]# mkdir -p /etc/cni/net.d

[root@linux-node2 ~]# vim /etc/cni/net.d/10-default.conf

{
        "name": "flannel",
        "type": "flannel",
        "delegate": {
            "bridge": "docker0",
            "isDefaultGateway": true,
            "mtu": 1400
        }
}

2.创建kubelet目录

[root@linux-node2 ~]# mkdir /var/lib/kubelet

3.创建kubelet服务配置

[root@k8s-node2 ~]# vim /usr/lib/systemd/system/kubelet.service

[Unit]

Description=Kubernetes Kubelet

Documentation=https://github.com/GoogleCloudPlatform/kubernetes

After=docker.service

Requires=docker.service

[Service]

WorkingDirectory=/var/lib/kubelet

ExecStart=/opt/kubernetes/bin/kubelet \

  --address=192.168.56.12 \

  --hostname-override=192.168.56.12 \

  --pod-infra-container-image=mirrorgooglecontainers/pause-amd64:3.0 \

  --experimental-bootstrap-kubeconfig=/opt/kubernetes/cfg/bootstrap.kubeconfig \

  --kubeconfig=/opt/kubernetes/cfg/kubelet.kubeconfig \

  --cert-dir=/opt/kubernetes/ssl \

  --network-plugin=cni \

  --cni-conf-dir=/etc/cni/net.d \

  --cni-bin-dir=/opt/kubernetes/bin/cni \

  --cluster-dns=10.1.0.2 \

  --cluster-domain=cluster.local. \

  --hairpin-mode hairpin-veth \

  --allow-privileged=true \

  --fail-swap-on=false \

  --logtostderr=true \

  --v=2 \

  --logtostderr=false \

  --log-dir=/opt/kubernetes/log

Restart=on-failure

RestartSec=5

4.启动Kubelet

[root@linux-node2 ~]# systemctl daemon-reload

[root@linux-node2 ~]# systemctl enable kubelet

[root@linux-node2 ~]# systemctl start kubelet

5.查看服务状态

[root@linux-node2 kubernetes]# systemctl status kubelet

6.查看csr请求 注意是在linux-node1上执行。

[root@linux-node1 ~]# kubectl get csr

NAME                                                   AGE       REQUESTOR           CONDITION

node-csr-0_w5F1FM_la_SeGiu3Y5xELRpYUjjT2icIFk9gO9KOU   1m        kubelet-bootstrap   Pending

7.批准kubelet 的 TLS 证书请求

[root@linux-node1 ~]# kubectl get csr|grep 'Pending' | awk 'NR>0{print $1}'| xargs kubectl certificate approve

执行完毕后，查看节点状态已经是Ready的状态了 [root@linux-node1 ssl]# kubectl get node NAME STATUS ROLES AGE VERSION

部署Kubernetes Proxy

1.配置kube-proxy使用LVS

[root@linux-node2 ~]# yum install -y ipvsadm ipset conntrack

2.创建 kube-proxy 证书请求

[root@linux-node1 ~]# cd /usr/local/src/ssl/

[root@linux-node1 ~]# vim kube-proxy-csr.json

{
  "CN": "system:kube-proxy",
  "hosts": [],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "BeiJing",
      "L": "BeiJing",
      "O": "k8s",
      "OU": "System"
    }
  ]
}

3.生成证书

[root@linux-node1~]# cfssl gencert -ca=/opt/kubernetes/ssl/ca.pem \

   -ca-key=/opt/kubernetes/ssl/ca-key.pem \

   -config=/opt/kubernetes/ssl/ca-config.json \

   -profile=kubernetes  kube-proxy-csr.json | cfssljson -bare kube-proxy

4.分发证书到所有Node节点

[root@linux-node1 ssl]# cp kube-proxy*.pem /opt/kubernetes/ssl/

[root@linux-node1 ssl]# scp kube-proxy*.pem 192.168.56.12:/opt/kubernetes/ssl/

[root@linux-node1 ssl]# scp kube-proxy*.pem 192.168.56.12:/opt/kubernetes/ssl/

5.创建kube-proxy配置文件

[root@linux-node2 ~]# kubectl config set-cluster kubernetes \

   --certificate-authority=/opt/kubernetes/ssl/ca.pem \

   --embed-certs=true \

   --server=https://192.168.56.11:6443 \

   --kubeconfig=kube-proxy.kubeconfig

Cluster "kubernetes" set.

[root@linux-node2 ~]# kubectl config set-credentials kube-proxy \

   --client-certificate=/opt/kubernetes/ssl/kube-proxy.pem \

   --client-key=/opt/kubernetes/ssl/kube-proxy-key.pem \

   --embed-certs=true \

   --kubeconfig=kube-proxy.kubeconfig

User "kube-proxy" set.

[root@linux-node2 ~]# kubectl config set-context default \

   --cluster=kubernetes \

   --user=kube-proxy \

   --kubeconfig=kube-proxy.kubeconfig

Context "default" created.

[root@linux-node2 ~]# kubectl config use-context default --kubeconfig=kube-proxy.kubeconfig

Switched to context "default".

6.分发kubeconfig配置文件

[root@linux-node1 ssl]# cp kube-proxy.kubeconfig /opt/kubernetes/cfg/

[root@linux-node1 ~]# scp kube-proxy.kubeconfig 192.168.56.12:/opt/kubernetes/cfg/

[root@linux-node1 ~]# scp kube-proxy.kubeconfig 192.168.56.13:/opt/kubernetes/cfg/

7.创建kube-proxy服务配置

[root@linux-node2 bin]# mkdir /var/lib/kube-proxy

[root@k8s-node2 ~]# vim /usr/lib/systemd/system/kube-proxy.service

[Unit]
Description=Kubernetes Kube-Proxy Server
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=network.target
[Service]
WorkingDirectory=/var/lib/kube-proxy
ExecStart=/opt/kubernetes/bin/kube-proxy \
  --bind-address=192.168.56.12 \
  --hostname-override=192.168.56.12 \
  --kubeconfig=/opt/kubernetes/cfg/kube-proxy.kubeconfig \
--masquerade-all \
  --feature-gates=SupportIPVSProxyMode=true \
  --proxy-mode=ipvs \
  --ipvs-min-sync-period=5s \
  --ipvs-sync-period=5s \
  --ipvs-scheduler=rr \
  --logtostderr=true \
  --v=2 \
  --logtostderr=false \
  --log-dir=/opt/kubernetes/log
Restart=on-failure
RestartSec=5
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target

8.启动Kubernetes Proxy

[root@linux-node2 ~]# systemctl daemon-reload

[root@linux-node2 ~]# systemctl enable kube-proxy

[root@linux-node2 ~]# systemctl start kube-proxy

9.查看服务状态 查看kube-proxy服务状态

[root@linux-node2 scripts]# systemctl status kube-proxy

检查LVS状态

[root@linux-node2 ~]# ipvsadm -L -n

IP Virtual Server version 1.2.1 (size=4096)

Prot LocalAddress:Port Scheduler Flags

  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn

TCP  10.1.0.1:443 rr persistent 10800

  -> 192.168.56.11:6443           Masq    1      0          0        

如果你在两台实验机器都安装了kubelet和proxy服务，使用下面的命令可以检查状态：

[root@linux-node1 ssl]#  kubectl get node

NAME            STATUS    ROLES     AGE       VERSION

192.168.56.12   Ready     <none>    22m       v1.10.1

192.168.56.13   Ready     <none>    3m        v1.10.1

linux-node3节点请自行部署。

1.为Flannel生成证书

[root@linux-node1 ~]# vim flanneld-csr.json

{
  "CN": "flanneld",
  "hosts": [],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "BeiJing",
      "L": "BeiJing",
      "O": "k8s",
      "OU": "System"
    }
  ]
}

2.生成证书

[root@linux-node1 ~]# cfssl gencert -ca=/opt/kubernetes/ssl/ca.pem \

   -ca-key=/opt/kubernetes/ssl/ca-key.pem \

   -config=/opt/kubernetes/ssl/ca-config.json \

   -profile=kubernetes flanneld-csr.json | cfssljson -bare flanneld

3.分发证书

[root@linux-node1 ~]# cp flanneld*.pem /opt/kubernetes/ssl/

[root@linux-node1 ~]# scp flanneld*.pem 192.168.56.12:/opt/kubernetes/ssl/

[root@linux-node1 ~]# scp flanneld*.pem 192.168.56.13:/opt/kubernetes/ssl/

4.下载Flannel软件包

[root@linux-node1 ~]# cd /usr/local/src

# wget

 https://github.com/coreos/flannel/releases/download/v0.10.0/flannel-v0.10.0-linux-amd64.tar.gz

[root@linux-node1 src]# tar zxf flannel-v0.10.0-linux-amd64.tar.gz

[root@linux-node1 src]# cp flanneld mk-docker-opts.sh /opt/kubernetes/bin/

复制到linux-node2节点

[root@linux-node1 src]# scp flanneld mk-docker-opts.sh 192.168.56.12:/opt/kubernetes/bin/

[root@linux-node1 src]# scp flanneld mk-docker-opts.sh 192.168.56.13:/opt/kubernetes/bin/

复制对应脚本到/opt/kubernetes/bin目录下。

[root@linux-node1 ~]# cd /usr/local/src/kubernetes/cluster/centos/node/bin/

[root@linux-node1 bin]# cp remove-docker0.sh /opt/kubernetes/bin/

[root@linux-node1 bin]# scp remove-docker0.sh 192.168.56.12:/opt/kubernetes/bin/

[root@linux-node1 bin]# scp remove-docker0.sh 192.168.56.13:/opt/kubernetes/bin/

5.配置Flannel

[root@linux-node1 ~]# vim /opt/kubernetes/cfg/flannel

FLANNEL_ETCD="-etcd-endpoints=https://192.168.56.11:2379,https://192.168.56.12:2379,https://192.168.56.13:2379"
FLANNEL_ETCD_KEY="-etcd-prefix=/kubernetes/network"
FLANNEL_ETCD_CAFILE="--etcd-cafile=/opt/kubernetes/ssl/ca.pem"
FLANNEL_ETCD_CERTFILE="--etcd-certfile=/opt/kubernetes/ssl/flanneld.pem"
FLANNEL_ETCD_KEYFILE="--etcd-keyfile=/opt/kubernetes/ssl/flanneld-key.pem"

复制配置到其它节点上

[root@linux-node1 ~]# scp /opt/kubernetes/cfg/flannel 192.168.56.12:/opt/kubernetes/cfg/

[root@linux-node1 ~]# scp /opt/kubernetes/cfg/flannel 192.168.56.13:/opt/kubernetes/cfg/

6.设置Flannel系统服务

[root@linux-node1 ~]# vim /usr/lib/systemd/system/flannel.service

[Unit]
Description=Flanneld overlay address etcd agent
After=network.target
Before=docker.service
[Service]
EnvironmentFile=-/opt/kubernetes/cfg/flannel
ExecStartPre=/opt/kubernetes/bin/remove-docker0.sh
ExecStart=/opt/kubernetes/bin/flanneld ${FLANNEL_ETCD} ${FLANNEL_ETCD_KEY} ${FLANNEL_ETCD_CAFILE} ${FLANNEL_ETCD_CERTFILE} ${FLANNEL_ETCD_KEYFILE}
ExecStartPost=/opt/kubernetes/bin/mk-docker-opts.sh -d /run/flannel/docker
Type=notify
[Install]
WantedBy=multi-user.target
RequiredBy=docker.service

复制系统服务脚本到其它节点上

# scp /usr/lib/systemd/system/flannel.service 192.168.56.12:/usr/lib/systemd/system/

# scp /usr/lib/systemd/system/flannel.service 192.168.56.13:/usr/lib/systemd/system/

Flannel CNI集成

下载CNI插件https://github.com/containernetworking/plugins/releases

wget https://github.com/containernetworking/plugins/releases/download/v0.7.1/cni-plugins-amd64-v0.7.1.tgz

[root@linux-node1 ~]# mkdir /opt/kubernetes/bin/cni

[root@linux-node1 src]# tar zxf cni-plugins-amd64-v0.7.1.tgz -C /opt/kubernetes/bin/cni

# scp -r /opt/kubernetes/bin/cni/* 192.168.56.12:/opt/kubernetes/bin/cni/

# scp -r /opt/kubernetes/bin/cni/* 192.168.56.13:/opt/kubernetes/bin/cni/

创建Etcd的key

/opt/kubernetes/bin/etcdctl --ca-file /opt/kubernetes/ssl/ca.pem --cert-file /opt/kubernetes/ssl/flanneld.pem --key-file /opt/kubernetes/ssl/flanneld-key.pem \

      --no-sync -C https://192.168.56.11:2379,https://192.168.56.12:2379,https://192.168.56.13:2379 \

mk /kubernetes/network/config '{ "Network": "10.2.0.0/16", "Backend": { "Type": "vxlan", "VNI": 1 }}' >/dev/null 2>&1

启动flannel

[root@linux-node1 ~]# systemctl daemon-reload

[root@linux-node1 ~]# systemctl enable flannel

[root@linux-node1 ~]# chmod +x /opt/kubernetes/bin/*

[root@linux-node1 ~]# systemctl start flannel

查看服务状态

[root@linux-node1 ~]# systemctl status flannel

配置Docker使用Flannel

[root@linux-node1 ~]# vim /usr/lib/systemd/system/docker.service

[Unit] #
在Unit下面修改After和增加Requires
After=network-online.target firewalld.service flannel.service
Wants=network-online.target
Requires=flannel.service
[Service] #
增加
EnvironmentFile=-/run/flannel/docker
Type=notify
EnvironmentFile=-/run/flannel/docker
ExecStart=/usr/bin/dockerd $DOCKER_OPTS

将配置复制到另外两个阶段

# scp /usr/lib/systemd/system/docker.service 192.168.56.12:/usr/lib/systemd/system/

# scp /usr/lib/systemd/system/docker.service 192.168.56.13:/usr/lib/systemd/system/

重启Docker

[root@linux-node1 ~]# systemctl daemon-reload

[root@linux-node1 ~]# systemctl restart docker

1.创建一个测试用的deployment

[root@linux-node1 ~]# kubectl run net-test --image=alpine --replicas=2 sleep 360000

2.查看获取IP情况

[root@linux-node1 ~]# kubectl get pod -o wide

NAME                        READY     STATUS    RESTARTS   AGE       IP          NODE

net-test-74f45db489-gmgv8   1/1       Running   0          1m        10.2.83.2   192.168.56.13

net-test-74f45db489-pr5jc   1/1       Running   0          1m        10.2.59.2   192.168.56.12

3.测试联通性

ping 10.2.83.2

Kubernetes Dashboard

创建CoreDNS

[root@linux-node1 ~]# kubectl create -f coredns.yaml

[root@linux-node1 ~]# kubectl get pod -n kube-system

NAME                                    READY     STATUS    RESTARTS   AGE

coredns-77c989547b-9pj8b                1/1       Running   0          6m

coredns-77c989547b-kncd5                1/1       Running   0          6m

创建Dashboard

[root@linux-node1 ~]# kubectl create -f dashboard/

[root@linux-node1 ~]# kubectl cluster-info

Kubernetes master is running at https://192.168.56.11:6443

kubernetes-dashboard is running at https://192.168.56.11:6443/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

kubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep admin-user | awk '{print $1}')

猜你喜欢

转载自blog.csdn.net/qq_28470425/article/details/82842989
今日推荐
基于大语言模型的开源知识库问答系统 MaxKB GitHub Star 数量突破 5,000 个！
美国拟限制 AI 大模型出口中国和俄罗斯
苹果将与 OpenAI 达成协议，将 ChatGPT 应用于 iPhone
openKylin 社区生态委员会第六次会议圆满召开
阿里云正式发布通义千问 2.5
Python 3.13 发布首个 Beta：实验性自由线程模式和 JIT、改进交互式解释器
Stack Overflow 拿我的代码去训练 AI 大模型，还封了我的账号
Pop!_OS 的 COSMIC 桌面完成 App Store 上架工作
《2024 年一季度互联网投融资运行情况》研究报告
报告：Django 仍然是 74% 开发者的首选
15 年前上了“FFmpeg 耻辱柱”，今天他还得谢谢咱——腾讯QQPlayer一雪前耻？
TIOBE 5 月榜单：Fortran “复活”进入 Top 10
周排行
记一下去大梅沙的准备（2018-05-26）
Spring 注解 事务
基于HTTP协议的客户端缓存
阿里云rds 备份和还原
[PHP] 几个拖慢 PHP 程序/API 运行速度的点
python 代码风格------------PEP8规则
js控制json生成菜单——自制菜单（一）
将字符串: 'k:1|k1:2|k2:3|k3:4 ' ,处理成 python 字典: {'k':1, 'k1':2, ...}
微信小程序转支付宝小程序
Qt551.窗口滚动条
每日归档
更多
2024-05-13(18)
2024-05-12(0)
2024-05-11(38)
2024-05-10(38)
2024-05-09(35)
2024-05-08(42)
2024-05-07(14)
2024-05-06(40)
2024-05-05(0)
2024-05-04(7)

代码天地 © 2018 codetd.com

境外服务器   最新电视剧2022