RabbitMQ is an open source message broker software (also known as message-oriented middleware) that implements the Advanced Message Queuing Protocol (AMQP)
-
Scalability: cluster services
-
Message persistence: Persist messages from the memory to the hard disk, and then load them from the hard disk to the memory
In our work, we often use Rabbitmq, which has a single node and a cluster.
Install RabbitMQ through this Kubernetes, and automatically discover registered clusters by inserting [rabbitmq_management, rabbitmq_peer_discovery_k8s]:
RabbitMQ is deployed through StatefulSet and can be accessed through domain names for easy processing
(1) Environment
Need Kubernetes cluster
Download the deployment file
下载地址:https://github.com/dotbalo/k8s/tree/master/k8s-rabbitmq-cluster
(2) Create a namespace
kubectl create ns public-service
If you do not use public-service, you need to change the public-service of all yaml files to your namespace.
sed -i "s#public-service#YOUR_NAMESPACE#g" *.yaml
(3) Modify the configuration file
在rabbitmq-configmap.yaml配置,添加这两个字段,解决密码不生效的问题
default_pass = RABBITMQ_PASS
default_user = RABBITMQ_USER
PV here uses nfs. The test environment, here I created it manually, or you can use a plug-in to create it automatically, and recycle it.
[root@k8s-master01 k8s-rabbitmq-cluster]# cat rabbitmq-pv.yaml
apiVersion: v1
kind: PersistentVolume
metadata:
name: pv-rmq-1
spec:
capacity:
storage: 1Gi
accessModes:
- ReadWriteMany
volumeMode: Filesystem
persistentVolumeReclaimPolicy: Recycle
storageClassName: "rmq-storage-class"
nfs:
# real share directory
path: /ifs/kubernetts/rabbitmq-cluster-0
# nfs real ip
server: 192.168.0.109
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: pv-rmq-2
spec:
capacity:
storage: 1Gi
accessModes:
- ReadWriteMany
volumeMode: Filesystem
persistentVolumeReclaimPolicy: Recycle
storageClassName: "rmq-storage-class"
nfs:
# real share directory
path: /ifs/kubernetts/rabbitmq-cluster-1
# nfs real ip
server: 192.168.0.109
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: pv-rmq-3
spec:
capacity:
storage: 1Gi
accessModes:
- ReadWriteMany
volumeMode: Filesystem
persistentVolumeReclaimPolicy: Recycle
storageClassName: "rmq-storage-class"
nfs:
# real share directory
path: /ifs/kubernetts/rabbitmq-cluster-2
# nfs real ip
server: 192.168.0.109
(4) Create a cluster
[root@k8s-master01 k8s-rabbitmq-cluster]# kubectl apply -f .
statefulset.apps/rmq-cluster created
configmap/rmq-cluster-config created
persistentvolume/pv-rmq-1 created
persistentvolume/pv-rmq-2 created
persistentvolume/pv-rmq-3 created
serviceaccount/rmq-cluster created
role.rbac.authorization.k8s.io/rmq-cluster created
rolebinding.rbac.authorization.k8s.io/rmq-cluster created
secret/rmq-cluster-secret created
service/rmq-cluster created
service/rmq-cluster-balancer created
Check the container log and show that this is normal
kubectl logs -f rmq-cluster-0 -n public-service
'/etc/rabbitmq/rabbitmq.conf' -> '/var/lib/rabbitmq/rabbitmq.conf'
2021-03-11 02:24:03.469 [info] <0.9.0> Feature flags: list of feature flags found:
2021-03-11 02:24:03.469 [info] <0.9.0> Feature flags: feature flag states written to
。。。。。。。。。。。。。。。
。。。。。。。。。。。。。。。
2021-03-11 02:24:04.273 [info] <0.692.0> Statistics database started.
2021-03-11 02:24:04.274 [info] <0.691.0> Starting worker pool 'management_worker_pool' with 3 processes in it
completed with 5 plugins.
2021-03-11 02:24:04.411 [info] <0.9.0> Server startup complete; 5 plugins started.
* rabbitmq_management
* rabbitmq_management_agent
* rabbitmq_web_dispatch
* rabbitmq_peer_discovery_k8s
* rabbitmq_peer_discovery_common
2021-03-11 02:24:23.135 [info] <0.434.0> node 'rabbit@rmq-cluster-1.rmq-cluster.public-service.svc.cluster.local' up
2021-03-11 02:24:23.904 [info] <0.434.0> rabbit on node 'rabbit@rmq-cluster-1.rmq-cluster.public-service.svc.cluster.local' up
2021-03-11 02:24:44.524 [info] <0.434.0> node 'rabbit@rmq-cluster-2.rmq-cluster.public-service.svc.cluster.local' up
2021-03-11 02:24:45.477 [info] <0.434.0> rabbit on node 'rabbit@rmq-cluster-2.rmq-cluster.public-service.svc.cluster.local' up
Create resources as follows
[root@k8s-master01 ~]# kubectl get pod,sts,svc,ep,pv,pvc -n public-service
NAME READY STATUS RESTARTS AGE
pod/rmq-cluster-0 1/1 Running 0 8m10s
pod/rmq-cluster-1 1/1 Running 0 7m34s
pod/rmq-cluster-2 1/1 Running 0 7m18s
NAME READY AGE
statefulset.apps/rmq-cluster 3/3 8m11s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/rmq-cluster ClusterIP None <none> 5672/TCP 8m11s
service/rmq-cluster-balancer NodePort 10.96.58.102 <none> 15672:31824/TCP,5672:31993/TCP 8m11s
NAME ENDPOINTS AGE
endpoints/rmq-cluster 10.244.32.129:5672,10.244.58.225:5672,10.244.85.246:5672 8m11s
endpoints/rmq-cluster-balancer 10.244.32.129:5672,10.244.58.225:5672,10.244.85.246:5672 + 3 more... 8m11s
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
persistentvolume/pv-rmq-1 1Gi RWX Recycle Bound public-service/rabbitmq-storage-rmq-cluster-2 rmq-storage-class 8m11s
persistentvolume/pv-rmq-2 1Gi RWX Recycle Bound public-service/rabbitmq-storage-rmq-cluster-1 rmq-storage-class 8m11s
persistentvolume/pv-rmq-3 1Gi RWX Recycle Bound public-service/rabbitmq-storage-rmq-cluster-0 rmq-storage-class 8m11s
persistentvolume/pv0001 2Gi RWO Recycle Bound default/test-pvc2 slow 9d
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
persistentvolumeclaim/rabbitmq-storage-rmq-cluster-0 Bound pv-rmq-3 1Gi RWX rmq-storage-class 8m11s
persistentvolumeclaim/rabbitmq-storage-rmq-cluster-1 Bound pv-rmq-2 1Gi RWX rmq-storage-class 7m34s
persistentvolumeclaim/rabbitmq-storage-rmq-cluster-2 Bound pv-rmq-1 1Gi RWX rmq-storage-class 7m18s
[root@k8s-master01 ~]#
(5) Verify the cluster
Can be exposed through the nodeport port of svc, or ingress to access the rabbitmq visualization interface
default_user = RABBITMQ_USER #账号
default_pass = RABBITMQ_PASS #密码
(6) Cluster expansion and shrinkage
Expansion, if PV is used, it is recommended to check if PV is enough, otherwise the status is always pending
kubectl scale statefulset -n public-service --replicas=4 rmq-cluster
--replicas=4 改这个值,就可以用来扩容和缩容
https://github.com/dotbalo/k8s/tree/master/k8s-rabbitmq-cluster