Hi, I'm Sidhartha Mani, an engineer at Rancher Labs, and for the past few months, I've been using Apache Mesos, an open source resource manager and scheduler, for deploying infrastructure. Mesos is very good at resource understanding and capacity management, but for more advanced workload or container management, many users choose a framework like Marathon. Such a framework can provide framework-provided features such as load balancing, service discovery, rolling upgrades, application composability, continuous deployment, and more.
Many Rancher community members also use Mesos and feel the need for a system that combines fault-tolerance, multi-tenancy, and the private container service capabilities Rancher provides. A community user from AT&T, Marcel Neuhausler, proposes an extensive design and design such an integrated workflow. Arranges the capabilities of VMs and Rancher's capabilities and manages the deployment of containers for those VMs. He wrote a Mesos-based framework for Rancher, which proved to be a great starting point for the framework I'm going to talk about today.
In this blog, I will describe this idea and this software (Rancher Mesos framework), through the act of working with Marcel. This framework can be used to set up large-scale production jobs like Hadoop, Kafka, ElasticSearch, etc. in Docker containers, as well as any part of the DevOps process, such as build, test, staging, or production application environments.
In this section, I will describe the framework's architecture and show how to configure it on your local environment.
Rancher Mesos Architecture
Below is a diagram to explain the Rancher Mesos framework:
As you can see, Rancher integrates with Mesos, using the standard Mesos interface - the Mesos framework. The various components are:
1.mesos-master: mesos-master is the process of the cluster mesos-master running on the machine. It maintains and monitors the mesos-slave, and handles resource provisioning, task initiation, task monitoring, fault tolerance, and messaging, etc.
2. Mesos Slave:
3. Rancher Server: This is a cluster of hosts running the docker container of rancher/server. It maintains and monitors the rancher host and provides users with a multi-tenant container service, including container management and infrastructure features such as private deployment environments, container networking, docker build support, service discovery, rolling upgrades, registry management, etc.
4. Rancher Hosts: These are provided using resources provided by Mesos hosts. These hosts run docker and rancher/agent run, for rancher's private network, containers are scheduled for various tasks and include hosts.
5. Rancher-Mesos Scheduler: It is a two-tier application, Rancher external event handler and Mesos scheduler. Event handlers are used to listen for events from Rancher created hosts. When the Rancher-Mesos scheduler receives a create host event, it adds the event to the event queue. Once provided a suitable slave schedules tasks, executes events, and creates them on the Rancher host if it has sufficient capacity.
6. Rancher-Mesos Executor: The called process provides an available slave to create a host for the rancher. This process creates a vm using QEMU-KVM, using bridged networking. Install on these vms and the rancher/agent registers it with the rancher server.
7. Rancher-Mesos Framework: is used to refer to Rancher-Mesos Scheduler and Rancher-Mesos Executor collectively.
Workflow for Rancher Mesos
1. The user will click on Add Host in the UI, which will provide a host among an available Mesos slaves. The allocation of Mesos slaves is determined by the Mesos master.
2. Once the host is provisioned, it will register itself to the rancher server. It will be displayed in the UI and the user can view statistics, execute shells or start/stop containers as normal.
See the following diagram explaining the sequence of events using the rancher server:
1. When you click Add Host in the UI, the rancher server creates a physicalhost.create event.
2. This event is received by all external subscribed to this event handler. In this case, the Rancher-Mesos scheduler subscribes to the event.
3. When an event is received, the scheduler can save the event in the event queue.
4. The scheduler then waits for the resource to come from an idle host provided by the Mesos Master.
5. Once the scheduler receives the resource offer, it can retrieve the oldest event from the queue and start the task of the offer host.
6. The task starts executing Rancher Mesos. The executor uses QEMU-KVM to start a new virtual machine.
7. Then install in the new virtual machine.
8. The executor instructs the new virtual machine and rancher must act as a host server.
Configuring and Running the Rancher Framework
In this section, I'll show you how to configure the architecture and try it out on your computer. We will use the VMware fusion pro virtualized setup as it requires changing network configuration and its easy to work with.
Download the Ubuntu desktop 14.04.2 iso. In VMware fusion, select Add > Have Disk or Image. Make sure nested virtualization is enabled, and have at least 2 GB of RAM before booting.
Enable Nested Virtualization
Click on settings >
Processors and Memory >
Advanced Options >
Enable Hypervisor Applications- 1
- 2
- 3
- 4
- 1
- 2
- 3
- 4
1. The first step is to set the network configuration. We need to set up a bridge network for eth0. Before operation, make sure that the bridge-utils package has been installed, use sudo apt-get install bridge-utils, and configure your /etc/ as follows network/interfaces file.
auto lo
iface lo inet loopback
auto eth0
iface eth0 inet manual
auto br0
iface br0 inet dhcp
bridge_ports eth0
bridge_stp off
bridge_fd 0
bridge_maxwait 0- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
Then run the ifup -a command, which will read the configuration file and set up the bridge interface.
2. Next install the necessary package. First, you need git, execute this command sudo apt-get install
git
sudo apt-get install -y qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils
echo 'allow br0' > /etc/qemu/bridge.conf- 1
- 2
- 3
- 1
- 2
- 3
Then, install the executor (you need golang, mercurial and Godeps)
go get -d github.com/wlan0/rancher-mesos-executor
Cd $GOPATH/src/github.com/wlan0/rancher-mesos-executor && ./scripts/build
sudo cp build/rancher-mesos-executor /bin/
$GOPATH/src/github.com/wlan0/rancher-mesos-executor && ./scripts/build
sudo cp build/rancher-mesos-executor /bin/ - 1
- 2
- 3
- 4
- 5
- 1
- 2
- 3
- 4
- 5
Finally install docker
wget -qO- https://get.docker.com | sh- 1
- 1
3. Start the rancher-server service
这将开始rancher-server在80端口- 1
- 1
4. Install mesos-master and mesos-slave
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv E56151BF
echo "deb http://repos.mesosphere.io/ubuntu/ trusty main" > /etc/apt/sources.list.d/mesosphere.list
sudo apt-get -y update
sudo apt-get -y install mesos
service zookeeper stop
sudo apt-get -y remove --purge zookeeper
echo manual > /etc/init/mesos-master.override- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 1
- 2
- 3
- 4
- 5
- 6
- 7
5. Start mesos-master and mesos-slave
sudo nohup mesos-master --work_dir=$(pwd) --ip=$IP &
sudo nohup mesos-slave --master=$IP:5050 --ip=$IP &- 1
- 2
- 1
- 2
6. Start rancher-mesos scheduler
CATTLE_URL=http://$IP:8080/v1 CATTLE_ACCESS_KEY=service CATTLE_SECRET_KEY=servicepass MESOS_MASTER=$IP:5050 IP_CIDR=$IP/24 rancher-mesos-scheduler- 1
- 1
7. From a browser, go to IP:8080 to see the rancher UI. Now, I have to do some other things to get the UI to call mesos, the rancher uses docker's machine to add resources from the cloud, and I haven't had time to create a custom Mesos-machine driver. To add a host, click Add Host in any environment and select the "Rackspace" icon, use any invalid credentials, and click Create. You should see a host's infrastructure tab added to the rancher. Wait a few minutes for the host to connect successfully. You can start containers using this host. Note: The part of the authentication handler I handle externally ( rancher-mesos-scheduler ) ignores the cloud type and always provides the convenience host, which will be driven from any different cloud drivers and APIs. In the future I will add a proper driver Mesos to create the host functionality.
8. Every time you provide the host, the console to create the VM will pop up on your screen. You can disable this in a production environment.
Hope this gives you an idea how to deploy rancher as a framework for Mesos. Thanks again to Marcel Neuhausler from AT&T for working with our team, and all other community members for proposing or suggesting this integration.
http://blog.csdn.net/lixudan/article/details/48545911