Mesos installation articles (1)

Mesos was born as a research project at UC Berkeley and is now a project in the Apache Incubator. The Mesos computing framework is a cluster manager that provides efficient resource isolation and sharing across distributed applications or frameworks, running Hadoop, MPI, Hypertable, and Spark. Enables ZooKeeper to implement fault-tolerant replication, use Linux Containers to isolate tasks, and support multiple resource scheduling allocations.



Overall Architecture

Apache Mesos consists of four components, Mesos-master, mesos-slave, framework and executor.

Explanation of basic terms in mesos

1

Mesos-master: It is the core of the entire system and is responsible for managing the frameworks (managed by frameworks_manager) and slaves (managed by slaves_manager) accessing mesos, and assigns resources on slaves to a certain strategy. framework (managed by the independent pluggable module Allocator).

2


Mesos-slave: Responsible for receiving and executing commands from mesos-master, managing mesos-tasks on nodes, and allocating resources for each task. The mesos-slave sends its own resources to the mesos-master, and the Allocator module in the mesos-master decides which framework to allocate the resources to. Currently, the resources considered are CPU and memory. That is to say, the mesos-slave will The number of CPUs and the amount of memory are sent to the mesos-master, and when the user submits a job, he needs to specify the number of CPUs and the amount of memory required by each task. In this way, when the task is running, the mesos-slave will put the task into the fixed resource containing the task. run in the linux container to achieve the effect of resource isolation.

3



Framework: Refers to external computing frameworks, such as Hadoop, Mesos, etc. These computing frameworks can access mesos through registration, so that mesos can perform unified management and resource allocation. Mesos requires that the accessible framework must have a scheduler module, which is responsible for task scheduling within the framework. When a framework wants to access mesos, it needs to modify its own scheduler in order to register with mesos and obtain the resources allocated to itself by mesos, so that its own scheduler allocates these resources to tasks in the framework, that is, It is said that the entire mesos system adopts the two-layer scheduling framework

4


Executor: the executor, which is installed on the mesos-slave and used to start the tasks in the computing framework.


Installation and deployment




1

Introduction to deployment environment

Kernel version:
[root@ctn-7-11 ~]# uname -a
Linux ctn-7-11.ptmind.com 3.10.0-327.4.4.el7.x86_64 #1 SMP Tue Jan 5 16:07:00 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

system version:
[root@ctn-7-11 ~]# more /etc/redhat-release
CentOS Linux release 7.2.1511 (Core)

docker version:
[root@ctn -7-13 ~]# docker --version
Docker version 1.10.3, build 20f81dd

Add hostnames to all hosts in the cluster
[root@ctn-7-12 ~]# more /etc/hosts
---Slightly---
172.16.7.12 ctn-7-12.ptmind.com ctn-7-12
172.16.7.13 ctn-7-13.ptmind .com ctn-7-13
172.16.7.14 ctn-7-14.ptmind.com ctn-7-14

is easy to configure the server ssh password-free login to get through
[root@ctn-7-11 workspace]# for ip in $( cat /workspace/mesoslist);do echo ------$ip-----;ssh root@$ip hostname;done
------172.16.7.11-----
ctn-7-11 .ptmind.com
------172.16.7.12-----
ctn-7-12.ptmind.com
------172.16.7.13-----
ctn-7-13.ptmind.com
------172.16.7.14-----
ctn-7-14.ptmind.com


1

Preparation before installation

Close firewall
sudo systemctl stop firewalld
sudo systemctl disable firewalld


install docker
sudo yum update
curl -sSL https://get.docker.com/ | sh

install common software packages
yum install git lrzsz wget elinks nc -y


add yum source (all hosts, install one of them)
rpm -Uvh http:/ /repos.mesosphere.io/el/7/noarch/RPMS/mesosphere-el-repo-7-1.noarch.rpm
rpm -Uvh http://repos.mesosphere.io/el/7/noarch/RPMS/mesosphere -el-repo-7-2.noarch.rpm
rpm -Uvh http://repos.mesosphere.io/el/7/noarch/RPMS/mesosphere-el-repo-7-3.noarch.rpm


3

cluster service deployment Planning
Server IP Address Host Name Installation Service
172.16.7.12ctn-7-12.ptmind.com mesos-master , zookeeper, mesos-slave , chronos
172.16.7.13ctn-7-13.ptmind.com mesos-master , zookeeper, mesos -slave ,
172.16.7.14ctn-7-14.ptmind.com mesos-master , zookeeper, mesos-slave, marathon


More content is being updated, Silver Shield Tai'an will continue to release the latest and most useful information for you, so stay tuned!