Bowen directory
a, GlusterFS overview
1, GlusterFS characteristics
2, the term GlusterFS
3, stacked modular architecture
Second, the principle of GlusterFS
1, GlusterFS workflow
2, the elastic HASH algorithm
three, GlusterFS volume type
1, distributed volumes
2, the strip roll
3, copy volume
4, distributed striped volume
5, the volume distributed replication

A, GlusterFS Overview

GlusterFS is an open source distributed file system, but also the core of Scale-Out storage solutions Gluster, there is a strong ability to scale in terms of data storage, by expanding the different nodes can support PB level of storage capacity. GlusterFS With TCP / IP or InfiniBand RDMA network will bring together dispersed storage resources, providing the same storage services, and use the space for a single global command to manage data. GlusterFS stackable user space, and no element based design, it can provide excellent performance data for a variety of loads.

GlusterFS mainly by the storage server, and the client NFS / Samba storage gateway (alternatively, be selected to use) composition. GlusterFS architecture biggest design feature is no metadata server component, which helps improve performance, reliability and stability of the entire system. Most of conventional distributed file system server through metadata stored metadata, the metadata comprising directory information, the directory structure on the storage node, this design is very efficient when browsing the directory, but there are some drawbacks, such as the single point of failure Once the metadata server fails, then even if the node includes high redundancy, will collapse the entire storage system, and the distributed file system is no GlusterFS membered server based design, the transverse intensity data expansion capability, with high reliability sex and storage efficiency. GlusterFS supports TCP / IP and InfiniBand RDMA high-speed networking, client data can be accessed through the acoustic GlusterFS agreement, the other did not run GlusterFS client terminal can access the data through storage gateway through NFS / CIFS protocol standard. As shown below:
GlusterFS Distributed File System overview

1, GlusterFS features

Scalability and high performance;

High availability;

Global unified namespace;

Standards-based protocols;

Elastic volume management;

2, GlusterFS term

Brick (memory block): refers to the trusted hosts provided by the host cell dedicated partition for the physical storage is GlusterFS the basic storage unit for storing the directory while the server is trusted to provide both storage pool, store directory format consists of the absolute path of the directory server and, as a representation sERVER: EXPORT, such as: 192.168.1.4/date/mydir/;

Volume (logical volume): a logical volume is a set of Brick. Volume is a logical data storage device, similar to the LVM logical volume. Most Gluster management operation is performed on a roll;

FUSE: is a kernel module that allows users to create their own file systems without modifying the kernel code;

VFS: Interface kernel space to user space it provides access to the disk;

Glusterd (background management process): to be run on each node in the cluster storage;

3, stacked modular architecture

如下图所示，GlusterFS采用模块化、堆栈式的架构，可以根据需求配置定制化的应用环境，如大文件存储、海量小文件存储、云存储、多传输协议应用等。通过对模块进行各种组合，即可实现复杂的功能。例如，Replicate模块可实现RAID1，Stripe模块可实现RAID0，通过两者的组合可实现RAID10和RAID01，同时获得更高的性能及可靠性。
GlusterFS Distributed File System overview
GlusterFS是模块化堆栈式的架构设计。模块成为Translator，是GlusterFS提供的一种强大的机制，借助这种良好定义的接口可以高效简便地扩展文件系统的功能。
1）服务器与客户端的设计高度模块化的同事模块接口是兼容的，同一个transtator可同事在客户端和服务器加载；

2）GlusterFS中所有的功能都是通过transtator实现的，其中客户端要比服务器更复杂。所以功能的重点主要集中在客户端上；

二、GlusterFS的工作原理

1、GlusterFS的工作流程

GlusterFS Distributed File System overview
图中所示只是GlusterFS数据访问的一个概要图。
1）客户端或应用程序通过GlusterFS的挂在点访问数据；

2）Linux系统内核通过VFS API收到请求并处理；

3）VFS将数据递交给FUSE内核文件系统，并向系统注册了一个实际的文件系统FUSE，而FUSE文件系统则是将数据过/dev/fuse设备文件递交给GlusterFS client端。可以将FUSE文件系统理解为一个代理；

4）GlusterFS client收到数据后。client根据配置文件对数据进行处理；

5）经过GlusterFS client处理后，通过网络将数据传递至远端的GlusterFS Server，并且将数据写入服务器存储设备；

2、弹性HASH算法

弹性HASH算法使用Davies-Meyer算法，通过HASH算法得到一个32位的整数范围，假设逻辑卷中有N个存储单位Brick，则32位的整数范围被划分为N个连续的子空间，每个空间对应一个Brick。当用户或应用程序访问某一个命名空间时，通过对该命名空间计算HASH值，根据该HASH值对应的32位整数空间定位数据所在的Brick。优点如下：

保证数据平均分布在每个Brick中；

解决了对元数据服务器的依赖，进而解决了单点故障及访问瓶颈；

三、GlusterFS的卷类型

GlusterFS支持七种卷，即分布式卷、条带卷、复制卷、分布式条带卷、分布式复制卷、条带复制卷和分布式条带复制卷，这七种卷可以满足不同应用对高性能、高可用的需求。

分布式卷（Distribute volume）：文件通过HASH算法分布到所有Brick Server上，这种卷是Glusterf的基础；以文件为单位根据HASH算法散列到不同的Brick，其实只是扩大了磁盘空间，如果有一个磁盘损坏，数据也将丢失，属于文件级的RAID0，不具备容错能力；

条带卷（Stripe volume）：类似于RAID0，文件被分为数据块并以轮询的方式分布到多个Brick Server上，文件存储以数据块为单位，支持大文件存储，文件越大，读取效率越高；

复制卷（Replica volume）：将文件同步到多个Brick上，使其具备多个文件副本，属于文件级RAID1，具有容错能力。因为数据分散到多个Brick中，所以读性能得到了很大提升，但写性能下降；

分布式条带卷（Distribute Stripe volume）：Brick Server数量是条带数（数据块分布的Brick数量）的倍数，兼备分布式卷和条带卷的特点；

分布式复制卷（Distribute Replica volume）：Brick Server数量是镜像数（数据副本数量）的倍数，具有分布式卷和复制卷的特点；

条带复制卷（Stripe Replica volume）：类似于RAID10，同时具有条带卷和复制卷的特点；

分布式条带复制卷（Distribute Stripe Replica volume）：三种基本卷的复合卷，通常用于类Map Reduce应用；

1、分布式卷

分布式卷是GlusterFS的默认卷，在创建卷时，默认选项就是创建分布式卷。在该模式下，并没有对文件进行分块处理，文件直接存储在某个Server节点上。直接使用本地文件系统进行文件存储，大部分Linux命令和工具可以继续正常使用。需要通过扩展文件属性保存HASH值，目前支持的底层文件系统有ext3、ext4、ZFS、XFS等。
由于使用本地文件系统，所以存取效率并没有提高，反而会因为网络通信的原因而有所降低；另外支持超大型文件也会有一定的难度，因为分布式卷不会对文件进行分块处理。虽然ext4已经可以支持最大16TB的单个文件，但是本地存储设备的容量实在有限。
如图所示，File1和File2存放在Server1，而File3存放在Server2，文件都是随机存储，一个文件要么在Server1上，要么在Server2上，不能分块同时存放在Server1和Server2上。
GlusterFS Distributed File System overview

分布式卷具有如下特点：

文件分布在不同的服务器，布局别冗余性；

更容易廉价地扩展卷的大小；

单点故障会造成数据丢失；

依赖于底层的数据保护；

创建分布式卷：

[root@cecentos01 ~]# gluster volume create dis-volume server1:/dir1 server2:/dir2
<!--创建一个名为dis-volume的分布卷，文件将根据HASH分布在
server1：/dir1、server2:/dir2中-->
Creation of dis-volume has been successful
Please start the volume to access data

2、条带卷

Stripe模式相当于RAID0，在该模式下，根据偏移量将文件分成N块（N个条带节点），轮询地存储在每个Brick Server节点。节点把每个数据块都作为普通文件存入本地文件系统中，通过扩展属性记录总块数和每块的序号。在配置时指定的条带数必须等于卷中Brick所包含的存储服务器数，在存储大文件时，性能尤为突出，但是不具备冗余性。
如下图所示，将文件存放在不同服务器里，File被分割为6段，1、3、5放在server1，2、4、6放在server2。
GlusterFS Distributed File System overview

条带卷具有如下特点：

数据被分割成更小块分布到块服务器群中的不同条带区；

分布减少了负载且更小的文件加速了存取的速度；

没有数据冗余；

创建条带卷：

[root@centos01 ~]# gluster volume create stripe-volume stripe 2 transport tcp server1:/dir1 server2:/dir2
<!--创建一个名为Stripe-volume的条带卷，文件将分块轮询地存储在
server1:/dir1 、server2:/dir2两个Brick中-->
Creation of rep-volume has been successful
Please start the volume to access data

3、复制卷

复制模式，也称为AFR，相当于RAID1。即同一文件保存一份或多份副本，每个节点保存相同的内容和目录结构。复制模式因为要保存副本，所以磁盘利用率较低。如果多个节点上的存储空间不一致，那么将按照木桶效应取最低节点的容量作为该卷的总容量。在配置复制卷时，复制数必须等于卷中Brick所包含的存储服务器数，复制卷具备冗余性，即使一个节点损坏，也不影响数据的正常使用。
如下图所示，将文件存放在服务器里，File1和File2同时存放在Server1和Server2上，相当于Server2中的文件是Server1中文件的副本。
GlusterFS Distributed File System overview

复制卷具有以下特点：

卷中所有的服务器均保存一个完整的副本；

卷的副本数量可由客户创建的时候决定；

至少有两个块服务器或者更多的服务器；

具有冗余性；

创建复制卷：

[root@centos01 ~]# gluster volume create rep-volume replica 2 transport tcp server1:/dir1 server2:/dir2
<!--创建名为rep-volume的复制卷，文件将同时存储两个副本，
分别在Server1:/dir1和Server2:/dir2两个Brick中-->
Creation of rep-volume has been successful
Please start the volume to access data

4、分布式条带卷

Article distributed with both volumes and striped volumes distributed function, mainly for large file access processing, distributed create a striped volume requires a minimum of four servers.
As shown below, File1 and File2 positioned by the distributed function and volume respectively to Server1 Server2. From Server1, the File1 divided into four segments, wherein the 1, 3 Server1 exp1 directory; 2,4 exp2 Server1 in the directory. In the Server2, the File2 is also divided into four segments, in which 1, 3 Server2 exp3 directory, 2,4 exp4 Server2 in the directory.
GlusterFS Distributed File System overview

Create a strip distributed with volume:

[root@centos01 ~]# gluster volume create dis-stripe stripe 2 transport tcp server1:/dir1 server2:/dir2 server3:/dir3 server4:/dir4
<!--创建了一个名为dis-stripe的分布式条带卷，配置分布式的条带卷时，
卷中Brick所包含的存储服务器数必须是条带数的倍数(>=2倍)-->
Creation of rep-volume has been successful
Please start the volume to access data

When creating a volume, if the number is equal to the storage server or replicate strip, create the striped or replicated volume; if the number of the article storage server strap 2 times or even more replication volume, then creates distribution formula striped or distributed replication volume.

5, distributed replicated volumes

Distributed Replicated Volume balance between functionality and volume of distributed replicated volumes, mainly used in case of need redundancy.
As shown below, File1 and File2 positioned by the distributed function and volume respectively to Server1 Server2. When storing File1, File1 according to characteristics of the replication volume, there will be two identical copies, respectively, and Server2 Server1 exp1 directory in the directory exp2, when storing File2, File2 according to characteristics of the replication volume, there will be two identical copies, respectively Server3 in exp3 directory and Server4 in exp4 directory.
GlusterFS Distributed File System overview

Creating a distributed replicated volumes:

[root@centos01 ~]# gluster volume create dis-rep replica 2 transport tcp server1:/dir1 server2:/dir2 server3:/dir3 server4:/dir4
<!--创建了一个名为dis-rep的分布式条带卷，配置分布式的复制卷时，
卷中Brick所包含的存储服务器数必须是条带数的倍数(>=2倍)-->
Creation of rep-volume has been successful
Please start the volume to access data

If there are 8 servers, when the replica is 2, the order of the server list, the servers 1 and 2 as a replication server 3, and 4 as a copy, the server 5 and 6 as a copy, the server 7 and 8 as a copy ; when the replica is 4, the order of the list of the server, the server 1/2/3/4 as a copy, a copy server as 5/6/7/8.

About GlusterFS to build a distributed file system cluster configuration will be detailed in the next blog post! ! !

-------- end of this article so far, thanks for reading --------

GlusterFS Distributed File System overview