A, MFS Distributed File System Description
1. What MFS that?
MooseFS is a Fault tolerant, distributed network file systemThat combines data are stored on multiple physical servers (a separate disk or partition) to ensure that there is a data backup copies. For access to the user (client), the entire distributed network file system looks like a cluster resource.
- Distributed File System benefits: centralized access, simplify operation, data disaster recovery, improved file access performance .
feature:
- mfs is a hierarchical tree structure
- Storage Support POSIX standard file attributes (permissions, last access and modification times)
- Support special file, such as: block, character equipment, pipes and sockets, file link (symbolic links and hard links)
- Supports access to the file system based on IP address and password
2, MFS features
- High reliability : a plurality of copies of each data may be provided, and may be stored on different hosts
- Scalability : it easy to dynamically expand storage capacity by increasing the entire file system disk capacity or increase the number of host host
- Highly fault tolerance : Deleted files can be reserved (a file system-level recycle bin) based on a configurable time period
- High data consistency : Even when the file is written to / visit, we can still complete a consistent snapshot of the document
3, MFS relevant principles
Distributed Principle
Distributed File System is to some shared files across multiple computers folder, the set time to a shared folder, users want to access these folders, just open a folder, you can see all Link to this shared folder within a folder.
MFS principle
MFS is a fault-tolerant network distributed file system, which is stored in the data dispersed across multiple physical servers, and it is presented to the user in a unified resource.
4, the four components
MooseFS file system mainly consists of four components, namely, the management server, log server metadata, the data storage server, a client , relevant explained as follows:
-
Management server (Master Server)：The master server manages the entire file system mfs,apart from Distribution UserExternal request, metadata is also used to store information for each data file the entire file system, the size of metadata (metadata) information includes a file (also can be a directory, socket, pipes, equipment, etc.), properties, etc. File Location Path and copying the data file space recovery and restoration of the control of multiple chunk server node . Similar master server load balancing lvs, lvs only difference is that the algorithm based on distribution request, and master to distribute requests based on memory of metadata information. This master can have only one state in the active work.
-
Metadata log server (metalogger Server) : Change the metadata backup management server master log file, the file type changelog_ml. *. MfsIn order to carry out work on the master server when there is a problem, you can go through a simple operation make the new master server. This is very similar to the Mysql master-slave synchronization, but he was not mysql application data locally from the library above, but only receives metadata information file related to record when a file is written to the primary server. This backup can have one or more, similar lvs from the load balancer.
-
Data storage server (Chunk Servers)：After any number of commercial servers, to store the data file, the data file is written to a data server, will synchronize the backup data on another server according to an algorithm.
-
The client (Client): Any number of hosts, you can mfsmount process and the management server (receive and change metadata) and data server (to change the actual file data) to communicate
5, MFS data processing
FIG Master Server as represented by triangles, Chunk Server represented by a circular, Client represented by a square .
5.1 reading process process
Reading process, The following four steps:
- client客户端访问主服务器master，获取文件实体的位置等相关信息。
- 主服务器master查询缓存记录，把文件实体的位置等相关信息（Chunk Server的IP地址和Chunk编号）发给client客户端。
- Client客户端根据拿到的信息去访问对应的存储实体数据的服务器（data servers或者chunk servers）。
- 存储实体数据的服务器（data servers或者chunk servers）把对应的数据返回给Client客户端。
When a plurality of read data MFS client, the server acts as a Master to these clients distribution routes guiding action, the return data returned to the requesting client terminal directly by different data servers, such patterns can greatly mitigate system and network bottlenecks primary server, increasing the throughput of the entire system.
5.2 process writing process
Writing process,As shown below:
- Client客户端访问主服务器master，请求写入数据。
- 主服务器master查询缓存记录，如果是新文件，则会联系后面的数据服务器创建对应的chunk对象准备存放文件。
- 数据服务器返回创建对应的chunk对象成功给主服务器。
- 主服务器master把文件实体的位置等相关信息发给client客户端。
- Client客户端访问对应的数据服务器写数据。
- 数据服务器之间进行数据同步，互相确认成功。
- 数据服务器返回成功写入信息给Client客户端。
- Client客户端回报给主服务器master写入结束。
5.3 to delete data process
5.4 modify the data process
6, MFS default port
| port | effect |
|---|---|
| 9419 | metalogger listening port address (default is 9419), and log data source binding. Regular and master synchronization data terminal |
| 9420 | Chunkserver connected to port address (default is 9420), the communication node |
| 9421 | The port address of the client connected to the external (default is 9421) |
| 9422 | MFS chunck and MFS Client side communication port |
| 9425 | MFS master web interface listening port, view the overall operational status |
