Fastdfs
concept:
Is an open source distributed file system similar to Google FS is pure C language development.
FastDFS is a lightweight open source distributed file system, its file management features include: file storage, file synchronization, file access (file upload, file download), etc., to solve the high-capacity storage and load balancing issues. Particularly suitable for the carrier to file online services, such as photo album sites, video sites, and more.
FastDFS is designed for Internet applications tailored to the purpose of the pursuit of high performance and scalability. It can be seen as key value pair file-based storage system, called a distributed file storage service more appropriate.
Official Forum http://bbs.chinaunix.net/forum-240-1.html
Architecture design:
There are three roles in FastDFS, Client is the first client, the second is the Tracker tracking server, and the third is the Storage storage server. Client Client which is provided by the official C, Java and PHP call API, this will be mentioned later. For tracking servers and storage servers out more about us here.
About the storage server, by way of a packet, a file server within the same group are identical (for synchronous thread synchronization), it is not the communication between the groups. The storage server will take the initiative to track the current state periodically reports to the server, the server can have multiple tracking between multiple is like, there is no master-slave.
About client is not required to store any information about the storage server. General user's request is not a direct request to the storage server, but will request tracking server . For example, after a client needs to upload a file, this time tracking server will assign a group to identify the client for uploading files from the storage server, in which case the client get information group, and began to direct the storage server group interaction, no longer need to carry out transit through tracking server.
which is:
Tracker Servre: tracking server, mainly to do work scheduling, load balancing role played during the visit. Recording state information in memory, the cluster group and the storage server, and the Client Storage is connected to the hub. Because all the information in memory, Tracker server performance is very high, a relatively large cluster (for example, hundreds of group) had three is enough.
Storage Server: storage servers, files and file attributes (meta data) are stored on the storage server.
Deployment:
A tracker both sets of storage node
Extended Storage Server
Instructions:
Upload mechanism:
- Storge upload status information to the timing tracker
- Client transmits an upload request to trackClient
- trackClient query storge available
- trackClient returned to the client's ip and port information storge
- Client According to the information of the low Ang StorgeServer upload files
- StrogeServer generation file_id, write the contents uploaded to disk
- StorgeServer return File_id (path information, and file name) to the Client
- Client information storage file
Download mechanism:
1) Client inquiries storage tracker download a file, the file parameter identification (the volume name and file name)
2) tracker returns an available storage
3) client communications and storage complete direct file download
Scenario:
FastDFS appropriate storage range between 4KB to 500M, it is more likely to store small and medium sized files such as pictures site, a short video sites, documents, app download station.
FastDFS的用户有支付宝、京东、赶集网、58同城、UC、51CTO和一些网盘公司，可以说目前用到FastDFS的公司特别多，因为移动互联网的兴起，一些短视频、电子书、小音频和一些app，都在十几兆或者一两百兆左右，使用FastDFS十分合适。