Article directory
foreword
FastDFS is an open source lightweight distributed file system. It manages files. Its functions include: file storage, file synchronization, file access (file upload, file download), etc. It solves the problems of large-capacity storage and load balancing. It is especially suitable for online services with files as the carrier, such as photo album websites, video websites and so on.
FastDFS is tailor-made for the Internet, fully considers redundant backup, load balancing, linear expansion and other mechanisms, and pays attention to high availability, high performance and other indicators. Using FastDFS is easy to build a set of high-performance file server clusters to provide file upload and download and other services.
Equipping a public network server as a file upload and download costs a lot of resources and overhead. If you build the FastDFS service locally, you only need to use the cpolar intranet port forwarding tool to configure the file to be accessed through the Internet, which greatly reduces Server fees and overheads, this article introduces the detailed tutorial of building FastDFS, and combines Nginx+cpolar tools to realize remote access to files.
1. Build the FastDFS file system locally
1.1 Environment installation
Since fastDFS is written in c language, we need to installgcc
yum -y install gcc-c++
FastDFS is dependent libevent库
and needs to be installed. Execute the following command:
yum -y install libevent
1.2 Install libfastcommon
Enter the local directory
cd /usr/local
libfastcommon is officially provided by FastDFS, and libfastcommon contains some basic libraries needed for FastDFS to run. Click the link below to get it:
https://github.com/happyfish100/libfastcommon/releases/tag/V1.0.7
After getting it, upload it to local
the directory
Unzip libfastcommon
tar -zxvf libfastcommonV1.0.7.tar.gz
Go to the unzipped folder
cd libfastcommon-1.0.7/
compile
./make.sh
Install
./make.sh install
After libfastcommon is installed, it will automatically copy the library files to the next /usr/lib64
directory. Since the FastDFS program refers to the usr/lib directory, it is necessary to copy the library files under /usr/lib64 to /usr/lib.
cp /usr/lib64/libfastcommon.so /usr/lib/
1.3 Install FastDFS
Enter the local directory
cd /usr/local
Click the following address to download the FastDFS_v5.05.tar.gz file:
https://sourceforge.net/projects/fastdfs/
After downloading the file, upload FastDFS_v5.05.tar.gz
it to/usr/local/
unzip files
tar -zxvf FastDFS_v5.05.tar.gz
Go to the unzipped folder
cd FastDFS
compile
./make.sh
Install
./make.sh install
After the installation is successful, copy the files under conf in the installation directory to /etc/fdfs/
the following
cd conf/
copy
cp * /etc/fdfs/
1.4 Configure Tracker
Next, configure and enter /etc/fdfs
the directory after the installation is successful
cd /etc/fdfs/
Modify tracker.conf
vim tracker.conf
base_path=/home/yuqing/fastdfs
change tobase_path=/home/fastdfs
Save and exit after editing, and then create a correspondingfastdfs文件夹
mkdir /home/fastdfs
then start
/usr/bin/fdfs_trackerd /etc/fdfs/tracker.conf restart
Set boot
vim /etc/rc.d/rc.local
Add the run command line to the file
/usr/bin/fdfs_trackerd /etc/fdfs/tracker.conf restart
1.5 Configuring Storage
Enter the /etc/fdfs directory
cd /etc/fdfs
Modify storage.conf
vim storage.conf
base_path=/home/yuqing/fastdfs
change tobase_path=/home/fastdfs
store_path0=/home/yuqing/fastdfs
change tostore_path0=/home/fastdfs/fdfs_storage
Continue to modify, set the tracker server: IP, tracker_server=Linux LAN ip: 22122, save and exit after setting
Create the corresponding fdfs_storage folder
mkdir /home/fastdfs/fdfs_storage
then start
/usr/bin/fdfs_storaged /etc/fdfs/storage.conf restart
Set up automatic startup
vim /etc/rc.d/rc.local
Add the run command line to the file, then save
/usr/bin/fdfs_storaged /etc/fdfs/storage.conf restart
1.6 Test upload and download
If FastDFS is installed successfully, you can test upload and download operations through the /usr/bin/fdfs_test command.
Modify /etc/fdfs/client.conf
vim /etc/fdfs/client.conf
Modify the following content, then save:
base_path=/home/fastdfs
tracker_server=Linux局域网ip:22122
Next, test uploading files. Here, a picture is uploaded to /usr/local
the directory for use as a test.
FastDFS upload file format:
/usr/bin/fdfs_test /etc/fdfs/client.conf upload+ path to upload files
Then upload the test23.png image to FastDFS according to the format
/usr/bin/fdfs_test /etc/fdfs/client.conf upload /usr/local/test23.png
After execution, we can see the return address information, which means success, but we can’t access it yet, we need to combine Nginx to access and download
1.7 Integration with Nginx
Click the download link below to download fastdfs-nginx-module_v1.16:
[https://sourceforge.net/projects/fastdfs/files/FastDFS%20Nginx%20Module%20Source%20Code/fastdfs-nginx-module_v1.16.tar.gz/download](https://sourceforge.net/projects/fastdfs/files/FastDFS Nginx Module Source Code/fastdfs-nginx-module_v1.16.tar.gz/download)
Enter the /usr/local/folder
cd /usr/local/
uploadfastdfs-nginx-module_v1.16.tar.gz
Unzip fastdfs-nginx-module_v1.16.tar.gz
tar -zxvf fastdfs-nginx-module_v1.16.tar.gz
Enter src, then edit the configuration file
cd fastdfs-nginx-module/src/
Modify the config file
vim config
Modify these three locations, change the path that was originally /usr/local/ to /usr/, then save and exit
Then copy mod_FastDFS.conf under FastDFS-nginx-module/src to /etc/fdfs/
cp mod_fastdfs.conf /etc/fdfs/
Modify the contents of mod_FastDFS.conf:
vim /etc/fdfs/mod_fastdfs.conf
The modification is as follows:
base_path=/home/fastdfs
tracker_server=Linux LAN ip address: 22122
url_have_group_name=true #url contains group name
store_path0=/home/fastdfs/fdfs_storage #Specify file storage path
Next, copy libfdfsclient.so to /usr/lib
cp /usr/lib64/libfdfsclient.so /usr/lib/
Just create the nginx/client directory
mkdir -p /var/temp/nginx/client
1.8 Install Nginx
First install the serial environment
yum -y install pcre pcre-devel
yum -y install zlib zlib-devel
yum -y install openssl openssl-devel
Then click to download the Nginx package:
Upload to the root directory
Unzip and enter the unzipped folder
tar -zxvf nginx-1.16.1.tar.gz && cd nginx-1.16.
Add the FastDFS-nginx-module module and execute the following command:
./configure \
--prefix=/usr/local/nginx \
--pid-path=/var/run/nginx/nginx.pid \
--lock-path=/var/lock/nginx.lock \
--error-log-path=/var/log/nginx/error.log \
--http-log-path=/var/log/nginx/access.log \
--with-http_gzip_static_module \
--http-client-body-temp-path=/var/temp/nginx/client \
--http-proxy-temp-path=/var/temp/nginx/proxy \
--http-fastcgi-temp-path=/var/temp/nginx/fastcgi \
--http-uwsgi-temp-path=/var/temp/nginx/uwsgi \
--http-scgi-temp-path=/var/temp/nginx/scgi \
--add-module=/usr/local/fastdfs-nginx-module/src
then compile
make
Install
make install
1.9 Configure Nginx
Modify the Nginx configuration file
vim /usr/local/nginx/conf/nginx.conf
Add server, save and exit after adding:
server {
listen 8089;#为了防止80端口,我们改成8089
server_name 192.168.59.133;#这里需要使用你本机的局域网ip
location /group1/M00/{
ngx_fastdfs_module;
}
}
Start Nginx
/usr/local/nginx/sbin/nginx
If the following error occurs when starting nginx:
nginx: [emerg] open() “/var/run/nginx/nginx.pid” failed (2: No such file or directory)
You can execute the following command and start again:
mkdir /var/run/nginx
Set it to start automatically at boot.
vim /etc/rc.d/rc.local
Add the running command line to the file, then save and exit:
/usr/local/nginx/sbin/nginx
turn off firewall
service iptables stop
Set the firewall not to start automatically
chkconfig iptables off
2. LAN test access FastDFS
Then according to the format, test upload the test23.png image we uploaded above to FastDFS
/usr/bin/fdfs_test /etc/fdfs/client.conf upload /usr/local/test23.png
After execution, we can see the return address information, indicating success
Then use the above address to visit in the browser, you can see the pictures we uploaded, and fastDFS is built
3. Install cpolar intranet penetration
Above, we installed the fastDFS distributed file service on the local Linux virtual machine. Next, we installed the cpolar intranet penetration tool. Through the http public network address of cpolar, we can easily access the files uploaded in fastDFS remotely, without having to Register a domain name to purchase a cloud server. The following are the steps to install cpolar intranet penetration
cpolar official website: https://www.cpolar.com/
- Install commands using one-click scripts
curl -L https://www.cpolar.com/static/downloads/install-release-cpolar.sh | sudo bash
- token authentication
Log in to the background of the cpolar official website, click the verification on the left to view your own authentication token, and then paste the token in the command line:
cpolar authtoken xxxxxxx
- Add a service to the system
sudo systemctl enable cpolar
- Start the cpolar service
sudo systemctl start cpolar
4. Configure the public network access address
After successfully starting the cpolar service, we visit the Linux LAN ip address + port 9200 on the browser, and log in to the cpolar web UI management interface.
After successful login, click Tunnel Management on the left dashboard - Create Tunnel, and create an http protocol tunnel pointing to the Nginx8089 port set above:
- Tunnel name: can be customized, be careful not to duplicate the existing tunnel name
- protocol: http
- Local address: 8089
- Domain name type: choose a random domain name for free
- Region: Select China VIP
click创建
Then open the online tunnel list, view and copy the public network address
Then open the browser, enter the public network address, access the files we uploaded in fastDFS, and the remote access is successful
5. Fixed public network address
Since the tunnel just created uses a random temporary address, the address will change within 24 hours. For long-term and more stable remote access, we will configure this public network address as fixed next.
5.1 Reserved second-level subdomains
You need to upgrade to the basic package or above to support the configuration of second-level subdomains. If you need to download files, you can upgrade to a higher package to obtain greater bandwidth and improve download speed.
Log in to the background of the cpolar official website , click on the dashboard on the left 预留
, find 保留二级子域名
, and reserve a second-level subdomain name for the http tunnel.
- Region: Select the server region
- Name: Fill in the second-level subdomain name you want to keep (customizable)
- Description: Remarks, which can be customized
This example reserves a fasttest
second-level subdomain named . After the subdomain name is successfully reserved, we copy the subdomain name, and then we need to configure it in the tunnel.
5.2 Configure the second-level subdomain name
Log in to the cpolar web ui management interface. 隧道管理
Click —— on the left dashboard 隧道列表
, find the tunnel that needs to configure the second-level subdomain name, and click on the right编辑
Modify the tunnel information and configure the second-level subdomain name into the tunnel:
- Domain Type: Select instead
二级子域名
- Sub Domain: Fill in the second-level subdomain name we just reserved (in this example
fasttest
)
After the modification is complete, click更新
状态
After the tunnel is successfully updated, click —— on the left dashboard 在线隧道列表
, and you can see the public network address of the tunnel, which has been updated to a second-level subdomain name, and copy the public network address.
6. Test access to fixed second-level subdomains
Let's test access to the successfully configured second-level subdomain name, open the browser, enter the public network address plus the resource path, access the files we uploaded in fastDFS, and the remote access will be successful if the picture appears. Now, our only private second-level subdomain name in the entire network has been created.
And this address will not change randomly anymore, it is fixed, as long as the tunnel is online, we can use this public network address to remotely access anytime and anywhere, without the need for a public network IP or setting up a router.
Reprinted from the article of cpolar pole point cloud: FastDFS and Nginx combine to build a file server, and internal network penetration to achieve public network access