A file system is a method of storing and organizing computer files, directories, and the data they contain, which simplifies finding and accessing files, directories, and data. If you are using a single computer, it is likely that more than one file system is in use. The file system can provide rich scalability. It can be written as a wrapper around the underlying filesystem to manage data within it and provide an enhanced, feature-rich filesystem (such as cvsfs-fuse, which provides a filesystem interface to CVS; or Wayback file system, which provides a file backup mechanism for preserving original data files).
Filesystem development was the work of kernel developers before the advent of userspace filesystems. Creating a filesystem requires knowledge of kernel programming and kernel technologies such as vfs. Debugging requires expertise in C and C++. But other developers need to be proficient in manipulating the file system to add personalization features (such as adding history or forward caching) and improving it.
Introduction to FUSE
With FUSE you can develop a fully functional file system: it has a simple API library, can be accessed by unprivileged users, and can be implemented securely. More importantly, FUSE's past performance fully demonstrates its stability.
With FUSE, you can develop filesystems like executable binaries, which need to be linked against the FUSE library -- in other words, the filesystem framework doesn't require you to know filesystem internals and kernel module programming.
As far as filesystems are concerned, userspace filesystems are no longer a novel design. Examples of commercial and academic implementations of userspace filesystems include:
LUFS is a hybrid userspace filesystem framework that provides transparent support for a myriad of filesystems for any application. Most LUFS consists of a kernel module and a user-space daemon. Fundamentally, it delegates most VFS calls to a dedicated daemon.
UserFS allows user processes to mount like a normal filesystem. This conceptual prototype provides ftpfs, which can provide anonymous FTP access using a file system interface.
The Ufo Project is a global file system for Solaris that allows users to treat remote files as if they were local files.
OpenAFS is an open source version of Andrew FileSystem.
CIFS is short for Common Internet FileSystem.
Unlike these commercial and academic implementations, FUSE brings this filesystem design power to Linux. Because FUSE uses executable programs (rather than shared objects like LUFS), it simplifies debugging and development of programs. FUSE is available on 2.4.x and 2.6.x kernels and now supports Java™ bindings, so you can write filesystems without being limited to C and C++. (See Resources for more on userland filesystems using FUSE.)
To create a filesystem in FUSE, you need to install a FUSE kernel module, then use the FUSE library and API to create your own filesystem.
To develop a filesystem, first download the FUSE source code (see Resources) and expand the package: tar -zxvf fuse-2.2.tar.gz. This creates a FUSE directory, which holds the source code. The contents of the fuse-2.2 directory are as follows:
./doc contains documentation related to FUSE. Now, there is only one file, how-fuse-works.
./kernel contains the source code for the FUSE kernel modules (of course you don't need to know the mechanics of these codes for developing a filesystem with FUSE).
./include contains the FUSE API headers, which you need to create the filesystem. The only thing you need now is fuse.h.
./lib holds the source code to create the FUSE libraries, which you need to link with your binary to create the filesystem.
./util is the source code of the FUSE tool library.
./example of course contains some examples for your reference, such as fusexmp.null and the hello filesystem.
Compile and install FUSE
Run the configure script in the fuse-2.2 directory: ./configure. This creates the required makefiles etc.
Run ./make to compile libraries, binaries and kernel modules. Look at the file ./kernel/fuse.ko in the kernel directory - this is the kernel module file. Also look at fuse.o, mount.o, and helper.o in the lib directory.
Run ./make install to complete the FUSE installation.
Another option: you can skip this step if you wish to install the module into the kernel yourself using insmod. For example: /usr/local/sbin/insmod ./kernel/fuse.ko or /sbin/insmod ./kernel/fuse.ko. Remember to use root privileges to install the required modules.
If desired, the above steps can be completed in only one step. In the fuse-2.2 directory, run ./configure; make; make install;.
IMPORTANT: When compiling FUSE, you need to have kernel headers or source code on your system. For simplicity, make sure to put the kernel sources in the /usr/src/ directory.
custom file system
Now let's create a filesystem so that an older Linux kernel can be used to access the AFS space on a Linux system with the latest kernel. You need two processes: a server process running on an older Linux kernel, and a FUSE client process running on a Linux system with the latest kernel. Whenever a request arrives on your FUSE client process, it contacts the remote server process. To communicate, this filesystem uses the RX RPC code, which is part of AFS, so you need to compile OpenAFS.
AFS-FUSE file system overview
Compile OpenAFS
Download the OpenAFS Linux source code and expand the source code.
In the directory where the source code was expanded, run ./make ./configure --enable-transarc-paths. If ./configure does not understand the sysname used for compilation, use the --with-afs-sysname option to provide the correct sysname.
To compile on the Linux 2.4 kernel, use the following command: ./configure --enable-transarc-paths --with-afs-sysname=i386_linux24.
Run ./make, then ./make dest. Check for errors during compilation.
If all goes well with the compilation process, the AFS source tree is ready to use. Now you need to prepare a development directory afsfuse. Inside this directory, create two other directories:
The include directory includes the header files for OpenAFS and FUSE's include directories.
The lib directory contains library files for OpenAFS and FUSE.
Copy the header and library files.
First, copy the AFS header files from the OpenAFS directory by copying all the directories and files in dest\i386_linux24\include to the include directory. Then copy the FUSE include directory in the fuse-2.2 directory to this directory. Repeat the same steps for the library files, copying them all into the lib directory.
Create the structure of the application.
For both sets of processes, you need to use two sets of files. Use the naming convention afsfuse_client.* to name files for the client process; use the naming convention afsfuse_server.* to name files for the server process.
So you have an afsfuse_client.c file that contains the code for the FUSE process; an afsfuse_server.c file that contains the server code used by the process running on the remote machine; a makefile; an rxgen file that creates RPC header files (eg afsfuse.xg).
The afsfuse_client.c file creates the afsfuse_client process code, which is called by the FUSE filesystem to create the filesystem (use fuse-2.2/example/fusexmp.c to create this file).
define the required functions
To use FUSE to create a filesystem, you declare a structure variable of type fuse_operations and pass it to the fuse_main function. There is a pointer in the fuse_operations structure to the function that needs to be called when the appropriate operation is performed.
Create client and stub files
Next use rxgen to compile the afsfuse.xg file to create the client and stub files. From the directory containing the source code for afsfuse_server and afsfuse_client, run the command openafs-1.2.13/i386_linux24/dest/bin/rxgen afsfuse.xg. This creates the following files:
afsfuse.cs.c is the client stub code that links with afsfuse_client.c.
afsfuse.h is a header file that contains various definitions for your FUSE RX code.
afsfuse.ss.c is the server stub code (proxy code) that links with the afsfuse_server code.
afsfuse.xdr.c contains the code used to handle the three structures defined in afsfuse.xg.
Now add some code to afsfuse_client.c and afsfuse_server.c that does the actual work. Most of the calls look like this:
Our_call_in_afs_fuse_client(). Parse the parameters and prepare to execute the RPC. Call afsfuse_server on RX [RPC]. Combining parameters. Copies these values into the row parameter passed to this function.
Our_call_in_afs_fuse_server(). Combining parameters. Call a function specific to the local file system or AFS. Analyze parameters to prepare for RPC execution. Generate RX RPC calls.
The afsfuse_client.c call looks like this:
int afsfuse_readlink(const char *path, char *buf, size_t size){
rx_connection *local& int ret& char *buffer = malloc (512)&
memset(buffer,0,512)& memset(buf,0,size)& local = getconnection()&
ret = rxc_rxc_readlink(local,path,512,&buffer) // rpc call
relconnection(local)&
strncpy(buf,buffer,512-1)&
// <- demarshall the paramets
return ret&
}
The afsfuse_server.c call looks like this:
Listing 3. afsfuse_server.c call
int rxc_rxc_readlink( struct rx_call *call, char * path, afs_uint32 size, char**data)
{ int ret& char lbuff[512] ={0}&
translatePath(path,lbuff)& //<- make filesystem call
*data = malloc(512)&
res = readlink(lbuff, *data, 512-1)&
if(res == -1) return -errno& (*data)[res] = '\0'& return 0&
}
Simply, you can add code in other functions to enhance the file system.
You need to create a makefile to compile the code. Remember to include the following options when compiling the code for afsfuse_client: -D_FILE_OFFSET_BITS=64 and -DFUSE_USE_VERSION=22.