Ceph can provide file, block and object storage are three types of forms, but most of the major storage form is block storage . Ceph block storage may interface directly with cloud computing platform, such as OpenStack like. Further, Ceph provides access block storage API and kernel modules. Kernel module solves the problem of access Ceph block storage through the bare metal. This paper describes about the client user mode memory block architecture and basic read process, then the subsequent detailed kernel module implemented in the client.
1 Ceph storage block of FIG.
Before introducing the client to realize we need to introduce the basic principles of Ceph block storage implementations. We know that block storage (disk) in the user-side rendering is actually a linear space, the blue bar shown in Figure 2. Ceph approach is to cut the space for the same length of the segment, and is represented by an object store, and the object name is the disk LBA.
FIG 2 Ceph block storage principle
For example, we created a 1G memory block, and using Ceph default policy (4MB each with an object storage). So, if there is write data (such as 1KB) at the beginning of the disk, the client will create an object of 4MB, the name of the format image_hash_lba. 3 is an example of a target block storage for storing data, LBA red box where the content of the data is.
FIG 3 stores the name of the object instance
In general, the block is converted to Ceph storage to store the object .
Ceph block storage API
Before subsequent analysis block storage architecture, we first look at the client's API and use. Here are some code fragments, this code is not complete, there is no exception handling, merely to illustrate the basic flow.
Aware of the above process, we know that there are associated with the block stored in two categories, namely, RBD and Image. Wherein the storage system is the block RBD associated class to implement the relevant block storage management operations , such as creating a mirror delete, create a replication pair mirroring group and the like mirroring operation. The Image class is specific mirror (disk) for operations, such as read and write data mirroring, snapshot and mirroring properties create delete and so on.
Ceph block storage client basic structure
在介绍块存储架构之前，我们先看一下块存储的使用流程。从上面代码可以看到这里核心的有2点，一个是创建一个IO上下文，这个是实现对象级数据读写的，我们在librados中曾经介绍过；另外一个是通过rbd.open打开一个块存储设备，而这里面最为重要的是传入了IO上下文参数（IoCtx）。
如果我们再继续深入介绍之前，我们先看一下rbd.open函数，如下是该函数的源代码（省略了部分不重要的内容）。从代码中可以看出这里的核心是创建了一个ImageCtx对象，并且通过参数传出。而传出的参数就是我们后面需要使用的访问磁盘的对象。同时，这里还需要注意的是构建ImageCtx对象时讲IO上下文传入其中，因此后续的IO操作就可以通过该上下文接口实现。
对于Ceph块存储客户端可以简单的划分为3层，分别是API接口层、块存储逻辑层和lirados层。当然，实际情况比这个要复杂的多（例如前文librados就包含4层），我们这里为了简化描述，暂且认为只包含这3层。后续，我们将深入介绍没一层。
图2 块设备基本架构
结合上述3层架构，我们看一下每层涉及的类。在API层主要包括RBD和Image 2个类，关于这两个类的作用我们在前文已经做过介绍，这里不在赘述。
图3 块存储结构类图
在块存储逻辑层最主要的类是ImageCtx类，其中主要包含CephContext、ImageRquestWQ和ObjectDispatcher类。其中ImageCtx类为块设备（Image）上下文类，该类负责处理块设备相关的，核心是实现块设备线性空间到对象存储之间的转换。除此之外，还包括一些高级功能，包括磁盘镜像、快照和磁盘组等。
ImageRequestWQ above class is a class queue IO related asynchronous request interface layer is cached to the queue, and IO via the transmission interface from inside. ObjectDispatcher class is responsible for the request (object request) is sent to the cluster RADOS converted.
On the write data flow
Understand the basic architecture of RBD in front of the main classes involved and, let's write data as an example to explain the main processes. The writing process is initiated write function of the Image class began. Throughout the call flow shown in Figure 4.
FIG 4 basic flow of data write
Details we will not be introduced here point to note is to call a function named file_to_extents, the function will block device IO IO request into a request for an object. In this function, it will be the IO IO cleavage sequences suitable for operation of the object image according to the attribute and the offset of a block layer IO. And further to the lower layer specific processing.
Well, first here today, and today we introduce the basic architecture and write data flow Ceph block storage client, and class involves a simple introduction. Follow-up, we will further in-depth details, description block storage to achieve critical processes.