Abstract: Mesos provides sophisticated, well-thought-out APIs for many different user scenarios. Persistent volumes are a feature introduced by the new acceptOffers API. Persistent volumes allow users to build database frameworks for Mesos, which can make data persistent in the event of any unforeseen failures and errors that affect the entire system. This article is selected from "Building Distributed Applications with the Mesos Framework".
Until recently, Mesos was only able to run services without storing data to disk. This is because there is no way to reserve the required disk blocks. Since Mesos version 0.23, it is possible to reserve disks.
We can think of Mesos as a deployment system. Wouldn't it be nice if the MySQL database could automatically back up itself and create new copies on demand? Or what if you had a simple, self-service REST API capable of creating new Riak and Cassandra clusters? Work on building a database framework for Mesos began in 2014. The problem with these frameworks is that each host must create a special data partition and manage it outside of Mesos. Using persistent volumes, projects like Apache Cotton (used by MySQL) and the Cassandra and Riak Mesos frameworks can be started and maintained independently.
In the design of Mesos, disk space is ephemeral and is segregated by task. This is usually a good thing unless the user wants to persist data. To solve this problem, Mesos introduces a new subtype of disk resource called volume. A volume is a disk block assigned to a task and mounted at a specific location. The API for this function is almost identical to the Marathon API for mounting host volumes (see "Mounting Host Volumes" for details). Users can even create non-persistent volumes, which can be useful when wanting to expose multiple independent disks to Mesos.
Let's examine how to create and use persistent volumes.
There are two acceptOffers Operations used to create and destroy persistent volumes. Not surprisingly, they are called Create and Destroy. Persistent volumes can only be created on disk resources that have been reserved. Typically, users reserve resources, create volumes, and start tasks in a single acceptOffers, as shown in the following example.
A persistent volume resource is the same as a regular disk resource, but it has the field disk set to the appropriate DiskInfo. DiskInfo names the persistent volume so that it can be mounted with the name of the nested string subfield persistence.id. DiskInfo's Volume must use RW mode (because Mesos 0.24 only supports RW). The container_path field of the Volume specifies the mount location of the container in the task sandbox.
The Persistent Volume API is quite new, so no production frameworks use it yet. It also has some limitations, like volumes must always be mounted as RW, and there is no way to expose multiple disks, nor any disk or I/O isolation. Backward compatibility of the API is guaranteed even after new features and functionality are added. Therefore, projects like Apache Cotton have integrated persistent volumes in their codebase.
This article is selected from "Building Distributed Applications with the Mesos Framework", click this link to view this book on the official website of the blog post. If you want to get more exciting articles in time, you can search for "Blog Viewpoint" in WeChat or scan the QR code below and follow.
