With the development of Internet, cloud computing and big data such as information technology, more and more applications rely on the storage and processing of massive data, such as intelligent monitoring, electronic commerce, geographic information, etc. These applications require a massive picture storage and retrieval. Because pictures are mostly small files (less than 80% the size of several MB), to GFS, HDFS as the representative for distributed storage system streaming access large files, if directly used to store pictures, due to the expansion of the metadata in the extended and performance are serious problems.
In order to solve the problem in small HDFS file storage area, the usual practice is to first merge many small files into one large file and then saved to HDFS, while the index for these small files for quick access. Typical technologies include Hadoop comes Archive, SequenceFile, but all require the user to write their own procedures to achieve with small files. In order to achieve a small file merge transparent to the user, need to solve the problem HDFS small file system level. Thesis explored for specific application scenarios, but not universal.
SandStone MOS massive image Solutions
For mass storage problem of picture data, cedar rock mass object storage (SandStone MOS) solution uses a distributed architecture to the center, while leveraging software-defined approach to achieve the capacity expansion of hundreds of PB grade scale under a single name space conditions, business anytime, anywhere access and unlimited data storage location.
Small in lifting massive file access performance, SandStone MOS use hashing to achieve an efficient level access to hundreds of millions of files. For document retrieval difficult, SandStone MOS Support tab function automatically sets the label file storage, combined with the business in order to better meet the efficient retrieval.
In addition, SandStone MOS ease of use and maintainability are also beyond the same level of product, which uses "x86 general-purpose servers + storage software" decoupling distributed architecture, the underlying storage space and the upper space to separate business logic, soft the hardware upgrade will not affect the normal operation of the entire system.
Even if the system has to use more applications update, it will not affect the storage space. It is worth mentioning that, SandStone MOS's first distributed data storage disk roaming feature that can help business users will be gradual replacement of old hardware, does not affect the normal operation of the business.
In the object storage area of focus and continuous innovation, the result is highly recognized by the industry. Last August, Shan Yan became the first data center computing innovation tested by Intel released a report China cloud and software-defined storage vendors.
At the same time, as a cloud data fir Rock National Information Technology Standards Committee computing standards working group members units and China Open Source Cloud Alliance governing units, it has participated in the preparation of distributed storage and formulation of national standards Ceph industry white paper.
Today, the object storage being brought out more vitality, a storage revolution is on. Fir rock data represented innovative storage vendors, is by constantly cultivating object storage areas, to help companies build smarter IT infrastructure, promote the traditional storage towards a more efficient, more intensive, comprehensive evolution direction easier to use.