A reference to an object store, you probably think of "store massive files." But with the widespread deployment scenarios of recent years, big data, cloud computing, business agility requirements of customers increasingly high, as much as possible to streamline business processes and improve business efficiency becomes more important. Faced with these changing needs, living behind the storage device can not be limited to "store" the basic requirements. Some of the leading object storage vendors have begun to think, how to originally external business processing capabilities into storage up, so do storage "more thing," and with business requirements enabling applications to help customers achieve greater value.
In order to meet customer requirements for part business agility and efficiency in handling the massive picture scene, SandStone MOS new picture processing functions in V5.4 version, so that the entire business process images processing are completed within the object store, let the process processing from "inefficient red tape" to "simple and efficient." This article is the characteristic thematic interpretation.
First, the photo processing business led to massive high data throughput
As the number of enterprises of all types of unstructured file documents, photographs and audio and video gradually increase ten million to hundreds of millions of rapid growth level, and continued to show exponential explosion of the distributed object store has been gradually replace the traditional NAS storage, become the standard of today's unstructured data stored within the enterprise. Unstructured data (especially image files), are usually closely related business processes. For example, the bank's housing loan approval business, application materials submitted by buyers usually contains various documents scanned copy of ID card, household, personal income and proof of purchase contracts, and in the form of pictures of long-term archiving. The entire approval process includes a variety of processing demands of the picture, such as to improve browsing efficiency by thumbnails, by playing a watermark in order to retroactively Source, as well as to fit different client display specifications by adjusting the image size. To meet the customer's picture storage, extraction, processing needs of this series, the current common object storage system is how to achieve it?
Figure 1. Based on common object storage architecture image processing business
As shown in FIG. 1, Common Object storage system stores only the picture file service, when the service system needs to acquire images (e.g., thumbnail) after treatment, substantially goes through the following four steps:
-
Business application to send the picture to read the original object storage request
-
Business applications download the original image to the service server
-
Business applications to upload pictures to the original picture processing system to request thumbnails
- Business applications get the original picture thumbnail picture processing system returns
Obviously, the whole process of storage systems and picture processing system for frequent interaction. When the business is actually running lines, due to the original high-resolution images are generally large files, high-definition, the above steps 2 and 3 are typically business network will bring huge traffic pressure, take up a lot of network bandwidth; the other because the process step more, which will increase the system response time, resulting in poor customer access experience.
As described above, the storage system and the object picture processing system independently, the former only responsible for "save", which is only responsible for "treating", and the middle of the "extraction" process inefficiencies. For the user, this is clearly not the best solution.
Cedar Rock based on years of industry practice, to accurately capture this long-standing pain points, and proposes specific solutions.
Figure 2. SandStone MOS-based image processing business structure
2, SandStone MOS inside the object by the storage system integrated image processing, picture processing procedure will be omitted during original picture service network on the copy back and forth. The advantage of this is: not only relieve pressure on the business network bandwidth, improve system responsiveness and interactive experience; also be able to give full play to the surplus of computing power storage node CPU resources to ensure efficient use of storage performance.
Second, the integrated image processing capabilities, intelligent storage upgrade
Now that you know SandStone MOS unique in image processing, then it is how to achieve it?
Figure 3.SandStone MOS Image processing business architecture implementation
As shown in FIG. 3, SandStone MOS bottom of the storage system common server hardware, each storage drives, object store on the gateway server is responsible for managing an OSD responsible S3 object storage logic package interface and interact with the read and write the OSD data, the load balancer is responsible for implementing write request traffic routing and availability load balancing among a plurality of objects stored in the gateway.
Object storage gateway when sending read and write requests to the service application load balancer through S3 SDK, load balancer according to the routing algorithm (e.g., poll) assigns the request to a corresponding object store and the gateway interacts with the OSD data is completed to read write.
Further, SandStone MOS object storage module in addition to providing the gateway S3 Object Access Protocol, supports commonly used image processing functions, including image transcoding, image compression, and a thumbnail image text watermark, specifications are as follows.
-
Picture interface supports transcoding to convert the picture into JPG, PNG, BMP, WEBP, GIF, TIFF and other formats
-
Image compression interface supports JPEG, PNG format. After slimming the same quality, same resolution, same format, greatly reduce the file size
-
Thumbnail interface supports image format conversion, thumbnail, clipping function. Simply fill in a few parameters, you can operate the picture thumbnail, generate a variety of thumbnails, the interface can support the processing of raw image formats: PSD, JPG, PNG, BMP, WEBP, GIF, TIFF
- Text watermark interface supports image types include: JPEG, PNG, PSD, GIF, TIFF, BMP
When the object storage system integrated image processing functions, by standard business applications S3 access entry, storage and processing procedure to complete the picture, business applications only need to read the original image S3 interface corresponding increase in processing parameters can be get the processed image.
Third, the media processing capacity increased by 50% to help customers improve efficiency
Simplified business structure, reduce hardware costs
In ordinary business object storage architecture, because the object itself does not support image storage processing, storage and processing of images were required to access the object storage service address and photo services address, which leads to complex business logic needs to be done for the two systems of logic , low efficiency of business processes. By using SandStone MOS distributed object storage systems, business applications only need to maintain an access address and the SDK can be, the overall structure has been greatly simplified, effective streamlining of IT equipment, reduce hardware costs.
Free network bandwidth pressure to ensure stable performance
SandStone MOS integrated image processing by the object within the storage system, eliminating the need to process image processing on the original image copy of the network traffic back and forth, the pressure reduces the bandwidth of the network service, other service systems to avoid crowding leads due to bandwidth risk of paralysis or even access Caton, effective protection of sustained and stable performance.
Picture greatly shorten the processing time of the request, make business more agile
By avoiding cross-network access business, which eliminates the need for interaction between the original picture storage system and the picture processing system, picture processing tasks can be completed within the nearest SandStone MOS. Contrast to the common object storage solution, under the same premise power operator, image processing overall efficiency SandStone MOS can enhance up to 50%, great protection business agility to help clients respond to rapid processing needs in a massive scene pictures .
Give full play to force surplus CPU count, increase hardware utilization
Usually in the business pressure is relatively low scenario, or for reasons of storage system expansion planning, resource utilization, CPU server hardware is usually only about 20% -30%, SandStone MOS image processing architecture can not only reduce traffic across copy data network, enhance picture processing efficiency, but also to give full play to the surplus computing power of CPU resources on the storage node, ensure adequate force protection under the multi-service operator scenario.
SandStone MOS processing capabilities by integrating images, while convenience store massive files, make images of the whole process of business processes within the object store completed, avoid the frequent interaction of traditional programs stored in the picture processing system, not only well done. " storage, "the basic requirements, but also to combine the needs of the scene, and efficient process to complete the picture in the internal memory to help customers achieve greater value.