A Quick Look at Object Storage

A Quick Look at Object Storage

Common Storage Types

1. SAN storage -> block storage

A storage method that directly faces the underlying disk, based on sectors, directly accesses the hardware media that stores data

Typical equipment: disk array, hard disk, virtual hard disk

Advantages: High performance, low latency Reason: Directly facing the bottom layer

Disadvantages: the price of the san system is high, and the scalability is relatively poor

2. NAS storage -> file storage

File storage has a file system, which mainly stores data in the form of files, and uses files as the transmission protocol to manage all directories and files in a hierarchical tree structure. Find the files you need and realize network storage through TCP/IP

Typical equipment: FTP, NFS server

Advantages: good scalability, cheap price, easy user management

Disadvantages: low bandwidth, high latency, not conducive to use in high-performance clusters

3. Object storage

Simply put, block storage reads and writes quickly, which is not conducive to sharing

File storage reads and writes slowly, which is good for sharing

Can you get a fast read and write, which is good for sharing? So there is object storage, which combines the advantages of NAS and SAN

3.1 Features

• unstructured storage

Suitable for storing unstructured data, such as documents, text, pictures, XML, HTML, various reports, audio and video information, etc. that are seen in daily life are all unstructured data

• multi-node

Take Tencent Cloud Object Storage OSS as an example. The storage computer rooms of OSS are spread all over the world. In short, the system will select the nearest computer room for users to store and speed up data transmission and access.

• Flat Structure –> Bucket

There is no concept of folders in object storage, and all data is stored at the same level. As mentioned above, for data stored in object storage, there is no need to know where it is stored, and the data can be quickly obtained only through "credentials"

• Elastic expansion

The capacity of object storage is above EB level; 1EB ≈ 1*10^9 GB. The capacity is still rising. Simply put, no matter how much data you have, you can store it with confidence.

1、1GB=1024MB

2、1TB=1024GB

3、1PB=1024TB

4、1EB=1024PB

3.2 Basic Concepts ⭐️⭐️⭐️⭐️⭐️

Bucket+File Key

• Bucket: It is a logical concept, and files are stored in this bucket. It can be understood as a parent folder, and files are stored under this parent folder
• File Key: It can understand the file name, which is the globally unique identifier (UID) of the object. The server and users do not need to know the physical address of the data, and can also use it to obtain files from the bucket, delete files, etc.

• data: the data body of the file
• Metadata
  • Metadata: A bit similar to data tags, there is no limit to the type and number of tag entries, which can be various description information of objects
  • If the object is a photo of a person, the metadata can be name, gender, nationality, age, location, time, etc.
  • In traditional file storage, this type of information belongs to the file itself and is packaged and stored together with the file. In object storage, metadata is independent and not encapsulated inside the data.
  • The benefits of metadata are very obvious, which can greatly speed up the sorting of objects, as well as classification and search

3.3 Traditional file storage VS object storage

We can think of the process of reading and writing unstructured data as the process of parking in a parking lot and finding a car

File Storage Equivalent to Self-Parking

Object storage is the equivalent of valet parking

• traditional file storage

File storage is like self-parking. You need to find the location (directory) of the parking (file) by yourself, record the route (path) to find the car, and find your car through the recorded route when picking up the car.

• object storage

The staff (object storage) will park the car (object) for you at a certain location in the parking lot (barrel) according to certain rules, and give you a voucher (object ID). When picking up the car, you only need to show the parking voucher Let the staff take out your car for you, and you don't need to know where the car is parked

3.4 Application scenarios

Object storage is suitable for storing unstructured data such as documents, pictures, and videos, and it makes data use more convenient

• traditional way
  • The teaching video is stored in the computer of each lecturer. Before the lecture, it needs to be copied to the classroom with a USB flash drive for playback.
  • The teaching video is stored in a certain network disk, which needs to be downloaded before the lecture, or the application of the network disk is installed on the classroom computer, which supports online preview and playback
• object storage
  • The video is stored in the object storage, and the corresponding object address (simply understood as a link) is obtained. In class, open the browser, enter the object address, and play online

3.5 Common Object Storage

• Alibaba Cloud OSS
• Tencent Cloud COS
• HUAWEI CLOUD OBS
• Qiniu Yun
• SM.MS
• GitHub