About the author: A cloud computing network operation and maintenance personnel, sharing the technology and dry goods of network and operation and maintenance every day.
Motto: Keep your head down and hurry on your way, be respectful
Personal homepage: Homepage of Netdou
Table of contents
1. Introduction to traditional storage:
1. The development history of storage
2. History and trend of hard disk development
4. Background of RAID technology
1. Storage backup introduction
2. Introduction to storage disaster recovery
Intra-city disaster recovery center:
3. Introduction and usage scenarios of cloud storage OSS
2. Cloud storage classification
3. Alibaba Cloud object storage service oss
4. The position of OSS in the Alibaba Cloud platform architecture
5. Comparison between OSS and self-built server storage
6. Cloud storage OSS application scenario - website/app dynamic and static separation
7. Cloud storage OSS application scenario - mass file storage
8. Cloud storage OSS application scenario - cloud data processing
9. Cloud storage OSS application scenario - data download acceleration
4. Basic overview of cloud storage OSS
(4) Access ID & Access Key (API key)
4. OSS data organization structure
6. Management method of cloud storage OSS
5. Upload and download of cloud storage OSS data
2. Upload files through the Alibaba Cloud console
3. Upload and download files through API
6. Image processing and static website hosting
1. Cloud storage OSS image processing
1. Data security and reliability
3. Cross-region replication steps
4. Security control: Bucket permission control
5. Cloud storage OSS anti-leech
6. Server-side encrypted storage
7. Operations supported by server-side encryption
8. Supported encryption algorithms
learning target:
1. Understand the basic knowledge and common technologies of traditional storage
2. Familiar with the application scenarios of cloud storage and the basic function overview of cloud storage
3. How to use cloud storage to upload and download data and related management functions
4. Understand the security authority control and data security of cloud storage
1. Introduction to traditional storage:
1. The development history of storage
2. History and trend of hard disk development
3. SDD Hard Disk Introduction
SDD hard disk
It generally refers to a storage device composed of a control unit and a solid-state storage unit (DRAM or Flash chip), and currently supports SATA, FC, and SAS interfaces.
Compared with mechanical hard drives, SSD has the following advantages:
>Fast data access speed: Compared with traditional hard disks, it has excellent performance in terms of intensive I0 read and write and low latency;
>Low failure rate: there are no mechanical parts inside, and all flash memory chips are used inside
>Green and environmental protection: quiet when working, low heat generation, fast heat dissipation
>Lighter weight: it is beneficial to carry the mobile device.
4. Background of RAID technology
5. Introduction to RAID
RAID (Redundant Array of Independent Disks) is Redundant Array of Independent Disks. Fundamentally speaking, a certain amount of data redundancy is formed in a certain way through multiple hard disks, so that when one or more hard disks are damaged, the data can still be saved by such redundancy and will not be lost.
2. Host access storage mode
Current network storage technologies include:
- Direct Attached Storage (DAS: Direct Attached Storage)
- Network storage device (NAS: NetworkAttached Storage)
- Storage Network (SAN:Storage Area Network)
1. Storage backup introduction
- full backup
- incremental backup
- full backup
Full backup:
is a full backup of the data.
Incremental backup:
It is to back up the files that have been modified after the last full backup or incremental backup
The advantage is that the amount of backup data is small and the time required is short;
The disadvantage is that multiple backup data are required when restoring, and there is a greater risk of problems
Full backup:
All files will be backed up every time it is backed up;
The advantage is that the recovery speed is relatively fast, because only the last full backup is required;
The disadvantage is that each backup takes a long time.
2. Introduction to storage disaster recovery
Intra-city disaster recovery center:
The distance between the production center and the disaster backup center is relatively close, and it is relatively easy to realize synchronous mirroring of data, which can ensure data integrity and zero data loss.
It can prevent potential risks and hidden dangers such as fire and building damage, but it is powerless against hidden dangers such as wars, earthquakes, and floods.
Remote backup center:
The production center and the backup center are across metropolitan areas, and the distance is relatively long.
Backing up data through asynchronous mirroring/replication does not guarantee zero data loss.
If the mirror is synchronized over a long distance, the transaction efficiency is too low and the communication cost is high.
Three Centers in Two Places:
Combining the advantages of "same city + different places".
In the case that the remote backup center has complete disaster takeover capabilities, the establishment of an intra-city backup site can enable the intra-city disaster recovery center to have application takeover capabilities.
It is also possible to make the intra-city disaster recovery center just a synchronous data mirroring site.
3. Introduction and usage scenarios of cloud storage OSS
1. Global data storage trends
The total amount of data on the earth so far:
In 2006, individual users had just entered the Tb era, and a total of about 180 EB of data was newly generated globally;
In 2011, this figure reached 1.8ZB;
And some market research institutions predict:
By 2020, the total amount of data in the entire world will increase by 44 times, reaching 35.2ZB (1ZB=10(ZTB)
2. Cloud storage classification
3. Alibaba Cloud object storage service oss
Object Storage Service (OSS for short) is an Internet-oriented distributed storage service.
It has the characteristics of mass, security, high performance, high reliability and low cost.
OSS is very suitable for storing a large amount of unstructured data of different sizes and formats, such as videos, images, logs, text files, etc. The size of a single data ranges from 1 byte to 48.8T, and the number of files that can be stored is unlimited, thus providing Internet applications with massive storage capacity.
4. The position of OSS in the Alibaba Cloud platform architecture
OSS is an important part of Alibaba Cloud, built on the Feitian core platform, and is a standard cloud computing storage service provided by Alibaba Cloud
5. Comparison between OSS and self-built server storage
6. Cloud storage OSS application scenario - website/app dynamic and static separation
Developers can directly use OSS and BGP bandwidth to realize direct download of ultra-low-latency data, or cooperate with Alibaba Cloud CDN acceleration service to distribute pictures, audio and video, and mobile application updates to provide the best experience and other scenarios.
7. Cloud storage OSS application scenario - mass file storage
It is suitable for the storage of massive files such as pictures, audio and video, and logs. It supports various terminal devices, web site programs, and mobile applications to directly write or read data to OSS, and supports two methods of streaming writing and file writing.
8. Cloud storage OSS application scenario - cloud data processing
After uploading files to OSS, you can cooperate with Media Transcoding Service (MTS), Image Processing Service (IMG), Batch Computing Service, and Offline Data Processing Service (ODPS) to fully tap the value of your data and lead the transformation from IT to DT.
9. Cloud storage OSS application scenario - data download acceleration
You can use OSS and BGP bandwidth to realize direct download of ultra-low-latency data, and also cooperate with Alibaba Cloud CDN acceleration service to provide the best experience for your pictures, audio and video, mobile application update distribution, and other scenarios.
4. Basic overview of cloud storage OSS
-
Basic concepts of OSS
（1）Object
- Each file of the user is an Object
- file size limit
- The Put Object method cannot exceed 5G8 at most, use multijpart to upload
- The object size of the method does not exceed 48.8T8
- Object国含key.datafDmeta data
（2）Bucket
- Each Object must be included in the bucket
- Bucket names are globally unique in the entire OSS and cannot be modified
- The total number of storage spaces created by the same Tongli Cloud account in the same region cannot exceed 100 Buckets
- There is no limit to the number of stored Objects in each Bucket, and there is no limit to the capacity of the storage space
- An application can correspond to one or more Buckets
（3）Service
The virtual storage space provided to the user, the user can have one or more Buckets in this storage space
(4) Access ID & Access Key (API key)
The user identifies the user and makes a signature price certificate for accessing OSS
4. OSS data organization structure
5. Page Console
6. Management method of cloud storage OSS
5. Upload and download of cloud storage OSS data
1. oss to upload and download
2. Upload files through the Alibaba Cloud console
1. Enter the OSS management console interface.
2. Click the name of the storage space you want to upload files to to open the storage space management page.
3. Click the file name to enter the management page of all files in the storage space
4. Click Upload File to open the file selection dialog.
5. Select the file to upload, then click Open, the file is uploaded successfully, click the refresh button to view the uploaded file
1. Enter the OSS management console interface.
2. Click the Bucket list where the target file is located to open the file management page.
3. Click the file name to open the file management page.
4. Click the Get Details link corresponding to the target file, and a dialog box will pop up to provide the URL address to access the file.
3. Upload and download files through API
upload files
The user uploads a single Object using the Put Object method in the OSS API
download file
Download the uploaded file (Object), Object download is done using HTTP GET request, Get Object
It is used to obtain an Object, and this operation requires the user to have read permission on the Object.
Upload and download files through SDK
6. Image processing and static website hosting
1. Cloud storage OSS image processing
2. Static website hosting
1. Users can configure their buckets as static website hosting mode through the OSS console - "Bucket List" - "Select Specified Bucket" - "Basic Settings" - "Static Page"
2. Access city name: http://<custom city name>/, such as: http://cloudstorage.aliyunihao.net/
3. The hosted static website supports the following functions:
a) Default homepage
(Index Document Support)
a) Default 404 page
(Error Document Support
7. Cloud storage OSS security
1. Data security and reliability
2. Cross-region replication
- Enterprise-oriented cloud storage disaster recovery solution
- Cross-Region Data Replication
- Data update initiates synchronization in real time
- Realize data remote disaster recovery
- Ensure business continuity in extreme environments
- Support two-way synchronization, regional multi-active
- Upload data nearby
3. Cross-region replication steps
Step 1: Cloud storage OSS console - "Bucket list -" select the corresponding Bucket - "Overview -" Cross-region replication
Step 2: Set attributes such as the region, Bucket, and data synchronization object of the synchronization target
4. Security control: Bucket permission control
OSS provides bucket-level permission access control
> Public-read-write
Anyone can perform PUT, Get and Delete operations on the Object in the Bucket
> Public-read
The creator of the Bucket can write to the Object in the Bucket (including PUT and Delete Object)
Anyone (including map name access) can read the Object in the Bucket (Get Object)
>Private
Only the creator of the Bucket can read and write objects in the Bucket (including PUT and Get Object)
Others cannot access the Objects in this Bucket
The default permission is private when creating a Bucket. You can modify the permission of the Bucket through the Put Bucket ACL interface of OSS
5. Cloud storage OSS anti-leech
OSS is a fee-for-use service. In order to prevent users' data on OSS from being stolen by others, OSS supports an anti-leeching method based on the header field referer in the HTTP header.
Currently, only through the OSS console can you set the whitelist of the referer. field for a bucket and whether to allow access to requests with an empty referer field.
For a bucket named oss-example, if its referer whitelist is set to http://www.aliyun.com, then all requests whose referer is http://www.aliyun.com can access objects in the bucket of oss-example
6. Server-side encrypted storage
1. OSS supports encryption and encoding of data uploaded by users on the server side
2. Specify x-oss-server-side-encryption=AES2563 when uploading the file.
3. For encrypted files, the access method is the same as that of non-encrypted files
- When the file is uploaded, OSS encrypts the user data and saves it
- When the file is downloaded, OSS automatically decrypts the file and declares it in the returned HTTP request
This file uses server-side encryption
7. Operations supported by server-side encryption
Content-Length : ContentLength
- Put Object
- Copy Object
- Initiate Multipart Upload
8. Supported encryption algorithms
AES256
9. Installation Control
Creation is not easy, please pay attention, like, collect, thank you~