HCIP Study Notes-Storage Service Planning-4

Storage Service Overview

1.1 Development Trend of Data Storage

• From the IBM mainframe era (the mainframe's file function, network function and storage function are encapsulated in a set environment), to the x86 era (data in the database is stored in the x86 architecture server in the computer room), and then to the virtualization era (Using distributed technology to store data in virtual machines), storage media components are becoming cloud-based (businesses go to the cloud, and data is also stored in the cloud along with the business), and network protocols are moving towards ALL IP.

1.2 Traditional cloud storage and off-cloud storage

• According to the server type, it is divided into: closed system storage and open system storage. The storage of the open system is divided into: built-in storage and external storage. Plug-in storage is divided into: direct-attached storage (DAS for short) and fabric-attached storage (fabric-attached storage, FAS for short) according to the connection method; network storage is further divided into: network access storage ( Network-Attached Storage, referred to as NAS hw35 and Storage Area Network (Storage Area Network, referred to as SAN).
• DAS: Although DAS is relatively old, it is still very suitable for small and medium-sized enterprises with small data volume and high requirements for disk access speed.
• NAS: NAS is mostly suitable for file servers to store unstructured data. Although it is limited by the speed of Ethernet, it has flexible deployment and low cost. 2903hw3580290
• SAN: SAN is suitable for large-scale applications or database systems, the disadvantage is high cost and complexity
• Block storage: Block storage splits data into blocks and stores each block separately. Each data block has a unique identifier 290, so the storage system can store smaller data in the most convenient location. This means that some data can be stored in a Linux environment and some in a Windows unit.
• File storage: File storage is also called file-level storage or file-based storage, and data is stored in folders in the form of a single piece of information.
• Object Storage: Object storage, also known as object-based storage, is a flat structure in which files are split into parts and spread across multiple pieces of hardware. In object storage, data is broken down into discrete units called "objects" and kept in a single repository, rather than as files in folders or chunks on a server

1.3 Usage Scenarios of Huawei Cloud Storage

• RPO (Recovery Point Objective) is the data recovery point objective, which mainly refers to the amount of data loss that the business system can tolerate.
• RTO (Recovery Time Objective) is the recovery time objective, which mainly refers to the longest time that the business can stop the service that can be tolerated, that is, the shortest time period required from the occurrence of the disaster to the recovery of the service function of the business system

Storage Service Planning

2.1 Block storage service EVS

2.1.1 EVS

• A few notes:
  • When an instance mounts a cloud disk, the number of mounted cloud disks will vary according to the instance rules. It depends on the specific situation.
  • The instance and the cloud disk must be in the same region, otherwise the mount will fail.
  • When creating a replica for EVS, the replica will be created in the same region as the EVS.

2.1.2 EVS specification type

• Use scene recommendation
  • High IO cloud hard disk: system disk of each operating system
  • SSD type disk: usually used as a data disk

2.1.3 Encryption function

• If the security administrator uses the encryption function for the first time, the operation process is as follows:
  • Authorize EVS to access KMS. After the authorization is successful, the system will create a default master key "evs/default" for the user. This key is used to encrypt the cloud disk.
  • Note: The encryption of the cloud disk depends on KMS. When using the encryption function for the first time, you need to authorize EVS to access KMS. After the authorization is successful, all users in the user group do not need to authorize the encryption function again.
  • Select a key. The keys that users can choose to use are as follows: Default master key. evs/default".0
  • User master key, the key created by the user before using the cloud disk encryption function, or the newly created key 0. After the security administrator successfully uses the encryption function, all users in area B can directly use the encryption function.
• If user E (ordinary user) uses the encryption function for the first time, the operation process is as follows.
  • When user E uses the encryption function, the system prompts that the authority is insufficient, and the EVS cannot be authorized to access the KMS. 0
  • Contact the security administrator and ask the security administrator to authorize EVS to access KMS.
• After the authorization is successful, user E and all users in area B can directly use the encryption function without contacting the security administrator for authorization.

2.1.4 Capacity Expansion Function

• To expand the capacity of cloud hard disk in the management console:
  • To expand the capacity of the cloud disk in the management console, you need to select the corresponding expansion method according to the state of the cloud disk. How to check the status of the cloud hard disk. The cloud hard disk whose status is "in use" means that the cloud hard disk has been mounted to the cloud server. It is necessary to judge whether the cloud hard disk supports expansion in the "in use" state according to the constraints and restrictions. If yes, you can directly expand the capacity Cloud disk capacity. If it is not supported, you need to uninstall the cloud disk and then expand it. A cloud disk whose status is "Available" means that the cloud disk is not attached to any cloud server, and the capacity of the cloud disk can be directly expanded

2.1.5 Snapshot function

• Daily backup data
  • By regularly creating snapshots of the cloud hard disk, the daily backup of data can be realized, which can deal with data loss or inconsistency caused by misoperation, virus and hacker attacks.
• Quickly restore data:
  • Before major operations such as application software upgrades or business data migration, you can create one or more snapshots. Once a problem occurs during the upgrade or migration process, you can use the snapshot to restore the business to the data state at the snapshot creation point in time.
• Rapid deployment of multiple services
  • Multiple cloud hard disks with the same data can be quickly created through the same snapshot, so that data resources can be provided for multiple businesses at the same time.

Applicable scene

2.2 Object Storage Service OBS

• A bucket is a container for storing objects in OBS. Object storage provides a flat storage method based on buckets and objects. All objects in a bucket are at the same logical level, eliminating the multi-level tree directory structure in the file system. Each bucket has its own attributes such as storage category, access permission, and region. Users can create buckets with different storage categories and access permissions in different regions, and configure more advanced attributes to meet the storage demands of different scenarios.
• Both the OBS system and a single bucket have no restrictions on the total data capacity and the number of objects/files, providing users with the ability to store large storage capacity, suitable for storing any type of files, and suitable for ordinary users, websites, enterprises, and developers. OBS is a The service for Internet access provides a web service interface based on the HTTP/HTTPS protocol. Users can connect to computers on the Internet anytime and anywhere, and access and manage data stored in OBS through the OBS management console or various OBS tools. In addition, OBS supports SDK and OBS API interfaces, enabling users to easily manage their own data stored on OBS and develop various types of upper-layer business applications.

2.2.1 Three storage categories of OBS

• Standard storage:
  • For example: big data, mobile applications, popular videos, social pictures and other scenarios.
• Infrequently accessed storage:
  • For example: file synchronization/sharing, enterprise backup and other scenarios. Compared with standard storage, infrequent access storage has the same data persistence, throughput, and access latency, and the cost is lower, but the availability is slightly lower than standard storage.
• Archive storage:
  • For example: data archiving, long-term backup and other scenarios. Archive storage is secure, durable, and extremely low-cost, and can be used as an alternative to tape libraries. To keep costs low, data retrieval times can vary from minutes to hours.

2.2.2 Comparison of three storage categories of OBS

• When configuring the data storage policy of the object storage service OBS, you can choose multi-AZ storage or single 80AZ storage based on business scenarios. When multi-AZ storage is selected, the data between multiple AZs can ensure data persistence up to 99.9999999999% and business continuity up to 99.995% through means such as disaster recovery, which is much higher than the traditional architecture
• Durability: A durability of 99.9999999999% (12 nines) means that the average annual object loss rate is expected to be 0.0000000001%. For example, if a user stores 100 million objects in OBS, it is estimated that one object may be lost every 10,000 years on average.
• Availability: Availability can also be understood as business continuity. Availability of 99.995% means that if OBS is accessed continuously for 100,000 minutes (about 69 days), the period of inaccessibility will not exceed 5 minutes.

2.2.3 Life cycle management function

• Action explanation:
• Storage class conversion: When an object storage class is converted to another storage class.
• Delete after expiration: When the object expires, it will be deleted by the object store.
• Additional Notes:
  • There is no limit to the number of lifecycle rules for a single bucket, but the total size of the XML descriptions of all lifecycle rules in a bucket cannot exceed 20 KB.
  • The minimum storage period for archive storage is 90 days. If the object is converted and the archive storage time is less than the minimum storage time, you need to make up the storage fee for the remaining days.
• Object Storage Class Conversion Limits(1):
  • Only standard storage objects are supported for conversion to infrequent access storage objects
  • Only standard storage or infrequent access storage objects are converted to archive storage objects
• Object Storage Class Switching Limits(2)
  • Converting low-frequency access storage objects to standard storage objects requires manual conversion
  • To convert an archive storage object to a standard storage or infrequent access storage object, you need to manually restore the object and then manually convert the storage class.

2.2.4 Cross-region replication

• When configuring cross-region replication rules, users can request to replicate some objects based on prefix matching, or request to replicate all objects in the bucket. Objects copied to the target bucket are exact copies of the objects in the source bucket. They have the same object name and metadata including: object content, size, last modified time, creator, version number, user-defined metadata, and ACL. By default, the storage class of the copied object is consistent with that of the source object. Users can also hw35802903 specify 358 storage classes for replication objects.
• Copied content:
  • Newly uploaded objects (except archive storage objects)
  • There are updated objects, for example, the content of the object is updated or the ACP of the successfully copied object is updated
  • Historical objects in the bucket (you need to enable the "Synchronize historical objects" function, except for archive storage objects)

2.2.5 Multi-version control function

• A bucket stores two objects, object one and object two. Because the version control function is enabled on the storage bucket, the user sees that the current version of object 1 is version 3. By querying the history record p0, you can see that the non-current versions of object 1 include version 1 and version 2. The user sees the current version of object two as deleted because the object is now marked for deletion. By querying the history, you can see that the non-current version of object 2 has version 1

2.2.6 Server-side encryption function

• Server-Side Encryption with KMS Managed Keys (SSE-KMS)
  • The user first needs to create a key in KMS (or use the default key provided by KMS), and use this key for server-side encryption when the user uploads objects in OBS
• Server-Side Encryption (SSE-C) with Client Provided Encryption Key:
  • OBS uses the key provided by the user and the MD5 value of the key for server-side encryption

2.2.7 Event notification function

• Supported event types:
• OBS can create objects through APIs such as PUT, POST, and COPY, and configure the corresponding event types to notify users when using specific APIs to create objects. Users can also request all created object notifications using the ObjectCreated:* event types.
  • ObjectCreated:* (all upload operations)
  • ObjectCreated:Put (upload object)
  • ObjectCreated: Post (upload object using browser). 0
  • ObjectCreated; 8Copy (copy object).
  • ObjectCreated: CompleteMultipartUpload (Merge segment)
• By using the ObjectRemoved event type, users can enable notification when an object or a batch of objects is removed from storage.
• Users can use the ObjectRemoved:Delete event type to request to be notified when an object is deleted or when a versioned object is permanently deleted. Alternatively, use the ObjectRemoved:DeleteMarkerCreated request to be notified when a delete marker is created for a versioned object. You can also use ObjectRemoved:* to request to be notified every time an object is removed.
  • ObjectRemoved: 4 (all delete operations)
  • ObjectRemoved: Delete (delete object)
  • ObjectRemoved: DeleteMarkerCreated (generated deletemarker object)

2.2.8 Applicable scenarios

• The big data solutions provided by OBS are mainly for scenarios such as massive data storage analysis, historical data detailed query, massive behavior log analysis, and public affairs analysis and statistics.
• Mass data storage analysis:
  • Typical scenarios: PB-level data storage, batch data analysis, millisecond-level data detailed query, etc. Historical data detailed query:
  • Typical scenarios: flow audit, equipment historical energy consumption analysis, trajectory playback, vehicle driving behavior analysis, refined monitoring, etc.
• Massive behavior log analysis:
  • Typical scenarios: analysis of learning habits, analysis of operation logs, analysis and query of system operation logs, etc.
• Public affairs analysis statistics:
  • Typical scenarios: crime tracking, related case query, traffic congestion analysis, scenic spot popularity statistics, etc.

2.3 File storage service SFS

2.3.1 Flexible File Service SFS

2.3.2 SFS Turbo

2.3.3 Applicable scenarios

2.3.4 Differences between EVS, SFS and OBS

2.4 Summary of Three Storage Services

• Cloud Disk (EVS): The entire raw disk space is mapped to the host VM, and users can freely format the storage into a file system for use as needed.
• Flexible File Service ( SFS ): It is like a shared folder. The file system already exists, and users can directly store their data on the file storage, such as Windows remote directory sharing.
• Object Storage Service (OBS): Each data corresponds to a unique id. In object-oriented storage, there is no longer a directory hierarchy similar to the file system. Completely flat storage means that the data can be directly located according to the id of the object Location.

2.5 Dedicated distributed storage service DSS

• Rich specifications:
  • High IO: high performance, high expansion, and high reliability, suitable for application scenarios with relatively high performance, high read and write speed requirements, and 1hw35802903 real-time data storage requirements.
  • Ultra-high IO: Low latency, high performance, suitable for low latency, high read and write rate requirements, and data-intensive application scenarios.
• Elastic expansion:
  • On-demand expansion: the storage pool can be expanded according to business needs
  • Linear performance growth: supports online expansion of disks under DSS, and performance linear growth to meet business needs.
• Safe and reliable:
  • More than three copies: data durability is as high as 99.9999999%.
  • Data encryption: Both the system disk and data disk support data encryption to protect data security
• Backup restore:
  • Cloud backup service: "Backups can be created for disks under dedicated distributed storage, and backup data can be used to roll back disks to maximize data security and correctness, ensuring business security

2.6 The difference between DSS and EVS

2.6.1 Applicable scenarios

• Enterprise customers: IDC Hosting customers, a securities settlement company, etc.
• Customers will use EVS shared storage and DSS dedicated storage services for business. ECS shared storage provides daily storage for daily corporate offices, development departments, testing departments, and databases. DSS dedicated storage provides storage for the core business data running on the BMS.

Content Delivery Network Service

3.1 Why CDN is needed

• Realize cross-operator, cross-regional full network coverage: Various factors such as interconnection failure, regional ISP geographical limitations, and limited export bandwidth have caused the website to be inaccessible regionally. CDN acceleration can cover lines around the world. By cooperating with operators, deploying IDC resources, and rationally deploying CDN edge distribution storage nodes at backbone node providers across the country to fully utilize bandwidth resources and balance source station traffic.
• Ensure website security: CDN's load balancing and distributed storage technology can enhance the reliability of the website and virtually add a protective umbrella to the user's website to deal with most Internet attacks. The anti-attack system can also prevent the website from being maliciously attacked
• Off-site backup: When a server fails unexpectedly, the system will call other nearby healthy server nodes to provide services, thereby providing nearly 100% reliability, which allows the website to never go down.
• Saving cost and investment: Using CDN acceleration can realize the national laying of the website, without considering the purchase of servers and subsequent hosting operation and maintenance, mirror synchronization between servers, and no need to worry about management and maintenance technicians, saving manpower, energy and financial resources.
• Let users focus more on their business: CDN acceleration vendors generally provide one-stop services, and their business is not limited to CDN. There are also supporting cloud storage, big data services, video cloud services, etc., and generally provide 7 x 24 operation and maintenance monitoring support, Ensure the smooth flow of the network at any time. And put more energy into the development of its own core business.

3.2 Content Delivery Network CDN

• HUAWEI CLOUD CDN caches source site resources on acceleration nodes all over the world. When end users access resources, there is no need to return to the source. CDN will At that time, the IP address of the CDN node that can respond to the user the fastest is provided to the user, so that the user can obtain the website content at the fastest speed.
• HUAWEI CLOUD CDN has abundant acceleration node resources, with 2000+ acceleration nodes in mainland China and 800+ acceleration nodes outside mainland China. There are sufficient bandwidth resources, and the bandwidth output capacity of the whole network is not less than 150.Tbps. It covers mainstream operators such as Telecom, China Unicom, China Mobile, and Education Network, as well as many small and medium-sized operators. Covering more than 130 countries/regions and supporting more than 1,600 operator networks. Ensure that user requests are accurately dispatched to the optimal edge node, providing effective and stable acceleration effects.

3.3 Working principle of CDN

3.4 Static Content Acceleration

• Dynamic Data: Web Programs
• Static data: pictures, videos, audio, etc.

3.5 Content anti-tampering

• Widely used in security-sensitive communications on the World Wide Web, such as transaction payments, etc.

3.6 Support range back to source

• The function of the Range information is to specify the range of the returned data in the HTTP request header, that is, the position of the first section and the position of the last byte. For example: Range: bytes=0-100 is to request the data content of the first 101 bytes of the file.
• Range back-to-source can effectively shorten the distribution time of large files, improve the efficiency of back-to-source, and reduce the consumption of back-to-source

3.7 Anti-leech

3.8 Case: CDN Accelerates OBS Bucket Files

Backup plan

4.1 Why do you need data backup

4.2 Cloud Backup CBR

• backup:
  • Backup refers to the backup data generated when a backup object executes a backup task, including all data required for backup object recovery
• repository:
  • Cloud Backup uses a repository to hold backups. Before creating a backup, you need to create at least one repository and bind a server or disk to the repository. Backups made by the server or disk are stored in the bound repository.
  • There are two types of repositories: backup repositories and replication repositories. The backup repository is used to store the backup generated by the backup object, and the replication repository is used to store the backup generated by the replication operation.
  • Backups generated by different types of backup objects need to be stored in different types of repositories
• Strategy: Divided into backup strategy and replication strategy.
  • Backup strategy: When it is necessary to perform an automatic backup operation on the backup object, a backup strategy can be set. By setting the backup task execution time, period and backup data retention rules in the policy, and binding the backup repository to the backup policy, automatic backup can be performed for the repository.
  • Replication Policy: When you need to perform automatic replication operations on backups or repositories, you can set a replication policy. By setting the execution time and cycle of the replication task and the backup data retention rules in the policy, and binding the backup repository to the replication policy, automatic replication can be performed for the repository. Backups resulting from replication need to be stored in the replication repository.

4.3 Differences of the four backups

• The backups produced by cloud backup can be divided into several types:
  • Cloud disk backup: Cloud disk backup provides data protection based on snapshot technology for cloud disks
  • Cloud server backup: Cloud server backup provides data protection for elastic cloud servers based on the consistent snapshot technology of multi-cloud disks. At the same time, the backup generated by the server that does not deploy database and other applications is server backup, and the backup generated by the server that deploys database and other applications is database server backup
  • SFS Turbo Backup: SFS Turbo Backup provides data protection for SFS Turbo file systems
  • Hybrid cloud backup: Hybrid cloud backup provides data protection for offline backup storage of backup data in OceanStor Dorado arrays and VMware server backups.

4.4 CBR configuration method

• Users can also mix the two methods according to business conditions. For example, according to the importance of data, all servers/file systems can be bound to the same repository, and the repository can be bound to a backup strategy for daily backup protection. Some of the servers/file systems that store very important data perform one-time backups from time to time as needed to ensure data security.

4.5 The difference between cloud disk backup and snapshot

4.6 Applicable scenarios

Disaster recovery plan

5.1 Storage disaster recovery service SDRS

• RPO=0: Based on 8+ years of self-developed storage layer synchronous replication technology, zero data loss is guaranteed.
• Minute-level RTO: After a disaster occurs, failover is completed in minutes.
• Online disaster recovery drill: On-demand online disaster recovery drills can be performed without affecting the business to test the feasibility and effectiveness of the disaster recovery solution.
• Three-step disaster recovery: You can easily realize business disaster recovery protection on the cloud by creating a protection group, creating a protection instance, and enabling protection in three steps.
• One-key disaster recovery switching: SDRS supports one-key disaster recovery switching. After the switching is completed, the business can be resumed by quickly starting ECS ​​manually.
• Business granular protection: SDRS supports business-based disaster recovery protection, that is, multiple ECSs of the same business are added to the same SDRS protection group for disaster recovery protection
• No need to install plug-ins: No need to install plug-ins in the disaster recovery ECS, simple deployment No charge for the disaster recovery end ECS: Under normal conditions, the disaster recovery end ECS is in the shutdown state, and there is no charge for computing resources TCO is reduced by 60%: Compared with traditional disaster recovery This solution saves costs such as hardware and power maintenance, and the disaster recovery TCO is reduced by about 60%.
• Automatic network migration: After the disaster recovery switch is completed, the IP, MAC, and EIP of the ECS can be migrated to the new production site without reconfiguring the IP and EIP

5.2 The difference between SDRS and CBR

5.3 Applicable scenario: cross-AZ disaster recovery

thinking questions