HHU Cloud Computing Final Review (Part 1) Google, Amazon AWS, Azure

The first half of the review notes for the cloud computing course at the Business School of Hohai University is
only for the test sites, so it is not comprehensive, and it is reserved for the students who need it in the future.
This article mainly talks about Google Cloud Computing, Amazon AWS, and Microsoft Azure. Important knowledge points of the system

Chapter 1 Introduction

Cloud computing is a mode of providing computing services, including servers, storage, databases, networks, software, analysis and other services, which are provided to users all over the world through the Internet ("cloud"). According to the type of service, cloud computing can generally be divided into three categories: infrastructure as a service (IaaS), platform as a service (PaaS) and software as a service (SaaS).

The following are detailed introductions and typical examples of these three service types:

1. Infrastructure as a Service (IaaS for short) : Provides computing resources, including infrastructure services such as servers, storage, and network hardware. Users can purchase or release resources at any time according to needs to cope with business changes. This service allows users to fully control all the details of the infrastructure, but at the same time requires users to be responsible for the operation, maintenance and management of the system. Typical IaaS service providers include Alibaba Cloud and Tencent Cloud.
2. Platform as a Service (PaaS for short) : Provides a complete development and deployment environment, allowing developers to focus on writing code without managing the underlying infrastructure. PaaS usually includes services such as operating systems, databases, and middleware. Users only need to focus on application development and do not need to manage the underlying infrastructure. Huawei Cloud offers a range of PaaS services, and Alibaba Cloud and Tencent Cloud offer similar services.
3. Software as a Service (SaaS for short) : Provide software applications through the network. Users do not need to purchase and install software, they only need to use the network. SaaS usually adopts a subscription system, and users only need to pay for using the software, without purchasing and maintaining software and hardware. Alibaba's DingTalk and Tencent's WeChat Work are typical examples of SaaS services.

The above are the three main service types of cloud computing. It should be noted that these three service types are not mutually exclusive, and many cloud service providers will provide these three services at the same time.

Chapter 2 Google Cloud Computing

2.1 Google File System (GFS)

GFS was originally designed to meet Google's big data storage and processing needs. The system consists of a master server (Master Server), multiple chunk servers (Chunk Server), and a series of client libraries (Client).

• Client is the access interface provided by GFS to the application
• The main server Master is responsible for the management of metadata (metadata), including namespace, access control information, and file block information. The main server does not directly participate in the reading and writing of files, thus avoiding becoming a bottleneck.
• Chunk servers store files as fixed-size chunks (64MB by default) and replicate them at the master's direction to provide fault tolerance.

The GFS file writing process is roughly as follows:

• The client asks the master server which chunk servers hold the file.
• After the master server returns the chunk server information, the client sends a write request to the chunk server.
• The chunk server accepts the write data and upon success notifies the master server to update the metadata.

升级原理？

2.2 MapReduce和Hadoop

Distributed data processing

MapReduce is a big data processing programming model designed by Google, and Apache Hadoop is a well-known open source implementation of this model.

MapReduce mainly includes three stages: Map stage, Shuffle stage and Reduce stage. In the Map phase, the input data is divided into multiple blocks and processed in parallel on each node of the cluster to generate a series of key-value pairs. The next Shuffle stage will sort and group these key-value pairs according to the key, so that the values ​​with the same key can be gathered together. Then in the Reduce phase, the corresponding value is aggregated according to the key of the key-value pair.

Here is an example of this process:

Suppose we need to process a large text file and the goal is to count the number of occurrences of each word. We can use MapReduce to accomplish this task.

1. Map stage : Each Map task processes a part of the file. It reads the text and outputs each word as a key-value pair, where the key is the word itself and the value is the count of the word (initially 1). For example, if the input is "apple banana apple", the output of the Map stage is [("apple", 1), ("banana", 1), ("apple", 1)].
2. Shuffle stage : In this stage, the system will automatically sort and group all key-value pairs according to the key, so that all the same keys will be gathered together. In our case, the Shuffle stage will collect all ("apple", 1) and ("banana", 1) together.
3. Reduce phase : In this phase, each Reduce task will receive the results of the Shuffle phase, and then aggregate all values ​​of the same key. For example, for "apple", the input of the Reduce phase is [("apple", 1), ("apple", 1)], it will add up these counts, and the output result is [("apple", 2) ].

In this way, through MapReduce, we can efficiently process a large amount of data on a large-scale cluster.

• The specific execution process:
  • The general execution process of MapReduce is shown in the following figure:

The six steps in the figure correspond to the following:

1. The input file is first divided into M data segments, and the size of each data segment is generally from 16MB to 64MB. The user program then creates numerous copies of the program across the cluster.
2. Except for one master, the replica programs are all worker programs, and the master assigns M map tasks and R reduce tasks. The master assigns a map task or reduce task to an idle worker.
3. The worker program assigned the map task reads the corresponding input data fragment, parses and processes the key-value pair, generates and outputs the intermediate key-value pair result, and caches it in the memory buffer.
4. Periodically, the intermediate results in the buffer are divided into R parts by a user-specified partition function (for example hash(key) mod R) and stored on local disk. When the task is completed, the storage locations cached on the local disk will be passed back to the master, and the master is responsible for transferring these storage locations to the reduce workers.
5. After the reduce worker program receives the data storage location information sent by the master program, it uses RPC to read the cached data from the disk of the host where the map worker resides, and then aggregates the data with the same key value by sorting the keys. If the intermediate data is too large to be sorted in memory, it must be sorted externally.
6. The reduce worker program traverses the sorted intermediate data. For each unique intermediate key value, the reduce worker passes the key value and its related value value set to the user-defined reduce function for processing, and the processing output is appended to the corresponding The output file for the partition. Since the sorted intermediate data is processed sequentially, each output file fragment is ordered internally.

2.3 Distributed lock service Chubby

Chubby is Google's distributed lock service. It provides synergy with other Google systems by providing coarse-grained locking and the ability to store small amounts of data.

• Chubby communication protocol (read textbook P30)

• master server error

  • During the Chubby communication, the client interacts with the Chubby server by sending RPC requests. When the main server fails, it will perform failover, and the backup server will take over its role to ensure the availability of the system.

• lease mechanism

  • The lease mechanism is an important means for Chubby to ensure service availability. When a client acquires a lock, it also acquires a lease, usually valid for tens of seconds. As long as the client's lease has not expired, it can think it holds that lock. If the client renews the lease with the Chubby server during the lease period, the lease can continue to be valid. If the server does not receive a lease renewal request, it will assume that the client has crashed and release the lock, allowing other clients to acquire it.

For example, suppose you are writing a document in Google Docs, and your colleagues want to edit the document at the same time. Chubby locks can prevent you from modifying the same part of the document at the same time during this process, causing conflicts. When you edit a paragraph, your client will acquire a Chubby lock to ensure that no one else can modify the paragraph while you are editing it. If your lease expires and is not renewed, Chubby will assume that you have stopped editing, release the lock, and allow others to edit the paragraph.

2.4 Distributed structured data table Bigtable

Video: https://www.bilibili.com/video/BV1bj41137BY/

Bigtable is Google's distributed storage system, mainly used to store structured data. Bigtable mainly consists of three parts: client library (Client Library), a main server (Master Server) and multiple sub-table servers (Tablet Server).

Data in Bigtable is organized in rows and columns, and each row is uniquely identified by a row key.

storage form

• <row key, column key, timestamp> -> content

• How row labels are stored

  • The way row keys are stored in Bigtable is sorted lexicographically (should mean alphabetical order), which makes the operation of reading adjacent row keys very efficient. **This feature makes Bigtable very suitable for operations that need to be read according to a certain range.

• Advantages of Inverting Row Labels

  • Web pages in the same address domain will be stored in consecutive positions in the table, which is convenient for users to search and analyze

  • Inversion is convenient for data compression, which can greatly improve the compression ratio

  • explain:

    • domain name:

      • In " www.example.com/news ", "example.com" is the main domain name, "www" is a subdomain name under this main domain name, and "/news" is a path or page under this subdomain name.
      • In "news.example.com", "example.com" is still the main domain name, but this time "news" is a subdomain under this main domain name.

      A subdomain is a branch of the main domain name, and is usually used to indicate different functions or services under the main domain name. For example, many websites will have "blog.example.com" to host their blog, or "shop.example.com" to host their online store.

    • The designer of Bigtable chose to use the URL in reverse order mainly to better optimize the query efficiency of the "domain name", especially when the URL has many levels of sub-domain names, for example:

      1. news.example.com
      2. sports.example.com
      3. finance.example2.com

      If the URLs are sorted in positive order, the pages of the same main domain name (example.com or example2.com) in the above three URLs may not necessarily be put together, but sorting in reverse order can do this:

      1. com.example.news
      2. com.example.sports
      3. com.example2.finance

main server

• The main role of the main server

  • 

child table

• tablet

  • Tables in Bigtable are made up of many small pieces called "tablets" in Bigtable. Each tablet represents a part of the range of row keys, so that the entire table can be divided into many small pieces that are processed in parallel, so-called "sub-tables". Tablets facilitate distributed storage and parallel processing of tables.
  • Conceptually a subtable is a collection of rows

• A tablet server is a server in Bigtable that is responsible for processing one or more tablet tables. It handles read and write requests for subtables and splits subtables into smaller units if necessary.

• SSTable

  • Basic schematic of SSTable format
    • SSTable is Google's internal data storage format designed for Bigtable. All SSTable files are stored on GFS, and users can query the corresponding value by key.
    • Each SSTable contains a series of blocks (block) and a block index (block index), used to efficiently search and read data.
    • 

• The actual composition of the child table

  • A subtable consists of multiple SSTables and log files
  • SSTables of different subtables can be shared
  • Only one log file is kept on each child table server
  • Bigtable stipulates that the contents of the log are sorted by key value
  • The number of sub-tables saved on each sub-table server can range from tens to thousands, usually around 100
  • 

• Subtable address composition

  • The data of a subtable (tablet) is stored in one or more SSTables. Each SSTable stores sorted key-value pairs. This sorting feature makes operations such as range queries and sequential reads very efficient.
  • There is an important mapping between subtables and SSTables: each subtable knows which SSTables its data is stored in. When a subtable server needs to read or write the data of a subtable, it will use this mapping relationship to find the corresponding SSTable, and then operate in the SSTable.
  • 

• Three forms of data compression

  • Minor compression, combined compression, primary compression
  • 

The data of the subtable is finally written to GFS, and the physical form of the subtable in GFS is several SSTable files

The cluster includes a main server and sub-servers. The main server is responsible for allocating slices to sub-servers, while the sub-servers are fully responsible for specific data services.

But don't mistakenly think that the sub-server really stores the data (except memtable data in the memory), the real location of the data is only known by GFS, the main server assigning the sub-table to the sub-server should mean that the sub-server has obtained the sub-table All SSTable file names, the subserver can know which SSTable file the required data is in through some indexing mechanisms, and then read the data of the SSTable file from GFS. This SSTable file may be distributed on several chunkservers.

Bigtable-related optimization techniques

Bigtable uses some performance optimization techniques: BWT (Burrows-Wheeler Transform) and Bloom filter.

1. BWT (Burrows-Wheeler Transform): It is an algorithm for data compression, mainly used to transform string data. In BWT, the characters of the string will be arranged in a certain order, and then a new string is selected. The order of characters in this new string will improve the subsequent compression algorithm (such as move-to-front transform, run-length encoding) efficiency. In other words, BWT increases the efficiency of compression by changing the order of the characters in the string and making the originally randomly distributed characters more concentrated. It is worth noting that this transformation is reversible, which means that the data transformed and compressed by BWT can be completely restored. ???

2. Bloom Filter: A Bloom filter is a highly space-efficient probabilistic data structure used to detect whether an element is a member of a set. Its main feature is that there is a certain false positive rate, but it will never be missed. The false positive rate refers to querying an element that does not exist in the collection, and the Bloom filter may mistakenly think that it is in the collection; the false negative refers to querying an element that exists in the collection, and the Bloom filter will not be wrong Think it's not in the set. In Bigtable, the Bloom filter is used to reduce unnecessary disk read operations: when we query an element, we first use the Bloom filter to judge it. If the judgment result is "not present", we can avoid disk reads; If the judgment result is "in", it needs to further read data from the disk to determine.

   • Suppose we have a phone book, and we need to determine whether a phone number is in the phone book. If we directly query the phone book, we may need to look through the entire phone book, which takes a long time. Whereas if we use Bloom filter then we can get the answer in a very short time. We first add all the phone numbers in the phone book to the Bloom filter, which produces a bit array. Then, when we need to check whether a phone number is in the phone book, we only need to check this bit array. If the Bloom filter judges the result as "not in", then we can be sure that the phone number is definitely not in the phone book, thereby avoiding unnecessary inquiries. If the result of the Bloom filter is "in", then we need to further check the phone book to determine. In this way, in most cases, we can get the answer in a very short time, which greatly improves the query efficiency.

In general, BWT is an algorithm used to improve data compression efficiency, while Bloom filter is a data structure used to reduce unnecessary disk reads, thereby improving query efficiency. Both play an important role in Bigtable.

2.5 Distributed storage system Megastore

https://www.jianshu.com/p/7c4d0ab911f6

Megastores

Megastore is a distributed storage system of Google. It is built on the basis of Bigtable and provides users with some relational database features including ACID transactions.

The basic architecture of Megastore includes the following parts:

• Entity Group: Megastore organizes data into entity groups (Entity Groups). Each entity group is internally stored in the manner of Bigtable, and each entity group can provide transactions with ACID semantics (like a small database). ACID transactions cannot be provided between entity groups.

• Replicas: In order to improve data availability and fault tolerance, Megastore will replicate the data of each entity group to multiple physical locations.

• Paxos: Megastore uses the Paxos algorithm to guarantee consistency among multiple replicas of each entity group. Every write operation of Megastore needs to be submitted by the majority of Paxos.

• Catch-up : In order to ensure the consistency of data between different regions, Megastore provides a Catch-up mechanism, that is, when a copy lags behind other copies, it can obtain the lost update through the Catch-up process.

• Product: Megastore is widely used in many Google products. For example, the data storage of Google App Engine is implemented based on Megastore.

ACID semantics

ACID refers to the four basic characteristics of the correct execution of database transactions: atomicity (Atomicity), consistency (Consistency), isolation (Isolation) and persistence (Durability).

1. A (Atomicity) Atomicity : This means that the entire transaction is an indivisible unit. All operations in the transaction are either all submitted successfully, or all failed and rolled back. For the operation of a transaction, the system guarantees that all operations are either completed or completely completed. If you don't do it, it is impossible to stay in the middle link. The atomicity of transactions is achieved through Undo and Redo.

2. C (Consistency) Consistency : A transaction must transform the database from one consistent state to another. Consistency is related to business. For example, if account A transfers money to account B, no matter whether the transfer is successful or not, the outgoing amount of account A must be consistent with the incoming amount of account B. This is business consistency.

3. I (lsolation)- Isolation : When multiple users access the database concurrently, the transactions opened by the database for each user cannot be interfered by the operations of other transactions, and multiple concurrent transactions must be isolated from each other. That is to achieve such an effect: For any two concurrent transactions T1 and T2, from the perspective of transaction T1, T2 either ends before T1 starts, or starts after T1 ends, which makes each transaction feel No other transactions in the system are executed in parallel.

4. D (Durability) Persistence : Once a transaction is committed, its modifications are permanently stored in the database. Even if the system crashes after the transaction is committed, the persistence of the transaction can be guaranteed after restarting.

basic structure

• Full copy : Complete logs and data are stored in Bigtable .

• Witness copy : Participate in voting when a resolution cannot be generated during the execution of the Paxos algorithm . Therefore, for this copy, Bigtable only stores its log and does not store specific data .

• Read-only copy : Cannot participate in voting , the function is only to read consistent data up to a certain point in time in the recent past. If read operations can tolerate these stale data, read replicas can transmit data over a large geographic space without exacerbating write latency.

• The deployment of Megastore requires a client function library and several servers . The application connects to this client function library, which executes the Paxos algorithm, and there is also a service called the coordinator (Fast Reads). To understand the role of this service, you first need to understand the fast read and fast write mechanisms

• You can think of Megastore as a super-large library, which has many branches around the world, that is, servers. Among these branch libraries, some libraries (full copies) stock full books and detailed borrowing records. There are also libraries (witness copies) that do not stock books, but they record all borrowings. In addition, there are some libraries (read-only copies) that only allow readers to read books, but do not record borrowing information. When readers want to borrow books, they need to connect to the client function library (that is, the lending desk), and the lending desk will execute a set of processes (similar to the Paxos algorithm) to ensure that the process of borrowing books is correct. The coordinator service is like a quick query service, which can quickly tell you where the book you are looking for is. This is the mechanism of quick reading.

Core Technology - Copy

• Data read:

• Before a current read, it is necessary to ensure that the data on at least one copy is up-to-date, that is to say, all updates previously submitted to the log must go to the copy and be guaranteed to take effect on the copy. ** This process is called Catchup

• 

Think of the process as if you go shopping at the supermarket. You have a shopping list in your hand, and this list needs to be kept up to date before you enter the supermarket, which means that all the changes you make at home (such as adding or deleting a certain product) have been updated to this list, this The process is Catchup. Then you shop in the supermarket according to the list, this is the process of data reading. If during the shopping process, you suddenly find that an item is missing from the list, then you need to go home and update the list again, which is to ensure that the list (that is, the copy) is up to date.

Chapter 3 Amazon AWS

3.1 Dynamo

Cloud Computing | AWS | Dynamo

• Dynamo

  • The underlying storage architecture supports only simple key-value pair storage
  • centerless model
  • Central idea: Dynamo achieves high availability and scalability through distributed storage in the data center.
  • Architecture form: Dynamo uses a consistent hash algorithm for data distribution, so that the system can balance the load and minimize data migration when nodes change dynamically

• Windows Azure is a cloud-based application execution environment that provides computing services, storage services, and various services for communicating with applications. In Azure, the storage layer uses a multi-copy replication mechanism to achieve data persistence and reliability by replicating data to different physical nodes.

  The main difference between the two lies in their design goals and implementation methods. Dynamo focuses on high availability and scalability, is suitable for unstructured data, and uses the final consistency model; while Azure is a more general platform that provides computing, storage, and communication services, suitable for storing structured data, and uses a strong consistency model.

Consistent Hash Algorithm

• Consistent Hash Algorithm: Haogang: 7-minute video explaining the consistent hash algorithm in detail
  • In the consistent hash algorithm, the relationship between virtual nodes and physical nodes can be simply understood as: each physical node corresponds to multiple virtual nodes, and each virtual node stores a part of the data of the physical nodes. By adding virtual nodes, we can evenly distribute the load when the number of physical nodes changes, and minimize the amount of data that needs to be migrated.
  • Using more virtual nodes has several benefits:
    • 1) It can distribute data more evenly and reduce the problem of data skew. This is because the performance of each physical node is not necessarily the same in reality, and the introduction of virtual nodes can make nodes with different performance bear different loads;
    • 2) When adding or deleting physical nodes, only a small amount of data needs to be migrated, reducing the overhead of data migration;
    • 3) Improve the scalability and stability of the system.
  • In the consistent hash algorithm, the complexity of finding a specific key (key) is usually $O\left(log\right(n))$ , where$n$ is the number of nodes (servers) in the hash ring.
  • The consistent hashing algorithm can be further improved, for example, by introducing replication and data fragmentation mechanisms, which can further improve system availability and data security.

redundant backup

• Weak Quorum mechanism with adjustable parameters
  • $R+W>N$ can guarantee that when the number of faulty nodes does not exceed one, users can obtain at least one copy of the latest data. where$W$ represents at least the number of copies that need to be written for a successful write operation,$R$ indicates the minimum number of copies that must be returned to the user by the server for a successful read operation,$N$ represents the number of replicas for each datastore.
• Why Redundant Backup Improves Data Security
  • Redundant backup is a common means to protect data security and availability. The principle of redundant backup is to store copies of data in multiple places. If one storage location fails and data is lost or corrupted, a copy of that data can be obtained from another location. For example, DynamoDB provides high availability and data durability by storing copies of data in multiple availability zones.
  • In fact, redundant backup can not only improve data security, but also increase data availability. When a storage node fails, the system can quickly obtain data from other storage nodes, avoiding system unavailability due to single point of failure.

Membership and error detection

P96

• Dynamo is a non-central architecture, each member node needs to save the routing information of other nodes

• In distributed systems, membership and error detection are very important issues. Membership refers to determining which nodes in the system are participating in the operation. Error detection refers to determining which nodes in the system may have failed.

• Membership (Membership) refers to determining which nodes are active in a distributed system, that is, the nodes currently participating in the operation. Since in a distributed system, nodes may go offline due to failure or other reasons, a mechanism is needed to track and identify which nodes are online.

  • Suppose you are participating in a multiplayer online game, and the servers of this game are distributed. Each player will connect to a server node. At this time, membership is to identify which players (nodes) are currently online and participating in the game.

• Failure Detection : It refers to identifying which nodes have failed in a distributed system. This is because in a large-scale distributed system, failures are the norm. Some nodes may not work due to various reasons such as hardware failures and network problems.

  • Continuing with the above game as an example, suppose that in your team, a player suddenly goes offline (maybe due to network failure, computer crash, etc.). The task of error detection is to find out that the player is offline as soon as possible, so that the game can take appropriate measures (such as letting the AI ​​take over, or finding a new player to join, etc.).

Membership and error detection are usually implemented through a protocol called " Gossip ". In this mechanism, each node periodically sends messages to other nodes that it is "alive". If a node does not hear from another node for a period of time, it assumes that node is offline or has failed.

• If a new node joins a system with a total number of nodes of N, and spreads in an optimal way (that is, the two nodes in each communication exchange node information for the first time), then the time required to spread the new node throughout the entire system The complexity is $O(logn)$

3.2 Elastic computing cloud EC2

• Geography and Availability Zones

• EC2 is short for Amazon's Elastic Compute Cloud, which provides scalable computing capabilities. Users can start applications on virtual machines in Amazon's computing environment. Various services, such as providing GPU

• AWS global infrastructure is cloud infrastructure distributed around the world. These infrastructures include geographic regions and availability zones.

  • A geographic region is the physical location of the AWS cloud around the world, and each region contains at least two availability zones.
  • Availability Zones represent data centers that are geographically separated , but within the same geographic area with low network latency. It is usually divided according to whether there is an independent power supply system and cooling system.
  • EC2 consists of multiple geographic regions, and each geographic region contains multiple availability zones.

• Incorporating this idea into the application architecture can improve the availability and disaster recovery capabilities of the application. For example, by deploying an application in multiple Availability Zones, even if there is a problem in one Availability Zone, the application can run normally in other Availability Zones.

3.3 Simple Storage Service S3

S3, Simple Storage Service, is an object storage service provided by Amazon. It uses objects as units and provides the ability to store and retrieve any amount of data on the Internet.

• Bucket : A bucket is like a folder or directory in Amazon S3, which is used to store objects (data). Each bucket has a globally unique name in S3, and all objects stored in S3 must be contained in some bucket. Users can set bucket access permissions to control which users can access objects in the bucket. Buckets can also be configured to store data in specific geographic regions.
• Object : An object is mainly composed of data and metadata, and is the basic element in a bucket, similar to a file. Each object contains data about the file itself and some metadata (such as the type of file, creation date, etc.). In S3, an object is uniquely identified by its key in a bucket , similar to how a file is uniquely identified by its path and filename in a file system.

We can compare the relationship between buckets and objects to a large warehouse (Bucket) and various commodities (Object) in the warehouse in real life. A warehouse is a place used to store various commodities, and a commodity is the basic unit in a warehouse. Each product has its own tags (such as the name of the product, production date, etc.), which is like metadata of the object. In the warehouse, each item has its fixed location, which is like the key of the object in the S3 bucket.

In this example, if we want to find a specific item, we first need to know which warehouse (bucket) it is in, and then we can find the item (object) based on the tag (metadata) or location (key) of the item.

3.4 Non-relational database

The difference between it and relational database

Textbook P108

• Relational database : such as MySQL, PostgreSQL, etc., is a database based on a relational model, in which data is stored in the form of tables . Relational databases emphasize data consistency and transactionality, and follow the ACID principles (atomicity, consistency, isolation, and durability). This type of database is suitable for applications with complex query requirements, such as joins, grouping, and aggregation.
• Non-relational databases (such as SimpleDB, DynamoDB) : Non-relational databases, also known as NoSQL databases, are mainly designed to handle large-scale data and can provide high-throughput, low-latency data access and storage. Such databases usually do not support or only provide limited transactional functionality, and may not follow ACID principles, but BASE principles (Basic Availability, Soft State, Eventual Consistency). For example, DynamoDB is a key-value store system known for its elastic scalability and predictable performance.

3.5 Relational Database Service RDS

AWS provides a managed service for MySQL called Amazon RDS. It handles many of the day-to-day administrative tasks of the database, such as backups, patching, and failover. Users can choose to run RDS instances in a single availability zone, or they can replicate across multiple availability zones for higher availability and fault tolerance.

RDS (Relational Database Service) is a relational database service provided by Amazon Web Services (AWS). MySQL is an open source relational database management system (RDBMS). Therefore, you can think of MySQL as a "product" and RDS as a "service".

In the RDS service, you can choose different types of database engines, including MySQL, PostgreSQL, MariaDB, Oracle, SQL Server, etc. Therefore, MySQL is one of the databases supported by RDS. Using the RDS service of AWS, users can easily deploy, expand and manage databases. RDS will handle many database operation and maintenance tasks for users, such as backup, software patch update, and failover.

Simply put, you can simply download and install MySQL on your own server, which requires you to handle all the maintenance and administration yourself. And if you choose to use AWS's RDS service and choose MySQL as the database engine, then AWS will handle many operation and maintenance tasks for you, and you only need to focus on using the database.

3.6 Content push service CloudFront

A CDN, or Content Delivery Network, is a service that replicates content on servers in different geographical locations so that users can access it faster. By using a CDN, you can bring your content closer to your users so they can get it faster.

the difference:

Amazon CloudFront and Azure CDN are examples of such services. Both of them can cache your content (such as static and dynamic content of the website, video streaming, API calls, etc.) in edge locations around the world, so that the content can be served to users faster.

? ? ? What is the difference between the two, Azure should have a cache

As an example, let's say you have a website in the East US, and your users are spread all over the world. Without a CDN, a user from Australia would need to send a request to a server in the US-East and then return the result. This increases latency. However, if you use a CDN, your website content will be replicated to servers around the world, including Australia. As a result, users from Australia can now fetch content from servers near them, significantly reducing latency.

Chapter 4 Microsoft's Cloud Computing Service Azure

4.1 Five parts

• Let's start with an analogy

1. Compute Services : You can think of Compute Services in Azure as leased computers . Just like you rent a computer in an Internet cafe, you can do anything you want on it, such as browsing the web, writing code, and even running a server. The computing service provided by Azure is to let you rent a computer on the cloud, and you can run your application on it
2. Storage service : Storage service in Azure is like a warehouse that you rent , where you can store anything you want to keep. Azure provides Blob (similar to a big box, you can put anything), table storage (like a huge Excel table, used to store structured data), queue storage (like a queuing system, used to store pending messages), and disk storage (like your computer's hard drive, used to store various files and data).
3. Fabric Controller : The Fabric Controller is like an intelligent building manager or building manager. It is responsible for coordinating and managing resources in cloud services, such as servers, networks, and storage . Just like a building manager decides which company rents which office, how power is distributed, and how network connections are configured, Fabric controllers are responsible for provisioning and managing the lifecycle of Azure service instances, allocating and managing network and storage resources.
4. Content Delivery Network (CDN) : Azure's CDN is like a chain of convenience stores around the world. Suppose you have a bakery in New York, and your bread is so popular that people all over the world want to eat it. But it's not realistic to have someone in Australia come to New York to buy bread. So, you decide to open stores all over the world so everyone can buy fresh bread at a store near them. This is what Azure CDN does. It caches your content (websites, videos, software, etc.) on nodes around the world . When users request these content, they can be obtained from the nearest node to improve access speed.
5. Windows Azure Connect : Azure Connect is like a bridge that connects your local network with Azure's virtual network . Let's say you have two islands, one on-premises (your local network) and one in the cloud (Azure's virtual network). You'll need a bridge to connect the two islands so you can travel between them easily. Azure Connect is the bridge, allowing you to securely and easily transfer data between on-premises and cloud networks.

• specific definition

1. Computing service (Compute) : Provides virtual machines (VMs) for running applications, supports different operating systems, and multiple programming languages, and can be expanded on demand.
2. Storage service (Storage) : Provides large-scale, highly available, and persistent cloud storage, including Blob (object) storage, file storage, queue storage, table storage, and disk storage.
3. Fabric Controller : Mainly used to deploy, manage and monitor applications.
4. CDN (Content Delivery Network) : Provides a global content distribution network service, and optimizes the speed at which users obtain data by caching content to edge nodes around the world.
5. Networking : Provide virtual network, load balancing, VPN, traffic management and other network functions to help you establish a secure and private connection between the cloud and the local environment .

4.2 Three Examples

The Azure service platform provides three types of roles (Roles) for applications to run code: Web Role, Worker Role and VM Role. These roles define the operating environment of application instances running on the Azure service platform.

1. Web Role : Provides an automatically managed, hosted IIS environment. Developers can publish ASP.NET, WCF services or other IIS-compatible applications to Web Role to handle users' HTTP/HTTPS requests.
2. Worker Role : Provides a common Windows environment where developers can run any type of program. Worker Roles are often used as background processing tasks, such as processing data from Web Roles, running scripts or other long-running tasks.
3. VM Role : Provides a virtual machine environment with a higher degree of freedom. Users can customize the Windows environment of VM Role as needed, and install required applications to run specific tasks or services.

1. Web Role : A Web Role is like a restaurant offering catering. It can accept and process client (user) requests, similar to a restaurant serving food to hungry people. In Azure, Web Role is a service running in Microsoft's public cloud environment that handles external HTTP or HTTPS requests, just like processing orders in a restaurant.
2. Worker Role : The Worker Role is like a chef in the back kitchen of a restaurant, responsible for cooking food. In Azure, Worker Role is to perform tasks running in the background, such as processing data from Web Role, running scripts or other long-running tasks. They are like chefs, turning raw materials (data) into gourmet meals (useful information).
3. VM Role : VM Role is like a rented apartment, you can decorate and place furniture according to your needs. In Azure, VM Role provides a virtual machine environment. Users can configure this environment according to their own needs and install the required software. This role gives users more freedom to define their cloud service environment, just like you can freely arrange it in a rented apartment.

4.3 Data structure

1. Blob : Used to store large amounts of unstructured data, such as pictures, audio, video, or log files.
2. Table : Provides scalable non-relational structured data storage, suitable for storing large amounts of non-relational data.
3. Queue : Provides an extensible message queue service for asynchronously passing messages between applications.

4.4 Spoke backup idea

Synchronization between different SQL Azure databases is done using a "hub-and-spoke" model

In this model, the central node (Hub) is responsible for receiving and distributing data, and the edge nodes (Spokes) are responsible for storing data.

Here the Hub is a SQL Azure database, and other Hubs can be SQL Azure databases or SQL Server databases

We can compare this model to an airline's route network. In this example, the airline's large hub airport (such as Atlanta or Amsterdam) acts as the central node, and the other smaller airports act as the edge nodes. All flights (data) will pass through the hub airport (Hub) and then be distributed to different destinations (Spokes).

In Azure's data center network, a similar model is used. The main data center (Hub) is responsible for receiving all user data and synchronizing the data to each edge data center (Spokes). This way, no matter where users access their data, they get the fastest response time because they always get it from the closest data center.

Moreover, if an edge data center fails, user data can still be obtained from the main data center or other edge data centers, thus ensuring data availability and consistency.