Authorization instructions: This article authorizes the official Amazon Cloud Technology article to forward and rewrite the rights, including but not limited to Amazon Cloud Technology official channels such as Amazon Cloud Technology Developer Community, Zhihu, self-media platforms, third-party developer media, etc.
Foreword:
In today's society, the rapid growth of data makes efficient and secure storage solutions increasingly critical. For developers and enterprises, having a storage solution that meets business needs is critical. However, with so many options available, how to find a more suitable storage solution has become a question worth pondering.
Article directory
1. Amazon S3 Express One Zone launched
As we said above, when we choose a storage product, we need to consider multiple factors such as data scale requirements, security costs, etc., in order to find the best storage solution.
The storage-related product Amazon S3 Express One Zone was released at the recently held Amazon Cloud Technology re:Invent 2023 conference. This is a new storage class designed to provide the highest performance and lowest latency object storage for the products customers access most.
When it comes to Amazon S3, some friends may have limited understanding of it. Simply put, Amazon S3 is a powerful object storage service that not only provides industry-leading scalability, data availability, and security, but also has excellent performance. Amazon S3 focuses on protecting and storing industry data of all sizes, providing businesses and users with trusted storage solutions.
The Amazon S3 Express One Zone we introduced today is further optimized and improved on the basis of Amazon S3. Compared with the previous S3, S3 Express One Zone is more outstanding in speed,Especially for those jobs that are frequently accessed and most latency-sensitive, such as real-time advertising analysis and machine learning training, Such workloads typically involve millions to tens of gigabytes of data, and Express One Zone is designed as a new S3 storage class focused on building and storing the most frequently accessed data . 
Express One Zone leverages specialized hardware and software to accelerate data processing, significantly reducing latency for frequently accessed data and delivering more consistent performance compared to traditional S3 storage. Supports millions of requests per minute, making it one of the fastest object stores in the cloud, up to ten times faster than S3 standard storage. This not only improves performance but also reduces operating costs, especially for scenarios that require high-speed data access and low latency.
2. Efficient cost reduction optimization and accelerated data processing in multiple scenarios
Amazon S3 Express One Zone works by combining catalog buckets with an open source framework for machine learning to reduce latency, processing time, compute costs, and API request costs to improve performance and lower total cost of ownership. At the same time, it can also significantly improve processing efficiency in scenarios such as interactive data analysis. The working principle diagram is as follows
For machine learning and artificial intelligence training, this service can accelerate model training and development and enable more efficient processing of model data sets, thereby significantly shortening the training cycle; , S3 Express One Zone can quickly complete computing-intensive HPC workloads and improve computing efficiency;, S3 Express One Zone can provide targeted advertising content with millisecond request latency, optimize personalized advertising, quickly respond to dynamic placements, and improve Advertising effect. In terms of real-time advertising, the service enables financial models to be extended to higher granularity levels, accelerating the overall modeling speed through low request latency, and providing more flexible and efficient financial modeling. model experience. For financial modelingIn high-performance computing (HPC) scenarios S3 Express One Zone can speed up queries, support rapid analysis of PB-level data, and provide users with real-time data insights. In terms of interactive data analysis,
S3 Express One Zone 相对于传统 S3 具有以下优点：
1.Supported bucket types: S3 Express One Zone introduces directory buckets so that objects can only be stored in this specific type of bucket. This design may provide finer storage control.
2.Durability: S3 Express One Zone data is redundantly stored on multiple devices within a single Availability Zone , with higher availability. In contrast, traditional S3's data redundancy spans multiple availability zones, resulting in higher availability goals but potentially higher costs.
3.Deletion behavior: S3 Express One Zone will recursively delete object paths, including empty directories, when deleting objects. . This helps manage storage space more conveniently.
4.DeleteObjects: The object key in the request: The object key is required to contain at least one non-space character, add the following Validity and security of the request.
5.Zone and zone endpoint: When using S3 Express One Zone, need to be specified in all client requests area to provide clearer request location.
6.Differences in multipart upload: S3 Express One Zone supports multipart upload, but the object creation date, It is different from the traditional S3 in terms of the continuity of use of part numbers. This makes S3 Express One Zone more suitable for specific upload needs.
These features and behavioral adjustments make S3 Express One Zone more suitable for the most latency-sensitive applications with relatively low availability requirements, providing developers with a cost-effective storage option.
3. Service experience and use
1. Create bucket and upload files
1. First complete the login onofficial website, come to the user console interface, search for AmazonS3 service in the search bar above, you can create a general Buckets and directory buckets. Here we choose to create a directory bucket
2. Configure the bucket related information according to personal needs. Here, select Directory - New for the bucket type, because what we want to experience is the function of Amazon S3 Express One Zone.
3. After configuring the relevant information, we only need to wait for a short time to see that the directory bucket has been created. Users can create objects in the directory bucket Manifest files for more flexible data management and monitoring
4. Click to enter the directory bucket we just created. Because it was just created, no related operations have been performed yet. Here, we click the upload button to store some local resources into our bucket
5. After entering the upload interface, we can drag and drop target files and folders, or upload files and folder resources locally.S3 Express One Zone supports directory Perform batch operations on the bucket, including inventory operations. Users can easily manipulate and manage objects in catalog buckets while retaining the validity and integrity of the inventory
6. After selecting the uploaded resources, we only need to wait for a moment to see the upload status. As shown in the picture, it has been uploaded successfully. Here I uploaded some photos.S3 Express One Zone provides the option to choose the checksum algorithm, including Secure Hash Algorithm (SHA) and cyclic redundancy. Checksum (CRC). Users can select appropriate checksum algorithms based on actual needs to verify data integrity during upload or download. This flexibility enables users to customizebased on their data integrity requirements
7. After that, we can check the relevant attribute permissions and some indicators of the bucket, and at the same time, we can move and delete storage objects and create folders.
2. Create access points and batch processing
1. In addition to some of the basic operations above, we can also create access points, which are named network endpoints connected to buckets that can be used to perform S3 object operations.
2. After clicking Create Access Point, after configuring the relevant information, wait a moment and you will see that the access point has been created successfully. You can see the bucket owner account ID and access point alias corresponding to the access point.
3. In addition to creating a single access point, you can also create object Lambda access points and multi-region access points, select according to developer needs
4. At the same time, developers can also use Amazon S3 objects to perform large-scale batch operations. Batch operations can run a single operation on the list of amazing S3 objects specified in the manifest.
Here I experienced using the Amazon S3 platform to perform a series of related operations. At the same time, the official also provides a variety of usage methods, such as: Amazon Cloud Technology Command Line Interface (Amazon Cloud Technology CLI) and Amazon S3 REST API etc. Friends can read the official documentation for more professional guidance ->Amazon S3 official documentation
3. Use API to operate S3 Express One Zone
Creates a directory bucket and you can immediately experience very low-latency read and write operations. This experience can be achieved by establishing an endpoint connection (VPC), and can also perform API operations through regional and regional endpoints to manage object and directory buckets.
To create a directory bucket using the API, you need to go through the following steps:
1. Have appropriate IAM permissions : In IAM Under the user's identity, ensure that the user has the s3express:CreateBucket permission. This can be accomplished by adding permissions like the following to the user's IAM policy. For example: Please ensure that the directory bucket that will be created has the correct name
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowAccessRegionalEndpointAPIs",
"Effect": "Allow",
"Action": [
"s3express:DeleteBucket",
"s3express:DeleteBucketPolicy",
"s3express:CreateBucket",
"s3express:PutBucketPolicy",
"s3express:GetBucketPolicy",
"s3express:ListAllMyDirectoryBuckets"
],
"Resource": "arn:xxx:s3express:us-west-2:111122223333:bucket/DOC-EXAMPLE-BUCKET--usw2-az1--x-s3/*"
},
{
"Sid": "AllowCreateSession",
"Effect": "Allow",
"Action": "s3express:CreateSession",
"Resource": "*"
}
]
}
2. Use API calls to create directory buckets: Use tools such as Amazon Cloud SDK or Amazon Cloud CLI to call the CreateBucket API operation. The following is an example using the Amazon Web Services CLI:
xxx s3api create-bucket --bucket your-bucket-name --region your-region
Replace "your-bucket-name" with the name of the directory bucket you want to create, and "your-region" with the Amazon Cloud Technology region where you want the bucket to be located.
3. Verify creation: After the create request is successful, you can verify the directory bucket by inspecting the returned information or using other Amazon Web Services S3 API operations. create. For example, you can list buckets using the following command:
xxx s3api list-buckets
For details, please refer toAmazon S3 Express One Zone usage documentation
4. Break through tradition and look to the future
Amazon S3 Express One Zone is a high-performance single-zone Amazon S3 storage class built to provide consistent single-digit millisecond data access applications for the most latency-sensitive customers. Compared with S3 standard storage, S3 Express One Zone has lower latency and higher cost performance. Data access speed is faster than standard storage, and the request cost is as low as 3%. This means applications can immediately benefit from faster data access, and using the Amazon S3 API, developers can easily expand or shrink storage.
At the same time, S3 Express One Zone provides performance elasticity, which not only meets the needs of extremely latency-sensitive applications, but also supports hundreds of thousands of requests per second. Data is stored in a new bucket type, Amazon S3 directory buckets, each directory bucket supports hundreds of thousands of transactions per second, regardless of key name or access mode .
Amazon S3 Express One Zone is especially suitable for scenarios that are extremely sensitive to latency.By using dedicated software and hardware to accelerate data processing, it can reduce the data latency of high-frequency access. the burden it brings.
Amazon S3 Express One Zone provides customers with an innovative solution for high-performance storage that delivers significant performance improvements and cost-effectiveness compared to traditional build methods. Traditional construction methods often require management across multiple storage systems and make compromises in data availability and durability. Amazon S3 Express One Zone stores data in Amazon S3 directory buckets through a sophisticated integration of dedicated hardware and software to achieve faster data access.
Compared with traditional construction methods, Amazon S3 Express One Zone performs well in performance. Its data access speed is 10 times faster than Amazon S3 Standard. It is especially suitable for performance-critical applications that are extremely sensitive to latency. Applications such as query-intensive analytics. In addition, by placing data and computing resources in the same Amazon Cloud Technology Availability Zone, customers not only have more flexibility to expand or shrink storage, but also can run workloads at lower computing costs, reducing overall costs.
Whether it is for developers or enterprise users, Amazon S3 Express One Zone provides powerful performance and broad applicability. In terms of data management and operation, it is flexible and highly performant, allowing users to better adapt to changing needs. As an upgraded version of Amazon S3, Amazon S3 Express One Zone once again demonstrates the technical strength of Amazon Cloud Technology in the field of cloud storage, providing users with a better data storage experience.