AWS S3
Business scene
challenge
- Web server for transferring large files may cause network latency
- User generated content needs to be allocated to all of the web server
solution
- Static assets stored in S3
- The S3 object to disclosed, users can access
S3's benefits
- Or access via HTTP API
- S3 is redundantly stored objects, at least three copies of the available storage area
- S3 uses a checksum to verify the data integrity
- To provide better version control enables users to save, retrieve and restore versions of each object
- Multiple copies of data consistency
S3 property
Buckets Buckets
- S3 storage containers
- Bucket name can contain up to 63 characters, and the world must be unique AWS
- Best practice is to contain and comply with specifications DNS domain name to name the Bucket
- Create a Bucket in the area close to the user to optimize performance
- Each user can create up to 100 buckets
- Buckets need to specify the area, but you can not specify the available area
Object Object
- Each entity or file stored in S3 is an object
- Maximum support single object 5TB
- Objects and metadata from the document itself composed of
- Document itself seems to be just a bunch of bytes in the stream S3
- Metadata contains the name and key objects, system metadata including MD5, last modification date and other system information, user metadata is the user's option
- Request object used to obtain Baotou
- x-amz-meta-
- x-amz-storage-class
- Content-MD5
- Content-Length
data structure
Key (Key): can be considered is the name of the data
value (Value): represents the data itself is
a version number (Version ID): It is important to enable the bucket for version control
metadata (Metadata): description about the data, such as said data creation time, modification time, file type, file size and other information
access control information: to manage access to files in the Bucket
Key
Identifying each object, the only storage bucket
UTF-8 characters maximum 1024 bytes, slash and the like can contain
the unique identification name + Key bucket S3 object composed
best practice is to use naming / name the object logical organization, but remember that this does not mean that the file is organized hierarchically, S3 object is always in a plane rather than a true System file
S3 objects based on their unique identification to access the Internet directly addressed, URL is its unique identification
Object Label
Each object is added up to 10 label
after label markers can be used IAM Policy Management
Characteristics S3
S3 operation
Interface operation based REST API, will map HTTP verbs to CRUD operations
actually use more advanced interfaces (such as programming languages, SDK, CLI and console) to call the REST API
Availability and durability
S3 storage infrastructure using highly durable and available
Persistent = 11 9
availability 4 = 9
Recommended version control, cross-domain zone copy, MFA delete other ways to prevent users from accidentally caused by a logic error
consistency
S3 will automatically replicate data automatically in the region
S3 is an eventually consistent system to ensure all get are the same
For the new object to write after reading PUTS provide consistency
to provide consistency for the final Overwrite PUTS and DELETE
S3 pricing strategy
The storage capacity of
a read operation
of data transmission
Advanced features S3
storage levels
S3-Standard
It provides high durability, high availability, low latency and high performance storage objects
S3-Standard-IA
Standard high availability offering lower but still more than usable area of deployment, but is designed to lower long-term access and data access frequency,
using the scale on storage for more than 30 days to provide more low-cost options, minimal fees particle size of 128KB, minimum 30 days.
IA-S3-ONEZONE
Similar to the Standard-IA, but only an available area of deployment provides high availability
for non-critical data easy to reproduce, to replace RRS
INTELLIGENT_TIERING
Automatic hierarchical data, similar to the persistence and availability IA
can provide a standard read and write performance, but not the same, depending on the frequency of read and write requests
additional monitoring and automatic hierarchical cost, minimum 30 days, less than 128K the object is not eligible for automatic tiered but still charges.
S3 reduce redundant memory RRS
Applicable to non-critical data easy to reproduce or derived data that supports low latency access, does not support high availability (4 9 Persistent)
is now no longer recommended, using the S3-ONEZONE-IA substitute
Glacier
Very low cost of archive data storage, real-time data access is not required