Resolving "Access Denied" Errors Encountered When Accessing Amazon S3 Objects

As a user of Amazon S3, you may encounter "Access Denied" errors when trying to access objects in an S3 bucket. These errors indicate that the request lacks valid credentials or permission policies to perform the requested operation.

In this blog post, I'll walk through various troubleshooting steps and configuration checks to resolve "Access Denied" errors when accessing S3 objects.

The Amazon cloud technology developer community provides developers with global development technology resources. There are technical documents, development cases, technical columns, training videos, activities and competitions, etc. Help Chinese developers connect with the world's most cutting-edge technologies, ideas, and projects, and recommend outstanding Chinese developers or technologies to the global cloud community. If you haven't paid attention/favorite yet, please don't rush over when you see this, click here to make it your technical treasure house!

Automate Documentation Using Amazon Systems Manager

Amazon Web Services provides two Systems Manager automation documents that can help diagnose access issues with your S3 buckets:

AmazonSupport-TroubleshootS3PublicRead - Use this document to check for public read access issues with your S3 buckets.

AmazonSupport-TroubleshootS3AccessSameAccount - Use this document to diagnose access denied errors from your own S3 bucket.

These documents automatically run diagnostics and provide recommended solutions based on your bucket configuration. I highly recommend using them as a first troubleshooting step.

To run these documents:

Open the Amazon Systems Manager console and go to the Automation section .
Search for the document name.
Click "Execute Automation".
Specify the required parameters, such as the S3 bucket name.
Review the results and implement the recommendations.

Automated documentation will check bucket policies, object ownership, user credentials, and more to identify root causes. This can save you hours of manual troubleshooting.

Check ownership of buckets and objects

An "Access Denied" error may occur if the object being accessed has a different owner than the bucket. By default, even if an uploader uploads an object to your bucket, the uploader owns the object.

Follow the steps below to check object ownership:

Use the Amazon CLI to get your account's canonical ID:

aws s3api list-buckets --query "Owner.ID"

2. Get the canonical ID of the object owner:

aws s3api list-objects --bucket mybucket --prefix myobject

3. If the IDs do not match, the object owner must grant you the ACL of Full Control:

aws s3api put-object-acl --bucket mybucket --key myobject --acl bucket-owner-full-control

4. Update object ownership by copying the object to itself:

aws s3 cp s3://mybucket/myobject s3://mybucket/myobject

To prevent this issue from occurring in the future, it is required to set the bucket-owner-full-control ACL on object uploads and enable S3 object ownership .

Check bucket policy and IAM policy

Carefully check the bucket policy and IAM user policy for statements that might inadvertently deny access.

pay attention:

Deny statements containing conditions such as multi-factor authentication , IP address, VPC , etc.
Operations such as s3:GetObject or s3:PutObjectAcl are missing
There are extra spaces or typos in the ARN
overly restrictive subject element

For example, this bucket policy denies access based on VPC:

{
  "Id": "Policy1234567890123",
  "Version": "2012-10-17", 
  "Statement": [
    {
      "Sid": "Statement1",
      "Action": [
        "s3:GetObject"
      ],
      "Effect": "Allow",
      "Resource": "arn:aws-cn:s3:::DOC-EXAMPLE-BUCKET/*",
      "Principal": "*"
    },
    {
      "Sid": "Statement2",
      "Action": [
        "s3:GetObject" 
      ],
      "Effect": "Deny",
      "Resource": "arn:aws-cn:s3:::DOC-EXAMPLE-BUCKET/*",
      "Condition": {
        "StringNotEquals": {
          "aws:SourceVpce": "vpce-1a2b3c4d"
        }
      },
      "Principal": "*"  
    }
  ]
}

In the above policy, Statement1 allows fetching of objects. But if the request is not from VPC endpoint vpce-1a2b3c4d, Statement2 denies the same GET access. Therefore, users outside of that VPC endpoint will receive an "Access Denied" error.

And this policy is missing the critical s3:PutObjectAcl operation:

{
  "Id": "Policy1234567890123",
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Stmt1234567890123",
      "Action": [ 
        "s3:PutObject"
      ],
      "Effect": "Allow",
      "Resource": "arn:aws-cn:s3:::DOC-EXAMPLE-BUCKET/*",
      "Principal": {
        "AWS": [
          "arn:aws-cn:iam::111122223333:user/Dave" 
        ]
      }
    }
  ]
}

The above policy allows only s3:PutObject operations. If a user attempts to modify an object's ACL using s3:PutObjectAcl, they will receive an "Access Denied" error because the operation is not permitted.

Ideally, there is no problem running the AWS Policy Builder tool to validate the policy.

If there are IAM permission boundaries set on the identity , check them too.

Check S3 public access blocking settings

If you get access denied on public object requests, review the S3 public access blocking settings at the account and bucket level. These settings can override permissions that allow public access.

Check these settings on your account and bucket using the S3 console.

Review user credentials

Make sure the IAM user or role accessing the bucket has the proper credentials configured:

For the CLI , check the configured credentials:

aws configure list

For EC2 , check the role attached to the instance:

aws sts get-caller-identity

For temporary credentials obtained through STS , check the S3 permissions in the session policy associated with the assumed role.

Check VPC endpoint policies

If accessing S3 through a VPC endpoint , make sure the endpoint policy grants the required permissions. This policy controls which buckets/objects can be accessed through this endpoint.

For example:

{
  "Id": "Policy1234567890123",  
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Stmt1234567890123",
      "Action": [
        "s3:GetObject",
        "s3:PutObject",
        "s3:ListBucket"
      ],
      "Effect": "Allow",
      "Resource": [
        "arn:aws-cn:s3:::DOC-EXAMPLE-BUCKET",
        "arn:aws-cn:s3:::DOC-EXAMPLE-BUCKET/*"
      ],
      "Principal": "*"
    }
  ] 
}

The above VPC endpoint policy:

Allow getting, putting and listing objects in the DOC-EXAMPLE-BUCKET bucket
Use wildcards to allow access to any object in that bucket
Applies to all principals accessed through the VPC endpoint
but only allows access to the DOC-EXAMPLE-BUCKET bucket

Check S3 access point policy

If using an S3 access point , the access point policy must allow access to the underlying bucket. So check the access point policy and bucket policy.

Check for missing objects

If the object being accessed does not exist in the bucket, S3 returns an "Access Denied" error instead of a 404. Check if the object actually exists:

aws s3api head-object --bucket mybucket --key myobject

If it doesn't exist, solve the actual object problem.

Confirm KMS encryption key access

If S3 objects encrypted using Amazon KMS (SSE-KMS) cannot be accessed by users with valid permissions, make sure that:

The KMS key policy grants the required permissions (eg kms:Decrypt)
If the IAM user is not in the same account as the KMS key, the IAM policy also includes KMS permissions

Specify Requester Payment Parameters

If requestor-pays for the bucket is enabled , the cross-account user must pass --request-payer the parameter:

aws s3 cp s3://mybucket/myobject . --request-payer requester

Check Amazon Organizations Policies

Verify that the Amazon Organizations service control policy allows your account to access S3. An explicit deny policy will override any allow policy.

This covers the main areas to check when troubleshooting "Access Denied" errors on S3 objects. Some points:

Automated Diagnosis Using Systems Manager Automated Documentation
Review bucket policies, object ownership, and access point policies
Check resource policies such as IAM permissions and VPC endpoint policies
If using ephemeral tokens, verify credentials and session policies
Check for object existence and special characters

Fix these for your configuration and you should be able to access your S3 objects without encountering any "Access Denied" errors. If you have any other troubleshooting tips, please let me know in the comments!