AWS PrivateLink for Amazon S3 (original) (raw)

With AWS PrivateLink for Amazon S3, you can provision interface VPC endpoints (interface endpoints) in your virtual private cloud (VPC). These endpoints are directly accessible from applications that are on premises over VPN and AWS Direct Connect, or in a different AWS Region over VPC peering.

Interface endpoints are represented by one or more elastic network interfaces (ENIs) that are assigned private IP addresses from subnets in your VPC. Requests to Amazon S3 over interface endpoints stay on the Amazon network. You can also access interface endpoints in your VPC from on-premises applications through AWS Direct Connect or AWS Virtual Private Network (AWS VPN). For more information about how to connect your VPC with your on-premises network, see the AWS Direct Connect User Guide and the AWS Site-to-Site VPN User Guide.

For general information about interface endpoints, see Interface VPC endpoints (AWS PrivateLink) in the AWS PrivateLink Guide.

Topics

Types of VPC endpoints for Amazon S3

You can use two types of VPC endpoints to access Amazon S3: gateway endpoints and interface endpoints (by using AWS PrivateLink). A gateway endpoint is a gateway that you specify in your route table to access Amazon S3 from your VPC over the AWS network. Interface endpoints extend the functionality of gateway endpoints by using private IP addresses to route requests to Amazon S3 from within your VPC, on premises, or from a VPC in another AWS Region by using VPC peering or AWS Transit Gateway. For more information, see What is VPC peering? and Transit Gateway vs VPC peering.

Interface endpoints are compatible with gateway endpoints. If you have an existing gateway endpoint in the VPC, you can use both types of endpoints in the same VPC.

Gateway endpoints for Amazon S3 Interface endpoints for Amazon S3
In both cases, your network traffic remains on the AWS network.
Use Amazon S3 public IP addresses Use private IP addresses from your VPC to access Amazon S3
Use the same Amazon S3 DNS names Require endpoint-specific Amazon S3 DNS names
Do not allow access from on premises Allow access from on premises
Do not allow access from another AWS Region Allow access from a VPC in another AWS Region by using VPC peering or AWS Transit Gateway
Not billed Billed

For more information about gateway endpoints, see Gateway VPC endpoints in the_AWS PrivateLink Guide_.

VPC limitations apply to AWS PrivateLink for Amazon S3. For more information, see Interface endpoint considerations and AWS PrivateLink quotas in the_AWS PrivateLink Guide_. In addition, the following restrictions apply.

AWS PrivateLink for Amazon S3 does not support the following:

Creating a VPC endpoint

To create a VPC interface endpoint, see Create a VPC endpoint in the AWS PrivateLink Guide.

Accessing Amazon S3 interface endpoints

When you create an interface endpoint, Amazon S3 generates two types of endpoint-specific, S3 DNS names: Regional and zonal.

Endpoint-specific S3 DNS names can be resolved from the S3 public DNS domain.

Private DNS

Private DNS options for VPC interface endpoints simplify routing S3 traffic over VPC endpoints and help you take advantage of the lowest-cost network path available to your application. You can use private DNS options to route Regional S3 traffic without updating your S3 clients to use the endpoint-specific DNS names of your interface endpoints, or managing DNS infrastructure. With private DNS names enabled, Regional S3 DNS queries resolve to the private IP addresses of AWS PrivateLink for the following endpoints:

AWS PrivateLink for Amazon S3 does not support Using Amazon S3 dual-stack endpoints. If you use an S3 dual-stack DNS name as a private DNS name, your IPv6 traffic will either be dropped or, if your virtual private cloud (VPC) has an internet gateway, your IPv6 traffic will be routed over the internet gateway in your VPC.

If you have a gateway endpoint in your VPC, you can automatically route in-VPC requests over your existing S3 gateway endpoint and on-premises requests over your interface endpoint. This approach allows you to optimize your networking costs by using gateway endpoints, which are not billed, for your in-VPC traffic. Your on-premises applications can use AWS PrivateLink with the help of the inbound Resolver endpoint. Amazon provides a DNS server, called the Route 53 Resolver, for your VPC. An inbound Resolver endpoint forwards DNS queries from the on-premises network to Route 53 Resolver.

Important

To take advantage of the lowest cost network path when using Enable private DNS only for inbound endpoints, a gateway endpoint must be present in your VPC. The presence of a gateway endpoint helps ensure that in-VPC traffic always routes over the AWS private network when the Enable private DNS only for inbound endpoints option is selected. You must maintain this gateway endpoint while you have the Enable private DNS only for inbound endpoints option selected. If you want to delete your gateway endpoint you must first clear Enable private DNS only for inbound endpoints.

If you want to update an existing interface endpoint to Enable private DNS only for inbound endpoints, first confirm that your VPC has an S3 gateway endpoint. For more information about gateway endpoints and managing private DNS names, seeGateway VPC endpoints and Manage DNS names respectively in the AWS PrivateLink Guide.

The Enable private DNS only for inbound endpoints option is available only for services that support gateway endpoints.

For more information about creating a VPC endpoint that uses Enable private DNS only for inbound endpoints, see Create an interface endpoint in the AWS PrivateLink Guide.

Using the VPC console

In the console you have two options: Enable DNS name andEnable private DNS only for inbound endpoints. Enable DNS name is an option supported by AWS PrivateLink. By using the Enable DNS name option, you can use Amazon’s private connectivity to Amazon S3, while making requests to the default public endpoint DNS names. When this option is enabled, customers can take advantage of the lowest cost network path available to their application.

When you enable private DNS names on an existing or new VPC interface endpoint for Amazon S3, the Enable private DNS only for inbound endpoints option is selected by default. If this option is selected, your applications use only interface endpoints for your on-premises traffic. This in-VPC traffic automatically uses the lower-cost gateway endpoints. Alternatively, you can clear Enable private DNS only for inbound endpoints to route all S3 requests over your interface endpoint.

Using the AWS CLI

If you don't specify a value for PrivateDnsOnlyForInboundResolverEndpoint, it will default to true. However, before your VPC applies your settings, it performs a check to make sure that you have a gateway endpoint present in the VPC. If a gateway endpoint is present in the VPC, the call succeeds. If not, you will see the following error message:

To set PrivateDnsOnlyForInboundResolverEndpoint to true, the VPC`vpce_id` must have a gateway endpoint for the service.

For a new VPC Interface endpoint

Use the private-dns-enabled and dns-options attributes to enable private DNS through the command line. ThePrivateDnsOnlyForInboundResolverEndpoint option in the dns-options attribute must be set to true. Replace the `user input placeholders` with your own information.


aws ec2 create-vpc-endpoint \
--region us-east-1 \
--service-name s3-service-name \
--vpc-id client-vpc-id \
--subnet-ids client-subnet-id \ 
--vpc-endpoint-type Interface  \
--private-dns-enabled  \
--ip-address-type ip-address-type \ 
--dns-options PrivateDnsOnlyForInboundResolverEndpoint=true \
--security-group-ids client-sg-id

For an existing VPC endpoint

If you want to use private DNS for an existing VPC endpoint, use the following example command and replace the `user input placeholders` with your own information.


aws ec2 modify-vpc-endpoint \
--region us-east-1 \
--vpc-endpoint-id client-vpc-id \
--private-dns-enabled \
--dns-options PrivateDnsOnlyForInboundResolverEndpoint=false

If you want to update an existing VPC endpoint to enable private DNS only for the Inbound Resolver, use the following example and replace the sample values with your own.


aws ec2 modify-vpc-endpoint \
--region us-east-1 \
--vpc-endpoint-id client-vpc-id \
--private-dns-enabled \
--dns-options PrivateDnsOnlyForInboundResolverEndpoint=true

Accessing buckets, access points, and Amazon S3 Control API operations from S3 interface endpoints

You can use the AWS CLI or AWS SDKs to access buckets, S3 access points, and Amazon S3 Control API operations through S3 interface endpoints.

The following image shows the VPC console Details tab, where you can find the DNS name of a VPC endpoint. In this example, the VPC endpoint ID (vpce-id) is vpce-0e25b8cdd720f900e and the DNS name is*.vpce-0e25b8cdd720f900e-argc85vg.s3.us-east-1.vpce.amazonaws.com.

The Details tab in the VPC console.

When using the DNS name to access a resource, replace * with the appropriate value. The appropriate values to use in place of * are as follows:

For example, to access a bucket, use a DNS name like this:

bucket.vpce-0e25b8cdd720f900e-argc85vg.s3.us-east-1.vpce.amazonaws.com

For examples of how to use DNS names to access buckets, access points, and Amazon S3 Control API operations, see the following sections of AWS CLI examples and AWS SDK examples.

For more information about how to view your endpoint-specific DNS names, see Viewing endpoint service private DNS name configuration in the VPC User Guide.

AWS CLI examples

To access S3 buckets, S3 access points, or Amazon S3 Control API operations through S3 interface endpoints in AWS CLI commands, use the --region and --endpoint-url parameters.

Example: Use an endpoint URL to list objects in your bucket

In the following example, replace the bucket name `my-bucket`, Region`us-east-1`, and the DNS name of the VPC endpoint ID`vpce-1a2b3c4d-5e6f.s3.us-east-1.vpce.amazonaws.com` with your own information.

aws s3 ls s3://my-bucket/ --region us-east-1 --endpoint-url https://bucket.vpce-1a2b3c4d-5e6f.s3.us-east-1.vpce.amazonaws.com

Example: Use an endpoint URL to list objects from an access point

aws s3api list-objects-v2 --bucket arn:aws:s3:us-east-1:123456789012:accesspoint/accesspointexamplename --region us-east-1 --endpoint-url https://accesspoint.vpce-1a2b3c4d-5e6f.s3.us-east-1.vpce.amazonaws.com

If you can't run the command successfully, update your AWS CLI to the latest version and try again. For more information on the update instructions, see Installing or updating the latest version of the AWS CLI in the_AWS Command Line Interface User Guide_.

aws s3api list-objects-v2 --bucket accesspointexamplename-8tyekmigicmhun8n9kwpfur39dnw4use1a-s3alias --region us-east-1 --endpoint-url https://bucket.vpce-1a2b3c4d-5e6f.s3.us-east-1.vpce.amazonaws.com  
aws configure set default.s3.addressing_style virtual

Then, in the following example, replace the access point alias `accesspointexamplename-8tyekmigicmhun8n9kwpfur39dnw4use1a-s3alias`, the Region `us-east-1`, and the VPC endpoint ID`vpce-1a2b3c4d-5e6f.s3.us-east-1.vpce.amazonaws.com` with your own information. For more information about access point alias, see Access point for general purpose buckets aliases. 

aws s3api list-objects-v2 --bucket accesspointexamplename-8tyekmigicmhun8n9kwpfur39dnw4use1a-s3alias --region us-east-1 --endpoint-url https://accesspoint.vpce-1a2b3c4d-5e6f.s3.us-east-1.vpce.amazonaws.com
Example: Use an endpoint URL to list jobs with an S3 control API operation

In the following example, replace the Region`us-east-1`, the VPC endpoint ID`vpce-1a2b3c4d-5e6f.s3.us-east-1.vpce.amazonaws.com`, and the account ID `12345678` with your own information.

aws s3control --region us-east-1 --endpoint-url https://control.vpce-1a2b3c4d-5e6f.s3.us-east-1.vpce.amazonaws.com list-jobs --account-id 12345678

AWS SDK examples

To access S3 buckets, S3 access points, or Amazon S3 Control API operations through S3 interface endpoints when using the AWS SDKs, update your SDKs to the latest version. Then configure your clients to use an endpoint URL for accessing a bucket, access point, or Amazon S3 Control API operations through S3 interface endpoints.

SDK for Python (Boto3)

Example: Use an endpoint URL to access an S3 bucket

In the following example, replace the Region`us-east-1` and VPC endpoint ID`vpce-1a2b3c4d-5e6f.s3.us-east-1.vpce.amazonaws.com` with your own information. 

s3_client = session.client(
service_name='s3',
region_name='us-east-1',
endpoint_url='https://bucket.vpce-1a2b3c4d-5e6f.s3.us-east-1.vpce.amazonaws.com'
)
Example: Use an endpoint URL to access an S3 access point

In the following example, replace the Region`us-east-1` and VPC endpoint ID`vpce-1a2b3c4d-5e6f.s3.us-east-1.vpce.amazonaws.com` with your own information. 

ap_client = session.client(
service_name='s3',
region_name='us-east-1',
endpoint_url='https://accesspoint.vpce-1a2b3c4d-5e6f.s3.us-east-1.vpce.amazonaws.com'
)
Example: Use an endpoint URL to access the Amazon S3 Control API

In the following example, replace the Region`us-east-1` and VPC endpoint ID`vpce-1a2b3c4d-5e6f.s3.us-east-1.vpce.amazonaws.com` with your own information. 

control_client = session.client(
service_name='s3control',
region_name='us-east-1',
endpoint_url='https://control.vpce-1a2b3c4d-5e6f.s3.us-east-1.vpce.amazonaws.com'
)

SDK for Java 1.x

Example: Use an endpoint URL to access an S3 bucket

In the following example, replace the VPC endpoint ID`vpce-1a2b3c4d-5e6f.s3.us-east-1.vpce.amazonaws.com` with your own information. 

// bucket client
final AmazonS3 s3 = AmazonS3ClientBuilder.standard().withEndpointConfiguration(
        new AwsClientBuilder.EndpointConfiguration(
                "https://bucket.vpce-1a2b3c4d-5e6f.s3.us-east-1.vpce.amazonaws.com",
                Regions.DEFAULT_REGION.getName()
        )
).build();
List<Bucket> buckets = s3.listBuckets();
Example: Use an endpoint URL to access an S3 access point

In the following example, replace the VPC endpoint ID`vpce-1a2b3c4d-5e6f.s3.us-east-1.vpce.amazonaws.com` and ARN`us-east-1:123456789012:accesspoint/prod` with your own information. 

// accesspoint client
final AmazonS3 s3accesspoint = AmazonS3ClientBuilder.standard().withEndpointConfiguration(
        new AwsClientBuilder.EndpointConfiguration(
                "https://accesspoint.vpce-1a2b3c4d-5e6f.s3.us-east-1.vpce.amazonaws.com",
                Regions.DEFAULT_REGION.getName()
        )
).build();
ObjectListing objects = s3accesspoint.listObjects("arn:aws:s3:us-east-1:123456789012:accesspoint/prod");
Example: Use an endpoint URL to access an Amazon S3 Control API operation

In the following example, replace the VPC endpoint ID`vpce-1a2b3c4d-5e6f.s3.us-east-1.vpce.amazonaws.com` with your own information. 

// control client
final AWSS3Control s3control = AWSS3ControlClient.builder().withEndpointConfiguration(
        new AwsClientBuilder.EndpointConfiguration(
                "https://control.vpce-1a2b3c4d-5e6f.s3.us-east-1.vpce.amazonaws.com",
                Regions.DEFAULT_REGION.getName()
        )
).build();
final ListJobsResult jobs = s3control.listJobs(new ListJobsRequest());

SDK for Java 2.x

Example: Use an endpoint URL to access an S3 bucket

In the following example, replace the VPC endpoint ID`vpce-1a2b3c4d-5e6f.s3.us-east-1.vpce.amazonaws.com` and the Region `Region.US_EAST_1` with your own information.

// bucket client
Region region = Region.US_EAST_1;
s3Client = S3Client.builder().region(region)
                   .endpointOverride(URI.create("https://bucket.vpce-1a2b3c4d-5e6f.s3.us-east-1.vpce.amazonaws.com"))
                   .build()
Example: Use an endpoint URL to access an S3 access point

In the following example, replace the VPC endpoint ID`vpce-1a2b3c4d-5e6f.s3.us-east-1.vpce.amazonaws.com` and the Region `Region.US_EAST_1` with your own information.

// accesspoint client
Region region = Region.US_EAST_1;
s3Client = S3Client.builder().region(region)
                   .endpointOverride(URI.create("https://accesspoint.vpce-1a2b3c4d-5e6f.s3.us-east-1.vpce.amazonaws.com"))
                   .build()
Example: Use an endpoint URL to access the Amazon S3 Control API

In the following example, replace the VPC endpoint ID`vpce-1a2b3c4d-5e6f.s3.us-east-1.vpce.amazonaws.com` and the Region `Region.US_EAST_1` with your own information.

// control client
Region region = Region.US_EAST_1;
s3ControlClient = S3ControlClient.builder().region(region)
                                 .endpointOverride(URI.create("https://control.vpce-1a2b3c4d-5e6f.s3.us-east-1.vpce.amazonaws.com"))
                                 .build()

Updating an on-premises DNS configuration

When using endpoint-specific DNS names to access the interface endpoints for Amazon S3, you don’t have to update your on-premises DNS resolver. You can resolve the endpoint-specific DNS name with the private IP address of the interface endpoint from the public Amazon S3 DNS domain.

Using interface endpoints to access Amazon S3 without a gateway endpoint or an internet gateway in the VPC

Interface endpoints in your VPC can route both in-VPC applications and on-premises applications to Amazon S3 over the Amazon network, as illustrated in the following diagram.

Data-flow diagram showing access to Amazon S3 using an interface endpoint and AWS PrivateLink.

The diagram illustrates the following:

Using gateway endpoints and interface endpoints together in the same VPC to access Amazon S3

You can create interface endpoints and retain the existing gateway endpoint in the same VPC, as the following diagram shows. By taking this approach, you allow in-VPC applications to continue accessing Amazon S3 through the gateway endpoint, which is not billed. Then, only your on-premises applications would use interface endpoints to access Amazon S3. To access Amazon S3 this way, you must update your on-premises applications to use endpoint-specific DNS names for Amazon S3.

Data-flow diagram showing access to Amazon S3 using gateway endpoints and interface endpoints.

The diagram illustrates the following:

For more information about gateway endpoints, see Gateway VPC endpoints in the_VPC User Guide_.

Creating a VPC endpoint policy for Amazon S3

You can attach an endpoint policy to your VPC endpoint that controls access to Amazon S3. The policy specifies the following information:

You can also use Amazon S3 bucket policies to restrict access to specific buckets from a specific VPC endpoint by using the aws:sourceVpce condition in your bucket policy. The following examples show policies that restrict access to a bucket or to an endpoint.

Topics

Example: Restricting access to a specific bucket from a VPC endpoint

You can create an endpoint policy that restricts access to only specific Amazon S3 buckets. This type of policy is useful if you have other AWS services in your VPC that use buckets. The following bucket policy restricts access to only the`` `amzn-s3-demo-bucket1` ``. To use this endpoint policy, replace `` `amzn-s3-demo-bucket1` `` with the name of your bucket.

{
  "Version": "2012-10-17",
  "Id": "Policy1415115909151",
  "Statement": [
    { "Sid": "Access-to-specific-bucket-only",
      "Principal": "*",
      "Action": [
        "s3:GetObject",
        "s3:PutObject"
      ],
      "Effect": "Allow",
      "Resource": ["arn:aws:s3:::amzn-s3-demo-bucket1",
                   "arn:aws:s3:::amzn-s3-demo-bucket1/*"]
    }
  ]
}

Example: Restricting access to buckets in a specific account from a VPC endpoint

You can create an endpoint policy that restricts access to only the S3 buckets in a specific AWS account. To prevent clients within your VPC from accessing buckets that you don't own, use the following statement in your endpoint policy. The following example statement creates a policy that restricts access to resources owned by a single AWS account ID, `111122223333`.

{
  "Statement": [
    {
      "Sid": "Access-to-bucket-in-specific-account-only",
      "Principal": "*",
      "Action": [
        "s3:GetObject",
        "s3:PutObject"
      ],
      "Effect": "Deny",
      "Resource": "arn:aws:s3:::*",
      "Condition": {
        "StringNotEquals": {
          "aws:ResourceAccount": "111122223333"
        }
      }
    }
  ]
}
Note

To specify the AWS account ID of the resource being accessed, you can use either theaws:ResourceAccount or the s3:ResourceAccount key in your IAM policy. However, be aware that some AWS services rely on access to AWS managed buckets. Therefore, using the aws:ResourceAccount ors3:ResourceAccount key in your IAM policy might also affect access to these resources.

Example: Restricting access to a specific VPC endpoint in the S3 bucket policy

Example: Restricting access to a specific VPC endpoint in the S3 bucket policy

The following Amazon S3 bucket policy allows access to a specific bucket,`` `amzn-s3-demo-bucket2` ``, from only the VPC endpoint`vpce-1a2b3c4d`. The policy denies all access to the bucket if the specified endpoint is not being used. The aws:sourceVpce condition specifies the endpoint and doesn't require an Amazon Resource Name (ARN) for the VPC endpoint resource, only the endpoint ID. To use this bucket policy, replace`` `amzn-s3-demo-bucket2` `` and`vpce-1a2b3c4d` with your bucket name and endpoint.

Important
{
  "Version": "2012-10-17",
  "Id": "Policy1415115909152",
  "Statement": [
    { "Sid": "Access-to-specific-VPCE-only",
      "Principal": "*",
      "Action": "s3:*",
      "Effect": "Deny",
      "Resource": ["arn:aws:s3:::amzn-s3-demo-bucket2",
                   "arn:aws:s3:::amzn-s3-demo-bucket2/*"],
      "Condition": {"StringNotEquals": {"aws:sourceVpce": "vpce-1a2b3c4d"}}
    }
  ]
}

For more policy examples, see Endpoints for Amazon S3 in the VPC User Guide.

For more information about VPC connectivity, see Network-to-VPC connectivity options in the AWS whitepaper Amazon Virtual Private Cloud Connectivity Options.