Access AWS S3 from Lambda within VPC

musingsole :

Overall, I'm pretty confused by using AWS Lambda within a VPC. The problem is Lambda is timing out while trying to access an S3 bucket. The solution seems to be a VPC Endpoint.

I've added the Lambda function to a VPC so it can access an RDS hosted database (not shown in the code below, but functional). However, now I can't access S3 and any attempt to do so times out.

I tried creating a VPC S3 Endpoint, but nothing has changed.

VPC Configuration

I'm using a simple VPC created by default whenever I first made an EC2 instance. It has four subnets, all created by default.

VPC Route Table

_Destination - Target - Status - Propagated_

172.31.0.0/16 - local - Active - No

pl-63a5400a (com.amazonaws.us-east-1.s3) - vpce-b44c8bdd - Active - No

0.0.0.0/0 - igw-325e6a56 - Active - No

Simple S3 Download Lambda:

import boto3
import pymysql
from StringIO import StringIO

def lambda_handler(event, context):
    s3Obj = StringIO()

    return boto3.resource('s3').Bucket('marineharvester').download_fileobj('Holding - Midsummer/sample', s3Obj)
Geoff :

With boto3, the S3 urls are virtual by default, which then require internet access to be resolved to region specific urls. This causes the hanging of the Lambda function until timeout.

To resolve this requires use of a Config object when creating the client, which tells boto3 to create path based S3 urls instead:

import boto3 
import botocore

client = boto3.client('s3', 'ap-southeast-2', config=botocore.config.Config(s3={'addressing_style':'path'}))

Note that the region in the call must be the region to which you are deploying the lambda and VPC Endpoint.

Then you will be able to use the pl-xxxxxx prefix list for the VPC Endpoint within the Lambda's security group, and still access S3.

Here is a working CloudFormation script that demonstrates this. It creates an S3 bucket, a lambda (that puts records into the bucket) associated to a VPC containing only private subnets and the VPC Endpoint, and necessary IAM roles.

この記事はインターネットから収集されたものであり、転載の際にはソースを示してください。

侵害の場合は、連絡してください[email protected]

編集
0

コメントを追加

0

関連記事

分類Dev

Access AWS S3 from Lambda within VPC

分類Dev

Access AWS S3 from Lambda within VPC

分類Dev

Access RDS from VPC Lambda

分類Dev

Can not use zip from S3 for AWS Lambda

分類Dev

aws access s3 from spark using IAM role

分類Dev

How to read csv file from s3 bucket in AWS Lambda?

分類Dev

Importing libraries in AWS Lambda function code from S3 bucket

分類Dev

R - How to copy and paste files from one location to another within an s3 bucket (using aws.s3)?

分類Dev

AWS API Gateway restricted access from S3 static web page only

分類Dev

Unable to access AWS s3 bucket from Private Google cloud composer

分類Dev

Boto does not save s3 file to disc on AWS Lambda

分類Dev

AWS Lambda using s3 getObject function nothing happening

分類Dev

NoSuchKey in resize images using python on AWS (lambda + s3)

分類Dev

VPC構成でAWS Lambdaを追加すると、S3にアクセスするときにタイムアウトが発生する

分類Dev

VPC構成でAWS Lambdaを追加すると、S3にアクセスするときにタイムアウトが発生する

分類Dev

VPC構成でAWS Lambdaを追加すると、S3にアクセスするときにタイムアウトが発生する

分類Dev

How to let AWS lambda in a VPC to publish SNS notification?

分類Dev

Make Lambda access EC2 port through a VPC

分類Dev

Lambda function in isolated VPC subnet can't access SSM parameter

分類Dev

How to give a VPC and all its instances access to a AWS Security Group

分類Dev

AWS Kinesis FirehoseからLambda、LambdaからS3へのJava

分類Dev

AWS Java Sdkを使用したAWS LambdaとS3の統合

分類Dev

Downloading folders from aws s3, cp or sync?

分類Dev

Get object from AWS S3 as a stream

分類Dev

Export big data from PostgreSQL to AWS s3

分類Dev

Uploading a Dataframe to AWS S3 Bucket from SageMaker

分類Dev

Upload to S3 from React Native with AWS Amplify

分類Dev

Export data from QlikSense cloud to AWS S3 bucket

分類Dev

Download Inventory File from AWS S3

Related 関連記事

  1. 1

    Access AWS S3 from Lambda within VPC

  2. 2

    Access AWS S3 from Lambda within VPC

  3. 3

    Access RDS from VPC Lambda

  4. 4

    Can not use zip from S3 for AWS Lambda

  5. 5

    aws access s3 from spark using IAM role

  6. 6

    How to read csv file from s3 bucket in AWS Lambda?

  7. 7

    Importing libraries in AWS Lambda function code from S3 bucket

  8. 8

    R - How to copy and paste files from one location to another within an s3 bucket (using aws.s3)?

  9. 9

    AWS API Gateway restricted access from S3 static web page only

  10. 10

    Unable to access AWS s3 bucket from Private Google cloud composer

  11. 11

    Boto does not save s3 file to disc on AWS Lambda

  12. 12

    AWS Lambda using s3 getObject function nothing happening

  13. 13

    NoSuchKey in resize images using python on AWS (lambda + s3)

  14. 14

    VPC構成でAWS Lambdaを追加すると、S3にアクセスするときにタイムアウトが発生する

  15. 15

    VPC構成でAWS Lambdaを追加すると、S3にアクセスするときにタイムアウトが発生する

  16. 16

    VPC構成でAWS Lambdaを追加すると、S3にアクセスするときにタイムアウトが発生する

  17. 17

    How to let AWS lambda in a VPC to publish SNS notification?

  18. 18

    Make Lambda access EC2 port through a VPC

  19. 19

    Lambda function in isolated VPC subnet can't access SSM parameter

  20. 20

    How to give a VPC and all its instances access to a AWS Security Group

  21. 21

    AWS Kinesis FirehoseからLambda、LambdaからS3へのJava

  22. 22

    AWS Java Sdkを使用したAWS LambdaとS3の統合

  23. 23

    Downloading folders from aws s3, cp or sync?

  24. 24

    Get object from AWS S3 as a stream

  25. 25

    Export big data from PostgreSQL to AWS s3

  26. 26

    Uploading a Dataframe to AWS S3 Bucket from SageMaker

  27. 27

    Upload to S3 from React Native with AWS Amplify

  28. 28

    Export data from QlikSense cloud to AWS S3 bucket

  29. 29

    Download Inventory File from AWS S3

ホットタグ

アーカイブ