Dependency: This article requires an understanding of the basics of AWS architecture design
If you want to process API requests asynchronously or add queues to your application architecture, you're in the right place.
This article describes how to integrate Amazon API Gateway as a proxy for SQS (Simple Queue Service).
Create an IAM Lambda execution role
additional strategy
AWSLambdaVPCAccessExecutionRole
AWSLambdaRole
SecretsManagerReadWrite
AmazonSSMFullAccess
AmazonS3FullAccess
CloudWatchFullAccess
trust relationship
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": [
"secretsmanager.amazonaws.com",
"lambda.amazonaws.com"
]
},
"Action": "sts:AssumeRole"
}
]
}
Create an IAM APIGateway SQS role
additional strategy
AWSLambdaVPCAccessExecutionRole
AWSLambdaRole
AWSLambdaSQSExecutionRole
AmazonAPIGatewayPushToCloudWatchLogs
AmazonAPIGatewayAdministrator
SecretsManagerReadWrite
trust relationship
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": [
"apigateway.amazonaws.com",
"lambda.amazonaws.com"
]
},
"Action": "sts:AssumeRole"
}
]
}
Create the APIGateway SQS lambda
AWSTemplateFormatVersion: 2010-09-09
Parameters:
Environment:
Type: String
Default: DEV
SubProjectName:
Description: The name of the sub project
Type: String
#TODO:
Default: your-subsystem-name
EnvironmentName:
Type: String
#TODO:
Default: d
CustomerName:
Description: The name of the customer
Type: String
#TODO:
Default: your-company-name
ProjectName:
Description: The name of the project
Type: String
#TODO:
Default: your-project-name
ProjectType:
Type: String
#TODO:
Default: iot
HostedZoneName:
Description: The name of route53 hosted
Type: String
#TODO:
Default: dev.xxxx.cn
Certificate:
Type: String
#TODO:
Default: xxxx-xxxx-xxx-xxx-xxxx
SQSKmsMasterKeyId:
Type: String
#TODO:
Default: xxxx-xxxx-xxx-xxx-xxxx
S3BktName:
Type: String
#TODO:
Default: xxxx-xxxx-d-s3
BaseLayerS3FileName:
Type: String
#TODO:
Default: xxxxxxxxxxxxxx
LambdaSubnetIds:
Description: The subnet ids of the lambda service
Type: List<AWS::EC2::Subnet::Id>
#TODO:
Default: subnet-xxxxxxxxxx,subnet-xxxxxxxxxx
LambdaSecurityGroups:
Description: security groups for lambda
Type: List<AWS::EC2::SecurityGroup::Id>
#TODO:
Default: sg-xxxxxxxxxx
Resources:
#create apigateway
APIGateway:
Type: AWS::ApiGateway::RestApi
Properties:
Description: API Endpoint to receive JSON payloads and queue in SQS
Name: !Sub ${
CustomerName}-${
ProjectName}-${
EnvironmentName}-agw
DisableExecuteApiEndpoint: true
Parameters:
endpointConfigurationTypes: REGIONAL
ignore: documentation
EndpointConfiguration:
Types:
- REGIONAL
Tags:
- Key: ApplName
Value: your-app-name
ProdDeployment:
Type: AWS::ApiGateway::Deployment
DependsOn:
- PostMethod
- CapPostMethod
Properties:
RestApiId: !Ref APIGateway
ProdStage:
Properties:
DeploymentId: !Ref ProdDeployment
RestApiId: !Ref APIGateway
StageName: !Ref EnvironmentName
Type: AWS::ApiGateway::Stage
V1Resource:
Type: AWS::ApiGateway::Resource
Properties:
ParentId: !GetAtt APIGateway.RootResourceId
PathPart: v1
RestApiId: !Ref APIGateway
#create api mapping
ApiDomainName:
Type: 'AWS::ApiGateway::DomainName'
Properties:
RegionalCertificateArn: !Sub arn:aws-cn:acm:${
AWS::Region}:${
AWS::AccountId}:certificate/${
Certificate}
DomainName: !Sub ${
ProjectName}${
ProjectType}.${
HostedZoneName}
EndpointConfiguration:
Types:
- REGIONAL
ApiPathMapping:
Type: 'AWS::ApiGateway::BasePathMapping'
Properties:
DomainName: !Ref ApiDomainName
RestApiId: !Ref APIGateway
Stage: !Ref ProdStage
#create api route53 record
Rount53DNSRecord:
Type: AWS::Route53::RecordSet
Properties:
HostedZoneName: !Sub ${
HostedZoneName}.
Comment: ApiGateway subsystem DNS.
Name: !Sub ${
ProjectName}${
ProjectType}.${
HostedZoneName}.
Type: A
AliasTarget:
HostedZoneId: !GetAtt 'ApiDomainName.RegionalHostedZoneId'
DNSName: !GetAtt 'ApiDomainName.RegionalDomainName'
#create lambda base layer
LambdaBaseLayer:
Type: AWS::Lambda::LayerVersion
Properties:
LayerName: !Sub ${
CustomerName}-${
ProjectName}-sqs-layerbase-${
EnvironmentName}-lambda
Description: SQS lambda base layer
Content:
S3Bucket: !Ref S3BktName
S3Key: !Ref BaseLayerS3FileName
CompatibleRuntimes:
- nodejs14.x
LicenseInfo: MIT
#create syb-system
Queue:
Type: AWS::SQS::Queue
Properties:
DelaySeconds: 0
MaximumMessageSize: 262144
MessageRetentionPeriod: 1209600
KmsMasterKeyId: !Ref SQSKmsMasterKeyId
QueueName: !Sub ${
CustomerName}-${
ProjectName}-${
SubProjectName}-${
EnvironmentName}-sqs
ReceiveMessageWaitTimeSeconds: 0
VisibilityTimeout: 30
Tags:
- Key: ApplName
Value: your-app-name
PolicySQS:
Type: AWS::SQS::QueuePolicy
Properties:
PolicyDocument:
Statement:
- Action: SQS:*
Effect: Allow
Principal: '*'
Resource: !GetAtt Queue.Arn
Version: '2012-10-17'
Queues:
- !Ref Queue
PostMethod:
Type: AWS::ApiGateway::Method
Properties:
AuthorizationType: CUSTOM
# Optional if SignatureAuthorizer is used
AuthorizerId: !Ref SignatureAuthorizer
# AuthorizationType: NONE
HttpMethod: POST
Integration:
Credentials: !Sub arn:aws-cn:iam::${
AWS::AccountId}:role/${
CustomerName}-${
ProjectName}-agw-sqs-${
EnvironmentName}-iamr
IntegrationHttpMethod: POST
IntegrationResponses:
- StatusCode: '200'
PassthroughBehavior: NEVER
RequestParameters:
integration.request.header.Content-Type: "'application/x-www-form-urlencoded'"
RequestTemplates:
application/json: Action=SendMessage&MessageBody=$input.body
Type: AWS
Uri: !Sub arn:aws-cn:apigateway:${
RegionName}:sqs:path/${
AWS::AccountId}/${
CustomerName}-${
ProjectName}-${
SubProjectName}-${
EnvironmentName}-sqs
MethodResponses:
- ResponseModels:
application/json: Empty
StatusCode: '200'
ResourceId: !Ref EnqueueResource
RestApiId: !Ref APIGateway
EnqueueResource:
Properties:
ParentId: !Ref V1Resource
PathPart: !Ref SubProjectName
RestApiId: !Ref APIGateway
Type: AWS::ApiGateway::Resource
#signature(optional)
SignatureAuthorizer:
Type: 'AWS::ApiGateway::Authorizer'
Properties:
AuthorizerCredentials: !Sub arn:aws-cn:iam::${
AWS::AccountId}:role/${
CustomerName}-${
ProjectName}-agw-sqs-${
EnvironmentName}-iamr
AuthorizerResultTtlInSeconds: '300'
AuthorizerUri: !Join
- ''
- - 'arn:aws-cn:apigateway:'
- !Ref 'AWS::Region'
- ':lambda:path/2015-03-31/functions/'
- !GetAtt
- SignatureLambdaFunction
- Arn
- /invocations
Type: REQUEST
IdentitySource: method.request.header.Auth
Name: !Sub ${
CustomerName}-${
ProjectName}-signature-${
EnvironmentName}-authorizer
RestApiId: !Ref APIGateway
SignatureLambdaFunction:
Type: AWS::Lambda::Function
Properties:
FunctionName: !Sub ${
CustomerName}-${
ProjectName}-signature-${
EnvironmentName}-lambda
Description: signature lambda
Code:
ZipFile: |
exports.handler = async (event) => {
const defaultAllowAllPolicy = {"principalId": "user","policyDocument": {"Version": "2012-10-17","Statement": [{"Action": "execute-api:Invoke","Effect": "Allow","Resource": "*"}]}};
return defaultAllowAllPolicy;
};
Handler: index.handler
Role: !Sub arn:aws-cn:iam::${
AWS::AccountId}:role/${
CustomerName}-${
ProjectName}-agw-sqs-${
EnvironmentName}-iamr
Runtime: nodejs14.x
Environment:
Variables:
# Signature secrets using the Secrets Manager API
SECRETS_MANAGER_NAME: !Sub ${
CustomerName}-${
ProjectName}-${
EnvironmentName}-secrets-manager
LOG_LEVEL: DEBUG
Tags:
- Key: ApplName
Value: your-app-name
#trigger lambda
LambdaFunction:
Type: AWS::Lambda::Function
Properties:
FunctionName: !Sub ${
CustomerName}-${
ProjectName}-${
SubProjectName}-${
EnvironmentName}-lambda
Description: sqs rais lambda
Code:
ZipFile: |
exports.handler = function(event, context) {
console.log('Empty lambda need to be replaced!');
};
Handler: index.handler
Layers:
- Ref: LambdaBaseLayer
Role: !Sub arn:aws-cn:iam::${
AWS::AccountId}:role/${
CustomerName}-${
ProjectName}-agw-sqs-${
EnvironmentName}-iamr
Runtime: nodejs14.x
Timeout: 900
# MemorySize: 512
VpcConfig:
SecurityGroupIds: !Ref LambdaSecurityGroups
SubnetIds: !Ref LambdaSubnetIds
Environment:
Variables:
# database secrets using the Secrets Manager API
SECRETS_MANAGER_NAME: !Join [ '-', [ !Ref CustomerName, !Ref ProjectName, !Ref EnvironmentName, 'secrets-manager' ] ]
LOG_LEVEL: DEBUG
Tags:
- Key: ApplName
Value: your-app-name
LambdaFunctionEventSourceMapping:
Type: AWS::Lambda::EventSourceMapping
Properties:
BatchSize: 10
Enabled: true
EventSourceArn: !GetAtt Queue.Arn
FunctionName: !GetAtt LambdaFunction.Arn
Question: Request Application/xml
1. "Integration Request"
1. Add HTTP headers
Content-Type ： 'application/x-www-form-urlencoded'
2. Add a mapping template
1) Add application/xml
Action=SendMessage&MessageBody={"body" : $input.json('$')}
2) Add application/json
Action=SendMessage&MessageBody={"body" : $input.json('$')}
2. "Integrated Response"
1.200 Add application/xml to the mapping template of the response
#set($inputParam=$input.path('$'))
$inputParam.body
## 这里可以加入你自己的XML格式
3. "Method Response"
1,200 Response body adding application/xml
保存即可
Issue: Use of authorized parties
1. Standard OAuth application
方法请求中选择授权方，新建Lambda，在Lambda中获取Head ->Authrization，编码验证令牌。
2. Parameter signature
参数列表+Head[签名]字段，客户端签名，服务器验证签名。 私钥
3. Security control
WAF控制进站，限制访问的源
4. Body signature
Signator does not support body parameter passing [I don’t know what the designer thinks]
If you want to verify the signature after parsing the body, the current feasible solution is:
ApiGateway: 开放接口集成
Signater: 微服务或身份验证（body除外）
Lambda：验证签名，放置队列SQS（不限制启动数量、性能要求高）
SQS：削峰
Lambda：订阅SQS（限制启动数量）
RDS或S3等持久化