Using AWS Secrets Manager with VPC endpoints and CloudFormation

AWS Secrets Manager helps you protect secrets needed to access your applications, services, and IT resources. The service enables you to easily rotate, manage, and retrieve database credentials, API keys, and other secrets throughout their lifecycle. Users and applications retrieve secrets with a call to Secrets Manager APIs, eliminating the need to hardcode sensitive information in plain text. Secrets Manager offers secret rotation with built-in integration for Amazon RDS, Amazon Redshift, and Amazon DocumentDB. Also, the service is extensible to other types of secrets, including API keys and OAuth tokens. In addition, Secrets Manager enables you to control access to secrets using fine-grained permissions and audit secret rotation centrally for resources in the AWS Cloud, third-party services, and on-premises. See AWS Secrets Manager for more information.

There are some configurations to access Secrets Manager from an AWS Lambda function when the Lambda function is in a VPC:

  1. Permission with Policies (AWSSecretsManagerGetSecretValuePolicy)

  2. Network configuration (VPC Endpoint)

  3. 443 Inbound rule for security group


SAM template yaml configurations

Creating a Secret

In this example, we are trying to create a database connection with credentials retrieved from Secrets Manager. It is assumed that Database is already created and is using the same secret to configure the database. For the best practice database is created with the CloudFormation and the secret is attached to the database with AWS::SecretsManager::SecretTargetAttachment. To simplify this example, those configurations are not displayed here.

MyDatabaseSecret:

Type: AWS::SecretsManager::Secret

Properties:

Name: !Sub '/${AWS::StackName}/database/root'

Description: RDS database admin credentials for Easy Integrations

GenerateSecretString:

SecretStringTemplate: '{"username": "root"}'

GenerateStringKey: "password"

ExcludeCharacters: '"@/\'


Security Group

Following security group is used for database and SecretManager access. If you do not need a database configuration, just skip the 3306.

  • 3306 port is for MySQL database

  • 443 is required to access SecretsManager VPCEndpoint

CidrIp block (CidrBlockPrivateSubnet1 and CidrBlockPrivateSubnet2) is referenced to database VPC subnet, needs to be replaced with the VPC's subnet IP block (example: 10.192.16.0/20 and 10.192.32.0/20).

MyDatabaseSecurityGroup:

Type: AWS::EC2::SecurityGroup

Properties:

GroupDescription: Database

VpcId: !Ref pubPrivateVPC

SecurityGroupIngress:

- { CidrIp: !Ref CidrBlockPrivateSubnet1, FromPort: 3306, ToPort: 3306, IpProtocol: tcp }

- { CidrIp: !Ref CidrBlockPrivateSubnet2, FromPort: 3306, ToPort: 3306, IpProtocol: tcp }

- { CidrIp: !Ref CidrBlockPrivateSubnet1, FromPort: 443, ToPort: 443, IpProtocol: tcp }

- { CidrIp: !Ref CidrBlockPrivateSubnet2, FromPort: 443, ToPort: 443, IpProtocol: tcp }


VPC Endpoint

Secrets manager endpoint is created to access from the VPC (defined as pubPrivateVPC reference). Make sure PrivateDnsEnabled is set to true.

SecretsManagerEndpoint:

Type: AWS::EC2::VPCEndpoint

Properties:

PolicyDocument:

Version: 2012-10-17

Statement:

- Effect: Allow

Principal: "*"

Action:

- "secretsmanager:*"

Resource:

- "*"

VpcEndpointType: Interface

PrivateDnsEnabled: true

ServiceName: !Sub com.amazonaws.${AWS::Region}.secretsmanager

VpcId: !Ref pubPrivateVPC

SecurityGroupIds: [ !GetAtt MyDatabaseSecurityGroup.GroupId ]

SubnetIds: [ !Ref privateSubnet1, !Ref privateSubnet2 ]


Lambda function

MyDatabaseSecret is used as reference for the AWSSecretsManagerGetSecretValuePolicy Policy.

RDS_SECRET_ARN is added to Environment variables which will be used in the code to fetch secret values.

MyCoolFunction:

Type: AWS::Serverless::Function

Properties:

FunctionName: my-cool-function

CodeUri: functions/my-cool-function

Handler: index.lambdaHandler

Runtime: nodejs14.x

Policies:

- AWSLambdaVPCAccessExecutionRole

- AWSLambdaRole

- AWSSecretsManagerGetSecretValuePolicy:

SecretArn: !Ref MyDatabaseSecret

VpcConfig:

SecurityGroupIds: [ !GetAtt MyDatabaseSecurityGroup.GroupId ]

SubnetIds: [ !Ref privateSubnet1, !Ref privateSubnet2 ]

Environment:

Variables:

RDS_SECRET_ARN: !Ref MyDatabaseSecret


Code (NodeJS Lambda function)

getDatabaseVariables used will give the secret value. RDS_SECRET_ARN is already added to environment variables and it holds the secret ARN.


const AWS = require("aws-sdk");

const secretsManagerClient = new AWS.SecretsManager({region: 'eu-west-1'});


const getSecretValue = async (secretArn) => {

console.log('Calling getSecretValue...');

console.log('secretArn', secretArn);

const data = await secretsManagerClient.getSecretValue({SecretId: secretArn}).promise();

console.log('Called secretsManagerClient.getSecretValue... data:', data);


let secret;

if ('SecretString' in data) {

secret = data.SecretString;

} else {

let buff = Buffer.from(data.SecretBinary, 'base64');

secret = buff.toString('ascii');

}

return secret ? JSON.parse(secret) : secret;

}


const getDatabaseVariables = async () => {

return await getSecretValue(process.env.RDS_SECRET_ARN);

}


Result:

{

"dbClusterIdentifier": "my-rds-cluster",

"password": "xxxxxxxxxxx",

"dbname": "my-db",

"engine": "mysql",

"port": 3306,

"host": "my-rds-cluster.cluster-xxxxxxxx.eu-west-1.rds.amazonaws.com",

"username": "root"

}


References: