AWS -

CloudFormation

CloudFormation is templating to create and manage AWS resource.

CloudFromation allows creating infrastructure with code. Using high-level programming languages to control IT systems.

Almost everything should be created using CloudFormation templates. Templates allow you to replicate complex sequence of AWS configuration. After startup, you can also easily change part of the system.

Running a template creates a stack. Stack is the running infrastructure. Template is the class, stack is the object instance, many stacks can rely on the same template.

aws cloudformation create-stack \
    --stack-name my-stack \
    --template-url https://s3.amazonaws.com/my-bucket/my-template.json

CloudFormation templates are JSON. The only currently valid AWSTemplateFormatVersion is 2010-09-09. Description is a good place to tell what the template is about.

{
    "AWSTemplateFormatVersion": "2010-09-09",
    "Description": "",
    "Parameters": {},
    "Conditions": {},
    "Mappings": {},
    "Resources": {},
    "Outputs": {}
}

Template JSON has special Fn and Ref markup. They are function definitions that are evaluated on read.

# Ref indicates intertemplate reference, the target resource must start first
{"Ref": "MyAwesomeHostVPC"}

# Fn::Join is used to create a single value from multiple.
{"Fn::Join", ["delimiter", ["row1\n", "row2\n", "row3\n"]]}

# Fn::Base64 is used to convert value to Base64
{"Fn:Base64": "value"}

# Fn::GetAtt allows reading values from other parts of the template
{"Fn::GetAtt": ["ApacheServer", "PrivateIp"]}

# Fn::Select selects array value by index
{"Fn::Select": ["0", {"Fn::GetAZs": ""}]}

# Fn::FindInMap allows reading mappings defined in templates
{"Fn::FindInMap": ["EC2RegionMap", {"Ref": "AWS::Region"}, "MyAMI1"]},

# Fn::Equals returns true if values are equal
{"Fn::Equals": [{"Ref": "AttachVolume"}, "yes"]}

Parameters defines what is customizable on the template. Fields that are used to customize the template at the start.

"Parameters": {
    "KeyName": {
        "Description": "Key pair name",
        "Type": "AWS::EC2::KeyPair::KeyName"
    },
    "VPC": {
        "Description": "Just select the one and only default VPC",
        "Type": "AWS::EC2::VPC::Id"
    },
    "Subnet": {
        "Description": "Just select one of the available subnets",
        "Type": "AWS::EC2::Subnet::Id"
    },
    "IpForSSH": {
        "Description": "Your public IP address to allow SSH access",
        "Type": "String"
    },
    "Lifetime": {
        "Description": "Lifetime in minutes (2-59)",
        "Type": "Number",
        "Default": "2",
        "MinValue": "2",
        "MaxValue": "59"
    },
    "AttachVolume": {
        "Description": "Should the volume be attached?",
        "Type": "String",
        "Default": "yes",
        "AllowedValues": ["yes", "no"]
    }
}

Conditions allows creating conditionals inside the templates.

"Parameters": {
    "AttachVolume": {
        "Description": "Should the volume be attached?",
        "Type": "String",
        "Default": "yes",
        "AllowedValues": ["yes", "no"]
    }
},
...
"Conditions": {
    "Attached": {"Fn::Equals": [{"Ref": "AttachVolume"}, "yes"]}
},
...
"VolumeAttachment": {
    "Type": "AWS::EC2::VolumeAttachment",
    "Condition": "Attached",
    "Properties": {
        "Device": "/dev/xvdf",
        "InstanceId": {"Ref": "Server"},
        "VolumeId": {"Ref": "Volume"}
    }
}

Mappings section defines a key-value mappings for Fn::FindInMap.

"Mappings": {
    "EC2RegionMap": {
        "ap-northeast-1": {"MyAMI1": "ami-cbf90ecb", "MyAMI2": "ami-03cf3903"},
        "ap-southeast-1": {"MyAMI1": "ami-68d8e93a", "MyAMI2": "ami-b49dace6"},
        "ap-southeast-2": {"MyAMI1": "ami-fd9cecc7", "MyAMI2": "ami-e7ee9edd"},
        "eu-central-1":   {"MyAMI1": "ami-a8221fb5", "MyAMI2": "ami-46073a5b"},
        "eu-west-1":      {"MyAMI1": "ami-a10897d6", "MyAMI2": "ami-6975eb1e"},
        "sa-east-1":      {"MyAMI1": "ami-b52890a8", "MyAMI2": "ami-fbfa41e6"},
        "us-east-1":      {"MyAMI1": "ami-1ecae776", "MyAMI2": "ami-303b1458"},
        "us-west-1":      {"MyAMI1": "ami-d114f295", "MyAMI2": "ami-7da94839"},
        "us-west-2":      {"MyAMI1": "ami-e7527ed7", "MyAMI2": "ami-69ae8259"}
    }
},
...
{"Fn::FindInMap": ["EC2RegionMap", {"Ref": "AWS::Region"}, "MyAMI1"]},

Resources defines all the started AWS resources. EC2 instances, security groups, VPCs etc.

Outputs defines what values a stack shows when running.

"Outputs": {
    "WebHostPublicName": {
        "Value": {"Fn::GetAtt": ["WebHost", "PublicDnsName"]},
        "Description": "connect via SSH as user ec2-user"
    }
}

EC2 instance UserData can be defined to specify startup scripts.

# User data mus be encoded in Base64.
"EC2Instance": {
    "Type": "AWS::EC2::Instance",
    "Properties": {
        "InstanceType": "t2.micro",
        "SecurityGroupIds": [{"Ref": "InstanceSecurityGroup"}],
        "KeyName": {"Ref": "KeyName"},
        "ImageId": {"Fn::FindInMap": [
            "EC2RegionMap",
            {"Ref": "AWS::Region"},
            "AmazonLinuxAMIHVMEBSBacked64bit"
        ]},
        "SubnetId": {"Ref": "Subnet"},
        "UserData": {"Fn::Base64": {"Fn::Join": ["", [
            "#!/bin/bash -ex\n",
            "export IPSEC_PSK=", {"Ref": "IPSecSharedSecret"}, "\n",
            "export VPN_USER=", {"Ref": "VPNUser"}, "\n",
            "export VPN_PASSWORD=", {"Ref": "VPNPassword"}, "\n",
            "export STACK_NAME=", {"Ref": "AWS::StackName"}, "\n",
            "export REGION=", {"Ref": "AWS::Region"}, "\n",
            "curl -s https://s3-us-west-2.amazonaws.com/arcana-vpn/vpn-setup.sh | bash -ex\n"
        ]]}}
    }
},

Random examples:

// in the template parameters...
"Lifetime": {
    "Description": "Lifetime in minutes (2-59)",
    "Type": "Number",
    "Default": "2",
    "MinValue": "2",
    "MaxValue": "59"
}

// in the EC2 instance...
"UserData": {"Fn::Base64": {"Fn::Join": ["", [
    "#!/bin/bash -ex\n",
    "INSTANCEID=`curl -s http://169.254.169.254/latest/meta-data/instance-id`\n",
    "echo \"aws --region ", {"Ref": "AWS::Region"}, " ec2 stop-instances --instance-ids $INSTANCEID\" | at now + ", {"Ref": "Lifetime"} ," minutes\n"
]]}}

"ServerIP": {
    "Description": "VPN connection endpoint IP",
    "Value": {"Fn::GetAtt": ["EC2Instance", "PublicIp"]}
},