1

Hello everyone.
I have a terraform project that uses the data resource to look up an AMI that is present in my AWS account.
I want to add LocalStack to the terraform project for local development and testing. Since LS emulates AWS services, I am not able to share/copy the AMI with LS. Moreover, I am unable to share the EBS block storage snapshot with LS.
Do you perhaps have any recommendations, suggestions to resolve this ?

2

Hi @jnax09,

Please have a look at our documentation Elastic Compute Cloud (EC2) | Docs (localstack.cloud),

Base Images

LocalStack uses a naming scheme to recognise and manage the containers and images associated with it. Containers are named localstack-ec2.<InstanceId>, while images are tagged localstack-ec2/<AmiName>:<AmiId>.

The Docker backend treats Docker images with the above naming scheme as AMIs. For example, the following command can be used to associate the Ubuntu Focal image as ami-000001.

$ docker tag ubuntu:focal localstack-ec2/ubuntu-focal-ami:ami-000001

These Docker-backed AMIs have the resource tag ec2_vm_manager:docker and can be listed with the following command:

$ awslocal ec2 describe-images --filters Name=tag:ec2_vm_manager,Values=docker

All other AMIs are ‘mocked’ and are based off the community edition of LocalStack. Attempting to launch Dockerised instances with these AMIs will return InvalidAMIID.NotFound error.

3

Thank for your reply @Marcel

This indeed works well with “public” AMIs/images. In my case, however, the AMI is a custom one and it is privately owned by my AWS account.

4

Could you please explain the steps you are taking?

5

In my terraform project, I have a data resource that looks up an AMI.

data "aws_ami" "centos5_ami" {
  most_recent = true
  owners      = ["*********"]

  filter {
    name   = "image-id"
    values = ["ami-***********"]
  }
}

As you can see the AMI is privately owned by my AWS.

I start Localstack using the following docker compose file:

version: "3.8"

services:
  localstack:
    container_name: localstack
    image: localstack/localstack
    ports:
      - "127.0.0.1:4566:4566"            # LocalStack Gateway
      - "127.0.0.1:4510-4559:4510-4559"  # external services port range
    environment:
      - MAIN_DOCKER_NETWORK=localstack_default
      - DEBUG=${DEBUG-}
      - DOCKER_HOST=unix:///var/run/docker.sock
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock"
    networks:
      - localstack_default
networks:
  localstack_default:
    name: localstack_default

When I proceed to run terraform plan, it is successful. However, upon running terraform apply, I get an error message about the AMI not found, which makes sense.

I don’t know whether it is possible but I have been trying to find a way to copy/share the AMI with Localstack. However, since Localstack enables us to emulate AWS services locally, I have been having a hard time figuring it out.

I hope that makes sense.

6

You will have to download the image on your local machine and tag it like I mentioned it in my previous comment.

7

Okay I will find a way to download the AMI locally and tag it accordingly. I will let you know how that goes.

8

Hey @Marcel. I have given this a try on a public AMI.

Here is what I have done:

  • I pulled the amazonlinux image: docker pull amazonlinux
  • then I tagged it accordingly: docker tag amazonlinux localstack-ec2/amazonlinux-ami:ami-000001

In my terraform project, I made the following change.

FROM:

data "aws_ami" "amazonlinux" {

  most_recent = true
  owners      = ["137112412989"]

  filter {
    name   = "name"
    values = ["amzn2-ami-hvm-2.0*-x86_64-gp2"]
  }
}

resource "aws_instance" "bastion" {
  ami                         = "data.aws_ami.amazonlinux.id"
  source_dest_check           = false
  instance_type               = var.instance_type
  subnet_id                   = var.subnet_id
  key_name                    = var.key_name
  vpc_security_group_ids      = compact(split(",", var.security_groups))
  monitoring                  = true
  associate_public_ip_address = true
  user_data                   = file(format("%s/user_data.sh", path.module))

  root_block_device {
    encrypted = true
  }

TO:

resource "aws_instance" "bastion" {
  ami                         = "ami-000001"
  source_dest_check           = false
  instance_type               = var.instance_type
  subnet_id                   = var.subnet_id
  key_name                    = var.key_name
  vpc_security_group_ids      = compact(split(",", var.security_groups))
  monitoring                  = true
  associate_public_ip_address = true
  user_data                   = file(format("%s/user_data.sh", path.module))

  root_block_device {
    encrypted = true
  }

Upon running terraform apply, I get the following error which I haven’t figured not yet.

Error: creating EC2 Instance: MissingParameter: The request must contain the parameter size or snapshotId
        status code: 400, request id: 37720c40-e508-49eb-9540-b17bec8ce11e

  on modules/vpc/stack-bastion/main.tf line 67, in resource "aws_instance" "bastion":
  67: resource "aws_instance" "bastion" {

Upon describing the ami-000001, this is what is returned:

{
  "Images": [
    {
      "Architecture": "arm64",
      "CreationDate": "2023-08-07T19:40:41.794331038Z",
      "ImageId": "ami-000001",
      "ImageLocation": "docker",
      "ImageType": "machine",
      "Public": true,
      "KernelId": "None",
      "OwnerId": "000000000000",
      "RamdiskId": "ari-1a2b3c4d",
      "State": "available",
      "BlockDeviceMappings": [
        {
          "DeviceName": "/dev/sda1",
          "Ebs": {
            "DeleteOnTermination": false,
            "SnapshotId": "snap-6d53f169",
            "VolumeSize": 15,
            "VolumeType": "standard"
          }
        }
      ],
      "Description": "None",
      "Hypervisor": "xen",
      "ImageOwnerAlias": "amazon",
      "Name": "amazonlinux-ami",
      "RootDeviceName": "/dev/sda1",
      "RootDeviceType": "standard",
      "Tags": [
        {
          "Key": "ec2_vm_manager",
          "Value": "docker"
        }
      ],
      "VirtualizationType": "None"
    }
  ]
}

What do you think I am missing or getting wrong ?

9

Hi @jnax09,

I have tried it and had to remove monitoring as it is not implemented. Otherwise, it got deployed.

As you mentioned, steps before:

$ docker pull amazonlinux
$ docker tag amazonlinux localstack-ec2/amazonlinux-ami:ami-000002
$ awslocal ec2 describe-images --filters Name=tag:ec2_vm_manager,Values=docker | grep ImageId

resource "aws_vpc" "vpc" {
  cidr_block = "10.0.0.0/16"
  }

resource "aws_subnet" "subnet" {
  vpc_id     = aws_vpc.vpc.id
  cidr_block = "10.0.0.0/24"
  }

resource "aws_security_group" "security_group" {
  vpc_id = aws_vpc.vpc.id
  }

resource "aws_key_pair" "deployer" {
  key_name   = "deployer-key"
  public_key = "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD3F6tyPEFEzV0LX3X8BsXdMsQz1x2cEikKDEY0aIj41qgxMCP/iteneqXSIFZBp5vizPvaoIR3Um9xK7PGoW8giupGn+EPuxIA4cDM4vzOqOkiMPhz5XK0whEjkVzTo4+S0puvDZuwIsdiW9mxhJc7tgBNL0cYlWSYVkz4G/fslNfRPW5mYAM49f4fhtxPb5ok4Q2Lg9dPKVHO/Bgeu5woMc7RY0p1ej6D4CKFE6lymSDJpW0YHX/wqE9+cfEauh7xZcG0q9t2ta6F6fmX0agvpFyZo8aFbXeUBr7osSCJNgvavWbM/06niWrOvYX2xwWdhXmXSrbX8ZbabVohBK41 email@example.com"
}

resource "aws_instance" "bastion" {
  ami                         = "ami-000002"
  source_dest_check           = false
  instance_type               = "t2.micro"
  subnet_id                   = aws_subnet.subnet.id
  key_name                    = aws_key_pair.deployer.id
  vpc_security_group_ids      = [aws_security_group.security_group.id]
  associate_public_ip_address = true
  user_data                   = file(format("%s/user_data.sh", path.module))
  
 root_block_device {
    encrypted = true
    volume_size = 8
  }
}
10

Thanks @Marcel. Besides removing monitoring, I see that you’ve added volume_size

11

Yes, I forgot to mention that it was necessary to add that.
Info in the documentation: AWS::EC2::Instance Ebs - AWS CloudFormation (amazon.com)

12

Indeed. Thanks @Marcel.

I have another blocker but unrelated to this. I will create another discussion for it.

1 Like