InternalError while calling route53 ChangeResourceRecordSets

Hi, In my terraform project, I have a module called acm that basically issues an AWS cert and create the necessary resources around it.

variable "fqdn" {}
variable "zone_id" {}
variable "service_name" {}
variable "commons" {
  type = map(string)
}

locals {
  common_tags = {
    X-Service = var.service_name
  }
}

resource "aws_acm_certificate" "this" {
  domain_name       = var.fqdn
  validation_method = "DNS"
  tags              = local.common_tags

  lifecycle {
    create_before_destroy = true
  }
}

resource "aws_route53_record" "this" {
  for_each = {
    for dvo in aws_acm_certificate.this.domain_validation_options : dvo.domain_name => {
      name   = dvo.resource_record_name
      record = dvo.resource_record_value
      type   = dvo.resource_record_type
    }
  }

  zone_id         = var.zone_id
  name            = each.value.name
  type            = each.value.type
  records         = [each.value.record]
  ttl             = 60
  allow_overwrite = true

  lifecycle {
    create_before_destroy = true
  }
}

resource "aws_acm_certificate_validation" "this" {
  certificate_arn         = aws_acm_certificate.this.arn
  validation_record_fqdns = [for record in aws_route53_record.this : record.fqdn]

  lifecycle {
    create_before_destroy = true
  }
}

The following declaration makes use of the acm module declared above:

module "domain_cert" {
  source       = "../../../modules/acm"
  fqdn         = "*.${module.deploy_env.pub_domain_v1}"
  zone_id      = local.route53_primary_zone_id
  service_name = var.service_name
  commons      = var.commons
}

The error I get, upon terraform apply:

module.domain_cert.aws_route53_record.this["*.localstack.example.com"]: Still creating... [53m31s elapsed]
module.domain_cert.aws_route53_record.this["*.localstack.example.com"]: Still creating... [53m41s elapsed]
module.domain_cert.aws_route53_record.this["*.localstack.example.com"]: Still creating... [53m51s elapsed]

Error: [ERR]: Error building changeset: InternalError: exception while calling route53.ChangeResourceRecordSets: ('Could not find entry %s for name %s in zones', TargetRecord(target='_c9edd76ee4a0e2a74388032f3861cc50.ykybfrwcxw.acm-validations.aws.', record_type=<RecordType.CNAME: 3>, record_id=None), '_d930b28be6c5927595552b219965053e.localstack.example.com.')
        status code: 500, request id: a63022f4-51d2-4bd1-9718-981472c2b5b6

I ran aws route53 list-resource-record-sets --hosted-zone-id "LLCX4US2DZK0YIT" | jq to get the list of record sets for the domain and got this:

{
  "ResourceRecordSets": [
    {
      "Name": "localstack.example.com.",
      "Type": "NS",
      "TTL": 172800,
      "ResourceRecords": [
        {
          "Value": "ns-2048.awsdns-64.com"
        },
        {
          "Value": "ns-2049.awsdns-65.net"
        },
        {
          "Value": "ns-2050.awsdns-66.org"
        },
        {
          "Value": "ns-2051.awsdns-67.co.uk"
        }
      ]
    },
    {
      "Name": "localstack.example.com.",
      "Type": "SOA",
      "TTL": 900,
      "ResourceRecords": [
        {
          "Value": "{'Value': 'ns-2048.awsdns-64.com. hostmaster.example.com. 1 7200 900 1209600 86400'}"
        }
      ]
    },
    {
      "Name": "_amazonses.localstack.example.com.",
      "Type": "TXT",
      "TTL": 600,
      "ResourceRecords": [
        {
          "Value": "\"a682b0aa-9f83-4612-818d-d657de5b23c3\""
        }
      ]
    },
    {
      "Name": "_d930b28be6c5927595552b219965053e.localstack.example.com.",
      "Type": "CNAME",
      "TTL": 60,
      "ResourceRecords": [
        {
          "Value": "_c9edd76ee4a0e2a74388032f3861cc50.ykybfrwcxw.acm-validations.aws."
        }
      ]
    },
    {
      "Name": "alb-api.localstack.example.com.",
      "Type": "A",
      "AliasTarget": {
        "HostedZoneId": "Z2P70J7EXAMPLE",
        "DNSName": "api-alb1.elb.localhost.localstack.cloud",
        "EvaluateTargetHealth": true
      }
    },
    {
      "Name": "alb-www1.localstack.example.com.",
      "Type": "A",
      "AliasTarget": {
        "HostedZoneId": "Z2P70J7EXAMPLE",
        "DNSName": "api-alb1.elb.localhost.localstack.cloud",
        "EvaluateTargetHealth": true
      }
    },
    {
      "Name": "alb-www2.localstack.example.com.",
      "Type": "A",
      "AliasTarget": {
        "HostedZoneId": "Z2P70J7EXAMPLE",
        "DNSName": "api-alb1.elb.localhost.localstack.cloud",
        "EvaluateTargetHealth": true
      }
    },
    {
      "Name": "api-alb1.localstack.example.com.",
      "Type": "A",
      "AliasTarget": {
        "HostedZoneId": "Z2P70J7EXAMPLE",
        "DNSName": "api-alb1.elb.localhost.localstack.cloud",
        "EvaluateTargetHealth": true
      }
    },
    {
      "Name": "es1.localstack.example.com.",
      "Type": "A",
      "AliasTarget": {
        "HostedZoneId": "Z2P70J7EXAMPLE",
        "DNSName": "alb-es1.elb.localhost.localstack.cloud",
        "EvaluateTargetHealth": false
      }
    }
  ]
}

Then, I ran aws acm list-certificates --query "CertificateSummaryList[?DomainName=='*.localstack.example.com'].{DomainName: DomainName, Status: Status}" --output json | jq to get the list of certs and got this:

[
  {
    "DomainName": "*.localstack.example.com",
    "Status": "ISSUED"
  }
]

I have troubleshooting this for a few days now and I can clueless at this point.

Thanks in advance for your assistance. Please let me know if you require any further details.

I have also realized that the nameserver that was returned when I initially created the hosted zone is different from the ones I get above when I queried the hosted zone.

The one that was initially returned is:

dns.localhost.localstack.cloud

Also, the error I get above, is the same for all records in the hosted zone.

Hi @jnax09 !
Thank you for reporting this issue.

There have been recent changes to route53, which seem to be related to your issue.

Could you attach the whole logs of all route53.ChangeResourceRecordSets in order?

If it is not too much work, a self-contained terraform snippet would be of great help as well, as there seem to be some variable definitions missing in your snippets. Of course, random sample data would be fine, as long as it triggers this issue.

Hi @dfangl. Thanks for your reply.

I have simplified the project. Here is how it looks now:

The acm module:

variable "fqdn" {}
variable "zone_id" {}

variable "service_name" {}

variable "commons" {
  type = map(string)
}

locals {
  common_tags = {
    X-Infra   = 1
  }
}


resource "aws_acm_certificate" "this" {
  domain_name       = var.fqdn
  validation_method = "DNS"
  tags              = local.common_tags

  lifecycle {
    create_before_destroy = true
  }
}

resource "aws_route53_record" "this" {
  for_each = {
    for dvo in aws_acm_certificate.this.domain_validation_options : dvo.domain_name => {
      name   = dvo.resource_record_name
      record = dvo.resource_record_value
      type   = dvo.resource_record_type
    }
  }

  zone_id         = var.zone_id
  name            = each.value.name
  type            = each.value.type
  records         = [each.value.record]
  ttl             = 60
  allow_overwrite = true

  lifecycle {
    create_before_destroy = true
  }
}

resource "aws_acm_certificate_validation" "this" {
  certificate_arn         = aws_acm_certificate.this.arn
  validation_record_fqdns = [for record in aws_route53_record.this : record.fqdn]

  lifecycle {
    create_before_destroy = true
  }
}

output "arn" {
  value = aws_acm_certificate_validation.this.certificate_arn
}

output "domain_name" {
  value = aws_acm_certificate.this.domain_name
}

The module usage:

module "domain_cert" {
  source       = "../../../modules/acm"
  fqdn         = "*.localstack.example.com"
  zone_id      = "*****6****" # I removed this.
  service_name = "es1"
  commons      =  { 
     env_tier = "test"
   }

Here are the logs you requested:

2023-08-28 20:26:07 2023-08-28T18:26:07.140  INFO --- [   asgi_gw_8] localstack.request.aws     : AWS route53.GetHostedZone => 200
2023-08-28 20:26:07 2023-08-28T18:26:07.162  INFO --- [   asgi_gw_0] localstack.request.aws     : AWS route53.ListResourceRecordSets => 200
2023-08-28 20:26:07 2023-08-28T18:26:07.178  INFO --- [   asgi_gw_7] localstack.request.aws     : AWS route53.ListTagsForResource => 200
2023-08-28 20:26:07 2023-08-28T18:26:07.614  INFO --- [   asgi_gw_4] localstack.request.aws     : AWS route53.GetHostedZone => 200
2023-08-28 20:26:07 2023-08-28T18:26:07.711  INFO --- [   asgi_gw_5] localstack.request.aws     : AWS route53.ChangeResourceRecordSets => 200
2023-08-28 20:26:07 2023-08-28T18:26:07.997  INFO --- [   asgi_gw_8] localstack.request.aws     : AWS acm.RequestCertificate => 200
2023-08-28 20:26:08 2023-08-28T18:26:08.018  INFO --- [   asgi_gw_9] localstack.request.aws     : AWS acm.DescribeCertificate => 200
2023-08-28 20:26:08 2023-08-28T18:26:08.034  INFO --- [   asgi_gw_6] localstack.request.aws     : AWS acm.DescribeCertificate => 200
2023-08-28 20:26:08 2023-08-28T18:26:08.056  INFO --- [   asgi_gw_1] localstack.request.aws     : AWS acm.ListTagsForCertificate => 200
2023-08-28 20:26:08 2023-08-28T18:26:08.141  INFO --- [   asgi_gw_3] localstack.request.aws     : AWS route53.GetHostedZone => 200
2023-08-28 20:26:08 2023-08-28T18:26:08.155  WARN --- [   asgi_gw_2] l.s.route53.provider       : UPSERT
2023-08-28 20:26:08 2023-08-28T18:26:08.156 ERROR --- [   asgi_gw_2] l.aws.handlers.logging     : exception during call chain: ('Could not find entry %s for name %s in zones', TargetRecord(target='_c9edd76ee4a0e2a74388032f3861cc50.ykybfrwcxw.acm-validations.aws.', record_type=<RecordType.CNAME: 3>, record_id=None), '_d930b28be6c5927595552b219965053e.localstack.example.com.')
2023-08-28 20:26:08 2023-08-28T18:26:08.158  INFO --- [   asgi_gw_2] localstack.request.aws     : AWS route53.ChangeResourceRecordSets => 500 (InternalError)
2023-08-28 20:26:08 2023-08-28T18:26:08.216  WARN --- [   asgi_gw_6] l.s.route53.provider       : UPSERT
2023-08-28 20:26:08 2023-08-28T18:26:08.216 ERROR --- [   asgi_gw_6] l.aws.handlers.logging     : exception during call chain: ('Could not find entry %s for name %s in zones', TargetRecord(target='_c9edd76ee4a0e2a74388032f3861cc50.ykybfrwcxw.acm-validations.aws.', record_type=<RecordType.CNAME: 3>, record_id=None), '_d930b28be6c5927595552b219965053e.localstack.example.com.')
2023-08-28 20:26:08 2023-08-28T18:26:08.216  INFO --- [   asgi_gw_6] localstack.request.aws     : AWS route53.ChangeResourceRecordSets => 500 (InternalError)
2023-08-28 20:26:08 2023-08-28T18:26:08.354  WARN --- [   asgi_gw_6] l.s.route53.provider       : UPSERT
2023-08-28 20:26:08 2023-08-28T18:26:08.354 ERROR --- [   asgi_gw_6] l.aws.handlers.logging     : exception during call chain: ('Could not find entry %s for name %s in zones', TargetRecord(target='_c9edd76ee4a0e2a74388032f3861cc50.ykybfrwcxw.acm-validations.aws.', record_type=<RecordType.CNAME: 3>, record_id=None), '_d930b28be6c5927595552b219965053e.localstack.example.com.')
2023-08-28 20:26:08 2023-08-28T18:26:08.355  INFO --- [   asgi_gw_6] localstack.request.aws     : AWS route53.ChangeResourceRecordSets => 500 (InternalError)
2023-08-28 20:26:08 2023-08-28T18:26:08.548  WARN --- [   asgi_gw_2] l.s.route53.provider       : UPSERT
2023-08-28 20:26:08 2023-08-28T18:26:08.548 ERROR --- [   asgi_gw_2] l.aws.handlers.logging     : exception during call chain: ('Could not find entry %s for name %s in zones', TargetRecord(target='_c9edd76ee4a0e2a74388032f3861cc50.ykybfrwcxw.acm-validations.aws.', record_type=<RecordType.CNAME: 3>, record_id=None), '_d930b28be6c5927595552b219965053e.localstack.example.com.')
2023-08-28 20:26:08 2023-08-28T18:26:08.549  INFO --- [   asgi_gw_2] localstack.request.aws     : AWS route53.ChangeResourceRecordSets => 500 (InternalError)
2023-08-28 20:26:08 2023-08-28T18:26:08.943  WARN --- [   asgi_gw_7] l.s.route53.provider       : UPSERT
2023-08-28 20:26:08 2023-08-28T18:26:08.943 ERROR --- [   asgi_gw_7] l.aws.handlers.logging     : exception during call chain: ('Could not find entry %s for name %s in zones', TargetRecord(target='_c9edd76ee4a0e2a74388032f3861cc50.ykybfrwcxw.acm-validations.aws.', record_type=<RecordType.CNAME: 3>, record_id=None), '_d930b28be6c5927595552b219965053e.localstack.example.com.')
2023-08-28 20:26:08 2023-08-28T18:26:08.944  INFO --- [   asgi_gw_7] localstack.request.aws     : AWS route53.ChangeResourceRecordSets => 500 (InternalError)
2023-08-28 20:26:09 2023-08-28T18:26:09.916  WARN --- [   asgi_gw_8] l.s.route53.provider       : UPSERT
2023-08-28 20:26:09 2023-08-28T18:26:09.916 ERROR --- [   asgi_gw_8] l.aws.handlers.logging     : exception during call chain: ('Could not find entry %s for name %s in zones', TargetRecord(target='_c9edd76ee4a0e2a74388032f3861cc50.ykybfrwcxw.acm-validations.aws.', record_type=<RecordType.CNAME: 3>, record_id=None), '_d930b28be6c5927595552b219965053e.localstack.example.com.')
2023-08-28 20:26:09 2023-08-28T18:26:09.917  INFO --- [   asgi_gw_8] localstack.request.aws     : AWS route53.ChangeResourceRecordSets => 500 (InternalError)
2023-08-28 20:26:10 2023-08-28T18:26:10.911  WARN --- [  asgi_gw_10] l.s.route53.provider       : UPSERT
2023-08-28 20:26:10 2023-08-28T18:26:10.912 ERROR --- [  asgi_gw_10] l.aws.handlers.logging     : exception during call chain: ('Could not find entry %s for name %s in zones', TargetRecord(target='_c9edd76ee4a0e2a74388032f3861cc50.ykybfrwcxw.acm-validations.aws.', record_type=<RecordType.CNAME: 3>, record_id=None), '_d930b28be6c5927595552b219965053e.localstack.example.com.')
2023-08-28 20:26:10 2023-08-28T18:26:10.913  INFO --- [  asgi_gw_10] localstack.request.aws     : AWS route53.ChangeResourceRecordSets => 500 (InternalError)
2023-08-28 20:26:13 2023-08-28T18:26:13.374  WARN --- [   asgi_gw_2] l.s.route53.provider       : UPSERT
2023-08-28 20:26:13 2023-08-28T18:26:13.374 ERROR --- [   asgi_gw_2] l.aws.handlers.logging     : exception during call chain: ('Could not find entry %s for name %s in zones', TargetRecord(target='_c9edd76ee4a0e2a74388032f3861cc50.ykybfrwcxw.acm-validations.aws.', record_type=<RecordType.CNAME: 3>, record_id=None), '_d930b28be6c5927595552b219965053e.localstack.example.com.')
2023-08-28 20:26:13 2023-08-28T18:26:13.374  INFO --- [   asgi_gw_2] localstack.request.aws     : AWS route53.ChangeResourceRecordSets => 500 (InternalError)
2023-08-28 20:26:18 2023-08-28T18:26:18.316  INFO --- [   asgi_gw_0] localstack.request.aws     : AWS route53.GetHostedZone => 200
2023-08-28 20:26:18 2023-08-28T18:26:18.332  INFO --- [  asgi_gw_10] localstack.request.aws     : AWS route53.ChangeResourceRecordSets => 200
2023-08-28 20:26:19 2023-08-28T18:26:19.321  WARN --- [   asgi_gw_2] l.s.route53.provider       : UPSERT
2023-08-28 20:26:19 2023-08-28T18:26:19.322 ERROR --- [   asgi_gw_2] l.aws.handlers.logging     : exception during call chain: ('Could not find entry %s for name %s in zones', TargetRecord(target='_c9edd76ee4a0e2a74388032f3861cc50.ykybfrwcxw.acm-validations.aws.', record_type=<RecordType.CNAME: 3>, record_id=None), '_d930b28be6c5927595552b219965053e.localstack.example.com.')
2023-08-28 20:26:19 2023-08-28T18:26:19.322  INFO --- [   asgi_gw_2] localstack.request.aws     : AWS route53.ChangeResourceRecordSets => 500 (InternalError)
2023-08-28 20:26:31 2023-08-28T18:26:31.018  WARN --- [   asgi_gw_8] l.s.route53.provider       : UPSERT
2023-08-28 20:26:31 2023-08-28T18:26:31.018 ERROR --- [   asgi_gw_8] l.aws.handlers.logging     : exception during call chain: ('Could not find entry %s for name %s in zones', TargetRecord(target='_c9edd76ee4a0e2a74388032f3861cc50.ykybfrwcxw.acm-validations.aws.', record_type=<RecordType.CNAME: 3>, record_id=None), '_d930b28be6c5927595552b219965053e.localstack.example.com.')
2023-08-28 20:26:31 2023-08-28T18:26:31.020  INFO --- [   asgi_gw_8] localstack.request.aws     : AWS route53.ChangeResourceRecordSets => 500 (InternalError)
2023-08-28 20:26:34 2023-08-28T18:26:34.751  INFO --- [   asgi_gw_5] localstack.request.aws     : AWS route53.GetChange => 200
2023-08-28 20:26:34 2023-08-28T18:26:34.787  INFO --- [   asgi_gw_0] localstack.request.aws     : AWS route53.GetHostedZone => 200
2023-08-28 20:26:34 2023-08-28T18:26:34.817  INFO --- [  asgi_gw_10] localstack.request.aws     : AWS route53.ListResourceRecordSets => 200
2023-08-28 20:26:45 2023-08-28T18:26:45.364  INFO --- [   asgi_gw_3] localstack.request.aws     : AWS route53.GetChange => 200
2023-08-28 20:26:45 2023-08-28T18:26:45.393  INFO --- [   asgi_gw_8] localstack.request.aws     : AWS route53.GetHostedZone => 200
2023-08-28 20:26:45 2023-08-28T18:26:45.414  INFO --- [   asgi_gw_5] localstack.request.aws     : AWS route53.ListResourceRecordSets => 200
2023-08-28 20:26:55 2023-08-28T18:26:55.890  WARN --- [  asgi_gw_10] l.s.route53.provider       : UPSERT
2023-08-28 20:26:55 2023-08-28T18:26:55.891 ERROR --- [  asgi_gw_10] l.aws.handlers.logging     : exception during call chain: ('Could not find entry %s for name %s in zones', TargetRecord(target='_c9edd76ee4a0e2a74388032f3861cc50.ykybfrwcxw.acm-validations.aws.', record_type=<RecordType.CNAME: 3>, record_id=None), '_d930b28be6c5927595552b219965053e.localstack.example.com.')
2023-08-28 20:26:55 2023-08-28T18:26:55.892  INFO --- [  asgi_gw_10] localstack.request.aws     : AWS route53.ChangeResourceRecordSets => 500 (InternalError)
2023-08-28 20:27:38 2023-08-28T18:27:38.200  WARN --- [   asgi_gw_0] l.s.route53.provider       : UPSERT
2023-08-28 20:27:38 2023-08-28T18:27:38.202 ERROR --- [   asgi_gw_0] l.aws.handlers.logging     : exception during call chain: ('Could not find entry %s for name %s in zones', TargetRecord(target='_c9edd76ee4a0e2a74388032f3861cc50.ykybfrwcxw.acm-validations.aws.', record_type=<RecordType.CNAME: 3>, record_id=None), '_d930b28be6c5927595552b219965053e.localstack.example.com.')
2023-08-28 20:27:38 2023-08-28T18:27:38.203  INFO --- [   asgi_gw_0] localstack.request.aws     : AWS route53.ChangeResourceRecordSets => 500 (InternalError)
2023-08-28 20:27:38 2023-08-28T18:27:38.272  WARN --- [  asgi_gw_10] l.s.route53.provider       : UPSERT
2023-08-28 20:27:38 2023-08-28T18:27:38.273 ERROR --- [  asgi_gw_10] l.aws.handlers.logging     : exception during call chain: ('Could not find entry %s for name %s in zones', TargetRecord(target='_c9edd76ee4a0e2a74388032f3861cc50.ykybfrwcxw.acm-validations.aws.', record_type=<RecordType.CNAME: 3>, record_id=None), '_d930b28be6c5927595552b219965053e.localstack.example.com.')
2023-08-28 20:27:38 2023-08-28T18:27:38.273  INFO --- [  asgi_gw_10] localstack.request.aws     : AWS route53.ChangeResourceRecordSets => 500 (InternalError)
2023-08-28 20:27:38 2023-08-28T18:27:38.392  WARN --- [   asgi_gw_2] l.s.route53.provider       : UPSERT
2023-08-28 20:27:38 2023-08-28T18:27:38.392 ERROR --- [   asgi_gw_2] l.aws.handlers.logging     : exception during call chain: ('Could not find entry %s for name %s in zones', TargetRecord(target='_c9edd76ee4a0e2a74388032f3861cc50.ykybfrwcxw.acm-validations.aws.', record_type=<RecordType.CNAME: 3>, record_id=None), '_d930b28be6c5927595552b219965053e.localstack.example.com.')
2023-08-28 20:27:38 2023-08-28T18:27:38.393  INFO --- [   asgi_gw_2] localstack.request.aws     : AWS route53.ChangeResourceRecordSets => 500 (InternalError)
2023-08-28 20:27:38 2023-08-28T18:27:38.572  WARN --- [   asgi_gw_3] l.s.route53.provider       : UPSERT
2023-08-28 20:27:38 2023-08-28T18:27:38.573 ERROR --- [   asgi_gw_3] l.aws.handlers.logging     : exception during call chain: ('Could not find entry %s for name %s in zones', TargetRecord(target='_c9edd76ee4a0e2a74388032f3861cc50.ykybfrwcxw.acm-validations.aws.', record_type=<RecordType.CNAME: 3>, record_id=None), '_d930b28be6c5927595552b219965053e.localstack.example.com.')
2023-08-28 20:27:38 2023-08-28T18:27:38.573  INFO --- [   asgi_gw_3] localstack.request.aws     : AWS route53.ChangeResourceRecordSets => 500 (InternalError)
2023-08-28 20:27:39 2023-08-28T18:27:39.073  WARN --- [   asgi_gw_8] l.s.route53.provider       : UPSERT
2023-08-28 20:27:39 2023-08-28T18:27:39.073 ERROR --- [   asgi_gw_8] l.aws.handlers.logging     : exception during call chain: ('Could not find entry %s for name %s in zones', TargetRecord(target='_c9edd76ee4a0e2a74388032f3861cc50.ykybfrwcxw.acm-validations.aws.', record_type=<RecordType.CNAME: 3>, record_id=None), '_d930b28be6c5927595552b219965053e.localstack.example.com.')
2023-08-28 20:27:39 2023-08-28T18:27:39.075  INFO --- [   asgi_gw_8] localstack.request.aws     : AWS route53.ChangeResourceRecordSets => 500 (InternalError)
2023-08-28 20:27:39 2023-08-28T18:27:39.844  WARN --- [   asgi_gw_0] l.s.route53.provider       : UPSERT
2023-08-28 20:27:39 2023-08-28T18:27:39.844 ERROR --- [   asgi_gw_0] l.aws.handlers.logging     : exception during call chain: ('Could not find entry %s for name %s in zones', TargetRecord(target='_c9edd76ee4a0e2a74388032f3861cc50.ykybfrwcxw.acm-validations.aws.', record_type=<RecordType.CNAME: 3>, record_id=None), '_d930b28be6c5927595552b219965053e.localstack.example.com.')
2023-08-28 20:27:39 2023-08-28T18:27:39.845  INFO --- [   asgi_gw_0] localstack.request.aws     : AWS route53.ChangeResourceRecordSets => 500 (InternalError)
2023-08-28 20:27:41 2023-08-28T18:27:41.497  WARN --- [  asgi_gw_10] l.s.route53.provider       : UPSERT
2023-08-28 20:27:41 2023-08-28T18:27:41.497 ERROR --- [  asgi_gw_10] l.aws.handlers.logging     : exception during call chain: ('Could not find entry %s for name %s in zones', TargetRecord(target='_c9edd76ee4a0e2a74388032f3861cc50.ykybfrwcxw.acm-validations.aws.', record_type=<RecordType.CNAME: 3>, record_id=None), '_d930b28be6c5927595552b219965053e.localstack.example.com.')
2023-08-28 20:27:41 2023-08-28T18:27:41.497  INFO --- [  asgi_gw_10] localstack.request.aws     : AWS route53.ChangeResourceRecordSets => 500 (InternalError)
2023-08-28 20:27:42 2023-08-28T18:27:42.391  WARN --- [   asgi_gw_2] l.s.route53.provider       : UPSERT
2023-08-28 20:27:42 2023-08-28T18:27:42.391 ERROR --- [   asgi_gw_2] l.aws.handlers.logging     : exception during call chain: ('Could not find entry %s for name %s in zones', TargetRecord(target='_c9edd76ee4a0e2a74388032f3861cc50.ykybfrwcxw.acm-validations.aws.', record_type=<RecordType.CNAME: 3>, record_id=None), '_d930b28be6c5927595552b219965053e.localstack.example.com.')
2023-08-28 20:27:42 2023-08-28T18:27:42.392  INFO --- [   asgi_gw_2] localstack.request.aws     : AWS route53.ChangeResourceRecordSets => 500 (InternalError)
2023-08-28 20:27:43 2023-08-28T18:27:43.573  WARN --- [   asgi_gw_3] l.s.route53.provider       : UPSERT
2023-08-28 20:27:43 2023-08-28T18:27:43.573 ERROR --- [   asgi_gw_3] l.aws.handlers.logging     : exception during call chain: ('Could not find entry %s for name %s in zones', TargetRecord(target='_c9edd76ee4a0e2a74388032f3861cc50.ykybfrwcxw.acm-validations.aws.', record_type=<RecordType.CNAME: 3>, record_id=None), '_d930b28be6c5927595552b219965053e.localstack.example.com.')
2023-08-28 20:27:43 2023-08-28T18:27:43.573  INFO --- [   asgi_gw_3] localstack.request.aws     : AWS route53.ChangeResourceRecordSets => 500 (InternalError)
2023-08-28 20:27:50 2023-08-28T18:27:50.441  WARN --- [   asgi_gw_5] l.s.route53.provider       : UPSERT
2023-08-28 20:27:50 2023-08-28T18:27:50.442 ERROR --- [   asgi_gw_5] l.aws.handlers.logging     : exception during call chain: ('Could not find entry %s for name %s in zones', TargetRecord(target='_c9edd76ee4a0e2a74388032f3861cc50.ykybfrwcxw.acm-validations.aws.', record_type=<RecordType.CNAME: 3>, record_id=None), '_d930b28be6c5927595552b219965053e.localstack.example.com.')
2023-08-28 20:27:50 2023-08-28T18:27:50.443  INFO --- [   asgi_gw_5] localstack.request.aws     : AWS route53.ChangeResourceRecordSets => 500 (InternalError)
2023-08-28 20:28:04 2023-08-28T18:28:04.355  WARN --- [  asgi_gw_10] l.s.route53.provider       : UPSERT
2023-08-28 20:28:04 2023-08-28T18:28:04.355 ERROR --- [  asgi_gw_10] l.aws.handlers.logging     : exception during call chain: ('Could not find entry %s for name %s in zones', TargetRecord(target='_c9edd76ee4a0e2a74388032f3861cc50.ykybfrwcxw.acm-validations.aws.', record_type=<RecordType.CNAME: 3>, record_id=None), '_d930b28be6c5927595552b219965053e.localstack.example.com.')
2023-08-28 20:28:04 2023-08-28T18:28:04.356  INFO --- [  asgi_gw_10] localstack.request.aws     : AWS route53.ChangeResourceRecordSets => 500 (InternalError)
2023-08-28 20:28:34 2023-08-28T18:28:34.052  WARN --- [   asgi_gw_5] l.s.route53.provider       : UPSERT
2023-08-28 20:28:34 2023-08-28T18:28:34.053 ERROR --- [   asgi_gw_5] l.aws.handlers.logging     : exception during call chain: ('Could not find entry %s for name %s in zones', TargetRecord(target='_c9edd76ee4a0e2a74388032f3861cc50.ykybfrwcxw.acm-validations.aws.', record_type=<RecordType.CNAME: 3>, record_id=None), '_d930b28be6c5927595552b219965053e.localstack.example.com.')
2023-08-28 20:28:34 2023-08-28T18:28:34.056  INFO --- [   asgi_gw_5] localstack.request.aws     : AWS route53.ChangeResourceRecordSets => 500 (InternalError)
2023-08-28 20:29:05 2023-08-28T18:29:05.908  WARN --- [   asgi_gw_3] l.s.route53.provider       : UPSERT
2023-08-28 20:29:05 2023-08-28T18:29:05.910 ERROR --- [   asgi_gw_3] l.aws.handlers.logging     : exception during call chain: ('Could not find entry %s for name %s in zones', TargetRecord(target='_c9edd76ee4a0e2a74388032f3861cc50.ykybfrwcxw.acm-validations.aws.', record_type=<RecordType.CNAME: 3>, record_id=None), '_d930b28be6c5927595552b219965053e.localstack.example.com.')
2023-08-28 20:29:05 2023-08-28T18:29:05.912  INFO --- [   asgi_gw_3] localstack.request.aws     : AWS route53.ChangeResourceRecordSets => 500 (InternalError)
2023-08-28 20:29:11 2023-08-28T18:29:11.757  WARN --- [   asgi_gw_8] l.s.route53.provider       : UPSERT
2023-08-28 20:29:11 2023-08-28T18:29:11.757 ERROR --- [   asgi_gw_8] l.aws.handlers.logging     : exception during call chain: ('Could not find entry %s for name %s in zones', TargetRecord(target='_c9edd76ee4a0e2a74388032f3861cc50.ykybfrwcxw.acm-validations.aws.', record_type=<RecordType.CNAME: 3>, record_id=None), '_d930b28be6c5927595552b219965053e.localstack.example.com.')
2023-08-28 20:29:11 2023-08-28T18:29:11.758  INFO --- [   asgi_gw_8] localstack.request.aws     : AWS route53.ChangeResourceRecordSets => 500 (InternalError)

Please let me know if you need anything else from me. I’d be happy to provide it.

Thanks

Hi @dfangl do you perhaps have an update for me ?

I think I know where the issue is, I will try to get a fix out today, or latest tomorrow. I will report back if there are any roadblocks preventing this timeline.

It will take another day, sorry about the delay.

No worries. Thanks for the update @dfangl

This issue should now be resolved on the latest localstack/localstack-pro image. Can you report back if it works for you?