Lambda DELETE_FAILED: ApiGatewayDeployment

Platform
1

Currently when redeploying my lambda to localstack using serverless framework i’m experiencing this error message.

DELETE_FAILED: ApiGatewayDeployment1706805707391 (AWS::ApiGateway::Deployment)
An error occurred (BadRequestException) when calling the DeleteDeployment operation: Active stages pointing to this deployment must be moved or deleted

The only fix i’ve found for this in the interim has been to stop my localstack docker container and then restart but this means i need to go and create my ssm parameters all over again every time i want to redeploy code changes…

Initial deployment works fine but this happens on if i run serverless deploy --stage local for a second time?

EDIT:

Here is my serverless.ts file

import type { AWS } from '@serverless/typescript';
import graphApiWebhook from "@functions/graphApiWebhook";
import consumeSqsCallRecord from "@functions/consumeSqsCallRecord";
import createGraphSubscription from "@functions/createGraphSubscription";
import { config } from 'dotenv';
config();

const serverlessConfiguration: AWS = {
  service: 'teams-webhook',
  frameworkVersion: '3',
  plugins: ['serverless-esbuild', 'serverless-dotenv-plugin', 'serverless-offline', 'serverless-localstack'],
  provider: {
    name: 'aws',
    runtime: 'nodejs18.x',
    region: 'eu-west-1',
    timeout: 10,
    apiGateway: {
      minimumCompressionSize: 1024,
      shouldStartNameWithService: true,
    },
    iam: {
      role: {
        statements: [
          {
            Effect: 'Allow',
            Action: 'sqs:*', // Temporarily broadened for debugging
            Resource: 'arn:aws:sqs:eu-west-1:*:teams_webhook_test',
          },
        ],
      },
    },
    environment: {
      AWS_NODEJS_CONNECTION_REUSE_ENABLED: '1',
      NODE_OPTIONS: '--enable-source-maps --stack-trace-limit=1000',
      // Ensure these environment variables are correctly set in your .env file
      DB_TYPE: process.env.DB_TYPE,
      DB_HOST: process.env.DB_HOST,
      DB_USERNAME: process.env.DB_USERNAME,
      DB_PASSWORD: process.env.DB_PASSWORD,
      DB_DATABASE: process.env.DB_DATABASE,
      AWS_SQS_QUEUE_URL: process.env.AWS_SQS_QUEUE_URL,
      GRAPH_API_ACCESS_TOKEN: process.env.GRAPH_API_ACCESS_TOKEN,
    },
  },
  resources: {
    Resources: {
      TeamsWebhookTest: {
        Type: "AWS::SQS::Queue",
        Properties: {
          QueueName: "teams_webhook_test",
          VisibilityTimeout: 300,
          MessageRetentionPeriod: 864000,
        },
      },
    },
  },
  functions: { graphApiWebhook, consumeSqsCallRecord, createGraphSubscription},
  package: { individually: true },
  custom: {
    esbuild: {
      bundle: true,
      minify: false,
      sourcemap: true,
      exclude: ['aws-sdk'],
      target: 'node18',
      define: { 'require.resolve': undefined },
      platform: 'node',
      concurrency: 10,
    },
    localstack: {
      stages: ['local'],
      host: 'http://localhost',
      edgePort: 4566,
      autostart: true,
      lambda: {
        mountCode: false, // Changed for debugging
      },
    },
    // Static API Gateway deployment ID for debugging
    apiGatewayDeploymentId: {
      local: 'ApiGatewayDeploymentStatic', // Static ID for debugging
    },
  },
};

module.exports = serverlessConfiguration;
2

Hi @alexpower,

Not all CloudFormation resources support delete operation. You can find the list in our documentation CloudFormation | Docs (localstack.cloud)

4

Ah okay, thanks for your response @Marcel !

What is the usual workaround for this? I’ve simply been stopping and starting the container to deploy changes is this what most other people are doing?

5

Yes, that’s what we propose for now. You could try using the internal endpoints as well to reset the state Internal Endpoints | Docs (localstack.cloud).

6

Generally just deleting / re-creating the stack should work as well, if not, we’d definitely prioritze a fix.

The only known issue right now that we can’t quickly add a fix for are updates like UpdateStack or CreateChangeSet with UPDATE type.

7

Yeah i don’t think deleting & re-creating the stack works.

Running serverless deploy --stage local for a second time whilst my app is already running yields the delete failed error which i assume is simply attempting to delete the stack.

If it helps you i can send my serverless.ts file and the full error message so you can see if it’s simply a setup issue on my part.

This doesn’t happen if i deploy the app to aws however which is why i’m of the understanding that it’s a localstack issue rather than a setup/code issue but i could be wrong.

8

Hello @alexpower!

This particular issue for AWS::ApiGateway::Deployment should now be fixed with Fix issues in CFn resources and filter notfound exceptions for delete operations by dominikschubert · Pull Request #10273 · localstack/localstack · GitHub.

You can give it a try by pulling the latest tag of your image (localstack/localstack or localstack/localstack-pro) and see if that fixes this particular issue.

It still could be an issue with UPDATE detection, but this could maybe unblock you.

Thanks!

9

Thank you @bentsku ! I’ll give this a run through later to see if it’s rectified the issue :slight_smile: !