Running Lambda in localstack inside a k8s strucutre

Hey all! I’m making a setup of a lambda deploy using localstack. My current infra is: I’m using kind to create a k8s cluster on a docker container, and inside this k8s cluster, i’m running a instance of localstack from official community image. When I’m trying to run a lambda on this localstack instance, my lambda is aparently starting without errors, but i’m unable to invoke this function. Is there any special config I need to do?

1 Like

image
looking at the state of my lambda, it appears to be active, but when I try invoking, always get timeout

found more logs about my error:

2024-01-16T18:48:37.789 DEBUG --- [-functhread9] l.s.a.i.version_manager    : Got invocation event 1c0b03d3-4fe3-4fa4-bdfe-98961d230b8b in loop
2024-01-16T18:48:37.789 DEBUG --- [-functhread9] l.s.a.i.version_manager    : Starting new environment
2024-01-16T18:48:37.789 DEBUG --- [-functhread9] l.s.a.i.docker_runtime_exe : Creating service endpoint for function arn:aws:lambda:us-east-1:000000000000:function:shareholders-ninety-day-positions-report:$LATEST executor a395346b672ee9467d2408781712073f
2024-01-16T18:48:37.789 DEBUG --- [-functhread9] l.s.a.i.docker_runtime_exe : Finished creating service endpoint for function arn:aws:lambda:us-east-1:000000000000:function:shareholders-ninety-day-positions-report:$LATEST executor a395346b672ee9467d2408781712073f
2024-01-16T18:48:37.789 DEBUG --- [-functhread9] l.s.a.i.docker_runtime_exe : Assigning container name of k8s-dev-env-control-plane-lambda-shareholders-ninety-day-positions-report-a395346b672ee9467d2408781712073f to executor a395346b672ee9467d2408781712073f
2024-01-16T18:48:37.797  INFO --- [   asgi_gw_1] localstack.request.aws     : AWS sts.AssumeRole => 200
2024-01-16T18:48:37.799 DEBUG --- [rt:$LATEST_0] l.u.c.container_client     : Getting ipv4 address for container k8s-dev-env-control-plane in network kind.
2024-01-16T18:48:37.805  INFO --- [rt:$LATEST_0] l.u.container_networking   : Determined main container target IP: 172.18.0.3
2024-01-16T18:48:37.809 DEBUG --- [rt:$LATEST_0] plugin.manager             : no extensions found in namespace localstack.hooks.lambda_start_docker_executor
2024-01-16T18:48:37.809 DEBUG --- [rt:$LATEST_0] l.u.c.docker_sdk_client    : Stopping container: k8s-dev-env-control-plane-lambda-shareholders-ninety-day-positions-report-a395346b672ee9467d2408781712073f
2024-01-16T18:48:37.811 DEBUG --- [rt:$LATEST_0] l.s.a.i.runtime_environmen : Unable to shutdown runtime handler 'a395346b672ee9467d2408781712073f'

Hi @mz.marcotuliocnd,

I would suggest to set the DEBUG=1 or LS_LOG=trace to get more details from the lambda invocation.

Try a simple lambda function (Lambda | Docs) and see whether it works or not.

Hello — Would like to chime in here, and mention that specific features such as execution of Lambda functions as Kubernetes pods are only available in the LocalStack Pro image.

Please find the docs here: Kubernetes | Docs

hey, thanks for the awnser guys! I was using version 2.1.0 of localstack, when I upgraded to 3.0.2, it started showing me the message saying that Docker images containers are a pro feature. So, I changed to zip file, and it worked. Thanks!

1 Like

Hi - Im having a similar issue. Could I just compare by setup to yours and see if you have any suggestions?

Im running the community edition of localstack in Kind on an m1 Mac via the latest Helm chart.
Im deploying a simple hello world lambda function as a zip using the default docker executor.

I am able to query the localstack lambda service to verify this installation completes successfully.

When I invoke the function:

  • the localstack logs show the lambda runner container being created
  • I can docker ps to show the runner container has been started on my host machine docker

However - the call then times out without any reply.
The localstack logs then show a connection timeout error connecting to an ip address starting 172.
This IP address corresponds to that of the lambda runner container that is spun up.

Im guessing I need to somehow configure Kind to allow access to that cidr block on my local machine ? Or configure localstack to use containerd and launch the runner containers on a node in the cluster ?

My kind configuration is standard except for the following I have had to add to allow kind to create containers using the host machine docker:

extraMounts:

  • hostPath: /var/run/docker.sock
    containerPath: /var/run/docker.sock

My localstack helm values then completes this wiring as:
extraEnvVars:
- name: DOCKER_HOST
value: unix:///var/run/docker.sock

Thanks for reading, and any suggestions gratefully received.

So as always the act of writing down the problem leads to the solution.
I just needed to configure localstack lambda to use the docker network that kind created to allow localstack to talk to the containers created on the host docker.

- name: LAMBDA_DOCKER_NETWORK
  value: kind

Thanks

1 Like