Is there a guide or any sort of docs on how to properly configure LocalStack to work with layers provided by AWS?
I’m trying to figure out how to use a aws-otel-python layer for local testing.
I tried using aws lambda add-layer-version-permission but that seemed to inject the wrong account ID in the layer ARN and I wasn’t able to figure out how to override it directly.
I’d appreciate any help here (as AWS is not my forte).
Hi @steamship_doug,
There is a documentation section on how to reference Lambda layers from AWS.
Lambda | Docs (localstack.cloud)
@Marcel ,
Thanks. Unfortunately, that command does not work for the layer I want to add. It inserts a different ID into the full layer than the one needed. This, I believe, is an issue as I do not own (nor did I create) the layer in question.
So, instead of adding a perm, I get Layer version arn:aws:lambda:us-west-2:<redacted>:layer:aws-otel-python-arm64-ver-1-17-0:1 does not exist.
Could you please provide a detailed description of the process you followed to obtain the layer?
I managed to get it downloaded and added to the Lambda function.
I’ve used the layer ARN from AWS Distro for OpenTelemetry Lambda Support For Python | AWS Open Distro for OpenTelemetry (aws-otel.github.io).
There may be an issue with the region, as LocalStack is using us-east-1 as a default region, and the ARN you have provided is pointing to us-west-2.
awslocal lambda create-function --function-name func1 --runtime python3.8 --role arn:aws:iam::000000000000:role/lambda-role --handler testlambda.handler --timeout 30 --zip-file fileb:///tmp/testlambda.zip --layers "arn:aws:lambda:us-east-1:901920570463:layer:aws-otel-python-arm64-ver-1-17-0:1"
{
"FunctionName": "func1",
"FunctionArn": "arn:aws:lambda:us-east-1:000000000000:function:func1",
"Runtime": "python3.8",
"Role": "arn:aws:iam::000000000000:role/lambda-role",
"Handler": "testlambda.handler",
"CodeSize": 276,
"Description": "",
"Timeout": 30,
"MemorySize": 128,
"LastModified": "2023-05-08T05:46:07.585175+0000",
"CodeSha256": "/2Z9Q0bWrDlJLJZiF/hFBs4n32EXHeb/EKrUGVhXBMQ=",
"Version": "$LATEST",
"TracingConfig": {
"Mode": "PassThrough"
},
"RevisionId": "c1febbb7-97aa-475b-8f8b-5b89965b2a1e",
"Layers": [
{
"Arn": "arn:aws:lambda:us-east-1:901920570463:layer:aws-otel-python-arm64-ver-1-17-0:1",
"CodeSize": 16374056
}
],
"State": "Pending",
"StateReason": "The function is being created.",
"StateReasonCode": "Creating",
"PackageType": "Zip",
"Architectures": [
"x86_64"
],
"EphemeralStorage": {
"Size": 512
},
"SnapStart": {
"ApplyOn": "None",
"OptimizationStatus": "Off"
},
"RuntimeVersionConfig": {
"RuntimeVersionArn": "arn:aws:lambda:us-east-1::runtime:8eeff65f6809a3ce81507fe733fe09b835899b99481ba22fd75b5a7338290ec1"
}
}
If you enable detailed logs with debugging enabled at DEBUG=1 or LS_LOG=trace, you should see something similar to the following.
localstack_main | 2023-05-08T05:45:58.139 DEBUG --- [ asgi_gw_0] l.s.a.l.aws_layer_fetcher : Request layer download URL for layer arn:aws:lambda:us-east-1:901920570463:layer:aws-otel-python-arm64-ver-1-17-0:1 from AWS proxy https://layerfetcher.production.aws.localstack.cloud/layers/arn:aws:lambda:us-east-1:901920570463:layer:aws-otel-python-arm64-ver-1-17-0:1
localstack_main | 2023-05-08T05:46:00.032 DEBUG --- [ asgi_gw_0] l.s.a.l.aws_layer_fetcher : Download layer zip from S3 bucket for layer arn:aws:lambda:us-east-1:901920570463:layer:aws-otel-python-arm64-ver-1-17-0:1
Marcel,
Thanks for your response. I was able to get localstack-pro to download the layer, but then I got stuck in a never ending deploy loop. Here are the logs:
localstack_main | 2023-05-05T21:25:13.012 DEBUG --- [functhread97] l.s.a.i.docker_runtime_exe : Creating service endpoint for function arn:aws:lambda:us-west-2:000000000000:function:E57319A7FEB442E3BA23928B6B56D468:$LATEST executor ded1ccdad4d976bd234421a85a21e65e
localstack_main | 2023-05-05T21:25:13.012 DEBUG --- [functhread97] l.s.a.i.docker_runtime_exe : Finished creating service endpoint for function arn:aws:lambda:us-west-2:000000000000:function:E57319A7FEB442E3BA23928B6B56D468:$LATEST executor ded1ccdad4d976bd234421a85a21e65e
localstack_main | 2023-05-05T21:25:13.012 DEBUG --- [functhread97] l.s.a.i.docker_runtime_exe : Assigning container name of localstack-main-lambda-e57319a7feb442e3ba23928b6b56d468-ded1ccdad4d976bd234421a85a21e65e to executor ded1ccdad4d976bd234421a85a21e65e
localstack_main | 2023-05-05T21:25:13.019 INFO --- [ asgi_gw_3] localstack.request.aws : AWS sts.AssumeRole => 200
localstack_main | 2023-05-05T21:25:13.022 DEBUG --- [68:$LATEST_0] l.s.a.i.docker_runtime_exe : Executing start docker executor pro-hook for function arn:aws:lambda:us-west-2:000000000000:function:E57319A7FEB442E3BA23928B6B56D468:$LATEST
localstack_main | 2023-05-05T21:25:13.023 DEBUG --- [68:$LATEST_0] l.u.c.docker_sdk_client : Creating container with attributes: {'mount_volumes': None, 'ports': None, 'cap_add': None, 'cap_drop': None, 'security_opt': None, 'dns': '127.0.0.1', 'additional_flags': '', 'workdir': None, 'privileged': None, 'labels': None, 'ulimits': None, 'command': None, 'detach': None, 'entrypoint': '/var/rapid/init', 'env_vars': {'AWS_DEFAULT_REGION': 'us-west-2', 'AWS_REGION': 'us-west-2', 'AWS_LAMBDA_FUNCTION_NAME': 'E57319A7FEB442E3BA23928B6B56D468', 'AWS_LAMBDA_FUNCTION_MEMORY_SIZE': 2048, 'AWS_LAMBDA_FUNCTION_VERSION': '$LATEST', 'AWS_LAMBDA_INITIALIZATION_TYPE': 'on-demand', 'AWS_LAMBDA_LOG_GROUP_NAME': '/aws/lambda/E57319A7FEB442E3BA23928B6B56D468', 'AWS_LAMBDA_LOG_STREAM_NAME': '2023/05/05/[$LATEST]ded1ccdad4d976bd234421a85a21e65e', 'AWS_ACCESS_KEY_ID': '<redacted>', 'AWS_SECRET_ACCESS_KEY': '<redacted>', 'AWS_SESSION_TOKEN': '<redacted>', 'LAMBDA_TASK_ROOT': '/var/task', 'LAMBDA_RUNTIME_DIR': '/var/runtime', 'AWS_XRAY_CONTEXT_MISSING': 'LOG_ERROR', 'AWS_XRAY_DAEMON_ADDRESS': '127.0.0.1:2000', '_AWS_XRAY_DAEMON_PORT': '2000', '_AWS_XRAY_DAEMON_ADDRESS': '127.0.0.1', 'TZ': ':UTC', 'AWS_LAMBDA_FUNCTION_TIMEOUT': 900, 'LOCALSTACK_HOSTNAME': '172.18.0.3', 'EDGE_PORT': '443', 'AWS_ENDPOINT_URL': 'http://172.18.0.3:443', 'LOCALSTACK_RUNTIME_ID': 'ded1ccdad4d976bd234421a85a21e65e', 'LOCALSTACK_RUNTIME_ENDPOINT': 'http://172.18.0.3:443/_localstack_lambda/ded1ccdad4d976bd234421a85a21e65e', '_HANDLER': 'steamship.invocable.entrypoint.safe_handler', 'AWS_EXECUTION_ENV': 'Aws_Lambda_python3.8', 'AWS_LAMBDA_EXEC_WRAPPER': '/opt/otel-instrument', 'LOCALSTACK_INIT_LOG_LEVEL': 'debug', 'LOCALSTACK_ENABLE_DNS_SERVER': '1', 'AWS_CA_BUNDLE': ''}, 'image_name': 'public.ecr.aws/lambda/python:3.8', 'interactive': None, 'name': 'localstack-main-lambda-e57319a7feb442e3ba23928b6b56d468-ded1ccdad4d976bd234421a85a21e65e', 'network': 'nludb_default', 'platform': 'linux/amd64', 'remove': None, 'self': <localstack.utils.container_utils.docker_sdk_client.SdkDockerClient object at 0xffff85b4d8d0>, 'tty': None, 'user': None}
localstack_main | 2023-05-05T21:25:13.076 DEBUG --- [68:$LATEST_0] localstack.packages.api : Installation of awslambda-runtime skipped (already installed).
localstack_main | 2023-05-05T21:25:13.076 DEBUG --- [68:$LATEST_0] l.u.c.docker_sdk_client : Copying file /var/lib/localstack/lib/awslambda-runtime/v0.1.17-pre/arm64/. into localstack-main-lambda-e57319a7feb442e3ba23928b6b56d468-ded1ccdad4d976bd234421a85a21e65e:/
localstack_main | 2023-05-05T21:25:13.311 DEBUG --- [68:$LATEST_0] l.u.c.docker_sdk_client : Copying file /tmp/lambda/awslambda-us-west-2-tasks/E57319A7FEB442E3BA23928B6B56D468-d7dbbb25-5a31-4c95-9b98-ee35d0fa5021/code/. into localstack-main-lambda-e57319a7feb442e3ba23928b6b56d468-ded1ccdad4d976bd234421a85a21e65e:/var/task
localstack_main | 2023-05-05T21:25:14.198 DEBUG --- [68:$LATEST_0] l.u.c.docker_sdk_client : Copying file /tmp/lambda/awslambda-us-west-2-tasks/aws-otel-python-amd64-ver-1-17-0-aa428427-f56d-4352-877f-9ee41e674c66/code/. into localstack-main-lambda-e57319a7feb442e3ba23928b6b56d468-ded1ccdad4d976bd234421a85a21e65e:/opt
localstack_main | 2023-05-05T21:25:14.806 DEBUG --- [ asgi_gw_0] l.u.c.docker_sdk_client : Removing container: localstack-main-lambda-e57319a7feb442e3ba23928b6b56d468-e46b44b2a45c2ea6325294dbe9a1e478
localstack_main | 2023-05-05T21:25:14.997 INFO --- [ asgi_gw_0] localstack.request.http : POST /_localstack_lambda/e46b44b2a45c2ea6325294dbe9a1e478/status/e46b44b2a45c2ea6325294dbe9a1e478/error => 202
localstack_main | 2023-05-05T21:25:15.272 DEBUG --- [68:$LATEST_0] l.u.c.docker_sdk_client : Starting container localstack-main-lambda-e57319a7feb442e3ba23928b6b56d468-ded1ccdad4d976bd234421a85a21e65e
localstack_main | 2023-05-05T21:25:15.508 DEBUG --- [68:$LATEST_0] l.u.c.container_client : Getting ipv4 address for container localstack-main-lambda-e57319a7feb442e3ba23928b6b56d468-ded1ccdad4d976bd234421a85a21e65e in network nludb_default.
localstack_main | 2023-05-05T21:25:15.876 WARN --- [ asgi_gw_2] l.s.a.i.executor_endpoint : Execution environment startup failed: {"errorMessage":"Error: exit status 1","errorType":"Extension.Crash"}
localstack_main | 2023-05-05T21:25:15.876 DEBUG --- [ asgi_gw_2] l.u.c.docker_sdk_client : Stopping container: localstack-main-lambda-e57319a7feb442e3ba23928b6b56d468-ded1ccdad4d976bd234421a85a21e65e
localstack_main | 2023-05-05T21:25:16.014 DEBUG --- [functhread97] l.s.a.i.version_manager : Detected no active environments for version arn:aws:lambda:us-west-2:000000000000:function:E57319A7FEB442E3BA23928B6B56D468:$LATEST. Starting one...
Hi @steamship_doug,
The logs indicate that there may be a mix of platforms affecting the start.
Copying file /var/lib/localstack/lib/awslambda-runtime/v0.1.17-pre/**arm64**
Copying file /tmp/lambda/awslambda-us-west-2-tasks/aws-otel-python-**amd64**
Please have a look at our Configuration | Docs (localstack.cloud).
LAMBDA_IGNORE_ARCHITECTURE=0(default) — Whether to ignore the AWS architectures (x86_64 or arm64) configured for the lambda function. Set to1to run cross-platform compatible lambda functions natively (i.e., Docker selects architecture).
Marcel,
Even with that set, I still see the following logs:
localstack_main | 2023-05-10T21:41:08.449 DEBUG --- [functhread61] l.s.a.i.version_manager : Starting new environment
localstack_main | 2023-05-10T21:41:08.449 DEBUG --- [functhread61] l.s.a.i.docker_runtime_exe : Creating service endpoint for function arn:aws:lambda:us-west-2:000000000000:function:513E7AF34F934BE5B64118A4855DD540:$LATEST executor 46e40577e5ce7731f285f0b15b7d759d
localstack_main | 2023-05-10T21:41:08.449 DEBUG --- [functhread61] l.s.a.i.docker_runtime_exe : Finished creating service endpoint for function arn:aws:lambda:us-west-2:000000000000:function:513E7AF34F934BE5B64118A4855DD540:$LATEST executor 46e40577e5ce7731f285f0b15b7d759d
localstack_main | 2023-05-10T21:41:08.449 DEBUG --- [functhread61] l.s.a.i.docker_runtime_exe : Assigning container name of localstack-main-lambda-513e7af34f934be5b64118a4855dd540-46e40577e5ce7731f285f0b15b7d759d to executor 46e40577e5ce7731f285f0b15b7d759d
localstack_main | 2023-05-10T21:41:08.458 INFO --- [ asgi_gw_1] localstack.request.aws : AWS sts.AssumeRole => 200
localstack_main | 2023-05-10T21:41:08.462 DEBUG --- [40:$LATEST_0] l.s.a.i.docker_runtime_exe : Executing start docker executor pro-hook for function arn:aws:lambda:us-west-2:000000000000:function:513E7AF34F934BE5B64118A4855DD540:$LATEST
localstack_main | 2023-05-10T21:41:08.462 DEBUG --- [40:$LATEST_0] l.u.c.docker_sdk_client : Creating container with attributes: {'mount_volumes': None, 'ports': None, 'cap_add': None, 'cap_drop': None, 'security_opt': None, 'dns': '127.0.0.1', 'additional_flags': '', 'workdir': None, 'privileged': None, 'labels': None, 'ulimits': None, 'command': None, 'detach': None, 'entrypoint': '/var/rapid/init', 'env_vars': {'AWS_DEFAULT_REGION': 'us-west-2', 'AWS_REGION': 'us-west-2', 'AWS_LAMBDA_FUNCTION_NAME': '513E7AF34F934BE5B64118A4855DD540', 'AWS_LAMBDA_FUNCTION_MEMORY_SIZE': 2048, 'AWS_LAMBDA_FUNCTION_VERSION': '$LATEST', 'AWS_LAMBDA_INITIALIZATION_TYPE': 'on-demand', 'AWS_LAMBDA_LOG_GROUP_NAME': '/aws/lambda/513E7AF34F934BE5B64118A4855DD540', 'AWS_LAMBDA_LOG_STREAM_NAME': '2023/05/10/[$LATEST]46e40577e5ce7731f285f0b15b7d759d', 'AWS_ACCESS_KEY_ID': <redacted>, 'AWS_SECRET_ACCESS_KEY': <redacted>, 'AWS_SESSION_TOKEN': <redacted>, 'LAMBDA_TASK_ROOT': '/var/task', 'LAMBDA_RUNTIME_DIR': '/var/runtime', 'AWS_XRAY_CONTEXT_MISSING': 'LOG_ERROR', 'AWS_XRAY_DAEMON_ADDRESS': '127.0.0.1:2000', '_AWS_XRAY_DAEMON_PORT': '2000', '_AWS_XRAY_DAEMON_ADDRESS': '127.0.0.1', 'TZ': ':UTC', 'AWS_LAMBDA_FUNCTION_TIMEOUT': 900, 'LOCALSTACK_HOSTNAME': '172.18.0.3', 'EDGE_PORT': '443', 'AWS_ENDPOINT_URL': 'http://172.18.0.3:443', 'LOCALSTACK_RUNTIME_ID': '46e40577e5ce7731f285f0b15b7d759d', 'LOCALSTACK_RUNTIME_ENDPOINT': 'http://172.18.0.3:443/_localstack_lambda/46e40577e5ce7731f285f0b15b7d759d', '_HANDLER': 'steamship.invocable.entrypoint.safe_handler', 'AWS_EXECUTION_ENV': 'Aws_Lambda_python3.8', 'AWS_LAMBDA_EXEC_WRAPPER': '/opt/otel-instrument', 'LOCALSTACK_INIT_LOG_LEVEL': 'debug', 'LOCALSTACK_ENABLE_DNS_SERVER': '1', 'AWS_CA_BUNDLE': ''}, 'image_name': 'public.ecr.aws/lambda/python:3.8', 'interactive': None, 'name': 'localstack-main-lambda-513e7af34f934be5b64118a4855dd540-46e40577e5ce7731f285f0b15b7d759d', 'network': 'nludb_default', 'platform': None, 'remove': None, 'self': <localstack.utils.container_utils.docker_sdk_client.SdkDockerClient object at 0xffffb41b58d0>, 'tty': None, 'user': None}
localstack_main | 2023-05-10T21:41:08.508 DEBUG --- [40:$LATEST_0] localstack.packages.api : Installation of awslambda-runtime skipped (already installed).
localstack_main | 2023-05-10T21:41:08.509 DEBUG --- [40:$LATEST_0] l.u.c.docker_sdk_client : Copying file /var/lib/localstack/lib/awslambda-runtime/v0.1.17-pre/arm64/. into localstack-main-lambda-513e7af34f934be5b64118a4855dd540-46e40577e5ce7731f285f0b15b7d759d:/
localstack_main | 2023-05-10T21:41:08.695 DEBUG --- [40:$LATEST_0] l.u.c.docker_sdk_client : Copying file /tmp/lambda/awslambda-us-west-2-tasks/513E7AF34F934BE5B64118A4855DD540-19c04eff-7739-45d5-8447-fd56b7025119/code/. into localstack-main-lambda-513e7af34f934be5b64118a4855dd540-46e40577e5ce7731f285f0b15b7d759d:/var/task
localstack_main | 2023-05-10T21:41:09.050 DEBUG --- [40:$LATEST_0] l.u.c.docker_sdk_client : Copying file /tmp/lambda/awslambda-us-west-2-tasks/aws-otel-python-amd64-ver-1-17-0-c5ef4fb8-e1c1-4545-82fc-f36fd90fea48/code/. into localstack-main-lambda-513e7af34f934be5b64118a4855dd540-46e40577e5ce7731f285f0b15b7d759d:/opt
localstack_main | 2023-05-10T21:41:09.687 DEBUG --- [ asgi_gw_0] l.u.c.docker_sdk_client : Removing container: localstack-main-lambda-513e7af34f934be5b64118a4855dd540-51f69525c07b0fe0bb33e5778e765110
localstack_main | 2023-05-10T21:41:09.832 INFO --- [ asgi_gw_0] localstack.request.http : POST /_localstack_lambda/51f69525c07b0fe0bb33e5778e765110/status/51f69525c07b0fe0bb33e5778e765110/error => 202
localstack_main | 2023-05-10T21:41:09.907 DEBUG --- [40:$LATEST_0] l.u.c.docker_sdk_client : Starting container localstack-main-lambda-513e7af34f934be5b64118a4855dd540-46e40577e5ce7731f285f0b15b7d759d
localstack_main | 2023-05-10T21:41:10.148 DEBUG --- [40:$LATEST_0] l.u.c.container_client : Getting ipv4 address for container localstack-main-lambda-513e7af34f934be5b64118a4855dd540-46e40577e5ce7731f285f0b15b7d759d in network nludb_default.
localstack_main | 2023-05-10T21:41:10.498 WARN --- [ asgi_gw_0] l.s.a.i.executor_endpoint : Execution environment startup failed: {"errorMessage":"Error: exit status 1","errorType":"Extension.Crash"}
And if I try to use the same architecture by using the amd64 version of the layer, I still see:
localstack_main | 2023-05-10T21:46:07.982 DEBUG --- [ asgi_gw_0] l.u.c.docker_sdk_client : Stopping container: localstack-main-lambda-afcd09a80fba42a4850322893ce1d4f5-425cab15c5dcb0121a086523a86dbbb6
localstack_main | 2023-05-10T21:46:08.706 DEBUG --- [functhread61] l.s.a.i.version_manager : Detected no active environments for version arn:aws:lambda:us-west-2:000000000000:function:AFCD09A80FBA42A4850322893CE1D4F5:$LATEST. Starting one...
localstack_main | 2023-05-10T21:46:08.708 DEBUG --- [functhread61] l.s.a.i.version_manager : Starting new environment
localstack_main | 2023-05-10T21:46:08.708 DEBUG --- [functhread61] l.s.a.i.docker_runtime_exe : Creating service endpoint for function arn:aws:lambda:us-west-2:000000000000:function:AFCD09A80FBA42A4850322893CE1D4F5:$LATEST executor 5d61d5662a1538fa5e60c39cac3e92de
localstack_main | 2023-05-10T21:46:08.709 DEBUG --- [functhread61] l.s.a.i.docker_runtime_exe : Finished creating service endpoint for function arn:aws:lambda:us-west-2:000000000000:function:AFCD09A80FBA42A4850322893CE1D4F5:$LATEST executor 5d61d5662a1538fa5e60c39cac3e92de
localstack_main | 2023-05-10T21:46:08.709 DEBUG --- [functhread61] l.s.a.i.docker_runtime_exe : Assigning container name of localstack-main-lambda-afcd09a80fba42a4850322893ce1d4f5-5d61d5662a1538fa5e60c39cac3e92de to executor 5d61d5662a1538fa5e60c39cac3e92de
localstack_main | 2023-05-10T21:46:08.728 INFO --- [ asgi_gw_3] localstack.request.aws : AWS sts.AssumeRole => 200
localstack_main | 2023-05-10T21:46:08.737 DEBUG --- [F5:$LATEST_0] l.s.a.i.docker_runtime_exe : Executing start docker executor pro-hook for function arn:aws:lambda:us-west-2:000000000000:function:AFCD09A80FBA42A4850322893CE1D4F5:$LATEST
localstack_main | 2023-05-10T21:46:08.737 DEBUG --- [F5:$LATEST_0] l.u.c.docker_sdk_client : Creating container with attributes: {'mount_volumes': None, 'ports': None, 'cap_add': None, 'cap_drop': None, 'security_opt': None, 'dns': '127.0.0.1', 'additional_flags': '', 'workdir': None, 'privileged': None, 'labels': None, 'ulimits': None, 'command': None, 'detach': None, 'entrypoint': '/var/rapid/init', 'env_vars': {'AWS_DEFAULT_REGION': 'us-west-2', 'AWS_REGION': 'us-west-2', 'AWS_LAMBDA_FUNCTION_NAME': 'AFCD09A80FBA42A4850322893CE1D4F5', 'AWS_LAMBDA_FUNCTION_MEMORY_SIZE': 2048, 'AWS_LAMBDA_FUNCTION_VERSION': '$LATEST', 'AWS_LAMBDA_INITIALIZATION_TYPE': 'on-demand', 'AWS_LAMBDA_LOG_GROUP_NAME': '/aws/lambda/AFCD09A80FBA42A4850322893CE1D4F5', 'AWS_LAMBDA_LOG_STREAM_NAME': '2023/05/10/[$LATEST]5d61d5662a1538fa5e60c39cac3e92de', 'AWS_ACCESS_KEY_ID': <redacted>, 'AWS_SECRET_ACCESS_KEY': <redacted>, 'AWS_SESSION_TOKEN': <redacted>, 'LAMBDA_TASK_ROOT': '/var/task', 'LAMBDA_RUNTIME_DIR': '/var/runtime', 'AWS_XRAY_CONTEXT_MISSING': 'LOG_ERROR', 'AWS_XRAY_DAEMON_ADDRESS': '127.0.0.1:2000', '_AWS_XRAY_DAEMON_PORT': '2000', '_AWS_XRAY_DAEMON_ADDRESS': '127.0.0.1', 'TZ': ':UTC', 'AWS_LAMBDA_FUNCTION_TIMEOUT': 900, 'LOCALSTACK_HOSTNAME': '172.18.0.3', 'EDGE_PORT': '443', 'AWS_ENDPOINT_URL': 'http://172.18.0.3:443', 'LOCALSTACK_RUNTIME_ID': '5d61d5662a1538fa5e60c39cac3e92de', 'LOCALSTACK_RUNTIME_ENDPOINT': 'http://172.18.0.3:443/_localstack_lambda/5d61d5662a1538fa5e60c39cac3e92de', '_HANDLER': 'steamship.invocable.entrypoint.safe_handler', 'AWS_EXECUTION_ENV': 'Aws_Lambda_python3.8', 'AWS_LAMBDA_EXEC_WRAPPER': '/opt/otel-instrument', 'LOCALSTACK_INIT_LOG_LEVEL': 'debug', 'LOCALSTACK_ENABLE_DNS_SERVER': '1', 'AWS_CA_BUNDLE': ''}, 'image_name': 'public.ecr.aws/lambda/python:3.8', 'interactive': None, 'name': 'localstack-main-lambda-afcd09a80fba42a4850322893ce1d4f5-5d61d5662a1538fa5e60c39cac3e92de', 'network': 'nludb_default', 'platform': None, 'remove': None, 'self': <localstack.utils.container_utils.docker_sdk_client.SdkDockerClient object at 0xffff9b4698d0>, 'tty': None, 'user': None}
localstack_main | 2023-05-10T21:46:08.781 DEBUG --- [F5:$LATEST_0] localstack.packages.api : Installation of awslambda-runtime skipped (already installed).
localstack_main | 2023-05-10T21:46:08.781 DEBUG --- [F5:$LATEST_0] l.u.c.docker_sdk_client : Copying file /var/lib/localstack/lib/awslambda-runtime/v0.1.17-pre/arm64/. into localstack-main-lambda-afcd09a80fba42a4850322893ce1d4f5-5d61d5662a1538fa5e60c39cac3e92de:/
localstack_main | 2023-05-10T21:46:08.996 DEBUG --- [F5:$LATEST_0] l.u.c.docker_sdk_client : Copying file /tmp/lambda/awslambda-us-west-2-tasks/AFCD09A80FBA42A4850322893CE1D4F5-35ed8e28-9973-4a5d-8715-614c467dc0fe/code/. into localstack-main-lambda-afcd09a80fba42a4850322893ce1d4f5-5d61d5662a1538fa5e60c39cac3e92de:/var/task
localstack_main | 2023-05-10T21:46:09.586 DEBUG --- [F5:$LATEST_0] l.u.c.docker_sdk_client : Copying file /tmp/lambda/awslambda-us-west-2-tasks/aws-otel-python-amd64-ver-1-17-0-4bc4341e-c730-4c4f-ac3c-d3afd8a89d00/code/. into localstack-main-lambda-afcd09a80fba42a4850322893ce1d4f5-5d61d5662a1538fa5e60c39cac3e92de:/opt
localstack_main | 2023-05-10T21:46:09.837 DEBUG --- [ asgi_gw_2] l.u.c.docker_sdk_client : Removing container: localstack-main-lambda-afcd09a80fba42a4850322893ce1d4f5-e7dc94abcf276d04d2c653c3f5298936
localstack_main | 2023-05-10T21:46:10.589 INFO --- [ asgi_gw_2] localstack.request.http : POST /_localstack_lambda/e7dc94abcf276d04d2c653c3f5298936/status/e7dc94abcf276d04d2c653c3f5298936/error => 202
localstack_main | 2023-05-10T21:46:11.216 DEBUG --- [F5:$LATEST_0] l.u.c.docker_sdk_client : Starting container localstack-main-lambda-afcd09a80fba42a4850322893ce1d4f5-5d61d5662a1538fa5e60c39cac3e92de
localstack_main | 2023-05-10T21:46:11.463 DEBUG --- [F5:$LATEST_0] l.u.c.container_client : Getting ipv4 address for container localstack-main-lambda-afcd09a80fba42a4850322893ce1d4f5-5d61d5662a1538fa5e60c39cac3e92de in network nludb_default.
localstack_main | 2023-05-10T21:46:11.820 WARN --- [ asgi_gw_1] l.s.a.i.executor_endpoint : Execution environment startup failed: {"errorMessage":"Error: exit status 1","errorType":"Extension.Crash"}
localstack_main | 2023-05-10T21:46:11.820 DEBUG --- [ asgi_gw_1] l.u.c.docker_sdk_client : Stopping container: localstack-main-lambda-afcd09a80fba42a4850322893ce1d4f5-5d61d5662a1538fa5e60c39cac3e92de
What do I need to do to run this layer?
Some more info: it seems to lead to a constantly churning set of docker containers trying to run python 3.8:
Here are the logs I could capture in the brief moments one of those containers was running:
2023-05-10 14:58:44 time="2023-05-10T21:58:44Z" level=debug msg="No code archives set. Skipping download." func=main.DownloadCodeArchives file="/home/runner/work/lambda-runtime-init/lambda-runtime-init/cmd/localstack/codearchive.go:21"
2023-05-10 14:58:44 time="2023-05-10T21:58:44Z" level=debug msg="Process running as root user." func=main.main file="/home/runner/work/lambda-runtime-init/lambda-runtime-init/cmd/localstack/main.go:146" euid=0 gid=0 uid=0 username=root
2023-05-10 14:58:44 time="2023-05-10T21:58:44Z" level=debug msg="Process running as non-root user." func=main.main file="/home/runner/work/lambda-runtime-init/lambda-runtime-init/cmd/localstack/main.go:148" euid=993 gid=990 uid=993 username=sbx_user1051
2023-05-10 14:58:44 time="2023-05-10T21:58:44Z" level=debug msg="Hot reloading disabled." func=main.RunHotReloadingListener file="/home/runner/work/lambda-runtime-init/lambda-runtime-init/cmd/localstack/awsutil.go:150"
2023-05-10 14:58:44 time="2023-05-10T21:58:44Z" level=debug msg="Runtime API Server listening on 127.0.0.1:9001" func="go.amzn.com/lambda/rapi.(*Server).Listen" file="/home/runner/work/lambda-runtime-init/lambda-runtime-init/lambda/rapi/server.go:94"
2023-05-10 14:58:44 time="2023-05-10T21:58:44Z" level=info msg="Configure environment for Init Caching." func="go.amzn.com/lambda/rapid.(*rapidContext).acceptStartRequestForInitCaching" file="/home/runner/work/lambda-runtime-init/lambda-runtime-init/lambda/rapid/start.go:391"
2023-05-10 14:58:44 time="2023-05-10T21:58:44Z" level=info msg="extensionsDisabledByLayer(/opt/disable-extensions-jwigqn8j) -> stat /opt/disable-extensions-jwigqn8j: no such file or directory" func=go.amzn.com/lambda/rapid.extensionsDisabledByLayer file="/home/runner/work/lambda-runtime-init/lambda-runtime-init/lambda/rapid/start.go:363"
2023-05-10 14:58:44 time="2023-05-10T21:58:44Z" level=debug msg="Received RUNNING" func="go.amzn.com/lambda/rapidcore.(*Server).Init" file="/home/runner/work/lambda-runtime-init/lambda-runtime-init/lambda/rapidcore/server.go:533"
2023-05-10 14:58:44 time="2023-05-10T21:58:44Z" level=debug msg="API request - POST /2020-01-01/extension/register, Headers:map[Accept-Encoding:[gzip] Content-Length:[32] Lambda-Extension-Name:[collector] User-Agent:[Go-http-client/1.1]]" func=go.amzn.com/lambda/rapi/middleware.AccessLogMiddleware.func1.1 file="/home/runner/work/lambda-runtime-init/lambda-runtime-init/lambda/rapi/middleware/middleware.go:76"
2023-05-10 14:58:44 time="2023-05-10T21:58:44Z" level=debug msg="API request - POST /2020-01-01/extension/register, Headers:map[Accept-Encoding:[gzip] Content-Length:[32] Lambda-Extension-Name:[collector] User-Agent:[Go-http-client/1.1]]" func=go.amzn.com/lambda/rapi/middleware.AccessLogMiddleware.func1.1 file="/home/runner/work/lambda-runtime-init/lambda-runtime-init/lambda/rapi/middleware/middleware.go:76"
2023-05-10 14:58:44 time="2023-05-10T21:58:44Z" level=info msg="External agent collector (cc9b5e8d-9e45-4072-9e24-2f4ccf78391f) registered, subscribed to [INVOKE SHUTDOWN]" func="go.amzn.com/lambda/rapi/handler.(*agentRegisterHandler).registerExternalAgent" file="/home/runner/work/lambda-runtime-init/lambda-runtime-init/lambda/rapi/handler/agentregister.go:104"
2023-05-10 14:58:44 time="2023-05-10T21:58:44Z" level=debug msg="Preregister runtime" func=go.amzn.com/lambda/rapid.doInit file="/home/runner/work/lambda-runtime-init/lambda-runtime-init/lambda/rapid/start.go:189"
2023-05-10 14:58:44 time="2023-05-10T21:58:44Z" level=debug msg="Start runtime" func=go.amzn.com/lambda/rapid.doInit file="/home/runner/work/lambda-runtime-init/lambda-runtime-init/lambda/rapid/start.go:222"
2023-05-10 14:58:44 time="2023-05-10T21:58:44Z" level=debug msg="API request - PUT /2022-07-01/telemetry, Headers:map[Accept-Encoding:[gzip] Content-Length:[212] Content-Type:[application/json] Lambda-Extension-Identifier:[cc9b5e8d-9e45-4072-9e24-2f4ccf78391f] User-Agent:[Go-http-client/1.1]]" func=go.amzn.com/lambda/rapi/middleware.AccessLogMiddleware.func1.1 file="/home/runner/work/lambda-runtime-init/lambda-runtime-init/lambda/rapi/middleware/middleware.go:76"
2023-05-10 14:58:44 time="2023-05-10T21:58:44Z" level=warning msg="First fatal error stored in appctx: Extension.Crash" func=go.amzn.com/lambda/appctx.StoreFirstFatalError file="/home/runner/work/lambda-runtime-init/lambda-runtime-init/lambda/appctx/appctxutil.go:156"
2023-05-10 14:58:44 time="2023-05-10T21:58:44Z" level=warning msg="Process 15(collector) exited: exit status 1" func="go.amzn.com/lambda/core.(*Watchdog).GoWait.func1" file="/home/runner/work/lambda-runtime-init/lambda-runtime-init/lambda/core/watchdog.go:67"
2023-05-10 14:58:44 time="2023-05-10T21:58:44Z" level=debug msg="Canceling flows: exit status 1" func="go.amzn.com/lambda/core.(*Watchdog).CancelFlows.func1" file="/home/runner/work/lambda-runtime-init/lambda-runtime-init/lambda/core/watchdog.go:82"
2023-05-10 14:58:44 time="2023-05-10T21:58:44Z" level=error msg="Init failed" func=go.amzn.com/lambda/rapid.handleStartError file="/home/runner/work/lambda-runtime-init/lambda-runtime-init/lambda/rapid/start.go:478" InvokeID= error="exit status 1"
2023-05-10 14:58:44 time="2023-05-10T21:58:44Z" level=info msg="Received signal" func=go.amzn.com/lambda/rapidcore.signalHandler file="/home/runner/work/lambda-runtime-init/lambda-runtime-init/lambda/rapidcore/sandbox.go:255" signal=terminated
2023-05-10 14:58:44 time="2023-05-10T21:58:44Z" level=info msg="Shutting down..." func=go.amzn.com/lambda/rapidcore.NewSandboxBuilder.func1 file="/home/runner/work/lambda-runtime-init/lambda-runtime-init/lambda/rapidcore/sandbox.go:95"
2023-05-10 14:58:45 time="2023-05-10T21:58:45Z" level=debug msg="API request - GET /2018-06-01/runtime/invocation/next, Headers:map[Accept:[*/*]]" func=go.amzn.com/lambda/rapi/middleware.AccessLogMiddleware.func1.1 file="/home/runner/work/lambda-runtime-init/lambda-runtime-init/lambda/rapi/middleware/middleware.go:76"
2023-05-10 14:58:45 time="2023-05-10T21:58:45Z" level=debug msg="API request - GET /2018-06-01/runtime/invocation/next, Headers:map[Accept:[*/*]]" func=go.amzn.com/lambda/rapi/middleware.AccessLogMiddleware.func1.1 file="/home/runner/work/lambda-runtime-init/lambda-runtime-init/lambda/rapi/middleware/middleware.go:76"
2023-05-10 14:58:44 2023-05-10T21:58:44Z [Info] Initializing AWS X-Ray daemon unknown
2023-05-10 14:58:44 2023-05-10T21:58:44Z [Debug] Listening on UDP 127.0.0.1:2000
2023-05-10 14:58:44 2023-05-10T21:58:44Z [Info] Using buffer memory limit of 119 MB
2023-05-10 14:58:44 2023-05-10T21:58:44Z [Info] 1904 segment buffers allocated
2023-05-10 14:58:44 2023-05-10T21:58:44Z [Debug] Using Endpoint read from Config file: http://172.18.0.3:443
2023-05-10 14:58:44 2023-05-10T21:58:44Z [Debug] Using proxy address:
2023-05-10 14:58:44 2023-05-10T21:58:44Z [Debug] Fetch region us-west-2 from commandline/config file
2023-05-10 14:58:44 2023-05-10T21:58:44Z [Info] Using region: us-west-2
2023-05-10 14:58:44 2023-05-10T21:58:44Z [Info] HTTP Proxy server using X-Ray Endpoint : http://172.18.0.3:443
2023-05-10 14:58:44 2023-05-10T21:58:44Z [Debug] Using Endpoint: http://172.18.0.3:443
2023-05-10 14:58:44 2023-05-10T21:58:44Z [Debug] Batch size: 10
2023-05-10 14:58:44 2023-05-10T21:58:44Z [Info] Starting proxy http server on 127.0.0.1:2000
2023-05-10 14:58:44 {"level":"info","ts":1683755924.4557374,"msg":"Launching OpenTelemetry Lambda extension","version":"v0.28.0"}
2023-05-10 14:58:44 {"level":"info","ts":1683755924.4573357,"logger":"telemetryAPI.Listener","msg":"Listening for requests","address":"sandbox:53612"}
2023-05-10 14:58:44 {"level":"info","ts":1683755924.4574277,"logger":"telemetryAPI.Client","msg":"Subscribing","baseURL":"http://127.0.0.1:9001/2022-07-01/telemetry"}
2023-05-10 14:58:44 {"level":"error","ts":1683755924.4582152,"logger":"telemetryAPI.Client","msg":"Subscription failed"}
2023-05-10 14:58:44 {"level":"fatal","ts":1683755924.458258,"msg":"Cannot register Telemetry API client","error":"request to http://127.0.0.1:9001/2022-07-01/telemetry failed: 404[404 Not Found] 404 page not found\n"}