Upgrade to 3.4 - Cannot load Lambda state: object has no attribute 'logging_config'

Hi there, I wanted to use the local lambda option for testing things out locally before deployment and got a corrupted/invalid state with my lambda after upgrading from 3.2 to 3.4:


LocalStack version: 3.4.1.dev20240425063645
LocalStack Docker container id: 3310928911c6
LocalStack build date: 2024-04-25
LocalStack build git hash: c7494f2

2024-05-22T16:13:21.037  INFO --- [  MainThread] l.bootstrap.licensingv2    : Successfully requested and activated new license 0843xxxxxxxxx:pro 🔑✅
2024-05-22T16:13:21.470  INFO --- [  MainThread] l.p.snapshot.plugins       : registering ON_STARTUP load strategy
2024-05-22T16:13:21.851  INFO --- [  MainThread] l.p.snapshot.plugins       : registering SCHEDULED save strategy
2024-05-22T16:13:21.871  INFO --- [  MainThread] l.extensions.platform      : loaded 0 extensions
2024-05-22T16:13:21.877  INFO --- [-functhread4] hypercorn.error            : Running on https://0.0.0.0:4566 (CTRL + C to quit)
2024-05-22T16:13:21.877  INFO --- [-functhread4] hypercorn.error            : Running on https://0.0.0.0:4566 (CTRL + C to quit)
2024-05-22T16:13:21.877  INFO --- [-functhread4] hypercorn.error            : Running on https://0.0.0.0:443 (CTRL + C to quit)
2024-05-22T16:13:21.877  INFO --- [-functhread4] hypercorn.error            : Running on https://0.0.0.0:443 (CTRL + C to quit)
Ready.
2024-05-22T16:13:23.492  INFO --- [  MainThread] l.p.snapshot.plugins       : restoring state of all services on startup
2024-05-22T16:13:23.501  INFO --- [  MainThread] l.persistence.manager      : Loading state for sts took 5 ms
2024-05-22T16:13:23.547  INFO --- [  MainThread] l.services.s3.v3.provider  : Using /var/lib/localstack/state/s3 as storage path for s3 assets
2024-05-22T16:13:23.594  INFO --- [  MainThread] l.persistence.manager      : Loading state for s3 took 89 ms
2024-05-22T16:13:24.808  INFO --- [  MainThread] l.persistence.manager      : Loading state for dynamodb took 1214 ms
2024-05-22T16:13:24.854  WARN --- [  MainThread] l.s.lambda_.provider       : Failed to restore function version arn:aws:lambda:us-east-1:000000000000:function:novelcrafter-local:$LATEST
Traceback (most recent call last):
  File "/opt/code/localstack/.venv/lib/python3.11/site-packages/localstack/services/lambda_/provider.py", line 259, in on_after_state_load
    new_config = dataclasses.replace(fn_version.config, state=new_state)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dataclasses.py", line 1496, in replace
    changes[f.name] = getattr(obj, f.name)
                      ^^^^^^^^^^^^^^^^^^^^
AttributeError: 'VersionFunctionConfiguration' object has no attribute 'logging_config'
2024-05-22T16:13:24.855  INFO --- [  MainThread] l.persistence.manager      : Loading state for lambda took 47 ms
2024-05-22T16:14:16.666  INFO --- [   asgi_gw_0] localstack.request.http    : OPTIONS /2015-03-31/functions/ => 204
2024-05-22T16:14:16.681 ERROR --- [   asgi_gw_0] l.aws.handlers.logging     : exception during call chain: 'VersionFunctionConfiguration' object has no attribute 'logging_config'
2024-05-22T16:14:16.682  INFO --- [   asgi_gw_0] localstack.request.aws     : AWS lambda.ListFunctions => 500 (InternalError)
2024-05-22T16:14:16.800 ERROR --- [   asgi_gw_0] l.aws.handlers.logging     : exception during call chain: 'VersionFunctionConfiguration' object has no attribute 'logging_config'
2024-05-22T16:14:16.800  INFO --- [   asgi_gw_0] localstack.request.aws     : AWS lambda.ListFunctions => 500 (InternalError)
2024-05-22T16:14:16.891 ERROR --- [   asgi_gw_0] l.aws.handlers.logging     : exception during call chain: 'VersionFunctionConfiguration' object has no attribute 'logging_config'
2024-05-22T16:14:16.891  INFO --- [   asgi_gw_0] localstack.request.aws     : AWS lambda.ListFunctions => 500 (InternalError)
2024-05-22T16:14:17.175 ERROR --- [   asgi_gw_0] l.aws.handlers.logging     : exception during call chain: 'VersionFunctionConfiguration' object has no attribute 'logging_config'
2024-05-22T16:14:17.176  INFO --- [   asgi_gw_0] localstack.request.aws     : AWS lambda.ListFunctions => 500 (InternalError)

This is how the lambda was created initially:

# awslocal lambda create-function \
    --> function-name novelcrafter-local \
    --runtime>  nodejs20.x \
    --zip-> file fileb://lambda.zip \
    --handler>  lambda.handler \
    --> role arn:aws:iam::000000000000:role/lambda-role
{
    "FunctionName": "novelcrafter-local",
    "FunctionArn": "arn:aws:lambda:us-east-1:000000000000:function:novelcrafter-local",
    "Runtime": "nodejs20.x",
    "Role": "arn:aws:iam::000000000000:role/lambda-role",
    "Handler": "lambda.handler",
    "CodeSize": 1428529,
    "Description": "",
    "Timeout": 3,
    "MemorySize": 128,
    "LastModified": "2024-05-22T16:09:22.282351+0000",
    "CodeSha256": "uvNOuLlLTKOO8BRwKDoYKnHDcpAQL34tvL6XLsAV+LI=",
    "Version": "$LATEST",
    "TracingConfig": {
        "Mode": "PassThrough"
    },
    "RevisionId": "314a1516-97b6-4aba-a6cd-489f81a6b8fd",
    "State": "Pending",
    "StateReason": "The function is being created.",
    "StateReasonCode": "Creating",
    "PackageType": "Zip",
    "Architectures": [
        "x86_64"
    ],
    "EphemeralStorage": {
        "Size": 512
    },
    "SnapStart": {
        "ApplyOn": "None",
        "OptimizationStatus": "Off"
    },
    "RuntimeVersionConfig": {
        "RuntimeVersionArn": "arn:aws:lambda:us-east-1::runtime:8eeff65f6809a3ce81507fe733fe09b835899b99481ba22fd75b5a7338290ec1"
    }
}
# awslocal lambda create-function-url-config \
   >  --function-name novelcrafter-local \
  >   --auth-type NONE
{
    "FunctionUrl": "http://jwohzmjpzrmghec6339uwpigwfc9zfkl.lambda-url.us-east-1.localhost.localstack.cloud:4566/",
    "FunctionArn": "arn:aws:lambda:us-east-1:000000000000:function:novelcrafter-local",
    "AuthType": "NONE",
    "CreationTime": "2024-05-22T16:09:29.009957+0000"
}

Hey @spaceemotion! We’re aware that there are some compatibility issues between versions when using persistence. Typically, you’ll have to re-create your persistent state after you’ve upgraded. Please clear your localstack volume directory on your host machine and give it another try.

Does that mean I can’t upgrade the version unless I want to loose all local data? Can I downgrade to 3.2 again and keep working with that version instead?

Hello @spaceemotion,

You can still use the old version of LocalStack Docker image.
Just remove the current version, pull the older version with docker pull localstack/localstack-pro:3.2.0

I can’t upgrade the version unless I want to loose all local data?

The other services were loaded successfully, only lambda functions needs to be re-deployed. You can then create a new saved state that will be loaded without issue.

1 Like

Thanks I’ll try that later.

I actually tried to upgrade because I couldn’t get the lambda function call to work. The address couldn’t be resolved, and I even whitelisted localstack.cloud on my DNS.

I tried to do a couple tests, but it doesn’t work (none of the subdomains work. I even have to use HTTP paths for S3, since the subdomains have the same issue as the lambdas):

spaceemotion@NOSTROMO ~> traceroute jwohzmjpzrmghec6339uwpigwfc9zfkl.lambda-url.us-east-1.localhost.localstack.cloud
jwohzmjpzrmghec6339uwpigwfc9zfkl.lambda-url.us-east-1.localhost.localstack.cloud: No address associated with hostname
Cannot handle "host" cmdline arg `jwohzmjpzrmghec6339uwpigwfc9zfkl.lambda-url.us-east-1.localhost.localstack.cloud' on position 1 (argc 1)
spaceemotion@NOSTROMO ~ [2]> nslookup jwohzmjpzrmghec6339uwpigwfc9zfkl.lambda-url.us-east-1.localhost.localstack.cloud
Server:         172.23.128.1
Address:        172.23.128.1#53                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 Non-authoritative answer:
*** Can't find jwohzmjpzrmghec6339uwpigwfc9zfkl.lambda-url.us-east-1.localhost.localstack.cloud: No answer                                                                                                                                                                               

I am running this in WSL2 on Win 10, using Docker. Here is how that looks like:

  localstack:
    container_name: "${LOCALSTACK_DOCKER_NAME-novelcrafter_localstack_main}"
    image: localstack/localstack-pro:3.2
    ports:
      - "127.0.0.1:4566:4566"            # LocalStack Gateway
      - "127.0.0.1:4510-4559:4510-4559"  # external services port range
    environment:
      - DEBUG=${DEBUG-}
      - SERVICES=dynamodb,s3,lambda
      - LOCALSTACK_AUTH_TOKEN="ls-xxxxxx"
      - SNAPSHOT_SAVE_STRATEGY=SCHEDULED
      - SNAPSHOT_LOAD_STRATEGY=ON_STARTUP
      - PERSISTENCE=1
    volumes:
      - "localstack_pods:/var/lib/localstack"
      - "./docker/files:/home/external"
      - "/var/run/docker.sock:/var/run/docker.sock"

I apologize, but I’m not clear on what you’re referring to. Are you still discussing the persistence issue, or is it something else?

I would suggest that you test out the Lambda URL by following steps from our documentation Lambda | Docs (localstack.cloud).

I would also recommend using the most default docker-compose.yml.

There is also no need to use SERVICES environment variable, as they are started when necessary.