SSL shutdown timed out error in localstack S3

Platform
1

I am new to localstack. I am trying to use localstack S3 in docker-compose like below

  localstack:
    image: localstack/localstack:3.4
    container_name: localstack
    ports:
      - "4566-4597:4566-4597"
    environment:
      - SERVICES=s3
      - DEBUG=${DEBUG- }
      - DOCKER_HOST=unix:///var/run/docker.sock
    volumes:
      - "$PWD/setup-s3.sh:/etc/localstack/init/ready.d/init-aws.sh"
      - "/var/run/docker.sock:/var/run/docker.sock"

I keep on seeing lot of exception like below in the docker console logs

2024-05-22T16:11:28.012 ERROR --- [-functhread6] asyncio                    : Unhandled exception in client_connected_cb
transport: <asyncio.sslproto._SSLProtocolTransport object at 0xffff5074ce30>
Traceback (most recent call last):
  File "/opt/code/localstack/.venv/lib/python3.11/site-packages/hypercorn/asyncio/run.py", line 109, in _server_callback
    await TCPServer(app, loop, config, context, reader, writer)
  File "/opt/code/localstack/.venv/lib/python3.11/site-packages/hypercorn/asyncio/tcp_server.py", line 74, in run
    await self._close()
  File "/opt/code/localstack/localstack/utils/server/http2_server.py", line 79, in _close
    return await _close_orig(self)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/code/localstack/.venv/lib/python3.11/site-packages/hypercorn/asyncio/tcp_server.py", line 117, in _close
    await self.writer.wait_closed()
  File "/usr/local/lib/python3.11/asyncio/streams.py", line 364, in wait_closed
    await self._protocol._get_close_waiter(self)
TimeoutError: SSL shutdown timed out

Can someone let me know how can I get around this?

Environment

  • MacOS Sonoma 14.4.1
  • Docker Desktop 4.5.0
2

Hello @tuk!

The logs are not telling a lot here, could you try to explain what you are doing and how are you interacting with the LocalStack container?

Could you also try to add the following environment variable to see if that would fix your issue? GATEWAY_SERVER=twisted (this is now the default with the latest image).

If you’re only using the S3 service, you could also use the S3-only docker image, available with the following tag: localstack/localstack:s3-latest.

3

I changed to using localstack:s3-latest. But now it is throwing the below error.

LocalStack version: 3.4.1.dev
LocalStack Docker container id: 1aa76389ab7f
LocalStack build date: 2024-04-25
LocalStack build git hash: 9287822

2024-05-23T04:48:54.766  INFO --- [  MainThread] localstack.utils.bootstrap : Execution of "start_runtime_components" took 2477.67ms
2024-05-23T04:48:55.468  INFO --- [  MainThread] localstack.utils.bootstrap : Execution of "_load_service_plugin" took 699.44ms
2024-05-23T04:48:55.469  INFO --- [  MainThread] localstack.utils.bootstrap : Execution of "require" took 699.74ms
2024-05-23T04:48:55.469  INFO --- [  MainThread] localstack.utils.bootstrap : Execution of "preload_services" took 702.62ms

Ready.

S3 Configuration started

/etc/localstack/init/ready.d/init-aws.sh: line 11: aws: command not found

2024-05-23T04:48:55.480 ERROR --- [  MainThread] localstack.runtime.init    : Error while running script Script(path='/etc/localstack/init/ready.d/init-aws.sh', stage=READY, state=ERROR): Script /etc/localstack/init/ready.d/init-aws.sh returned a non-zero exit code 127

My setup-s3.sh looks like below

#!/usr/bin/env bash
set -e
export TERM=ansi
export AWS_ACCESS_KEY_ID=foobar
export AWS_SECRET_ACCESS_KEY=foobar
export AWS_DEFAULT_REGION=eu-west-2
export PAGER=

echo "S3 Configuration started"

aws --endpoint-url=http://localhost:4566 s3 mb s3://local-payload-bucket
aws --endpoint-url=http://localhost:4566 s3 mb s3://local-config-bucket

echo "S3 Configured"

I am using AWS V2 SDK client to connect to localstack S3 as mentioned here and doing normal get, put, etc

4

If I add GATEWAY_SERVER=twisted I do not see the exceptions. My working localstack section in docker-compose looks like below

  localstack:
    image: localstack/localstack:3.4
    container_name: localstack
    ports:
      - "4566-4597:4566-4597"
    environment:
      - SERVICES=s3
      - GATEWAY_SERVER=twisted
      - DEBUG=${DEBUG- }
      - DOCKER_HOST=unix:///var/run/docker.sock
    volumes:
      - "$PWD/setup-s3.sh:/etc/localstack/init/ready.d/init-aws.sh"
      - "/var/run/docker.sock:/var/run/docker.sock"

Can you please help me with the below queries now?

  1. I am using 3.4 version. Is not that the latest version? If yes then why I have to add GATEWAY_SERVER=twisted seperately?
  2. What does GATEWAY_SERVER=twisted do?
  3. I am using S3 only. How can I use localstack/localstack:s3-latest? Is there a way for me to specify the version in localstack/localstack:s3-latest instead of using latest always. My goal is to first test latest and then move to that version?
  4. localstack/localstack:s3-latest is not working with init-aws.sh I specified in my last reply. Is there a way for me to make this work?
5

I’ll try to reply point per point:

  1. 3.4 is our latest tagged version. We release a latest version containing all the changes we currently develop, and our next tagged release will be 3.5, in early June. GATEWAY_SERVER=twisted will not be needed then, as it will be default.
  2. GATEWAY_SERVER=twisted changes our underlying HTTP server used to serve HTTP requests. We’ve had issue with our previous implementation and worked lately to fix those issues, which also gave performance improvements. As it seems it fixes your issue, I’d advice you to keep your working configuration, and once we release 3.5, just remove the configuration flag as it will be default.
  3. & .4 For now, the advantage of using the S3-only Docker image is that it is lightweight. Sadly, being lightweight means we removed some utilities, like the AWS CLI, which is why your init scripts are not working anymore. You would need to run pip install awscli as the first command in your bash script to make it work.
    Coming back to the S3 image tags, we currently do not release tagged versions of the S3 image, and only have the s3-latest tag. Knowing this for now, I’d suggest you continue working with localstack/localstack:3.4 or localstack/localstack:latest.

I hope this answers your question, please do not hesitate if you have more!

1 Like