cross posted here: Nxdomain when trying to use local stack for S3 testing - Questions / Help - Elixir Programming Language Forum
I have set up local stack within docker-compose so that I can do integration testing with s3, however with my setup I seem to be running into a nxdomain error that I can’t seem to figure out, below is my setup and some of my code for testing - am I missing something or doing something wrong? I know that I was originally using host networking but switched to bridge network
error
#0 16.15 20:51:48.018 application=ex_aws domain=elixir file=lib/ex_aws/request.ex function=request_and_retry/7 line=78 mfa=ExAws.Request.request_and_retry/7 module=ExAws.Request pid=<0.591.0> [warn] ExAws: HTTP ERROR: :nxdomain for URL: "http://localstack:4566/test-bucket/" ATTEMPT: 9
#0 18.28
#0 18.28 20:51:50.156 application=ex_aws domain=elixir file=lib/ex_aws/request.ex function=request_and_retry/7 line=78 mfa=ExAws.Request.request_and_retry/7 module=ExAws.Request pid=<0.591.0> [warn] ExAws: HTTP ERROR: :nxdomain for URL: "http://localstack:4566/test-bucket/" ATTEMPT: 10
#0 18.31
#0 18.31
#0 18.31 1) test get_object/2 get existing file from s3 (S3Test)
#0 18.31 test/s3_test.exs:12
#0 18.31 ** (ExAws.Error) ExAws Request Error!
#0 18.31
#0 18.31 {:error, :nxdomain}
#0 18.31
#0 18.31 code: ExAws.request!(ExAws.S3.put_bucket("test-bucket", "us-west-2"))
#0 18.31 stacktrace:
#0 18.31 (ex_aws 2.3.2) lib/ex_aws.ex:89: ExAws.request!/2
#0 18.31 test/s3_test.exs:28: (test)
config/test.exs
config :ex_aws,
access_key_id: "test",
secret_access_key: "test",
region: "us-west-2"
config :ex_aws, :s3,
scheme: "http://",
host: "localstack",
port: 4566,
region: "us-west-2"
s3_test.exs
test "get existing file from s3" do
# Create some random bytes to store in the file.
contents = :crypto.strong_rand_bytes(100)
# We set up an on_exit callback to empty and then delete the bucket
# when the test exit, so that the next test has a clean slate.
on_exit(fn ->
"test-bucket"
|> ExAws.S3.list_objects()
|> ExAws.stream!()
|> Enum.each(&ExAws.request!(ExAws.S3.delete_object("test-bucket", &1.key)))
ExAws.request!(ExAws.S3.delete_bucket("test-bucket"))
end)
# Create bucket.
ExAws.request!(ExAws.S3.put_bucket("test-bucket", "us-west-2"))
# Upload a file.
ExAws.request!(ExAws.S3.put_object("test-bucket", "my/random/file", contents))
# Now, we run our code and assert on its behavior.
{:ok, object} = S3Helper.get_object("test-bucket", "my/random/file")
assert(Map.has_key?(object, :status_code) and object.status_code == 200)
end
s3 logic
@ex_aws_mod Application.compile_env(:tls_worker, [:test_doubles, :ex_aws], ExAws)
def get_object(bucket, path) do
ExAws.S3.get_object(bucket, path)
|> @ex_aws_mod.request(region: Application.get_env(:ex_aws, :region))
end
docker-compose
version: "3"
services:
test:
build:
context: .
dockerfile: Dockerfile.test
args:
MIX_ENV: test
ENVIRONMENT: test
volumes:
- /app
working_dir: /app
command: MIX_ENV=test mix test
depends_on:
- localstack
networks:
- test
stdin_open: true
tty: true
localstack:
container_name: "${LOCALSTACK_DOCKER_NAME-localstack_main}"
image: localstack/localstack
environment:
- AWS_DEFAULT_REGION="us-west-2"
- SERVICES="s3"
- EDGE_PORT="4566"
networks:
- test
ports:
- "4566:4566"
volumes:
- "${LOCALSTACK_VOLUME_DIR:-./volume}:/var/lib/localstack"
- "/var/run/docker.sock:/var/run/docker.sock"
networks:
test:
driver: bridge