About a year ago we have released the first version of LocalStack Extensions: Extensions are a powerful mechanism to plug additional functionality into LocalStack - ranging from additional service emulators, to value-add features like Chaos Engineering, request logging, cloud resource replication/proxying, and more. An initial list of extensions can be found in this repository as well as in the LocalStack Web app.
Over the last couple of months we have been experimenting with a LocalStack Snowflake emulator extension, which allows to develop and test your Snowflake data pipelines entirely on your local machine!
Below is a simple sample snippet that illustrates how you can use the Python Snowflake client library to connect to the the Snowflake emulator running on your local machine (note that the hostname
snowflake.localhost.localstack.cloud resolves to a local IP address):
connection = snowflake.connector.connect(
connection.cursor().execute("CREATE TABLE table1(col1 INT)")
This extension is still highly experimental and covers only a small subset of the Snowflake API and functionality - although some of the key features are already available today, including:
- Basic operations on warehouses, databases, schemas, and tables (e.g., Using the Python Connector | Snowflake Documentation)
- Storing files in user/data/named stages (Choosing an Internal Stage for Local Files | Snowflake Documentation)
- Snowpark libraries (e.g., Snowpark Developer Guide for Python | Snowflake Documentation)
- Snowpipe streaming with Kafka connector (Using Snowflake Connector for Kafka with Snowpipe Streaming | Snowflake Documentation)
- … and quite a bit more!
Below is a sneak preview of the extension running in LocalStack, and using DBeaver to connect to it and run Snowflake SQL queries against it:
We are currently looking for early adopters who would like to give it a try with this extension, provide feedback on the current functionality, and help us shape the roadmap going forward.
If you’re interested in learning more, please reach out to us directly - either in this thread, or via DM, or via email to firstname.lastname@example.org .
We’re looking forward to getting your feedback and learning more about your use cases for developing & testing data pipelines locally with Snowflake!