❄️ Introducing the LocalStack Snowflake extension (experimental)

1

Update: The information in this post is slightly outdated. In the meantime, we have released a public preview version of the LocalStack Snowflake emulator - see blog post here: Introducing LocalStack for Snowflake: The new emulator to build & test data pipelines locally - LocalStack

Hi all,

About a year ago we have released the first version of LocalStack Extensions: Extensions are a powerful mechanism to plug additional functionality into LocalStack - ranging from additional service emulators, to value-add features like Chaos Engineering, request logging, cloud resource replication/proxying, and more. An initial list of extensions can be found in this repository as well as in the LocalStack Web app.

Over the last couple of months we have been experimenting with a LocalStack :snowflake: Snowflake emulator extension, which allows to develop and test your Snowflake data pipelines entirely on your local machine!

Below is a simple sample snippet that illustrates how you can use the Python Snowflake client library to connect to the the Snowflake emulator running on your local machine (note that the hostname snowflake.localhost.localstack.cloud resolves to a local IP address):

connection = snowflake.connector.connect(
    user="test",
    password="test",
    account="test",
    database="test",
    host="snowflake.localhost.localstack.cloud",
)
connection.cursor().execute("CREATE TABLE table1(col1 INT)")

This extension is still highly experimental and covers only a small subset of the Snowflake API and functionality - although some of the key features are already available today, including:

Below is a sneak preview of the extension running in LocalStack, and using DBeaver to connect to it and run Snowflake SQL queries against it:

We are currently looking for early adopters who would like to give it a try with this extension, provide feedback on the current functionality, and help us shape the roadmap going forward.

If you’re interested in learning more, please reach out to us directly - either in this thread, or via DM, or via email to support@localstack.cloud .

We’re looking forward to getting your feedback and learning more about your use cases for developing & testing data pipelines locally with Snowflake!

2 Likes
2

Hi @whummer , Does this extension support Snowflake Java libraries or JDBC?
If yes, could you please share documents to get started with it?

3

Hi @harindra , yes it does support Java libraries and JDBC.

We do not have extensive documentation for the extension yet, unfortunately, as it is not generally available yet. Can you please reach out to us via email (support@localstack.cloud), or in our Community Slack channel (Slack) - happy to discuss the next steps there to give you access. Thanks!

4

Hi @whummer, I am interested in testing out your Snowflake extension to see how it integrates with my service.
I found this localstack-extension-snowflake - pypi Package Security Analysis - Socket and have already installed the extension. Can you please guide me on how to enable Snowflake service and how to check if Snowflake is working properly?

1 Like
5

Does your tool allow for testing of Dynamic Tables, Streams, Tasks?

1 Like
6

Hi @whummer, I would also be greatly interested in trying this feature. I’ll send an email,
Thanks,

1 Like
7

@whummer I am also interested

1 Like
8

@whummer I’m also interested. would love to try it

1 Like
9

Hi @whummer, this seems to really cover a concrete need.
Looking forward to discover more.
Thanks!

1 Like
10

Hi @whummer , Does this extension support Snowflake Connector for .NET (GitHub - snowflakedb/snowflake-connector-net: Snowflake Connector for .NET)?
If yes, could you please share documents to get started with it? And what is a valid connection string?

1 Like
11

Hi @annaZh,

Please find some relevant docs for using Snowflake connectors here: Snowflake Drivers | Docs

As of now, the docs cover Python & Node.js, but a similar setup would work for .NET among others as well.

1 Like
12

Hi @HarshCasper ,

I tried to follow these instructions Installation | Docs (using Docker Compose), but I had problems with the connection string. When I run the command conn.Open(); I get an exception 'Error: Snowflake Internal Error: Unable to connect. One or more errors occurred. (A task was canceled.) SqlState: 08006, VendorCode: 270001, QueryId: '.

Could you please tell me what I’m doing wrong?
Maybe there’s something wrong with my connection string?

My Docker Compose:

version: "3.8"

services:
  localstack:
    container_name: "localstack-main"
    image: localstack/snowflake
    ports:
      - "127.0.0.1:4566:4566"
    environment:
      - LOCALSTACK_AUTH_TOKEN=${LOCALSTACK_AUTH_TOKEN:?}
    volumes:
      - "./volume:/var/lib/localstack"

My connection string:
var connectionString = "user=test;password=test;account=test;db=test;host=snowflake.localhost.localstack.cloud";

My code:

var connectionString = "user=test;password=test;account=test;db=test;host=snowflake.localhost.localstack.cloud";
using (var conn = new SnowflakeDbConnection())
{
    conn.ConnectionString = connectionString;
    conn.Open();

    using (IDbCommand cmd = conn.CreateCommand())
    {
        cmd.CommandText = "create or replace table test(id int, text varchar);";
        cmd.ExecuteNonQuery();

        
    }

    conn.Close();
}
13

Hi @whummer,

I’m trying to integrate this extension with my java 17 codebase. So far is working good except the prepared statements. There is any reason why the prepared statements cannot be used with localstack environment?

Thank you!

1 Like
14

Would you like to move the conversation to our Slack community? It would be much easier to provide assistance in a 1o1 fashion :slight_smile:

1 Like