Snowflake setup guide
Ingesting data from Snowflake into Propel.
This guide covers how to:
- Find your Snowflake Account Identifier
- Ensure change tracking is enabled on your Snowflake tables.
- Create a Snowflake user, role, and warehouse.
- (Optional) Configure Snowflake Network Policy.
- Create a Snowflake Data Pool
Requirements
- You have a Propel account.
- You have a Snowflake account.
- You have a Snowflake user with the
SECURITYADMIN
,SYSADMIN
, andACCOUNTADMIN
system-defined roles.
Step 1: Find your Snowflake Account Identifier
Your Snowflake Account Identifier is a string composed of your account locator, region, and cloud.
The Snowflake Account Identifier is different from the Snowflake Account Name.
Here is how to find your Snowflake Account Identifier:
Option 1: Using SQL
To find your Snowflake Account Identifier, run the following queries in your Snowflake worksheet:
and
Option 2: Using the UI
Using the UI, you can find your Snowflake Account Identifier by clicking on your account menu on the bottom left.
The Snowflake Account Identifier is a string in the following format depending on the region and cloud:
<account_locator>
or<account_locator>.<cloud_region_id>
or<account_locator>.<cloud_region_id>.<cloud>
Where:
cloud_region_id
is the identifier for the cloud region (dictated by the cloud platform).cloud
is the identifier for the cloud platform (aws, azure, or gcp).
The following are examples of the Snowflake Account Locator for different regions and clouds, for the full list see the Snowflake documentation.
Save your Snowflake Account Identifier and region as you will need them in the next steps.
Step 2: Ensure change tracking is enabled
Before setting up a Snowflake Data Pool (Propel’s high-speed data store and cache), it’s important to make sure CHANGE_TRACKING is enabled on the tables that you want to use as the underlying data.
You can enable it with the following command:
Step 3: Create the Snowflake user, role, and warehouse
Next, you need to create the Snowflake user, role, and warehouse for Propel.
We recommend creating a dedicated user, role, and warehouse for Propel. By creating a dedicated role and user, you can apply the principle of least privilege to give it only the necessary privileges. Additionally, by having a dedicated warehouse, you can monitor and control costs and ensure you have the necessary compute resources to operate the integration.
Copy the script below to a new Snowflake worksheet and select the “All Queries” checkbox.
Make sure you replace the values for the user_password
, database_name
, and schema_name
variables with the following values before running the script.
user_password
it must be at least 8 characters long, contain at least 1 digit, 1 uppercase letter and 1 lowercase letterdatabase_name
with the name of the Snowflake database where your schema is located, for exampleANALYTICS
.schema_name
with the database and schema name where your tables are located, for exampleANALYTICS.PUBLIC
.
You will need to run this script with a user that has SECURITYADMIN
and ACCOUNTADMIN
system-defined roles.
Step 4: (Optional) Configure Snowflake Network Policy
If you have defined a Snowflake Network Policy, you need to update it to include the Propel IP address by following the instructions on modifying network policies.
Propel IP Addresses:
Step 5: Create a Snowflake Data Pool
Now that you have created the Snowflake user, role, and warehouse, you can create a Snowflake Data Pool.
Create a Data Pool
Go to the “Data Pools” section in the Console, click “Create Data Pool” and click on the “Snowflake” tile.
If you create a Snowflake Data Pool for the first time, you must create your Snowflake credentials for Propel to connect to your Snowflake account.
Enter credential details
To create your Snowflake credentials, you will need the following details:
-
Credentials name: A name to identify the credentials in Propel.
-
Snowflake Account Identifier: The account identifier from Step 1.
-
Database: The database where your schema is located, for example
ANALYTICS
. -
Schema: The database and schema name where your tables are located, for example
PUBLIC
. -
Warehouse: The warehouse you created in Step 3.
-
Role: The role you created in Step 3.
-
User: The user you created in Step 3.
-
Password: The password for the user you created in Step 3.
Test your credentials
After entering your Snowflake credentials, click “Create and test credentials” to ensure Propel can successfully connect to your Snowflake account.
If the connection is successful, you will see a confirmation message. If not, check your entered credentials and try again.
Introspect your Snowflake tables
Here, you will see a list of tables available to ingest. If you don’t see the table you want to ingest, make sure your user has the right permissions to access the table.
Select table and columns
Continue with the setup flow, and then select the table as well as the columns you want to use.
Configure data type and settings
Select whether your data is “Append-only” or “Mutable data”.
To learn more, read out guide on Selecting table engine and sorting key.
Answer the questions in the wizard to complete the setup.
Confirm your table settings and click “Continue”.
Set sync interval
Specify how often you want Propel to sync your data.
Name your Data Pool
You can enable access policies later to restrict access to the data pool.
Click “Create Data Pool” to complete the setup.
Validate setup
Propel will connect to your Snowflake account, create a stream on the table to capture change data and sync it to Propel.
Preview data
Then you can click “Preview Data” to make sure your data is synchronized correctly.
Was this page helpful?