Ingest data from Snowflake.
Feature name | Supported | Notes |
---|---|---|
Syncs inserts, updates, and deletes. | ✅ | See How Propel syncs inserts, updates, and deletes. |
Re-sync | ✅ | See Re-sync section. |
Configurable sync interval | ✅ | See How Propel syncs section. It can be configured to occur at intervals ranging from every minute to every 24 hours. |
Sync Pausing / Resuming | ✅ | |
Real-time updates | ✅ | See the Real-time updates section. |
Real-time deletes | ✅ | See the Real-time deletes section. |
Batch Delete API | ✅ | See Batch Delete API. |
Batch Update API | ✅ | See Batch Update API. |
API configurable | ✅ | See API docs. |
Terraform configurable | ✅ | See Terraform docs. |
Structure | Requirement |
---|---|
Table | Change tracking enabled |
View | Change tracking enabled on underlying tables |
Dynamic Tables | No additional setup (change tracking always enabled) |
AddColumnToDataPool
job.
For breaking changes like column deletions or type modifications, recreate the Data Pool.
See our Changing Schemas section for more details.
dbt full-refresh
operation re-creates the tablereSyncSnowflakeDataPool
mutation for API-based re-syncing.
Snowflake Type | Propel Type | Notes |
---|---|---|
DATETIME , TIMESTAMP_NTZ , TIMESTAMP_LTZ , TIMESTAMP_TZ , TIMESTAMP(p) | TIMESTAMP | Timestamps without a timezone will be synced as UTC. |
DATE | DATE | |
BOOLEAN | BOOLEAN | |
NUMBER(p≤9, s=0) | INT32 | |
NUMBER(p≤18, s=0) | INT64 | |
NUMBER(p≤9, s>0) | FLOAT | |
NUMBER(p≤18, s>0) , NUMBER(p>18, s) | DOUBLE | |
NUMBER(p, s) , DECIMAL(p, s) , NUMERIC(p, s) | Depends on precision and scale | |
NUMBER(38, s>0) , INT , INTEGER , BIGINT , SMALLINT , TINYINT , BYTEINT | DOUBLE | Snowflake maps INT types to NUMBER(38, 0) , which Propel represents as a DOUBLE as it can have numbers larger than INT64 . See Snowflake numeric types. |
FLOAT , FLOAT4 , FLOAT8 , DOUBLE , DOUBLE PRECISION , REAL | DOUBLE | |
VARCHAR , CHAR , CHARACTER , STRING , TEXT , BINARY , VARBINARY | STRING | |
TIME | STRING | |
VARIANT , OBJECT , ARRAY | JSON | |
GEOGRAPHY , GEOMETRY | Not supported |
How long does the data take to sync from Snowflake to Propel?
Does Propel read through to Snowflake on every query?
Can you sync only certain columns from Snowflake into a Data Pool?
What happens if Snowflake is down?
When does the Data Pool syncing interval start?
LIVE
or when syncing is resumed.