New features, fixes, and improvements.
Ingest data from Amazon Firehose streams into Propel’s Serverless ClickHouse with zero configuration.
A customizable React component for selecting dashboard time ranges, with seamless Propel Query API integration.
Stream Segment events directly to Propel for real-time analytics.
The Data Grid is a React component for easily visualizing data in a table format.
Features you no longer need to build:
DynamoDB excels as a transactional database, but it’s not ideal for analytics involving aggregations and GROUP BY queries.
That’s where Propel’s DynamoDB connector shines. It allows you to replicate DynamoDB data in real time to Propel’s Serverless ClickHouse, enabling advanced analytics that are both fast and cost-effective.
Key features:
Added support for SHOW commands including:
SHOW DATABASES
SHOW TABLES
SHOW COLUMNS
SHOW INDEXES
SHOW CREATE DATABASE
SHOW CREATE TABLE
SHOW FUNCTIONS
SHOW ENGINES
Added support for system tables including:
system.databases
system.tables
system.columns
system.aggregate_function_combinators
system.formats
system.functions
system.licenses
system.settings
system.settings_changed
system.table_engines
system.table_functions
system.time_zones
Improved PopSQL compatibility:
date_time_output_format = 'simple'
when Faraday user agent is detectedEnhanced timezone handling:
Fixed bug where tables were added without “propel” database qualifier
Changed currentDatabase()
to return “propel”
Added support for database query parameter and x-clickhouse-database header
Added support for wait_end_of_query
parameter used by Python clients
Improved authentication handling for missing X-ClickHouse-Key header
Added support for client_protocol_version in ClickHouse HTTP endpoint
We’ve launched a native ClickHouse SQL HTTP interface that lets you query Propel’s Serverless ClickHouse directly. You can use any standard ClickHouse library or SDK to run SELECT
and SHOW
queries against your data.
To get started, use the following connection details:
Parameter | Value |
---|---|
Host | https://clickhouse.us-east-2.propeldata.com |
Port | 8443 |
Database | propel |
user | Your Propel Application ID |
password | Your Propel Application secret |
Here’s an example of querying data using the ClickHouse HTTP interface with curl:
Learn more about the ClickHouse SQL interface.
We’ve completely revamped our documentation to make it easier for developers to get started and be successful with Propel.
The new docs include the following sections:
Serverless ClickHouse: Connection details, supported SQL syntax, SQL reference, functions, and everything you need to know to manage, transform and query your data.
Ingestion: Guides on ingesting data from a variety of sources to Propel, including Webhooks, events streams, and data warehouses.
Query APIs: Guides and examples for using the SQL and Query APIs for common analytics use cases and data visualizations.
Embeddable UIs: Guides and documentation for embedding Propel’s visualization components into your applications.
Explore the new docs.
You can now authenticate server-side applications using HTTP Basic Authentication with your Application ID and secret.
Learn more about server-side authentication.
The <GroupBy>
component is a dropdown menu that allows users to dynamically change the grouping of data for multidimensional analysis.
Learn more about the <GroupBy>
component.
X-ClickHouse-User
and X-ClickHouse-Key
headers.PENDING
state.chartConfigProps
in the <TimeSeries>
component.groupBy
functionality to <TimeSeries>
and <FilterProvider>
components for dynamic data grouping.<Leaderboard>
component to properly sync with time range from <FilterProvider>
.legendPosition
prop in <PieChart>
component.accentColors
prop to <Leaderboard>
component for visual styling.DATA_POOL_READ
, DATA_POOL_STATS
, METRIC_READ
, and METRIC_STATS
scopes will no longer be able to set these scopes as they are now deprecatedDATA_POOL_READ
, DATA_POOL_STATS
, METRIC_READ
, and METRIC_STATS
API scopes.DATA_POOL_QUERY
, METRIC_QUERY
API scopes to allow seeing the schema for the Data Pools and columns they have access to.AND
and OR
operators=
to include >
, <
, LIKE
, and IN
among others.position
, locate
, positionCaseInsensitive
, positionUTF8
, positionCaseInsensitiveUTF8
- multiSearch functions for various use cases (e.g., AllPositions
, FirstPosition
, FirstIndex
)match
, REGEXP
, multiMatchAny
, multiMatchAnyIndex
, multiMatchAllIndices
- Fuzzy matching: multiFuzzyMatchAny
, multiFuzzyMatchAnyIndex
, multiFuzzyMatchAllIndices
extract
, extractAll
, extractAllGroupsHorizontal
, extractAllGroupsVertical
- like
, notLike
, ilike
, notILike
functionsngramDistance
, ngramSearch
(with case-sensitive and UTF8 variants)- countSubstrings
, countMatches
(with case-insensitive options)regexpExtract
, hasSubsequence
, hasToken
(with various options for case sensitivity and UTF8)<Container>
, <Flex>
, <Grid>
, and <Card>
.<Text>
and <Heading>
.<Tabs>
component for tab-based layouts with support for Card components as tabs.gray
) from "@propeldata/ui-kit/colors"
for consistent styling across the application.<TimeRangePicker>
now integrates seamlessly with <FilterProvider>
for improved data filtering across components.<TimeGrainPicker>
component that also integrated seamlessly with the <FilterProvider>
.groupBy
functionality to the <TimeSeries>
component, enabling data grouping and more flexible visualizations.<SimpleFilter>
component with clearable functionality. Developers can use the disableClearable
prop to turn off this feature if needed.accentColor
with accentColors
to provide more versatile theming options.partition_by
, order_by
, and primary_key
fields. Users can now use the syntax [""]
to define these fields as empty when needed. This improvement allows for more precise control over Data Pool settings.NOT_FOUND
error when the requested resource doesn’t exist. This applies to Materialized Views, Copy Jobs, and Data Pool access policies, improving error handling and user experience.<SimpleFilter>
component styles with select-based components like the <TimeRangePicker>
for a more uniform user interface.order_by
, partition_by
, and primary_key
columns were being applied in an incorrect order when defining Table Settings during Data Pool creation. We replaced Set
with List
to ensure order is preserved in fields where it’s critical.arrayJoin
: Allows for the expansion of arrays into separate rows.JSONExtractArrayRaw
: Extracts an array from a JSON string.JSONExtractKeys
: Retrieves keys from a JSON object.JSONArrayLength
: Determines the length of a JSON array.tupleNames
: Returns the names of tuple elements.tupleElement
: Extracts a specific element from a tuple.Our most powerful Propellers, the P1_LARGE and P1_X_LARGE, which can read 250 and 500 million rows per second respectively, now have a significantly lower price.
Propeller | Old price | New price | % Price drop |
---|---|---|---|
P1_MEDIUM | $0.10 per GB read | $0.06 per GB read | 40% |
P1_LARGE | $0.12 per GB read | $0.07 per GB read | 42% |
P1_X_LARGE | $0.15 per GB read | $0.08 per GB read | 47% |
As we gain scale, we are committed to passing those savings to our customers.
Customers can now invite their team members to their Propel account. This feature enhances collaboration by allowing multiple users to access and work on the same account. Team members can share resources, manage Data Pools, and streamline their workflows within a single, unified account.
Log in and invite your teammates.
FROM
statement.ver
param for MergeTree tables, causing creation to fail.FINAL
clause was not being correctly added to the underlying ClickHouse queries, and _propel_is_deleted
filtering wasn’t working in some cases.We’re introducing Materialized Views in Propel’s Serverless ClickHouse as a powerful tool for data transformation. Developers can leverage Materialized Views to reshape, filter, or enrich data with SQL. Materialized Views are persistent query results that update dynamically as the original data changes.
The key benefit? Data is transformed in real time. No scheduling. No full-refreshes.
Learn more about Materialized Views
OpenAI has announced the acquisition of Rockset, and as a result, the Rockset service will cease to operate. For those unfamiliar with Rockset, it was a cloud-hosted real-time analytics database that enabled millisecond-latency queries for aggregations and joins, similar to Propel.
We are pleased to announce the immediate availability of the Rockset Migration Service. This service is designed to offer a seamless transition for companies from Rockset.
To get started with the migration process, please schedule a kick-off call with our team here.
We are thrilled to announce that Propel now supports customizable table engines and sorting keys for all Data Pools. What does this mean? Better query performance, more cost-efficient reads, and support for real-time updates and delete on any Data Pool type.
Table engines in Propel’s Serverless ClickHouse determine how tables store, process, read, and update their data.
The sorting key is a set of one or more columns that Propel uses to organize the rows within a table. It determines the order of the rows in the table and significantly impacts the query performance. If the rows are sorted well, Propel can efficiently skip over unneeded rows and thus optimize query performance.
This enhancement provides users with more flexibility and control over their data, allowing them to optimize their data pools for their specific use cases.
Learn more about the table engine and sorting key
timestamp
field in the Data Pool API.uniqueId
and tenantId
fields are now deprecated in the Webhook Connection Settings objectWe have significantly expanded our SQL function support, extending it to a broad range of functions for PostgreSQL and ClickHouse SQL dialects, as well as unique Propel functions. This improvement offers developers greater flexibility and control when querying, transforming, and managing data.
Learn more about Propel SQL function support.
We’ve rolled out an updated Console navigation. The new menu structure and design organizes the Console into two primary sections: “Data” and “API”. The “Data” section houses all Serverless ClickHouse-related functionalities, and “API” contains all API-related functionalities.
Log in to the Console to see the new navigation.
We introduced a GraphQL Schema Explorer in the Console. Developers can now actively search through the Propel API GraphQL schema, access API endpoints with ease, and directly download the schema from a provided URL.
Check out the new GraphQL Schema Explorer in the Console.
filters
, you can now provide filterSql
. For example, see the filterSql
parameter in the counter API.filterSql
parameter that supports SQL-style filters.The ClickHouse Data Pool enables you to read through to your self-hosted ClickHouse or ClickHouse Cloud rather than syncing data to Propel. This allows you to utilize the data in your analytic dashboards, reports, and workflows directly from your ClickHouse instance through the Propel APIs and UI components.
Learn more about the ClickHouse Data Pool.
The GraphQL Playground enables you to run GraphQL queries directly from the Console, offering a simple way to interact with your data when building applications.
Key Features:
Log in to the Console and click “Playground”, then select “API: GraphQL”.
_propel_synced_at
that was incorrectly set for some Webhook Data Pools, resulting in out-of-range values.The Webhook Data Pool now ingests events 10x faster. We have optimized ingestion so that data is available within single digit seconds.
Learn more about the Webhook Data Pool.
With Propel’s new SQL Playground, you can now execute SQL queries directly from the Console. It provides you with an easy way to explore your data when building applications.
Key Features:
Log in to the Console and click “Playground”, then select “API: SQL”.
The Airbyte destination lets you synchronize data from over 350+ sources to Propel’s Serverless ClickHouse infrastructure. It provides an easy way to power your customer-facing analytics and data applications with data from any SaaS application, database, or platform supported by Airbyte.
Learn more about the Airbyte destination.
disable_partial_success=true
query parameter, you can ensure that, if any individual event in a batch of events fails validation, the entire request will fail. For example: https://webhooks.us-east-2.propeldata.com/v1/WHK00000000000000000000000000?disable_partial_success=true
The new Kafka Data Pool lets you ingest real-time streaming data into Propel. It provides an easy way to power real-time dashboards, streaming analytics, and workflows with a low-latency data API on top of your Kafka topics.
Learn more about the Kafka Data Pool.
We are introducing a new, generous free tier! It includes up to $15 of usage per month, and the best part is, it does not expire.
Sign up and get started today.
We are introducing Schema Evolution for Data Pools with the ability to add new columns to your Data Pools. Now, you can add new columns to your Data Pools, allowing you to evolve your data schema as your needs grow and change.
Learn more about the Add column to Data Pool operation.
The new batch delete operation helps you stay GDPR compliant by providing a straightforward way to permanently delete data from a Data Pool. Meanwhile, the batch update operation helps maintain data integrity and facilitates data backfilling in the event of schema changes. Both operations can be done via the Console or API.
Learn more about batch updates and deletes.
The Propel UI Kit now features logging capabilities for faster development and clean logging in production. By default, all errors are logged to the browser’s console. This behavior can be customized using the LogProvider component. The LogProvider uses React’s context mechanism to propagate log settings to nested components, allowing for specific component logging. Available log levels include “error”, “warn”, “info”, or “debug”.
Learn more about the React UI Kit’s logging controls.
The Fivetran destination lets you synchronize data from over 400 sources to Propel’s Serverless ClickHouse infrastructure.
Learn more about our Fivetran destination.
The ClickHouse Data Pool reads through to your self-hosted ClickHouse or ClickHouse Cloud rather than syncing data to Propel.
Learn more about the ClickHouse “read-through” Data Pool.
toStartOfWeek
, toStartOfMonth
, and toStartOfYear
SQL functions.NOW()
and CURRENT_DATE
functions in SQL.INTERVAL
in SQL.timestamp
can be supplied to TimeRangeInput
when querying.engine
, partitionBy
, primaryKey
, and orderBy
) for their Data Pools via the API.timestamp
via the API.engine
, partitionBy
, primaryKey
, and orderBy
) when creating a Data Pool via the API.processedRecords
instead of newRecords
in the Processed Records column for the Syncs table.You can now query any Data Pool using SQL over the GraphQL API. Need to join, group by, or perform complex queries? No problem. Propel’s SQL supports PostgreSQL syntax, including joins, unions, and common table expressions for more complex queries. The SQL API allows you to query your data however you’d like, and Propel’s multi-tenant access policies ensure that customers can only query their own data.
You can now connect any BI tool or PostgreSQL client to Propel. Essentially, Propel mimics a PostgreSQL instance, providing a seamless connection to a variety of tools or client applications.
For SaaS applications, this simplifies the process of providing a customer-facing SQL interface for custom reporting and data sharing.
Learn more about the SQL interface.
The new Data Grid API efficiently retrieves individual records from a Data Pool, with the added convenience of built-in pagination, filtering, and sorting. It’s perfect for displaying data in a table format, making it ideal for data tables with individual events, orders, requests, or log messages.
Learn more about the Data Grid API
The new Records by ID API is optimized for quick, unique ID lookups. It returns the records corresponding to the given IDs. This API can present detailed record information in a data table or record detail page.
Learn more about the Records by ID API.
The new Top Values API returns the most common values in a specified column ordered by frequency. The Top Values API can populate UI filters, prompt available values to AI agents, or showcase trending values within a column.
You can now control the look and feel of all your UI components in one theme. The theme of the UI Kit determines all essential visual elements, including the colors of components, the depth of shadows, and the overall light or dark appearance of the interface. We provide light and dark themes out of the box and the ability to customize your own theme.
Learn more about themes in the UI Kit.
You can now easily fetch and refresh API access tokens from the frontend. The new AccessTokenProvider
component allows you to provide a function that fetches an access token from your backend. Using this function, the provider will serve the fetched access token to all its child components and automatically refresh the token when it expires.
Learn more about the Access Token Provider.
The new Filter component simplifies the process of adding filters to your dashboards. It uses Propel’s Top Values API to fill the dropdown list with unique values from a specific column, arranged by their frequency.
Learn more about the filter component.
The PieChart component is designed to create pie or doughnut charts using the Leaderboard API.
Learn more about the Pie Chart component.
Propel’s UI Kit provides prebuilt React components for querying data from Propel’s GraphQL API. These components can be used to query data for custom visualizations or to build with third-party libraries such as D3.js, Recharts, Nivo, or Chart.js.
Learn more about the Query Hooks.
toStartOfWeek
, toStartOfMonth
, and toStartOfYear
SQL functions.data_pool:read
scope to list Data Pools and their schemas.timeRange
optional in GraphQL API.New features, fixes, and improvements.
Ingest data from Amazon Firehose streams into Propel’s Serverless ClickHouse with zero configuration.
A customizable React component for selecting dashboard time ranges, with seamless Propel Query API integration.
Stream Segment events directly to Propel for real-time analytics.
The Data Grid is a React component for easily visualizing data in a table format.
Features you no longer need to build:
DynamoDB excels as a transactional database, but it’s not ideal for analytics involving aggregations and GROUP BY queries.
That’s where Propel’s DynamoDB connector shines. It allows you to replicate DynamoDB data in real time to Propel’s Serverless ClickHouse, enabling advanced analytics that are both fast and cost-effective.
Key features:
Added support for SHOW commands including:
SHOW DATABASES
SHOW TABLES
SHOW COLUMNS
SHOW INDEXES
SHOW CREATE DATABASE
SHOW CREATE TABLE
SHOW FUNCTIONS
SHOW ENGINES
Added support for system tables including:
system.databases
system.tables
system.columns
system.aggregate_function_combinators
system.formats
system.functions
system.licenses
system.settings
system.settings_changed
system.table_engines
system.table_functions
system.time_zones
Improved PopSQL compatibility:
date_time_output_format = 'simple'
when Faraday user agent is detectedEnhanced timezone handling:
Fixed bug where tables were added without “propel” database qualifier
Changed currentDatabase()
to return “propel”
Added support for database query parameter and x-clickhouse-database header
Added support for wait_end_of_query
parameter used by Python clients
Improved authentication handling for missing X-ClickHouse-Key header
Added support for client_protocol_version in ClickHouse HTTP endpoint
We’ve launched a native ClickHouse SQL HTTP interface that lets you query Propel’s Serverless ClickHouse directly. You can use any standard ClickHouse library or SDK to run SELECT
and SHOW
queries against your data.
To get started, use the following connection details:
Parameter | Value |
---|---|
Host | https://clickhouse.us-east-2.propeldata.com |
Port | 8443 |
Database | propel |
user | Your Propel Application ID |
password | Your Propel Application secret |
Here’s an example of querying data using the ClickHouse HTTP interface with curl:
Learn more about the ClickHouse SQL interface.
We’ve completely revamped our documentation to make it easier for developers to get started and be successful with Propel.
The new docs include the following sections:
Serverless ClickHouse: Connection details, supported SQL syntax, SQL reference, functions, and everything you need to know to manage, transform and query your data.
Ingestion: Guides on ingesting data from a variety of sources to Propel, including Webhooks, events streams, and data warehouses.
Query APIs: Guides and examples for using the SQL and Query APIs for common analytics use cases and data visualizations.
Embeddable UIs: Guides and documentation for embedding Propel’s visualization components into your applications.
Explore the new docs.
You can now authenticate server-side applications using HTTP Basic Authentication with your Application ID and secret.
Learn more about server-side authentication.
The <GroupBy>
component is a dropdown menu that allows users to dynamically change the grouping of data for multidimensional analysis.
Learn more about the <GroupBy>
component.
X-ClickHouse-User
and X-ClickHouse-Key
headers.PENDING
state.chartConfigProps
in the <TimeSeries>
component.groupBy
functionality to <TimeSeries>
and <FilterProvider>
components for dynamic data grouping.<Leaderboard>
component to properly sync with time range from <FilterProvider>
.legendPosition
prop in <PieChart>
component.accentColors
prop to <Leaderboard>
component for visual styling.DATA_POOL_READ
, DATA_POOL_STATS
, METRIC_READ
, and METRIC_STATS
scopes will no longer be able to set these scopes as they are now deprecatedDATA_POOL_READ
, DATA_POOL_STATS
, METRIC_READ
, and METRIC_STATS
API scopes.DATA_POOL_QUERY
, METRIC_QUERY
API scopes to allow seeing the schema for the Data Pools and columns they have access to.AND
and OR
operators=
to include >
, <
, LIKE
, and IN
among others.position
, locate
, positionCaseInsensitive
, positionUTF8
, positionCaseInsensitiveUTF8
- multiSearch functions for various use cases (e.g., AllPositions
, FirstPosition
, FirstIndex
)match
, REGEXP
, multiMatchAny
, multiMatchAnyIndex
, multiMatchAllIndices
- Fuzzy matching: multiFuzzyMatchAny
, multiFuzzyMatchAnyIndex
, multiFuzzyMatchAllIndices
extract
, extractAll
, extractAllGroupsHorizontal
, extractAllGroupsVertical
- like
, notLike
, ilike
, notILike
functionsngramDistance
, ngramSearch
(with case-sensitive and UTF8 variants)- countSubstrings
, countMatches
(with case-insensitive options)regexpExtract
, hasSubsequence
, hasToken
(with various options for case sensitivity and UTF8)<Container>
, <Flex>
, <Grid>
, and <Card>
.<Text>
and <Heading>
.<Tabs>
component for tab-based layouts with support for Card components as tabs.gray
) from "@propeldata/ui-kit/colors"
for consistent styling across the application.<TimeRangePicker>
now integrates seamlessly with <FilterProvider>
for improved data filtering across components.<TimeGrainPicker>
component that also integrated seamlessly with the <FilterProvider>
.groupBy
functionality to the <TimeSeries>
component, enabling data grouping and more flexible visualizations.<SimpleFilter>
component with clearable functionality. Developers can use the disableClearable
prop to turn off this feature if needed.accentColor
with accentColors
to provide more versatile theming options.partition_by
, order_by
, and primary_key
fields. Users can now use the syntax [""]
to define these fields as empty when needed. This improvement allows for more precise control over Data Pool settings.NOT_FOUND
error when the requested resource doesn’t exist. This applies to Materialized Views, Copy Jobs, and Data Pool access policies, improving error handling and user experience.<SimpleFilter>
component styles with select-based components like the <TimeRangePicker>
for a more uniform user interface.order_by
, partition_by
, and primary_key
columns were being applied in an incorrect order when defining Table Settings during Data Pool creation. We replaced Set
with List
to ensure order is preserved in fields where it’s critical.arrayJoin
: Allows for the expansion of arrays into separate rows.JSONExtractArrayRaw
: Extracts an array from a JSON string.JSONExtractKeys
: Retrieves keys from a JSON object.JSONArrayLength
: Determines the length of a JSON array.tupleNames
: Returns the names of tuple elements.tupleElement
: Extracts a specific element from a tuple.Our most powerful Propellers, the P1_LARGE and P1_X_LARGE, which can read 250 and 500 million rows per second respectively, now have a significantly lower price.
Propeller | Old price | New price | % Price drop |
---|---|---|---|
P1_MEDIUM | $0.10 per GB read | $0.06 per GB read | 40% |
P1_LARGE | $0.12 per GB read | $0.07 per GB read | 42% |
P1_X_LARGE | $0.15 per GB read | $0.08 per GB read | 47% |
As we gain scale, we are committed to passing those savings to our customers.
Customers can now invite their team members to their Propel account. This feature enhances collaboration by allowing multiple users to access and work on the same account. Team members can share resources, manage Data Pools, and streamline their workflows within a single, unified account.
Log in and invite your teammates.
FROM
statement.ver
param for MergeTree tables, causing creation to fail.FINAL
clause was not being correctly added to the underlying ClickHouse queries, and _propel_is_deleted
filtering wasn’t working in some cases.We’re introducing Materialized Views in Propel’s Serverless ClickHouse as a powerful tool for data transformation. Developers can leverage Materialized Views to reshape, filter, or enrich data with SQL. Materialized Views are persistent query results that update dynamically as the original data changes.
The key benefit? Data is transformed in real time. No scheduling. No full-refreshes.
Learn more about Materialized Views
OpenAI has announced the acquisition of Rockset, and as a result, the Rockset service will cease to operate. For those unfamiliar with Rockset, it was a cloud-hosted real-time analytics database that enabled millisecond-latency queries for aggregations and joins, similar to Propel.
We are pleased to announce the immediate availability of the Rockset Migration Service. This service is designed to offer a seamless transition for companies from Rockset.
To get started with the migration process, please schedule a kick-off call with our team here.
We are thrilled to announce that Propel now supports customizable table engines and sorting keys for all Data Pools. What does this mean? Better query performance, more cost-efficient reads, and support for real-time updates and delete on any Data Pool type.
Table engines in Propel’s Serverless ClickHouse determine how tables store, process, read, and update their data.
The sorting key is a set of one or more columns that Propel uses to organize the rows within a table. It determines the order of the rows in the table and significantly impacts the query performance. If the rows are sorted well, Propel can efficiently skip over unneeded rows and thus optimize query performance.
This enhancement provides users with more flexibility and control over their data, allowing them to optimize their data pools for their specific use cases.
Learn more about the table engine and sorting key
timestamp
field in the Data Pool API.uniqueId
and tenantId
fields are now deprecated in the Webhook Connection Settings objectWe have significantly expanded our SQL function support, extending it to a broad range of functions for PostgreSQL and ClickHouse SQL dialects, as well as unique Propel functions. This improvement offers developers greater flexibility and control when querying, transforming, and managing data.
Learn more about Propel SQL function support.
We’ve rolled out an updated Console navigation. The new menu structure and design organizes the Console into two primary sections: “Data” and “API”. The “Data” section houses all Serverless ClickHouse-related functionalities, and “API” contains all API-related functionalities.
Log in to the Console to see the new navigation.
We introduced a GraphQL Schema Explorer in the Console. Developers can now actively search through the Propel API GraphQL schema, access API endpoints with ease, and directly download the schema from a provided URL.
Check out the new GraphQL Schema Explorer in the Console.
filters
, you can now provide filterSql
. For example, see the filterSql
parameter in the counter API.filterSql
parameter that supports SQL-style filters.The ClickHouse Data Pool enables you to read through to your self-hosted ClickHouse or ClickHouse Cloud rather than syncing data to Propel. This allows you to utilize the data in your analytic dashboards, reports, and workflows directly from your ClickHouse instance through the Propel APIs and UI components.
Learn more about the ClickHouse Data Pool.
The GraphQL Playground enables you to run GraphQL queries directly from the Console, offering a simple way to interact with your data when building applications.
Key Features:
Log in to the Console and click “Playground”, then select “API: GraphQL”.
_propel_synced_at
that was incorrectly set for some Webhook Data Pools, resulting in out-of-range values.The Webhook Data Pool now ingests events 10x faster. We have optimized ingestion so that data is available within single digit seconds.
Learn more about the Webhook Data Pool.
With Propel’s new SQL Playground, you can now execute SQL queries directly from the Console. It provides you with an easy way to explore your data when building applications.
Key Features:
Log in to the Console and click “Playground”, then select “API: SQL”.
The Airbyte destination lets you synchronize data from over 350+ sources to Propel’s Serverless ClickHouse infrastructure. It provides an easy way to power your customer-facing analytics and data applications with data from any SaaS application, database, or platform supported by Airbyte.
Learn more about the Airbyte destination.
disable_partial_success=true
query parameter, you can ensure that, if any individual event in a batch of events fails validation, the entire request will fail. For example: https://webhooks.us-east-2.propeldata.com/v1/WHK00000000000000000000000000?disable_partial_success=true
The new Kafka Data Pool lets you ingest real-time streaming data into Propel. It provides an easy way to power real-time dashboards, streaming analytics, and workflows with a low-latency data API on top of your Kafka topics.
Learn more about the Kafka Data Pool.
We are introducing a new, generous free tier! It includes up to $15 of usage per month, and the best part is, it does not expire.
Sign up and get started today.
We are introducing Schema Evolution for Data Pools with the ability to add new columns to your Data Pools. Now, you can add new columns to your Data Pools, allowing you to evolve your data schema as your needs grow and change.
Learn more about the Add column to Data Pool operation.
The new batch delete operation helps you stay GDPR compliant by providing a straightforward way to permanently delete data from a Data Pool. Meanwhile, the batch update operation helps maintain data integrity and facilitates data backfilling in the event of schema changes. Both operations can be done via the Console or API.
Learn more about batch updates and deletes.
The Propel UI Kit now features logging capabilities for faster development and clean logging in production. By default, all errors are logged to the browser’s console. This behavior can be customized using the LogProvider component. The LogProvider uses React’s context mechanism to propagate log settings to nested components, allowing for specific component logging. Available log levels include “error”, “warn”, “info”, or “debug”.
Learn more about the React UI Kit’s logging controls.
The Fivetran destination lets you synchronize data from over 400 sources to Propel’s Serverless ClickHouse infrastructure.
Learn more about our Fivetran destination.
The ClickHouse Data Pool reads through to your self-hosted ClickHouse or ClickHouse Cloud rather than syncing data to Propel.
Learn more about the ClickHouse “read-through” Data Pool.
toStartOfWeek
, toStartOfMonth
, and toStartOfYear
SQL functions.NOW()
and CURRENT_DATE
functions in SQL.INTERVAL
in SQL.timestamp
can be supplied to TimeRangeInput
when querying.engine
, partitionBy
, primaryKey
, and orderBy
) for their Data Pools via the API.timestamp
via the API.engine
, partitionBy
, primaryKey
, and orderBy
) when creating a Data Pool via the API.processedRecords
instead of newRecords
in the Processed Records column for the Syncs table.You can now query any Data Pool using SQL over the GraphQL API. Need to join, group by, or perform complex queries? No problem. Propel’s SQL supports PostgreSQL syntax, including joins, unions, and common table expressions for more complex queries. The SQL API allows you to query your data however you’d like, and Propel’s multi-tenant access policies ensure that customers can only query their own data.
You can now connect any BI tool or PostgreSQL client to Propel. Essentially, Propel mimics a PostgreSQL instance, providing a seamless connection to a variety of tools or client applications.
For SaaS applications, this simplifies the process of providing a customer-facing SQL interface for custom reporting and data sharing.
Learn more about the SQL interface.
The new Data Grid API efficiently retrieves individual records from a Data Pool, with the added convenience of built-in pagination, filtering, and sorting. It’s perfect for displaying data in a table format, making it ideal for data tables with individual events, orders, requests, or log messages.
Learn more about the Data Grid API
The new Records by ID API is optimized for quick, unique ID lookups. It returns the records corresponding to the given IDs. This API can present detailed record information in a data table or record detail page.
Learn more about the Records by ID API.
The new Top Values API returns the most common values in a specified column ordered by frequency. The Top Values API can populate UI filters, prompt available values to AI agents, or showcase trending values within a column.
You can now control the look and feel of all your UI components in one theme. The theme of the UI Kit determines all essential visual elements, including the colors of components, the depth of shadows, and the overall light or dark appearance of the interface. We provide light and dark themes out of the box and the ability to customize your own theme.
Learn more about themes in the UI Kit.
You can now easily fetch and refresh API access tokens from the frontend. The new AccessTokenProvider
component allows you to provide a function that fetches an access token from your backend. Using this function, the provider will serve the fetched access token to all its child components and automatically refresh the token when it expires.
Learn more about the Access Token Provider.
The new Filter component simplifies the process of adding filters to your dashboards. It uses Propel’s Top Values API to fill the dropdown list with unique values from a specific column, arranged by their frequency.
Learn more about the filter component.
The PieChart component is designed to create pie or doughnut charts using the Leaderboard API.
Learn more about the Pie Chart component.
Propel’s UI Kit provides prebuilt React components for querying data from Propel’s GraphQL API. These components can be used to query data for custom visualizations or to build with third-party libraries such as D3.js, Recharts, Nivo, or Chart.js.
Learn more about the Query Hooks.
toStartOfWeek
, toStartOfMonth
, and toStartOfYear
SQL functions.data_pool:read
scope to list Data Pools and their schemas.timeRange
optional in GraphQL API.