Skip to main content
· 4 min read

September 2024

Improvements

Ingestion

  • Improved Webhook Data Pool performance: Increased request per second (RPS) and concurrent request capacity 4x, significantly reducing 429 responses.
  • The Webhook Data Pool now accepts payloads up to 1 MiB.

Console

  • Enhanced account security: Users can now set up Multi-Factor Authentication (MFA) for their accounts.
  • Improved user experience: The last selected Propeller for the Data Pool’s Preview Data will be remembered and displayed for future sessions.
  • Updated console for deprecated API Scope: Applications without prior access to DATA_POOL_READ, DATA_POOL_STATS, METRIC_READ, and METRIC_STATS scopes will no longer be able to set these scopes as they are now deprecated

API

  • Deprecated DATA_POOL_READ, DATA_POOL_STATS, METRIC_READ, and METRIC_STATS API scopes.
  • Expanded DATA_POOL_QUERY, METRIC_QUERY API scopes to allow seeing the schema for the Data Pools and columns they have access to.
  • Improved the resiliency of copy jobs for large tables, reducing errors and preventing jobs from getting stuck. Enhanced error messages to provide more informative feedback when jobs fail.
  • Propellers no longer have a limit on maximum bytes to read making them more powerful.
  • Materialize View creation now tests the SQL query prior to creating the Materialized View. This protects developers from creating Materialized Views with incorrect SQL. I handles the case when queries take a long time for large datasets gracefully.

SQL

  • Enhanced JOIN clause functionality:
    • Added support for multiple column expressions using AND and OR operators
    • Expanded operator support beyond = to include >, <, LIKE, and IN among others.
  • Increased query result size limit from 500KB to 10MB, allowing for larger data retrieval
  • New ClickHouse functions for enhanced string manipulation and searching:
    • Position and Search functions like:- position, locate, positionCaseInsensitive, positionUTF8, positionCaseInsensitiveUTF8- multiSearch functions for various use cases (e.g., AllPositions, FirstPosition, FirstIndex)
    • Pattern Matching:- match, REGEXP, multiMatchAny, multiMatchAnyIndex, multiMatchAllIndices- Fuzzy matching: multiFuzzyMatchAny, multiFuzzyMatchAnyIndex, multiFuzzyMatchAllIndices
    • Extraction and Comparison:- extract, extractAll, extractAllGroupsHorizontal, extractAllGroupsVertical- like, notLike, ilike, notILike functions
    • N-gram and Substring Operations:- ngramDistance, ngramSearch (with case-sensitive and UTF8 variants)- countSubstrings, countMatches (with case-insensitive options)
    • Specialized String Functions:- regexpExtract, hasSubsequence, hasToken (with various options for case sensitivity and UTF8)

Embeddable UI (0.11.1)

  • New layout components: <Container>, <Flex>, <Grid> , and <Card>.
  • New typography components: <Text> and <Heading> .
  • New <Tabs> component for tab-based layouts with support for Card components as tabs.
  • Added importable theme colors: Users can now import specific colors (e.g., gray) from "@propeldata/ui-kit/colors" for consistent styling across the application.
  • <TimeRangePicker> now integrates seamlessly with <FilterProvider> for improved data filtering across components.
  • Added new <TimeGrainPicker> component that also integrated seamlessly with the <FilterProvider> .
  • Improved React compatibility: All components are now compatible with React Server Components (RSC) and exported as client-side components, enhancing performance and flexibility.
  • Enhanced debugging capabilities: Components now log prop mismatch errors, facilitating easier troubleshooting and development.
  • Added groupBy functionality to the <TimeSeries> component, enabling data grouping and more flexible visualizations.
  • Enhanced the <SimpleFilter> component with clearable functionality. Developers can use the disableClearable prop to turn off this feature if needed.
  • Improved color customization: Replaced accentColor with accentColors to provide more versatile theming options.

Terraform (v1.3.4)

  • Enhanced flexibility in Data Pool configuration: Added support for explicitly setting empty values for partition_by, order_by, and primary_key fields. Users can now use the syntax [""] to define these fields as empty when needed. This improvement allows for more precise control over Data Pool settings.

Fixes

Console

  • Resolved issues with Google sign-in for accepting invitations.
  • Fixed a bug that prevented sign-ups for users who had previously registered but not verified their email.
  • Fixed a bug that caused the SQL Console to attempt to send a Propeller for Applications resulting in an error.

API

  • The API now returns a NOT_FOUND error when the requested resource doesn't exist. This applies to Materialized Views, Copy Jobs, and Data Pool access policies, improving error handling and user experience.

Embeddable UI

  • Fixed inconsistent border radius across components for improved visual coherence.
  • Aligned <SimpleFilter> component styles with select-based components like the <TimeRangePicker> for a more uniform user interface.

Terraform (v1.3.4)

  • Fixed an issue where order_by, partition_by, and primary_key columns were being applied in an incorrect order when defining Table Settings during Data Pool creation. We replaced Set with List to ensure order is preserved in fields where it's critical.