Creating tables

Propel’s Serverless ClickHouse supports creating tables (Data Pools) via the API and Terraform.

Data Pools in Propel are ClickHouse tables with a pipeline to ingest data from supported data sources. With the API, you can create an empty Data Pool that creates table in Propel’s Serverless ClickHouse.

See the Create Data Pool API reference documentation.

API Arguments

The table below describes the input parameters for the createDataPoolV2 API. These parameters allow you to customize your table, including specifying the schema, table engine, ordering, and access control.

Argument	Description	Required
uniqueName	The Data Pool's unique name. If not specified, Propel will set the ID as the unique name.	No
description	The Data Pool's description.	No
timestamp	The table's default timestamp column. Used as the default timestamp for the query APIs.	No
columns	The list of column names and their types.	Yes
accessControlEnabled	Enables or disables access control for the Data Pool. If the Data Pool has access control enabled, Applications must be assigned Data Pool Access Policies in order to query the Data Pool and its Metrics.	No
tableSettingsInput	The tableSettingsInput object define how the Data Pool's table is created in ClickHouse. See table settings below.	No

Table settings

The table settings define how the table is created in ClickHouse. To learn more about table engines, read our guide on How to select a table engine and sorting key.

The tableSettingsInput object has the following properties:

Field	Type	Required	Description
engine	TableEngineInput	No	The ClickHouse table engine for the Data Pool's table. This field is optional. A default will be chosen based on the Data Pool's timestamp and uniqueId values, if specified.
partitionBy	array of String	No	The PARTITION BY clause for the Data Pool's table. This field is optional. A default will be chosen based on the Data Pool's timestamp and uniqueId values, if specified.
primaryKey	array of String	No	The PRIMARY KEY clause for the Data Pool's table. This field is optional. A default will be chosen based on the Data Pool's timestamp and uniqueId values, if specified.
orderBy	array of String	No	The ORDER BY clause for the Data Pool's table. This field is optional. A default will be chosen based on the Data Pool's timestamp and uniqueId values, if specified.

Example: Create a MergeTree table

This example shows how to create a Data Pool using the API and Terraform.

API

mutation {
  createDataPoolV2(
    input: {
      uniqueName: "New TacoSoft Demo Data"
      description: "A sample dataset consisting of orders for a taco ordering SaaS"
      timestamp: { columnName: "created_at" }
      columns: [
        { columnName: "quantity", type: INT32, isNullable: false },
        { columnName: "taco_name", type: STRING, isNullable: false },
        { columnName: "sauce_name", type: STRING, isNullable: false },
        { columnName: "restaurant_id", type: STRING, isNullable: false },
        { columnName: "restaurant_name", type: STRING, isNullable: false },
        { columnName: "taco_total_price", type: FLOAT, isNullable: false },
        { columnName: "order_item_id", type: STRING, isNullable: false },
        { columnName: "tortilla_id", type: STRING, isNullable: false },
        { columnName: "toppings", type: JSON, isNullable: false },
        { columnName: "sauce_id", type: STRING, isNullable: false },
        { columnName: "taco_unit_price", type: FLOAT, isNullable: false },
        { columnName: "order_id", type: STRING, isNullable: false },
        { columnName: "order_item_generated_at", type: TIMESTAMP, isNullable: false },
        { columnName: "taco_id", type: STRING, isNullable: false },
        { columnName: "timestamp", type: TIMESTAMP, isNullable: false },
        { columnName: "tortilla_name", type: STRING, isNullable: false }
      ]
      accessControlEnabled: true
      tableSettings: {
        engine: {
          mergeTree: {
            type: MERGE_TREE
          }
        }
        orderBy: ["timestamp"]
      }
    }
  ) {
    dataPool {
      id
      uniqueName
      accessControlEnabled
      description
      tableSettings {
        orderBy
      }
    }
  }
}

Terraform

resource "propel_data_pool" "tacosoft_demo_data" {
  unique_name = "New TacoSoft Demo Data"
  description = "A sample dataset consisting of orders for a taco ordering SaaS"
  timestamp   = "created_at"

  access_control_enabled = true

  table_settings {
    engine {
      type = "MERGE_TREE"
    }
    order_by = ["timestamp"]
  }

  column {
    name     = "quantity"
    type     = "INT32"
    nullable = false
  }
  column {
    name     = "taco_name"
    type     = "STRING"
    nullable = false
  }
  column {
    name     = "sauce_name"
    type     = "STRING"
    nullable = false
  }
  column {
    name     = "restaurant_id"
    type     = "STRING"
    nullable = false
  }
  column {
    name     = "restaurant_name"
    type     = "STRING"
    nullable = false
  }
  column {
    name     = "taco_total_price"
    type     = "FLOAT"
    nullable = false
  }
  column {
    name     = "order_item_id"
    type     = "STRING"
    nullable = false
  }
  column {
    name     = "tortilla_id"
    type     = "STRING"
    nullable = false
  }
  column {
    name     = "toppings"
    type     = "JSON"
    nullable = false
  }
  column {
    name     = "sauce_id"
    type     = "STRING"
    nullable = false
  }
  column {
    name     = "taco_unit_price"
    type     = "FLOAT"
    nullable = false
  }
  column {
    name     = "order_id"
    type     = "STRING"
    nullable = false
  }
  column {
    name     = "order_item_generated_at"
    type     = "TIMESTAMP"
    nullable = false
  }
  column {
    name     = "taco_id"
    type     = "STRING"
    nullable = false
  }
  column {
    name     = "timestamp"
    type     = "TIMESTAMP"
    nullable = false
  }
  column {
    name     = "tortilla_name"
    type     = "STRING"
    nullable = false
  }
}

---