QuestDB
Official · maintained by Marmotmarmotdata/questdb Discover tables, views and materialized views from QuestDB instances
The QuestDB plugin discovers tables, views and materialized views from QuestDB instances. It captures the designated timestamp, partitioning, WAL, deduplication and TTL settings of each table, QuestDB column types verbatim, row counts, sizes on disk and partition counts, and links views to the tables they read.
It connects over the PostgreSQL wire protocol (port 8812) with github.com/jackc/pgx/v5 and reads QuestDB's own table functions (tables(), table_columns(), table_partitions(), views(), materialized_views()), since QuestDB's pg_catalog is minimal. QuestDB has a single database and no schemas, so no Database asset is created and objects are addressed by their bare name.
Required Permissions
QuestDB open source has a single login, admin with password quest by default, and no per-table permissions. On QuestDB Enterprise, use a user that can SELECT from the tables to discover; see QuestDB's access control documentation for the exact grants.
In the UI
Point-and-click, no config file needed.
- 1 Open Runs Create pipeline
- 2 Pick QuestDB from the plugin list.
- 3 Fill in the wizard, set a schedule, save.
With the CLI
Save a YAML config, then run marmot ingest.
name: my-questdb-pipeline
runs:
- questdb:
host: "<host>"$ marmot ingest -c ingest.yamlNot using plugins? Other ways to populate Marmot
Configuration
14 top-level fields. * marks required fields.
tags multiselect Tags to apply to discovered assets
external_links object[] External links to show on all assets
name string Display name for the link
icon string Icon identifier for the link
url string URL to the external resource
filter object Filter discovered assets by name (regex)
include multiselect Include patterns for resource names (regex)
exclude multiselect Exclude patterns for resource names (regex)
host string QuestDB server hostname or IP address
port int PostgreSQL wire protocol port
- default
- 8812
user string Username for authentication
- default
- admin
password password Password for authentication (QuestDB ships with quest)
database string Database name, QuestDB has only one
- default
- qdb
ssl_mode select SSL mode (disable, require)
- default
- disable
include_columns bool Whether to include column information in table metadata
- default
- true
include_views bool Whether to discover views
- default
- true
include_materialized_views bool Whether to discover materialized views
- default
- true
include_statistics bool Whether to include row counts, column counts and sizes
- default
- true
exclude_system_tables bool Whether to exclude QuestDB internal tables (sys.*, _*, telemetry)
- default
- true
Assets emitted
Metadata this plugin attaches to each discovered asset.
QuestDB
QuestDBFieldsThe metadata fields the QuestDB plugin emits for table and view assets.
host stringQuestDB server hostname
port intPostgreSQL wire protocol port
table_name stringTable or view name
object_type stringObject type (table, view, materialized_view)
designated_timestamp stringColumn the table is ordered and partitioned by
partition_by stringPartition interval (NONE, HOUR, DAY, WEEK, MONTH, YEAR)
partition_count intNumber of partitions on disk
wal_enabled boolWhether writes go through the write-ahead log
dedup boolWhether deduplication on upsert keys is enabled
ttl stringRetention period, for example 1 WEEK
max_uncommitted_rows intRows buffered before an out-of-order commit
o3_max_lag intOut-of-order commit lag in microseconds
materialized boolWhether a view is materialized
base_table stringTable a materialized view is built from
refresh_type stringMaterialized view refresh type (immediate, timer, manual, period)
refresh_period stringMaterialized view refresh interval, for example 1 HOUR
last_refresh stringWhen the materialized view last started refreshing
Column
QuestDBColumnFieldsThe per-column fields embedded in an asset's schema.
column_name stringColumn name
data_type stringQuestDB data type as reported, for example SYMBOL or GEOHASH(8c)
is_nullable boolAlways true, QuestDB columns are nullable
designated_timestamp boolWhether this is the designated timestamp column
indexed boolWhether the symbol column has a bitmap index
symbol_capacity intDistinct values a symbol column is sized for
symbol_cached boolWhether the symbol table is cached in memory
upsert_key boolWhether the column is a deduplication upsert key