DuckDB
Core pluginmarmotdata/duckdb
Discover schemas, tables, views and foreign key relationships from DuckDB database files
The DuckDB plugin discovers schemas, tables, views and foreign key relationships from DuckDB database files.
File Sources
The path field accepts local paths, S3 URIs (s3://bucket/key) or Git URIs (git::https://...). For S3 and Git sources, the file is downloaded to a temporary directory before discovery and cleaned up afterwards.
See File Sources for the full list of supported backends, authentication options and configuration examples.
In the UI
Point-and-click, no config file needed.
- 1 Open
Runs→Create Pipeline - 2 Pick DuckDB from the plugin list.
- 3 Fill in the wizard, set a schedule, save.
With the CLI
Save a YAML config, then run marmot ingest.
name: my-duckdb-pipeline
runs:
- duckdb:
path: "<path>"marmot ingest -c ingest.yaml Not using plugins? Other ways to populate Marmot
Configuration
10 top-level fields.
tags multiselect Tags to apply to discovered assets
external_links object[] External links to show on all assets
name string Display name for the link
icon string Icon identifier for the link
url string URL to the external resource
filter object Filter discovered assets by name (regex)
include multiselect Include patterns for resource names (regex)
exclude multiselect Exclude patterns for resource names (regex)
s3_source object S3 file source configuration
source_type = s3credentials object AWS credentials for S3 access
use_default bool Use AWS credentials from environment or default profile (recommended)
true id string AWS access key ID
secret password AWS secret access key
token password AWS session token
profile string AWS profile to use from shared credentials file
role string AWS IAM role ARN to assume
role_external_id string External ID for cross-account role assumption
region string AWS region for services
endpoint string Custom endpoint URL for AWS services
git_source object Git repository file source configuration
source_type = giturl string Git repository URL
ref string Branch, tag or commit to check out
main path string Subdirectory within the repository
token password Personal access token for HTTPS auth
ssh_key_path string Path to SSH private key for SSH auth
path string Path to the DuckDB database file (local path, s3://bucket/key or git::url)
include_columns bool Whether to include column information in table metadata
true enable_metrics bool Whether to include table metrics (row counts and sizes)
true discover_foreign_keys bool Whether to discover foreign key relationships
true exclude_system_schemas bool Whether to exclude system schemas (information_schema, pg_catalog)
true Assets Emitted
Metadata fields this plugin attaches to each discovered asset.
Duck DB
DuckDBFieldsDuckDBFields describes the metadata fields DuckDB emits for table and view assets. It is kept as a documentation-only struct so downstream tooling can introspect the shape of the metadata map.
path stringPath to the DuckDB database file
schema stringSchema name
table_name stringTable or view name
object_type stringObject type (BASE TABLE, VIEW)
row_count intEstimated row count
size intEstimated size in bytes
comment stringObject comment/description
Duck DB Column
DuckDBColumnFieldsDuckDBColumnFields describes the per-column fields embedded in an asset's schema.
column_name stringColumn name
data_type stringColumn data type
is_nullable boolWhether null values are allowed
column_default stringDefault value expression
Duck DB Foreign Key
DuckDBForeignKeyFieldsDuckDBForeignKeyFields describes the fields of a discovered foreign key relationship.
constraint_name stringForeign key constraint name
source_schema stringSchema of the referencing table
source_table stringName of the referencing table
source_column stringColumn in the referencing table
target_schema stringSchema of the referenced table
target_table stringName of the referenced table
target_column stringColumn in the referenced table