PostgreSQL
Core pluginmarmotdata/postgresql
Discover databases, schemas, and tables from PostgreSQL instances
The PostgreSQL plugin discovers databases, schemas, and tables from PostgreSQL instances. It captures column information, table metrics, and foreign key relationships for lineage.
Required Permissions
The user needs read access to the information schema:
GRANT CONNECT ON DATABASE your_db TO marmot_reader;
GRANT USAGE ON SCHEMA public TO marmot_reader;
GRANT SELECT ON ALL TABLES IN SCHEMA public TO marmot_reader;In the UI
Point-and-click, no config file needed.
- 1 Open
Runs→Create Pipeline - 2 Pick PostgreSQL from the plugin list.
- 3 Fill in the wizard, set a schedule, save.
With the CLI
Save a YAML config, then run marmot ingest.
name: my-postgresql-pipeline
runs:
- postgresql:
host: "<host>"
user: "<user>"marmot ingest -c ingest.yaml Not using plugins? Other ways to populate Marmot
Configuration
13 top-level fields.
tags multiselect Tags to apply to discovered assets
external_links object[] External links to show on all assets
name string Display name for the link
icon string Icon identifier for the link
url string URL to the external resource
filter object Filter discovered assets by name (regex)
include multiselect Include patterns for resource names (regex)
exclude multiselect Exclude patterns for resource names (regex)
host string PostgreSQL server hostname or IP address
port int PostgreSQL server port
5432 user string Username for authentication
password password Password for authentication
ssl_mode select SSL mode (disable, require, verify-ca, verify-full)
disable include_databases bool Whether to discover databases
true include_columns bool Whether to include column information in table metadata
true enable_metrics bool Whether to include table metrics
true discover_foreign_keys bool Whether to discover foreign key relationships
true exclude_system_schemas bool Whether to exclude system schemas (pg_*)
true Assets Emitted
Metadata fields this plugin attaches to each discovered asset.
Postgres
PostgresFieldsPostgresFields represents PostgreSQL-specific metadata fields
host stringPostgreSQL server hostname
port intPostgreSQL server port
database stringDatabase name
schema stringSchema name
table_name stringObject name
object_type stringObject type (table, view, materialized_view)
owner stringObject owner
size intObject size in bytes
row_count intApproximate row count
created stringCreation timestamp
comment stringObject comment/description
encoding stringDatabase encoding
collate stringDatabase collation
ctype stringDatabase character classification
is_template boolWhether database is a template
allow_connections boolWhether connections to this database are allowed
connection_limit intMaximum allowed connections
Postgres Column
PostgresColumnFieldsPostgresColumnFields represents PostgreSQL column-specific metadata fields
column_name stringColumn name
data_type stringData type
is_nullable boolWhether null values are allowed
column_default stringDefault value expression
is_primary_key boolWhether column is part of primary key
comment stringColumn comment/description
Postgres Foreign Key
PostgresForeignKeyFieldsPostgresForeignKeyFields represents PostgreSQL foreign key relationship fields
constraint_name stringForeign key constraint name
source_schema stringSchema of the referencing table
source_table stringName of the referencing table
source_column stringColumn in the referencing table
target_schema stringSchema of the referenced table
target_table stringName of the referenced table
target_column stringColumn in the referenced table