dbt

marmotdata/dbt

Ingest metadata from dbt (Data Build Tool) projects including models, tests, and lineage

Category transformation

Status experimental

Features

AssetsLineage

The DBT plugin ingests metadata from dbt (Data Build Tool) projects, including models, sources, seeds, and lineage relationships. It reads dbt's generated artifacts to understand your data transformation layer and how it connects to your warehouse.

Prerequisites

Before Marmot can ingest your dbt project, you need to generate the artifact files in your project's target/ directory.

Required

Generate manifest.json by running:

dbt compile

Recommended

Generate catalog.json for column types and statistics:

dbt docs generate

File Sources

The target_path field accepts local paths, S3 URIs (s3://bucket/prefix) or Git URIs (git::https://...). For S3 and Git sources, the target directory is downloaded to a temporary location before discovery and cleaned up afterwards.

See File Sources for the full list of supported backends, authentication options and configuration examples.

In the UI

Point-and-click, no config file needed.

1 Open Runs → Create Pipeline
2 Pick dbt from the plugin list.
3 Fill in the wizard, set a schedule, save.

Read the full UI guide

With the CLI

Save a YAML config, then run marmot ingest.

ingest.yaml

name: my-dbt-pipeline
runs:
  - dbt:
      target_path: "<target_path>"
      project_name: "<project_name>"

$ marmot ingest -c ingest.yaml

Read the full CLI guide

Not using plugins? Other ways to populate Marmot

Terraform

Declare assets as infrastructure-as-code.

Pulumi

Native TypeScript, Python, or Go resources.

Kubernetes Operator

Sync assets from Kubernetes custom resources.

Configuration

15 top-level fields.

tags multiselect

Tags to apply to discovered assets

external_links object[]

External links to show on all assets

name string

Display name for the link

icon string

Icon identifier for the link

url string

URL to the external resource

filter object

Filter discovered assets by name (regex)

include multiselect

Include patterns for resource names (regex)

exclude multiselect

Exclude patterns for resource names (regex)

s3_source object

S3 file source configuration

shown when source_type = s3

credentials object

AWS credentials for S3 access

use_default bool

Use AWS credentials from environment or default profile (recommended)

default true

id string

AWS access key ID

secret password

AWS secret access key

token password

AWS session token

profile string

AWS profile to use from shared credentials file

role string

AWS IAM role ARN to assume

role_external_id string

External ID for cross-account role assumption

region string

AWS region for services

endpoint string

Custom endpoint URL for AWS services

git_source object

Git repository file source configuration

shown when source_type = git

url string

Git repository URL

ref string

Branch, tag or commit to check out

default main

path string

Subdirectory within the repository

token password

Personal access token for HTTPS auth

ssh_key_path string

Path to SSH private key for SSH auth

target_path string

Path to DBT target directory containing manifest.json, catalog.json, etc. (local path, s3://bucket/prefix or git::url)

project_name string

DBT project name

environment string

Environment name (e.g., production, staging)

default production

include_manifest bool

Include manifest.json for model definitions

default true

include_catalog bool

Include catalog.json for table/column descriptions

default true

include_run_results bool

Include run_results.json for test results

default false

include_sources_json bool

Include sources.json for source definitions

default false

discover_models bool

Discover DBT models

default true

discover_sources bool

Discover DBT sources

default true

discover_tests bool

Discover DBT tests

default false

Assets Emitted

Metadata fields this plugin attaches to each discovered asset.

DBT Model

DBTModelFields

22 fields

DBTModelFields represents DBT model-specific metadata fields

dbt_unique_id string

DBT's unique identifier for this node

dbt_package string

DBT package name

dbt_path string

Path to the model file

dbt_original_path string

Original path in the DBT project

dbt_materialized string

Materialization type (table, view, incremental, ephemeral)

database string

Target database name

schema string

Target schema name

model_name string

DBT model name

table_name string

Physical table/view name in database

fully_qualified_name string

Fully qualified name (database.schema.table)

project_name string

DBT project name

environment string

Deployment environment (dev, prod, etc)

alias string

Table alias if different from model name

adapter_type string

Database adapter type (postgres, snowflake, bigquery, etc)

dbt_version string

DBT version used to generate this model

last_run_status string

Status of the last DBT run (success, error, skipped)

last_run_execution_time float

Execution time of last run in seconds

last_run_message string

Message from last DBT run

last_run_failures int

Number of failures in last run

raw_sql string

Raw SQL before compilation

owner string

Table/view owner from database catalog

catalog_comment string

Comment from database catalog

DBT Source

DBTSourceFields

12 fields

DBTSourceFields represents DBT source-specific metadata fields

dbt_unique_id string

DBT's unique identifier for this source

dbt_package string

DBT package name

database string

Source database name

schema string

Source schema name

table_name string

Source table name

fully_qualified_name string

Fully qualified name (database.schema.table)

source_name string

DBT source name

identifier string

Physical table identifier

loaded bool

Whether source was loaded at time of DBT execution

project_name string

DBT project name

environment string

Deployment environment

freshness_checked bool

Whether freshness checks are configured

DBT Seed

DBTSeedFields

9 fields

DBTSeedFields represents DBT seed-specific metadata fields

dbt_unique_id string

DBT's unique identifier for this seed

dbt_package string

DBT package name

database string

Target database name

schema string

Target schema name

table_name string

Seed table name

fully_qualified_name string

Fully qualified name (database.schema.table)

seed_path string

Path to seed CSV file

project_name string

DBT project name

environment string

Deployment environment

DBT Column

DBTColumnFields

5 fields

DBTColumnFields represents DBT column-specific metadata fields

column_name string

Column name

data_type string

Column data type

column_description string

Column description from DBT

column_tags string[]

Tags applied to this column

column_comment string

Column comment from database catalog

DBT Config

DBTConfigFields

6 fields

DBTConfigFields represents DBT config-specific metadata fields Fields with config_ prefix contain DBT model configuration

config_enabled bool

Whether model is enabled

config_materialized string

Materialization strategy from config

config_tags string

Tags from config

config_persist_docs bool

Whether to persist documentation to database

config_full_refresh bool

Whether to perform full refresh

config_on_schema_change string

Behavior when schema changes (append_new_columns, fail, ignore)

DBT Stats

DBTStatsFields

6 fields

DBTStatsFields represents DBT catalog statistics fields Fields with stat_ prefix contain statistics from database catalog

stat_row_count int

Number of rows

stat_bytes int

Size in bytes

stat_last_modified string

Last modification timestamp

stat_num_rows int

Number of rows (alternative)

stat_approximate_count int

Approximate row count

stat_size float

Table size