Apache Superset

marmotdata/superset

Discover dashboards, charts, datasets and database connections from Apache Superset

Dashboards Assets Lineage Experimental

The Superset plugin discovers dashboards, charts, datasets and database connections from Apache Superset through its REST API.

Dashboards contain their charts, datasets feed the charts that read them, and a dataset is linked to the table it reads in the database Superset connects to, using the identity that database's own Marmot plugin gives the table. Virtual datasets are linked to the tables named after FROM and JOIN in their SQL.

Required Permissions

Log in with a user that can read dashboards, charts, datasets and databases. The built-in Gamma role is enough for dashboards, charts and datasets; reading database connection details (GET /api/v1/database/{id}/connection) needs can_read on Database, which Alpha and Admin have.

Table lineage

A dataset is linked to a table only for backends Marmot has a naming rule for: PostgreSQL, MySQL, MariaDB, BigQuery, ClickHouse, SQLite, DuckDB, Athena (as Glue), Snowflake, Redshift, SQL Server, CockroachDB, Oracle, Druid, Trino and Databricks. The edge appears in Marmot once that database has been ingested with its own plugin.

In the UI

Point-and-click, no config file needed.

1 Open Runs Create pipeline
2 Pick Apache Superset from the plugin list.
3 Fill in the wizard, set a schedule, save.

Read the UI guide

With the CLI

Save a YAML config, then run marmot ingest.

ingest.yaml yaml

name: my-superset-pipeline
runs:
  - superset:
      host: "<host>"
      username: "<username>"
      password: "***"

$ marmot ingest -c ingest.yaml

Read the CLI guide

Not using plugins? Other ways to populate Marmot

Terraform Declare assets as infrastructure-as-code. Pulumi Native TypeScript, Python, or Go resources. Kubernetes Operator Sync assets from Kubernetes custom resources.

Configuration

14 top-level fields. * marks required fields.

tags multiselect

Tags to apply to discovered assets

external_links object[]

External links to show on all assets

name string

Display name for the link

icon string

Icon identifier for the link

url string

URL to the external resource

filter object

Filter discovered assets by name (regex)

include multiselect

Include patterns for resource names (regex)

exclude multiselect

Exclude patterns for resource names (regex)

host string

Superset URL (e.g. https://superset.example.com)

username string

Username to log in with

password password

Password to log in with

provider select

Authentication provider (db or ldap)

default: db

db ldap

verify_ssl bool

Verify the server's TLS certificate

default: true

include_charts bool

Discover charts

default: true

include_datasets bool

Discover datasets

default: true

include_databases bool

Discover database connections

default: true

include_draft bool

Include unpublished (draft) dashboards

default: true

discover_lineage bool

Link dashboards, charts, datasets and the tables they read

default: true

page_size int

Objects per API page

default: 100

Assets emitted

Metadata this plugin attaches to each discovered asset.

Dashboard

SupersetDashboardFields

9 fields

The metadata fields the plugin emits for Dashboard assets.

id int

Dashboard id in Superset

slug string

URL slug

url string

Link to the dashboard

published bool

Whether the dashboard is published

status string

Dashboard status (published, draft)

owners string[]

Names of the owners

tags string[]

Tags applied in Superset

changed_on string

Last change time (UTC)

chart_count int

Number of charts on the dashboard

Chart

SupersetChartFields

10 fields

The metadata fields the plugin emits for Chart assets.

id int

Chart id in Superset

viz_type string

Superset visualisation type

chart_type string

Normalised chart type (Table, Line, Bar, Pie, ...)

datasource_id int

Id of the dataset the chart reads

datasource_type string

Kind of datasource (table)

dataset string

Dataset the chart reads (schema.table)

url string

Link to the chart

owners string[]

Names of the owners

dashboards string[]

Titles of the dashboards the chart is on

changed_on string

Last change time (UTC)

Dataset

SupersetDatasetFields

11 fields

The metadata fields the plugin emits for Data Model Object assets (Superset datasets).

id int

Dataset id in Superset

kind string

physical (a table) or virtual (a SQL query)

database string

Name of the Superset database connection

database_id int

Id of the database connection

backend string

SQLAlchemy dialect of the database (postgresql, snowflake, ...)

schema string

Schema the dataset lives in

table_name string

Table name, or the virtual dataset's name

url string

Link to the dataset

owners string[]

Names of the owners

description string

Dataset description

changed_on string

Last change time (UTC)

Column

SupersetColumnFields

4 fields

The per-column fields embedded in a dataset asset's schema.

column_name string

Column name

data_type string

Column data type

description string

Column description

expression string

SQL expression of a calculated column

Database

SupersetDatabaseFields

9 fields

The metadata fields the plugin emits for DataSource assets (Superset database connections).

id int

Database id in Superset

backend string

SQLAlchemy dialect (postgresql, snowflake, ...)

driver string

SQLAlchemy driver

host string

Database host

port string

Database port

database string

Database or catalog the connection opens

sqlalchemy_uri string

Connection URI with the password masked

expose_in_sqllab bool

Whether the database is available in SQL Lab

allow_dml bool

Whether SQL Lab may run DML against it