Apache Hive
Official · maintained by Marmotmarmotdata/hive Discover databases, tables and views from Apache Hive through HiveServer2
The Hive plugin discovers databases, tables, views and materialized views from Apache Hive through HiveServer2. It captures columns with their Hive types and comments, partition and bucket layout, storage formats, table statistics, view queries and foreign key constraints for lineage.
It speaks the HiveServer2 Thrift protocol with the pure-Go github.com/beltran/gohive driver over the binary or HTTP transport, with NONE, NOSASL, LDAP or KERBEROS authentication and optional TLS.
Required Permissions
The user needs SELECT on the databases to discover. Discovery runs SHOW DATABASES, DESCRIBE DATABASE EXTENDED, SHOW TABLES, SHOW VIEWS, SHOW MATERIALIZED VIEWS, DESCRIBE FORMATTED, SHOW PARTITIONS and, with include_ddl, SHOW CREATE TABLE. Sample data previews run SELECT * ... LIMIT 20.
Notes
- Row counts and sizes come from the metastore statistics (
numRows,totalSize), which are as fresh as the lastANALYZE TABLEor stats-collecting write. - Kerberos needs a build with the driver's
kerberostag and the GSSAPI C library:go build -tags kerberos .. The published binaries are built without it and report an error whenauth: KERBEROSis used. - Hive 4 creates non-ACID tables as external tables (
TRANSLATED_TO_EXTERNAL), soobject_typereports what Hive itself reports.
In the UI
Point-and-click, no config file needed.
- 1 Open Runs Create pipeline
- 2 Pick Apache Hive from the plugin list.
- 3 Fill in the wizard, set a schedule, save.
With the CLI
Save a YAML config, then run marmot ingest.
name: my-hive-pipeline
runs:
- hive:
host: "<host>"
username: "<username>"
password: "***"$ marmot ingest -c ingest.yamlNot using plugins? Other ways to populate Marmot
Configuration
20 top-level fields. * marks required fields.
tags multiselect Tags to apply to discovered assets
external_links object[] External links to show on all assets
name string Display name for the link
icon string Icon identifier for the link
url string URL to the external resource
filter object Filter discovered assets by name (regex)
include multiselect Include patterns for resource names (regex)
exclude multiselect Exclude patterns for resource names (regex)
host string HiveServer2 hostname or IP address
port int HiveServer2 port
- default
- 10000
auth select Authentication mechanism (NONE, NOSASL, LDAP, KERBEROS)
- default
- NONE
username string Username (hive when empty)
password password Password, required for LDAP
kerberos_service_name string Service part of the HiveServer2 Kerberos principal
- default
- hive
transport select Thrift transport (binary or http)
- default
- binary
http_path string Endpoint path for the http transport
- default
- cliservice
ssl bool Connect over TLS
- default
- false
ssl_skip_verify bool Skip TLS certificate verification
- default
- false
databases multiselect Databases to discover (all when empty)
exclude_databases multiselect Databases to skip
- default
- ["sys","information_schema"]
include_columns bool Include column information
- default
- true
include_views bool Include views and materialized views
- default
- true
include_statistics bool Include row counts and sizes from table statistics
- default
- true
include_partitions bool Count the partitions of partitioned tables
- default
- true
include_ddl bool Store the SHOW CREATE TABLE output in metadata
- default
- false
Assets emitted
Metadata this plugin attaches to each discovered asset.
Database
HiveDatabaseFieldsThe metadata fields the Hive plugin emits for database assets.
host stringHiveServer2 hostname
port intHiveServer2 port
database stringDatabase name
comment stringDatabase comment
location stringWarehouse location of the database
owner stringDatabase owner
owner_type stringOwner type (USER, ROLE, GROUP)
parameters objectDatabase properties (DBPROPERTIES)
hive_version stringHive version reported by the server
table_count intNumber of tables discovered in the database
view_count intNumber of views and materialized views discovered in the database
Table
HiveTableFieldsThe metadata fields the Hive plugin emits for table and view assets.
host stringHiveServer2 hostname
port intHiveServer2 port
database stringDatabase the object belongs to
table_name stringTable or view name
object_type stringObject type (managed, external, view, materialized_view)
owner stringObject owner
owner_type stringOwner type (USER, ROLE, GROUP)
created stringCreation time (RFC 3339)
last_access stringLast access time (RFC 3339), when Hive tracks it
last_ddl stringTime of the last DDL change (RFC 3339)
location stringStorage location
input_format stringHadoop input format class
output_format stringHadoop output format class
serde stringSerDe class
compressed boolWhether the storage is marked compressed
num_buckets intNumber of buckets, for bucketed tables
bucket_columns string[]Columns the table is bucketed by
sort_columns string[]Columns each bucket is sorted by
transactional boolWhether the table is ACID (transactional)
num_files intNumber of files, from table statistics
comment stringTable comment
partition_columns string[]Partition columns
partition_count intNumber of partitions
partition_values string[]First 20 partition specs (dt=2026-01-01)
parameters objectTable properties not surfaced as their own field
ddl stringSHOW CREATE TABLE output, when include_ddl is set
Column
HiveColumnFieldsThe per-column fields embedded in an asset's schema.
column_name stringColumn name
data_type stringHive data type as declared, complex types included
is_nullable boolFalse when a NOT NULL constraint covers the column
is_primary_key boolWhether the column is part of the primary key constraint
is_partition_column boolWhether the column is a partition column
description stringColumn comment
default_expression stringDefault value from a DEFAULT constraint