Google Cloud Storage

The Google Cloud Storage plugin discovers buckets from GCP projects. It captures bucket metadata including location, storage class, encryption settings, and lifecycle rules.

Connection Examples

Required Permissions

The service account needs the following IAM roles:

• Storage Bucket Viewer (roles/storage.bucketViewer) - For discovering and listing buckets

Or use a custom role with these permissions:

• storage.buckets.list
• storage.buckets.get
• storage.objects.list (if using object count)

Keyless authentication

On Marmot Cloud or Marmot Enterprise the pipeline can present its own identity instead of a service account key. Set workload_identity_provider to a Workload Identity Federation provider that trusts your Marmot instance as an OIDC issuer, and grant the pipeline's subject, pipeline:<name> as reported by the pipeline API, the role above directly (principal://iam.googleapis.com/<pool>/subject/pipeline:<name>), or grant it roles/iam.workloadIdentityUser on a service account named in service_account. No key exists anywhere; Marmot mints a short-lived token for each run.

name: my-gcs-pipeline
runs:
  - gcs:
      project_id: "<project_id>"

$ marmot ingest -c ingest.yaml