Skip to content

Latest commit

 

History

History
39 lines (28 loc) · 3.6 KB

README.md

File metadata and controls

39 lines (28 loc) · 3.6 KB
Raw

Basic Dataplex Usage

This example illustrates showcase the basic dataplex features.

It will:

  • Create a lake
  • Create a zone within that lake
  • Create an asset within the zone that is within our lake

Inputs

Name Description Type Default Required
assets List of asset objects to create 
list(object({
    id           = string,
    lake_id      = string,
    zone_id      = string,
    display_name = optional(string),
    description  = optional(string),
    labels       = optional(map(string), {}),
    discovery_spec = object({
      enabled          = bool,
      schedule         = optional(string),
      include_patterns = optional(list(string), []),
      exclude_patterns = optional(list(string), []),
      csv_options = optional(object({
        delimiter              = optional(string, ","),
        header_rows            = optional(number, 0)
        disable_type_inference = optional(bool, false),
        encoding               = optional(string, "UTF-8")
      }))
      json_options = optional(object({
        disable_type_inference = optional(bool, false),
        encoding               = optional(string, "UTF-8")
      })),
    }),
    resource_spec = object({
      name = string,
      type = string
    })
  }))
 [] no
project_id The project to host your resources in string n/a yes
location The location to host your resources in string n/a yes
lakes List of lake objects to create 
list(object({
    id                 = string,
    display_name       = optional(string),
    description        = optional(string),
    labels             = optional(map(string), {}),
    dataproc_metastore = optional(string)
  }))
 [] no
zones List of zone objects to create 
list(object({
    id           = string,
    lake_id      = string,
    type         = string,
    display_name = optional(string),
    description  = optional(string),
    labels       = optional(map(string), {}),
    discovery_spec = object({
      enabled          = bool,
      schedule         = optional(string),
      include_patterns = optional(list(string), []),
      exclude_patterns = optional(list(string), []),
      csv_options = optional(object({
        delimiter              = optional(string, ","),
        header_rows            = optional(number, 0)
        disable_type_inference = optional(bool, false),
        encoding               = optional(string, "UTF-8")
      }))
      json_options = optional(object({
        disable_type_inference = optional(bool, false),
        encoding               = optional(string, "UTF-8")
      })),
    }),
    resource_spec = object({
      location_type = string
    })
  }))
 [] no

Outputs

Name Description
asset_self_links Map of self links to the defined Dataplex assets
asset_uids Map of UIDs to the defined Dataplex assets
lake_self_links Map of self links to the defined Dataplex lakes
lake_uids Map of UIDs to the defined Dataplex lakes
zone_self_links Map of self links to the defined Dataplex zones
zone_uids Map of UIDs to the defined Dataplex zones

Example Usage

See terraform.tfvars for some of the example usage.

Notes

Assets

Remember the assets you discover must be in the same region(s) as the location policy of the lake/zone you create.