Manage Vault resources programmatically with Terraform

Use Terraform to manage policies, namespaces, and plugins in Vault.

Before you start

• You must have Terraform installed.
• You must have the Terraform Vault provider configured.
• You must have admin access to your Terraform installation. If you do not have admin access, you can still generate the relevant configuration files, but you will need to have someone else apply the changes.
• You must have a Vault server running.

Step 1: Create a resource file for namespaces

Terraform Vault provider supports a vault_namespace resource type for managing Vault namespaces:

resource "vault_namespace" "<TERRAFORM_RESOURCE_NAME>" {  path = "<VAULT_NAMESPACE>"}

To manage your Vault namespaces in Terraform:

1. Use the vault namespace list command to identify any unmanaged namespaces that you need to migrate. For example:

   $ vault namespace listKeys----admin/

2. Create a new Terraform Vault Provider resource file called vault_namespaces.tf that defines vault_namespace resources for each of the new or existing namespaces resources you want to manage.

   For example, to migrate the admin namespace in the example and create a new dev namespace:

   resource "vault_namespace" "admin_ns" {  path = "admin"}resource "vault_namespace" "dev_ns" {  path = "dev"}

Step 2: Create a resource file for secret engines

Terraform Vault provider supports discrete types for the different auth, secret, and database plugin types in Vault.

To migrate a secret engine, use the vault_mount resource type:

resource "vault_mount" "<TERRAFORM_RESOURCE_NAME>" {  path = "<VAULT_NAMESPACE>"  type = "<VAULT_PLUGIN_TYPE>"}

To manage your Vault secret engines in Terraform:

1. Use the vault secret list command to identify any unmanaged secret engines that you need to migrate. For example:

   $ vault secrets list | grep -vEw '(cubbyhole|identity|sys)'Path          Type         Accessor              Description----          ----         --------              -----------transit/      transit      transit_8291b949      n/a

2. Use the -namespace flag to check for unmanaged secret engines under any namespaces you identified in the previous step. For example, to check for secret engines under the admin namespace:

   $ vault secrets list -namespace=admin | grep -vEw '(cubbyhole|identity|sys)'Path           Type            Accessor                 Description----           ----            --------                 -----------admin_keys/    kv              kv_87edfc65              n/a

3. Create a new Terraform Vault Provider resource file called vault_secrets.tf that defines vault_mount resources for each of the new or existing secret engines you want to manage.

   For example, to migrate the transit and admin_keys secret engines in the example and enable a new kv engine under the new dev namespace called dev_keys:

   resource "vault_mount" "transit_plugin" {  path = "transit"  type = "transit"}resource "vault_mount" "admin_keys_plugin" {  namespace = vault_namespace.admin_ns.path  path = "admin_keys"  type = "kv"  options = {    version = "2"  }}resource "vault_mount" "dev_keys_plugin" {  namespace = vault_namespace.dev_ns.path  path = "dev_keys"  type = "kv"  options = {    version = "2"  }}

Step 3: Create a resource file for policies

Terraform Vault provider supports a vault_policy resource type for managing Vault policies:

resource "vault_policy" "<TERRAFORM_RESOURCE_NAME>" {  name = "<VAULT_POLICY_NAME>"  policy = <<EOT    <VAULT_POLICY_DEFINITION>  EOT}

To manage your Vault policies in Terraform:

1. Use the vault policy list command to identify any unmanaged policies that you need to migrate. For example:

   $ vault policy list | grep -vEw 'root'default

2. Create a Terraform Vault Provider resource file called vault_policies.tf that defines vault_mount resources for each policy resource you want to manage in Terraform. You can use the following bash code to write all your existing, non-root policies to the file:

   for vpolicy in $(vault policy list | grep -vw root) ; do  echo "resource \"vault_policy\" \"vault_$vpolicy\" {"  echo "  name = \"$vpolicy\""  echo "  policy = <<EOT"  vault policy read $vpolicy  echo "EOT"  echo "}"  echo ""done > vault_policies.tf

3. Update the vault_policies.tf file with any new policies you want to add. For example, to create a policy for the example dev_keys secret engine:

   resource "vault_policy" "dev_team_policy" {name = "dev_team"policy = <<EOT path vault_mount.dev_keys_plugin.path {   capabilities = ["create", "update"] } EOT}

Step 4: Update your Terraform configuration

1. Create a vault directory wherever you keep your deployment configuration files for Terraform.

2. Save your new resource files to your new Vault configuration directory.

3. Use terraform fmt to adjust the formatting (if needed) of your new configuration files:

   $ terraform fmtvault_namespaces.tfvault_secrets.tfvault_policies.tf

4. Use terraform validate to confirm the new configuration is valid:

   $ terraform validateSuccess! The configuration is valid.

Step 5: Import preexisting root-level resources

Use the terraform import command to import the preexisting root-level resources.

For example, import the admin namespace, default policy, and transit plugin from the previous steps:

$ terraform import vault_namespace.admin_ns adminvault_namespace.admin_ns: Importing from ID "admin"...vault_namespace.admin_ns: Import prepared!  Prepared vault_namespace for importvault_namespace.admin_ns: Refreshing state... [id=admin]Import successful!The resources that were imported are shown above. These resources are now inyour Terraform state and will henceforth be managed by Terraform.

$ terraform import vault_policy.default_policy defaultvault_policy.default_policy: Importing from ID "default"...vault_policy.default_policy: Import prepared!  Prepared vault_policy for importvault_policy.default_policy: Refreshing state... [id=default]Import successful!The resources that were imported are shown above. These resources are now inyour Terraform state and will henceforth be managed by Terraform.

$ terraform import vault_mount.transit_plugin transitvault_mount.transit_plugin: Importing from ID "transit"...vault_mount.transit_plugin: Import prepared!  Prepared vault_mount for importvault_mount.transit_plugin: Refreshing state... [id=transit]Import successful!The resources that were imported are shown above. These resources are now inyour Terraform state and will henceforth be managed by Terraform.

Step 6: Import preexisting nested resources

To import resources that belong to a previously unmanaged namespace, you must set the TERRAFORM_VAULT_NAMESPACE_IMPORT environment variable before importing.

For example, to import the admin_keys secret engine from the admin namespace:

1. Set TERRAFORM_VAULT_NAMESPACE_IMPORT to the admin Vault namespace:

   $ export TERRAFORM_VAULT_NAMESPACE_IMPORT="admin"

2. Import the vault_mount resource admin_keys:

   $ terraform import vault_mount.admin_keys_plugin admin_keysvault_mount.admin_keys_plugin: Importing from ID "admin_keys"...vault_mount.admin_keys_plugin: Import prepared!  Prepared vault_mount for importvault_mount.admin_keys_plugin: Refreshing state... [id=admin_keys]Import successful!The resources that were imported are shown above. These resources are now inyour Terraform state and will henceforth be managed by Terraform.

3. Unset the TERRAFORM_VAULT_NAMESPACE_IMPORT variable when you finish importing child resources:

   $ unset TERRAFORM_VAULT_NAMESPACE_IMPORT

Step 6: Verify the import

1. Use the terraform state show command to check your Terraform state file and verify the resources imported successfully. For example, to check the admin_keys resource:

   $ terraform state show vault_mount.admin_keys_plugin# vault_mount.admin_keys_plugin:resource "vault_mount" "admin_keys" {    accessor                     = "kv_87edfc65"    allowed_managed_keys         = []    audit_non_hmac_request_keys  = []    audit_non_hmac_response_keys = []    default_lease_ttl_seconds    = 0    description                  = null    external_entropy_access      = false    id                           = "admin_keys"    local                        = false    max_lease_ttl_seconds        = 0    namespace                    = "admin"    options                      = {        "version" = "2"    }    path                         = "admin_keys"    seal_wrap                    = false    type                         = "kv"

2. For each of the migrated resources, compare the accessor value from your Terraform state to the accessor value in Vault. For example, to confirm the accessor for admin_keys:

   $ vault secrets list -namespace="admin" | grep -vEw '(cubbyhole|identity|sys)'Path           Type            Accessor                 Description----           ----            --------                 -----------admin_keys/    kv              kv_87edfc65              n/a

Step 7: Add new Vault resources

1. Run terraform plan to confirm the new resources that Terraform will manage:

   $ terraform planvault_policy.default_policy: Refreshing state... [id=default]vault_namespace.admin_ns: Refreshing state... [id=admin/]vault_mount.transit_plugin: Refreshing state... [id=transit]vault_mount.admin_keys_plugin: Refreshing state... [id=admin_keys]Terraform used the selected providers to generate the following execution plan.Resource actions are indicated with the following symbols:  + createTerraform will perform the following actions:  # vault_mount.dev_keys_plugin will be created  + resource "vault_mount" "dev_keys" {      + accessor                     = (known after apply)      + audit_non_hmac_request_keys  = (known after apply)      + audit_non_hmac_response_keys = (known after apply)      + default_lease_ttl_seconds    = (known after apply)      + external_entropy_access      = false      + id                           = (known after apply)      + max_lease_ttl_seconds        = (known after apply)      + namespace                    = "dev"      + options                      = {          + "version" = "2"        }      + path                         = "dev_keys"      + seal_wrap                    = (known after apply)      + type                         = "kv"    }  # vault_namespace.dev_ns will be created  + resource "vault_namespace" "dev" {      + custom_metadata = (known after apply)      + id              = (known after apply)      + namespace_id    = (known after apply)      + path            = "dev"      + path_fq         = (known after apply)    }  # vault_policy.dev_team will be created  + resource "vault_policy" "dev_team" {      + id     = (known after apply)      + name   = "dev_team"      + policy = <<-EOT            path vault_mount.dev_keys_plugin.path {                  capabilities = ["create", "update"]                }        EOT    }Plan: 3 to add, 0 to change, 0 to destroy.

2. Run terraform apply to create the new resources:

   $ terraform applyvault_namespace.dev_ns: Creating...vault_namespace.dev_ns: Creation complete after 0s [id=dev/]vault_mount.dev_keys_plugin: Creating...vault_mount.dev_keys_plugin: Creation complete after 0s [id=dev_keys]vault_policy.dev_team: Creating...vault_policy.dev_team: Creation complete after 0s [id=dev_team]Apply complete! Resources: 3 added, 0 changed, 0 destroyed.

3. Use the terraform state show command to check your Terraform state file and verify the new resources created successfully. For example, to check the dev_keys resource:

   $ terraform state show vault_mount.dev_keys_plugin# vault_mount.dev_keys_plugin:resource "vault_mount" "dev_keys" {    accessor                     = "kv_b3d2dd6f"    allowed_managed_keys         = []    audit_non_hmac_request_keys  = []    audit_non_hmac_response_keys = []    default_lease_ttl_seconds    = 0    description                  = null    external_entropy_access      = false    id                           = "dev_keys"    local                        = false    max_lease_ttl_seconds        = 0    namespace                    = "dev"    options                      = {        "version" = "2"    }    path                         = "dev_keys"    seal_wrap                    = false    type                         = "kv"}

4. Confirm that your Vault instance can use the new resources. For example, to confirm the dev_keys resources:

   $ vault secrets list -namespace="dev" | grep -vEw '(cubbyhole|identity|sys)'Path          Type            Accessor                 Description----          ----            --------                 -----------dev_keys/     kv              kv_b3d2dd6f              n/a

Next steps

1. Review the best practices for programmatic Vault management.