• Overview
    • Automated PKI Infrastructure
    • Data Encryption & Tokenization
    • Database Credential Rotation
    • Dynamic Secrets
    • Identity-based Access
    • Key Management
    • Kubernetes Secrets
    • Secrets Management
  • Enterprise
  • Tutorials
  • Docs
  • API
  • Community
GitHubTry Cloud
Download
    • v1.11.x (latest)
    • v1.10.x
    • v1.9.x
    • v1.8.x
    • v1.7.x
    • v1.6.x
    • v1.5.x
    • v1.4.x
  • What is Vault?
  • Use Cases
    • CLI Quick Start
    • HCP Quick Start
    • Developer Quick Start

  • Browser Support
  • Installing Vault
    • Overview
    • Architecture
    • High Availability
    • Integrated Storage
    • Security Model
    • Telemetry
    • Token Authentication
    • Key Rotation
    • Replication
    • Limits and Maximums
    • Overview
    • 'Dev' Server
    • Seal/Unseal
    • Namespace API Lock
    • Lease, Renew, and Revoke
    • Authentication
    • Tokens
    • Identity
    • OIDC Provider
    • Response Wrapping
    • Policies
    • Password Policies
    • Username Templating
    • High Availability
    • Storage
      • Overview
      • Autopilot
    • PGP, GnuPG, and Keybase
    • Recovery Mode
    • Resource Quotas
      • Overview
      • FAQ
    • Transform
    • Mount Migration
    • Overview
      • Overview
      • TCP
    • replication
      • Overview
      • AliCloud KMS
      • AWS KMS
      • Azure Key Vault
      • GCP Cloud KMS
      • OCI KMS
      • HSM PKCS11 ENT
      • Vault Transit
    • sentinel
      • Overview
      • Consul
      • Kubernetes
      • Overview
      • Aerospike
      • Alicloud OSS
      • Azure
      • Cassandra
      • CockroachDB
      • Consul
      • CouchDB
      • DynamoDB
      • Etcd
      • Filesystem
      • FoundationDB
      • Google Cloud Spanner
      • Google Cloud Storage
      • In-Memory
      • Manta
      • MSSQL
      • MySQL
      • OCI Object Storage
      • PostgreSQL
      • Integrated Storage (Raft)
      • S3
      • Swift
      • Zookeeper
    • telemetry
    • ui
    • Log Completed Requests
    • Entropy Augmentation ENT
    • kms_library ENT
    • Overview
    • agent
      • Overview
      • disable
      • enable
      • list
      • Overview
      • disable
      • enable
      • help
      • list
      • move
      • tune
    • debug
    • delete
      • Overview
      • delete
      • destroy
      • enable-versioning
      • get
      • list
      • metadata
      • patch
      • put
      • rollback
      • undelete
      • Overview
      • lookup
      • renew
      • revoke
      • Overview
      • get
      • inspect
    • list
    • login
    • monitor
    • namespace
      • Overview
      • diagnose
      • generate-root
      • init
      • key-status
      • members
      • migrate
      • raft
      • rekey
      • rotate
      • seal
      • step-down
      • unseal
      • usage
    • path-help
      • Overview
      • deregister
      • info
      • list
      • register
      • reload
      • Overview
      • delete
      • fmt
      • list
      • read
      • write
    • read
      • Overview
      • disable
      • enable
      • list
      • move
      • tune
    • server
    • ssh
    • status
      • Overview
      • capabilities
      • create
      • lookup
      • renew
      • revoke
    • unwrap
    • version
    • version-history
    • write
    • Token Helpers
    • Overview
      • Overview
        • Overview
        • AliCloud
        • AppRole
        • AWS
        • Azure
        • Cert
        • CF
        • GCP
        • JWT
        • Kerberos
        • Kubernetes
        • Overview
        • File
      • Overview
        • Overview
        • Kubernetes
    • Templates
    • Windows service

    • Overview
    • Active Directory
    • AliCloud
    • AWS
    • Azure
    • Consul
    • Cubbyhole
      • Overview
      • Cassandra
      • Couchbase
      • Elasticsearch
      • HanaDB
      • IBM Db2
      • InfluxDB
      • MongoDB
      • MongoDB Atlas
      • MSSQL
      • MySQL/MariaDB
      • Oracle
      • PostgreSQL
      • Redshift
      • Snowflake
      • Custom
    • Google Cloud
    • Google Cloud KMS
      • Overview
      • Identity Tokens
      • OIDC Identity Provider
      • Overview
      • Azure Key Vault
      • AWS KMS
      • GCP Cloud KMS
      • Overview
      • K/V Version 1
      • K/V Version 2
    • KMIP ENTERPRISE
    • Kubernetes
    • MongoDB Atlas
    • Nomad
    • OpenLDAP
      • Overview
      • Setup and Usage
      • Quick Start - Root CA Setup
      • Quick Start - Intermediate CA Setup
      • Considerations
      • Rotation Primitives
    • RabbitMQ
      • Overview
      • Signed Certificates
      • SSH OTP
      • Dynamic Key
    • Terraform Cloud
    • TOTP
      • Overview
      • FF3-1 Tweak Usage
      • Tokenization Transform ENTERPRISE
    • Transit
    • Venafi (Certificates)
    • Overview
    • AppRole
    • AliCloud
    • AWS
    • Azure
    • Cloud Foundry
    • GitHub
    • Google Cloud
      • Overview
      • OIDC Providers
    • Kerberos
    • Kubernetes
    • LDAP
      • Overview
      • FAQ
    • Oracle Cloud Infrastructure
    • Okta
    • RADIUS
    • TLS Certificates
    • Tokens
    • Username & Password

    • App ID DEPRECATED
    • Overview
    • File
    • Syslog
    • Socket
    • Overview
    • Plugin Architecture
    • Plugin Development
    • Plugin Management
    • Plugin Portal
  • Vault Integration Program
  • Troubleshoot

    • Overview
      • Overview
      • Agent Injector vs. Vault CSI Provider
        • Overview
        • Running Vault
        • Enterprise Licensing
        • Running Vault on OpenShift
        • Configuration
          • Overview
          • Development
          • Standalone with Load Balanced UI
          • Standalone with TLS
          • Standalone with Audit Storage
          • External Vault
          • Using Kubernetes Auth Method
          • HA Cluster with Consul
          • HA Cluster with Raft
          • HA Enterprise Cluster with Raft
          • HA Enterprise DR Clusters with Raft
          • HA Enterprise Performance Clusters with Raft
          • Vault Agent Injector TLS Configuration
          • Vault Agent Injector TLS with Cert-Manager
        • Overview
        • Annotations
        • Installation
        • Examples
        • Overview
        • Installation
        • Configurations
        • Examples
      • Overview
      • Vault Lambda Extension
      • Running Vault
      • Overview
      • Installation
      • Configuration
      • Troubleshooting
      • Overview
      • Installation
      • Configuration
      • Upgrading
      • Troubleshooting

    • Overview
    • Upgrade Plugins
    • Upgrade to 1.11.x
    • Upgrade to 1.10.x
    • Upgrade to 1.9.x
    • Upgrade to 1.8.x
    • Upgrade to 1.7.x
    • Upgrade to 1.6.3
    • Upgrade to 1.6.2
    • Upgrade to 1.6.1
    • Upgrade to 1.6.0
    • Upgrade to 1.5.3
    • Upgrade to 1.5.2
    • Upgrade to 1.5.1
    • Upgrade to 1.5.0
    • Upgrade to 1.4.6
    • Upgrade to 1.4.5
    • Upgrade to 1.4.4
    • Upgrade to 1.4.1
    • Upgrade to 1.4.0
    • Upgrade to 1.3.10
    • Upgrade to 1.3.9
    • Upgrade to 1.3.8
    • Upgrade to 1.3.5
    • Upgrade to 1.3.4
    • Upgrade to 1.3.3
    • Upgrade to 1.3.2
    • Upgrade to 1.3.0
    • Upgrade to 1.2.7
    • Upgrade to 1.2.6
    • Upgrade to 1.2.5
    • Upgrade to 1.2.4
    • Upgrade to 1.2.1
    • Upgrade to 1.2.0
    • Upgrade to 1.1.2
    • Upgrade to 1.1.1
    • Upgrade to 1.1.0
    • Upgrade to 1.0.0
    • Upgrade to 0.11.6
    • Upgrade to 0.11.2
    • Upgrade to 0.11.0
    • Upgrade to 0.10.4
    • Upgrade to 0.10.2
    • Upgrade to 0.10.0
    • Upgrade to 0.9.6
    • Upgrade to 0.9.3
    • Upgrade to 0.9.2
    • Upgrade to 0.9.1
    • Upgrade to 0.9.0
    • Upgrade to 0.8.0
    • Upgrade to 0.7.0
    • Upgrade to 0.6.4
    • Upgrade to 0.6.3
    • Upgrade to 0.6.2
    • Upgrade to 0.6.1
    • Upgrade to 0.6.0
    • Upgrade to 0.5.1
    • Upgrade to 0.5.0

    • Overview
    • 1.11.0
    • 1.10.0
    • 1.9.0
    • 1.8.0
    • 1.7.0
    • 1.6.0
    • 1.5.0

    • Overview
    • FAQ

    • Overview
    • Feature Deprecation Notice and Plans
    • License
    • Client Count
    • Login MFA
    • Server Side Consistent Token

  • Glossary

    • Overview
      • Overview
      • Autoloading
      • FAQ
    • Replication
      • Overview
      • Behavioral Changes
      • Security
    • Automated Integrated Storage Snapshots
    • Automated Upgrades
    • Redundancy Zones
    • Lease Count Quotas
    • Entropy Augmentation
      • Overview
      • FIPS 140-2 Inside Vault
      • Seal Wrap for FIPS 140-2
    • Seal Wrap
    • Namespaces
    • Performance Standbys
    • Eventual Consistency
    • Control Groups
    • Managed Keys
      • Overview
      • Duo MFA
      • Okta MFA
      • PingID MFA
      • TOTP MFA
      • Overview
      • Examples
      • Properties
    • HCP Vault

The Vault website is being redesigned to help you find what you are looking for more effectively.

Type '/' to Search

»Highly Available Vault Enterprise Disaster Recovery Clusters with Integrated Storage (Raft)

Important Note: This chart is not compatible with Helm 2. Please use Helm 3.6+ with this chart.

The following is an example of creating a disaster recovery cluster using Vault Helm.

For more information on Disaster Recovery, see the official documentation.

For license configuration refer to Running Vault Enterprise.

»Primary Cluster

First, create the primary cluster:

helm install vault-primary hashicorp/vault \
  --set='server.image.repository=hashicorp/vault-enterprise' \
  --set='server.image.tag=1.10.3-ent' \
  --set='server.ha.enabled=true' \
  --set='server.ha.raft.enabled=true'
helm install vault-primary hashicorp/vault \
  --set='server.image.repository=hashicorp/vault-enterprise' \
  --set='server.image.tag=1.10.3-ent' \
  --set='server.ha.enabled=true' \
  --set='server.ha.raft.enabled=true'

Next, initialize and unseal vault-primary-0 pod:

kubectl exec -ti vault-primary-0 -- vault operator init
kubectl exec -ti vault-primary-0 -- vault operator unseal
kubectl exec -ti vault-primary-0 -- vault operator init
kubectl exec -ti vault-primary-0 -- vault operator unseal

Finally, join the remaining pods to the Raft cluster and unseal them. The pods will need to communicate directly so we'll configure the pods to use the internal service provided by the Helm chart:

kubectl exec -ti vault-primary-1 -- vault operator raft join http://vault-primary-0.vault-primary-internal:8200
kubectl exec -ti vault-primary-1 -- vault operator unseal

kubectl exec -ti vault-primary-2 -- vault operator raft join http://vault-primary-0.vault-primary-internal:8200
kubectl exec -ti vault-primary-2 -- vault operator unseal
kubectl exec -ti vault-primary-1 -- vault operator raft join http://vault-primary-0.vault-primary-internal:8200
kubectl exec -ti vault-primary-1 -- vault operator unseal

kubectl exec -ti vault-primary-2 -- vault operator raft join http://vault-primary-0.vault-primary-internal:8200
kubectl exec -ti vault-primary-2 -- vault operator unseal

To verify if the Raft cluster has successfully been initialized, run the following.

First, login using the root token on the vault-primary-0 pod:

kubectl exec -ti vault-primary-0 -- vault login
kubectl exec -ti vault-primary-0 -- vault login

Next, list all the raft peers:

kubectl exec -ti vault-primary-0 -- vault operator raft list-peers

Node                                    Address                        State       Voter
----                                    -------                        -----       -----
a1799962-8711-7f28-23f0-cea05c8a527d    vault-primary-0.vault-primary-internal:8201    leader      true
e6876c97-aaaa-a92e-b99a-0aafab105745    vault-primary-1.vault-primary-internal:8201    follower    true
4b5d7383-ff31-44df-e008-6a606828823b    vault-primary-2.vault-primary-internal:8201    follower    true
kubectl exec -ti vault-primary-0 -- vault operator raft list-peers

Node                                    Address                        State       Voter
----                                    -------                        -----       -----
a1799962-8711-7f28-23f0-cea05c8a527d    vault-primary-0.vault-primary-internal:8201    leader      true
e6876c97-aaaa-a92e-b99a-0aafab105745    vault-primary-1.vault-primary-internal:8201    follower    true
4b5d7383-ff31-44df-e008-6a606828823b    vault-primary-2.vault-primary-internal:8201    follower    true

»Secondary Cluster

With the primary cluster created, next create a secondary cluster and enable disaster recovery replication.

helm install vault-secondary hashicorp/vault \
  --set='server.image.repository=hashicorp/vault-enterprise' \
  --set='server.image.tag=1.10.3-ent' \
  --set='server.ha.enabled=true' \
  --set='server.ha.raft.enabled=true'
helm install vault-secondary hashicorp/vault \
  --set='server.image.repository=hashicorp/vault-enterprise' \
  --set='server.image.tag=1.10.3-ent' \
  --set='server.ha.enabled=true' \
  --set='server.ha.raft.enabled=true'

Next, initialize and unseal vault-secondary-0 pod:

kubectl exec -ti vault-secondary-0 -- vault operator init
kubectl exec -ti vault-secondary-0 -- vault operator unseal
kubectl exec -ti vault-secondary-0 -- vault operator init
kubectl exec -ti vault-secondary-0 -- vault operator unseal

Finally, join the remaining pods to the Raft cluster and unseal them. The pods will need to communicate directly so we'll configure the pods to use the internal service provided by the Helm chart:

kubectl exec -ti vault-secondary-1 -- vault operator raft join http://vault-secondary-0.vault-secondary-internal:8200
kubectl exec -ti vault-secondary-1 -- vault operator unseal

kubectl exec -ti vault-secondary-2 -- vault operator raft join http://vault-secondary-0.vault-secondary-internal:8200
kubectl exec -ti vault-secondary-2 -- vault operator unseal
kubectl exec -ti vault-secondary-1 -- vault operator raft join http://vault-secondary-0.vault-secondary-internal:8200
kubectl exec -ti vault-secondary-1 -- vault operator unseal

kubectl exec -ti vault-secondary-2 -- vault operator raft join http://vault-secondary-0.vault-secondary-internal:8200
kubectl exec -ti vault-secondary-2 -- vault operator unseal

To verify if the Raft cluster has successfully been initialized, run the following.

First, login using the root token on the vault-secondary-0 pod:

kubectl exec -ti vault-secondary-0 -- vault login
kubectl exec -ti vault-secondary-0 -- vault login

Next, list all the raft peers:

kubectl exec -ti vault-secondary-0 -- vault operator raft list-peers

Node                                    Address                        State       Voter
----                                    -------                        -----       -----
a1799962-8711-7f28-23f0-cea05c8a527d    vault-secondary-0.vault-secondary-internal:8201    leader      true
e6876c97-aaaa-a92e-b99a-0aafab105745    vault-secondary-1.vault-secondary-internal:8201    follower    true
4b5d7383-ff31-44df-e008-6a606828823b    vault-secondary-2.vault-secondary-internal:8201    follower    true
kubectl exec -ti vault-secondary-0 -- vault operator raft list-peers

Node                                    Address                        State       Voter
----                                    -------                        -----       -----
a1799962-8711-7f28-23f0-cea05c8a527d    vault-secondary-0.vault-secondary-internal:8201    leader      true
e6876c97-aaaa-a92e-b99a-0aafab105745    vault-secondary-1.vault-secondary-internal:8201    follower    true
4b5d7383-ff31-44df-e008-6a606828823b    vault-secondary-2.vault-secondary-internal:8201    follower    true

»Enable Disaster Recovery Replication On Primary

With the initial clusters setup, we can now configure them for disaster recovery replication.

First, on the primary cluster, enable replication:

kubectl exec -ti vault-primary-0 -- vault write -f sys/replication/dr/primary/enable primary_cluster_addr=https://vault-primary-active:8201
kubectl exec -ti vault-primary-0 -- vault write -f sys/replication/dr/primary/enable primary_cluster_addr=https://vault-primary-active:8201

Next, create a token the secondary cluster will use to configure replication:

kubectl exec -ti vault-primary-0 -- vault write sys/replication/dr/primary/secondary-token id=secondary
kubectl exec -ti vault-primary-0 -- vault write sys/replication/dr/primary/secondary-token id=secondary

The token in the output will be used when configuring the secondary cluster.

»Enable Disaster Recovery Replication On Secondary

Using the token created in the last step, enable disaster recovery replication on the secondary:

kubectl exec -ti vault-secondary-0 -- vault write sys/replication/dr/secondary/enable token=<TOKEN FROM PRIMARY>
kubectl exec -ti vault-secondary-0 -- vault write sys/replication/dr/secondary/enable token=<TOKEN FROM PRIMARY>

Last, delete the remainder secondary pods and unseal them using the primary unseal token after Kubernetes reschedules them:

kubectl delete pod vault-secondary-1
kubectl exec -ti vault-secondary-1 -- vault operator unseal <PRIMARY UNSEAL TOKEN>

kubectl delete pod vault-secondary-2
kubectl exec -ti vault-secondary-2 -- vault operator unseal <PRIMARY UNSEAL TOKEN>
kubectl delete pod vault-secondary-1
kubectl exec -ti vault-secondary-1 -- vault operator unseal <PRIMARY UNSEAL TOKEN>

kubectl delete pod vault-secondary-2
kubectl exec -ti vault-secondary-2 -- vault operator unseal <PRIMARY UNSEAL TOKEN>
github logoEdit this page
DocsAPILearnCommunityPrivacySecurityPress KitConsent Manager