»Upgrading Vault

These are general upgrade instructions for Vault for both non-HA and HA setups. Please ensure that you also read any version-specific upgrade notes which can be found in the sidebar.

Vault upgrades are designed such that large jumps (ie 1.3.10 -> 1.7.x) are supported. The upgrade notes for each intervening version must be reviewed. The upgrade notes may describe additional steps or configuration to update before, during, or after the upgrade.

»Tutorial

Refer to the Vault Upgrade Standard Procedure tutorial to learn how to upgrade Vault Enterprise.

»Agent

The Vault Agent is an API client of the Vault Server. Vault APIs are almost always backwards compatible. When they are not, this is called out in the upgrade guide for the new Vault version, and there is a lengthy deprecation period. The Vault Agent version can lag behind the Vault Server version, though we recommend keeping all Vault instances up to date with the most recent minor Vault version to the extent possible.

»Testing the Upgrade

It's always a good idea to try to ensure that the upgrade will be successful in your environment. The ideal way to do this is to take a snapshot of your data and load it into a test cluster. However, if you are issuing secrets to third party resources (cloud credentials, database credentials, etc.) ensure that you do not allow external network connectivity during testing, in case credentials expire. This prevents the test cluster from trying to revoke these resources along with the non-test cluster.

»OSS to Enterprise Installations

Upgrading to Vault Enterprise installations follow the same steps as OSS upgrades except that the Vault Enterprise binary is to be used and the license file applied, when applicable. The Enterprise binary and license file can be obtained through your HashiCorp sales team.

»Non-HA Installations

Upgrading non-HA installations of Vault is as simple as replacing the Vault binary with the new version and restarting Vault. Any upgrade tasks that can be performed for you will be taken care of when Vault is unsealed.

Always use SIGINT or SIGTERM to properly shut down Vault.

Be sure to also read and follow any instructions in the version-specific upgrade notes.

»HA Installations

This is our recommended upgrade procedure, and the procedure we use internally at HashiCorp. However, you should consider how to apply these steps to your particular setup since HA setups can differ on whether a load balancer is in use, what addresses clients are being given to connect to Vault (standby + leader, leader-only, or discovered via service discovery), etc.

Whatever method you use, you should ensure that you never fail over from a newer version of Vault to an older version. Our suggested procedure is designed to prevent this.

Please note that Vault does not support true zero-downtime upgrades, but with proper upgrade procedure the downtime should be very short (a few hundred milliseconds to a second depending on how the speed of access to the storage backend).

Perform these steps on each standby:

1. Properly shut down Vault on the standby node via SIGINT or SIGTERM
2. Replace the Vault binary with the new version; ensure that mlock() capability is added to the new binary with setcap
3. Start the standby node
4. Unseal the standby node
5. Verify vault status shows correct Version and HA Mode is standby
6. Review the node's logs to ensure successful startup and unseal

At this point all standby nodes will be upgraded and ready to take over. The upgrade will not be complete until one of the upgraded standby nodes takes over active duty. To do this:

1. Properly shut down the remaining (active) node

   Note: It is important that you shut the node down properly. This will perform a step-down and release the HA lock, allowing a standby node to take over with a very short delay. If you kill Vault without letting it release the lock, a standby node will not be able to take over until the lock's timeout period has expired. This is backend-specific but could be ten seconds or more.

2. Replace the Vault binary with the new version; ensure that mlock() capability is added to the new binary with setcap

3. Start the node

4. Unseal the node

5. Verify vault status shows correct Version and HA Mode is standby

6. Review the node's logs to ensure successful startup and unseal

Internal upgrade tasks will happen after one of the upgraded standby nodes takes over active duty.

Be sure to also read and follow any instructions in the version-specific upgrade notes.

»Replication Installations

Upgrading installations of Vault which participate in Enterprise Replication requires the following basic order of operations:

• Upgrade the replication secondary instances first using appropriate guidance from the previous sections depending on whether each secondary instance is Non-HA or HA
• Verify functionality of each secondary instance after upgrading
• When satisfied with functionality of upgraded secondary instances, upgrade the primary instance

Here is an example of upgrading four Vault replicated Vault clusters:

In the above scenario, the ideal upgrade procedure would be as follows, verifying functionality after each cluster upgrade.

1. Upgrade Clusters B and D, using the HA upgrade process above. These clusters have no downstream clusters, so they should be upgraded first, but the ordering of B vs D does not matter.
2. Upgrade Cluster C, which now has an upgraded downstream cluster (Cluster D). Because Cluster C is a cluster, it should also use the HA upgrade process.
3. Finally, upgrade Cluster A. All clusters downstream of A will already be upgraded. It should be upgraded last, as it is a Performance Primary and a DR Primary.