Type '/' to Search

»Azure Secrets Engine

The Azure secrets engine dynamically generates Azure service principals along with role and group assignments. Vault roles can be mapped to one or more Azure roles, and optionally group assignments, providing a simple, flexible way to manage the permissions granted to generated service principals.

Each service principal is associated with a Vault lease. When the lease expires (either during normal revocation or through early revocation), the service principal is automatically deleted.

If an existing service principal is specified as part of the role configuration, a new password will be dynamically generated instead of a new service principal. The password will be deleted when the lease is revoked.

Microsoft is shutting down their Azure Active Directory API and will be retiring it in 2022. If you are currently using this secret engine, you will need to update the credentials to include Microsoft Graph API permissions and specify the use_microsoft_graph_api configuration value as true. See the API Docs for more details.

»Setup

Most secrets engines must be configured in advance before they can perform their functions. These steps are usually completed by an operator or configuration management tool.

  1. Enable the Azure secrets engine:

    $ vault secrets enable azure
Success! Enabled the azure secrets engine at: azure/

    $ vault secrets enable azure
Success! Enabled the azure secrets engine at: azure/

    By default, the secrets engine will mount at the name of the engine. To enable the secrets engine at a different path, use the -path argument.

  2. Configure the secrets engine with account credentials:

    $ vault write azure/config \
    subscription_id=$AZURE_SUBSCRIPTION_ID \
    tenant_id=$AZURE_TENANT_ID \
    client_id=$AZURE_CLIENT_ID \
    client_secret=$AZURE_CLIENT_SECRET \
    use_microsoft_graph_api=true

Success! Data written to: azure/config

    $ vault write azure/config \
    subscription_id=$AZURE_SUBSCRIPTION_ID \
    tenant_id=$AZURE_TENANT_ID \
    client_id=$AZURE_CLIENT_ID \
    client_secret=$AZURE_CLIENT_SECRET \
    use_microsoft_graph_api=true

Success! Data written to: azure/config

    If you are running Vault inside an Azure VM with MSI enabled, client_id and client_secret may be omitted. For more information on authentication, see the authentication section below.

  3. Configure a role. A role may be set up with either an existing service principal, or a set of Azure roles that will be assigned to a dynamically created service principal.

To configure a role called "my-role" with an existing service principal:

$ vault write azure/roles/my-role \
    application_object_id=<existing_app_obj_id> \
    ttl=1h

$ vault write azure/roles/my-role \
    application_object_id=<existing_app_obj_id> \
    ttl=1h

Alternatively, to configure the role to create a new service principal with Azure roles:

$ vault write azure/roles/my-role ttl=1h azure_roles=-<<EOF
    [
        {
            "role_name": "Contributor",
            "scope":  "/subscriptions/<uuid>/resourceGroups/Website"
        }
    ]
EOF

$ vault write azure/roles/my-role ttl=1h azure_roles=-<<EOF
    [
        {
            "role_name": "Contributor",
            "scope":  "/subscriptions/<uuid>/resourceGroups/Website"
        }
    ]
EOF

Roles may also have their own TTL configuration that is separate from the mount's TTL. For more information on roles see the roles section below.

»Usage

After the secrets engine is configured and a user/machine has a Vault token with the proper permissions, it can generate credentials. The usage pattern is the same whether an existing or dynamic service principal is used.

To generate a credential using the "my-role" role:

$ vault read azure/creds/my-role

Key                Value
---                -----
lease_id           azure/creds/sp_role/1afd0969-ad23-73e2-f974-962f7ac1c2b4
lease_duration     60m
lease_renewable    true
client_id          408bf248-dd4e-4be5-919a-7f6207a307ab
client_secret      ad06228a-2db9-4e0a-8a5d-e047c7f32594

$ vault read azure/creds/my-role

Key                Value
---                -----
lease_id           azure/creds/sp_role/1afd0969-ad23-73e2-f974-962f7ac1c2b4
lease_duration     60m
lease_renewable    true
client_id          408bf248-dd4e-4be5-919a-7f6207a307ab
client_secret      ad06228a-2db9-4e0a-8a5d-e047c7f32594

This endpoint generates a renewable set of credentials. The application can login using the client_id/client_secret and will have access provided by configured service principal or the Azure roles set in the "my-role" configuration.

»Roles

Vault roles let you configure either an existing service principal or a set of Azure roles, along with role-specific TTL parameters. If an existing service principal is not provided, the configured Azure roles will be assigned to a newly created service principal. The Vault role may optionally specify role-specific ttl and/or max_ttl values. When the lease is created, the more restrictive of the mount or role TTL value will be used.

»Application Object IDs

If an existing service principal is to be used, the Application Object ID must be set on the Vault role. This ID can be found by inspecting the desired Application with the az CLI tool, or via the Azure Portal. Note that the Application Object ID must be provided, not the Application ID.

»Azure Roles

If dynamic service principals are used, Azure roles must be configured on the Vault role. Azure roles are provided as a JSON list, with each element describing an Azure role and scope to be assigned. Azure roles may be specified using the role_name parameter ("Owner"), or role_id ("/subscriptions/.../roleDefinitions/..."). role_id is the definitive ID that's used during Vault operation; role_name is a convenience during role management operations. All roles must exist when the configuration is written or the operation will fail. The role lookup priority is:

  1. If role_id is provided, it is validated and the corresponding role_name updated.
  2. If only role_name is provided, a case-insensitive search-by-name is made, succeeding only if exactly one matching role is found. The role_id field will updated with the matching role ID.

The scope must be provided for every role assignment.

»Azure Groups

If dynamic service principals are used, a list of Azure groups may be configured on the Vault role. When the service principal is created, it will be assigned to these groups. Similar to the format used for specifying Azure roles, Azure groups may be referenced by either their group_name or object_id. Group specification by name must yield a single matching group.

Example of role configuration:

$ vault write azure/roles/my-role \
    ttl=1h \
    max_ttl=24h \
    azure_roles=@az_roles.json \
    azure_groups=@az_groups.json

$ cat az_roles.json
[
  {
    "role_name": "Contributor",
    "scope":  "/subscriptions/<uuid>/resourceGroups/Website"
  },
  {
    "role_id": "/subscriptions/<uuid>/providers/Microsoft.Authorization/roleDefinitions/<uuid>",
    "scope":  "/subscriptions/<uuid>"
  },
  {
    "role_name": "This won't matter as it will be overwritten",
    "role_id": "/subscriptions/<uuid>/providers/Microsoft.Authorization/roleDefinitions/<uuid>",
    "scope":  "/subscriptions/<uuid>/resourceGroups/Database"
  }
]

$ cat az_groups.json
[
  {
    "group_name": "foo",
  },
  {
    "group_name": "This won't matter as it will be overwritten",
    "object_id": "a6a834a6-36c3-4575-8e2b-05095963d603"
  }
]

$ vault write azure/roles/my-role \
    ttl=1h \
    max_ttl=24h \
    azure_roles=@az_roles.json \
    azure_groups=@az_groups.json

$ cat az_roles.json
[
  {
    "role_name": "Contributor",
    "scope":  "/subscriptions/<uuid>/resourceGroups/Website"
  },
  {
    "role_id": "/subscriptions/<uuid>/providers/Microsoft.Authorization/roleDefinitions/<uuid>",
    "scope":  "/subscriptions/<uuid>"
  },
  {
    "role_name": "This won't matter as it will be overwritten",
    "role_id": "/subscriptions/<uuid>/providers/Microsoft.Authorization/roleDefinitions/<uuid>",
    "scope":  "/subscriptions/<uuid>/resourceGroups/Database"
  }
]

$ cat az_groups.json
[
  {
    "group_name": "foo",
  },
  {
    "group_name": "This won't matter as it will be overwritten",
    "object_id": "a6a834a6-36c3-4575-8e2b-05095963d603"
  }
]

»Authentication

The Azure secrets backend must have sufficient permissions to read Azure role information and manage service principals. The authentication parameters can be set in the backend configuration or as environment variables. Environment variables will take precedence. The individual parameters are described in the configuration section of the API docs.

If the client ID or secret are not present and Vault is running on an Azure VM, Vault will attempt to use Managed Service Identity (MSI) to access Azure. Note that when MSI is used, tenant and subscription IDs must still be explicitly provided in the configuration or environment variables.

»MS Graph Permissions

The following Azure permissions should be assigned to the service principal provided to Vault for managing Azure:

Permission NameType
Application.Read.AllApplication
Application.ReadWrite.AllApplication
Application.ReadWrite.OwnedByApplication
Directory.Read.AllApplication
Directory.ReadWrite.AllApplication
Group.Read.AllApplication
Group.ReadWrite.AllApplication
GroupMember.Read.AllApplication
GroupMember.ReadWrite.AllApplication
Permission NameType
Application.Read.AllDelegated
Application.ReadWrite.AllDelegated
Directory.AccessAsUser.AllDelegated
Directory.Read.AllDelegated
Directory.ReadWrite.AllDelegated
Group.Read.AllDelegated
Group.ReadWrite.AllDelegated
GroupMember.Read.AllDelegated
GroupMember.ReadWrite.AllDelegated

Additionally the following IAM roles are required and need to be added to the service principal using the Azure portal:

  • "Owner" role

»AAD (Azure Active Directory)

Microsoft is shutting down their Azure Active Directory API and will be retiring it in 2022. If you are currently using this secret engine, you will need to update the credentials to include Microsoft Graph API permissions and specify the use_microsoft_graph_api configuration value as true. See the API Docs for more details.

As part of the retirement of Azure Active Directory, new service principals can no longer be granted Azure Active Directory permissions using the Azure Portal. To add the neccessary API permissions for AAD, the Azure CLI is suggested. The following commands will add the correct privileges to an existing service principal:

APP_ID='<App ID Here>'

az ad app permission add \
    --id ${APP_ID?} \
    --api 00000002-0000-0000-c000-000000000000 \
    --api-permissions \
        1cda74f2-2616-4834-b122-5cb1b07f8a59=Role \
        78c8a3c8-a07e-4b9e-af1b-b5ccab50a175=Role \
        970d6fa6-214a-4a9b-8513-08fad511e2fd=Scope \
        311a71cc-e848-46a1-bdf8-97ff7156d8e6=Scope

az ad app permission grant --id ${APP_ID?} --api 00000002-0000-0000-c000-000000000000

APP_ID='<App ID Here>'

az ad app permission add \
    --id ${APP_ID?} \
    --api 00000002-0000-0000-c000-000000000000 \
    --api-permissions \
        1cda74f2-2616-4834-b122-5cb1b07f8a59=Role \
        78c8a3c8-a07e-4b9e-af1b-b5ccab50a175=Role \
        970d6fa6-214a-4a9b-8513-08fad511e2fd=Scope \
        311a71cc-e848-46a1-bdf8-97ff7156d8e6=Scope

az ad app permission grant --id ${APP_ID?} --api 00000002-0000-0000-c000-000000000000

Additionally the following IAM roles are required and need to be added to the service principal using the Azure portal:

  • "Owner" role

»Migrating From AAD To Microsoft Graph

In this example we will migrate the Azure secret engine from using Azure Active Directory (AAD) to Microsoft Graph.

First, create a new service principal with the proper permissions for managing Azure accounts. After granting the appropriate permissions, the following will be needed from the service principal to configure the secret engine:

  1. Application (client) ID
  2. Directory (tenant) ID
  3. Client Secret
  4. Subscription ID

Finally, the Azure secret engine configuration can be updated with the new values:

TENANT_ID='<Directory (Tenant) ID Here>'
CLIENT_ID='Application (Client) ID Here>'
CLIENT_SECRET='<Client Secret Here>'
SUBSCRIPTION_ID='<Subcription ID Here>'

vault write azure/config \
        subscription_id=$SUBSCRIPTION_ID  \
        client_id=$CLIENT_ID \
        client_secret=$CLIENT_SECRET \
        tenant_id=$TENANT_ID \
        use_microsoft_graph_api=true

TENANT_ID='<Directory (Tenant) ID Here>'
CLIENT_ID='Application (Client) ID Here>'
CLIENT_SECRET='<Client Secret Here>'
SUBSCRIPTION_ID='<Subcription ID Here>'

vault write azure/config \
        subscription_id=$SUBSCRIPTION_ID  \
        client_id=$CLIENT_ID \
        client_secret=$CLIENT_SECRET \
        tenant_id=$TENANT_ID \
        use_microsoft_graph_api=true

»Choosing between dynamic or existing service principals

Dynamic service principals are preferred if the desired Azure resources can be provided via the RBAC system and Azure roles defined in the Vault role. This form of credential is completely decoupled from any other clients, is not subject to permission changes after issuance, and offers the best audit granularity.

Access to some Azure services cannot be provided with the RBAC system, however. In these cases, an existing service principal can be set up with the necessary access, and Vault can create new passwords for this service principal. Any changes to the service principal permissions affect all clients. Furthermore, Azure does not provide any logging with regard to which credential was used for an operation.

An important limitation when using an existing service principal is that Azure limits the number of passwords for a single Application. This limit is based on Application object size and isn't firmly specified, but in practice hundreds of passwords can be issued per Application. An error will be returned if the object size is reached. This limit can be managed by reducing the role TTL, or by creating another Vault role against a different Azure service principal configured with the same permissions.

»Additional Notes

  • If a referenced Azure role doesn't exist, a credential will not be generated. Service principals will only be generated if all role assignments are successful. This is important to note if you're using custom Azure role definitions that might be deleted at some point.

  • Azure roles are assigned only once, when the service principal is created. If the Vault role changes the list of Azure roles, these changes will not be reflected in any existing service principal, even after token renewal.

  • The time required to issue a credential is roughly proportional to the number of Azure roles that must be assigned. This operation make take some time (10s of seconds are common, and over a minute has been seen).

  • Service principal credential timeouts are not used. Vault will revoke access by deleting the service principal.

  • The Application Name for dynamic service principals will be prefixed with vault-. Similarly the keyId of any passwords added to an existing service principal will begin with ffffff. These may be used to search for Vault-created credentials using the az tool or Portal.

»Help & Support

The Azure secrets engine is written as an external Vault plugin and thus exists outside the main Vault repository. It is automatically bundled with Vault releases, but the code is managed separately.

Please report issues, add feature requests, and submit contributions to the vault-plugin-secrets-azure repo on GitHub.

»Tutorial

Refer to the Azure Secrets Engine tutorial to learn how to use the AWS secrets engine to dynamically generate AWS credentials.

»API

The Azure secrets engine has a full HTTP API. Please see the Azure secrets engine API docs for more details.