Vault is an open-source tool that provides a secure, reliable way to store and distribute secrets like API keys, access tokens, and passwords. Software like Vault can be critically important when deploying applications that require the use of secrets or sensitive data.
https://learn.hashicorp.com/tutorials/vault/getting-started-intro?in=vault/getting-started
In this tutorial, we will:
- Install Vault and configure it as a system service
- Initialize an encrypted on-disk data store
- Store and retrieve a sensitive value using a dedicate account/policy
As with any service that manages sensitive information, you should consider reading additional documentation regarding Vault’s deployment best practices before using it in a production environment.
Vault’s production hardening guide covers topics such as policies, root tokens, and auditing.
Requirements : Debian (>9) or Ubuntu (>18) distribution, curl and unzip package installed
sudo apt-get update && sudo apt-get install curl unzip
Step 1 – Installing Vault
there are two different approaches to install Vault : oldschool or lazy.
Oldschool approach :
Go to Vault download page, to get latest version number, or at the time of writing, you can get it with following command.
vault_v=$(curl -s https://releases.hashicorp.com/vault/ | grep vault -m1 | sed -r 's/<a href=[^>]+>//g;s/<\/a>//g' | sed 's/vault_//' | xargs)
Change export variable with $vault_v (export VAULT_VERSION=”$vault_v”) or the version number found on the website (eg 1.6.0)
export VAULT_VERSION="1.6.0"
export VAULT_URL="https://releases.hashicorp.com/vault"
curl --remote-name ${VAULT_URL}/${VAULT_VERSION}/vault_${VAULT_VERSION}_linux_amd64.zip
curl --remote-name ${VAULT_URL}/${VAULT_VERSION}/vault_${VAULT_VERSION}_SHA256SUMS
Verify the integrity of zip archive :
grep vault_${VAULT_VERSION}_linux_amd64.zip vault_${VAULT_VERSION}_SHA256SUMS | sha256sum -c -
Each line in the SHA256SUMS file has a checksum and a filename, one for each zip archive that HashiCorp provides.
The grep portion of the above command prints the line with the checksum and filename of the 64-bit Linux binary, then pipes (|) that line to the next command. The SHA-256 command checks, -c, that the file with the filename from that line matches the checksum from that line.
Running the command should indicate the archive is OK.
With the checksum verification complete, unzip the downloaded package and move the vault binary to /usr/bin/. Check vault is available on the system path.
unzip vault_${VAULT_VERSION}_linux_amd64.zip
sudo chown root:root vault
sudo mv vault /usr/bin/
Set a Linux capability flag on the binary. This adds extra security by letting the binary perform memory locking without unnecessarily elevating its privileges.
sudo setcap cap_ipc_lock=+ep /usr/bin/vault
Enable autocomplete feature
vault -autocomplete-install complete -C /usr/bin/vault vault
Lazy approach :
curl -fsSL https://apt.releases.hashicorp.com/gpg | sudo apt-key add - sudo apt-add-repository "deb [arch=amd64] https://apt.releases.hashicorp.com $(lsb_release -cs) main" sudo apt-get update && sudo apt-get install vault
You can now use the vault command. Try checking Vault’s version to make sure it works.
vault --version
The Vault executable is now installed on your server, so the next step is to configure it to run as a system service.
Step 2 – Creating the Vault Unit File
Systemd is a system and service manager. In order to start Vault as a system service, we need to set up the following things:
- A system user for the Vault daemon to run as
- A data directory to store Vault’s information
- Vault’s configuration file
- The systemd unit file itself.
Please note that we’re using the filesystem backend to store encrypted secrets on the local filesystem at /var/lib/vault. This is suitable for local or single-server deployments that do not need to be replicated. Other Vault backends, such as the Consul backend, will store encrypted secrets at rest within a distributed key/value store.
Create a vault system user.
sudo useradd -r -d /var/lib/vault -s /bin/nologin vault
Here, we use /var/lib/vault as the user’s home directory. This will be used as the Vault data directory. We also set the shell to /bin/nologin to restrict the user as a non-interactive system account.
You may have expected error : useradd: Warning: missing or non-executable shell '/bin/nologin'
Set the ownership of /var/lib/vault to the vault user and the vault group exclusively.
sudo install -o vault -g vault -m 750 -d /var/lib/vault
Now let’s set up Vault’s configuration file, /etc/vault.hcl.
Create vault.hcl using nano or your favorite text editor.
sudo nano /etc/vault.hcl
Paste the following into the file, and make sure to substitute your IP or domain name. You can also set 127.0.0.1 : we are installing a standalone server, and this service is listenning on its own network instance.
# We enable Vault's UI
ui=true
# Backend location
backend "file" {
path = "/var/lib/vault"
}
listener "tcp" {
tls_disable = 1
}
# Change IP with the host's IP address
api_addr = "http://IP_or_SERVERNAME:8200"
cluster_addr = "http://IP_or_SERVERNAME:8200"
Save and close the file, then secure the Vault configuration file’s permissions by only allowing the vault user to read it.
sudo chown vault:vault /etc/vault.hcl sudo chmod 640 /etc/vault.hcl
Next, to let Systemd manage the persistent Vault daemon, create a unit file at /etc/systemd/system/vault.service.
sudo nano /etc/systemd/system/vault.service
Copy and paste the following into the file. This allows Vault to run in the background as a persistent system service daemon.
[Unit] Description="Vault - A tool for managing secrets" Documentation=https://www.vaultproject.io/docs/ Requires=network-online.target After=network-online.target ConditionFileNotEmpty=/etc/vault.hcl StartLimitIntervalSec=60 StartLimitBurst=3 [Service] User=vault Group=vault ProtectSystem=full ProtectHome=read-only PrivateTmp=yes PrivateDevices=yes SecureBits=keep-caps AmbientCapabilities=CAP_IPC_LOCK Capabilities=CAP_IPC_LOCK+ep CapabilityBoundingSet=CAP_SYSLOG CAP_IPC_LOCK NoNewPrivileges=yes ExecStart=/usr/bin/vault server -config=/etc/vault.hcl ExecReload=/bin/kill --signal HUP $MAINPID KillMode=process KillSignal=SIGINT Restart=on-failure RestartSec=5 TimeoutStopSec=30 LimitNOFILE=65536 LimitMEMLOCK=infinity [Install] WantedBy=multi-user.target
The complete list of service unit options is extensive, but the most important configuration options to note are:
- ConditionFileNotEmpty ensures that the /etc/vault.hcl configuration file exists.
- User and Group, which control the user permissions that the Vault daemon will run with.
- ExecStart, which points to the executable that we installed previously and defines what to start to run the service.
- ExecReload, which is called when Vault reloads its configuration file, e.g., when running systemctl reload vault.
- [Install], which lets us run this service persistently at startup so we don’t need to start it manually after reboots.
Step 3 – Initializing Vault
When you first start Vault, it will be uninitialized, which means that it is not ready to get and store data.
sudo systemctl start vault
You can run a quick check to confirm the service has started successfully.
sudo systemctl status vault
user@labs:/# sudo systemctl status vault
● vault.service - "Vault - A tool for managing secrets"
Loaded: loaded (/etc/systemd/system/vault.service; disabled; vendor preset: enabled)
Active: active (running) since Fri 2020-12-04 15:15:53 GMT; 2s ago
Docs: https://www.vaultproject.io/docs/
Main PID: 1899 (vault)
Tasks: 8 (limit: 544)
Memory: 124.2M
CGroup: /system.slice/vault.service
└─1899 /usr/bin/vault server -config=/etc/vault.hcl
Dec 04 15:15:53 chef-node vault[1899]: Go Version: go1.15.4
Dec 04 15:15:53 chef-node vault[1899]: Listener 1: tcp (addr: "127.0.0.1:8200", cluster address: "127.0.0.1:8201", max_request_duration: "1m30s", max_request_s
Dec 04 15:15:53 chef-node vault[1899]: Log Level: info
Dec 04 15:15:53 chef-node vault[1899]: Mlock: supported: true, enabled: true
Dec 04 15:15:53 chef-node vault[1899]: Recovery Mode: false
Dec 04 15:15:53 chef-node vault[1899]: Storage: file
Dec 04 15:15:53 chef-node vault[1899]: Version: Vault v1.6.0
Dec 04 15:15:53 chef-node vault[1899]: Version Sha: 7ce0bd9691998e0443bc77e98b1e2a4ab1e965d4
Dec 04 15:15:53 chef-node vault[1899]: ==> Vault server started! Log data will stream in below:
Dec 04 15:15:53 chef-node vault[1899]: 2020-12-04T15:15:53.839Z [INFO] proxy environment: http_proxy= https_proxy= no_proxy=
The output of that command should include several pieces of information : process ID, usage, state…
If the service is not active, take a look at the log lines at the end of the command’s output.
Next, we’ll set an environment variable to tell to the vault command how to connect to the Vault server. Here, Vault has been configured to listen on the host interface only, so set the VAULT_ADDR environment variable to the host HTTP endpoint.
export VAULT_ADDR=http://IP_or_SERVERNAME:8200
Confirm that the vault is in an uninitialized state by checking its status
vault status
user@labs:/# vault status Key Value --- ----- Seal Type shamir Initialized false Sealed true Total Shares 0 Threshold 0 Unseal Progress 0/0 Unseal Nonce n/a Version 1.6.0 Storage Type file HA Enabled false
There are two pieces of information that Vault will expose at initialization time that will not be available at any other point:
- Initial root token. This is equivalent to root permissions to your Vault deployment, which allows the management of all Vault policies, mounts, and so on.
- Unseal keys. These are used to unseal Vault when the daemon starts, which permits the Vault daemon to decrypt the backend secret store.
More specifically, Vault’s unsealing process decrypts the backend using a key formed by key shares. That being said, when initializing Vault, you may choose how many unseal keys to create and how many are necessary to successfully unseal Vault.
A typical, simple value for the unseal parameters would be to create three keys and require at least two of those keys at unseal time. This permits the important key shares to be separated and stored in distinct locations to ensure that compromising one is not sufficient to unseal Vault.
In other words, whenever Vault is started, at least two unseal keys will be required in order to make the service become available and ready to use. While sealed, the files that store the actual secret values will remain encrypted and inaccessible.
In this lab, we are keeping things simple, and use a single unseal key. Initialize Vault with following parameters:
vault operator init -key-shares=1 -key-threshold=1
user@labs:/# vault operator init -key-shares=1 -key-threshold=1 Unseal Key 1: ft6lFO4kd2BrLP8CyzZsoiwkNWIEdp98E+mCIp+a5AM= Initial Root Token: s.dMm9mFKw0VOYWagDaTVoi5jy Vault initialized with 1 key shares and a key threshold of 1. Please securely distribute the key shares printed above. When the Vault is re-sealed, restarted, or stopped, you must supply at least 1 of these keys to unseal it before it can start servicing requests. Vault does not store the generated master key. Without at least 1 key to reconstruct the master key, Vault will remain permanently sealed! It is possible to generate new unseal keys, provided you have a quorum of existing unseal keys shares. See "vault operator rekey" for more information.
Save unseal token and the initial root token in a secure way. For example, one option would be to store one unseal key in a password manager, another on a USB drive, and another in GPG-encrypted file.
Now unseal Vault using the newly created unseal token.
vault operator unseal
The command will ask for an unseal token:
user@labs:/# vault operator unseal Key (will be hidden):
[ Repeat unseal process, as many as key threshold you configured ]
The command’s output indicates that the unseal process completed successfully.
user@labs:/# vault status Unseal Key (will be hidden): Key Value --- ----- Seal Type shamir Initialized true Sealed false Total Shares 1 Threshold 1 Version 1.6.0 Storage Type file Cluster Name vault-cluster-04eeecd7 Cluster ID 44e3d50d-0ada-4fa0-7116-ad5fb2120295 HA Enabled false
Vault is now unsealed and ready for use.
These unseal steps are necessary whenever Vault is started or restarted.
Please not that unsealing is a distinct process from normal interaction with Vault (reading and writing values), which are authenticated by tokens. In the last step, we’ll create the necessary access tokens and policies to store secret values and read/write to specific paths in Vault.
We can now login and authenticate with initial root token provided
vault login
user@labs:/# vault login Token (will be hidden): Success! You are now authenticated. The token information displayed below is already stored in the token helper. You do NOT need to run "vault login" again. Future Vault requests will automatically use this token. Key Value --- ----- token s.dMm9mFKw0VOYWagDaTVoi5jy token_accessor WfTqkmitfX7Wn1FsmWUeuMzg token_duration ∞ token_renewable false token_policies ["root"] identity_policies [] policies ["root"]
Step 4 – Reading and Writing Secrets
There are several secret backends enumerated in the Vault documentation, but for this example we will use the generic secret backend. This backend stores simple key/value pairs in Vault.
To begin, we will enable a new Key/Value secret path, called “secret” and write a value to this path within Vault. We write an additional value for control access testing.
vault secrets enable -path=secret kv-v1 vault write secret/message value=mypassword vault write secret/testpolicy value=confidential
In this command, the secret/ prefix indicates that we are writing to the KV (Key Value) backend mounted at the secret path, and we are storing the key value at the path message with the value mypassword. We use the root token, which has superuser privileges, to write the generic secret. We are also using KV backend version 1. Version 2 brings secrets versionning support and updated ACL policy.
In a production scenario, you may store values like API keys or passwords. Although you may read the secret value using the root token (vault read secret/message), it is illustrative to generate a less privileged token with read-only permissions to our single secret.
Create a file called policy.hcl
nano policy.hcl
Populate the file with the following Vault policy, which defines read-only access to the secret path in your working directory:
path "secret/message" {
capabilities = ["read"]
}
Save and close the file. The following command will create a policy named readonly-policy with the rights of the policy.
vault policy write readonly-policy policy.hcl
You can now create a token with the rights specified in the policy.
vault token create -policy="readonly-policy"
The output will look like this:
root@labs:/# vault token create -policy="readonly-policy" Key Value --- ----- token s.vyNbGK4yzy1q22UnpgZexIGw token_accessor CItApu1eOqLW3R9z2Y7YzPZl token_duration 768h token_renewable true token_policies ["default" "readonly-policy"] identity_policies [] policies ["default" "readonly-policy"]
Now login with this new token
vault login
root@labs:/# vault login Token (will be hidden): Success! You are now authenticated. The token information displayed below is already stored in the token helper. You do NOT need to run "vault login" again. Future Vault requests will automatically use this token. Key Value --- ----- token s.QhUetY4yrokvT9Z6ZKBd1xM7 token_accessor CItApu1eOqLW3R9z2Y7YzPZl token_duration 767h50m3s token_renewable true token_policies ["default" "readonly-policy"] identity_policies [] policies ["default" "readonly-policy"]
You can use the value of readonly_token to access the data stored in the path secret/message (and no other secrets in Vault).
vault read secret/message
root@labs:/# vault read secret/message Key Value --- ----- refresh_interval 768h value mypassword
You can also test that this unprivileged token cannot perform other operations.
vault read secret/testpolicy
Output:
root@labs:/# vault read secret/testpolicy Error reading secret2/testpolicy: Error making API request. URL: GET http://127.0.0.1:8200/v1/secret2/testpolicy Code: 403. Errors: * 1 error occurred: * permission denied
This confirms that the less-privileged token cannot perform actions except from those explicitly stated in its Vault policy.
Conclusion
Job done ! You have now a working installation of Vault.
You can read Vault documentation to get additional information about additional ways to store and access secrets as well as alternative authentication methods.
This is a basic installation, perfect for labs or home as this installation is persistent.
Reference
https://learn.hashicorp.com/tutorials/vault/getting-started-install?in=vault/getting-started