|
| 1 | +--- |
| 2 | +type: docs |
| 3 | +linkTitle: KMS |
| 4 | +title: KMS (Key Management Service) |
| 5 | +description: Store secrets distributed across 3 data centers, compatible with existing HashiCorp Vault/Bao ecosystems |
| 6 | +keywords: |
| 7 | +- vault |
| 8 | +- key-value store |
| 9 | +- serverless database |
| 10 | +- hashicorp vault compatible |
| 11 | +- distributed storage |
| 12 | +- high availability |
| 13 | +--- |
| 14 | + |
| 15 | +Customers often asks us for a native solution to store and protect their secrets, in a secure, convenient and resilient way. Clever KMS is our answer to this need. It was built in collaboration with a client with specific security requirements, but we designed it so that it could also meet our own needs and those of any user of Clever Cloud platform. |
| 16 | + |
| 17 | +As with our Materia database offering, it's built on top of the open-source transactional database [FoundationDB](https://www.foundationdb.org/), widely used by Apple to store their metadata, which gives us strong guarantees on data reliability and a full‑fledged transactional model, allowing us to create a serverless multi‑tenant database. We developed compatibility layers with existing ecosystems such as HashiCorp Vault and Bao. |
| 18 | + |
| 19 | +> [!NOTE] Clever KMS is in private access |
| 20 | +> Ask for activation to your sales representative or [Clever Cloud support](https://console.clever-cloud.com/ticket-center-choice) |
| 21 | +
|
| 22 | +## Architecture |
| 23 | + |
| 24 | +Because of the distributed nature of Materia, Clever KMS is always up, which is crucial for a secrets‑management tool. All data in transit is fully encrypted with TLS until it reaches one of the Clever KMS nodes, where the TLS termination occurs. Each node is connected to FoundationDB through [WireGuard](https://www.wireguard.com/). At rest, your data is stored using the state‑of‑the‑art cryptographic cipher [XChaCha20‑Poly1305](https://en.wikipedia.org/wiki/ChaCha20-Poly1305). This ensures that your data is never accessible to third parties, either in transit or at rest. |
| 25 | + |
| 26 | +Powered by Materia and modern cryptography, Clever KMS provides a secure, reliable, and resilient platform able to store your keys, certificates, and sensitive data. |
| 27 | + |
| 28 | +```mermaid |
| 29 | +flowchart LR |
| 30 | +
|
| 31 | + subgraph ClientSide |
| 32 | + C[Client] |
| 33 | + LB[Load Balancer] |
| 34 | + end |
| 35 | +
|
| 36 | + subgraph KMS Cluster |
| 37 | + K1[Clever KMS #1] |
| 38 | + K2[Clever KMS #2] |
| 39 | + K3[Clever KMS #3] |
| 40 | + end |
| 41 | +
|
| 42 | + subgraph FDB |
| 43 | + FDBNode[Cluster FoundationDB] |
| 44 | + end |
| 45 | +
|
| 46 | + %% TLS from Client → LB → KMS Nodes |
| 47 | + C -- TLS --> LB |
| 48 | + LB -- TLS --> K1 |
| 49 | + LB -- TLS --> K2 |
| 50 | + LB -- TLS --> K3 |
| 51 | +
|
| 52 | + %% WireGuard between KMS Nodes and FDB |
| 53 | + K1 -- wireguard --> FDBNode |
| 54 | + K2 -- wireguard --> FDBNode |
| 55 | + K3 -- wireguard --> FDBNode |
| 56 | +``` |
| 57 | + |
| 58 | +## Prerequisites |
| 59 | + |
| 60 | +To use Clever KMS you'll need : |
| 61 | +- An authorized access for your organisation |
| 62 | +- Clever Tools with KMS commands |
| 63 | +- A compatible Vault CLI |
| 64 | + |
| 65 | +To check if you have access to Clever KMS for your organisation, run the following command in your terminal: |
| 66 | + |
| 67 | +```bash |
| 68 | +clever features enable kms |
| 69 | +clever kms --org <your_org_id> |
| 70 | +``` |
| 71 | + |
| 72 | +## Create a Clever KMS add‑on |
| 73 | + |
| 74 | +To create a Kubernetes cluster, use the Console or the following command in Clever Tools: |
| 75 | + |
| 76 | +```bash |
| 77 | +clever addon create kms myKMS --org <your_org_id> |
| 78 | +``` |
| 79 | + |
| 80 | +The add-on is immediately created and you can start using it. You will receive two tokens, based on [Eclipse Biscuit](https://biscuitsec.org/): |
| 81 | + |
| 82 | +- Root token: for administrative tasks |
| 83 | +- User token: for day‑to‑day tasks |
| 84 | + |
| 85 | +You can use HTTP API compatible with Vault, or any compatible client such as [Hashicorp Vault CLI](https://developer.hashicorp.com/vault/docs/commands). Such tools will need the KMS endpoint exported as `VAULT_ADDR` environment variable: |
| 86 | + |
| 87 | +```bash |
| 88 | +export VAULT_ADDR="https://kms.eu-fr-1.services.clever-cloud.com:8200" |
| 89 | +``` |
| 90 | + |
| 91 | +First step is to login with root token and initialize your vault, this will display your unseal shares.: |
| 92 | + |
| 93 | +```bash |
| 94 | +❯ vault login |
| 95 | +❯ vault operator init |
| 96 | + |
| 97 | +Unseal Key 1: AQcEUzVTOmXm8O-lQavD4lCPKRTdXiwQAokb_uA2GjmJ |
| 98 | +Unseal Key 2: AjoHPHK1ixLe5Dr9gV4STroKv_Kk3oHIQX_x-vAmo1xN |
| 99 | +Unseal Key 3: A2GogTjZ617v85dV-C89TihYEV9C5AS8C3n1jz0aZCDC |
| 100 | +Unseal Key 4: BPd1uTtjoel0VU4-Zkq3LSu_Dh89EgUsFKVhP_8fstSt |
| 101 | +Unseal Key 5: BazaBHEPwaVFQuOWHzuYLbntoLLbKIBYXqNlSjIjdagi |
| 102 | + |
| 103 | +Initial Root Token: XKvufz9aKdfnQg042uziwt2HuTtkqWRIjx-LLQrdRQY |
| 104 | + |
| 105 | +Vault initialized with 5 key shares and a key threshold of 3. Please securely |
| 106 | +distribute the key shares printed above. When the Vault is re-sealed, |
| 107 | +restarted, or stopped, you must supply at least 3 of these keys to unseal it |
| 108 | +before it can start servicing requests. |
| 109 | + |
| 110 | +Vault does not store the generated root key. Without at least 3 keys to |
| 111 | +reconstruct the root key, Vault will remain permanently sealed! |
| 112 | + |
| 113 | +It is possible to generate new unseal keys, provided you have a quorum |
| 114 | +of existing unseal keys shares. See "bao operator rotate-keys" for more |
| 115 | +information. |
| 116 | +``` |
| 117 | + |
| 118 | +> [!IMPORTANT] Don't lose unseal keys |
| 119 | +> They are the only way to unseal your vault. If you lose the entire quorum required (here 3), you won't be able to unseal your vault again. |
| 120 | +
|
| 121 | +## Unseal a Clever KMS vault |
| 122 | + |
| 123 | +Once logged in and initialized, if the vault is sealed, unseal it with the required number of unseal keys: |
| 124 | + |
| 125 | +```bash |
| 126 | +vault operator unseal |
| 127 | +Unseal Key (will be hidden): |
| 128 | +``` |
| 129 | + |
| 130 | +This will summarize the state of the vault. |
| 131 | + |
| 132 | +```bash |
| 133 | +Key Value |
| 134 | +--- ----- |
| 135 | +Seal Type n/a |
| 136 | +Initialized true |
| 137 | +Sealed true |
| 138 | +Total Shares 5 |
| 139 | +Threshold 3 |
| 140 | +Unseal Progress 1/3 |
| 141 | +Unseal Nonce n/a |
| 142 | +Version 0.9.2 |
| 143 | +``` |
| 144 | + |
| 145 | +The current progress can be found in the "Unseal Progress" field. You need to repeat the operation two more times to complete the unseal process. Once successful, the output of the last command will display the new "Sealed = false" state: |
| 146 | + |
| 147 | +```bash |
| 148 | +Key Value |
| 149 | +--- ----- |
| 150 | +Seal Type n/a |
| 151 | +Initialized true |
| 152 | +Sealed false |
| 153 | +Total Shares 5 |
| 154 | +Threshold 3 |
| 155 | +Version 0.9.2 |
| 156 | +``` |
| 157 | + |
| 158 | +Your vault is now able to handle requests. At any time, you can seal the vault: |
| 159 | + |
| 160 | +```bash |
| 161 | +vault operator seal |
| 162 | +Success! Vault is sealed. |
| 163 | +``` |
| 164 | + |
| 165 | +## Manage KMS add‑on secrets through Clever Tools |
| 166 | + |
| 167 | +You can manage your KMS add-on secrets directly from Clever Tools: |
| 168 | + |
| 169 | +```bash |
| 170 | +clever kms # List your key-value secrets |
| 171 | +clever kms put mySecret myValue # Create a secret |
| 172 | +clever kms patch mySecret myNewValue # Update a secret |
| 173 | +clever kms destroy mySecret # Destroy a secret |
| 174 | +``` |
| 175 | + |
| 176 | +## Manage KMS add‑on secrets through vault CLI |
| 177 | + |
| 178 | +As Clever KMS is compatible with Vault API and CLI, you can login with user token and use more advanced features: |
| 179 | + |
| 180 | +```bash |
| 181 | +# Create and get a secret |
| 182 | +vault kv put -mount=secret identity_user_1 firstname=John lastname=Doe |
| 183 | +vault kv get -mount=secret identity_user_1 |
| 184 | + |
| 185 | +# Update a secret (creates a new version) |
| 186 | +vault kv put -mount=secret identity_user_1 firstname=John lastname=Doe phone=666 |
| 187 | +vault kv get -version=1 -mount=secret identity_user_1 |
| 188 | + |
| 189 | +# You can mark the latest version of the secret as "deleted" |
| 190 | +# This doesn't remove it from storage but makes it invisible to get requests |
| 191 | +vault kv delete -mount=secret identity_user_1 |
| 192 | + |
| 193 | +# Now the remaining version is version 1 |
| 194 | +vault kv get -mount=secret identity_user_1 |
| 195 | + |
| 196 | +# Repeating the operation again will make the secret disappear from the get API |
| 197 | +vault kv delete -mount=secret identity_user_1 |
| 198 | +# If you try to get it, response will be "No value found at secret/data/identity_user_1" |
| 199 | +vault kv get -mount=secret identity_user_1 |
| 200 | + |
| 201 | +# To reverse the operation, you can undelete a version |
| 202 | +vault kv undelete -mount=secret -versions=2 identity_user_1 |
| 203 | +# This will restore this specific version but not others |
| 204 | +vault kv get -version=1 -mount=secret identity_user_1 |
| 205 | +vault kv get -mount=secret identity_user_1 |
| 206 | + |
| 207 | +# Destroy a secret, it won't be recoverable |
| 208 | +vault kv destroy -mount=secret -versions=1 identity_user_1 |
| 209 | +vault kv undelete -mount=secret -versions=1 identity_user_1 |
| 210 | + |
| 211 | +# Version 1 can't be restored, but version 2 is still available and readable. |
| 212 | +vault kv get -mount=secret -version=1 identity_user_1 |
| 213 | +vault kv get -mount=secret -version=2 identity_user_1 |
| 214 | +``` |
| 215 | + |
| 216 | +## Transit secrets |
| 217 | + |
| 218 | +You can create a secret and use it as a symmetric key to encrypt and decrypt data in transit. |
| 219 | + |
| 220 | +```bash |
| 221 | +❯ vault write -f transit/keys/my-key |
| 222 | +Success! Data written to: transit/keys/my-key |
| 223 | +``` |
| 224 | + |
| 225 | +Once the transit key `my-key` is created, it can be used to encrypt data. All plaintext data must be base64‑encoded, as your payload can be any binary data, not only text but also files (PDF, certificates, etc.): |
| 226 | + |
| 227 | +```bash |
| 228 | +❯ vault write transit/encrypt/my-key plaintext=$(echo "my secret data" | base64) |
| 229 | +Key Value |
| 230 | +--- ----- |
| 231 | +ciphertext vault:v1:ZsBtzrDN-aAWxeQ_phnFi64UI44k7ntdWqoK7jKFt7GEz3HUaxXVYTzKevS2HlosSWWN91Uv_aDFF4ID |
| 232 | +``` |
| 233 | + |
| 234 | +The ciphertext isn't the same between two calls with the same payload, making it non‑predictable: |
| 235 | + |
| 236 | +```bash |
| 237 | +❯ vault write transit/encrypt/my-key plaintext=$(echo "my secret data" | base64) |
| 238 | +Key Value |
| 239 | +--- ----- |
| 240 | +ciphertext vault:v1:75Le-Zs1fzJP-_l-vPdePoabvNXYGsBtcj0Ii4LcJ_eNHeImfNWz1aAH82uyPcGgyfExcyNk5Ds9lW-C |
| 241 | +``` |
| 242 | + |
| 243 | +The ciphertext isn't stored by Clever KMS; it must be stored by the client in its own datastore (not necessarily secured), along with the transit key name used to generate the ciphertext. |
| 244 | + |
| 245 | +### Decrypt ciphertext |
| 246 | + |
| 247 | +To get back the original payload, call the decrypt route with the ciphertext and the transit key name: |
| 248 | +```bash |
| 249 | +❯ vault write transit/decrypt/my-key ciphertext=vault:v1:75Le-Zs1fzJP-_l-vPdePoabvNXYGsBtcj0Ii4LcJ_eNHeImfNWz1aAH82uyPcGgyfExcyNk5Ds9lW-C |
| 250 | +Key Value |
| 251 | +--- ----- |
| 252 | +plaintext bXkgc2VjcmV0IGRhdGEK |
| 253 | + |
| 254 | +❯ base64 --decode <<< "bXkgc2VjcmV0IGRhdGEK" |
| 255 | +my secret data |
| 256 | +``` |
| 257 | + |
| 258 | +Decryption is stable even if the ciphertext looks totally different. |
| 259 | + |
| 260 | +```bash |
| 261 | +❯ vault write transit/decrypt/my-key ciphertext=vault:v1:ZsBtzrDN-aAWxeQ_phnFi64UI44k7ntdWqoK7jKFt7GEz3HUaxXVYTzKevS2HlosSWWN91Uv_aDFF4ID |
| 262 | +Key Value |
| 263 | +--- ----- |
| 264 | +plaintext bXkgc2VjcmV0IGRhdGEK |
| 265 | +``` |
| 266 | + |
| 267 | +With a non‑existent key: |
| 268 | + |
| 269 | +```bash |
| 270 | +❯ vault write transit/decrypt/not-existing ciphertext=vault:v1:ZsBtzrDN-aAWxeQ_phnFi64UI44k7ntdWqoK7jKFt7GEz3HUaxXVYTzKevS2HlosSWWN91Uv_aDFF4ID |
| 271 | +Error writing data to transit/decrypt/not-existing: Error making API request. |
| 272 | + |
| 273 | +URL: PUT http://localhost:8202/v1/transit/decrypt/not-existing |
| 274 | +Code: 400. Raw Message: |
| 275 | + |
| 276 | +"Unable to decrypt the secret" |
| 277 | +``` |
| 278 | + |
| 279 | +With an existing but different key: |
| 280 | + |
| 281 | +```bash |
| 282 | +❯ vault write transit/decrypt/my-key2 ciphertext=vault:v1:ZsBtzrDN-aAWxeQ_phnFi64UI44k7ntdWqoK7jKFt7GEz3HUaxXVYTzKevS2HlosSWWN91Uv_aDFF4ID |
| 283 | +Error writing data to transit/decrypt/my-key2: Error making API request. |
| 284 | + |
| 285 | +URL: PUT http://localhost:8202/v1/transit/decrypt/my-key2 |
| 286 | +Code: 400. Raw Message: |
| 287 | + |
| 288 | +"Unable to decrypt the secret" |
| 289 | +``` |
| 290 | + |
| 291 | +This leaves very little hint for an enumeration attack. |
| 292 | + |
| 293 | +> [!IMPORTANT] Keep your transit key name safe |
| 294 | +> If you lose it, your won't be able to decrypt ciphertext |
| 295 | +
|
| 296 | +### Rotate transit key |
| 297 | + |
| 298 | +You may want to rotate transit keys for security reasons. This will create a new version of the key. If your key was created at version 1: |
| 299 | + |
| 300 | +```bash |
| 301 | +❯ vault write transit/encrypt/my-key plaintext=$(echo "my secret data" | base64) |
| 302 | +Key Value |
| 303 | +--- ----- |
| 304 | +ciphertext vault:v1:xLmLv2_uegkD0PSus6Efcp6Ie7CFNlU9wRPj0oXHb6rgtqRXa2FURPF_I3aYZay1td0LsjaDh4d2GsSz |
| 305 | +``` |
| 306 | + |
| 307 | +The resulting ciphertext will be prefixed by "vault:v1". |
| 308 | + |
| 309 | +If you rotate the transit key: |
| 310 | + |
| 311 | +```bash |
| 312 | +❯ vault write -f transit/keys/my-key/rotate |
| 313 | +Success! Data written to: transit/keys/my-key/rotate |
| 314 | +``` |
| 315 | + |
| 316 | +Your next call to the encrypt route with the rotated transit key, it will create a ciphertext prefixed by "vault:v2". |
| 317 | + |
| 318 | +```bash |
| 319 | +❯ vault write transit/encrypt/my-key plaintext=$(echo "my secret data" | base64) |
| 320 | +Key Value |
| 321 | +--- ----- |
| 322 | +ciphertext vault:v2:GHyO4zNEHVr7ERW0FSwpFbdMDgQClX7qDekRNsngUd5r5hDW8pFGbQfHjDV-jgjRz7u1gECsoYe5Fw4C |
| 323 | +``` |
| 324 | + |
| 325 | +Both encrypted ciphertexts are decryptable, even if the key has been rotated. |
| 326 | + |
| 327 | +```bash |
| 328 | +❯ vault write transit/decrypt/my-key ciphertext=vault:v1:xLmLv2_uegkD0PSus6Efcp6Ie7CFNlU9wRPj0oXHb6rgtqRXa2FURPF_I3aYZay1td0LsjaDh4d2GsSz |
| 329 | +Key Value |
| 330 | +--- ----- |
| 331 | +plaintext bXkgc2VjcmV0IGRhdGEK |
| 332 | + |
| 333 | +❯ vault write transit/decrypt/my-key ciphertext=vault:v2:GHyO4zNEHVr7ERW0FSwpFbdMDgQClX7qDekRNsngUd5r5hDW8pFGbQfHjDV-jgjRz7u1gECsoYe5Fw4C |
| 334 | +Key Value |
| 335 | +--- ----- |
| 336 | +plaintext bXkgc2VjcmV0IGRhdGEK |
| 337 | +``` |
| 338 | + |
| 339 | +This allows you to smoothly handle the lifecycle of your secrets. |
| 340 | + |
| 341 | +### Rewrap ciphertext with the latest transit key version |
| 342 | + |
| 343 | +Rewrapping involves decrypting the v1 ciphertext and then encrypting to v2. |
| 344 | + |
| 345 | +```bash |
| 346 | +❯ vault write transit/rewrap/my-key ciphertext=vault:v1:xLmLv2_uegkD0PSus6Efcp6Ie7CFNlU9wRPj0oXHb6rgtqRXa2FURPF_I3aYZay1td0LsjaDh4d2GsSz |
| 347 | +Key Value |
| 348 | +--- ----- |
| 349 | +ciphertext vault:v2:JwgZBtUnHD9aJWDRlYz4s_RTt-Gw4nOiBT2PJc2Gxc12zp_YrZBVlEiSl00kzjYQqGJp8entrgFN0nrl |
| 350 | +``` |
| 351 | + |
| 352 | +This rewrapped secret is now decrypted with version 2 of the transit key. |
| 353 | + |
| 354 | +```bash |
| 355 | +❯ vault write transit/decrypt/my-key ciphertext=vault:v2:JwgZBtUnHD9aJWDRlYz4s_RTt-Gw4nOiBT2PJc2Gxc12zp_YrZBVlEiSl00kzjYQqGJp8entrgFN0nrl |
| 356 | +Key Value |
| 357 | +--- ----- |
| 358 | +plaintext bXkgc2VjcmV0IGRhdGEK |
| 359 | +``` |
0 commit comments