BYOK Encryption Configuration

BYOK (Bring Your Own Key) lets organizations encrypt audit log fields with their own Key Management Service (KMS) keys instead of relying on platform-managed encryption. When enabled, the platform uses envelope encryption: a per-org Data Encryption Key (DEK) encrypts audit data locally, and the customer’s KMS key wraps (encrypts) the DEK.

For the technical design — KMS abstraction layer, EnvelopeEncryptor, DEK caching, and fail-closed/fail-open semantics — see Security Architecture: BYOK Envelope Encryption.

For a quick overview of the admin API, see Admin Operations: BYOK Encryption.

---

How BYOK works

When BYOK is enabled for an organization:

1. The platform calls generate_dek() on the configured KMS provider to obtain a plaintext DEK and a wrapped (encrypted) copy.
2. Audit log fields are encrypted locally using the plaintext DEK with AES-256-GCM (12-byte nonce, nonce-prefixed ciphertext).
3. The ciphertext and wrapped DEK are stored in the audit_logs table. The plaintext DEK is cached in memory only — never persisted to disk or database.
4. To read encrypted audit data, the platform sends the wrapped DEK to the KMS for unwrapping, then decrypts locally.

Non-BYOK organizations are completely unaffected — the existing plaintext audit path is unchanged.

---

Prerequisites

Before enabling BYOK:

• The platform must have network connectivity to the KMS endpoint.
• The KMS key must have permissions for encrypt, decrypt, and generate-data-key operations.
• The authentication method (managed identity, service principal, IAM role, or access key) must be configured on the platform host.
• You need platform admin privileges (require_admin role).

---

Supported KMS providers

Provider	kms_type value	Authentication methods	Status
Azure Key Vault	vault	managed_identity, service_principal, token	Phase B
AWS KMS	aws_kms	IAM role, access key	Phase B
HashiCorp Vault	hashicorp_vault	Token, AppRole	Planned
GCP Cloud KMS	gcp_kms	Service account, workload identity	Planned
Mock (testing)	mock	None	Available

Note

Azure Key Vault and AWS KMS providers are defined in the KMS abstraction layer but not yet production-implemented (Phase B). The mock provider is available for integration testing and development. HashiCorp Vault and GCP Cloud KMS are on the roadmap.

---

Enable BYOK for an organization

1. Choose a KMS provider from the table above and ensure your KMS key is provisioned.
2. Create the KMS configuration for the org via the API or Cloud portal.
3. Validate connectivity to confirm the platform can reach the KMS and perform encrypt/decrypt operations.
4. BYOK activates automatically on the next audit event after a successful validation.

Create KMS configuration

curl -s -X POST \
  "https://platform.example.com/api/v1/admin/orgs/${ORG_ID}/kms" \
  -H "Authorization: Bearer ${ADMIN_TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{
    "kms_type": "vault",
    "endpoint": "https://myorg-vault.vault.azure.net",
    "key_id": "arbitex-audit-key",
    "auth_method": "managed_identity",
    "enabled": true
  }'

Request fields:

Field	Type	Required	Description
kms_type	string	Yes	Provider type — see table above
endpoint	string	Yes	KMS endpoint URL
key_id	string	Yes	Key identifier within the KMS
auth_method	string	Yes	Authentication method for the provider
enabled	boolean	No	Whether encryption is active (default: true)

The configuration is stored in the customer_kms_configs table with a unique index on org_id — each organization can have at most one KMS configuration.

Validate connectivity

After creating the configuration, validate that the platform can reach and use the KMS:

curl -s -X POST \
  "https://platform.example.com/api/v1/admin/orgs/${ORG_ID}/kms/validate" \
  -H "Authorization: Bearer ${ADMIN_TOKEN}"

The validation runs a full test cycle: generate a DEK, encrypt sample data, then decrypt and verify the result matches the original. A successful response confirms the KMS is reachable and the key has the required permissions.

Success response (200 OK):

{
  "status": "valid",
  "provider": "vault",
  "key_id": "arbitex-audit-key",
  "latency_ms": 45
}

Failure response (422):

{
  "status": "invalid",
  "error": "KMS connection failed: timeout after 5000ms",
  "provider": "vault"
}

---

Provider configuration

Azure Key Vault

Prerequisites:

• An Azure Key Vault instance with a key created (RSA or EC).
• The Arbitex platform’s managed identity (or service principal) must have Key Encrypt, Key Decrypt, and Key Unwrap permissions on the key.

Configuration:

curl -s -X POST \
  "https://platform.example.com/api/v1/admin/orgs/${ORG_ID}/kms" \
  -H "Authorization: Bearer ${ADMIN_TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{
    "kms_type": "vault",
    "endpoint": "https://myorg-vault.vault.azure.net",
    "key_id": "arbitex-audit-key",
    "auth_method": "managed_identity",
    "enabled": true
  }'

Auth methods:

Method	auth_method	When to use
Managed identity	managed_identity	Recommended for Azure-hosted deployments — no credentials to manage
Service principal	service_principal	Non-Azure hosts or multi-tenant scenarios
Token	token	Short-lived access tokens for testing or migration

For Kubernetes deployments using Azure workload identity, use managed_identity — the workload identity federation maps to a managed identity transparently.

AWS KMS

Prerequisites:

• An AWS KMS key (symmetric, ENCRYPT_DECRYPT usage).
• The platform’s IAM role or access key must have kms:Encrypt, kms:Decrypt, and kms:GenerateDataKey permissions on the key ARN.

Configuration:

curl -s -X POST \
  "https://platform.example.com/api/v1/admin/orgs/${ORG_ID}/kms" \
  -H "Authorization: Bearer ${ADMIN_TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{
    "kms_type": "aws_kms",
    "endpoint": "https://kms.us-east-1.amazonaws.com",
    "key_id": "arn:aws:kms:us-east-1:123456789012:key/mrk-abc123",
    "auth_method": "iam_role",
    "enabled": true
  }'

For cross-region disaster recovery, use an AWS multi-region key (MRK) so replicas can decrypt in any region.

Mock (testing)

The mock provider performs in-memory encrypt/decrypt without any external KMS. Use it for integration testing and development environments.

curl -s -X POST \
  "https://platform.example.com/api/v1/admin/orgs/${ORG_ID}/kms" \
  -H "Authorization: Bearer ${ADMIN_TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{
    "kms_type": "mock",
    "endpoint": "mock://localhost",
    "key_id": "test-key",
    "auth_method": "none",
    "enabled": true
  }'

Caution

Never use the mock provider in production. Mock-encrypted audit entries cannot be decrypted if the platform restarts (the in-memory key is lost).

---

View and update configuration

View current configuration:

curl -s -X GET \
  "https://platform.example.com/api/v1/admin/orgs/${ORG_ID}/kms" \
  -H "Authorization: Bearer ${ADMIN_TOKEN}"

Response:

{
  "org_id": "org_01j...",
  "kms_type": "vault",
  "endpoint": "https://myorg-vault.vault.azure.net",
  "key_id": "arbitex-audit-key",
  "auth_method": "managed_identity",
  "enabled": true,
  "created_at": "2026-03-15T10:30:00Z",
  "updated_at": "2026-03-15T10:30:00Z"
}

Update configuration:

curl -s -X PUT \
  "https://platform.example.com/api/v1/admin/orgs/${ORG_ID}/kms" \
  -H "Authorization: Bearer ${ADMIN_TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{
    "kms_type": "vault",
    "endpoint": "https://myorg-vault.vault.azure.net",
    "key_id": "arbitex-audit-key-v2",
    "auth_method": "managed_identity",
    "enabled": true
  }'

Disable BYOK without removing configuration:

curl -s -X PUT \
  "https://platform.example.com/api/v1/admin/orgs/${ORG_ID}/kms" \
  -H "Authorization: Bearer ${ADMIN_TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{
    "kms_type": "vault",
    "endpoint": "https://myorg-vault.vault.azure.net",
    "key_id": "arbitex-audit-key",
    "auth_method": "managed_identity",
    "enabled": false
  }'

When enabled is set to false, new audit events are stored in plaintext. Existing encrypted entries remain encrypted and readable as long as the KMS is reachable.

---

Key rotation

To rotate the KMS key:

1. Create a new key version in your KMS (Azure Key Vault auto-versions; AWS KMS uses key rotation policies).
2. Update the Arbitex KMS configuration with the new key_id.
3. Validate the new key.

# Update to the new key version
curl -s -X PUT \
  "https://platform.example.com/api/v1/admin/orgs/${ORG_ID}/kms" \
  -H "Authorization: Bearer ${ADMIN_TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{
    "kms_type": "vault",
    "endpoint": "https://myorg-vault.vault.azure.net",
    "key_id": "arbitex-audit-key-v2",
    "auth_method": "managed_identity",
    "enabled": true
  }'

# Validate the new key works
curl -s -X POST \
  "https://platform.example.com/api/v1/admin/orgs/${ORG_ID}/kms/validate" \
  -H "Authorization: Bearer ${ADMIN_TOKEN}"

Key rotation is non-disruptive:

• The platform generates a new DEK on the next audit event using the new key.
• Existing encrypted entries remain readable — each entry stores the kms_key_ref used for wrapping, so the platform knows which key version to use for decryption.
• The DEK cache (1-hour TTL, 256-org LRU) flushes naturally. After the cache entry expires, new DEKs are generated with the new key.

Tip

For Azure Key Vault, enable automatic key rotation in the vault settings. When the key auto-rotates, the latest version is used automatically — you only need to update the Arbitex key_id if the key name changes.

---

Remove BYOK

To remove BYOK and revert to platform-managed encryption:

curl -s -X DELETE \
  "https://platform.example.com/api/v1/admin/orgs/${ORG_ID}/kms" \
  -H "Authorization: Bearer ${ADMIN_TOKEN}"

Caution

Removing a KMS configuration does not decrypt existing audit entries. They remain encrypted and will show "[encrypted - KMS unavailable]" in exports and the admin UI. To restore access to historical encrypted entries, re-create the KMS configuration with the same key.

---

Cloud portal

The Cloud portal provides a visual KMS configuration interface at Settings > Encryption (/portal/encryption). The portal supports:

• Status badge: connected / not configured / error
• Provider selection: dropdown with supported provider types
• Connection details: endpoint URL, key ID, authentication method
• Validate button: runs a test encrypt/decrypt cycle
• Enable/disable toggle: activates or deactivates BYOK without removing the configuration
• Remove button: deletes the KMS configuration entirely

The portal UI calls the same admin API endpoints documented above.

---

Deployment considerations

Network connectivity

The platform must be able to reach the KMS endpoint. Ensure the following for each provider:

Provider	Default endpoint	Port	Protocol
Azure Key Vault	https://{vault-name}.vault.azure.net	443	HTTPS/TLS 1.2+
AWS KMS	https://kms.{region}.amazonaws.com	443	HTTPS/TLS 1.2+

For air-gapped or VPN-connected deployments, ensure the platform container can resolve and reach the KMS hostname. If using a private endpoint (Azure Private Link, AWS PrivateLink), configure DNS resolution accordingly.

Redis reservation

BYOK status resolution uses a 60-second cache (is_byok_enabled(org_id)) to avoid per-event database lookups. DEK caching uses in-memory LRU (not Redis). Redis db=2 is reserved for encryption-related caching — do not use it for other purposes.

Kubernetes workload identity

For Azure Kubernetes Service (AKS) deployments using workload identity federation:

1. Create a user-assigned managed identity with Key Vault permissions.
2. Federate the identity with the Kubernetes service account.
3. Set auth_method: "managed_identity" in the KMS configuration.

The platform automatically uses the pod’s federated identity token — no secrets or certificates need to be mounted.

---

Troubleshooting

KMS connectivity failures

Symptom	Cause	Resolution
validate returns timeout	Network path blocked	Check firewall rules, NSG, and DNS resolution to the KMS endpoint
validate returns 403	Insufficient permissions	Verify the platform identity has encrypt/decrypt/generate-data-key permissions on the KMS key
validate returns 404	Key not found	Confirm the key_id matches an existing key in the KMS
Audit events fail with AuditEncryptionError	KMS unavailable during write	Check KMS health; writes are fail-closed to prevent audit gaps
Audit exports show "[encrypted - KMS unavailable]"	KMS unavailable during read	Reads are fail-open; restore KMS connectivity to decrypt

DEK cache behavior

The DEK cache parameters:

Parameter	Value
TTL	1 hour (3600s)
Capacity	256 orgs
Eviction	LRU

If you rotate a key and want immediate effect, restart the platform process to flush the DEK cache. Otherwise, the cache entry expires naturally within 1 hour and a new DEK is generated with the new key.

BYOK status not updating

BYOK status (enabled/disabled) is cached for 60 seconds. After enabling or disabling BYOK, allow up to 60 seconds for all platform instances to pick up the change.

---

See also

• Security Architecture: BYOK Envelope Encryption — KMS abstraction, envelope encryption, DEK caching, fail-closed semantics
• Admin Operations: BYOK — Quick API reference
• Deployment Guide: BYOK Considerations — Network, Redis, Kubernetes notes