Infrastructure
Secrets Management
1Password-based secret management for local development, CI/CD, and Kubernetes
Philosophy
1Password is the single source of truth for all secrets across all environments. Secrets never exist as plaintext in the repository, local disk, or CI logs.
| Layer | Mechanism | Secrets on Disk? |
|---|---|---|
| Local development | .env.op files with op:// references, resolved by op run | No |
| CI/CD | GitHub Actions secrets injected at runtime | No |
| Kubernetes | 1Password Operator syncs vault items to K8s Secrets | Encrypted at rest (etcd) |
Local Development
.env.op Files
Each app has an .env.op file (committed to git) containing op:// URI references alongside literal defaults:
# Example: apps/lexilink/.env.op
STRIPE_SECRET_KEY=op://k3/lexilink-prod/STRIPE_SECRET_KEY # Resolved by op run
NEXT_PUBLIC_APP_URL=http://localhost:3000 # Passed through literally
op://references are resolved at runtime byop run— secrets never touch disk- Literal values pass through unchanged (non-secret dev defaults)
- Lexilink has additional
.env.op.stagingand.env.op.previewfiles for multi-environment support
Setup: OP_SERVICE_ACCOUNT_TOKEN
All dev:* scripts require a 1Password Service Account token:
# Add to ~/.zshrc (get token from 1Password admin)
export OP_SERVICE_ACCOUNT_TOKEN="ops_..."
source ~/.zshrc
Running with Secrets
The scripts/op-dev.sh script wraps op run for ephemeral secret resolution:
bun dev:lexilink # Dev with prod secrets
bun dev:lexilink:staging # Dev with staging secrets
bun dev:lexilink:preview # Dev with preview secrets
bun dev:calnexus # Dev calnexus with secrets
bun dev:planex # Dev planex with secrets
bun dev:archus # Dev archus with secrets
Each dev:* script runs: op run --env-file apps/<app>/.env.op -- turbo dev --filter=<app>
Regenerating .env.op Files
The scripts/generate-env-op.sh script regenerates .env.op files from 1Password item fields:
bun env:generate # Regenerate all .env.op files
./scripts/generate-env-op.sh lexilink --env staging # Regenerate one app/env
Runtime Validation with t3-env
All apps use t3-env (via @hn-monorepo/config) for runtime environment variable validation. The configuration uses emptyStringAsUndefined: true to catch blank values.
During Docker builds, CI=true is set in the builder stage to skip env validation at build time (secrets are not available during build).
CI/CD Secrets
GitHub Actions secrets are injected into workflow runs at runtime.
| Secret | Used By | Purpose |
|---|---|---|
HARBOR_USERNAME | deploy.yml | Harbor registry login |
HARBOR_PASSWORD | deploy.yml | Harbor registry login |
KUBECONFIG | deploy.yml | Base64-encoded kubeconfig for kubectl access |
CONVEX_ADMIN_KEY_<APP> | deploy.yml | Per-app Convex admin keys (7 apps) |
CONVEX_ADMIN_KEY_<APP>_STAGING | deploy.yml | Staging Convex admin keys |
CONVEX_ADMIN_KEY_<APP>_PREVIEW | deploy.yml | Preview Convex admin keys |
TURBO_TOKEN | ci.yml | Turborepo remote cache token |
TURBO_TEAM | ci.yml | Turborepo team identifier |
OP_SERVICE_ACCOUNT_TOKEN | deploy.yml | 1Password CLI access (if needed) |
Kubernetes Secrets (1Password Operator)
How It Works
The 1Password Connect Operator runs in op-system and watches for OnePasswordItem custom resources. Each app has a onepassworditem.yaml that maps a 1Password vault item to a Kubernetes Secret:
apiVersion: onepassword.com/v1
kind: OnePasswordItem
metadata:
name: lexilink-secrets
namespace: hn-apps
spec:
itemPath: "vaults/k3/items/lexilink-prod"
The operator creates a Kubernetes Secret named lexilink-secrets with fields from the vault item. Deployments reference these fields via secretKeyRef:
env:
- name: STRIPE_SECRET_KEY
valueFrom:
secretKeyRef:
name: lexilink-secrets
key: STRIPE_SECRET_KEY
Auto-Restart on Secret Change
Deployments use the auto-restart annotation to ensure pods restart when the underlying 1Password item changes:
annotations:
operator.1password.io/auto-restart: "true"
Important: Field Name Casing
The 1Password Operator preserves original casing from vault items. It does NOT lowercase field names. Ensure vault item fields match exactly what the application expects (typically SCREAMING_SNAKE_CASE).
Vault Structure
All secrets are stored in the k3 vault in 1Password. The naming convention is:
| Item Name | Environment | Used By |
|---|---|---|
<app>-prod | Production | hn-apps namespace |
<app>-staging | Staging | hn-staging namespace |
<app>-preview | Preview | hn-preview namespace |
convex-<app>-prod | Production | convex namespace |
convex-<app>-staging | Staging | hn-staging namespace |
convex-<app>-preview | Preview | hn-preview namespace |
Special Cases
- harbor-registry secret is manually managed (the
.dockerconfigjsonkey format is incompatible with the 1Password Operator) - auth-secret is shared across namespaces via
k8s/secrets/ - Vault item titles with em-dashes can break
op read— useop item get <id>instead
Multi-Environment Support
Currently only lexilink has full multi-environment support (production, staging, preview). Other apps default to prod only.
| Environment | Branch/Trigger | Namespace | 1Password Item |
|---|---|---|---|
| Production | Push to master | hn-apps / convex | lexilink-prod |
| Staging | Push to staging | hn-staging | lexilink-staging |
| Preview | PR label deploy:preview | hn-preview | lexilink-preview |
To add multi-environment support for another app, create the corresponding vault items in 1Password and follow the overlay pattern described in the Deployment page.
Troubleshooting
1Password secrets not syncing to Kubernetes
- Check operator logs:
kubectl logs -n op-system -l app=onepassword-connect - Verify the vault item exists:
op item get <app>-prod --vault k3 - Ensure field names use the expected casing (
AUTH_SECRET, notauth_secret) - Check the
OnePasswordItemresource status:kubectl describe onepassworditem -n hn-apps
Local dev: “op: error initializing client”
- Verify
OP_SERVICE_ACCOUNT_TOKENis set:echo $OP_SERVICE_ACCOUNT_TOKEN - Token may have expired — request a new one from the 1Password admin
- Ensure
opCLI is installed:brew install 1password-cli
Missing env var in Kubernetes pod
- Check the Deployment manifest references the correct secret name and key
- Verify the field exists in the 1Password vault item
- Check the Secret was created:
kubectl get secret <app>-secrets -n hn-apps -o yaml - Look for typos in field name casing