Production Launch Readiness
The operational gate that flips the platform from "F11 hardening complete" to "real clinic, real patients, real revenue." This is distinct from F11: F11 is the technical/regulatory feature set (features.md → F11); this document is the operational checklist that sequences the cutover and confirms every external dependency, every runbook, and every sign-off is in place.
This is not a feature spec. It's a gate document — a living checklist whose real-launch target is ~September 2026 (full launch: real clinics + real patients + the ~20k legacy-patient migration). Realigned to September full-launch, 2026-06-19.
Production is already live (opened early at the June-10 demo)
The production substrate (infra/envs/production/) went live 2026-06-05 and stays live continuously — it was opened early to serve the June-10 single-session demo and has been serving real patients since. The September gate is therefore graduation, not provisioning: the env, RDS, ECS services, crons, alarms, and budgets already exist (verify each against the Infrastructure checklist below). What September adds on top of the running env is:
- Real terms + re-consent — publish real v2
platform_termsplus each clinic'sorg_terms, which version-bumps every demo alumnus and migrated patient into a412re-consent on next request (the consent ledger'sRequireCurrentConsentsgate). - The ~20k legacy-patient migration — transform/import the ~20k legacy patients (11k+ treatment plans, 5k+ subscriptions) into the existing live prod DB, including catalog Bucket 3 (D2C first-party clinic + tier/premium seed). Tracked launch-gate item; plan closer to launch.
- Flip the deferred items live — media-service + clinic app to running desired counts, full programs feature + a seeded patient catalog, DSAR/erasure end-to-end, the k6 load gate, cross-region backups, and the steady-state HA posture (see the per-area checklists below for which are done vs. open).
The active September plan is june-launch.md; its scope folds into this gate.
The three gates, in order:
- Foundation gate (1E.3) — staging deployed, foundation acceptance test passes against AWS staging. No real patients. (foundation.md)
- Layer 2 / F-tier feature build-out — F1–F12 features built on top of the substrate. (features.md)
- Production launch readiness — this doc. The operational gate before real clinics see real production.
See decisions.md for the architectural decisions feeding this gate.
Acceptance criteria
The platform is ready for first paying clinic when all of the following are true:
- Foundation 1E.3 closed (staging gate passed) — CLOSED 2026-05-15
- All F-tier features required at launch are merged, tested, deployed to staging, and validated end-to-end
- F11 hardening complete (technical + regulatory features shipped, production environment provisioned and load-tested)
- All pre-launch wiring tasks below complete
- Romanian regulatory counsel has signed off on F11.0.5 findings
- Legacy-data migration runbook executed in dry-run against a copy of the legacy database
- Incident response on-call rotation exists and has been tested
- Sign-off list below is fully checked
If any one is missing, the platform is not launching today.
Pre-cutover gates
Grouped by area. Each must be green before the cutover runbook executes.
Live production incidents (hard launch blockers)
Because prod is already serving real patients, open production incidents are launch blockers in their own right — they have to be closed before the env graduates to the September full launch.
- [ ] 🚨 iOS video-playback failure — exercise videos fail to play on iOS Safari: the Bunny hotlink-protected URL returns a CoreMedia
403(only ~30% session-completion on iOS today). Root cause is hotlink protection rejecting the CoreMedia/AVPlayer request; the fix is token-authenticated signed Bunny URLs (the player requests a per-asset signed URL the CDN accepts) rather than relying on referer/origin hotlink rules. Both a current production incident and a hard launch blocker — a third of patients can't complete a session without it. Tracked launch-gate item.
Foundation + features
- [x] Foundation 1E.3 closed and locked (CLOSED 2026-05-15) —
apps/docs/implementation-plan/foundation.mdshows the 1A–1C + 1E.3 substrate checkboxes checked (1D admin surfaces remain partially in flight, tracked there) - [ ] F1–F8 features that are part of launch scope are merged, tested, and deployed to staging
- [ ] F9 telerehab is in launch scope OR explicitly deferred — confirm decision
- [ ] F10 Telemetry service is deployed if F9 ships at launch (telerehab depends on telemetry; see telemetry/index.md)
- [ ] F11 technical hardening complete: GDPR DSAR endpoint works end-to-end, prod KMS rotation tested, security scan passes, performance benchmarks recorded against staging-shape production environment
Infrastructure
The env is already live (opened 2026-06-05) — this section is now validate the live demo-prod env + flip the remaining HA/scale posture, not "provision from scratch." Items the Terraform already implements are pre-checked; the open boxes are the validation tests + steady-state decisions that haven't run yet.
[x] Production AWS environment provisioned via Terraform (
infra/envs/production/) and idempotent — done; live since 2026-06-05[x] RDS Multi-AZ instance running (
db.t4g.medium,multi_az = trueinmain.tf) — provisioned; failover still needs a validation test (see open item below)[x] The base crons (
api-partition-roll,telemetry-partition-roll,usage-quota-reset,usage-summary-rollup,check-providersat 1-min prod cadence) are defined incrons.tfvia EventBridge Scheduler → ECS RunTask — confirm each has actually run successfully in prod at least once (CloudWatch success-pattern + DLQ depth = 0)[x] CloudWatch alarms + alerts SNS topic configured per monitoring.md (
observability.tf) — done (ALB, per-service CPU/mem, NAT alarms enabled)[x] AWS Budgets alarms set at 50% / 80% / 100% (actual) + 100% (forecasted) of monthly budget (
observability.tf) — done[ ] Synthetic RDS-failover test — force a Multi-AZ failover (reboot-with-failover) and confirm pgbouncer reconnects + the app recovers within target; not yet exercised in prod
[ ] pgbouncer drain test — confirm one pgbouncer task can drain/redeploy without disrupting in-flight runtime traffic
[ ] Custom-hostname WAF decision — see the WAF-coverage open item under Operational readiness; the L7-WAF-for-custom-domains call still has to land
[ ] Steady-state
app_replicasdecision — the demo posture pins low replica counts; pick + apply the launch steady-state (and the matching media/clinic desired counts — see the demo→launch state delta under External services)[x] Layer 2 backups live and PROVEN RESTORABLE (2026-08-02).
cmd/backup-runner+pg_dumpinDockerfile.apishipped;enable_backup_cron = true; daily 02:00 UTC. Covers both databases — core and telemetry (telemetry is a separate database, so the core dump never contained it). Abackup-missingCloudWatch alarm fires on absence of a success heartbeat, and the heartbeat only publishes after the uploaded object is read back and re-checksummed.The first drill failed, and that is the point. Production could not be restored from its own backup:
principal_is_human()referencedprincipalsunqualified, andpg_dumprestores under an emptysearch_path, so the inline CHECK onplatform_membershipskilled the data load. Fixed in migration000039. No checksum or alarm could have caught it — only an actual restore.Verified green on 2026-08-02 via
./infra/scripts/restore-drill.sh production: core restored in 11s with all 13 structural counts matching source exactly (180 tables, 731 indexes, 2285 constraints, 246 functions, 109 RLS-enabled tables, 292 RLS policies, 371 partitions) and 119,865 rows across 62 tables; telemetry restored in 3.3s with 50,817 rows and 70 partitions.[ ] Recurring restore drills. Run monthly and after any change to the backup pipeline, schema tooling, or Postgres major version. Currently on-demand — decide whether to schedule it before launch.
[ ] ALB + Cloudflare end-to-end (HTTPS, WAF active, custom-hostname provisioning verified) — proven for the demo path; re-verify at launch scale
[x] Production origin is edge-only. The production ALB security group accepts inbound 443 solely from Cloudflare's published IP ranges (
data.cloudflare_ip_ranges→alb_ingress_cidrsinenvs/production/main.tf, consumed bymodules/network/main.tf). This is the control that makes every edge protection actually binding: without it an attacker reaches the origin directly, bypassing the WAF and forgingCF-Connecting-IPto rotate rate-limit buckets at will — the bypassinternal/shared/clientip/clientip.goflags in its own doc comment. Verified in Terraform 2026-08-09.[ ] No Cloudflare WAF or rate-limiting ruleset exists in Terraform — for any zone.
envs/{staging,production}/cloudflare.tfmanage zone settings (TLS 1.2 floor, always-HTTPS), managed headers, flexible-SSL and a www redirect, and nothing else. Whatever L7 protection exists today is dashboard-configured: unversioned, unreviewed, and invisible to anyone reading the repo. Land edge rate-limiting rules + the managed WAF ruleset as Terraform at the zone level, so clinic custom hostnames inherit them and the config is reviewable. Note this is a precondition for option (a) of the custom-hostname WAF decision below, which presumes zone WAF rules that do not currently exist.[ ] Staging's ALB is open to
0.0.0.0/0(modules/network/variables.tf, documented as an orange-cloud-masked escape hatch). Consequence worth stating: staging's origin is directly reachable and its rate-limit keys are forgeable, so staging is not a faithful rehearsal of production's edge posture. Decide whether to close it or accept it explicitly.[ ] App-layer rate limiting is not DDoS defence, and shouldn't be counted as one. By the time
ratelimit.Middlewarerejects a request it has already reached an ECS task, taken a connection, and cost a Redis round-trip. It also cannot answer a distributed flood at all: the public surfaces most likely to be targeted — org resolve (300/min in prod), public booking (5/min), holds (60/min) — are all IP-keyed, so rotating source IPs hands the attacker a fresh bucket per IP. Only the edge sees the aggregate. See P60.[ ] Video consultations (F5.5) have NO infrastructure at all —
grep -i video infra/matches nothing, while the feature is built and browser-tested locally. Needed before000049promotes: three secrets (VIDEO_ROOM_SECRETgenerated per env;DAILY_API_KEY+DAILY_WEBHOOK_SECRETas a bootstrap pair) wired into the api task-def, an EventBridge entry forvideo-usage-reconcilewith a success-pattern alarm, and a one-shotcmd/video-webhook-registerrun per environment. Partitions need no Terraform (video_session_eventsalready rolls underapi-partition-roll), but the promotion runbook must roll partitions immediately aftermigrate: a new partitioned table applied mid-month has no partition for that month until 02:00 the next day, and these rows arrive from a webhook, so the failure is silent — consultations run normally and leave no record. Full list in features.md → F5.5 — infra and ops.[ ] Async render pipeline — scale-out (deferred from staging). Staging runs a single, fixed media task; the durable render queue + dispatcher work without any scaling infra. Two pieces are intentionally not wired until production load justifies them (see async-render-pipeline.md → Production rollout):
- [ ] Media autoscaling on queue depth. API emits a CloudWatch metric (pending-render count / oldest-pending age); a Fargate target-tracking policy scales the media service out when the bake queue builds and back in when it drains. Until this ships,
429backpressure + the dispatcher's retry/backoff keep a single media task safe — just slower under a burst. - [ ] Redis concurrency token-bucket + status cache. The moment media runs more than one instance, add a Redis token-bucket sized to total media bake capacity (acquire before dispatch, release after) so the uncoordinated per-instance dispatchers don't over-dispatch past total capacity. Same trigger is the natural point to add the Redis cache for the patient "preparing" poll. Pinned to the first media scale-out so "later" isn't "too late."
- [ ] Media autoscaling on queue depth. API emits a CloudWatch metric (pending-render count / oldest-pending age); a Fargate target-tracking policy scales the media service out when the bake queue builds and back in when it drains. Until this ships,
Testing & CI
The 5 layers of testing the platform should have at production launch. Where each layer sits, what it catches, and what's already in place.
| Layer | What | Where it runs | Status |
|---|---|---|---|
| Unit | Pure-function tests with mocks. make test (Go race-detector) + pnpm test (TS). | Local + CI on every PR | ✅ Wired — ci.yml runs make/pnpm checks + tests on every PR (PR #33). |
| Integration | Real Postgres via testcontainers. make test-integration. Includes setup_clinic_test.go. | Local + CI on every PR | ✅ Wired — ci.yml runs make test-integration on every PR (PR #33). |
| Smoke | Deployed service responds correctly to a basic request. aws ecs wait services-stable + curl /healthz + jq assertion. | Post-deploy in GitHub Actions | ✅ Wired in both deploy-staging.yml and deploy-production.yml (the prod workflow asserts api + telemetry /healthz). |
| End-to-end | Real user flows through real UI against deployed env (Playwright/Cypress). | Nightly cron against staging | Deferred until F-tier features worth E2E-testing |
| Load | Performance + capacity under realistic traffic (k6/Locust). | On-demand against dedicated load-test env | Deferred until pre-launch |
Pre-launch wiring:
- [x] CI on every PR —
.github/workflows/ci.ymlrunspnpm check+make check+make test+make test-integrationon everypull_request(PR #33). Catches regressions before merge. ~5-15 min per PR; well under the GitHub Actions free-tier quota even at heavy PR volume. - [x] Production smoke test — already wired in
deploy-production.yml: each service job runsaws ecs wait services-stableand theapi+telemetryjobs additionallycurl -fsS .../healthz+ jq-assertstatus == "ok"(andpostgres/redisfor api). Notedeploy-production.ymlrebuilds images per-env (each job tags:${{ github.sha }}and docker-build-pushes; Next.js bakes env-specificNEXT_PUBLIC_*at build time) — there is no SHA/release-tag promotion of staging artifacts. Open follow-up: extend the prod smoke beyond/healthz(e.g. an authenticated canary) per the cutover acceptance step. - [ ] External synthetic monitor on production — Cloudflare Health Checks (free) or Better Stack / UptimeRobot. Continuous external probe of
https://api.restartix.pro/healthzfrom multiple regions; catches outages independent of the deploy pipeline. - [ ] Real-user monitoring — Sentry SDK in api + 3 Next.js apps captures unhandled errors with stack traces and user context. Already on the External services checklist below; cross-listed here because it's the production half of the testing picture.
- [ ] Nightly E2E against staging — Playwright suite covering ~5 golden paths (sign in, create org, schedule appointment, view treatment plan, sign out). Scheduled GHA workflow that runs against
https://*-staging.restartix.pro. Worth adding when ≥3 F-tier features are live. - [ ] Pre-launch load test — k6 script targeting expected launch-day traffic profile (20k users in first org migration; ~5k month-1 active; ~1k peak concurrent). Run against staging or a dedicated load-test env. Validates autoscaling thresholds + Aurora ACU sizing + pgbouncer pool capacity.
Test-fixture contact details — one email domain, one phone number:
Every fabricated email in a fixture, seed, factory or UI placeholder uses @restartix.com. Every fabricated phone number is 0755 617 273 (+40755617273 normalized).
Never invent one. An invented address or number is a real person's address or number roughly as often as it isn't — and the platform is being built toward transactional email and SMS. The day a test factory, a seeded demo org or a stray staging run points its notifier at production credentials, an invented contact detail becomes a message delivered to a stranger who never heard of the clinic. There is no recall, and under GDPR an unsolicited message to an uninvolved data subject is a reportable event, not an embarrassment.
Two narrow exceptions, both because the test is specifically about the shape of a contact detail:
- RFC 2606 reserved domains (
example.com,*.test,*.invalid,*.example) where the case exercises an unusual domain structure the house domain cannot express — a two-letter domain, a multi-label suffix, plus-addressing. These are permanently undeliverable by standard. - Ofcom's reserved drama range
+44 20 7946 0xxx, used in phone_test.go for the one case that needs a genuinely foreign calling code. Permanently unallocated.
Anything reserved is safe; anything registrable is not. clinic.com, example.ro and gmail.com are registrable, and all three were in fixtures before this rule was written (swept 2026-08-06).
The house number is also the number in every doc comment that illustrates the phone normalizer, so grep -rE '0?7[0-9]{2}[ .-]?[0-9]{3}[ .-]?[0-9]{3}' over the repo returns exactly one number. That is what makes the convention checkable rather than aspirational.
What we deliberately do NOT do:
- E2E or load tests against production with real user data — too risky.
- Full integration test suite (
setup_clinic_test.go) re-run against staging on every deploy — 1E.2 closed by scope reduction; the GHA smoke step covers the high-value cases. The full suite stays as local + PR-CI run. - Cross-cloud chaos engineering — overkill until you have multi-region production.
External services + sub-processors
- [ ] Sentry — org created, projects per service (API + 3 Next.js apps + Telemetry if F10 launches), source-map upload step in CI working, release tracking by image SHA, alert routing to Slack tested
- [ ] Cloudflare — Pro plan active, Cloudflare for SaaS configured, custom-hostnames API token in Secrets Manager, end-to-end test of clinic registering a custom domain
- [ ] Bunny CDN — account active in EU region, DPA reviewed and signed, Bunny Stream library set up, admin upload workflow tested with reference exercise videos, signed playback URLs working from Patient Portal
- [ ] Clerk — production-mode keys (not test keys), HIPAA-eligible plan if applicable, BAA signed, webhook endpoint working
- [ ] Daily.co — production keys, HIPAA-eligible plan if F5 telerehab video is in scope, BAA signed
- [ ] Anthropic — API keys provisioned for AI agent capabilities (foundation 1C.8); per-org budget controls active via 1C.7 metering; AI agent service shape decided and documented (foundation memory: AI agent runtime is still an open scoping question — close before agents go live)
- [ ] AWS SES (infra) — production identity verified, DKIM + SPF + DMARC configured, sandbox exit confirmed, account-level suppression list active, SES configuration set with bounce + complaint event destinations pointing at an SNS topic
- [ ] AWS SES (app-layer code, gap re-verified 2026-06-19) — the API still has no bounce/complaint webhook handler and no
notification_suppressiontable. Build before launch:- SNS topic
restartix-prod-ses-feedbacksubscribed to SES bounce + complaint events; HTTPS subscription points at a new public API endpoint with SNS signature verification - Migration adding a
notification_suppressiontable (recipient, reason:hard_bounce/complaint/manual,suppressed_at, source event ID) — RLS restricts reads to platform-admin (suppression is a platform concern, not per-org) - Webhook handler at
POST /v1/internal/ses-feedbackthat verifies the SNS signature, parses the SES event payload, inserts intonotification_suppression EmailChannelprecheck in internal/core/notify/: before dispatch, query suppression by recipient address; if present, mark the notification dead-lettered withdead_letter_reason='suppressed'rather than calling SESmigrations/core/000010_notifications.up.sqldeferred-list line about "Bounce / complaint webhooks + suppression list automation" gets ticked here
- SNS topic
- [ ] All sub-processors disclosed in the platform DPA template (1B.10) and the Romanian-localized version
September-gate items folded in from june-launch.md
The demo opened prod early with several features pinned off or deferred. September flips these live — each is a launch-gate row here, sourced from june-launch.md:
- [ ] Media service + clinic app live — both run at reduced/zero desired counts for the demo; bring
mediaandclinicto their launch desired counts (see the demo→launch state delta below) - [ ] Full programs feature + seeded patient catalog — the demo shipped a single curated session; September needs the full programs feature and a seeded patient-facing catalog
- [ ] DSAR / erasure end-to-end — patient access-export + anonymization-erasure validated against real (migrated) data, not just the demo path
- [ ] k6 load gate — see the Pre-launch load test item under Testing & CI; run against the ~20k migration + month-1 active profile before opening to real clinics
- [ ] Cross-region backups — reconcile with the "What's deliberately not in this gate" note below, which defers cross-region S3 backup replication to within the first quarter post-launch. Decision needed: is cross-region a launch blocker for September, or does it stay a post-launch Q-1 item? Whichever way it lands, it presupposes the daily logical backup is actually on — i.e.
enable_backup_cron = true(currentlyfalse, see Infrastructure) is the prerequisite.
Demo → launch state delta. Flipping out of the demo posture means, at minimum: (1) set
app_replicasto the launch steady-state (the demo pins it low); (2) raisemediadesired_countfrom its demo value (andclinicdesired_countfrom0); (3) remove the/dev/perfroute and thediag-*diagnostic harness that exist only for demo-window debugging. Capture the full delta in the cutover runbook so nothing demo-only leaks into the launched env.
Compliance + legal
- [ ] F11.0.5 Romanian compliance pass — counsel engaged, full findings documented, privacy notice template (1B.10) and DPA revised, ANSPDCP enforcement scan complete
- [ ] Sub-processor list published to a public-facing page on the platform website per GDPR Art. 28 transparency
- [ ] DPA template ready for clinic onboarding — countersigned versions stored per clinic
- [ ] MDR Class I posture confirmed by regulatory counsel (or upgraded to Class IIa with appropriate process changes — see CLAUDE.md → Medical Device Readiness)
- [ ] AWS BAA accepted via AWS Artifact (free; HIPAA-eligibility on the AWS account)
- [ ] Legacy product DPA termination plan — when the legacy product shuts down, what's the data-handover and termination notice to its current users
Data migration
- [ ] Legacy migration runbook in deployment.md → Runbook: launch-day legacy-data migration executed end-to-end as a dry run against a copy of the legacy database
- [ ] Row-count and integrity validation queries documented and known to pass against the dry-run output
- [ ] Legacy passwords (if migrating user accounts) — confirm Clerk's password import path or force-reset-on-first-login flow
- [ ] Patient consent re-acquisition flow ready — legacy consents may not satisfy the new consent ledger schema; per-purpose re-consent on first login if needed
- [ ] Rollback plan validated: PITR restore from "pre-launch" RDS snapshot works in <1h
Operational readiness
- [ ] On-call rotation documented and committed to (PagerDuty / Slack alerts / phone tree)
- [ ] Incident response playbook in monitoring.md → Incident Response Procedures reviewed by all on-call engineers
- [ ] Synthetic incident drill — chaos test from monitoring.md executed in staging; alerts fired correctly; on-call responded within target time
- [ ] Status page — public status page configured (statuspage.io / similar), automated by CloudWatch alarms or manual updates
- [ ] Support escalation path — first-line clinic support routes to a human; engineering escalation path defined; severity levels with response-time targets
- [ ]
support@restartix.promailbox provisioned — referenced in the break-glass email template (break_glass_opened.email.{en,ro}.tmpl). The template tells the recipient to contact support if anything looks wrong; without a working mailbox at that address, the email is misleading. Set up via Google Workspace / Microsoft 365 / Fastmail / similar with MX records pointing at the chosen provider. Same provisioning thread can landnoreply@restartix.pro(send-only, no inbox needed) and any other addresses the platform uses. - [ ] Cloudflare WAF coverage decision for clinic custom domains — by default, Cloudflare WAF rules on the
restartix.prozone do NOT extend to clinic custom hostnames (e.g.physio-bucharest.ro) via Cloudflare for SaaS. DDoS protection is always-on; L7 WAF is not. Three options at launch: (a) subscribe to Cloudflare's "WAF for SaaS" add-on so zone WAF rules extend to all custom hostnames, (b) reintroduce AWS WAF on the ALB to cover L7 attacks at the origin (reverses the "Cloudflare-only WAF" decision indecisions.md), or (c) accept the gap and rely on application-layer defenses. Decision needs to land before the first clinic with a custom domain handles real patient traffic. Two corrections from the 2026-08-09 audit: option (a) presumes zone WAF rules that do not exist in Terraform (see the edge items under Infrastructure — nothing but TLS/redirect/header settings is managed as code), and option (c) is weaker than it sounds, because app-layer rate limiting is IP-keyed and a distributed flood gets a fresh bucket per source IP. - [ ] Customer success runbook — what does first-clinic onboarding look like? Manual handholding for the first ~5 clinics, then standardized
- [ ] Documentation portal for clinics — admin-facing how-to guides for setup-a-clinic, manage-staff, configure-billing, etc.
Deferred: promote the 2026-08-11 entitlement + usage work
[ ] 24 commits sit unpushed on
stagingas of 2026-08-11, deferred by the platform owner. Production runs a binary behind its own schema. Nothing is broken — the one incompatibility was resolved by restoringorganization_entitlements.treatment_plans_enabled— but the gap grows with every further change, and the ORDER of promotion is now load-bearing:infra/scripts/000003-grandfather-org-entitlements.sql— before or with the deploy. The new code gates video onvideo_consultations_enabled, which isFALSEon production; without this the clinic loses video consultations the moment it ships.000004-entitlement-scope.sql,000004-tiers-is-public.sql— before. Additions; the new code needs the columns.000004-limit-doctrine.sql,000004-entitlement-cleanup.sql— any time. Data only.- Deploy.
000004b-drop-treatment-plans-column.sql— after, and only after. This is the one that took production down on 2026-08-11 when run early.
The rule the split encodes: schema additions can precede a deploy; removals must follow it. "Script first, deploy second" is safe only when the deploy follows immediately — with a human-gated deploy it is an outage with a wait attached.
[ ] Staging is at
000038and materially drifted — behind production, and never touched by this work. It is not a faithful rehearsal for any of the above until reset.
Sales + commercial readiness
[ ] First-clinic contract signed and includes the platform's standard MSA + DPA
[ ] Pricing locked for the launch tier (shared-mode default per tenant-isolation.md)
[ ] Billing flow tested end-to-end — clinic onboarded, subscription created, first invoice generated (manual via FGO at launch is fine; F12 engine ships later)
[ ] Patient consent flow at launch tested — patient signs up, accepts platform + clinic consents, onboards into the clinic, can withdraw consents granularly
[ ] Decide whether
tenancy_mode = 'dedicated'is worth building for the RestartiX clinic itself. Raised 2026-08-11, deliberately unanswered.The RestartiX clinic moved onto the dedicated tier in production on 2026-08-11 (commercial SKU: unlimited caps, all four commercial entitlements). Its
tenancy_modeis stillshared, and that is correct, not an oversight — the two are different axes that happen to share a word. The tier is what an org is billed; the tenancy mode is where its data physically lives. See glossary → Tenancy Mode.What dedicated tenancy actually means today is one structural change: a per-tenant auth-provider organisation (its own Clerk org). Own-S3 and own-CMK are explicitly NOT part of it — they ship later as entitlements, once an exit/portability tool and a documented crypto-shred runbook exist. See decisions.md → Why entitlements, not columns.
Nothing can provision it. Every creation path lands
shared; the column is a schema reservation and no Go code branches on it. Flipping it by hand would claim an isolation posture the infrastructure does not provide, which is a false compliance statement rather than a display bug.The question to answer before launch is whether being our own first dedicated tenant is worth the provisioner, or whether shared-mode isolation (RLS + prefix scoping + app-layer checks) is what we would honestly tell a customer is sufficient — because if it is sufficient for us, the argument for selling dedicated to anyone else gets harder, and if it is not, we are running our own clinic on a posture we would not sell. Either answer is informative; leaving it undecided is the only bad outcome.
Note also: the docs say the dedicated tier "maps to"
tenancy_mode = 'dedicated'. Nothing performs or enforces that mapping — the tier move on 2026-08-11 left the column untouched, correctly. That sentence describes an intention, not a behaviour.
Cutover runbook
Day-of sequence. Estimate: 2–6h depending on legacy-data migration size.
This is not a first-ever cold cutover. Production has been live and serving real patients since 2026-06-05. T-0 transforms/imports the ~20k legacy patients into the existing live prod DB (alongside the demo-alumni rows already there) — it does not stand up a new env. Run the import via the existing tooling: the SSM tunnel (
infra/scripts/production-tunnel.sh) +infra/scripts/production-apply.shfor schema/data steps.infra/scripts/production-reset-db.shexists but is destructive — it is not part of this runbook (the prod DB has real patient data; never reset it at cutover).
T-7 days
- [ ] Communicate cutover schedule to first clinic (start time, expected duration, what they need to do)
- [ ] Take a manual RDS snapshot of the live production DB as the pre-migration rollback point (the DB already holds real demo-era patient data — this is the restore target if the import goes wrong)
- [ ] Confirm all sub-processor health (Cloudflare, Sentry, Bunny, Clerk, Daily.co, SES, Anthropic)
- [ ] Final dry-run of the legacy migration against a copy
T-1 day
- [ ] Final go/no-go review with everyone on the sign-off list below
- [ ] Confirm on-call rotation knows it's "live tomorrow"
- [ ] Pre-warm Cloudflare cache rules; verify edge cert + custom-hostname for the first clinic
T-0 (cutover day)
The target DB is the already-live production database, reached via the SSM tunnel (production-tunnel.sh) + production-apply.sh. Steps 1–7 import the legacy data into it; the migration freeze becomes forward-only at this gate — once real migrated patients are in the live DB, migrations are no longer freely reorderable/wipeable (the early-dev "migrations are editable" rule ends here; from T-0 on, every schema change is a new forward migration).
- Maintenance mode on legacy — return 503 with a branded maintenance page; communicate to legacy users
- Snapshot legacy database — full pg_dump from the legacy host, verified
- Pull legacy dump to a workstation with the operations IAM role
- Run data-transform pipeline (
services/migration-tools/legacy-importper the runbook in deployment.md) — transform + import the ~20k patients (11k+ treatment plans, 5k+ subscriptions) into the live prod DB via the SSM tunnel - Publish real v2 terms + version-bump re-consent — publish the real
platform_termsand each clinic'sorg_terms; the version bump puts every demo alumnus and every freshly-migrated patient into a412re-consent gate on next request (the consent ledger'sRequireCurrentConsents). Migrated patients accept on first login - Validate row counts against expected targets — block on any mismatch (account for the pre-existing demo-era rows already in the DB)
- Spot-check a few real legacy users in the new system — sign in, hit the re-consent gate, verify profile, see expected appointments
- Take a "post-import" RDS snapshot as a known-good launch state
- Remove maintenance mode, route DNS to production
- Synthetic acceptance test — run a small canary script against production: sign in, list orgs, list specialists, create a test appointment, confirm it persists
- Notify first clinic — they can start onboarding their staff and patients
- Watch monitoring for 24h — on-call active, alarm channels open, dashboards visible
T+24h, T+1 week, T+1 month
- [ ] T+24h: review error rate, p99 latency, no-failed-deploys, alarm-noise volume; if any of these is concerning, pause new clinic onboarding until resolved
- [ ] T+1 week: post-launch retrospective with the team; what surprised us, what we'd do differently, what runbooks need updating
- [ ] T+1 month: first paid invoice cycle complete (clinic charged successfully, AI cost roll-up correct, no dunning surprises)
Post-cutover monitoring (first 30 days)
What to watch, who watches, what triggers action.
| Signal | Where | Threshold | Action |
|---|---|---|---|
| 5xx error rate | CloudWatch alarm restartix-production-api-5xx | > 0.5% sustained 10m | On-call investigation |
| p99 latency | CloudWatch (API target group) | > 2s sustained 10m | On-call investigation |
| RDS connection saturation | CloudWatch | > 80% of max_connections | Capacity review |
| RDS replica lag (when read replicas exist) | CloudWatch | > 5s sustained | Replica health check |
| Clerk auth failures | Clerk dashboard + Sentry | Spike vs. baseline | Possible auth incident |
| Sentry new-error rate | Sentry | New issue class with high volume | Triage same-day |
| Bunny CDN delivery errors | Bunny dashboard | > 1% sustained | CDN health check |
| AWS spend trajectory | AWS Cost Explorer + Budgets | > forecast for the month | Cost review |
| Daily backup status | CloudWatch | Missed backup or checksum mismatch | Critical — investigate immediately |
| Audit-partition-roll cron | CloudWatch | Failure | Critical — ensure next-month partitions exist |
The full alarm catalogue lives in monitoring.md. This table is the post-launch focused subset.
Sign-off list
Before the cutover runbook executes, every signature here is required:
| Sign-off | Owner | Confirms |
|---|---|---|
| Engineering | Tech lead | F11 hardening complete; production environment validated; backup posture verified; incident response playbook reviewed |
| Regulatory / Compliance | Romanian counsel (F11.0.5) | Privacy notice + DPA templates approved; MDR class confirmed; data residency confirmed; sub-processor list published |
| Operations | On-call lead | On-call rotation in place; status page live; support escalation defined; chaos drill complete |
| Customer Success | First-clinic onboarding lead | First clinic ready; onboarding runbook tested; documentation portal usable |
| Commercial | Founder / business owner | First-clinic contract signed; pricing locked; billing flow tested |
What's deliberately not in this gate
To keep the gate honest about what blocks launch vs. what's nice-to-have:
- F12 Billing engine — not a launch blocker. First clinic gets manually-cut FGO invoices until the engine ships (features.md → F12).
- F13 Dedicated tenancy mode — deferred until first paying dedicated-mode contract.
- Multi-region / data residency per-tenant — out of scope per CLAUDE.md.
- Mobile apps — open decision in features.md; web-only is acceptable for launch.
- Customer-managed KMS — Phase 1 uses AWS-managed; CMK migration triggers documented in aws-infrastructure.md → Customer-managed KMS migration path.
- Datadog APM — CloudWatch + Sentry covers the launch; Datadog deferred until traffic + team scale justify it.
- Cross-region S3 backup replication — Layer 3 of backup-disaster-recovery.md deferred to within first quarter post-launch.
- Phase 2 read replicas — added when triggers in scaling-architecture.md → Lever 5 fire.
Related documentation
- implementation-plan.md — master plan
- foundation.md — foundation 1A–1E
- features.md — F1–F12 + F13
- aws-infrastructure.md — full topology and cost
- iac-layout.md — Terraform module structure
- deployment.md — CI/CD pipeline + runbooks (including legacy-data migration)
- scaling-architecture.md — connection math + scaling levers
- monitoring.md — alarms + incident response
- backup-disaster-recovery.md — RPO/RTO + DR drills
- decisions.md — architectural rationale
- external-providers.md — sub-processor list