P42 Cache Remediation
Status: applied and committed 2026-08-06 (08b13854, pushed to staging) — not deployed. Neither staging nor production is running it. Scope: packages/api-client/src/client.ts, packages/api-client/src/cache-tags.ts, and the updateTag call sites in apps/clinic + apps/console.
Result: 42 tagged reads → 19, of which 12 are TTL-bounded and 7 are seeded reference data. updateTag call sites: 49 → 10, all in the Console and every one matched by a read that still carries its tag — no orphans. make check passes.
The P42 tag cache invalidates only inside the process that issued the updateTag. This document is the inventory behind the fix, the per-tag verdict, and the reasoning for each. Full pattern context lives at patterns.md → P42; the decision that scoped this work is recorded at platform-completion.md → Phase 2.
What was actually measured
All 42 tagged reads live in one file — packages/api-client/src/client.ts. The write side is 49 call sites: 29 in clinic, 20 in console, 0 in portal.
Three findings correct what was recorded when the bug was first filed.
1. app_replicas = 2 describes the portal, not the clinic
From infra/envs/production/compute.tf:
| app | desired_count | min | max |
|---|---|---|---|
| portal | var.app_replicas | 2 | 8 |
| clinic | 1 | 1 | 8 (CPU 70% target) |
| console | 1 | 1 | 1 (fixed) |
So cross-instance divergence is live on the portal, latent on the clinic — it appears the moment a clinic is busy enough to autoscale past one task — and impossible on the console.
The clinic case is the awkward one. Almost every mutable tag is a clinic tag, and the clinic is the app whose divergence only starts under load. That is precisely when it is least reproducible and most expensive.
2. The portal has zero updateTag calls
The portal owns bookMyAppointment, rescheduleMyAppointment, cancelMyAppointment, uploadMyAppointmentFile and deleteMyAppointmentFile. None of them invalidates org:{id}:appointments or org:{id}:appointment:{id}.
For the portal→clinic path this is not a process-boundary problem at all — the invalidation was never written. A shared cacheHandler would not have fixed it.
3. One call site already overrides revalidate
41 of 42 reads are revalidate: false. listConsentPurposes bounds its org variant at 300s. The TTL escape hatch already exists and is proven in production; the remediation reuses it rather than inventing a mechanism.
The sharpest instance: the live calendar
apps/clinic/app/api/appointments/route.ts is a route handler written so live mode can own its data through SWR. Its own comment reads:
A route handler so live mode can own its data client-side through SWR and revalidate exactly when it needs to. […] a live view that can serve a stale answer is not live.
The listAppointments call inside it is tagged, revalidate: false. SWR's revalidateOnFocus and every manual mutate() round-trip to a permanently cached response. The same shape appears in /api/sessions, /api/sessions/[id] and /api/catalog/pickable.
The rule
Case-by-case judgement about "is this data static enough" is what produced the current state — every one of these 42 tags looked defensible when it was added. The remediation applies a rule instead:
A tag is safe only when nothing outside a single process can change the underlying data.
That is true for migration-seeded reference data and nothing else. Everything else either drops the tag or accepts a bounded TTL, which converts "stale forever" into "stale for at most N seconds" without any new infrastructure.
Note what the rule excludes that a looser one would not: data mutated by another service. session and exercise detail carry render rows that the media service writes asynchronously. No Next.js process — not even with a shared cacheHandler — is on that write path.
Verdicts
DROP the tag — 23 sites
Method names, not line numbers: the sweep's line numbers went stale the moment the edits landed.
| tag | methods in client.ts | why |
|---|---|---|
org:*:appointments | listAppointments, getAppointmentCalendarCounts | portal mutates and never invalidates; feeds the live SWR grid |
org:*:appointment:{id} | getAppointment, listAppointmentFiles | portal uploads and deletes files on this surface |
org:*:calendars, :calendar:{id} | listCalendars, getCalendar, listCalendarForms | mutable clinic config; clinic autoscales |
org:*:specialist:*:availability | listWeeklyHours, listScheduleOverrides | same |
org:*:custom-fields | listCustomFields | same |
org:*:form-templates, :form-template:{id} | listFormTemplates, getFormTemplate, listFormTemplateVersions | publish flips the column another page filters on |
org:*:sessions, :session:{id} | listSessions, getSession | media service writes render rows out-of-band |
platform:exercises, :exercise:{slug} | listExercises, getExercise | same out-of-band render state |
platform:exercise-taxonomy:{slug} | listExerciseTags, listExerciseInstructions, listExerciseContraindications, listExercisePrerequisites, getExercisePoseConfig | live authoring surface; console traffic is superadmin-only, so the cache buys almost nothing |
platform:legal-document-stale-orgs | listLegalDocumentStaleOrgs | a computed cross-org projection that moves when any org acts |
getAppointmentCalendarCounts is referenced by no app. Not because a heat-map was removed — the month heat-map does not exist yet. GET /v1/organizations/{id}/appointments/calendar is live on the Go API (handler.go:540) and the client wrapper is in place, waiting for a UI. Its tag is dropped; the method is kept.
KEEP the tag, add cacheRevalidate — 12 sites
Slow-moving data with no reliable in-process invalidator. 300s, exported as SLOW_MOVING_TTL_S from cache-tags.ts and matching the precedent listConsentPurposes already set for its org variant.
| tag | methods in client.ts | why a TTL rather than a drop |
|---|---|---|
platform:taxonomy:{axis} × 7 | listExerciseCategories, listExerciseBodyRegions, listExerciseEquipment, listExerciseMovementPatterns, listExerciseRecoveryPhases, listExerciseConditions, listExerciseSkillPrerequisites | read by clinic AND console, written only by console — the clinic's copy is stale forever after any vocabulary edit. Second confirmed cross-app leak. |
platform:consent-purposes-with-latest | listConsentPurposes (platform variant) | read by clinic + portal, written only by console. Same shape; the org variant already has this TTL. |
org:*:summary, org:*:settings | getOrganization, getOrganizationSettings | console-only read and write, so correct today only because console is pinned at max_capacity = 1. A TTL removes the dependency on an infra constant. |
platform:legal-document-templates, platform:consent-purpose-versions | listLegalDocumentTemplates, listPlatformConsentPurposeVersions | same |
The last four are the interesting category: they work in production right now. They are listed here because they work for a reason nobody writing a Terraform change would think to preserve. Raising console's max_capacity is a one-line change with no visible connection to cache correctness.
KEEP as-is — 7 sites
Migration-seeded, with no runtime mutation path anywhere in the codebase:
listPermissions · listRoleTemplates · getCustomFieldMeta · getFormTemplateMeta · getOfferingFormSlots · listPoseEngines · listPoseEngineLandmarks
Knock-on work — done in the same pass
Dropping 23 tags orphaned 39 updateTag writes across 13 files — every clinic write, and half the console's. A surviving updateTag whose read side no longer exists reads as a working invalidation contract to the next person, so all 39 went with them, along with the 13 now-unused builders in cache-tags.ts. Deleting the builders was the enforcement mechanism: the typecheck then located every orphaned write for free.
Several files wrapped their invalidation in a local helper — bust(), invalidateSession(), invalidateExercise(), invalidateFields(), invalidateTemplates(). Stripping the updateTag lines left some of them empty and others holding only a refresh(), so each was collapsed into a direct refresh() call. One, invalidateSession(), was left as an empty function still called from two places — the kind of residue that survives a mechanical sweep and reads as working code.
The refresh() half of the server-action contract is untouched and is now the whole contract on those surfaces. Every mutating action was re-checked for it; the three that deliberately do not refresh (bookHeldSlotAction, createPatientAction, createSessionNoRedirectAction) each document why, and each was already correct.
Corroboration found mid-pass
listClinicExercises in the same file was already uncached, with a comment diagnosing exactly this problem before it had a name:
Deliberately UNCACHED — the catalog is mutated from Console (a different Next.js process / different ECS task) while read from Clinic. P42's tagged-cache invalidation is per-process, so a Console publish would leave Clinic serving stale "no exercises yet" indefinitely.
Whoever wrote that reached the correct conclusion for one endpoint and did not generalise it. That is the argument for a rule rather than per-endpoint judgement.
Follow-ups
OPEN, found 2026-08-14 — the Portal does not revalidate when the family switcher changes person. Switching profiles leaves the previous person's data on screen until something else forces a fetch.
This is a scope-key bug of the same family as the ones this document exists for, and it got sharper the day the server side was fixed. Every patient-facing read is now scoped to the SELECTED person — /me/appointments, /me/forms and the catalog resolver all go through patientscope — so the API returns the right person's rows. The cache in front of it does not know the selection exists: the tag and the cache key are built from the org and the resource, and two different family members produce the same key.
The rule in this document already says what is wrong with that: tag scope must match data visibility. A key that omits the selected profile is a key that answers about the wrong person, and "the person on screen" is precisely the axis the data varies along. So the fix is one of:
- include the selected
patient_profile_idin the cache key for every patient-facing tagged read, or - drop the tag on those reads (the verdict this document reached for 23 other sites), or
- invalidate on switch — the switcher is a client action, so it can call
updateTag/refresh()the way a mutation does.
Worth settling together rather than per call site, because the switcher is a single control and a half-scoped cache is indistinguishable from a working one until the wrong child's appointment is on screen. Noted rather than fixed: it is a Portal-wide decision, not a change to the reads that just shipped.
Done in this pass. listSessions and getSession no longer take an orgId — it existed only to build the tag, and org scope comes from the client's header context. Six call sites updated. Three of them were left holding an unused const orgId = await requireCurrentOrganizationId(); that call is a guard (it throws when the request carries no tenant context), so the call stays and only the assignment went, with a comment saying so. An assigned-and-unused variable would have been deleted by the next reader, taking the guard with it.
Closed 2026-08-06 — measured, and the question turned out not to apply. Read-only CloudWatch pull against production (./infra/scripts/latency.sh production 6), while production was still running the pre-remediation code:
| app | requests / 6h |
|---|---|
| clinic | 0 |
| console | 0 |
| portal | 523 |
And the P42-affected endpoints returned zero rows from the API's own logs over the same window.
The reason is structural, not a quiet period. Production runs migration 000039. The tables behind almost every de-cached read — appointments (F5), calendars (F4), form_templates / custom_fields (F3) — arrive at 000042–000046 and are on no environment. Twenty of the twenty-three dropped tags are on endpoints that do not exist in production. The three families that do exist (/v1/sessions, /v1/admin/exercises, the exercise-taxonomy reads) are read only by clinic and console, both at zero traffic.
So this change cannot have made production slower: there is no traffic on any surface it touches. Production RDS sits at ~5% CPU. The item is closed rather than deferred.
It becomes a live question again when F1–F6 actually deploy and clinics begin using the clinic app. At that point it is ordinary launch-readiness performance work for new features, not a regression check on this change — and latency.sh is there to answer it in one command.
The baseline above is worth re-reading then for one reason: it establishes that the clinic app had no production traffic at all before F1–F6, so any number measured later is that feature set's own cost, with nothing to subtract.
What this does not do
It does not add a shared cacheHandler on Redis. That remains deferred rather than rejected — it would fix cross-app and cross-instance together and keep the performance win, at the cost of a dependency, a SOUP row, config in three apps, and Redis becoming a cache-availability dependency. Revisit only if measurement shows the uncached latency actually hurts.
It also does not close the portal's missing invalidation as such. Once the appointment tags are gone there is nothing left for the portal to invalidate, so the gap closes by construction — but if appointment caching is ever reintroduced, the portal write path has to be built at the same time.