# Posts Module

Hexagonal module for the **feed** (`GET /posts/published` and siblings): creation, listing, content versioning, polls/attachments/reactions/comments enrichment, scheduled publishing, key updates, livestream posts, pinned posts.

This doc focuses on the two things most likely to confuse an agent reading cold: **how the published-feed listing branches**, and **the feed-visibility cache** wired into it.

---

## Feed listing: the three modes

Entry: `postsController.listPublishedPosts` → `list` → `PostsService.cursorPaginationList`. The mode is chosen by `getCursorListMode(loggedUser, includeGroupPosts)`:

```
if (!shouldUserIgnoreSegmentation(user) && !includeGroupPosts)  → SEGMENTATION
if (shouldUseComposedFeedFetch && includeGroupPosts)            → COMPOSED
else                                                            → LEGACY
```

Inputs:
- **`includeGroupPosts`** (per request) = the `includeGroupPosts` query param **AND** `hasViewGroupPostsInFeedCommunityFeature(user)` (the *"view group posts in feed"* community feature; cached). When true, `postTypes = [FEED, GROUP]`; otherwise `[FEED]`.
- **`shouldUserIgnoreSegmentation(user)`** — true for users with the `IGNORE_SEGMENTATION` capability (admins). Cheap, in-memory (reads the user's capabilities).
- **`shouldUseComposedFeedFetch(instanceId)`** — feature flag `NEW_FEED_AND_GROUP_FETCH_ENABLED` (cached, LRU + Redis).

| Mode | When | How IDs are fetched | Segmentation applied? |
|------|------|---------------------|------------------------|
| **SEGMENTATION** | normal user, **feed only** (`!includeGroupPosts`) | `findPostIdsByCursorWithSegmentation` → `segmentationsService.getSegmentableIdsByUser` (the **heavy** query: `public OR item-match OR own`) | ✅ yes — resolves *which posts are segmented for the user* |
| **COMPOSED** | `includeGroupPosts` + FF on | `findAllIdsByComposedCursorIgnoringSegmentation` → UNION of a FEED query + a GROUP query (group ids from membership), composed cursor | ❌ no — feed posts unsegmented; group visibility via membership |
| **LEGACY** | everything else (e.g. admin ignoring segmentation, or `includeGroupPosts` with FF off) | `findAllIdsByCursorIgnoringSegmentation` (single query, scopes) | ❌ no |

All three converge: `findAllByIds(ids, order, includeGroups)` (hydration: `Posts ⋈ Users ⋈ PostContents [⋈ Groups]`, scopes `base`/`withGroup` on `PostDAO`) → `enrichPosts` (polls, attachments, streams) → controller adds `getViewedByPostId`, `addExtraInformationToList` (last comment, comment count, reactions, signed URLs), own-segmentation info (only for the user's own posts), edit links.

**Key takeaway:** the expensive per-user "what's segmented for me" query exists **only in SEGMENTATION mode** — i.e. instances with the group-posts-in-feed feature **OFF**. COMPOSED/LEGACY don't run it. That's exactly where the cache below applies.

---

## Feed-visibility cache (`PostFeedVisibilityCacheService`)

Caches the heavy SEGMENTATION-mode query for **page 1** of `/posts/published`, to cut DB load when many *distinct* users open the home at once. Lives in `business/services/postFeedVisibilityCacheService.ts`; wired into `PostsService.findPostIdsByCursorWithSegmentation`. Env TTL: `POSTS_FEED_VISIBILITY_CACHE_TTL` (default 3h).

### Mode (`off` / `shadow` / `on`)

Gated by env `POSTS_FEED_VISIBILITY_CACHE_MODE` (see `FeedVisibilityCacheMode`). Flipping is config-only (no deploy of new code):

- `off` — cache disabled. Live query always; no Redis ops, no metrics, no invalidation (`bumpFeedVisibilityCache` no-ops).
- `shadow` — measure only. Records a **presence-based** hit/miss (`recordShadowLookup`: read the key + version; on miss write a *marker* — a versioned value with an empty payload) and invalidates (bumps), but **serves the live computation**. Runs **one** heavy query (the live serve); it does NOT compute/store the real cached value, so there's no double-compute and no warm. The marker is not a valid value (`parseVersionedValue` → null), so flipping to `on` starts cold (first read recomputes). Measures the would-be hit-rate in prod with no risk.
- `on` — serves from the cache (computes + stores the real value).

Provisioned via env (not a secret): baseline in `infrastructure/modules/{rest_server,worker_server}/env.default` (currently `shadow`); per-environment value in `infrastructure/env/{env}/environment_overrides.tf` (then `terraform apply`). App-code default (unset env) is `off`. Integration tests force `on` via `test-integration/.env.test`.

> hit/miss semantics differ per mode: in `shadow` a marker counts as a hit (presence); in `on` a marker is a miss (forces recompute). You're always in one mode, so the metric is self-consistent within a mode.

### When it engages

When mode is `shadow`/`on`, in SEGMENTATION mode **and** for the cacheable first page — `isCacheableFeedVisibilityQuery`: `cursor == null && !q && no ids filter && no userId filter && isKeyUpdate === undefined`. Search, deep pagination, `findByPk`/pinned lookups (which pass `ids`), and `listMinePublishedPosts` (passes `userId`) all fall through to the live query unchanged.

### Signature-keyed (cross-user sharing)

Redis value key: `POST_FEED_VISIBILITY:{instanceId}:{signature}:{limit}`.

- `signature = sha1(sorted feed-relevant segmentation item codes of the user)`.
- **Feed-relevant** = the user's `UserSegmentationItems` filtered to items **actually used to segment feed posts** (`SegmentationsService.getItemIdsUsedBySegmentableType` — a per-item `EXISTS` that short-circuits; never an instance-wide `DISTINCT` scan). Items no post targets can't change visibility, so they're dropped — collapsing users who would otherwise fragment. **Everyone who only sees public posts shares one bucket** (empty signature).
- Computed **per-request from current data** → never stale → no leak.
- Users in the same audiences share one cache entry → a crowd of distinct users collapses into a handful of reads.

> Index note: the `EXISTS` is only cheap with an index that lets it probe `(instanceId, itemId)` among post-segmentations. The ideal is a partial index `("instanceId","itemId") WHERE "segmentableType"='post' AND "deletedAt" IS NULL`. Without it, for an item used by non-post segmentables but no post, the `EXISTS` can scan that item's rows to prove absence. Check `Segmentations` indexes before relying on the "cheap" claim.

### Value, read, and the per-user overlay

- The cached value is the **audience-visible set only** (`public OR item-match`, computed with `orUserIdIsLoggedUser=false`) → identical for every user of the signature, never leaks per-user content. Stored as `"{version}:{json}"` where json is an array of `[id, publicationDatetimeEpochMs]` (limit+1 entries).
- The author's **own posts** are NOT in the shared entry. They're re-added per-request as a cheap overlay (`postRepositoryPort.findOwnVisiblePostEntries`) and **merged** with the cached set by `(publicationDatetime, id) DESC`, sliced to `limit+1`. This is why the value stores timestamps. Rationale: the frontend shows the creator as part of the segment even if the saved segmentation excludes them, so the author must always see their own post — but baking `OR userId` into a shared entry would leak it to same-signature peers.
- The `UserSegmentations` direct-user-targeting branch is intentionally **not** overlaid (a prod check showed those posts exist only in test instances).

### Versioned invalidation (mirrors `GroupRedBubbleService`)

A per-instance counter `POST_FEED_VISIBILITY:ver:{instanceId}` is bumped via `PostsService.bumpFeedVisibilityCache`. **Invalidation is centralized in the service-layer transition methods — NOT in controllers.** Any new code path that changes a feed post's visibility must call `bumpFeedVisibilityCache` from the service method that performs the transition; do not add bumps in controllers (that's how the manual-publish and draft-approval paths were originally missed).

`bumpFeedVisibilityCache` is **transaction-safe**: if there's an active transaction (the CLS namespace `Sequelize.useCLS` uses for the request — the same one the repo writes attach to), the Redis bump is deferred with `transaction.afterCommit`, so it **never runs inside the DB transaction**; otherwise it runs immediately. (A lost CLS context degrades to an early/at-most-once bump → bounded staleness healed by the version check + TTL, never a wrong-tx commit.)

It bumps **only when the visible set can change** (`isFeedVisible(post)` = `type=FEED && state=POSTED && approvalStatus=APPROVED`):

| Transition | Service method | Covers |
|---|---|---|
| Create (visible now) | `createFeedPost` (if `isFeedVisible`) | create endpoint |
| Publish (manual + scheduled) | `publishWithoutEnrichment` (if `isFeedVisible`) | `POST /:id/publish` **and** the scheduled cron — single choke point |
| Approve pending FEED request | `reviewPendingRequests` (only `APPROVED` + `FEED`) | review endpoint |
| Approve draft → feed | `updatePostStatusFromDraft` (if `isFeedVisible`) | draft approval |
| Re-segment / public toggle | `update` (if segmentation params present) | update endpoint |
| Delete a visible post | `delete` (if `isFeedVisible`) | delete endpoint |
| Content-only edit, draft/scheduled creation, rejection | — | not bumped (visible set unchanged) |

On read, the value's tagged version is compared to the current instance version; a mismatch is a miss → recompute (self-healing, no NX). A **tightened** post (re-segmented away from a user) leaves the feed immediately via the bump; the 3h TTL is only a backstop. **User membership changes need no invalidation** — the user's signature changes, so they self-route to a different key.

### Metrics (Datadog, `MetricRecorderPort`)

- `humandMainApi.postsFeedVisibilityCache.lookup` — count, tag `result: hit|miss|error` → hit rate.
- `humandMainApi.postsFeedVisibilityCache.computeMs` — distribution, timed around the heavy compute (miss path) → DB-load signal.
- `humandMainApi.postsFeedVisibilityCache.bumpFailure` — count, emitted when an invalidation `incr` fails after retries (the bump is lost until the TTL self-heals). The `incr` is retried (shared `retry` helper, idempotent-safe) before this fires; alert on it.

### Gotchas

- The bump runs in the controller **post-transaction callback** (after the HTTP response is sent), so integration tests that mutate-then-read must **poll** (`eventually`) — see `test-integration/api/posts/feedVisibilityCache.test.ts`.
- Only **page 1** is cached; the cursor is a coordinate `(publicationDatetime, id)`, so page-2+ live queries stay consistent with the cached first page.
- The cache key's `limit` is the only varying dimension (`postTypes`/`states`/`approvalStates` are constant for the cacheable case).

---

## Testing

Integration: `test-integration/api/posts/` (e.g. `feedVisibilityCache.test.ts`, `posts.test.ts`). Use `CreateCommunity.new()`; the group-posts-in-feed feature is OFF by default, so feed reads hit SEGMENTATION mode (and the cache). To assert cache key sharing, count Redis keys `POST_FEED_VISIBILITY:{instanceId}:*` (the version key has the form `:ver:{instanceId}`, so it isn't matched by that pattern).