You are validating and fixing the latest agents-database changes on branch feat/memory-promotion-automation-clean.

Primary objective:
- The new automatic background processing for memory consolidation and promotion appears broken.
- Leave the repo with that added functionality actually working end to end.

Definition of done:
1. Find the intended flow for automatic consolidation/promotion/background processing.
2. Reproduce the current failure with concrete evidence.
3. Fix the broken pieces.
4. Validate end to end with runnable evidence, not just static reasoning.
5. Update any needed docs/scripts/tests to match the final behavior.
6. End with a concise summary in the final output including: root cause, files changed, validation run, and any residual risk.

Important operating rules:
- Work in /mnt/rpi/agents-database.
- Do not push or merge.
- Be careful not to commit or rely on transient DB contents from data/shared-agent-memory.sqlite3 unless truly required. That file is already dirty; avoid making the situation worse.
- Prefer minimal, robust fixes.
- Do not stop early. Keep going until the definition of done is met or you hit a real blocker.
- If useful, create your own tmux pi jobs for long-running or parallelizable subtasks using the native wrappers in /mnt/rpi/pi-config/bin:
  - pi-job-start
  - pi-job-check
  - pi-job-wait
  - pi-job-list
  - pi-job-stop
  Use them when they help you avoid blocking or to split investigation/validation work.
- Since this run is one-prompt only, self-manage aggressively: inspect logs/tests/scripts, form a plan, execute, validate, and summarize.
- For waiting/poll cadence, choose adaptively based on what you observe. Nothing is fixed for you.

Suggested first steps:
- Inspect git diff/status and relevant scripts/tests for the automation changes.
- Search for maintenance/background/consolidate/promote flows.
- Reproduce with the smallest meaningful command.
- Add or update tests that would have caught the bug.

When done, print a crisp final report with commands run and their outcomes.


**Considering next steps**

I need to take action, and I’m wondering if I should check my memory. But it seems like that might not be necessary. It could be more useful to focus on the project context instead. I think I'll check the git status; that should give me a clear idea of where I stand with my current work. It’s always good to stay updated!

bash

git status --short && echo '---' && git branch --show-current && echo '---' && git log --oneline -n 5

find . -maxdepth 2 \( -name '*maint*' -o -name '*consol*' -o -name '*promot*' -o -name '*background*' -o -name '*worker*' \) | sort


 M data/shared-agent-memory.sqlite3
?? scripts/export-db-snapshot-html.py
---
feat/memory-promotion-automation-clean
---
d2c7d58 promote qualifying memories automatically
2dab626 feat: add remote shared-memory HTTP API
d9fad54 docs: add launchd template and daemon runner
3894ec8 feat: add maintenance daemon and config CLI
70cfc5a test: cover maintenance scheduler and conflict gating


./docs/memory-consolidation-rfc.md
./scripts/run-maintenance-daemon.sh


**Investigating changes in commits**

I need to look into differences, especially in the last commit. This usually means checking the changes made and understanding how they might impact everything else. It might help to search through the commit history to clarify what’s been altered. I'm curious if there are specific files or code sections that have significant changes. Figuring this out will definitely assist me in understanding the overall project better. Let’s see what I can find!

rg -n "consolidat|promot|background|maintenance|daemon|scheduler|qualif|pending promotion|promote" -S .

git diff --stat HEAD~3..HEAD && echo '---' && git diff --name-only HEAD~3..HEAD


./src/shared_agent_memory/service.py:65:        self._maintenance_check_running = False
./src/shared_agent_memory/service.py:69:        self._run_opportunistic_maintenance(limit=10)
./src/shared_agent_memory/service.py:125:        self._run_opportunistic_maintenance(limit=1)
./src/shared_agent_memory/service.py:274:    def consolidate(self, inbox_items: list[str]) -> dict[str, Any]:
./src/shared_agent_memory/service.py:275:        promoted: list[str] = []
./src/shared_agent_memory/service.py:315:                promoted.append(memory_id)
./src/shared_agent_memory/service.py:316:        return {"promoted": promoted, "linked": linked}
./src/shared_agent_memory/service.py:318:    def consolidate_candidates(self, *, config: dict[str, Any]) -> dict[str, Any]:
./src/shared_agent_memory/service.py:321:            "promoted": 0,
./src/shared_agent_memory/service.py:331:        promote_threshold = float(config.get("promote_confidence", 0.7))
./src/shared_agent_memory/service.py:438:                if candidate["status"] == "inbox" and candidate.get("confidence", 0.0) >= promote_threshold:
./src/shared_agent_memory/service.py:443:                    stats["promoted"] += 1
./src/shared_agent_memory/service.py:444:                elif candidate["status"] == "active" and candidate["type"] in {"episode", "artifact", "decision", "project", "procedural", "semantic"} and candidate.get("confidence", 0.0) >= promote_threshold:
./src/shared_agent_memory/service.py:445:                    # Normalize qualifying active records as already-promoted durable memories.
./src/shared_agent_memory/service.py:446:                    stats["promoted"] += 1
./src/shared_agent_memory/service.py:494:            "maintenance_jobs",
./src/shared_agent_memory/service.py:495:            "maintenance_runs",
./src/shared_agent_memory/service.py:518:            "maintenance_jobs",
./src/shared_agent_memory/service.py:519:            "maintenance_runs",
./src/shared_agent_memory/service.py:538:        self._run_opportunistic_maintenance(limit=10)
./src/shared_agent_memory/service.py:620:        self._run_opportunistic_maintenance(limit=1)
./src/shared_agent_memory/service.py:629:        self._run_opportunistic_maintenance(limit=1)
./src/shared_agent_memory/service.py:688:        self._run_opportunistic_maintenance(limit=1)
./src/shared_agent_memory/service.py:765:        self._run_opportunistic_maintenance(limit=1)
./src/shared_agent_memory/service.py:804:        self._run_opportunistic_maintenance(limit=1)
./src/shared_agent_memory/service.py:832:        self._run_opportunistic_maintenance(limit=1)
./src/shared_agent_memory/service.py:888:        self._run_opportunistic_maintenance(limit=1)
./src/shared_agent_memory/service.py:963:            maintenance_jobs = self._count_if_table_exists(conn, "maintenance_jobs")
./src/shared_agent_memory/service.py:970:            "maintenance_jobs": maintenance_jobs,
./src/shared_agent_memory/service.py:1155:        self._run_opportunistic_maintenance(limit=1)
./src/shared_agent_memory/service.py:1158:    def _run_opportunistic_maintenance(self, *, limit: int) -> None:
./src/shared_agent_memory/service.py:1159:        if self._maintenance_check_running:
./src/shared_agent_memory/service.py:1161:        due_jobs = self.list_due_maintenance_jobs(limit=1)
./src/shared_agent_memory/service.py:1164:        self._maintenance_check_running = True
./src/shared_agent_memory/service.py:1168:            MemoryEngine(self).run_due_maintenance(limit=limit)
./src/shared_agent_memory/service.py:1172:            self._maintenance_check_running = False
./src/shared_agent_memory/service.py:1246:                "id": "job_consolidation_daily_10",
./src/shared_agent_memory/service.py:1247:                "job_type": "consolidation",
./src/shared_agent_memory/service.py:1258:                    "promote_confidence": 0.7,
./src/shared_agent_memory/service.py:1266:                "id": "job_consolidation_daily_15",
./src/shared_agent_memory/service.py:1267:                "job_type": "consolidation",
./src/shared_agent_memory/service.py:1278:                    "promote_confidence": 0.7,
./src/shared_agent_memory/service.py:1286:                "id": "job_consolidation_weekly",
./src/shared_agent_memory/service.py:1287:                "job_type": "consolidation",
./src/shared_agent_memory/service.py:1298:                    "promote_confidence": 0.7,
./src/shared_agent_memory/service.py:1307:            legacy = conn.execute("SELECT id FROM maintenance_jobs WHERE id = ?", ("job_consolidation_daily",)).fetchone()
./src/shared_agent_memory/service.py:1311:                    UPDATE maintenance_jobs
./src/shared_agent_memory/service.py:1315:                    ("deprecated: split into 10:00 and 15:00 jobs", utcnow(), "job_consolidation_daily"),
./src/shared_agent_memory/service.py:1318:                row = conn.execute("SELECT id FROM maintenance_jobs WHERE id = ?", (job["id"],)).fetchone()
./src/shared_agent_memory/service.py:1325:                    INSERT INTO maintenance_jobs (
./src/shared_agent_memory/service.py:1348:    def list_maintenance_jobs(self) -> list[dict[str, Any]]:
./src/shared_agent_memory/service.py:1350:            rows = conn.execute("SELECT * FROM maintenance_jobs ORDER BY next_due_at ASC").fetchall()
./src/shared_agent_memory/service.py:1351:            return [self._row_to_maintenance_job(row) for row in rows]
./src/shared_agent_memory/service.py:1353:    def list_due_maintenance_jobs(self, *, now: str | None = None, limit: int | None = None) -> list[dict[str, Any]]:
./src/shared_agent_memory/service.py:1355:        sql = "SELECT * FROM maintenance_jobs WHERE enabled = 1 AND next_due_at <= ? ORDER BY next_due_at ASC"
./src/shared_agent_memory/service.py:1362:            return [self._row_to_maintenance_job(row) for row in rows]
./src/shared_agent_memory/service.py:1364:    def get_maintenance_job(self, job_id: str) -> dict[str, Any]:
./src/shared_agent_memory/service.py:1366:            row = conn.execute("SELECT * FROM maintenance_jobs WHERE id = ?", (job_id,)).fetchone()
./src/shared_agent_memory/service.py:1369:            return self._row_to_maintenance_job(row)
./src/shared_agent_memory/service.py:1371:    def update_maintenance_job(
./src/shared_agent_memory/service.py:1386:        job = self.get_maintenance_job(job_id)
./src/shared_agent_memory/service.py:1410:                UPDATE maintenance_jobs
./src/shared_agent_memory/service.py:1430:        return self.get_maintenance_job(job_id)
./src/shared_agent_memory/service.py:1449:    def create_maintenance_run(self, job_id: str) -> dict[str, Any]:
./src/shared_agent_memory/service.py:1455:                INSERT INTO maintenance_runs (id, job_id, status, started_at, completed_at, summary, stats_json, error_message)
./src/shared_agent_memory/service.py:1460:        return self.get_maintenance_run(run_id)
./src/shared_agent_memory/service.py:1462:    def finish_maintenance_run(
./src/shared_agent_memory/service.py:1475:                UPDATE maintenance_runs
./src/shared_agent_memory/service.py:1481:        return self.get_maintenance_run(run_id)
./src/shared_agent_memory/service.py:1483:    def get_maintenance_run(self, run_id: str) -> dict[str, Any]:
./src/shared_agent_memory/service.py:1485:            row = conn.execute("SELECT * FROM maintenance_runs WHERE id = ?", (run_id,)).fetchone()
./src/shared_agent_memory/service.py:1488:            return self._row_to_maintenance_run(row)
./src/shared_agent_memory/service.py:1490:    def list_maintenance_runs(self, *, job_id: str | None = None, limit: int = 20) -> list[dict[str, Any]]:
./src/shared_agent_memory/service.py:1491:        sql = "SELECT * FROM maintenance_runs WHERE 1=1"
./src/shared_agent_memory/service.py:1500:            return [self._row_to_maintenance_run(row) for row in rows]
./src/shared_agent_memory/service.py:1563:    def _row_to_maintenance_job(self, row: Any) -> dict[str, Any]:
./src/shared_agent_memory/service.py:1569:    def _row_to_maintenance_run(self, row: Any) -> dict[str, Any]:
./src/shared_agent_memory/cli.py:56:    maintenance_tick = subparsers.add_parser("maintenance-tick")
./src/shared_agent_memory/cli.py:57:    maintenance_tick.add_argument("--limit", type=int)
./src/shared_agent_memory/cli.py:58:    maintenance_tick.add_argument("--job-id")
./src/shared_agent_memory/cli.py:60:    maintenance_status = subparsers.add_parser("maintenance-status")
./src/shared_agent_memory/cli.py:61:    maintenance_status.add_argument("--runs", type=int, default=10)
./src/shared_agent_memory/cli.py:63:    maintenance_config = subparsers.add_parser("maintenance-config")
./src/shared_agent_memory/cli.py:64:    maintenance_config.add_argument("job_id")
./src/shared_agent_memory/cli.py:65:    maintenance_config.add_argument("--cadence")
./src/shared_agent_memory/cli.py:66:    maintenance_config.add_argument("--interval-minutes", type=int)
./src/shared_agent_memory/cli.py:67:    maintenance_config.add_argument("--window-start")
./src/shared_agent_memory/cli.py:68:    maintenance_config.add_argument("--window-end")
./src/shared_agent_memory/cli.py:69:    maintenance_config.add_argument("--enable", action="store_true")
./src/shared_agent_memory/cli.py:70:    maintenance_config.add_argument("--disable", action="store_true")
./src/shared_agent_memory/cli.py:71:    maintenance_config.add_argument("--metadata-json")
./src/shared_agent_memory/cli.py:72:    maintenance_config.add_argument("--recompute-next", action="store_true")
./src/shared_agent_memory/cli.py:74:    maintenance_daemon = subparsers.add_parser("maintenance-daemon")
./src/shared_agent_memory/cli.py:75:    maintenance_daemon.add_argument("--interval-seconds", type=int, default=900)
./src/shared_agent_memory/cli.py:76:    maintenance_daemon.add_argument("--jitter-seconds", type=int, default=60)
./src/shared_agent_memory/cli.py:77:    maintenance_daemon.add_argument("--limit", type=int)
./src/shared_agent_memory/cli.py:78:    maintenance_daemon.add_argument("--job-id")
./src/shared_agent_memory/cli.py:79:    maintenance_daemon.add_argument("--once", action="store_true")
./src/shared_agent_memory/cli.py:99:    consolidate = subparsers.add_parser("consolidate")
./src/shared_agent_memory/cli.py:100:    consolidate.add_argument("memory_ids", nargs="+")
./src/shared_agent_memory/cli.py:169:    elif args.command == "maintenance-tick":
./src/shared_agent_memory/cli.py:170:        print(json.dumps(engine.run_due_maintenance(limit=args.limit, job_id=args.job_id), indent=2))
./src/shared_agent_memory/cli.py:171:    elif args.command == "maintenance-status":
./src/shared_agent_memory/cli.py:174:                {"jobs": service.list_maintenance_jobs(), "runs": service.list_maintenance_runs(limit=args.runs)},
./src/shared_agent_memory/cli.py:178:    elif args.command == "maintenance-config":
./src/shared_agent_memory/cli.py:179:        job = service.get_maintenance_job(args.job_id)
./src/shared_agent_memory/cli.py:200:        updated = service.update_maintenance_job(
./src/shared_agent_memory/cli.py:211:    elif args.command == "maintenance-daemon":
./src/shared_agent_memory/cli.py:213:            results = engine.run_due_maintenance(limit=args.limit, job_id=args.job_id)
./src/shared_agent_memory/cli.py:238:    elif args.command == "consolidate":
./src/shared_agent_memory/cli.py:239:        print(json.dumps(service.consolidate(args.memory_ids), indent=2))
./src/shared_agent_memory/engine.py:26:    def run_due_maintenance(self, *, limit: int | None = None, job_id: str | None = None) -> list[dict[str, Any]]:
./src/shared_agent_memory/engine.py:28:            jobs = [self.service.get_maintenance_job(job_id)]
./src/shared_agent_memory/engine.py:30:            jobs = self.service.list_due_maintenance_jobs(limit=limit)
./src/shared_agent_memory/engine.py:33:            run = self.service.create_maintenance_run(job["id"])
./src/shared_agent_memory/engine.py:35:                stats = self._run_maintenance_job(job)
./src/shared_agent_memory/engine.py:36:                self.service.finish_maintenance_run(
./src/shared_agent_memory/engine.py:49:                self.service.update_maintenance_job(
./src/shared_agent_memory/engine.py:58:                self.service.finish_maintenance_run(
./src/shared_agent_memory/engine.py:65:                self.service.update_maintenance_job(
./src/shared_agent_memory/engine.py:74:    def _run_maintenance_job(self, job: dict[str, Any]) -> dict[str, Any]:
./src/shared_agent_memory/engine.py:76:        if job_type == "consolidation":
./src/shared_agent_memory/engine.py:77:            return self.service.consolidate_candidates(config=job.get("metadata") or {})
./src/shared_agent_memory/schema.py:100:CREATE TABLE IF NOT EXISTS maintenance_jobs (
./src/shared_agent_memory/schema.py:117:CREATE TABLE IF NOT EXISTS maintenance_runs (
./src/shared_agent_memory/schema.py:126:    FOREIGN KEY(job_id) REFERENCES maintenance_jobs(id)
./src/shared_agent_memory/schema.py:222:CREATE INDEX IF NOT EXISTS idx_maintenance_jobs_due ON maintenance_jobs(next_due_at);
./src/shared_agent_memory/schema.py:223:CREATE INDEX IF NOT EXISTS idx_maintenance_runs_job ON maintenance_runs(job_id);
./tests/test_memory_service.py:31:    thread = threading.Thread(target=server.serve_forever, daemon=True)
./tests/test_memory_service.py:48:def test_captured_conversation_stays_inbox_until_consolidated(tmp_path: Path) -> None:
./tests/test_memory_service.py:62:    promoted = service.consolidate([memory_id])
./tests/test_memory_service.py:63:    assert promoted["promoted"] == [memory_id]
./tests/test_memory_service.py:339:def test_maintenance_jobs_exist_by_default(tmp_path: Path) -> None:
./tests/test_memory_service.py:342:    jobs = {job["id"] for job in service.list_maintenance_jobs()}
./tests/test_memory_service.py:344:    assert "job_consolidation_daily_10" in jobs
./tests/test_memory_service.py:345:    assert "job_consolidation_daily_15" in jobs
./tests/test_memory_service.py:346:    assert "job_consolidation_weekly" in jobs
./tests/test_memory_service.py:349:def test_maintenance_tick_records_run(tmp_path: Path) -> None:
./tests/test_memory_service.py:354:    service.update_maintenance_job("job_consolidation_daily_10", next_due_at=now)
./tests/test_memory_service.py:355:    results = engine.run_due_maintenance(job_id="job_consolidation_daily_10")
./tests/test_memory_service.py:358:    job = service.get_maintenance_job("job_consolidation_daily_10")
./tests/test_memory_service.py:359:    runs = service.list_maintenance_runs(job_id="job_consolidation_daily_10")
./tests/test_memory_service.py:365:def test_service_startup_runs_due_maintenance(tmp_path: Path) -> None:
./tests/test_memory_service.py:370:    service.update_maintenance_job("job_consolidation_daily_10", next_due_at=now)
./tests/test_memory_service.py:372:    job = restarted.get_maintenance_job("job_consolidation_daily_10")
./tests/test_memory_service.py:373:    runs = restarted.list_maintenance_runs(job_id="job_consolidation_daily_10")
./tests/test_memory_service.py:379:def test_write_operations_trigger_due_maintenance_check(tmp_path: Path) -> None:
./tests/test_memory_service.py:383:    service.update_maintenance_job("job_consolidation_daily_10", next_due_at=now)
./tests/test_memory_service.py:386:    job = service.get_maintenance_job("job_consolidation_daily_10")
./tests/test_memory_service.py:387:    runs = service.list_maintenance_runs(job_id="job_consolidation_daily_10")
./tests/test_memory_service.py:393:def test_high_confidence_inbox_memory_auto_promotes_on_write(tmp_path: Path) -> None:
./tests/test_memory_service.py:395:    service.update_maintenance_job("job_consolidation_daily_10", next_due_at="2026-03-20T10:00:00+00:00")
./tests/test_memory_service.py:399:            "id": "mem_auto_promote",
./tests/test_memory_service.py:403:            "title": "Auto promote me",
./tests/test_memory_service.py:435:    service.consolidate_candidates(
./tests/test_memory_service.py:442:            "promote_confidence": 1.0,
./scripts/run-maintenance-daemon.sh:13:  maintenance-daemon \
./docs/http-api.md:50:- `consolidate_candidates()` and maintenance jobs live behind `MemoryService`; the HTTP API exposes the store, not a separate memory subsystem.
./docs/http-api.md:51:- `episode` and `inbox` records are candidates, not canonical truth, until consolidation or explicit promotion.
./scripts/export-db-snapshot-html.py:120:    body {{ font-family: system-ui, -apple-system, Segoe UI, Roboto, sans-serif; margin: 0; padding: 24px; color: #111; background: #fafafa; }}
./scripts/export-db-snapshot-html.py:122:    .meta, .summary {{ background: white; border: 1px solid #ddd; border-radius: 10px; padding: 16px; margin-bottom: 20px; }}
./scripts/export-db-snapshot-html.py:124:    .card {{ background: #f6f8fa; border-radius: 8px; padding: 12px; border: 1px solid #e5e7eb; }}
./scripts/export-db-snapshot-html.py:125:    .table-wrap {{ overflow: auto; background: white; border: 1px solid #ddd; border-radius: 10px; }}
./scripts/export-db-snapshot-html.py:128:    th {{ position: sticky; top: 0; background: #f3f4f6; z-index: 1; }}
./docs/system-v1.md:12:- authoritative place to implement memory promotion, consolidation, and conflict handling
./docs/system-v1.md:26:- the configured CLI runner must be available for LLM consolidation steps (MiniMax M2.5 Free).
./docs/system-v1.md:38:- `consolidate_candidates(config)` handles dedup, conflict detection, promotion, and optional synthesis
./docs/system-v1.md:39:- `maintenance_jobs` + `maintenance_runs` drive scheduled consolidation
./docs/system-v1.md:42:Promotion and conflict semantics:
./docs/system-v1.md:47:- inbox memories can be promoted to `active` once they pass the configured threshold
./docs/system-v1.md:50:## Maintenance
./docs/system-v1.md:53:.venv/bin/python -m shared_agent_memory.cli --db data/shared-agent-memory.sqlite3 maintenance-status
./docs/system-v1.md:54:.venv/bin/python -m shared_agent_memory.cli --db data/shared-agent-memory.sqlite3 maintenance-tick --limit 10
./docs/system-v1.md:55:.venv/bin/python -m shared_agent_memory.cli --db data/shared-agent-memory.sqlite3 maintenance-daemon --interval-seconds 900
./docs/system-v1.md:60:- `systemd/user/agents-database-maintenance.service`
./docs/system-v1.md:61:- `scripts/run-maintenance-daemon.sh`
./docs/system-v1.md:65:- `~/.config/systemd/user/agents-database-maintenance.service`
./docs/system-v1.md:66:- enabled with `systemctl --user enable --now agents-database-maintenance.service`
./docs/system-v1.md:73:systemctl --user status agents-database-maintenance.service --no-pager
./docs/system-v1.md:74:systemctl --user show agents-database-maintenance.service --property=NRestarts,ExecMainPID,SubState
./docs/system-v1.md:75:journalctl --user -u agents-database-maintenance.service -n 50 --no-pager
./docs/system-v1.md:80:- `docs/launchd/agents-database-maintenance.plist`
./docs/system-v1.md:84:- the daemon now starts through `scripts/run-maintenance-daemon.sh`
./docs/system-v1.md:86:- the service was verified to auto-restart after a manual `systemctl --user kill agents-database-maintenance.service`
./docs/system-v1.md:87:- `MemoryService` also performs opportunistic catch-up checks on startup and after normal write operations, so due jobs do not depend exclusively on the long-lived daemon
./docs/system-v1.md:96:- `docs/memory-consolidation-rfc.md`
./docs/shared-agent-memory-rfc.md:19:- one local `memory-engine` handles ingestion, extraction, consolidation, embeddings, and index refresh
./docs/shared-agent-memory-rfc.md:53:   A local background worker responsible for asynchronous processing: extraction, consolidation, embedding generation, and index maintenance.
./docs/shared-agent-memory-rfc.md:95:4. `scheduled maintenance`
./docs/shared-agent-memory-rfc.md:96:   Background maintenance runs to generate embeddings, refresh indexes, suggest consolidation, and detect duplicates.
./docs/shared-agent-memory-rfc.md:105:- promotion of raw conversation snippets directly into durable truth
./docs/shared-agent-memory-rfc.md:129:3. `consolidate`
./docs/shared-agent-memory-rfc.md:132:4. `promote`
./docs/shared-agent-memory-rfc.md:133:   A consolidated record becomes `active` and available as durable memory.
./docs/shared-agent-memory-rfc.md:149:- Durable `profile` facts about Sebas should require explicit confirmation or a promotion rule.
./docs/shared-agent-memory-rfc.md:157:- Every promoted memory must retain source/evidence references.
./docs/shared-agent-memory-rfc.md:355:## Ingestion and Consolidation
./docs/shared-agent-memory-rfc.md:382:### Consolidation Rules
./docs/shared-agent-memory-rfc.md:384:Consolidation should:
./docs/shared-agent-memory-rfc.md:390:- promote only sufficiently grounded memories to `active`
./docs/shared-agent-memory-rfc.md:395:- candidate facts extracted from episodes stay `inbox` until consolidation.
./docs/shared-agent-memory-rfc.md:396:- profile/worldview memories should be promoted with explicit confirmation or policy.
./docs/shared-agent-memory-rfc.md:408:- `consolidate(inbox_items)`
./docs/shared-agent-memory-rfc.md:455:4. A conversation imported from outside the system stays raw until explicitly captured, then becomes `inbox`, then later may be promoted.
./docs/shared-agent-memory-rfc.md:493:  Not now because the system first needs reliable ingestion, status handling, and consolidation.
./docs/shared-agent-memory-rfc.md:511:Treat memory as a structured, evidence-backed, consolidating knowledge system.
./docs/memory-consolidation-rfc.md:1:# Memory Consolidation + Opportunistic Scheduler RFC
./docs/memory-consolidation-rfc.md:9:Add a periodic memory consolidation stage to `agents-database` that is explicit, auditable, and catch-up friendly on a laptop. The scheduler must run at the next available moment if the machine was offline at the planned time.
./docs/memory-consolidation-rfc.md:13:- Keep the existing capture→extract→consolidate→promote→index flow.
./docs/memory-consolidation-rfc.md:15:- Preserve originals; no hard deletes during consolidation.
./docs/memory-consolidation-rfc.md:18:- Scheduler is **opportunistic** (runs when available, not only at exact times).
./docs/memory-consolidation-rfc.md:25:- Autonomous truth rewriting without explicit promotion rules.
./docs/memory-consolidation-rfc.md:29:### 1) Opportunistic Scheduler (Next Available)
./docs/memory-consolidation-rfc.md:48:### 2) Consolidation Pipeline
./docs/memory-consolidation-rfc.md:59:4. **Promotion**: only promote to `active` when above confidence threshold.
./docs/memory-consolidation-rfc.md:77:### maintenance_jobs
./docs/memory-consolidation-rfc.md:82:CREATE TABLE IF NOT EXISTS maintenance_jobs (
./docs/memory-consolidation-rfc.md:84:  job_type TEXT NOT NULL,               -- e.g. consolidation
./docs/memory-consolidation-rfc.md:100:### maintenance_runs
./docs/memory-consolidation-rfc.md:105:CREATE TABLE IF NOT EXISTS maintenance_runs (
./docs/memory-consolidation-rfc.md:114:  FOREIGN KEY(job_id) REFERENCES maintenance_jobs(id)
./docs/memory-consolidation-rfc.md:138:CREATE INDEX IF NOT EXISTS idx_maintenance_jobs_due ON maintenance_jobs(next_due_at);
./docs/memory-consolidation-rfc.md:139:CREATE INDEX IF NOT EXISTS idx_maintenance_runs_job ON maintenance_runs(job_id);
./docs/memory-consolidation-rfc.md:143:## Consolidation Heuristics (v1)
./docs/memory-consolidation-rfc.md:151:- Promotion threshold: confidence >= 0.70
./docs/memory-consolidation-rfc.md:165:## Scheduler Details (v1)
./docs/memory-consolidation-rfc.md:169:- Daily light runs at 10:00 and 15:00: dedupe + promotion checks (no LLM by default).
./docs/memory-consolidation-rfc.md:175:for job in maintenance_jobs where enabled=1:
./docs/memory-consolidation-rfc.md:188:- `maintenance-daemon` is cooperative; run it via your preferred login/session manager for continuous checks.
./docs/memory-consolidation-rfc.md:194:.venv/bin/python -m shared_agent_memory.cli --db data/shared-agent-memory.sqlite3 maintenance-status
./docs/memory-consolidation-rfc.md:195:.venv/bin/python -m shared_agent_memory.cli --db data/shared-agent-memory.sqlite3 maintenance-tick
./docs/memory-consolidation-rfc.md:196:.venv/bin/python -m shared_agent_memory.cli --db data/shared-agent-memory.sqlite3 maintenance-config job_consolidation_daily_10 --window-start 10:00 --recompute-next
./docs/memory-consolidation-rfc.md:197:.venv/bin/python -m shared_agent_memory.cli --db data/shared-agent-memory.sqlite3 maintenance-daemon --interval-seconds 900
./docs/memory-consolidation-rfc.md:202:- `docs/launchd/agents-database-maintenance.plist`
./docs/memory-consolidation-rfc.md:206:- `systemd/user/agents-database-maintenance.service`
./docs/memory-consolidation-rfc.md:207:- install with `ln -sf /home/sebas/agents-database/systemd/user/agents-database-maintenance.service /home/sebas/.config/systemd/user/agents-database-maintenance.service`
./docs/memory-consolidation-rfc.md:208:- enable with `systemctl --user daemon-reload && systemctl --user enable --now agents-database-maintenance.service`
./docs/memory-consolidation-rfc.md:213:- 2026-03-26: RFC drafted with opportunistic scheduler + consolidation pipeline.
./docs/memory-consolidation-rfc.md:214:- 2026-03-26: Implemented maintenance tables, scheduler hooks, and consolidation runner (v1).
./docs/memory-consolidation-rfc.md:215:- 2026-03-27: Added maintenance config + daemon CLI commands.
./docs/memory-consolidation-rfc.md:216:- 2026-03-30: Fixed the maintenance daemon wrapper to run from a repo checkout by exporting `PYTHONPATH` and preferring `./.venv/bin/python`.
./docs/memory-consolidation-rfc.md:221:- Whether to run the engine as a long-lived background process or on-demand via CLI.
./docs/memory-consolidation-rfc.md:222:- Which storage to use for LLM usage counters (likely `maintenance_runs.stats_json`).
./docs/memory-consolidation-rfc.md:226:1. Add scheduler hook coverage beyond the daemon path if we want startup/write-triggered opportunistic runs inside normal service usage.
./docs/memory-consolidation-rfc.md:227:2. Add tests for daemon/bootstrap operation and job catch-up across missed windows.
./docs/memory-consolidation-rfc.md:228:3. Add LLM budget counters per run/day to `maintenance_runs.stats_json` or a dedicated table.
./docs/launchd/agents-database-maintenance.plist:6:    <string>com.sebas.agents-database.maintenance</string>
./docs/launchd/agents-database-maintenance.plist:10:      <string>/Users/sebas/agents-database/scripts/run-maintenance-daemon.sh</string>
./docs/launchd/agents-database-maintenance.plist:19:    <string>/Users/sebas/Library/Logs/agents-database/maintenance.out.log</string>
./docs/launchd/agents-database-maintenance.plist:21:    <string>/Users/sebas/Library/Logs/agents-database/maintenance.err.log</string>


 docs/http-api.md                               |  55 ++++
 docs/launchd/agents-database-maintenance.plist |  34 +++
 docs/memory-consolidation-rfc.md               |  34 ++-
 docs/system-v1.md                              |  60 +++-
 scripts/run-http-api.sh                        |  16 ++
 scripts/run-maintenance-daemon.sh              |  15 +
 src/shared_agent_memory/cli.py                 |   7 +
 src/shared_agent_memory/http_api.py            | 364 +++++++++++++++++++++++++
 src/shared_agent_memory/service.py             |  37 ++-
 systemd/user/agents-database-http.service      |  17 ++
 tests/test_memory_service.py                   | 288 ++++++++++++++-----
 11 files changed, 849 insertions(+), 78 deletions(-)
---
docs/http-api.md
docs/launchd/agents-database-maintenance.plist
docs/memory-consolidation-rfc.md
docs/system-v1.md
scripts/run-http-api.sh
scripts/run-maintenance-daemon.sh
src/shared_agent_memory/cli.py
src/shared_agent_memory/http_api.py
src/shared_agent_memory/service.py
systemd/user/agents-database-http.service
tests/test_memory_service.py


**Inspecting for test failures**

I need to run some tests because they’re likely failing. I'm considering inspecting relevant files, particularly those related to service, engine, or CLI. It’s essential to pinpoint where things might be going wrong. I want to make sure I understand the causes of these failures so I can address them effectively. I’ll get started on that and see what I can find in those areas!

pytest -q tests/test_memory_service.py -q


/bin/bash: line 1: pytest: command not found

Command exited with code 127


                ) VALUES (
                    :id, :trigger_type, :source_id, :requested_scope, :job_state, :processor_outcome,
                    :retry_count, :error_message, :created_at, :updated_at, :processed_at
                )
                """,
                event,
            )
        self._run_opportunistic_maintenance(limit=1)
        return {"source": source, "event": event}

    def _run_opportunistic_maintenance(self, *, limit: int) -> None:
        if self._maintenance_check_running:
            return
        due_jobs = self.list_due_maintenance_jobs(limit=1)
        if not due_jobs:
            return
        self._maintenance_check_running = True
        try:
            from .engine import MemoryEngine

            MemoryEngine(self).run_due_maintenance(limit=limit)
        except Exception:
            return
        finally:
            self._maintenance_check_running = False

    def _memory_filter_sql(self, sql: list[str], filters: dict[str, Any]) -> list[Any]:
        params: list[Any] = []
        allowed = {"project_id", "repo_id", "type", "subtype", "run_id", "task_id", "origin_agent", "url", "domain"}
        for key, value in filters.items():
            if key not in allowed or value is None:
                continue
            sql.append(f"AND {key} = ?")
            params.append(value)
        return params

    def _facet_score(self, memory: dict[str, Any], filters: dict[str, Any]) -> float:
        tracked = ["project_id", "repo_id", "type", "subtype", "run_id", "task_id", "origin_agent", "url", "domain"]
        applied = [key for key in tracked if filters.get(key) is not None]
        if not applied:
            return 0.0
        matched = sum(1 for key in applied if memory.get(key) == filters[key])
        return matched / len(applied)

    def _migrate_memory_row(self, row: dict[str, Any]) -> dict[str, Any] | None:
        metadata = self.store.loads(row.get("metadata_json")) or {}
        if row.get("schema_version") == SCHEMA_VERSION and not metadata.get("legacy_kind") and not metadata.get("legacy_system"):
            return None
        subtype = self._memory_subtype(metadata)
        origin_agent = row.get("origin_agent") or metadata.get("origin_agent") or metadata.get("legacy_system") or row.get("agent_id")
        run_id = row.get("run_id") or metadata.get("run_id") or metadata.get("legacy_run_id") or self._infer_run_id(row, subtype)
        task_id = row.get("task_id") or metadata.get("task_id")
        url = row.get("url") or metadata.get("url") or metadata.get("source_url") or self._infer_url(row, subtype)
        domain = row.get("domain") or metadata.get("domain") or (urlparse(url).netloc if url else None)
        clean = {key: value for key, value in metadata.items() if key not in {"legacy_kind", "legacy_system", "legacy_run_id", "run_id", "task_id", "url", "domain", "source_url", "record_kind", "schema"}}
        if subtype == "research_run":
            clean["goal"] = clean.get("goal") or _line_value(row["content"], "Goal") or row["title"].removeprefix("Research run: ").strip()
            clean["scope"] = clean.get("scope") or _line_value(row["content"], "Scope")
            clean["assumptions"] = clean.get("assumptions") or _line_value(row["content"], "Assumptions")
            clean["summary"] = clean.get("summary") or _line_value(row["content"], "Summary") or row["summary"]
        clean["schema"] = SCHEMA_VERSION
        clean["subtype"] = subtype
        record_type = row["type"]
        if subtype == "router_handoff":
            record_type = "episode"
        elif subtype == "research_source":
            record_type = "artifact"
        return {
            "id": row["id"],
            "schema_version": SCHEMA_VERSION,
            "type": record_type,
            "subtype": subtype,
            "origin_agent": origin_agent,
            "run_id": run_id,
            "task_id": task_id,
            "url": url,
            "domain": domain,
            "summary": clean.get("summary", row["summary"]),
            "metadata_json": self.store.dumps(clean),
        }

    def _migrate_task_row(self, row: dict[str, Any]) -> dict[str, Any] | None:
        metadata = self.store.loads(row.get("metadata_json")) or {}
        if row.get("schema_version") == SCHEMA_VERSION and not metadata.get("legacy_kind") and not metadata.get("legacy_system"):
            return None
        run_id = row.get("run_id") or metadata.get("run_id") or metadata.get("legacy_run_id")
        clean = {key: value for key, value in metadata.items() if key not in {"legacy_kind", "legacy_system", "legacy_run_id"}}
        clean["schema"] = SCHEMA_VERSION
        return {
            "id": row["id"],
            "schema_version": SCHEMA_VERSION,
            "run_id": run_id,
            "metadata_json": self.store.dumps(clean),
        }

    def _ensure_default_jobs(self) -> None:
        defaults = [
            {
                "id": "job_consolidation_daily_10",
                "job_type": "consolidation",
                "cadence": "daily",
                "interval_minutes": None,
                "window_start": "10:00",
                "window_end": None,
                "metadata": {
                    "mode": "daily_10",
                    "candidate_age_days": 0,
                    "max_candidates": 200,
                    "dedupe_high_threshold": 0.9,
                    "dedupe_mid_threshold": 0.75,
                    "promote_confidence": 0.7,
                    "conflict_threshold": 0.55,
                    "llm_enabled": False,
                    "llm_provider": "opencode",
                    "llm_command": ["opencode", "-m", "MiniMax M2.5 Free"],
                },
            },
            {
                "id": "job_consolidation_daily_15",
                "job_type": "consolidation",
                "cadence": "daily",
                "interval_minutes": None,
                "window_start": "15:00",
                "window_end": None,
                "metadata": {
                    "mode": "daily_15",
                    "candidate_age_days": 0,
                    "max_candidates": 200,
                    "dedupe_high_threshold": 0.9,
                    "dedupe_mid_threshold": 0.75,
                    "promote_confidence": 0.7,
                    "conflict_threshold": 0.55,
                    "llm_enabled": False,
                    "llm_provider": "opencode",
                    "llm_command": ["opencode", "-m", "MiniMax M2.5 Free"],
                },
            },
            {
                "id": "job_consolidation_weekly",
                "job_type": "consolidation",
                "cadence": "weekly",
                "interval_minutes": None,
                "window_start": None,
                "window_end": None,
                "metadata": {
                    "mode": "weekly",
                    "candidate_age_days": 0,
                    "max_candidates": 800,
                    "dedupe_high_threshold": 0.9,
                    "dedupe_mid_threshold": 0.75,
                    "promote_confidence": 0.7,
                    "conflict_threshold": 0.55,
                    "llm_enabled": True,
                    "llm_provider": "opencode",
                    "llm_command": ["opencode", "-m", "MiniMax M2.5 Free"],
                },
            },
        ]
        with self.store.connection() as conn:
            legacy = conn.execute("SELECT id FROM maintenance_jobs WHERE id = ?", ("job_consolidation_daily",)).fetchone()
            if legacy:
                conn.execute(
                    """
                    UPDATE maintenance_jobs
                    SET enabled = 0, last_summary = ?, updated_at = ?
                    WHERE id = ?
                    """,
                    ("deprecated: split into 10:00 and 15:00 jobs", utcnow(), "job_consolidation_daily"),
                )
            for job in defaults:
                row = conn.execute("SELECT id FROM maintenance_jobs WHERE id = ?", (job["id"],)).fetchone()
                if row:
                    continue
                now = utcnow()
                next_due = self._compute_next_due(now, job["cadence"], job["interval_minutes"], job["window_start"], job["window_end"])
                conn.execute(
                    """
                    INSERT INTO maintenance_jobs (
                        id, job_type, cadence, interval_minutes, window_start, window_end, next_due_at,
                        last_run_at, last_status, last_summary, enabled, metadata_json, created_at, updated_at
                    ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                    """,
                    (
                        job["id"],
                        job["job_type"],
                        job["cadence"],
                        job["interval_minutes"],
                        job["window_start"],
                        job["window_end"],
                        next_due,
                        None,
                        None,
                        None,
                        1,
                        self.store.dumps(job["metadata"]),
                        now,
                        now,
                    ),
                )

    def list_maintenance_jobs(self) -> list[dict[str, Any]]:
        with self.store.connection() as conn:
            rows = conn.execute("SELECT * FROM maintenance_jobs ORDER BY next_due_at ASC").fetchall()
            return [self._row_to_maintenance_job(row) for row in rows]

    def list_due_maintenance_jobs(self, *, now: str | None = None, limit: int | None = None) -> list[dict[str, Any]]:
        now = now or utcnow()
        sql = "SELECT * FROM maintenance_jobs WHERE enabled = 1 AND next_due_at <= ? ORDER BY next_due_at ASC"
        params: list[Any] = [now]
        if limit is not None:
            sql += " LIMIT ?"
            params.append(limit)
        with self.store.connection() as conn:
            rows = conn.execute(sql, params).fetchall()
            return [self._row_to_maintenance_job(row) for row in rows]

    def get_maintenance_job(self, job_id: str) -> dict[str, Any]:
        with self.store.connection() as conn:
            row = conn.execute("SELECT * FROM maintenance_jobs WHERE id = ?", (job_id,)).fetchone()
            if row is None:
                raise KeyError(job_id)
            return self._row_to_maintenance_job(row)

    def update_maintenance_job(
        self,
        job_id: str,
        *,
        cadence: str | None = None,
        interval_minutes: int | None = None,
        window_start: str | None = None,
        window_end: str | None = None,
        next_due_at: str | None = None,
        last_run_at: str | None = None,
        last_status: str | None = None,
        last_summary: str | None = None,
        enabled: bool | None = None,
        metadata: dict[str, Any] | None = None,
    ) -> dict[str, Any]:
        job = self.get_maintenance_job(job_id)
        if cadence is not None:

[322 more lines in file. Use offset=1388 to continue.]


from __future__ import annotations

from typing import Any

from .service import MemoryService, make_id, utcnow

class MemoryEngine:
    def __init__(self, service: MemoryService) -> None:
        self.service = service

    def process_pending(self, limit: int | None = None) -> list[dict[str, Any]]:
        events = self.service.list_pending_events()
        if limit is not None:
            events = events[:limit]

        processed: list[dict[str, Any]] = []
        for event in events:
            try:
                processed.append(self._process_event(event))
                self.service.mark_event(event["id"], "completed", outcome="extracted")
            except Exception as exc:  # pragma: no cover - defensive
                self.service.mark_event(event["id"], "failed", error_message=str(exc))
        return processed

    def run_due_maintenance(self, *, limit: int | None = None, job_id: str | None = None) -> list[dict[str, Any]]:
        if job_id:
            jobs = [self.service.get_maintenance_job(job_id)]
        else:
            jobs = self.service.list_due_maintenance_jobs(limit=limit)
        results: list[dict[str, Any]] = []
        for job in jobs:
            run = self.service.create_maintenance_run(job["id"])
            try:
                stats = self._run_maintenance_job(job)
                self.service.finish_maintenance_run(
                    run["id"],
                    status="completed",
                    summary=f"{job['job_type']} completed",
                    stats=stats,
                )
                next_due = self.service._compute_next_due(
                    utcnow(),
                    job["cadence"],
                    job.get("interval_minutes"),
                    job.get("window_start"),
                    job.get("window_end"),
                )
                self.service.update_maintenance_job(
                    job["id"],
                    next_due_at=next_due,
                    last_run_at=utcnow(),
                    last_status="completed",
                    last_summary=f"{job['job_type']} completed",
                )
                results.append({"job_id": job["id"], "status": "completed", "stats": stats})
            except Exception as exc:
                self.service.finish_maintenance_run(
                    run["id"],
                    status="failed",
                    summary=f"{job['job_type']} failed",
                    stats={},
                    error_message=str(exc),
                )
                self.service.update_maintenance_job(
                    job["id"],
                    last_run_at=utcnow(),
                    last_status="failed",
                    last_summary=str(exc),
                )
                results.append({"job_id": job["id"], "status": "failed", "error": str(exc)})
        return results

    def _run_maintenance_job(self, job: dict[str, Any]) -> dict[str, Any]:
        job_type = job.get("job_type")
        if job_type == "consolidation":
            return self.service.consolidate_candidates(config=job.get("metadata") or {})
        return {"skipped": True, "reason": f"unknown job_type {job_type}"}

    def _process_event(self, event: dict[str, Any]) -> dict[str, Any]:
        with self.service.store.connection() as conn:
            source = conn.execute("SELECT * FROM sources WHERE id = ?", (event["source_id"],)).fetchone()
            if source is None:
                raise KeyError(event["source_id"])
            source_dict = dict(source)

        content = source_dict["content"]
        summary = self.service._make_summary(content)
        memory = self.service.ingest(
            {
                "id": make_id("mem"),
                "type": "episode",
                "scope": self._normalize_scope(event["requested_scope"]),
                "status": "inbox",
                "source_kind": source_dict["source_kind"],
                "title": source_dict["title"],
                "content": content,
                "summary": summary,
                "confidence": 0.6,
                "freshness": 1.0,
                "observed_at": source_dict["created_at"],
                "source_ref": source_dict["source_ref"],
                "evidence_ref": source_dict["id"],
                "embedding": self.service._text_embedding(content),
                "metadata": {
                    "ingested_by": "memory-engine",
                    "trigger_type": event["trigger_type"],
                    "captured_at": utcnow(),
                },
            }
        )
        return {"event_id": event["id"], "memory_id": memory["id"]}

    def _normalize_scope(self, requested_scope: str) -> str:
        if requested_scope in {"global", "project", "repo", "agent", "session"}:
            return requested_scope
        return "session"


        updated = service.update_maintenance_job(
            args.job_id,
            cadence=args.cadence,
            interval_minutes=args.interval_minutes,
            window_start=args.window_start,
            window_end=args.window_end,
            next_due_at=next_due_at,
            enabled=enabled,
            metadata=metadata,
        )
        print(json.dumps(updated, indent=2))
    elif args.command == "maintenance-daemon":
        def _tick() -> dict[str, object]:
            results = engine.run_due_maintenance(limit=args.limit, job_id=args.job_id)
            return {"timestamp": time.time(), "results": results}

        print(json.dumps(_tick(), indent=2))
        if args.once:
            return
        while True:
            jitter = random.randint(0, max(0, args.jitter_seconds)) if args.jitter_seconds else 0
            time.sleep(max(1, args.interval_seconds) + jitter)
            print(json.dumps(_tick(), indent=2))
    elif args.command == "serve-http":
        run_http_server(db_path=args.db, host=args.host, port=args.port)
    elif args.command == "search":
        filters = {
            key: value
            for key, value in {
                "project_id": args.project_id,
                "repo_id": args.repo_id,
                "type": args.type,
            }.items()
            if value
        }
        print(json.dumps(service.search(args.query, args.scopes, filters, include_inbox=args.include_inbox), indent=2))
    elif args.command == "context":
        print(json.dumps(service.context_for(args.project, args.repo, args.agent, args.task), indent=2))
    elif args.command == "consolidate":
        print(json.dumps(service.consolidate(args.memory_ids), indent=2))
    elif args.command == "explain":
        print(json.dumps(service.explain(args.identifier), indent=2))
    elif args.command == "list-memories":
        metadata = {}
        for pair in args.metadata:
            key, _, value = pair.partition("=")
            if not key or not _:
                raise ValueError(f"Invalid metadata filter: {pair}")
            metadata[key] = value
        print(
            json.dumps(
                service.list_memories(
                    status=args.status,
                    scope=args.scope,
                    memory_type=args.type,
                    subtype=args.subtype,
                    project_id=args.project_id,
                    repo_id=args.repo_id,
                    run_id=args.run_id,
                    task_id=args.task_id,
                    origin_agent=args.origin_agent,
                    url=args.url,
                    domain=args.domain,
                    source_ref=args.source_ref,
                    evidence_ref=args.evidence_ref,
                    metadata=metadata or None,
                    limit=args.limit,
                ),
                indent=2,
            )
        )
    elif args.command == "task-bundle":
        print(json.dumps(service.task_bundle(args.task_id), indent=2))
    elif args.command == "audit-v2":
        print(json.dumps(service.audit_v2(), indent=2))
    elif args.command == "migrate-v2":
        print(json.dumps(service.migrate_v2(), indent=2))
    elif args.command == "export":
        payload = service.export()
        if args.output:

[9 more lines in file. Use offset=280 to continue.]


ls -la .venv/bin/python* && .venv/bin/python -m pytest -q tests/test_memory_service.py


lrwxrwxrwx 1 sebas sebas  7 Mar 30 14:13 .venv/bin/python -> python3
lrwxrwxrwx 1 sebas sebas 16 Mar 30 14:13 .venv/bin/python3 -> /usr/bin/python3
lrwxrwxrwx 1 sebas sebas  7 Mar 30 14:13 .venv/bin/python3.13 -> python3
..................                                                       [100%]
18 passed in 2.85s


        except HTTPError as exc:
            assert exc.code == 400
        else:
            raise AssertionError("Expected HTTP 400 for search by id")
    finally:
        server.shutdown()
        server.server_close()

def test_maintenance_jobs_exist_by_default(tmp_path: Path) -> None:
    service = make_service(tmp_path)

    jobs = {job["id"] for job in service.list_maintenance_jobs()}

    assert "job_consolidation_daily_10" in jobs
    assert "job_consolidation_daily_15" in jobs
    assert "job_consolidation_weekly" in jobs

def test_maintenance_tick_records_run(tmp_path: Path) -> None:
    service = make_service(tmp_path)
    engine = MemoryEngine(service)

    now = "2026-03-20T10:00:00+00:00"
    service.update_maintenance_job("job_consolidation_daily_10", next_due_at=now)
    results = engine.run_due_maintenance(job_id="job_consolidation_daily_10")

    assert results
    job = service.get_maintenance_job("job_consolidation_daily_10")
    runs = service.list_maintenance_runs(job_id="job_consolidation_daily_10")

    assert job["last_status"] == "completed"
    assert runs

def test_service_startup_runs_due_maintenance(tmp_path: Path) -> None:
    db_path = tmp_path / "memory.sqlite3"
    service = MemoryService(str(db_path))
    now = "2026-03-20T10:00:00+00:00"

    service.update_maintenance_job("job_consolidation_daily_10", next_due_at=now)
    restarted = MemoryService(str(db_path))
    job = restarted.get_maintenance_job("job_consolidation_daily_10")
    runs = restarted.list_maintenance_runs(job_id="job_consolidation_daily_10")

    assert job["last_status"] == "completed"
    assert runs

def test_write_operations_trigger_due_maintenance_check(tmp_path: Path) -> None:
    service = make_service(tmp_path)
    now = "2026-03-20T10:00:00+00:00"

    service.update_maintenance_job("job_consolidation_daily_10", next_due_at=now)
    service.create_project("proj_alpha", "Alpha")

    job = service.get_maintenance_job("job_consolidation_daily_10")
    runs = service.list_maintenance_runs(job_id="job_consolidation_daily_10")

    assert job["last_status"] == "completed"
    assert runs

def test_high_confidence_inbox_memory_auto_promotes_on_write(tmp_path: Path) -> None:
    service = make_service(tmp_path)
    service.update_maintenance_job("job_consolidation_daily_10", next_due_at="2026-03-20T10:00:00+00:00")

    memory = service.create_memory(
        {
            "id": "mem_auto_promote",
            "type": "episode",
            "scope": "project",
            "status": "inbox",
            "title": "Auto promote me",
            "content": "A durable session note that should become active automatically.",
            "confidence": 0.95,
            "freshness": 0.9,
        }
    )

    assert memory["status"] == "active"

def test_conflicts_only_for_selected_types(tmp_path: Path) -> None:
    service = make_service(tmp_path)
    service.create_memory(
        {
            "id": "dec_active",
            "type": "decision",
            "scope": "project",
            "status": "active",
            "title": "Decision A",
            "content": "Feature X is enabled for launch.",
        }
    )
    service.create_memory(
        {
            "id": "dec_inbox",
            "type": "decision",
            "scope": "project",
            "status": "inbox",
            "title": "Decision B",
            "content": "Feature X is not enabled for launch.",
        }
    )
    service.consolidate_candidates(
        config={
            "candidate_age_days": 0,
            "max_candidates": 10,
            "conflict_threshold": 0.1,
            "dedupe_high_threshold": 0.99,
            "dedupe_mid_threshold": 0.99,
            "promote_confidence": 1.0,
            "llm_enabled": False,
        }
    )
    with service.store.connection() as conn:
        conflicts = conn.execute("SELECT COUNT(*) AS count FROM memory_conflicts").fetchone()["count"]

    assert conflicts == 1

def test_migrate_v2_rewrites_legacy_rows_and_drops_legacy_tables(tmp_path: Path) -> None:
    service = make_service(tmp_path)
    with service.store.connection() as conn:
        conn.execute(
            """
            INSERT INTO memories (
                id, type, scope, status, source_kind, title, content, summary, confidence, freshness,
                created_at, updated_at, observed_at, source_ref, evidence_ref, embedding_json, metadata_json
            ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
            """,
            (
                "legacy_run",
                "episode",
                "global",
                "active",
                "run",
                "Research run: Legacy cleanup",
                "Goal: Legacy cleanup\nScope: Repo\nAssumptions: None\nSummary: Pending",

[81 more lines in file. Use offset=470 to continue.]


#!/usr/bin/env bash
set -euo pipefail

REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
DB_PATH="${DB_PATH:-${REPO_ROOT}/data/shared-agent-memory.sqlite3}"
INTERVAL_SECONDS="${INTERVAL_SECONDS:-900}"
JITTER_SECONDS="${JITTER_SECONDS:-60}"

export PATH="/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:${PATH}"

exec python3 -m shared_agent_memory.cli \
  --db "${DB_PATH}" \
  maintenance-daemon \
  --interval-seconds "${INTERVAL_SECONDS}" \
  --jitter-seconds "${JITTER_SECONDS}"


**Investigating script issues**

I’m noticing a potential issue with the wrapper being broken due to no PYTHONPATH or virtual environment. The documentation mentions this was fixed before, but it might have regressed. I need to reproduce the current failure and gather concrete evidence, which likely means inspecting and possibly running the scripts. I wonder if the end-to-end background processing via the HTTP API is connected. I should also search the docs for system-v1 to identify the intended flow for automatic background processing.


- exact duplicates are linked and archived
- conflicting claims produce explicit `memory_conflicts` rows and `contradicts` links
- inbox memories can be promoted to `active` once they pass the configured threshold
- the canonical output is always the shared DB record, not an external report

## Maintenance

```bash
.venv/bin/python -m shared_agent_memory.cli --db data/shared-agent-memory.sqlite3 maintenance-status
.venv/bin/python -m shared_agent_memory.cli --db data/shared-agent-memory.sqlite3 maintenance-tick --limit 10
.venv/bin/python -m shared_agent_memory.cli --db data/shared-agent-memory.sqlite3 maintenance-daemon --interval-seconds 900
```

Versioned service definitions:

- `systemd/user/agents-database-maintenance.service`
- `scripts/run-maintenance-daemon.sh`

Installed Linux user service:

- `~/.config/systemd/user/agents-database-maintenance.service`
- enabled with `systemctl --user enable --now agents-database-maintenance.service`
- `Restart=always` with `RestartSec=10`
- `loginctl show-user sebas --property=Linger` is `yes`, so the user service is eligible to come back after reboot without an interactive login

Operational checks:

```bash
systemctl --user status agents-database-maintenance.service --no-pager
systemctl --user show agents-database-maintenance.service --property=NRestarts,ExecMainPID,SubState
journalctl --user -u agents-database-maintenance.service -n 50 --no-pager
```

macOS launchd template:

- `docs/launchd/agents-database-maintenance.plist`

Current Linux runtime notes:

- the daemon now starts through `scripts/run-maintenance-daemon.sh`
- the wrapper exports `PYTHONPATH` for the repo checkout and prefers `./.venv/bin/python` when present
- the service was verified to auto-restart after a manual `systemctl --user kill agents-database-maintenance.service`
- `MemoryService` also performs opportunistic catch-up checks on startup and after normal write operations, so due jobs do not depend exclusively on the long-lived daemon

## Boundary

- new durable work: shared DB only
- legacy `~/personal-agent/data/personal-agent.sqlite3`: transition-only inspection and migration source

## RFCs

- `docs/memory-consolidation-rfc.md`


Execution (v1):

- Calls the configured CLI runner and passes the prompt via stdin.
- Job metadata can override the command via `llm_command`.

## Scheduler Details (v1)

### Job definition

- Daily light runs at 10:00 and 15:00: dedupe + promotion checks (no LLM by default).
- Weekly deep run: include LLM synthesis and abstraction.

### Next-available algorithm (pseudo)

```python
for job in maintenance_jobs where enabled=1:
  if now >= job.next_due_at:
     run_job(job)
     job.last_run_at = now
     job.next_due_at = compute_next_due(job)
```

`compute_next_due(job)` respects `window_start/window_end`; if outside window, schedule the next window start. If missed, run immediately and schedule the next window.

## Gotchas

- Job windows are interpreted as UTC (based on stored ISO timestamps).
- If the configured runner is unavailable, LLM synthesis is skipped without failing the job.
- `maintenance-daemon` is cooperative; run it via your preferred login/session manager for continuous checks.
- Launchd templates need path edits if your repo lives elsewhere.

CLI usage:

```bash
.venv/bin/python -m shared_agent_memory.cli --db data/shared-agent-memory.sqlite3 maintenance-status
.venv/bin/python -m shared_agent_memory.cli --db data/shared-agent-memory.sqlite3 maintenance-tick
.venv/bin/python -m shared_agent_memory.cli --db data/shared-agent-memory.sqlite3 maintenance-config job_consolidation_daily_10 --window-start 10:00 --recompute-next
.venv/bin/python -m shared_agent_memory.cli --db data/shared-agent-memory.sqlite3 maintenance-daemon --interval-seconds 900
```

Launchd template:

- `docs/launchd/agents-database-maintenance.plist`

Linux systemd user service:

- `systemd/user/agents-database-maintenance.service`
- install with `ln -sf /home/sebas/agents-database/systemd/user/agents-database-maintenance.service /home/sebas/.config/systemd/user/agents-database-maintenance.service`
- enable with `systemctl --user daemon-reload && systemctl --user enable --now agents-database-maintenance.service`
- requires `loginctl enable-linger sebas` if the machine should restart the user service after reboot without waiting for login

## Progress Log (living)

- 2026-03-26: RFC drafted with opportunistic scheduler + consolidation pipeline.
- 2026-03-26: Implemented maintenance tables, scheduler hooks, and consolidation runner (v1).
- 2026-03-27: Added maintenance config + daemon CLI commands.
- 2026-03-30: Fixed the maintenance daemon wrapper to run from a repo checkout by exporting `PYTHONPATH` and preferring `./.venv/bin/python`.
- 2026-03-30: Added and enabled a Linux `systemd --user` service with `Restart=always`; verified auto-restart after kill and confirmed `Linger=yes` for reboot persistence.

## Open Questions

- Whether to run the engine as a long-lived background process or on-demand via CLI.
- Which storage to use for LLM usage counters (likely `maintenance_runs.stats_json`).

## Proposed Next Steps

1. Add scheduler hook coverage beyond the daemon path if we want startup/write-triggered opportunistic runs inside normal service usage.
2. Add tests for daemon/bootstrap operation and job catch-up across missed windows.
3. Add LLM budget counters per run/day to `maintenance_runs.stats_json` or a dedicated table.
4. Decide whether to keep Linux `systemd --user` as the primary runtime and treat launchd as secondary documentation.


/usr/bin/python3: Error while finding module specification for 'shared_agent_memory.cli' (ModuleNotFoundError: No module named 'shared_agent_memory')

Command exited with code 1


**Inspecting systemd service**

I see that inspecting the systemd service is a priority. I need to check the configurations, logs, and status of the service to understand its operation better. It might reveal any issues or optimizations I could apply. I’ll ensure to follow up on this after addressing the other reports. There’s definitely a lot to consider when analyzing system services, and I want to get it all right!


ENOENT: no such file or directory, access '/mnt/rpi/agents-database/systemd/user/agents-database-maintenance.service'


from __future__ import annotations

import argparse
import json
import random
import time
from pathlib import Path

from .engine import MemoryEngine
from .http_api import run_http_server
from .service import MemoryService

def build_parser() -> argparse.ArgumentParser:
    parser = argparse.ArgumentParser(prog="shared-agent-memory")
    parser.add_argument("--db", default="shared-agent-memory.sqlite3")

    subparsers = parser.add_subparsers(dest="command", required=True)

    subparsers.add_parser("init-db")

    project = subparsers.add_parser("create-project")
    project.add_argument("project_id")
    project.add_argument("name")
    project.add_argument("--description")

    repo = subparsers.add_parser("create-repo")
    repo.add_argument("repo_id")
    repo.add_argument("name")
    repo.add_argument("--project-id")
    repo.add_argument("--path")

    profile = subparsers.add_parser("upsert-profile-fact")
    profile.add_argument("title")
    profile.add_argument("content")
    profile.add_argument("--source-ref")
    profile.add_argument("--evidence-ref")

    ingest = subparsers.add_parser("ingest")
    ingest.add_argument("payload_json")

    capture = subparsers.add_parser("capture-conversation")
    capture.add_argument("source_ref")
    capture.add_argument("scope")
    capture.add_argument("content")
    capture.add_argument("--capture-mode", default="manual_import")

    session = subparsers.add_parser("capture-session")
    session.add_argument("source_ref")
    session.add_argument("scope")
    session.add_argument("content")

    process = subparsers.add_parser("process-pending")
    process.add_argument("--limit", type=int)

    maintenance_tick = subparsers.add_parser("maintenance-tick")
    maintenance_tick.add_argument("--limit", type=int)
    maintenance_tick.add_argument("--job-id")

    maintenance_status = subparsers.add_parser("maintenance-status")
    maintenance_status.add_argument("--runs", type=int, default=10)

    maintenance_config = subparsers.add_parser("maintenance-config")
    maintenance_config.add_argument("job_id")
    maintenance_config.add_argument("--cadence")
    maintenance_config.add_argument("--interval-minutes", type=int)
    maintenance_config.add_argument("--window-start")
    maintenance_config.add_argument("--window-end")
    maintenance_config.add_argument("--enable", action="store_true")
    maintenance_config.add_argument("--disable", action="store_true")
    maintenance_config.add_argument("--metadata-json")
    maintenance_config.add_argument("--recompute-next", action="store_true")

    maintenance_daemon = subparsers.add_parser("maintenance-daemon")
    maintenance_daemon.add_argument("--interval-seconds", type=int, default=900)
    maintenance_daemon.add_argument("--jitter-seconds", type=int, default=60)
    maintenance_daemon.add_argument("--limit", type=int)
    maintenance_daemon.add_argument("--job-id")
    maintenance_daemon.add_argument("--once", action="store_true")

    serve_http = subparsers.add_parser("serve-http")
    serve_http.add_argument("--host", default="127.0.0.1")
    serve_http.add_argument("--port", type=int, default=8091)

    search = subparsers.add_parser("search")
    search.add_argument("query")
    search.add_argument("--scopes", nargs="*")
    search.add_argument("--project-id")
    search.add_argument("--repo-id")
    search.add_argument("--type")
    search.add_argument("--include-inbox", action="store_true")

    context = subparsers.add_parser("context")
    context.add_argument("--project")
    context.add_argument("--repo")
    context.add_argument("--agent")
    context.add_argument("--task")

    consolidate = subparsers.add_parser("consolidate")
    consolidate.add_argument("memory_ids", nargs="+")

    explain = subparsers.add_parser("explain")
    explain.add_argument("identifier")

    list_memories = subparsers.add_parser("list-memories")
    list_memories.add_argument("--status", default="active")
    list_memories.add_argument("--scope")
    list_memories.add_argument("--type")
    list_memories.add_argument("--subtype")
    list_memories.add_argument("--project-id")
    list_memories.add_argument("--repo-id")
    list_memories.add_argument("--run-id")
    list_memories.add_argument("--task-id")
    list_memories.add_argument("--origin-agent")
    list_memories.add_argument("--url")
    list_memories.add_argument("--domain")
    list_memories.add_argument("--source-ref")
    list_memories.add_argument("--evidence-ref")
    list_memories.add_argument("--metadata", action="append", default=[])
    list_memories.add_argument("--limit", type=int, default=50)

[168 more lines in file. Use offset=121 to continue.]


find . -path '*maintenance*.service' -o -path '*http*.service' | sort


**Investigating maintenance issues**

I'm noticing there’s no maintenance service file, even though the documentation suggests there should be one. This seems like it could be an issue worth looking into. I think I should inspect the `scripts/run-http-api.sh` file to identify any patterns or problems that might give me more insight. It's always a bit puzzling when things don’t line up with the docs, so I’m curious to dig deeper here.


#!/usr/bin/env bash
set -euo pipefail

REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
DB_PATH="${DB_PATH:-${REPO_ROOT}/data/shared-agent-memory.sqlite3}"
HOST="${HOST:-0.0.0.0}"
PORT="${PORT:-8091}"

export PATH="/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:${PATH}"
export PYTHONPATH="${REPO_ROOT}/src${PYTHONPATH:+:${PYTHONPATH}}"

exec python3 -m shared_agent_memory.cli \
  --db "${DB_PATH}" \
  serve-http \
  --host "${HOST}" \
  --port "${PORT}"


fatal: path 'systemd/user/agents-database-maintenance.service' does not exist in 'HEAD~1'

Command exited with code 128


**Considering maintenance and testing**

I'm thinking that maybe a maintenance service document was supposed to be added, but it seems the file is missing. The user is asking for an update to make sure the docs, scripts, and tests align with the final behavior. I’ll probably need to create a service file too. It might be necessary to develop a test for the wrapper, perhaps a unit test for script existence. I'll also check if there are any existing tests for the scripts in the repo structure.

find tests -maxdepth 2 -type f | sort && echo '---' && find systemd -maxdepth 3 -type f | sort && echo '---' && ls -la scripts


tests/conftest.py
tests/__pycache__/conftest.cpython-313-pytest-9.0.2.pyc
tests/__pycache__/test_memory_service.cpython-313-pytest-9.0.2.pyc
tests/test_memory_service.py
---
systemd/user/agents-database-http.service
---
total 24
drwxrwxr-x  2 sebas sebas 4096 Apr 23 12:41 .
drwxrwxr-x 11 sebas sebas 4096 Apr 22 16:57 ..
-rw-r--r--  1 sebas sebas 5757 Apr 23 12:41 export-db-snapshot-html.py
-rwxr-xr-x  1 sebas sebas  469 Apr 22 16:57 run-http-api.sh
-rw-r--r--  1 sebas sebas  493 Apr 22 14:06 run-maintenance-daemon.sh


**Inspecting script execution**

I’m thinking about whether I need to fix the script to make it executable. The "run-maintenance-daemon" isn't currently executable, which might matter for the systemd ExecStart if it relies on a bash path. If the service runs the script directly, it definitely needs that executable bit. Okay, let’s investigate the HTTP service file pattern to see what’s going on there. I want to make sure everything is set up correctly!


[Unit]
Description=Agents Database HTTP API
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
WorkingDirectory=/home/sebas/agents-database
Environment=DB_PATH=/home/sebas/agents-database/data/shared-agent-memory.sqlite3
Environment=HOST=0.0.0.0
Environment=PORT=8091
ExecStart=/home/sebas/agents-database/scripts/run-http-api.sh
Restart=always
RestartSec=10

[Install]
WantedBy=default.target


from __future__ import annotations

import sys
from pathlib import Path

ROOT = Path(__file__).resolve().parents[1]
SRC = ROOT / "src"

if str(SRC) not in sys.path:
    sys.path.insert(0, str(SRC))


edit

#!/usr/bin/env bash
set -euo pipefail

REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
DB_PATH="${DB_PATH:-${REPO_ROOT}/data/shared-agent-memory.sqlite3}"
INTERVAL_SECONDS="${INTERVAL_SECONDS:-900}"
JITTER_SECONDS="${JITTER_SECONDS:-60}"

export PATH="/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:${PATH}"

exec python3 -m shared_agent_memory.cli \
  --db "${DB_PATH}" \
  maintenance-daemon \
  --interval-seconds "${INTERVAL_SECONDS}" \
  --jitter-seconds "${JITTER_SECONDS}"

#!/usr/bin/env bash
set -euo pipefail

REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
DB_PATH="${DB_PATH:-${REPO_ROOT}/data/shared-agent-memory.sqlite3}"
INTERVAL_SECONDS="${INTERVAL_SECONDS:-900}"
JITTER_SECONDS="${JITTER_SECONDS:-60}"
PYTHON_BIN="${PYTHON_BIN:-}"

export PATH="/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:${PATH}"
export PYTHONPATH="${REPO_ROOT}/src${PYTHONPATH:+:${PYTHONPATH}}"

if [[ -z "${PYTHON_BIN}" ]]; then
  if [[ -x "${REPO_ROOT}/.venv/bin/python" ]]; then
    PYTHON_BIN="${REPO_ROOT}/.venv/bin/python"
  else
    PYTHON_BIN="python3"
  fi
fi

exec "${PYTHON_BIN}" -m shared_agent_memory.cli \
  --db "${DB_PATH}" \
  maintenance-daemon \
  --interval-seconds "${INTERVAL_SECONDS}" \
  --jitter-seconds "${JITTER_SECONDS}" \
  "$@"


Successfully replaced 1 block(s) in scripts/run-maintenance-daemon.sh.

    ...
  4 REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
  5 DB_PATH="${DB_PATH:-${REPO_ROOT}/data/shared-agent-memory.sqlite3}"
  6 INTERVAL_SECONDS="${INTERVAL_SECONDS:-900}"
  7 JITTER_SECONDS="${JITTER_SECONDS:-60}"
+ 8 PYTHON_BIN="${PYTHON_BIN:-}"
  8
  9 export PATH="/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:${PATH}"
+11 export PYTHONPATH="${REPO_ROOT}/src${PYTHONPATH:+:${PYTHONPATH}}"
 10
-11 exec python3 -m shared_agent_memory.cli \
+13 if [[ -z "${PYTHON_BIN}" ]]; then
+14   if [[ -x "${REPO_ROOT}/.venv/bin/python" ]]; then
+15     PYTHON_BIN="${REPO_ROOT}/.venv/bin/python"
+16   else
+17     PYTHON_BIN="python3"
+18   fi
+19 fi
+20
+21 exec "${PYTHON_BIN}" -m shared_agent_memory.cli \
 12   --db "${DB_PATH}" \
 13   maintenance-daemon \
 14   --interval-seconds "${INTERVAL_SECONDS}" \
-15   --jitter-seconds "${JITTER_SECONDS}"
+25   --jitter-seconds "${JITTER_SECONDS}" \
+26   "$@"


write

[Unit]
Description=Agents Database Maintenance Daemon
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
WorkingDirectory=/home/sebas/agents-database
Environment=DB_PATH=/home/sebas/agents-database/data/shared-agent-memory.sqlite3
Environment=INTERVAL_SECONDS=900
Environment=JITTER_SECONDS=60
ExecStart=/home/sebas/agents-database/scripts/run-maintenance-daemon.sh
Restart=always
RestartSec=10

[Install]
WantedBy=default.target


Successfully wrote 458 bytes to systemd/user/agents-database-maintenance.service


bash

chmod +x scripts/run-maintenance-daemon.sh && stat -c '%A %n' scripts/run-maintenance-daemon.sh systemd/user/agents-database-maintenance.service


-rwxr-xr-x scripts/run-maintenance-daemon.sh
-rw-r--r-- systemd/user/agents-database-maintenance.service


from __future__ import annotations

import json
import threading
from pathlib import Path
from urllib.error import HTTPError
from urllib.request import Request, urlopen

from http.server import ThreadingHTTPServer

from shared_agent_memory import MemoryEngine, MemoryService
from shared_agent_memory.http_api import build_handler
from shared_agent_memory.schema import SCHEMA_VERSION

def make_service(tmp_path: Path) -> MemoryService:
    return MemoryService(str(tmp_path / "memory.sqlite3"))

def _request_json(url: str, *, method: str = "GET", payload: dict | None = None) -> tuple[int, dict]:
    body = None if payload is None else json.dumps(payload).encode("utf-8")
    request = Request(url, data=body, method=method)
    if body is not None:
        request.add_header("Content-Type", "application/json")
    with urlopen(request) as response:  # noqa: S310 - local test server only
        return response.status, json.loads(response.read().decode("utf-8"))

def _start_api(service: MemoryService) -> tuple[ThreadingHTTPServer, str]:
    server = ThreadingHTTPServer(("127.0.0.1", 0), build_handler(service))
    thread = threading.Thread(target=server.serve_forever, daemon=True)
    thread.start()
    host, port = server.server_address
    return server, f"http://{host}:{port}"

def test_manual_profile_fact_is_immediately_searchable(tmp_path: Path) -> None:
    service = make_service(tmp_path)
    service.upsert_profile_fact("Communication", "Sebas prefers concise, direct communication.")

    results = service.search("concise communication", scopes=["global"])

    assert results["results"]
    assert results["results"][0]["memory"]["type"] == "profile"
    assert results["results"][0]["memory"]["schema_version"] == SCHEMA_VERSION

def test_captured_conversation_stays_inbox_until_consolidated(tmp_path: Path) -> None:
    service = make_service(tmp_path)
    engine = MemoryEngine(service)

    service.capture_conversation("chat-1", "project", "We agreed to keep one active machine at a time.")
    processed = engine.process_pending()

    memory_id = processed[0]["memory_id"]
    memory = service.get_memory(memory_id)
    assert memory["status"] == "inbox"

    search_without_inbox = service.search("active machine", scopes=["project"])
    assert search_without_inbox["results"] == []

    promoted = service.consolidate([memory_id])
    assert promoted["promoted"] == [memory_id]

    search_with_active = service.search("active machine", scopes=["project"])
    assert search_with_active["results"][0]["memory"]["id"] == memory_id

def test_context_bundle_combines_global_project_and_task_artifacts(tmp_path: Path) -> None:
    service = make_service(tmp_path)
    service.create_project("proj_alpha", "Alpha")
    service.create_repo("repo_alpha", "Repo Alpha", project_id="proj_alpha")
    service.upsert_profile_fact("Preferences", "Sebas wants agents to preserve clarity.")
    service.create_memory(
        {
            "type": "project",
            "scope": "project",
            "project_id": "proj_alpha",
            "repo_id": "repo_alpha",
            "title": "Project goal",

[470 more lines in file. Use offset=81 to continue.]


# Agents Database in System V1

`agents-database` is the durable store for System V1.

Canonical topology and machine bootstrap live in `~/personal-agent/docs/system-v1.md`.

## V1 Role

- canonical shared DB for memories, operational tasks, task runs, approvals, artifacts, and handoffs
- resumability anchor for multi-repo work
- migration target for legacy `personal-agent` state
- authoritative place to implement memory promotion, consolidation, and conflict handling

## Resume From Shared DB

Use the local CLI against `data/shared-agent-memory.sqlite3`:

```bash
python3 -m shared_agent_memory.cli --db data/shared-agent-memory.sqlite3 task-bundle task_<id>
python3 -m shared_agent_memory.cli --db data/shared-agent-memory.sqlite3 list-memories \
  --source-ref personal-agent:run:<run-id>
```

## Runtime Dependencies

- the configured CLI runner must be available for LLM consolidation steps (MiniMax M2.5 Free).
- local verification tools live in `./.venv` for this repo

## Memory pipeline

The memory pipeline lives in `MemoryService` / `MemoryEngine`, not in repo-local side scripts.

Current behavior:

- `ingest()` accepts durable memory writes
- `capture_session()` and `capture_conversation()` create raw source material
- `process_pending()` turns pending ingestion events into episode memories
- `consolidate_candidates(config)` handles dedup, conflict detection, promotion, and optional synthesis
- `maintenance_jobs` + `maintenance_runs` drive scheduled consolidation
- opportunistic checks run on startup and after writes

Promotion and conflict semantics:

- `episode` and `inbox` records are candidates, not truth by default
- exact duplicates are linked and archived
- conflicting claims produce explicit `memory_conflicts` rows and `contradicts` links
- inbox memories can be promoted to `active` once they pass the configured threshold
- the canonical output is always the shared DB record, not an external report

## Maintenance

```bash
.venv/bin/python -m shared_agent_memory.cli --db data/shared-agent-memory.sqlite3 maintenance-status
.venv/bin/python -m shared_agent_memory.cli --db data/shared-agent-memory.sqlite3 maintenance-tick --limit 10
.venv/bin/python -m shared_agent_memory.cli --db data/shared-agent-memory.sqlite3 maintenance-daemon --interval-seconds 900
```

Versioned service definitions:

- `systemd/user/agents-database-maintenance.service`
- `scripts/run-maintenance-daemon.sh`

Installed Linux user service:

- `~/.config/systemd/user/agents-database-maintenance.service`
- enabled with `systemctl --user enable --now agents-database-maintenance.service`
- `Restart=always` with `RestartSec=10`
- `loginctl show-user sebas --property=Linger` is `yes`, so the user service is eligible to come back after reboot without an interactive login

Operational checks:

```bash
systemctl --user status agents-database-maintenance.service --no-pager
systemctl --user show agents-database-maintenance.service --property=NRestarts,ExecMainPID,SubState
journalctl --user -u agents-database-maintenance.service -n 50 --no-pager
```

macOS launchd template:

- `docs/launchd/agents-database-maintenance.plist`

Current Linux runtime notes:

- the daemon now starts through `scripts/run-maintenance-daemon.sh`
- the wrapper exports `PYTHONPATH` for the repo checkout and prefers `./.venv/bin/python` when present
- the service was verified to auto-restart after a manual `systemctl --user kill agents-database-maintenance.service`
- `MemoryService` also performs opportunistic catch-up checks on startup and after normal write operations, so due jobs do not depend exclusively on the long-lived daemon

## Boundary

- new durable work: shared DB only
- legacy `~/personal-agent/data/personal-agent.sqlite3`: transition-only inspection and migration source

## RFCs

- `docs/memory-consolidation-rfc.md`


edit

from __future__ import annotations

import json
import threading
from pathlib import Path
from urllib.error import HTTPError
from urllib.request import Request, urlopen

from http.server import ThreadingHTTPServer

from __future__ import annotations

import json
import os
import subprocess
import threading
from pathlib import Path
from urllib.error import HTTPError
from urllib.request import Request, urlopen

from http.server import ThreadingHTTPServer

def test_run_maintenance_daemon_wrapper_runs_from_repo_checkout(tmp_path: Path) -> None:
    db_path = tmp_path / "memory.sqlite3"
    service = MemoryService(str(db_path))
    service.update_maintenance_job("job_consolidation_daily_10", next_due_at="2026-03-20T10:00:00+00:00")

    script = Path(__file__).resolve().parents[1] / "scripts" / "run-maintenance-daemon.sh"
    env = os.environ.copy()
    env["DB_PATH"] = str(db_path)
    env["INTERVAL_SECONDS"] = "1"
    env["JITTER_SECONDS"] = "0"
    result = subprocess.run(
        [str(script), "--once", "--job-id", "job_consolidation_daily_10"],
        capture_output=True,
        text=True,
        check=True,
        env=env,
    )

    restarted = MemoryService(str(db_path))
    job = restarted.get_maintenance_job("job_consolidation_daily_10")
    runs = restarted.list_maintenance_runs(job_id="job_consolidation_daily_10")

    assert '"status": "completed"' in result.stdout
    assert job["last_status"] == "completed"
    assert runs

def test_versioned_maintenance_systemd_service_exists() -> None:
    service_file = Path(__file__).resolve().parents[1] / "systemd" / "user" / "agents-database-maintenance.service"
    content = service_file.read_text()

    assert service_file.exists()
    assert "ExecStart=/home/sebas/agents-database/scripts/run-maintenance-daemon.sh" in content

def test_migrate_v2_rewrites_legacy_rows_and_drops_legacy_tables(tmp_path: Path) -> None:


Successfully replaced 2 block(s) in tests/test_memory_service.py.

   1 from __future__ import annotations
   2
   3 import json
+  4 import os
+  5 import subprocess
   4 import threading
   5 from pathlib import Path
   6 from urllib.error import HTTPError
   7 from urllib.request import Request, urlopen
     ...
 448
 449     assert conflicts == 1
 450
 451
+454 def test_run_maintenance_daemon_wrapper_runs_from_repo_checkout(tmp_path: Path) -> None:
+455     db_path = tmp_path / "memory.sqlite3"
+456     service = MemoryService(str(db_path))
+457     service.update_maintenance_job("job_consolidation_daily_10", next_due_at="2026-03-20T10:00:00+00:00")
+458
+459     script = Path(__file__).resolve().parents[1] / "scripts" / "run-maintenance-daemon.sh"
+460     env = os.environ.copy()
+461     env["DB_PATH"] = str(db_path)
+462     env["INTERVAL_SECONDS"] = "1"
+463     env["JITTER_SECONDS"] = "0"
+464     result = subprocess.run(
+465         [str(script), "--once", "--job-id", "job_consolidation_daily_10"],
+466         capture_output=True,
+467         text=True,
+468         check=True,
+469         env=env,
+470     )
+471
+472     restarted = MemoryService(str(db_path))
+473     job = restarted.get_maintenance_job("job_consolidation_daily_10")
+474     runs = restarted.list_maintenance_runs(job_id="job_consolidation_daily_10")
+475
+476     assert '"status": "completed"' in result.stdout
+477     assert job["last_status"] == "completed"
+478     assert runs
+479
+480
+481 def test_versioned_maintenance_systemd_service_exists() -> None:
+482     service_file = Path(__file__).resolve().parents[1] / "systemd" / "user" / "agents-database-maintenance.service"
+483     content = service_file.read_text()
+484
+485     assert service_file.exists()
+486     assert "ExecStart=/home/sebas/agents-database/scripts/run-maintenance-daemon.sh" in content
+487
+488
 452 def test_migrate_v2_rewrites_legacy_rows_and_drops_legacy_tables(tmp_path: Path) -> None:
 453     service = make_service(tmp_path)
 454     with service.store.connection() as conn:
 455         conn.execute(
     ...


**Validating file operations**

I think I need to check if the service file exists before attempting to read its text. But hey, the path itself does seem to be there. Maybe it's a good idea to run some tests and try to reproduce the issue before making any fixes? It could be smart to save the command output after fixing things too. I’ll definitely want to note any earlier failures to reference later. Now, let’s validate everything!

bash

.venv/bin/python -m pytest -q tests/test_memory_service.py

DB_PATH=/tmp/agents-db-maint-test.sqlite3 INTERVAL_SECONDS=1 JITTER_SECONDS=0 bash scripts/run-maintenance-daemon.sh --once --job-id job_consolidation_daily_10


....................                                                     [100%]
20 passed in 3.01s


**Validating background processing**

I need to validate the end-to-end auto-promote for background processing using a temporary database and a scheduled job. Existing tests seem to cover most aspects. I might want to run an explicit command script, perhaps a Python snippet, to demonstrate how memory auto-promotes work. I’m thinking about using `.venv/bin/python - <<'PY'`, which will let me run the real repository code. I can’t wait to see how this all ties together!

.venv/bin/python - <<'PY'
from pathlib import Path
from tempfile import TemporaryDirectory
from shared_agent_memory import MemoryService

with TemporaryDirectory() as d:
    db = str(Path(d) / 'memory.sqlite3')
    service = MemoryService(db)
    service.update_maintenance_job('job_consolidation_daily_10', next_due_at='2026-03-20T10:00:00+00:00')
    memory = service.create_memory({
        'id': 'mem_auto_promote_demo',
        'type': 'episode',
        'scope': 'project',
        'status': 'inbox',
        'title': 'Auto promote demo',
        'content': 'Durable note promoted by due maintenance on write.',
        'confidence': 0.95,
        'freshness': 0.9,
    })
    job = service.get_maintenance_job('job_consolidation_daily_10')
    runs = service.list_maintenance_runs(job_id='job_consolidation_daily_10')
    print({'memory_status': memory['status'], 'job_last_status': job['last_status'], 'run_count': len(runs)})
PY


Traceback (most recent call last):
  File "<stdin>", line 3, in <module>
ModuleNotFoundError: No module named 'shared_agent_memory'

Command exited with code 1


**Considering Python setup**

I’m trying to figure out whether I need to use PYTHONPATH or the -m flag. It seems like checking the environment variables could be a good approach to clarify that. Using the correct settings is essential, but I'm curious if there's a difference in how they impact script execution. Let's explore these options together to ensure everything runs smoothly!

PYTHONPATH=src .venv/bin/python - <<'PY'
from pathlib import Path
from tempfile import TemporaryDirectory
from shared_agent_memory import MemoryService

with TemporaryDirectory() as d:
    db = str(Path(d) / 'memory.sqlite3')
    service = MemoryService(db)
    service.update_maintenance_job('job_consolidation_daily_10', next_due_at='2026-03-20T10:00:00+00:00')
    memory = service.create_memory({
        'id': 'mem_auto_promote_demo',
        'type': 'episode',
        'scope': 'project',
        'status': 'inbox',
        'title': 'Auto promote demo',
        'content': 'Durable note promoted by due maintenance on write.',
        'confidence': 0.95,
        'freshness': 0.9,
    })
    job = service.get_maintenance_job('job_consolidation_daily_10')
    runs = service.list_maintenance_runs(job_id='job_consolidation_daily_10')
    print({'memory_status': memory['status'], 'job_last_status': job['last_status'], 'run_count': len(runs)})
PY


{'memory_status': 'active', 'job_last_status': 'completed', 'run_count': 1}


**Checking git changes**

I think I need to check the git diff to see what changes have been made. This can help me understand the differences between the files or commits. I should look for specific changes that were made, maybe paying attention to both additions and deletions. It's important to clearly see what's been altered so I can address anything that needs fixing. Let's dive into the diff and figure it out!

git status --short && echo '---' && git diff -- scripts/run-maintenance-daemon.sh systemd/user/agents-database-maintenance.service tests/test_memory_service.py


 M data/shared-agent-memory.sqlite3
 M scripts/run-maintenance-daemon.sh
 M tests/test_memory_service.py
?? scripts/export-db-snapshot-html.py
?? systemd/user/agents-database-maintenance.service
---
diff --git a/scripts/run-maintenance-daemon.sh b/scripts/run-maintenance-daemon.sh
old mode 100644
new mode 100755
index 835eb0a..7708531
--- a/scripts/run-maintenance-daemon.sh
+++ b/scripts/run-maintenance-daemon.sh
@@ -5,11 +5,22 @@ REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
 DB_PATH="${DB_PATH:-${REPO_ROOT}/data/shared-agent-memory.sqlite3}"
 INTERVAL_SECONDS="${INTERVAL_SECONDS:-900}"
 JITTER_SECONDS="${JITTER_SECONDS:-60}"
+PYTHON_BIN="${PYTHON_BIN:-}"

 export PATH="/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:${PATH}"
+export PYTHONPATH="${REPO_ROOT}/src${PYTHONPATH:+:${PYTHONPATH}}"

-exec python3 -m shared_agent_memory.cli \
+if [[ -z "${PYTHON_BIN}" ]]; then
+  if [[ -x "${REPO_ROOT}/.venv/bin/python" ]]; then
+    PYTHON_BIN="${REPO_ROOT}/.venv/bin/python"
+  else
+    PYTHON_BIN="python3"
+  fi
+fi
+
+exec "${PYTHON_BIN}" -m shared_agent_memory.cli \
   --db "${DB_PATH}" \
   maintenance-daemon \
   --interval-seconds "${INTERVAL_SECONDS}" \
-  --jitter-seconds "${JITTER_SECONDS}"
+  --jitter-seconds "${JITTER_SECONDS}" \
+  "$@"
diff --git a/tests/test_memory_service.py b/tests/test_memory_service.py
index 11efe67..9a38106 100644
--- a/tests/test_memory_service.py
+++ b/tests/test_memory_service.py
@@ -1,6 +1,8 @@
 from __future__ import annotations

 import json
+import os
+import subprocess
 import threading
 from pathlib import Path
 from urllib.error import HTTPError
@@ -449,6 +451,41 @@ def test_conflicts_only_for_selected_types(tmp_path: Path) -> None:
     assert conflicts == 1

+def test_run_maintenance_daemon_wrapper_runs_from_repo_checkout(tmp_path: Path) -> None:
+    db_path = tmp_path / "memory.sqlite3"
+    service = MemoryService(str(db_path))
+    service.update_maintenance_job("job_consolidation_daily_10", next_due_at="2026-03-20T10:00:00+00:00")
+
+    script = Path(__file__).resolve().parents[1] / "scripts" / "run-maintenance-daemon.sh"
+    env = os.environ.copy()
+    env["DB_PATH"] = str(db_path)
+    env["INTERVAL_SECONDS"] = "1"
+    env["JITTER_SECONDS"] = "0"
+    result = subprocess.run(
+        [str(script), "--once", "--job-id", "job_consolidation_daily_10"],
+        capture_output=True,
+        text=True,
+        check=True,
+        env=env,
+    )
+
+    restarted = MemoryService(str(db_path))
+    job = restarted.get_maintenance_job("job_consolidation_daily_10")
+    runs = restarted.list_maintenance_runs(job_id="job_consolidation_daily_10")
+
+    assert '"status": "completed"' in result.stdout
+    assert job["last_status"] == "completed"
+    assert runs
+
+
+def test_versioned_maintenance_systemd_service_exists() -> None:
+    service_file = Path(__file__).resolve().parents[1] / "systemd" / "user" / "agents-database-maintenance.service"
+    content = service_file.read_text()
+
+    assert service_file.exists()
+    assert "ExecStart=/home/sebas/agents-database/scripts/run-maintenance-daemon.sh" in content
+
+
 def test_migrate_v2_rewrites_legacy_rows_and_drops_legacy_tables(tmp_path: Path) -> None:
     service = make_service(tmp_path)
     with service.store.connection() as conn:


**Considering project maintenance**

I should think about mentioning any unrelated dirty files. It might make sense to inspect if the documents are already accurate, which they seem to be now. I'm wondering if there’s still a need for additional edits. Maybe I should save memory by creating a project fact about needing an "agents-database maintenance wrapper" in the PYTHONPATH. Using the agents-db API could be helpful but isn't strictly necessary. I'll store a concise record of these decisions.


---
name: memory-write
description: Save a durable high-signal memory into agents-database
---

# memory-write

## Use When

- new information is likely useful beyond current chat or task
- saving a decision, preference, project fact, reusable source/artifact, or handoff summary
- user asks to remember something durable

## Do Not Use When

- content is trivial, redundant, ephemeral, or likely useless later
- content is secret unless user explicitly asked to store it

## Required Workflow

1. If useful context may already exist, optionally check first:
   - `curl -fsSL "${AGENTS_DB_API:-http://100.116.176.16:8091}/api/search?q=<query>&scope=global&scope=project&limit=10"`
2. Choose best memory type:
   - `profile`: stable user preference, priority, recurring constraint
   - `decision`: chosen approach or operating rule
   - `project`: durable project fact or invariant
   - `artifact`: reusable source, claim, reference, output
   - `episode`: concise session summary, blocked state, handoff context
3. Build a concise payload with:
   - required: `type`, `scope`, `title`, `content`
   - recommended when known: `subtype`, `project_id`, `repo_id`, `task_id`, `run_id`, `source_ref`, `origin_agent`, `url`, `domain`, `metadata`
4. Save it with the helper:
   - `~/.agents/skills/memory-write/memory-write --type <type> --scope <scope> --title "<title>" --content "<content>"`
5. Confirm the created memory id and what was stored.

## Quick Examples

```bash
~/.agents/skills/memory-write/memory-write \
  --type profile \
  --subtype working_preference \
  --scope global \
  --title "User prefers concise status updates" \
  --content "Sebas prefers short, direct updates with business impact first." \
  --source-ref "conversation:2026-04-17" \
  --origin-agent personal-agent

~/.agents/skills/memory-write/memory-write \
  --type decision \
  --subtype operating_rule \
  --scope global \
  --title "Agents should save durable high-signal memory opportunistically" \
  --content "Treat agents-database as shared durable memory, not only search." \
  --metadata '{"source":"conversation"}'
```

## Rules

- prefer one strong memory over many weak ones
- keep `title` short and searchable
- keep `content` factual and compact
- use `episode` for summaries, not for every chat turn
- include `url` and `domain` for web-derived artifacts when known
- include `project_id`, `repo_id`, `task_id`, and `run_id` when context exists


**Considering storage options**

I'm thinking about storing the episode. While it may not be strictly necessary, it could be useful later on. I wonder if using a bash helper could simplify the process. It seems like a catchy solution, but I want to ensure it's indeed the right approach. Overall, I'll proceed with this plan but keep in mind the potential need for revisiting this storage choice in the future.

~/.agents/skills/memory-write/memory-write --type episode --scope project --title "agents-database maintenance wrapper fix" --content "On branch feat/memory-promotion-automation-clean, automatic maintenance was broken because scripts/run-maintenance-daemon.sh launched python3 without repo PYTHONPATH and without preferring .venv, causing ModuleNotFoundError for shared_agent_memory.cli from a repo checkout. Fixed wrapper, restored executable bit, added versioned systemd user service file, and added subprocess coverage for the wrapper path." --project-id agents-database --repo-id agents-database --source-ref "conversation:2026-04-30:maintenance-wrapper-fix"


Traceback (most recent call last):
  File "/home/sebas/.agents/skills/memory-write/memory-write", line 86, in <module>
    main()
    ~~~~^^
  File "/home/sebas/.agents/skills/memory-write/memory-write", line 80, in main
    with urllib.request.urlopen(req, timeout=30) as r:
         ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/urllib/request.py", line 189, in urlopen
    return opener.open(url, data, timeout)
           ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/urllib/request.py", line 495, in open
    response = meth(req, response)
  File "/usr/lib/python3.13/urllib/request.py", line 604, in http_response
    response = self.parent.error(
        'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.13/urllib/request.py", line 533, in error
    return self._call_chain(*args)
           ~~~~~~~~~~~~~~~~^^^^^^^
  File "/usr/lib/python3.13/urllib/request.py", line 466, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.13/urllib/request.py", line 613, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 500: FOREIGN KEY constraint failed

Command exited with code 1


~/.agents/skills/memory-write/memory-write --type episode --scope global --title "agents-database maintenance wrapper fix" --content "On branch feat/memory-promotion-automation-clean, automatic maintenance was broken because scripts/run-maintenance-daemon.sh launched python3 without repo PYTHONPATH and without preferring .venv, causing ModuleNotFoundError for shared_agent_memory.cli from a repo checkout. Fixed wrapper, restored executable bit, added versioned systemd user service file, and added subprocess coverage for the wrapper path." --source-ref "conversation:2026-04-30:maintenance-wrapper-fix"