| License | CC0 (see COPYING) |
| Language | Python 3 (PEP 668 host — always use a venv, never system pip) |
| Runtime deps | =model2vec=, =sqlite-vec= |
+| Recall | =prefetch()= injects up to 30 memories per turn; ids already returned in the session are filtered out |
| Embedding model | =minishlab/potion-multilingual-128M= (~540 MB, cached offline; the repo's unused ONNX export is skipped) |
-| Plugin metadata | plugin.yaml (version 1.1.1) |
+| Plugin metadata | plugin.yaml (version 1.2.0) |
| Public docs site | https://www.svjatoslav.eu/projects/ (served from Documentation/) |
* Purpose
session. It is a Hermes =MemoryProvider= plugin: the LLM stores and
recalls facts through four =retinue_memory_*= tools backed by
model2vec embeddings and sqlite-vec vector search, fully local,
-CPU-only, single SQLite file.
+CPU-only, single SQLite file. Retrieval is not only explicit: the
+=prefetch()= hook injects relevant memories before each turn (see
+"Recall algorithm" below).
Retinue is developed in this workspace and *deployed* into the active
Hermes home (~/.hermes/plugins/retinue/) by the =Install Retinue= script.
edit =~/.hermes/plugins/retinue/= directly; edit here and re-run the
installer.
+* Recall algorithm
+:PROPERTIES:
+:ID: 93c385e6-3e7d-4ebe-8e80-17cf8a8d47a3
+:END:
+
+Retinue pushes recall on its own and still answers explicit searches.
+Both paths read the same =memories= table and share one returned-set.
+
+1. =prefetch()= runs before every non-trivial turn with the current user
+ message and the session id. Hermes passes nothing else.
+2. The embedding query is built from the current message (max 600 chars)
+ plus the rolling topic recorded by =sync_turn()=: the previous two
+ user messages (200 chars each) and the first 200 chars of the last
+ assistant reply. Short follow-ups ("and the second one?") get their
+ subject from that topic text.
+3. =MemoryStore.search()= returns the top 30 rows (=PREFETCH_LIMIT=).
+ No similarity threshold, no character cap.
+4. Ids already returned in this session are dropped; the survivors are
+ recorded as returned.
+5. Nothing left means no injection. Otherwise the block is appended to
+ the user message: a =Retinue recall (N fresh):= line, then
+ =[id] content= per memory.
+6. The set is per session, in RAM only (=_seen=). =on_session_switch()=
+ and =on_pre_compress()= clear it; a restart starts empty.
+7. =retinue_memory_search= applies the same filter; =include_seen: true=
+ bypasses it.
+
+Full description: =Documentation/Development/index.org=, section
+"Automatic recall (prefetch)".
+
* Layout
:PROPERTIES:
:ID: a8c8c555-d1c1-4f96-b236-be8106e03a9a
#+begin_src json
{
"name": "retinue_memory_search",
- "description": "Search Retinue memories by semantic similarity. Use when you need to recall relevant context even if the user uses different wording than the stored memory.",
+ "description": "Search Retinue memories by semantic similarity. Use when you need to recall relevant context even if the user uses different wording than the stored memory. Memories already recalled earlier in this session are omitted unless include_seen is true.",
"parameters": {
"type": "object",
"properties": {
"query": {"type": "string", "description": "Natural-language search query."},
- "limit": {"type": "integer", "description": "Maximum number of results to return.", "default": 50}
+ "limit": {"type": "integer", "description": "Maximum number of results to return.", "default": 30},
+ "include_seen": {"type": "boolean", "description": "Also return memories already recalled earlier in this session (default false).", "default": false}
},
"required": ["query"]
}
:ID: e30a9286-6083-48ed-a861-61c72a5737b4
:END:
-Retrieval happens through exactly one concrete path: the
-=retinue_memory_search= tool. The =prefetch= hook is a no-op, so nothing
-is injected automatically — every retrieval is an explicit tool call
-visible in the conversation transcript.
+Retrieval has two paths that read the same =memories= table: automatic
+recall through the =prefetch= hook (below) and the explicit
+=retinue_memory_search= tool. The tool path returns what the LLM asks
+for; the automatic path returns only memories this session has not
+seen yet.
-The step-by-step mechanics:
+The step-by-step mechanics of the explicit path:
1. *The LLM is notified the tool exists.* At session start Hermes calls
=get_tool_schemas()= and places all four =retinue_memory_*= schemas
into the LLM's tool list next to the built-in tools. The
- =retinue_memory_search= schema declares two parameters: =query=
- (required string) and =limit= (optional integer, default 50).
+ =retinue_memory_search= schema declares three parameters: =query=
+ (required string), =limit= (optional integer, default 30), and
+ =include_seen= (optional boolean, default false — when true, memories
+ already returned in this session are included again).
Separately, the text of =src/retinue/system_prompt_block.txt= is
injected into the system prompt, instructing the LLM to run
=retinue_memory_search= before acting on a task.
(full stored text), =created= (ISO 8601 UTC timestamp), and
=similarity=. The =similarity= is computed in =provider.py= from the
raw sqlite-vec distance as =1 / (1 + distance)= — higher is better,
- 1.0 means identical vectors. (Raw distance in the database is
- unbounded L2 on the model's non-normalized embeddings, so typical
- real-world matches land in the 0.4–0.6 range; treat the score as a
- relative ranking signal, not an absolute confidence.)
+ 1.0 means identical vectors. The stored embeddings are unit-length
+ (model2vec normalizes them), so the raw distance is plain L2 and
+ cosine similarity is =1 - distance^2 / 2=; =1 / (1 + distance)=
+ compresses that into roughly 0.40–0.50 for everything in a real
+ store. Treat it as a relative ranking signal only.
#+begin_src
{"success": true, "result": [
scores near the noise floor) mean the store has nothing relevant;
the LLM proceeds without memory context.
+** Automatic recall (prefetch)
+:PROPERTIES:
+:CUSTOM_ID: automatic-recall
+:ID: 39a1ec7f-89f2-49a9-b86a-49f6294dcc70
+:END:
+
+1) Before every non-trivial turn Hermes calls the provider with the
+ current user message and the session id:
+
+ : prefetch("Why does it filter already returned ids?", session_id="20261023_101500_ab12cd")
+
+ Nothing else from the conversation is passed in.
+
+2) Retinue builds the embedding query from the current message plus a
+ rolling topic recorded by =sync_turn()= on previous turns: the two
+ previous user messages (200 characters each) and the first 200
+ characters of the last assistant reply. The current message is
+ capped at 600 characters. This is what gives a short follow-up
+ ("and the second one?") a subject to search for.
+
+3) =MemoryStore.search()= runs the same =vec0= KNN query as the tool
+ with a fixed budget of 30 rows (=PREFETCH_LIMIT= in =provider.py=).
+ There is no similarity threshold and no character cap.
+
+4) Ids already returned during this session are dropped from those 30
+ rows. The surviving ids are recorded as returned for the session.
+
+5) If nothing survives, =prefetch()= returns an empty string and Hermes
+ injects nothing on that turn. Otherwise the block is appended to the
+ user message and persisted with it:
+
+ : Retinue recall (2 fresh):
+ : [123] Retinue project: deployed with "Install Retinue" into ~/.hermes/plugins/retinue.
+ : [456] Retinue stores everything in retinue_memory.db (single SQLite file).
+
+6) The returned-set is per session and lives in RAM (=_seen= in
+ =provider.py=). It is cleared by =on_session_switch()= (/new,
+ /resume, /branch, /reset) and by =on_pre_compress()=, because
+ compression may summarize the injected block away and the memory
+ would otherwise stay invisible for the rest of the session.
+
+7) The set is not persisted. After a restart the first turn of the next
+ session injects its best matches again.
+
+8) The explicit =retinue_memory_search= tool applies the same filter;
+ =include_seen: true= returns already-recalled memories anyway.
+
+The net effect: a memory enters the conversation at most once per
+session. With a stable topic the injected block shrinks turn by turn
+and eventually returns nothing; when the subject changes, the new top
+30 contains fresh ids and recall resumes.
+
* Development and testing workflow
:PROPERTIES:
:CUSTOM_ID: development-and-testing-workflow
it runs offline, requires no server, and stores everything in a single
SQLite database.
+Before every turn Retinue also injects the memories relevant to the
+current message on its own — but only those not already returned in the
+same session, so a long conversation does not keep re-reading the same
+facts. The rules are in
+[[file:Development/index.org::#automatic-recall][Automatic recall]].
+
For technical details, read:
- [[file:Development/index.org::#memory-save][How memory save works]].
- [[file:Development/index.org::#memory-retrieval][How memory retrieval works]].
+- [[file:Development/index.org::#automatic-recall][Automatic recall (prefetch)]].
* Installation
:PROPERTIES:
name: retinue
-version: 1.1.1
+version: 1.2.0
description: "Semantic memory provider for Hermes Agent using model2vec and sqlite-vec."
hooks:
- sync_turn
import logging
import sqlite3
import struct
+import threading
from datetime import datetime, timezone
from pathlib import Path
from typing import Any
# ---------------------------------------------------------------------------
_model: Any | None = None
_model_dim: int | None = None
+# The provider warms the model on a background thread at startup while tool
+# calls may embed concurrently; without this lock both would load 500 MB.
+_model_lock = threading.Lock()
def _resolve_model_path() -> str:
if _model is not None:
return _model
- try:
- from model2vec import StaticModel
- except ImportError as e:
- raise RuntimeError(
- "model2vec is not installed. Run: pip install model2vec"
- ) from e
+ with _model_lock:
+ if _model is not None: # another thread finished while we waited
+ return _model
+
+ try:
+ from model2vec import StaticModel
+ except ImportError as e:
+ raise RuntimeError(
+ "model2vec is not installed. Run: pip install model2vec"
+ ) from e
- logger.info("Loading embedding model: %s", MODEL)
- _model = StaticModel.from_pretrained(_resolve_model_path())
- return _model
+ logger.info("Loading embedding model: %s", MODEL)
+ _model = StaticModel.from_pretrained(_resolve_model_path())
+ return _model
def embed_text(text: str) -> bytes:
db.commit()
return row
- def find_by_content(self, text: str) -> list[tuple]:
- """Find memories whose content contains the given substring."""
+ def find_exact(self, text: str) -> list[tuple]:
+ """Find the memories whose content is exactly `text`."""
+ db = self._get_db()
+ rows = db.execute(
+ f"SELECT rowid, content, created FROM {TABLE} WHERE content = ? ORDER BY rowid",
+ (text,),
+ ).fetchall()
+ return rows
+
+ def find_containing(self, text: str) -> list[tuple]:
+ """Find memories whose content contains `text` as a LITERAL substring.
+
+ `instr()` is used instead of LIKE so that `%` and `_` in the search
+ text stay ordinary characters: `find_containing("%")` must match only
+ memories that themselves contain a percent sign, never the whole store.
+ """
db = self._get_db()
rows = db.execute(
- f"SELECT rowid, content, created FROM {TABLE} WHERE content LIKE ? ORDER BY rowid",
- (f"%{text}%",),
+ f"SELECT rowid, content, created FROM {TABLE} "
+ f"WHERE instr(lower(content), lower(?)) > 0 ORDER BY rowid",
+ (text,),
).fetchall()
return rows
import json
import logging
import threading
+from collections import deque
+from dataclasses import dataclass
from pathlib import Path
from typing import Any, Dict, List, Optional, TYPE_CHECKING
logger = logging.getLogger(__name__)
+# Recall budget per prefetch: one search per turn, at most this many rows.
+PREFETCH_LIMIT = 30
+# Rolling topic that disambiguates short follow-up messages ("and the second
+# one?"): the previous user messages plus the tail of the last reply.
+TOPIC_USER_MESSAGES = 2
+TOPIC_USER_CHARS = 200
+TOPIC_ASSISTANT_CHARS = 200
+CURRENT_MESSAGE_CHARS = 600
+
# Try to inherit from Hermes' MemoryProvider ABC; fall back to a local stub
# when developing or testing outside of a Hermes process.
if TYPE_CHECKING:
pass
+try:
+ from agent.memory_provider import RecallStatus # type: ignore[assignment]
+except ImportError: # pragma: no cover - Hermes not available in plain Python env
+
+ @dataclass(frozen=True)
+ class RecallStatus: # type: ignore[no-redef]
+ """Local stand-in for Hermes' recall indicator record."""
+
+ provider_label: str
+ count: int
+ glyph: str = "🧠"
+
+
class RetinueMemoryProvider(MemoryProvider):
"""Semantic memory provider for Hermes Agent."""
self._store: MemoryStore | None = None
self._hermes_home: str | None = None
self._session_id: str = ""
- self._sync_thread: threading.Thread | None = None
+ # Session recall state: memory ids already returned to the LLM, the
+ # rolling topic used to build the next query, and the size of the last
+ # injection (for Hermes' recall indicator). All guarded by one lock:
+ # prefetch runs on its own thread, tool calls and sync_turn on others.
+ self._lock = threading.RLock()
+ self._seen: dict[str, set[int]] = {}
+ self._topic_users: deque[str] = deque(maxlen=TOPIC_USER_MESSAGES)
+ self._topic_assistant: str = ""
+ self._last_recall: int = 0
@property
def name(self) -> str:
def initialize(self, session_id: str, **kwargs) -> None:
"""Called once when Hermes starts a session."""
self._hermes_home = kwargs.get("hermes_home")
- self._session_id = session_id
+ with self._lock:
+ self._session_id = session_id
+ self._seen = {session_id: set()}
+ self._topic_users.clear()
+ self._topic_assistant = ""
+ self._last_recall = 0
if self._hermes_home:
base_dir = Path(self._hermes_home)
self._store = MemoryStore(base_dir=base_dir)
logger.info("Retinue memory provider initialized at %s", self._store.db_path)
+ # The first embed of a process costs ~3.5 s (snapshot lookup + 500 MB
+ # load). prefetch() runs at the start of turn 1 and the turn waits for
+ # it, so load the model in the background instead of on that path.
+ threading.Thread(
+ target=self._warm_model, name="retinue-model-warmup", daemon=True
+ ).start()
+
+ @staticmethod
+ def _warm_model() -> None:
+ try:
+ from retinue.memory import get_model
+
+ get_model()
+ logger.info("Retinue embedding model warmed up")
+ except Exception as e: # pragma: no cover - best effort
+ logger.debug("Retinue model warm-up failed: %s", e)
# ------------------------------------------------------------------
# System prompt guidance
},
{
"name": "retinue_memory_search",
- "description": "Search Retinue memories by semantic similarity. Use when you need to recall relevant context even if the user uses different wording than the stored memory.",
+ "description": "Search Retinue memories by semantic similarity. Use when you need to recall relevant context even if the user uses different wording than the stored memory. Memories already recalled earlier in this session are omitted unless include_seen is true.",
"parameters": {
"type": "object",
"properties": {
"limit": {
"type": "integer",
"description": "Maximum number of results to return.",
- "default": 50,
+ "default": 30,
+ },
+ "include_seen": {
+ "type": "boolean",
+ "description": "Also return memories already recalled earlier in this session (default false).",
+ "default": False,
},
},
"required": ["query"],
def _tool_retinue_memory_search(self, args: Dict[str, Any]) -> List[Dict[str, Any]]:
assert self._store is not None
- rows = self._store.search(args["query"], limit=args.get("limit", 50))
+ rows = self._store.search(args["query"], limit=args.get("limit", PREFETCH_LIMIT))
+ if not args.get("include_seen"):
+ rows = self._fresh(rows, self._session_key())
return [
{
"id": row[0],
pass
# ------------------------------------------------------------------
- # Optional hooks
+ # Optional hooks: recall, session state, memory mirror
# ------------------------------------------------------------------
def sync_turn(
self,
session_id: str = "",
messages: Optional[List[Dict[str, Any]]] = None,
) -> None:
- """Persist a completed turn. Currently a no-op; future versions will
- summarize the turn and store it."""
- def _sync() -> None:
- pass
+ """Record the turn as topic context for the next prefetch query.
+
+ Retrieval quality depends on the query carrying the conversation
+ topic: Hermes passes only the current user message to prefetch(), and
+ an elliptical follow-up ("and the second one?") embeds to nothing on
+ its own. The previous user messages plus the tail of the last reply
+ are kept here and prepended to the next query.
+ """
+ user = " ".join((user_content or "").split())
+ assistant = " ".join((assistant_content or "").split())
+ with self._lock:
+ if user:
+ self._topic_users.append(user)
+ self._topic_assistant = assistant
+
+ def prefetch(self, query: str, *, session_id: str = "") -> str:
+ """Return memories relevant to `query` that this session has not seen."""
+ if self._store is None:
+ return ""
- if self._sync_thread and self._sync_thread.is_alive():
- self._sync_thread.join(timeout=1.0)
- self._sync_thread = threading.Thread(target=_sync, daemon=True)
- self._sync_thread.start()
+ current = " ".join((query or "").split())
+ if not current:
+ return ""
+
+ key = self._session_key(session_id)
+ if session_id:
+ with self._lock:
+ self._session_id = session_id
+ self._seen.setdefault(key, set())
+
+ try:
+ rows = self._store.search(self._build_query(current), limit=PREFETCH_LIMIT)
+ except Exception as e:
+ logger.debug("Retinue prefetch search failed: %s", e)
+ return ""
+
+ fresh = self._fresh(rows, key)
+ with self._lock:
+ self._last_recall = len(fresh)
+ if not fresh:
+ return ""
+ return self._format(fresh)
+
+ def queue_prefetch(self, query: str, *, session_id: str = "") -> None:
+ """No-op: prefetch() costs ~0.5 ms (embed + KNN) and runs inline."""
+ return
+
+ def recall_status(self): # type: ignore[override]
+ """Report how many memories the last prefetch injected."""
+ with self._lock:
+ count = self._last_recall
+ if count <= 0:
+ return None
+ return RecallStatus(provider_label="Retinue", count=count)
+
+ def on_session_switch(
+ self,
+ new_session_id: str,
+ *,
+ parent_session_id: str = "",
+ reset: bool = False,
+ rewound: bool = False,
+ **kwargs,
+ ) -> None:
+ """/new, /resume, /branch, /reset, compression: forget what was returned."""
+ if not new_session_id:
+ return
+ with self._lock:
+ self._session_id = new_session_id
+ self._seen = {new_session_id: set()}
+ self._last_recall = 0
+ if reset or rewound:
+ self._topic_users.clear()
+ self._topic_assistant = ""
+
+ def on_pre_compress(self, messages: list[dict[str, Any]], **kwargs) -> str:
+ """Compression may summarize the injected recall text away; re-arm it.
+
+ Returns "" — Retinue contributes no text to the summary prompt.
+ """
+ with self._lock:
+ self._seen[self._session_key()] = set()
+ self._last_recall = 0
+ return ""
+
+ def _session_key(self, session_id: str = "") -> str:
+ """The session whose returned-set applies (tool calls carry no id)."""
+ return session_id or self._session_id or "default"
+
+ def _build_query(self, current: str) -> str:
+ """Current message plus the rolling topic, for the embedding query."""
+ with self._lock:
+ users = list(self._topic_users)
+ assistant = self._topic_assistant
+ parts = [current[:CURRENT_MESSAGE_CHARS]]
+ parts.extend(u[:TOPIC_USER_CHARS] for u in users)
+ if assistant:
+ parts.append(assistant[:TOPIC_ASSISTANT_CHARS])
+ return "\n".join(p for p in parts if p)
+
+ def _fresh(self, rows: list[tuple], key: str) -> list[tuple]:
+ """Drop rows already returned in this session and mark the rest."""
+ with self._lock:
+ seen = self._seen.setdefault(key, set())
+ fresh = [row for row in rows if row[0] not in seen]
+ seen.update(row[0] for row in fresh)
+ return fresh
+
+ @staticmethod
+ def _format(rows: list[tuple]) -> str:
+ """Compact recall block: one header line, then `[id] content`."""
+ lines = [f"Retinue recall ({len(rows)} fresh):"]
+ for rowid, content, _created, _distance in rows:
+ body = " ".join((content or "").split())
+ lines.append(f"[{rowid}] {body}")
+ return "\n".join(lines)
def shutdown(self) -> None:
if self._store is not None:
self._store.close()
self._store = None
- if self._sync_thread and self._sync_thread.is_alive():
- self._sync_thread.join(timeout=5.0)
+
+ def _mirror_delete(self, text: str) -> None:
+ """Delete the ONE Retinue memory that corresponds to `text`.
+
+ The built-in memory tool removes an entry by identity, never by
+ substring. Mirroring that as "delete every memory containing these
+ words" destroyed unrelated memories (a removal of a note mentioning
+ "Retinue project" also deleted every other note with those words).
+ So: delete exact matches; fall back to a substring match only when it
+ is unambiguous; otherwise delete nothing and log.
+ """
+ assert self._store is not None
+ if not text:
+ return
+
+ rows = self._store.find_exact(text)
+ if not rows:
+ candidates = self._store.find_containing(text)
+ if len(candidates) == 1:
+ rows = candidates
+ elif candidates:
+ logger.warning(
+ "Retinue mirror: removed text matches %d memories; "
+ "deleting none of them (ambiguous): %r",
+ len(candidates), text[:120],
+ )
+ return
+ else:
+ return
+
+ for row in rows:
+ self._store.delete(row[0])
def on_memory_write(
self,
memory tool: even if the LLM chooses `memory(action='add')`, the fact
can still be recalled later via `retinue_memory_search`.
"""
- if self._store is None or not content:
+ if self._store is None:
return
+ meta = metadata or {}
+ # `previous_content` is the entry the native store actually removed
+ # (Hermes pins it from the committed result). `old_text` is only the
+ # search argument and is NOT authoritative identity.
+ removed = str(meta.get("previous_content") or "")
+ selected = str(meta.get("old_text") or "")
+
try:
if action == "add":
- self._store.add(content=content)
+ if content:
+ self._store.add(content=content)
elif action == "remove":
- for row in self._store.find_by_content(content):
- self._store.delete(row[0])
+ self._mirror_delete(removed or content)
elif action == "replace":
- old_text = (metadata or {}).get("old_text", "")
- if old_text:
- for row in self._store.find_by_content(old_text):
- self._store.delete(row[0])
- self._store.add(content=content)
+ self._mirror_delete(removed or selected)
+ if content:
+ self._store.add(content=content)
except Exception as e:
logger.debug("Retinue on_memory_write failed: %s", e)
sys.path.insert(0, str(PROJECT_ROOT / "src"))
from retinue.memory import MemoryStore, embed_text, get_dim, get_model
-from retinue.provider import RetinueMemoryProvider
+from retinue.provider import (
+ CURRENT_MESSAGE_CHARS,
+ RetinueMemoryProvider,
+ TOPIC_ASSISTANT_CHARS,
+ TOPIC_USER_CHARS,
+)
from retinue.cli import main as cli_main
def test_on_memory_write() -> None:
- """Exercise the built-in memory mirror hook."""
+ """Exercise the built-in memory mirror hook (and its data-loss guards)."""
base_dir = Path(tempfile.mkdtemp(prefix="retinue-mirror-test-"))
try:
provider = RetinueMemoryProvider()
provider.initialize("mirror-session", hermes_home=str(base_dir))
+ assert provider._store is not None
+ store = provider._store
- # Simulate a built-in memory write being mirrored.
provider.on_memory_write(
action="add",
target="user",
assert len(data["result"]) == 1, data
print("Mirror add result:", data)
- # Replace the mirrored memory.
+ # Two memories that share a long phrase. Removing one of them must NOT
+ # take the other with it (the 2026-10-23 substring-delete bug).
+ keeper = "Project Retinue is developed in the workspace tree"
+ target = "Project Retinue stores embeddings with sqlite-vec"
+ store.add(keeper)
+ store.add(target)
+
provider.on_memory_write(
- action="replace",
- target="user",
- content="Default browser is Firefox",
- metadata={"tool_name": "memory", "old_text": "Prefers concise responses"},
+ action="remove",
+ target="memory",
+ content=target,
+ # Hermes pins the removed entry from the committed store result.
+ metadata={"previous_content": target, "old_text": "sqlite-vec"},
)
- # Verify the old content is gone by exact substring.
- assert provider._store is not None
- old_rows = provider._store.find_by_content("Prefers concise responses")
- assert len(old_rows) == 0, old_rows
- new_rows = provider._store.find_by_content("Default browser is Firefox")
- assert len(new_rows) == 1, new_rows
- print("Mirror replace cleared old content and added new content.")
+ assert store.find_exact(target) == [], "target memory still present"
+ assert len(store.find_exact(keeper)) == 1, "unrelated memory was deleted"
+ print("Mirror remove deleted exactly the removed entry; neighbour kept.")
- # Remove.
+ # A stale/partial text that matches several memories must delete
+ # nothing rather than guess.
+ before = len(store.list())
provider.on_memory_write(
action="remove",
- target="user",
- content="Default browser is Firefox",
- metadata={"tool_name": "memory"},
+ target="memory",
+ content="",
+ metadata={"old_text": "Project Retinue"},
+ )
+ assert len(store.list()) == before, "ambiguous removal deleted memories"
+ print("Mirror remove with an ambiguous old_text deleted nothing.")
+
+ # Replace identifies the old entry by previous_content alone (the old
+ # code ignored that field and kept a stale copy behind).
+ provider.on_memory_write(
+ action="replace",
+ target="memory",
+ content="Project Retinue stores embeddings in retinue_memory.db",
+ metadata={"previous_content": keeper},
+ )
+ assert store.find_exact(keeper) == [], "replace kept the old entry"
+ assert len(store.find_containing("retinue_memory.db")) == 1
+ print("Mirror replace swapped the entry via previous_content.")
+
+ # LIKE wildcards must be literal: "%" in the removed text used to match
+ # (and delete) the whole store.
+ store.add("Literal wildcard note: 50% of _tests_ pass")
+ assert len(store.find_containing("%")) == 1
+ underscored = store.find_containing("_")
+ assert underscored and all("_" in row[1] for row in underscored)
+ assert len(underscored) < len(store.list())
+ before = len(store.list())
+ provider.on_memory_write(
+ action="remove", target="memory", content="",
+ metadata={"previous_content": "%"},
)
- remaining = provider._store.find_by_content("Default browser is Firefox")
- assert len(remaining) == 0, remaining
- print("Mirror remove removed the memory.")
+ removed = before - len(store.list())
+ assert removed <= 1, f"wildcard removal deleted {removed} memories"
+ print("Wildcard characters are literal; '%' removal stayed local.")
provider.shutdown()
print("\nMirror tests passed.")
shutil.rmtree(base_dir, ignore_errors=True)
+def _prefetch_ids(block: str) -> list[int]:
+ """Memory ids in a formatted prefetch block."""
+ ids = []
+ for line in block.splitlines():
+ if line.startswith("[") and "]" in line:
+ ids.append(int(line[1:line.index("]")]))
+ return ids
+
+
+def test_prefetch_session_filter() -> None:
+ """Prefetch injects each memory once per session."""
+ base_dir = Path(tempfile.mkdtemp(prefix="retinue-prefetch-test-"))
+ try:
+ provider = RetinueMemoryProvider()
+ provider.initialize("session-a", hermes_home=str(base_dir))
+ assert provider._store is not None
+ store = provider._store
+
+ seeds = [
+ "Retinue stores memories in sqlite-vec",
+ "Retinue prefetch injects recall automatically before each turn",
+ "Retinue session filter drops ids already returned",
+ "Retinue documentation describes the recall algorithm",
+ "Retinue hooks live in provider.py",
+ ]
+ seed_ids = [store.add(text) for text in seeds]
+
+ first = provider.prefetch("How does Retinue prefetch work?", session_id="session-a")
+ assert first.startswith("Retinue recall ("), first[:80]
+ assert sorted(_prefetch_ids(first)) == sorted(seed_ids), first
+ assert provider.recall_status() is not None
+ status = provider.recall_status()
+ assert status is not None and status.count == len(seed_ids), status
+
+ # Same query again: everything is already in the session.
+ assert provider.prefetch("How does Retinue prefetch work?", session_id="session-a") == ""
+
+ # A memory added later still gets through.
+ late_id = store.add("Retinue prefetch budget is thirty memories")
+ second = provider.prefetch("How does Retinue prefetch work?", session_id="session-a")
+ assert _prefetch_ids(second) == [late_id], second
+
+ # Explicit search filters seen ids; include_seen restores them.
+ plain = json.loads(provider.handle_tool_call(
+ "retinue_memory_search", {"query": "Retinue", "limit": 10}))
+ assert plain["success"] is True and plain["result"] == [], plain
+ everything = json.loads(provider.handle_tool_call(
+ "retinue_memory_search",
+ {"query": "Retinue", "limit": 10, "include_seen": True}))
+ assert len(everything["result"]) == len(seed_ids) + 1, everything
+
+ # A new session starts with an empty returned-set.
+ provider.on_session_switch("session-b", reset=True)
+ assert provider.prefetch("How does Retinue prefetch work?", session_id="session-b") != ""
+ assert provider.prefetch("How does Retinue prefetch work?", session_id="session-b") == ""
+
+ # Compression re-arms recall: the injected text may be summarized away.
+ provider.on_pre_compress([])
+ assert provider.prefetch("How does Retinue prefetch work?", session_id="session-b") != ""
+
+ provider.shutdown()
+ print("\nPrefetch session-filter tests passed.")
+ finally:
+ shutil.rmtree(base_dir, ignore_errors=True)
+
+
+def test_rolling_topic_query() -> None:
+ """The prefetch query carries the previous turns, not just the last message."""
+ base_dir = Path(tempfile.mkdtemp(prefix="retinue-topic-"))
+ provider = RetinueMemoryProvider()
+ provider.initialize("topic", hermes_home=str(base_dir))
+ try:
+ provider.sync_turn(
+ "How do I add a new location to the Garmin Edge GPS?",
+ "Open the GPX folder over USB and drop the file in.",
+ )
+ provider.sync_turn(
+ "And what about the aukio-3d rasterizer performance?",
+ "Tile-parallel raster is faster; the bugreport-20261006-000922 fix helped.",
+ )
+ query = provider._build_query("and the second one?")
+ lowered = query.lower()
+ assert "garmin edge gps" in lowered, query
+ assert "aukio-3d rasterizer" in lowered, query
+ assert "tile-parallel raster" in lowered, query
+ assert "and the second one?" in query, query
+ # Bounded: current message + 2 user excerpts + 1 assistant excerpt.
+ assert len(query) <= (CURRENT_MESSAGE_CHARS + 2 * TOPIC_USER_CHARS
+ + TOPIC_ASSISTANT_CHARS + 3), len(query)
+
+ # Long messages are truncated to the caps.
+ provider.sync_turn("x" * 5000, "y" * 5000)
+ provider.sync_turn("z" * 5000, "w" * 5000)
+ long_query = provider._build_query("q" * 5000)
+ assert len(long_query) <= (CURRENT_MESSAGE_CHARS + 2 * TOPIC_USER_CHARS
+ + TOPIC_ASSISTANT_CHARS + 3), len(long_query)
+ print("\nRolling-topic query tests passed.")
+ finally:
+ provider.shutdown()
+ shutil.rmtree(base_dir, ignore_errors=True)
+
+
def test_cli() -> None:
"""Exercise the "Show Retinue memory contents" CLI (src/retinue/cli.py)."""
base_dir = Path(tempfile.mkdtemp(prefix="retinue-cli-test-"))
test_memory_store()
test_provider()
test_on_memory_write()
+ test_prefetch_session_filter()
+ test_rolling_topic_query()
test_cli()
print("\nAll standalone tests passed.")
projects / retinue.git / commitdiff