From 365e80a57d07930479faec4c41bc8118f3b0a9af Mon Sep 17 00:00:00 2001 From: Svjatoslav Agejenko Date: Sat, 11 Jul 2026 09:07:30 +0000 Subject: [PATCH] docs: split development documentation into dedicated page Move the technical architecture, install.sh, project structure, database layout, Hermes Agent integration, and development workflow sections from the main doc/index.org into a new doc/development/index.org file. The main index now keeps a shorter "Source code" section with the license, author, and source-code links, and links to the new dedicated Development page. This mirrors the branching documentation structure used in the sixth-3d project. --- doc/development/index.org | 453 ++++++++++++++++++++++++++++++++++++++ doc/index.org | 449 +------------------------------------ 2 files changed, 456 insertions(+), 446 deletions(-) create mode 100644 doc/development/index.org diff --git a/doc/development/index.org b/doc/development/index.org new file mode 100644 index 0000000..0c2e578 --- /dev/null +++ b/doc/development/index.org @@ -0,0 +1,453 @@ +#+SETUPFILE: ~/.emacs.d/org-styles/html/darksun.theme +#+TITLE: Development - Retinue +#+LANGUAGE: en +#+LATEX_HEADER: \usepackage[margin=1.0in]{geometry} +#+LATEX_HEADER: \usepackage{parskip} +#+LATEX_HEADER: \usepackage[none]{hyphenat} + +#+OPTIONS: H:20 num:20 +#+OPTIONS: author:nil + +[[file:../index.org][Back to main documentation]] + +* Architecture +:PROPERTIES: +:CUSTOM_ID: architecture +:ID: ad252f6e-8152-4dd4-95f6-7b52f773e740 +:END: + +Retinue uses: + +- =model2vec= :: Static embedding model (=minishlab/potion-multilingual-128M=, + ~500 MB download on first use, then offline). Understands meaning across + languages. +- =sqlite-vec= :: SQLite extension for vector similarity search. No + server, no daemon, no port. +- Unified store :: All memories share one table. Each memory has a + content string, a timestamp, and an auto-generated embedding. + +* What install.sh does +:PROPERTIES: +:CUSTOM_ID: what-install-sh-does +:ID: 9f6b3951-e8f0-4643-bd54-b35bdf528d6e +:END: + +=install.sh= is the deployment bridge between the Retinue workspace and the +active Hermes home. It is intended for iterative development: edit the +source tree, run the script, and test inside Hermes immediately. + +The script is idempotent, so you can run it after every code change. It +preserves runtime artifacts (the database, virtual environments, and caches) +while replacing the plugin source code. + +What it does, in order: + +1. Detects the active Hermes home from the =HERMES_HOME= environment variable, + or falls back to =$HOME/.hermes=. +2. Creates =$HERMES_HOME/plugins/retinue/= if it does not exist. +3. Resolves the Python interpreter that Hermes actually runs under (via the + =PYTHON= environment variable, common venv layouts, or the =hermes= + launcher shebang) and checks for =sqlite-vec= and =model2vec=. +4. Installs any missing dependencies in that interpreter with =pip=. +5. Copies the latest source tree into the plugin directory using =rsync=, + excluding =data/=, =.venv/=, =.hf-cache/=, =__pycache__/=, and =.git/= so + the runtime store and virtual environment are preserved. +6. Sets =memory.provider: retinue= with =hermes config set= if the CLI is + available. +7. Runs a lightweight import check to confirm the plugin loads. + +To target a specific Hermes profile or home directory: + +#+begin_src sh +HERMES_HOME="$HOME/.hermes/profiles/coder" bash install.sh +#+end_src + +* Project structure +:PROPERTIES: +:CUSTOM_ID: project-structure +:ID: 6d9ea612-7ecf-4eef-96e0-ab2e9b744a2e +:END: + +#+begin_src +Retinue/ +├── doc/ +│ └── index.org # This file +├── install.sh # Deploy into the active Hermes plugin directory +├── plugin.yaml # Hermes plugin metadata +├── __init__.py # Plugin entry point: register(ctx) +├── src/ # Plugin source package +│ └── retinue/ +│ ├── __init__.py # Package init +│ ├── memory.py # Memory engine: embeddings + sqlite-vec +│ └── provider.py # MemoryProvider subclass +├── cli.py # Optional: hermes retinue CLI +└── data/ # Created at runtime under the active Hermes home + └── memory.db # SQLite + sqlite-vec database +#+end_src + +* Database layout +:PROPERTIES: +:CUSTOM_ID: database-layout +:ID: 2e9bc319-c314-4a48-8fdf-0c6bac11f344 +:END: + +Retinue stores all data in a single SQLite virtual table under the active +Hermes home: + +#+begin_src text +$HERMES_HOME/retinue_memory.db +#+end_src + +*"memories" virtual table:* + +| Column | Type | Meaning | +|-------------+------------------+----------------------------------------------------------| +| =rowid= | INTEGER | Auto-increment primary key. Stable ID used for deletion. | +| =embedding= | float[MODEL_DIM] | The model2vec embedding vector for =content=. | +| =content= | TEXT | The full text of the memory/fact. | +| =created= | TEXT | ISO 8601 UTC timestamp when the row was inserted. | + +=MODEL_DIM= depends on the model (hardcoded to +=minishlab/potion-multilingual-128M=). The table is created with +=sqlite-vec= and supports cosine-similarity search. + +Why a /virtual/ table? In SQLite, a virtual table is created with +=CREATE VIRTUAL TABLE ... USING vec0(...)=. The =vec0= module from +=sqlite-vec= provides the storage and indexing logic for the +=embedding= column, which enables the =embedding MATCH ?= vector +similarity query. The table is still persisted in the same SQLite +=.db= file; “virtual” only means SQLite delegates the table engine to +the =sqlite-vec= extension. + +* Integration with Hermes Agent +:PROPERTIES: +:CUSTOM_ID: integration-with-hermes-agent +:ID: ca6a072f-b686-4dec-b50a-e4b83980be61 +:END: + +Retinue implements the Hermes =MemoryProvider= abstract base class from +=agent/memory_provider.py=. + +** Required methods +:PROPERTIES: +:CUSTOM_ID: required-methods +:ID: 845b8e5d-7e94-4b93-b147-d065f864eeb3 +:END: + +| Method | Purpose | Notes | +|------------------------------------------+---------------------------------------+-------------------------------------------------------------------------------------------| +| =name= (property) | Provider identifier, e.g. ="retinue"= | Must be unique | +| =is_available()= | Can this provider activate? | No network calls; check deps/config/env | +| =initialize(session_id, **kwargs)= | One-time startup | Receives =hermes_home=, =platform=, =agent_context=, etc. | +| =get_tool_schemas()= | Tool schemas to expose to the model | OpenAI function-calling format; return =[]= if context-only | +| =handle_tool_call(name, args, **kwargs)= | Execute a tool call | Must return a JSON string | +| =get_config_schema()= | Fields for =hermes memory setup= | List of field descriptors (key, description, secret, required, default, choices, env_var) | +| =save_config(values, hermes_home)= | Persist non-secret config | Secrets go to =.env=; env-var-only providers can no-op | + +** Optional hooks +:PROPERTIES: +:CUSTOM_ID: optional-hooks +:ID: f0792483-e9ea-4ad0-be64-7a4e9544a489 +:END: + +| Method | Purpose | +|---------------------------------------------------------------+---------------------------------------------------------------| +| =system_prompt_block()= | Static text injected into the system prompt | +| =prefetch(query, *, session_id="")= | Recall relevant context before each turn (should be fast) | +| =queue_prefetch(query)= | Pre-warm recall for the next turn | +| =sync_turn(user, assistant, *, session_id="", messages=None)= | Persist a completed turn (must be non-blocking) | +| =on_session_end(messages)= | Flush/extract at conversation end | +| =on_pre_compress(messages) -> str= | Capture insights before context compression | +| =on_memory_write(action, target, content, metadata=None)= | Mirror built-in =memory= writes to the provider | +| =on_delegation(task, result, **kwargs)= | Observe subagent work from the parent | +| =shutdown()= | Clean up connections/threads | +| =backup_paths() -> list[str]= | Extra on-disk paths outside =HERMES_HOME= for =hermes backup= | + +** Memory save +:PROPERTIES: +:CUSTOM_ID: memory-save +:ID: 6325a3ac-8629-461e-aa5e-62408ba73bde +:END: + +Hermes does not automatically persist every turn. A memory provider +can receive data through three separate paths, and the provider +decides what to store and how. + +*** Explicit memory tools (provider tools) +:PROPERTIES: +:CUSTOM_ID: explicit-memory-tools-provider-tools +:ID: a0b54263-fd4c-4997-a867-834ccce8a8b6 +:END: + +The tools returned by =get_tool_schemas()= are offered to the LLM +alongside Hermes' built-in tools. When the LLM decides a fact is worth +remembering, it calls one of them. The provider receives the call in +=handle_tool_call(name, args, **kwargs)= and must return a JSON +string. + +Retinue exposes these tools: + +- =retinue_memory_add= — store a fact. +- =retinue_memory_search= — search stored facts by semantic similarity. +- =retinue_memory_list= — list facts. +- =retinue_memory_delete= — delete a fact by ID. +- =retinue_memory_stats= — show store statistics. + +This is the primary path in Retinue. The LLM decides when to save, and +the user can explicitly ask "remember that ...". + +*** Built-in =memory= tool mirror (=on_memory_write=) +:PROPERTIES: +:CUSTOM_ID: built-in-memory-tool-mirror-on-memory-write +:ID: 25f1b5ee-0b77-412f-98f1-74d69b884f19 +:END: + +Hermes has a built-in =memory= tool that the LLM can call to write durable +facts. When that tool successfully commits a write, Hermes notifies the active +external memory provider through =on_memory_write(action, target, content, +metadata=None)=. + +Fields: + +- =action= — ="add"=, ="replace"=, or ="remove"=. +- =target= — ="memory"= or ="user"=. +- =content= — the text being written. +- =metadata= — provenance dictionary, commonly containing: + - =write_origin= — e.g. ="assistant_tool"= + - =execution_context= — e.g. ="foreground"= or ="background"= + - =session_id= / =parent_session_id= + - =platform= — e.g. ="cli"=, ="telegram"= + - =tool_name= — always ="memory"= for this hook + - =task_id= / =tool_call_id= — when the write happened inside a task + - =old_text= — present for =replace= actions + +Retinue implements this hook so that successful =memory= writes are mirrored +into the Retinue semantic store. They can then be searched via +=retinue_memory_search=. + +*** Turn-level extraction (=sync_turn=) +:PROPERTIES: +:CUSTOM_ID: turn-level-extraction-sync-turn +:ID: 52f3d0c2-5e75-4a49-90b1-50fca051ae1e +:END: + +After every completed turn, Hermes calls =sync_turn(user_content, +assistant_content, *, session_id="", messages=None)= on a background +thread. + +Fields: + +- =user_content= — the cleaned user message for this turn (skill + scaffolding stripped). +- =assistant_content= — the assistant's final response for this turn. +- =session_id= — the active Hermes session ID. +- =messages= — the full OpenAI-style conversation message list /as of + the completed turn/, including any assistant tool calls and tool + results. This is *not* just the latest pair; it is the entire + transcript so far. + +The provider can override this method to extract and store facts from +the raw conversation. It should be non-blocking — queue work to a +background thread if needed, because Hermes calls it synchronously and +a slow provider will stall the agent. + +Retinue's current implementation leaves =sync_turn= as a no-op. Future +versions may use it to summarize each turn and store extracted facts +automatically. + +*** Summary of who decides what to save +:PROPERTIES: +:CUSTOM_ID: summary-of-who-decides-what-to-save +:ID: f9bf3bb2-f15e-4f37-82d1-52254e12bfd0 +:END: + +| Path | Trigger | What the provider receives | +|----------------------+---------------------------------------------+------------------------------------------------------| +| Provider tools | LLM calls one of =get_tool_schemas()= tools | Tool name + structured arguments | +| Built-in memory tool | LLM calls =memory= | =on_memory_write(action, target, content, metadata)= | +| Turn sync | Hermes after every completed turn | =sync_turn(user, assistant, session_id, messages)= | + +** Memory retrieval +:PROPERTIES: +:CUSTOM_ID: memory-retrieval +:ID: e30a9286-6083-48ed-a861-61c72a5737b4 +:END: + +Retrieval is also provider-driven. Hermes calls =prefetch(query, +session_id="")= before each turn; the provider can return relevant +context as a formatted string. This text is injected into the system +prompt block and fenced as ==. + +In addition, the LLM can explicitly call provider tools (e.g. +=retinue_memory_search=) or the built-in =memory= tool to look up facts on +demand. + +Retinue currently uses the provider-tool path for retrieval; =prefetch= is a +no-op. + +** Relationship to built-in =memory= +:PROPERTIES: +:CUSTOM_ID: relationship-to-built-in-memory +:ID: 4aba035e-ab65-48c6-925c-49b68a3e65be +:END: + +Hermes always exposes the built-in =memory= tool. It stores compact, +high-signal notes in two text files (=MEMORY.md= and =USER.md=) that are +injected into the system prompt as a frozen snapshot. It is best for short, +durable facts: preferences, corrections, environment details, and reusable +lessons. + +When Retinue is active, the LLM sees two memory surfaces: + +- Built-in =memory= — compact, always-on, text-file storage. +- =retinue_memory_*= — local semantic memory with embeddings and similarity + search. + +The LLM chooses based on the tool descriptions. There is no hard-coded rule. +In practice: + +- Use =memory= for short, high-signal facts that should appear in every system + prompt (e.g. "User prefers concise responses", "Project uses pytest"). +- Use =retinue_memory_add= for richer knowledge that may be recalled later by + meaning (e.g. "ProjectX is hosted on our primary server"). + =retinue_memory_search= can find it even when the query uses different words. + +To avoid losing facts when the LLM picks the built-in =memory= tool, Retinue +implements =on_memory_write= so successful =memory= writes (add, replace, remove) +are mirrored into the Retinue semantic store. They can then be searched via +=retinue_memory_search=. + +** Tools exposed to Hermes +:PROPERTIES: +:CUSTOM_ID: tools-exposed-to-hermes +:ID: 04cf7ccf-e71a-4f91-a023-3dbf87d5dbf5 +:END: + +When Retinue is the active memory provider, Hermes can call these tools +natively. The exact names and schemas are defined in +=get_tool_schemas()= and dispatched through =handle_tool_call()=. + +*** =retinue_memory_add= +:PROPERTIES: +:CUSTOM_ID: retinue-memory-add +:ID: 69dbc374-8c72-4c8c-a46e-7617bc0c2b2d +:END: + +Store a memory with content. + +#+begin_src json +{ + "name": "retinue_memory_add", + "description": "Store a semantic memory in Retinue.", + "parameters": { + "type": "object", + "properties": { + "content": {"type": "string", "description": "The memory content"} + }, + "required": ["content"] + } +} +#+end_src + +*** =retinue_memory_search= +:PROPERTIES: +:CUSTOM_ID: retinue-memory-search +:ID: 8c479c9b-141e-4a12-9ee0-8f15e398c0c1 +:END: + +Semantic search across all memories. + +#+begin_src json +{ + "name": "retinue_memory_search", + "description": "Search Retinue memories by semantic similarity.", + "parameters": { + "type": "object", + "properties": { + "query": {"type": "string", "description": "Natural-language search query"}, + "limit": {"type": "integer", "default": 10} + }, + "required": ["query"] + } +} +#+end_src + +*** =retinue_memory_list= +:PROPERTIES: +:CUSTOM_ID: retinue-memory-list +:ID: 42c019eb-69d9-438f-b7c9-cdc1685a5526 +:END: + +List all stored memories. + +*** =retinue_memory_delete= +:PROPERTIES: +:CUSTOM_ID: retinue-memory-delete +:ID: 4d3fb3e3-6dc3-4184-9c23-dfd0714130db +:END: + +Delete a memory by ID. + +*** =retinue_memory_stats= +:PROPERTIES: +:CUSTOM_ID: retinue-memory-stats +:ID: 7d56b6c5-c889-43a3-980e-6b8ed8277ec1 +:END: + +Show memory store statistics. +* Development and testing workflow +:PROPERTIES: +:CUSTOM_ID: development-and-testing-workflow +:ID: 246ce71b-6841-49fb-a330-2bb83fc275d3 +:END: + +Retinue is developed in its own source tree and deployed into the active +Hermes home with =install.sh=. This makes iteration fast and safe: the +runtime =data/= directory lives in Hermes, while the source code lives in +the workspace. + +** Iterative development loop +:PROPERTIES: +:CUSTOM_ID: iterative-development-loop +:ID: b5b488be-6961-4b2a-ae81-f11d37b73c0b +:END: + +1. Edit source files in the Retinue workspace. +2. Run the installer to copy the latest code into Hermes: + + #+begin_src sh + cd /path/to/retinue + bash install.sh + #+end_src + +3. Verify the plugin loads: + + #+begin_src sh + hermes memory status + #+end_src + +4. Test through a Hermes session: + + #+begin_src sh + hermes chat -q "Remember that my default shell is fish." + hermes chat -q "What shell do I use?" + #+end_src + +5. Repeat from step 1. + +** Standalone testing +:PROPERTIES: +:CUSTOM_ID: standalone-testing +:ID: 24aa4d55-e730-451c-aa27-9b9f4e5959c7 +:END: + +You can also test Retinue without installing it into Hermes. The project +includes a standalone test runner: + +#+begin_src sh +cd /path/to/retinue +./run-tests.sh +#+end_src + +This uses a project-local =.venv= and caches downloaded models in +=.hf-cache=, both of which are excluded from deployment by =install.sh=. diff --git a/doc/index.org b/doc/index.org index 7177061..b3cc6aa 100644 --- a/doc/index.org +++ b/doc/index.org @@ -124,15 +124,11 @@ Hermes session: Hermes will call =retinue_memory_search= and retrieve the relevant memory. -* Development +* Source code :PROPERTIES: :CUSTOM_ID: development -:ID: d77ed683-4da0-4cf1-84ee-73544b93f28b :END: -This section explains in more detail how this plugin works, in case -you are a software developer and want to troubleshoot or improve it. - *This program is free software: released under Creative Commons Zero (CC0) license* @@ -149,444 +145,5 @@ you are a software developer and want to troubleshoot or improve it. : git clone https://www3.svjatoslav.eu/git/retinue.git -** Architecture -:PROPERTIES: -:CUSTOM_ID: architecture -:ID: ad252f6e-8152-4dd4-95f6-7b52f773e740 -:END: - -Retinue uses: - -- =model2vec= :: Static embedding model (=minishlab/potion-multilingual-128M=, - ~500 MB download on first use, then offline). Understands meaning across - languages. -- =sqlite-vec= :: SQLite extension for vector similarity search. No - server, no daemon, no port. -- Unified store :: All memories share one table. Each memory has a - content string, a timestamp, and an auto-generated embedding. - -** What install.sh does -:PROPERTIES: -:CUSTOM_ID: what-install-sh-does -:ID: 9f6b3951-e8f0-4643-bd54-b35bdf528d6e -:END: - -=install.sh= is the deployment bridge between the Retinue workspace and the -active Hermes home. It is intended for iterative development: edit the -source tree, run the script, and test inside Hermes immediately. - -The script is idempotent, so you can run it after every code change. It -preserves runtime artifacts (the database, virtual environments, and caches) -while replacing the plugin source code. - -What it does, in order: - -1. Detects the active Hermes home from the =HERMES_HOME= environment variable, - or falls back to =$HOME/.hermes=. -2. Creates =$HERMES_HOME/plugins/retinue/= if it does not exist. -3. Resolves the Python interpreter that Hermes actually runs under (via the - =PYTHON= environment variable, common venv layouts, or the =hermes= - launcher shebang) and checks for =sqlite-vec= and =model2vec=. -4. Installs any missing dependencies in that interpreter with =pip=. -5. Copies the latest source tree into the plugin directory using =rsync=, - excluding =data/=, =.venv/=, =.hf-cache/=, =__pycache__/=, and =.git/= so - the runtime store and virtual environment are preserved. -6. Sets =memory.provider: retinue= with =hermes config set= if the CLI is - available. -7. Runs a lightweight import check to confirm the plugin loads. - -To target a specific Hermes profile or home directory: - -#+begin_src sh -HERMES_HOME="$HOME/.hermes/profiles/coder" bash install.sh -#+end_src - -** Project structure -:PROPERTIES: -:CUSTOM_ID: project-structure -:ID: 6d9ea612-7ecf-4eef-96e0-ab2e9b744a2e -:END: - -#+begin_src -Retinue/ -├── doc/ -│ └── index.org # This file -├── install.sh # Deploy into the active Hermes plugin directory -├── plugin.yaml # Hermes plugin metadata -├── __init__.py # Plugin entry point: register(ctx) -├── src/ # Plugin source package -│ └── retinue/ -│ ├── __init__.py # Package init -│ ├── memory.py # Memory engine: embeddings + sqlite-vec -│ └── provider.py # MemoryProvider subclass -├── cli.py # Optional: hermes retinue CLI -└── data/ # Created at runtime under the active Hermes home - └── memory.db # SQLite + sqlite-vec database -#+end_src - -** Database layout -:PROPERTIES: -:CUSTOM_ID: database-layout -:ID: 2e9bc319-c314-4a48-8fdf-0c6bac11f344 -:END: - -Retinue stores all data in a single SQLite virtual table under the active -Hermes home: - -#+begin_src text -$HERMES_HOME/retinue_memory.db -#+end_src - -*"memories" virtual table:* - -| Column | Type | Meaning | -|-------------+------------------+----------------------------------------------------------| -| =rowid= | INTEGER | Auto-increment primary key. Stable ID used for deletion. | -| =embedding= | float[MODEL_DIM] | The model2vec embedding vector for =content=. | -| =content= | TEXT | The full text of the memory/fact. | -| =created= | TEXT | ISO 8601 UTC timestamp when the row was inserted. | - -=MODEL_DIM= depends on the model (hardcoded to -=minishlab/potion-multilingual-128M=). The table is created with -=sqlite-vec= and supports cosine-similarity search. - -Why a /virtual/ table? In SQLite, a virtual table is created with -=CREATE VIRTUAL TABLE ... USING vec0(...)=. The =vec0= module from -=sqlite-vec= provides the storage and indexing logic for the -=embedding= column, which enables the =embedding MATCH ?= vector -similarity query. The table is still persisted in the same SQLite -=.db= file; “virtual” only means SQLite delegates the table engine to -the =sqlite-vec= extension. - -** Integration with Hermes Agent -:PROPERTIES: -:CUSTOM_ID: integration-with-hermes-agent -:ID: ca6a072f-b686-4dec-b50a-e4b83980be61 -:END: - -Retinue implements the Hermes =MemoryProvider= abstract base class from -=agent/memory_provider.py=. - -*** Required methods -:PROPERTIES: -:CUSTOM_ID: required-methods -:ID: 845b8e5d-7e94-4b93-b147-d065f864eeb3 -:END: - -| Method | Purpose | Notes | -|------------------------------------------+---------------------------------------+-------------------------------------------------------------------------------------------| -| =name= (property) | Provider identifier, e.g. ="retinue"= | Must be unique | -| =is_available()= | Can this provider activate? | No network calls; check deps/config/env | -| =initialize(session_id, **kwargs)= | One-time startup | Receives =hermes_home=, =platform=, =agent_context=, etc. | -| =get_tool_schemas()= | Tool schemas to expose to the model | OpenAI function-calling format; return =[]= if context-only | -| =handle_tool_call(name, args, **kwargs)= | Execute a tool call | Must return a JSON string | -| =get_config_schema()= | Fields for =hermes memory setup= | List of field descriptors (key, description, secret, required, default, choices, env_var) | -| =save_config(values, hermes_home)= | Persist non-secret config | Secrets go to =.env=; env-var-only providers can no-op | - -*** Optional hooks -:PROPERTIES: -:CUSTOM_ID: optional-hooks -:ID: f0792483-e9ea-4ad0-be64-7a4e9544a489 -:END: - -| Method | Purpose | -|---------------------------------------------------------------+---------------------------------------------------------------| -| =system_prompt_block()= | Static text injected into the system prompt | -| =prefetch(query, *, session_id="")= | Recall relevant context before each turn (should be fast) | -| =queue_prefetch(query)= | Pre-warm recall for the next turn | -| =sync_turn(user, assistant, *, session_id="", messages=None)= | Persist a completed turn (must be non-blocking) | -| =on_session_end(messages)= | Flush/extract at conversation end | -| =on_pre_compress(messages) -> str= | Capture insights before context compression | -| =on_memory_write(action, target, content, metadata=None)= | Mirror built-in =memory= writes to the provider | -| =on_delegation(task, result, **kwargs)= | Observe subagent work from the parent | -| =shutdown()= | Clean up connections/threads | -| =backup_paths() -> list[str]= | Extra on-disk paths outside =HERMES_HOME= for =hermes backup= | - -*** Memory save -:PROPERTIES: -:CUSTOM_ID: memory-save -:ID: 6325a3ac-8629-461e-aa5e-62408ba73bde -:END: - -Hermes does not automatically persist every turn. A memory provider -can receive data through three separate paths, and the provider -decides what to store and how. - -**** Explicit memory tools (provider tools) -:PROPERTIES: -:CUSTOM_ID: explicit-memory-tools-provider-tools -:ID: a0b54263-fd4c-4997-a867-834ccce8a8b6 -:END: - -The tools returned by =get_tool_schemas()= are offered to the LLM -alongside Hermes' built-in tools. When the LLM decides a fact is worth -remembering, it calls one of them. The provider receives the call in -=handle_tool_call(name, args, **kwargs)= and must return a JSON -string. - -Retinue exposes these tools: - -- =retinue_memory_add= — store a fact. -- =retinue_memory_search= — search stored facts by semantic similarity. -- =retinue_memory_list= — list facts. -- =retinue_memory_delete= — delete a fact by ID. -- =retinue_memory_stats= — show store statistics. - -This is the primary path in Retinue. The LLM decides when to save, and -the user can explicitly ask "remember that ...". - -**** Built-in =memory= tool mirror (=on_memory_write=) -:PROPERTIES: -:CUSTOM_ID: built-in-memory-tool-mirror-on-memory-write -:ID: 25f1b5ee-0b77-412f-98f1-74d69b884f19 -:END: - -Hermes has a built-in =memory= tool that the LLM can call to write durable -facts. When that tool successfully commits a write, Hermes notifies the active -external memory provider through =on_memory_write(action, target, content, -metadata=None)=. - -Fields: - -- =action= — ="add"=, ="replace"=, or ="remove"=. -- =target= — ="memory"= or ="user"=. -- =content= — the text being written. -- =metadata= — provenance dictionary, commonly containing: - - =write_origin= — e.g. ="assistant_tool"= - - =execution_context= — e.g. ="foreground"= or ="background"= - - =session_id= / =parent_session_id= - - =platform= — e.g. ="cli"=, ="telegram"= - - =tool_name= — always ="memory"= for this hook - - =task_id= / =tool_call_id= — when the write happened inside a task - - =old_text= — present for =replace= actions - -Retinue implements this hook so that successful =memory= writes are mirrored -into the Retinue semantic store. They can then be searched via -=retinue_memory_search=. - -**** Turn-level extraction (=sync_turn=) -:PROPERTIES: -:CUSTOM_ID: turn-level-extraction-sync-turn -:ID: 52f3d0c2-5e75-4a49-90b1-50fca051ae1e -:END: - -After every completed turn, Hermes calls =sync_turn(user_content, -assistant_content, *, session_id="", messages=None)= on a background -thread. - -Fields: - -- =user_content= — the cleaned user message for this turn (skill - scaffolding stripped). -- =assistant_content= — the assistant's final response for this turn. -- =session_id= — the active Hermes session ID. -- =messages= — the full OpenAI-style conversation message list /as of - the completed turn/, including any assistant tool calls and tool - results. This is *not* just the latest pair; it is the entire - transcript so far. - -The provider can override this method to extract and store facts from -the raw conversation. It should be non-blocking — queue work to a -background thread if needed, because Hermes calls it synchronously and -a slow provider will stall the agent. - -Retinue's current implementation leaves =sync_turn= as a no-op. Future -versions may use it to summarize each turn and store extracted facts -automatically. - -**** Summary of who decides what to save -:PROPERTIES: -:CUSTOM_ID: summary-of-who-decides-what-to-save -:ID: f9bf3bb2-f15e-4f37-82d1-52254e12bfd0 -:END: - -| Path | Trigger | What the provider receives | -|----------------------+---------------------------------------------+------------------------------------------------------| -| Provider tools | LLM calls one of =get_tool_schemas()= tools | Tool name + structured arguments | -| Built-in memory tool | LLM calls =memory= | =on_memory_write(action, target, content, metadata)= | -| Turn sync | Hermes after every completed turn | =sync_turn(user, assistant, session_id, messages)= | - -*** Memory retrieval -:PROPERTIES: -:CUSTOM_ID: memory-retrieval -:ID: e30a9286-6083-48ed-a861-61c72a5737b4 -:END: - -Retrieval is also provider-driven. Hermes calls =prefetch(query, -session_id="")= before each turn; the provider can return relevant -context as a formatted string. This text is injected into the system -prompt block and fenced as ==. - -In addition, the LLM can explicitly call provider tools (e.g. -=retinue_memory_search=) or the built-in =memory= tool to look up facts on -demand. - -Retinue currently uses the provider-tool path for retrieval; =prefetch= is a -no-op. - -*** Relationship to built-in =memory= -:PROPERTIES: -:CUSTOM_ID: relationship-to-built-in-memory -:ID: 4aba035e-ab65-48c6-925c-49b68a3e65be -:END: - -Hermes always exposes the built-in =memory= tool. It stores compact, -high-signal notes in two text files (=MEMORY.md= and =USER.md=) that are -injected into the system prompt as a frozen snapshot. It is best for short, -durable facts: preferences, corrections, environment details, and reusable -lessons. - -When Retinue is active, the LLM sees two memory surfaces: - -- Built-in =memory= — compact, always-on, text-file storage. -- =retinue_memory_*= — local semantic memory with embeddings and similarity - search. - -The LLM chooses based on the tool descriptions. There is no hard-coded rule. -In practice: - -- Use =memory= for short, high-signal facts that should appear in every system - prompt (e.g. "User prefers concise responses", "Project uses pytest"). -- Use =retinue_memory_add= for richer knowledge that may be recalled later by - meaning (e.g. "ProjectX is hosted on our primary server"). - =retinue_memory_search= can find it even when the query uses different words. - -To avoid losing facts when the LLM picks the built-in =memory= tool, Retinue -implements =on_memory_write= so successful =memory= writes (add, replace, remove) -are mirrored into the Retinue semantic store. They can then be searched via -=retinue_memory_search=. - -*** Tools exposed to Hermes -:PROPERTIES: -:CUSTOM_ID: tools-exposed-to-hermes -:ID: 04cf7ccf-e71a-4f91-a023-3dbf87d5dbf5 -:END: - -When Retinue is the active memory provider, Hermes can call these tools -natively. The exact names and schemas are defined in -=get_tool_schemas()= and dispatched through =handle_tool_call()=. - -**** =retinue_memory_add= -:PROPERTIES: -:CUSTOM_ID: retinue-memory-add -:ID: 69dbc374-8c72-4c8c-a46e-7617bc0c2b2d -:END: - -Store a memory with content. - -#+begin_src json -{ - "name": "retinue_memory_add", - "description": "Store a semantic memory in Retinue.", - "parameters": { - "type": "object", - "properties": { - "content": {"type": "string", "description": "The memory content"} - }, - "required": ["content"] - } -} -#+end_src - -**** =retinue_memory_search= -:PROPERTIES: -:CUSTOM_ID: retinue-memory-search -:ID: 8c479c9b-141e-4a12-9ee0-8f15e398c0c1 -:END: - -Semantic search across all memories. - -#+begin_src json -{ - "name": "retinue_memory_search", - "description": "Search Retinue memories by semantic similarity.", - "parameters": { - "type": "object", - "properties": { - "query": {"type": "string", "description": "Natural-language search query"}, - "limit": {"type": "integer", "default": 10} - }, - "required": ["query"] - } -} -#+end_src - -**** =retinue_memory_list= -:PROPERTIES: -:CUSTOM_ID: retinue-memory-list -:ID: 42c019eb-69d9-438f-b7c9-cdc1685a5526 -:END: - -List all stored memories. - -**** =retinue_memory_delete= -:PROPERTIES: -:CUSTOM_ID: retinue-memory-delete -:ID: 4d3fb3e3-6dc3-4184-9c23-dfd0714130db -:END: - -Delete a memory by ID. - -**** =retinue_memory_stats= -:PROPERTIES: -:CUSTOM_ID: retinue-memory-stats -:ID: 7d56b6c5-c889-43a3-980e-6b8ed8277ec1 -:END: - -Show memory store statistics. -** Development and testing workflow -:PROPERTIES: -:CUSTOM_ID: development-and-testing-workflow -:ID: 246ce71b-6841-49fb-a330-2bb83fc275d3 -:END: - -Retinue is developed in its own source tree and deployed into the active -Hermes home with =install.sh=. This makes iteration fast and safe: the -runtime =data/= directory lives in Hermes, while the source code lives in -the workspace. - -*** Iterative development loop -:PROPERTIES: -:CUSTOM_ID: iterative-development-loop -:ID: b5b488be-6961-4b2a-ae81-f11d37b73c0b -:END: - -1. Edit source files in the Retinue workspace. -2. Run the installer to copy the latest code into Hermes: - - #+begin_src sh - cd /path/to/retinue - bash install.sh - #+end_src - -3. Verify the plugin loads: - - #+begin_src sh - hermes memory status - #+end_src - -4. Test through a Hermes session: - - #+begin_src sh - hermes chat -q "Remember that my default shell is fish." - hermes chat -q "What shell do I use?" - #+end_src - -5. Repeat from step 1. - -*** Standalone testing -:PROPERTIES: -:CUSTOM_ID: standalone-testing -:ID: 24aa4d55-e730-451c-aa27-9b9f4e5959c7 -:END: - -You can also test Retinue without installing it into Hermes. The project -includes a standalone test runner: - -#+begin_src sh -cd /path/to/retinue -./run-tests.sh -#+end_src - -This uses a project-local =.venv= and caches downloaded models in -=.hf-cache=, both of which are excluded from deployment by =install.sh=. +For detailed architecture, integration with Hermes Agent, and the +project development workflow, see the dedicated [[file:development/][Development]] page. -- 2.20.1