--- /dev/null
+#+SETUPFILE: ~/.emacs.d/org-styles/html/darksun.theme
+#+TITLE: Development - Retinue
+#+LANGUAGE: en
+#+LATEX_HEADER: \usepackage[margin=1.0in]{geometry}
+#+LATEX_HEADER: \usepackage{parskip}
+#+LATEX_HEADER: \usepackage[none]{hyphenat}
+
+#+OPTIONS: H:20 num:20
+#+OPTIONS: author:nil
+
+[[file:../index.org][Back to main documentation]]
+
+* Architecture
+:PROPERTIES:
+:CUSTOM_ID: architecture
+:ID: ad252f6e-8152-4dd4-95f6-7b52f773e740
+:END:
+
+Retinue uses:
+
+- =model2vec= :: Static embedding model (=minishlab/potion-multilingual-128M=,
+ ~500 MB download on first use, then offline). Understands meaning across
+ languages.
+- =sqlite-vec= :: SQLite extension for vector similarity search. No
+ server, no daemon, no port.
+- Unified store :: All memories share one table. Each memory has a
+ content string, a timestamp, and an auto-generated embedding.
+
+* What install.sh does
+:PROPERTIES:
+:CUSTOM_ID: what-install-sh-does
+:ID: 9f6b3951-e8f0-4643-bd54-b35bdf528d6e
+:END:
+
+=install.sh= is the deployment bridge between the Retinue workspace and the
+active Hermes home. It is intended for iterative development: edit the
+source tree, run the script, and test inside Hermes immediately.
+
+The script is idempotent, so you can run it after every code change. It
+preserves runtime artifacts (the database, virtual environments, and caches)
+while replacing the plugin source code.
+
+What it does, in order:
+
+1. Detects the active Hermes home from the =HERMES_HOME= environment variable,
+ or falls back to =$HOME/.hermes=.
+2. Creates =$HERMES_HOME/plugins/retinue/= if it does not exist.
+3. Resolves the Python interpreter that Hermes actually runs under (via the
+ =PYTHON= environment variable, common venv layouts, or the =hermes=
+ launcher shebang) and checks for =sqlite-vec= and =model2vec=.
+4. Installs any missing dependencies in that interpreter with =pip=.
+5. Copies the latest source tree into the plugin directory using =rsync=,
+ excluding =data/=, =.venv/=, =.hf-cache/=, =__pycache__/=, and =.git/= so
+ the runtime store and virtual environment are preserved.
+6. Sets =memory.provider: retinue= with =hermes config set= if the CLI is
+ available.
+7. Runs a lightweight import check to confirm the plugin loads.
+
+To target a specific Hermes profile or home directory:
+
+#+begin_src sh
+HERMES_HOME="$HOME/.hermes/profiles/coder" bash install.sh
+#+end_src
+
+* Project structure
+:PROPERTIES:
+:CUSTOM_ID: project-structure
+:ID: 6d9ea612-7ecf-4eef-96e0-ab2e9b744a2e
+:END:
+
+#+begin_src
+Retinue/
+├── doc/
+│ └── index.org # This file
+├── install.sh # Deploy into the active Hermes plugin directory
+├── plugin.yaml # Hermes plugin metadata
+├── __init__.py # Plugin entry point: register(ctx)
+├── src/ # Plugin source package
+│ └── retinue/
+│ ├── __init__.py # Package init
+│ ├── memory.py # Memory engine: embeddings + sqlite-vec
+│ └── provider.py # MemoryProvider subclass
+├── cli.py # Optional: hermes retinue <subcommand> CLI
+└── data/ # Created at runtime under the active Hermes home
+ └── memory.db # SQLite + sqlite-vec database
+#+end_src
+
+* Database layout
+:PROPERTIES:
+:CUSTOM_ID: database-layout
+:ID: 2e9bc319-c314-4a48-8fdf-0c6bac11f344
+:END:
+
+Retinue stores all data in a single SQLite virtual table under the active
+Hermes home:
+
+#+begin_src text
+$HERMES_HOME/retinue_memory.db
+#+end_src
+
+*"memories" virtual table:*
+
+| Column | Type | Meaning |
+|-------------+------------------+----------------------------------------------------------|
+| =rowid= | INTEGER | Auto-increment primary key. Stable ID used for deletion. |
+| =embedding= | float[MODEL_DIM] | The model2vec embedding vector for =content=. |
+| =content= | TEXT | The full text of the memory/fact. |
+| =created= | TEXT | ISO 8601 UTC timestamp when the row was inserted. |
+
+=MODEL_DIM= depends on the model (hardcoded to
+=minishlab/potion-multilingual-128M=). The table is created with
+=sqlite-vec= and supports cosine-similarity search.
+
+Why a /virtual/ table? In SQLite, a virtual table is created with
+=CREATE VIRTUAL TABLE ... USING vec0(...)=. The =vec0= module from
+=sqlite-vec= provides the storage and indexing logic for the
+=embedding= column, which enables the =embedding MATCH ?= vector
+similarity query. The table is still persisted in the same SQLite
+=.db= file; “virtual” only means SQLite delegates the table engine to
+the =sqlite-vec= extension.
+
+* Integration with Hermes Agent
+:PROPERTIES:
+:CUSTOM_ID: integration-with-hermes-agent
+:ID: ca6a072f-b686-4dec-b50a-e4b83980be61
+:END:
+
+Retinue implements the Hermes =MemoryProvider= abstract base class from
+=agent/memory_provider.py=.
+
+** Required methods
+:PROPERTIES:
+:CUSTOM_ID: required-methods
+:ID: 845b8e5d-7e94-4b93-b147-d065f864eeb3
+:END:
+
+| Method | Purpose | Notes |
+|------------------------------------------+---------------------------------------+-------------------------------------------------------------------------------------------|
+| =name= (property) | Provider identifier, e.g. ="retinue"= | Must be unique |
+| =is_available()= | Can this provider activate? | No network calls; check deps/config/env |
+| =initialize(session_id, **kwargs)= | One-time startup | Receives =hermes_home=, =platform=, =agent_context=, etc. |
+| =get_tool_schemas()= | Tool schemas to expose to the model | OpenAI function-calling format; return =[]= if context-only |
+| =handle_tool_call(name, args, **kwargs)= | Execute a tool call | Must return a JSON string |
+| =get_config_schema()= | Fields for =hermes memory setup= | List of field descriptors (key, description, secret, required, default, choices, env_var) |
+| =save_config(values, hermes_home)= | Persist non-secret config | Secrets go to =.env=; env-var-only providers can no-op |
+
+** Optional hooks
+:PROPERTIES:
+:CUSTOM_ID: optional-hooks
+:ID: f0792483-e9ea-4ad0-be64-7a4e9544a489
+:END:
+
+| Method | Purpose |
+|---------------------------------------------------------------+---------------------------------------------------------------|
+| =system_prompt_block()= | Static text injected into the system prompt |
+| =prefetch(query, *, session_id="")= | Recall relevant context before each turn (should be fast) |
+| =queue_prefetch(query)= | Pre-warm recall for the next turn |
+| =sync_turn(user, assistant, *, session_id="", messages=None)= | Persist a completed turn (must be non-blocking) |
+| =on_session_end(messages)= | Flush/extract at conversation end |
+| =on_pre_compress(messages) -> str= | Capture insights before context compression |
+| =on_memory_write(action, target, content, metadata=None)= | Mirror built-in =memory= writes to the provider |
+| =on_delegation(task, result, **kwargs)= | Observe subagent work from the parent |
+| =shutdown()= | Clean up connections/threads |
+| =backup_paths() -> list[str]= | Extra on-disk paths outside =HERMES_HOME= for =hermes backup= |
+
+** Memory save
+:PROPERTIES:
+:CUSTOM_ID: memory-save
+:ID: 6325a3ac-8629-461e-aa5e-62408ba73bde
+:END:
+
+Hermes does not automatically persist every turn. A memory provider
+can receive data through three separate paths, and the provider
+decides what to store and how.
+
+*** Explicit memory tools (provider tools)
+:PROPERTIES:
+:CUSTOM_ID: explicit-memory-tools-provider-tools
+:ID: a0b54263-fd4c-4997-a867-834ccce8a8b6
+:END:
+
+The tools returned by =get_tool_schemas()= are offered to the LLM
+alongside Hermes' built-in tools. When the LLM decides a fact is worth
+remembering, it calls one of them. The provider receives the call in
+=handle_tool_call(name, args, **kwargs)= and must return a JSON
+string.
+
+Retinue exposes these tools:
+
+- =retinue_memory_add= — store a fact.
+- =retinue_memory_search= — search stored facts by semantic similarity.
+- =retinue_memory_list= — list facts.
+- =retinue_memory_delete= — delete a fact by ID.
+- =retinue_memory_stats= — show store statistics.
+
+This is the primary path in Retinue. The LLM decides when to save, and
+the user can explicitly ask "remember that ...".
+
+*** Built-in =memory= tool mirror (=on_memory_write=)
+:PROPERTIES:
+:CUSTOM_ID: built-in-memory-tool-mirror-on-memory-write
+:ID: 25f1b5ee-0b77-412f-98f1-74d69b884f19
+:END:
+
+Hermes has a built-in =memory= tool that the LLM can call to write durable
+facts. When that tool successfully commits a write, Hermes notifies the active
+external memory provider through =on_memory_write(action, target, content,
+metadata=None)=.
+
+Fields:
+
+- =action= — ="add"=, ="replace"=, or ="remove"=.
+- =target= — ="memory"= or ="user"=.
+- =content= — the text being written.
+- =metadata= — provenance dictionary, commonly containing:
+ - =write_origin= — e.g. ="assistant_tool"=
+ - =execution_context= — e.g. ="foreground"= or ="background"=
+ - =session_id= / =parent_session_id=
+ - =platform= — e.g. ="cli"=, ="telegram"=
+ - =tool_name= — always ="memory"= for this hook
+ - =task_id= / =tool_call_id= — when the write happened inside a task
+ - =old_text= — present for =replace= actions
+
+Retinue implements this hook so that successful =memory= writes are mirrored
+into the Retinue semantic store. They can then be searched via
+=retinue_memory_search=.
+
+*** Turn-level extraction (=sync_turn=)
+:PROPERTIES:
+:CUSTOM_ID: turn-level-extraction-sync-turn
+:ID: 52f3d0c2-5e75-4a49-90b1-50fca051ae1e
+:END:
+
+After every completed turn, Hermes calls =sync_turn(user_content,
+assistant_content, *, session_id="", messages=None)= on a background
+thread.
+
+Fields:
+
+- =user_content= — the cleaned user message for this turn (skill
+ scaffolding stripped).
+- =assistant_content= — the assistant's final response for this turn.
+- =session_id= — the active Hermes session ID.
+- =messages= — the full OpenAI-style conversation message list /as of
+ the completed turn/, including any assistant tool calls and tool
+ results. This is *not* just the latest pair; it is the entire
+ transcript so far.
+
+The provider can override this method to extract and store facts from
+the raw conversation. It should be non-blocking — queue work to a
+background thread if needed, because Hermes calls it synchronously and
+a slow provider will stall the agent.
+
+Retinue's current implementation leaves =sync_turn= as a no-op. Future
+versions may use it to summarize each turn and store extracted facts
+automatically.
+
+*** Summary of who decides what to save
+:PROPERTIES:
+:CUSTOM_ID: summary-of-who-decides-what-to-save
+:ID: f9bf3bb2-f15e-4f37-82d1-52254e12bfd0
+:END:
+
+| Path | Trigger | What the provider receives |
+|----------------------+---------------------------------------------+------------------------------------------------------|
+| Provider tools | LLM calls one of =get_tool_schemas()= tools | Tool name + structured arguments |
+| Built-in memory tool | LLM calls =memory= | =on_memory_write(action, target, content, metadata)= |
+| Turn sync | Hermes after every completed turn | =sync_turn(user, assistant, session_id, messages)= |
+
+** Memory retrieval
+:PROPERTIES:
+:CUSTOM_ID: memory-retrieval
+:ID: e30a9286-6083-48ed-a861-61c72a5737b4
+:END:
+
+Retrieval is also provider-driven. Hermes calls =prefetch(query,
+session_id="")= before each turn; the provider can return relevant
+context as a formatted string. This text is injected into the system
+prompt block and fenced as =<memory-context>=.
+
+In addition, the LLM can explicitly call provider tools (e.g.
+=retinue_memory_search=) or the built-in =memory= tool to look up facts on
+demand.
+
+Retinue currently uses the provider-tool path for retrieval; =prefetch= is a
+no-op.
+
+** Relationship to built-in =memory=
+:PROPERTIES:
+:CUSTOM_ID: relationship-to-built-in-memory
+:ID: 4aba035e-ab65-48c6-925c-49b68a3e65be
+:END:
+
+Hermes always exposes the built-in =memory= tool. It stores compact,
+high-signal notes in two text files (=MEMORY.md= and =USER.md=) that are
+injected into the system prompt as a frozen snapshot. It is best for short,
+durable facts: preferences, corrections, environment details, and reusable
+lessons.
+
+When Retinue is active, the LLM sees two memory surfaces:
+
+- Built-in =memory= — compact, always-on, text-file storage.
+- =retinue_memory_*= — local semantic memory with embeddings and similarity
+ search.
+
+The LLM chooses based on the tool descriptions. There is no hard-coded rule.
+In practice:
+
+- Use =memory= for short, high-signal facts that should appear in every system
+ prompt (e.g. "User prefers concise responses", "Project uses pytest").
+- Use =retinue_memory_add= for richer knowledge that may be recalled later by
+ meaning (e.g. "ProjectX is hosted on our primary server").
+ =retinue_memory_search= can find it even when the query uses different words.
+
+To avoid losing facts when the LLM picks the built-in =memory= tool, Retinue
+implements =on_memory_write= so successful =memory= writes (add, replace, remove)
+are mirrored into the Retinue semantic store. They can then be searched via
+=retinue_memory_search=.
+
+** Tools exposed to Hermes
+:PROPERTIES:
+:CUSTOM_ID: tools-exposed-to-hermes
+:ID: 04cf7ccf-e71a-4f91-a023-3dbf87d5dbf5
+:END:
+
+When Retinue is the active memory provider, Hermes can call these tools
+natively. The exact names and schemas are defined in
+=get_tool_schemas()= and dispatched through =handle_tool_call()=.
+
+*** =retinue_memory_add=
+:PROPERTIES:
+:CUSTOM_ID: retinue-memory-add
+:ID: 69dbc374-8c72-4c8c-a46e-7617bc0c2b2d
+:END:
+
+Store a memory with content.
+
+#+begin_src json
+{
+ "name": "retinue_memory_add",
+ "description": "Store a semantic memory in Retinue.",
+ "parameters": {
+ "type": "object",
+ "properties": {
+ "content": {"type": "string", "description": "The memory content"}
+ },
+ "required": ["content"]
+ }
+}
+#+end_src
+
+*** =retinue_memory_search=
+:PROPERTIES:
+:CUSTOM_ID: retinue-memory-search
+:ID: 8c479c9b-141e-4a12-9ee0-8f15e398c0c1
+:END:
+
+Semantic search across all memories.
+
+#+begin_src json
+{
+ "name": "retinue_memory_search",
+ "description": "Search Retinue memories by semantic similarity.",
+ "parameters": {
+ "type": "object",
+ "properties": {
+ "query": {"type": "string", "description": "Natural-language search query"},
+ "limit": {"type": "integer", "default": 10}
+ },
+ "required": ["query"]
+ }
+}
+#+end_src
+
+*** =retinue_memory_list=
+:PROPERTIES:
+:CUSTOM_ID: retinue-memory-list
+:ID: 42c019eb-69d9-438f-b7c9-cdc1685a5526
+:END:
+
+List all stored memories.
+
+*** =retinue_memory_delete=
+:PROPERTIES:
+:CUSTOM_ID: retinue-memory-delete
+:ID: 4d3fb3e3-6dc3-4184-9c23-dfd0714130db
+:END:
+
+Delete a memory by ID.
+
+*** =retinue_memory_stats=
+:PROPERTIES:
+:CUSTOM_ID: retinue-memory-stats
+:ID: 7d56b6c5-c889-43a3-980e-6b8ed8277ec1
+:END:
+
+Show memory store statistics.
+* Development and testing workflow
+:PROPERTIES:
+:CUSTOM_ID: development-and-testing-workflow
+:ID: 246ce71b-6841-49fb-a330-2bb83fc275d3
+:END:
+
+Retinue is developed in its own source tree and deployed into the active
+Hermes home with =install.sh=. This makes iteration fast and safe: the
+runtime =data/= directory lives in Hermes, while the source code lives in
+the workspace.
+
+** Iterative development loop
+:PROPERTIES:
+:CUSTOM_ID: iterative-development-loop
+:ID: b5b488be-6961-4b2a-ae81-f11d37b73c0b
+:END:
+
+1. Edit source files in the Retinue workspace.
+2. Run the installer to copy the latest code into Hermes:
+
+ #+begin_src sh
+ cd /path/to/retinue
+ bash install.sh
+ #+end_src
+
+3. Verify the plugin loads:
+
+ #+begin_src sh
+ hermes memory status
+ #+end_src
+
+4. Test through a Hermes session:
+
+ #+begin_src sh
+ hermes chat -q "Remember that my default shell is fish."
+ hermes chat -q "What shell do I use?"
+ #+end_src
+
+5. Repeat from step 1.
+
+** Standalone testing
+:PROPERTIES:
+:CUSTOM_ID: standalone-testing
+:ID: 24aa4d55-e730-451c-aa27-9b9f4e5959c7
+:END:
+
+You can also test Retinue without installing it into Hermes. The project
+includes a standalone test runner:
+
+#+begin_src sh
+cd /path/to/retinue
+./run-tests.sh
+#+end_src
+
+This uses a project-local =.venv= and caches downloaded models in
+=.hf-cache=, both of which are excluded from deployment by =install.sh=.
Hermes will call =retinue_memory_search= and retrieve the relevant memory.
-* Development
+* Source code
:PROPERTIES:
:CUSTOM_ID: development
-:ID: d77ed683-4da0-4cf1-84ee-73544b93f28b
:END:
-This section explains in more detail how this plugin works, in case
-you are a software developer and want to troubleshoot or improve it.
-
*This program is free software: released under Creative Commons Zero
(CC0) license*
: git clone https://www3.svjatoslav.eu/git/retinue.git
-** Architecture
-:PROPERTIES:
-:CUSTOM_ID: architecture
-:ID: ad252f6e-8152-4dd4-95f6-7b52f773e740
-:END:
-
-Retinue uses:
-
-- =model2vec= :: Static embedding model (=minishlab/potion-multilingual-128M=,
- ~500 MB download on first use, then offline). Understands meaning across
- languages.
-- =sqlite-vec= :: SQLite extension for vector similarity search. No
- server, no daemon, no port.
-- Unified store :: All memories share one table. Each memory has a
- content string, a timestamp, and an auto-generated embedding.
-
-** What install.sh does
-:PROPERTIES:
-:CUSTOM_ID: what-install-sh-does
-:ID: 9f6b3951-e8f0-4643-bd54-b35bdf528d6e
-:END:
-
-=install.sh= is the deployment bridge between the Retinue workspace and the
-active Hermes home. It is intended for iterative development: edit the
-source tree, run the script, and test inside Hermes immediately.
-
-The script is idempotent, so you can run it after every code change. It
-preserves runtime artifacts (the database, virtual environments, and caches)
-while replacing the plugin source code.
-
-What it does, in order:
-
-1. Detects the active Hermes home from the =HERMES_HOME= environment variable,
- or falls back to =$HOME/.hermes=.
-2. Creates =$HERMES_HOME/plugins/retinue/= if it does not exist.
-3. Resolves the Python interpreter that Hermes actually runs under (via the
- =PYTHON= environment variable, common venv layouts, or the =hermes=
- launcher shebang) and checks for =sqlite-vec= and =model2vec=.
-4. Installs any missing dependencies in that interpreter with =pip=.
-5. Copies the latest source tree into the plugin directory using =rsync=,
- excluding =data/=, =.venv/=, =.hf-cache/=, =__pycache__/=, and =.git/= so
- the runtime store and virtual environment are preserved.
-6. Sets =memory.provider: retinue= with =hermes config set= if the CLI is
- available.
-7. Runs a lightweight import check to confirm the plugin loads.
-
-To target a specific Hermes profile or home directory:
-
-#+begin_src sh
-HERMES_HOME="$HOME/.hermes/profiles/coder" bash install.sh
-#+end_src
-
-** Project structure
-:PROPERTIES:
-:CUSTOM_ID: project-structure
-:ID: 6d9ea612-7ecf-4eef-96e0-ab2e9b744a2e
-:END:
-
-#+begin_src
-Retinue/
-├── doc/
-│ └── index.org # This file
-├── install.sh # Deploy into the active Hermes plugin directory
-├── plugin.yaml # Hermes plugin metadata
-├── __init__.py # Plugin entry point: register(ctx)
-├── src/ # Plugin source package
-│ └── retinue/
-│ ├── __init__.py # Package init
-│ ├── memory.py # Memory engine: embeddings + sqlite-vec
-│ └── provider.py # MemoryProvider subclass
-├── cli.py # Optional: hermes retinue <subcommand> CLI
-└── data/ # Created at runtime under the active Hermes home
- └── memory.db # SQLite + sqlite-vec database
-#+end_src
-
-** Database layout
-:PROPERTIES:
-:CUSTOM_ID: database-layout
-:ID: 2e9bc319-c314-4a48-8fdf-0c6bac11f344
-:END:
-
-Retinue stores all data in a single SQLite virtual table under the active
-Hermes home:
-
-#+begin_src text
-$HERMES_HOME/retinue_memory.db
-#+end_src
-
-*"memories" virtual table:*
-
-| Column | Type | Meaning |
-|-------------+------------------+----------------------------------------------------------|
-| =rowid= | INTEGER | Auto-increment primary key. Stable ID used for deletion. |
-| =embedding= | float[MODEL_DIM] | The model2vec embedding vector for =content=. |
-| =content= | TEXT | The full text of the memory/fact. |
-| =created= | TEXT | ISO 8601 UTC timestamp when the row was inserted. |
-
-=MODEL_DIM= depends on the model (hardcoded to
-=minishlab/potion-multilingual-128M=). The table is created with
-=sqlite-vec= and supports cosine-similarity search.
-
-Why a /virtual/ table? In SQLite, a virtual table is created with
-=CREATE VIRTUAL TABLE ... USING vec0(...)=. The =vec0= module from
-=sqlite-vec= provides the storage and indexing logic for the
-=embedding= column, which enables the =embedding MATCH ?= vector
-similarity query. The table is still persisted in the same SQLite
-=.db= file; “virtual” only means SQLite delegates the table engine to
-the =sqlite-vec= extension.
-
-** Integration with Hermes Agent
-:PROPERTIES:
-:CUSTOM_ID: integration-with-hermes-agent
-:ID: ca6a072f-b686-4dec-b50a-e4b83980be61
-:END:
-
-Retinue implements the Hermes =MemoryProvider= abstract base class from
-=agent/memory_provider.py=.
-
-*** Required methods
-:PROPERTIES:
-:CUSTOM_ID: required-methods
-:ID: 845b8e5d-7e94-4b93-b147-d065f864eeb3
-:END:
-
-| Method | Purpose | Notes |
-|------------------------------------------+---------------------------------------+-------------------------------------------------------------------------------------------|
-| =name= (property) | Provider identifier, e.g. ="retinue"= | Must be unique |
-| =is_available()= | Can this provider activate? | No network calls; check deps/config/env |
-| =initialize(session_id, **kwargs)= | One-time startup | Receives =hermes_home=, =platform=, =agent_context=, etc. |
-| =get_tool_schemas()= | Tool schemas to expose to the model | OpenAI function-calling format; return =[]= if context-only |
-| =handle_tool_call(name, args, **kwargs)= | Execute a tool call | Must return a JSON string |
-| =get_config_schema()= | Fields for =hermes memory setup= | List of field descriptors (key, description, secret, required, default, choices, env_var) |
-| =save_config(values, hermes_home)= | Persist non-secret config | Secrets go to =.env=; env-var-only providers can no-op |
-
-*** Optional hooks
-:PROPERTIES:
-:CUSTOM_ID: optional-hooks
-:ID: f0792483-e9ea-4ad0-be64-7a4e9544a489
-:END:
-
-| Method | Purpose |
-|---------------------------------------------------------------+---------------------------------------------------------------|
-| =system_prompt_block()= | Static text injected into the system prompt |
-| =prefetch(query, *, session_id="")= | Recall relevant context before each turn (should be fast) |
-| =queue_prefetch(query)= | Pre-warm recall for the next turn |
-| =sync_turn(user, assistant, *, session_id="", messages=None)= | Persist a completed turn (must be non-blocking) |
-| =on_session_end(messages)= | Flush/extract at conversation end |
-| =on_pre_compress(messages) -> str= | Capture insights before context compression |
-| =on_memory_write(action, target, content, metadata=None)= | Mirror built-in =memory= writes to the provider |
-| =on_delegation(task, result, **kwargs)= | Observe subagent work from the parent |
-| =shutdown()= | Clean up connections/threads |
-| =backup_paths() -> list[str]= | Extra on-disk paths outside =HERMES_HOME= for =hermes backup= |
-
-*** Memory save
-:PROPERTIES:
-:CUSTOM_ID: memory-save
-:ID: 6325a3ac-8629-461e-aa5e-62408ba73bde
-:END:
-
-Hermes does not automatically persist every turn. A memory provider
-can receive data through three separate paths, and the provider
-decides what to store and how.
-
-**** Explicit memory tools (provider tools)
-:PROPERTIES:
-:CUSTOM_ID: explicit-memory-tools-provider-tools
-:ID: a0b54263-fd4c-4997-a867-834ccce8a8b6
-:END:
-
-The tools returned by =get_tool_schemas()= are offered to the LLM
-alongside Hermes' built-in tools. When the LLM decides a fact is worth
-remembering, it calls one of them. The provider receives the call in
-=handle_tool_call(name, args, **kwargs)= and must return a JSON
-string.
-
-Retinue exposes these tools:
-
-- =retinue_memory_add= — store a fact.
-- =retinue_memory_search= — search stored facts by semantic similarity.
-- =retinue_memory_list= — list facts.
-- =retinue_memory_delete= — delete a fact by ID.
-- =retinue_memory_stats= — show store statistics.
-
-This is the primary path in Retinue. The LLM decides when to save, and
-the user can explicitly ask "remember that ...".
-
-**** Built-in =memory= tool mirror (=on_memory_write=)
-:PROPERTIES:
-:CUSTOM_ID: built-in-memory-tool-mirror-on-memory-write
-:ID: 25f1b5ee-0b77-412f-98f1-74d69b884f19
-:END:
-
-Hermes has a built-in =memory= tool that the LLM can call to write durable
-facts. When that tool successfully commits a write, Hermes notifies the active
-external memory provider through =on_memory_write(action, target, content,
-metadata=None)=.
-
-Fields:
-
-- =action= — ="add"=, ="replace"=, or ="remove"=.
-- =target= — ="memory"= or ="user"=.
-- =content= — the text being written.
-- =metadata= — provenance dictionary, commonly containing:
- - =write_origin= — e.g. ="assistant_tool"=
- - =execution_context= — e.g. ="foreground"= or ="background"=
- - =session_id= / =parent_session_id=
- - =platform= — e.g. ="cli"=, ="telegram"=
- - =tool_name= — always ="memory"= for this hook
- - =task_id= / =tool_call_id= — when the write happened inside a task
- - =old_text= — present for =replace= actions
-
-Retinue implements this hook so that successful =memory= writes are mirrored
-into the Retinue semantic store. They can then be searched via
-=retinue_memory_search=.
-
-**** Turn-level extraction (=sync_turn=)
-:PROPERTIES:
-:CUSTOM_ID: turn-level-extraction-sync-turn
-:ID: 52f3d0c2-5e75-4a49-90b1-50fca051ae1e
-:END:
-
-After every completed turn, Hermes calls =sync_turn(user_content,
-assistant_content, *, session_id="", messages=None)= on a background
-thread.
-
-Fields:
-
-- =user_content= — the cleaned user message for this turn (skill
- scaffolding stripped).
-- =assistant_content= — the assistant's final response for this turn.
-- =session_id= — the active Hermes session ID.
-- =messages= — the full OpenAI-style conversation message list /as of
- the completed turn/, including any assistant tool calls and tool
- results. This is *not* just the latest pair; it is the entire
- transcript so far.
-
-The provider can override this method to extract and store facts from
-the raw conversation. It should be non-blocking — queue work to a
-background thread if needed, because Hermes calls it synchronously and
-a slow provider will stall the agent.
-
-Retinue's current implementation leaves =sync_turn= as a no-op. Future
-versions may use it to summarize each turn and store extracted facts
-automatically.
-
-**** Summary of who decides what to save
-:PROPERTIES:
-:CUSTOM_ID: summary-of-who-decides-what-to-save
-:ID: f9bf3bb2-f15e-4f37-82d1-52254e12bfd0
-:END:
-
-| Path | Trigger | What the provider receives |
-|----------------------+---------------------------------------------+------------------------------------------------------|
-| Provider tools | LLM calls one of =get_tool_schemas()= tools | Tool name + structured arguments |
-| Built-in memory tool | LLM calls =memory= | =on_memory_write(action, target, content, metadata)= |
-| Turn sync | Hermes after every completed turn | =sync_turn(user, assistant, session_id, messages)= |
-
-*** Memory retrieval
-:PROPERTIES:
-:CUSTOM_ID: memory-retrieval
-:ID: e30a9286-6083-48ed-a861-61c72a5737b4
-:END:
-
-Retrieval is also provider-driven. Hermes calls =prefetch(query,
-session_id="")= before each turn; the provider can return relevant
-context as a formatted string. This text is injected into the system
-prompt block and fenced as =<memory-context>=.
-
-In addition, the LLM can explicitly call provider tools (e.g.
-=retinue_memory_search=) or the built-in =memory= tool to look up facts on
-demand.
-
-Retinue currently uses the provider-tool path for retrieval; =prefetch= is a
-no-op.
-
-*** Relationship to built-in =memory=
-:PROPERTIES:
-:CUSTOM_ID: relationship-to-built-in-memory
-:ID: 4aba035e-ab65-48c6-925c-49b68a3e65be
-:END:
-
-Hermes always exposes the built-in =memory= tool. It stores compact,
-high-signal notes in two text files (=MEMORY.md= and =USER.md=) that are
-injected into the system prompt as a frozen snapshot. It is best for short,
-durable facts: preferences, corrections, environment details, and reusable
-lessons.
-
-When Retinue is active, the LLM sees two memory surfaces:
-
-- Built-in =memory= — compact, always-on, text-file storage.
-- =retinue_memory_*= — local semantic memory with embeddings and similarity
- search.
-
-The LLM chooses based on the tool descriptions. There is no hard-coded rule.
-In practice:
-
-- Use =memory= for short, high-signal facts that should appear in every system
- prompt (e.g. "User prefers concise responses", "Project uses pytest").
-- Use =retinue_memory_add= for richer knowledge that may be recalled later by
- meaning (e.g. "ProjectX is hosted on our primary server").
- =retinue_memory_search= can find it even when the query uses different words.
-
-To avoid losing facts when the LLM picks the built-in =memory= tool, Retinue
-implements =on_memory_write= so successful =memory= writes (add, replace, remove)
-are mirrored into the Retinue semantic store. They can then be searched via
-=retinue_memory_search=.
-
-*** Tools exposed to Hermes
-:PROPERTIES:
-:CUSTOM_ID: tools-exposed-to-hermes
-:ID: 04cf7ccf-e71a-4f91-a023-3dbf87d5dbf5
-:END:
-
-When Retinue is the active memory provider, Hermes can call these tools
-natively. The exact names and schemas are defined in
-=get_tool_schemas()= and dispatched through =handle_tool_call()=.
-
-**** =retinue_memory_add=
-:PROPERTIES:
-:CUSTOM_ID: retinue-memory-add
-:ID: 69dbc374-8c72-4c8c-a46e-7617bc0c2b2d
-:END:
-
-Store a memory with content.
-
-#+begin_src json
-{
- "name": "retinue_memory_add",
- "description": "Store a semantic memory in Retinue.",
- "parameters": {
- "type": "object",
- "properties": {
- "content": {"type": "string", "description": "The memory content"}
- },
- "required": ["content"]
- }
-}
-#+end_src
-
-**** =retinue_memory_search=
-:PROPERTIES:
-:CUSTOM_ID: retinue-memory-search
-:ID: 8c479c9b-141e-4a12-9ee0-8f15e398c0c1
-:END:
-
-Semantic search across all memories.
-
-#+begin_src json
-{
- "name": "retinue_memory_search",
- "description": "Search Retinue memories by semantic similarity.",
- "parameters": {
- "type": "object",
- "properties": {
- "query": {"type": "string", "description": "Natural-language search query"},
- "limit": {"type": "integer", "default": 10}
- },
- "required": ["query"]
- }
-}
-#+end_src
-
-**** =retinue_memory_list=
-:PROPERTIES:
-:CUSTOM_ID: retinue-memory-list
-:ID: 42c019eb-69d9-438f-b7c9-cdc1685a5526
-:END:
-
-List all stored memories.
-
-**** =retinue_memory_delete=
-:PROPERTIES:
-:CUSTOM_ID: retinue-memory-delete
-:ID: 4d3fb3e3-6dc3-4184-9c23-dfd0714130db
-:END:
-
-Delete a memory by ID.
-
-**** =retinue_memory_stats=
-:PROPERTIES:
-:CUSTOM_ID: retinue-memory-stats
-:ID: 7d56b6c5-c889-43a3-980e-6b8ed8277ec1
-:END:
-
-Show memory store statistics.
-** Development and testing workflow
-:PROPERTIES:
-:CUSTOM_ID: development-and-testing-workflow
-:ID: 246ce71b-6841-49fb-a330-2bb83fc275d3
-:END:
-
-Retinue is developed in its own source tree and deployed into the active
-Hermes home with =install.sh=. This makes iteration fast and safe: the
-runtime =data/= directory lives in Hermes, while the source code lives in
-the workspace.
-
-*** Iterative development loop
-:PROPERTIES:
-:CUSTOM_ID: iterative-development-loop
-:ID: b5b488be-6961-4b2a-ae81-f11d37b73c0b
-:END:
-
-1. Edit source files in the Retinue workspace.
-2. Run the installer to copy the latest code into Hermes:
-
- #+begin_src sh
- cd /path/to/retinue
- bash install.sh
- #+end_src
-
-3. Verify the plugin loads:
-
- #+begin_src sh
- hermes memory status
- #+end_src
-
-4. Test through a Hermes session:
-
- #+begin_src sh
- hermes chat -q "Remember that my default shell is fish."
- hermes chat -q "What shell do I use?"
- #+end_src
-
-5. Repeat from step 1.
-
-*** Standalone testing
-:PROPERTIES:
-:CUSTOM_ID: standalone-testing
-:ID: 24aa4d55-e730-451c-aa27-9b9f4e5959c7
-:END:
-
-You can also test Retinue without installing it into Hermes. The project
-includes a standalone test runner:
-
-#+begin_src sh
-cd /path/to/retinue
-./run-tests.sh
-#+end_src
-
-This uses a project-local =.venv= and caches downloaded models in
-=.hf-cache=, both of which are excluded from deployment by =install.sh=.
+For detailed architecture, integration with Hermes Agent, and the
+project development workflow, see the dedicated [[file:development/][Development]] page.
projects / retinue.git / commitdiff