--- /dev/null
+A Java CLI tool for batch processing with large language models using
+CPU-based computation via llama.cpp. Designed for "slow but smart" AI
+tasks where batch processing is more valuable than real-time
+interaction.
+
+* Commands
+
+- *wizard* :: Interactive configuration wizard. Validates and fixes
+ configuration files, discovers new models in the models directory,
+ and updates configuration. Run this first to set up or update your
+ setup.
+
+- *process* :: Continuous task processor. Monitors the tasks directory
+ for files starting with =TOCOMPUTE:=, processes them using the
+ configured AI model, and appends results to the same file.
+
+- *listmodels* :: Lists all available models defined in the configuration.
+
+- *joinfiles* :: Utility to concatenate multiple files together (used
+ for preparing task files with context).
+
+- *addtaskheader* :: Adds a TOCOMPUTE header to task files.
+
+* Task File Format
+
+Task files are plain text files placed in the tasks directory. Format:
+
+#+BEGIN_SRC text
+TOCOMPUTE: skill=<skill_name> model=<model_alias> priority=<number>
+
+<user prompt content here>
+#+END_SRC
+
+- First line must start with =TOCOMPUTE:=
+- Parameters are optional: =skill= (defaults to "default"), =model=
+ (defaults per-skill or "default"), =priority= (defaults to 0)
+- Files are processed in priority order (higher priority = processed first)
+- After processing, the file =TOCOMPUTE:= heading in the file is renamed
+ to =DONE:= and contains the AI response
+
+* Configuration
+
+Configuration is stored in YAML format at
+=~/.config/alyverkko-cli/configuration.yaml=. Key settings:
+
+- =tasks_directory= :: Where task files are placed for processing
+- =models_directory= :: Directory containing GGUF model files
+- =skills_directory= :: Directory containing skill (prompt) YAML files
+- =llama_cli_path= :: Path to the llama-cli executable
+- =thread_count=, =batch_thread_count= :: CPU thread settings
+
+** Skills (Prompts)
+
+Skills are YAML files in the skills directory (e.g., =writer.yaml=):
+
+#+BEGIN_SRC yaml
+prompt: |
+ You are a helpful assistant...
+temperature: 0.7
+top_p: 0.9
+model_alias: "mistral"
+timeout_millis: 300000
+#+END_SRC
+
+Model selection hierarchy (highest priority first):
+
+1. Explicit =model= parameter in TOCOMPUTE line
+2. =model_alias= in skill configuration
+3. Model named "default" in configuration
+
+** Models
+
+Models are defined in the configuration with aliases, filesystem paths,
+and optional settings:
+
+#+BEGIN_SRC yaml
+models:
+ - alias: "mistral"
+ filesystem_path: "mistral-7b-instruct-v0.2.Q4_K_M.gguf"
+ context_size: 8192
+ final_answer_indicator: "</s>"
+#+END_SRC
+
+The =final_answer_indicator= splits the AI response into "internal
+thoughts" and "final answer" sections in the output file.
+
+* Source Structure
+
+- =src/main/java/eu/svjatoslav/alyverkko_cli/= :: Main package
+ - =Main.java= :: Entry point, command dispatcher
+ - =Command.java= :: Interface for all commands
+ - =commands/= :: Command implementations
+ - =task_processor/= :: Core batch processing logic
+ - =configuration/= :: Configuration model classes (Configuration, Model, SkillConfig)
+ - =Utils.java= :: Utility functions
#+OPTIONS: author:nil
* Introduction
+:PROPERTIES:
+:CUSTOM_ID: introduction
+:ID: b4da28df-7208-4c27-a1ea-98d8692c5012
+:END:
The *Älyverkko CLI* application is a user-friendly tool developed in
Java, specifically tailored to streamline the utilization of expansive
Note: project is still in early stage.
** Use cases
+:PROPERTIES:
+:CUSTOM_ID: use-cases
+:ID: 9aeba681-2d9d-442f-bc4b-3d61a294e596
+:END:
*** Domain: Natural Language Processing (NLP)
:PROPERTIES:
+:CUSTOM_ID: nlp-domain
:ID: e1b6aa9a-a27d-4019-8bd4-fceb0c606e23
:END:
*** Domain: Code Generation
:PROPERTIES:
+:CUSTOM_ID: code-generation-domain
:ID: f4c7d3e0-8c18-4123-a797-871ca73a7580
:END:
*** Domain: Content Creation
:PROPERTIES:
+:CUSTOM_ID: content-creation-domain
:ID: f38360ad-54f6-4f24-b299-f73a9faacabd
:END:
*Älyverkko CLI*.
** Why Bother With This Setup? (The Big Picture)
+:PROPERTIES:
+:CUSTOM_ID: why-this-setup
+:ID: 5862520c-db92-4a5e-9177-717e8dea66f8
+:END:
Before diving into steps, let's address the elephant in the room: *Why
go through this setup when ChatGPT is just a click away?*
generation, document analysis, content creation, etc.
* Concept glossary
+:PROPERTIES:
+:CUSTOM_ID: concept-glossary
+:ID: 3bff0680-db41-4f97-a2af-f677209dd430
+:END:
** General concepts
+:PROPERTIES:
+:CUSTOM_ID: general-concepts
+:ID: 2ae2e3ec-835f-45e1-bc15-618c8b2c3e85
+:END:
*** Task
:PROPERTIES:
+:CUSTOM_ID: task-concept
:ID: 140c53cb-8032-4a04-83ed-d1818b1cfc52
:END:
*** Skill
:PROPERTIES:
+:CUSTOM_ID: skill-concept
:ID: 6579abb4-8386-418b-9457-cae6c3345dfb
:END:
- See also:
*** Model
:PROPERTIES:
+:CUSTOM_ID: model-concept
:ID: 074af969-8a9b-407d-a605-6725fe4e8580
:END:
*** "TOCOMPUTE" Marker
:PROPERTIES:
+:CUSTOM_ID: tocompute-marker
:ID: cd4b622a-6b74-4fac-85fe-f5b056367824
:END:
*** "DONE" Marker
:PROPERTIES:
+:CUSTOM_ID: done-marker
:ID: aa69e23a-248a-4459-a36e-74b43948dba9
:END:
=TOCOMPUTE:= line to continue the conversation.
*** Priority
+:PROPERTIES:
+:CUSTOM_ID: priority-concept
+:ID: fc7d5e69-a2ef-4afb-b4c3-a549d0792373
+:END:
/Priority/ is an integer value specified in the =TOCOMPUTE:= header
(e.g., =priority=10=) that determines task processing order. Higher
*** System Prompt
:PROPERTIES:
+:CUSTOM_ID: system-prompt
:ID: 89bc60f0-89d4-4e10-ae80-8f824f2e3c55
:END:
*** User Prompt
:PROPERTIES:
+:CUSTOM_ID: user-prompt
:ID: 009e5410-f852-4faa-b81a-f9c98b056ae3
:END:
results - the AI can only work with what you provide.
*** Model Library
+:PROPERTIES:
+:CUSTOM_ID: model-library
+:ID: 5ec0618d-90bb-4b7e-913a-477da45f406f
+:END:
The /model library/ is the internal registry of all available AI
models configured in the system. It's constructed during startup
validation so your tasks can reference models by simple aliases.
*** GGUF Format
+:PROPERTIES:
+:CUSTOM_ID: gguf-format
+:ID: 7d8ae98c-5d47-422e-8d18-00496774cfd9
+:END:
/GGUF/ is the binary model format used by llama.cpp for AI inference.
*** llama.cpp
:PROPERTIES:
+:CUSTOM_ID: llama-cpp
:ID: 01b0d389-75d4-420f-8d5c-cae29900301f
:END:
system.
** Important files and directories
+:PROPERTIES:
+:CUSTOM_ID: important-files-directories
+:ID: ca3dce55-566b-427f-b6d1-630fcd245f76
+:END:
*** Configuration File
:PROPERTIES:
+:CUSTOM_ID: configuration-file
:ID: fd687508-0a76-4fee-9a1c-4031cb403c60
:END:
flexible hierarchy for managing complex workflows.
*** Skill Directory
+:PROPERTIES:
+:CUSTOM_ID: skill-directory
+:ID: c62dcf42-5017-4ba4-9e4d-af5517d4f968
+:END:
The /skill directory/ (configured via =skills_directory=) is the
filesystem location where YAML files defining AI behaviors are stored.
requiring Alyverkko CLI restart.
*** Task Directory
+:PROPERTIES:
+:CUSTOM_ID: task-directory
+:ID: c59976d5-cb22-44b2-a86c-7b39c40cedee
+:END:
The /task directory/ is the designated filesystem location where users
place task files for processing, configured via =tasks_directory= in
communication is actually /more/ productive.
** Generation parameters
+:PROPERTIES:
+:CUSTOM_ID: generation-parameters
+:ID: 5a253c87-2ae4-4335-a1a2-c9e07a208f7f
+:END:
*** Temperature
:PROPERTIES:
+:CUSTOM_ID: temperature
:ID: 24a0a54b-828b-4c78-8208-504390848fbc
:END:
*** Top-p (Nucleus Sampling)
:PROPERTIES:
+:CUSTOM_ID: top-p
:ID: 047f5bf7-e964-49ac-a666-c7ac75754e54
:END:
*** Repeat Penalty
:PROPERTIES:
+:CUSTOM_ID: repeat-penalty
:ID: 728f6daf-7a75-4e09-832b-5d449f0b4cae
:END:
*** Top-k
:PROPERTIES:
+:CUSTOM_ID: top-k
:ID: 2c8eb415-509c-4269-8a65-48b2e6662290
:END:
*** Min-p
:PROPERTIES:
+:CUSTOM_ID: min-p
:ID: 9ee338ae-bb79-4c3e-b262-928fe237cb17
:END:
maximum response reliability.
*** Thread Count
+:PROPERTIES:
+:CUSTOM_ID: thread-count
+:ID: 727a72a7-44d1-419d-8c3b-5b2fe224b933
+:END:
/Thread count/ specifies the number of CPU threads dedicated to the
core AI inference process (configured via =thread_count= in
gets fully utilized already very fast with just few threads.
*** Batch Thread Count
+:PROPERTIES:
+:CUSTOM_ID: batch-thread-count
+:ID: 133c0c3b-f1f0-454c-a159-cca5266d728a
+:END:
/Batch thread count/ specifies threads used for prompt preprocessing
(configured via =batch_thread_count=). This parameter affects how
help up to your CPU's logical core count.
*** Context Size Tokens
+:PROPERTIES:
+:CUSTOM_ID: context-size-tokens
+:ID: d6b978f2-1db0-4b14-93f3-30ed63d97e59
+:END:
/Context size tokens/ defines the maximum number of tokens
(word-pieces) a model can process, configured per-model via
it causes unpredictable or significantly degraded model output.
*** Timeout
+:PROPERTIES:
+:CUSTOM_ID: timeout
+:ID: 85ab8c92-9439-44bc-b8d5-cb64f1fd9270
+:END:
/Timeout/ is a parameter that specifies the maximum time (in
milliseconds) that the AI is allowed to run for a task. If this time
*** Parameter Precedence Hierarchy
:PROPERTIES:
+:CUSTOM_ID: parameter-precedence
:ID: 456dd42e-a474-4464-a14e-384c68713537
:END:
override broader ones.
** AI response post processing parameters
+:PROPERTIES:
+:CUSTOM_ID: response-post-processing
+:ID: 2096333d-1b08-40fb-b04a-94b739cb1c4b
+:END:
*** Final answer indicator
:PROPERTIES:
+:CUSTOM_ID: final-answer-indicator
:ID: 8da2522b-d34f-4632-9fdd-603c1a64febb
:END:
#+end_example
*** End of text marker
+:PROPERTIES:
+:CUSTOM_ID: end-of-text-marker
+:ID: a554f321-b387-4666-b7c4-768f0e7a4e96
+:END:
An /end of text marker/ is an optional string (e.g., "###", ”[end of
text]“) specified per-model that signals the AI has completed its
end of AI response.
* Installation
+:PROPERTIES:
+:CUSTOM_ID: installation
+:ID: d5ca2e96-2063-4236-94c8-d962d13ea484
+:END:
When you first encounter Älyverkko CLI, the setup process might seem
involved compared to cloud-based AI services. That's completely
assistant that works /for you/.
** Requirements
+:PROPERTIES:
+:CUSTOM_ID: requirements
+:ID: c0e57874-93c1-40fe-a129-e43e7e6dc810
+:END:
*Operating System:*
Älyverkko CLI is developed and tested on Debian 13 "Bookworm". It
input/output data.
** Your Setup Journey - What to Expect
+:PROPERTIES:
+:CUSTOM_ID: setup-journey
+:ID: 6a2f2fa2-6d40-46c6-bf45-69a261bd69ea
+:END:
Before we start actual setup, here's brief overview of what you'll be
doing:
** Installation
:PROPERTIES:
+:CUSTOM_ID: installation-steps
:ID: 0b705a37-9b84-4cd5-878a-fedc9ab09b12
:END:
** Alyverkko CLI daemon configuration
:PROPERTIES:
+:CUSTOM_ID: daemon-configuration
:ID: 0fcdae48-81c5-4ae1-bdb9-64ae74e87c45
:END:
Älyverkko CLI application configuration is done by editing YAML
: ~/.config/alyverkko-cli.yaml
*** Key Parameters Explained
+:PROPERTIES:
+:CUSTOM_ID: key-parameters
+:ID: bd4c9716-5f04-4e94-b713-ad0ff078f8a6
+:END:
**** Core Directories
+:PROPERTIES:
+:CUSTOM_ID: core-directories
+:ID: 75c76c6b-637c-42a7-a31a-6f6be9570747
+:END:
- =tasks_directory=: Where task files are placed for processing.
- =llama_cli_path=: Path to llama.cpp's *llama-completion* executable.
**** Generation Parameters
+:PROPERTIES:
+:CUSTOM_ID: generation-parameters-config
+:ID: 40ec81ad-cea1-4d41-a007-1f334f9b4117
+:END:
- =default_temperature=: (Optional) Creativity control (0-3, higher =
more creative).
1.0 = no penalty)
**** Performance Tuning
+:PROPERTIES:
+:CUSTOM_ID: performance-tuning
+:ID: 18acf2d6-7118-4f18-9a86-ac48a33c761b
+:END:
- =thread_count=: Sets the number of threads to be used by the AI
during response generation. RAM data transfer speed is usually
**** Model-Specific Settings
:PROPERTIES:
+:CUSTOM_ID: model-specific-settings
:ID: 4206c6e9-d116-4030-94a4-87bf6f82043f
:END:
answer indicator]].
*** Configuration file example
+:PROPERTIES:
+:CUSTOM_ID: configuration-example
+:ID: 0c577b27-df6f-429f-9ac2-f89105a73544
+:END:
The application is configured using a YAML-formatted configuration
file. Below is an example of how the configuration file might look:
#+end_src
*** Enlisting available models
+:PROPERTIES:
+:CUSTOM_ID: enlisting-models
+:ID: 47bb1776-2824-425f-a13b-c0a863c48c23
+:END:
Once Älyverkko CLI is installed and properly configured, you can run
following command at commandline to see what models are available to
it:
manually remove this suffix after fixing the model file path.
*** Self test
+:PROPERTIES:
+:CUSTOM_ID: self-test
+:ID: 781917b5-3d46-4368-9473-8a379088c91f
+:END:
The *selftest* command performs a series of checks to ensure the
system is configured correctly:
- The presence of the *llama.cpp* executable.
** Skill concept and configuration
+:PROPERTIES:
+:CUSTOM_ID: skill-configuration
+:ID: 1592510e-5a8c-4cf3-9851-4a96fae6434e
+:END:
+ See also: [[id:6579abb4-8386-418b-9457-cae6c3345dfb][Skill]] concept explanation.
*** Skill File Format
:PROPERTIES:
+:CUSTOM_ID: skill-file-format
:ID: 47fd0e4e-f86e-4c94-9656-ed76c0f9c2c5
:END:
*** Example Skill File
:PROPERTIES:
+:CUSTOM_ID: example-skill-file
:ID: 4fae9009-eb2e-4d3f-a43e-68ec8c43a7cd
:END:
: writer.yaml
See more example skills: [[https://www3.svjatoslav.eu/projects/alyverkko-cli/examples/skills/default.yaml][default.yaml]], [[https://www3.svjatoslav.eu/projects/alyverkko-cli/examples/skills/summary.yaml][summary.yaml]]
** Starting process daemon
+:PROPERTIES:
+:CUSTOM_ID: starting-daemon
+:ID: ab98110c-4603-40b1-a69c-72ddc13524e6
+:END:
Älyverkko CLI keeps continuously listening for and processing tasks
from a specified mail directory.
processing mode:
**** Start via command line interface
+:PROPERTIES:
+:CUSTOM_ID: start-cli
+:ID: bea72be2-994b-479c-bd7d-56300518fb5a
+:END:
1. Open your terminal.
close terminal window.
**** Start using your desktop environment application launcher
+:PROPERTIES:
+:CUSTOM_ID: start-desktop
+:ID: eeb89786-3b9a-4e3d-9699-75fd4119ccee
+:END:
1. Access the application launcher or application menu on your desktop
environment.
4. If you want to stop Älyverkko CLI, just close terminal window.
**** Start in the background as systemd system service
+:PROPERTIES:
+:CUSTOM_ID: start-systemd
+:ID: 85a79517-dd51-466b-8b12-3907c133e75a
+:END:
During Älyverkko CLI [[id:0b705a37-9b84-4cd5-878a-fedc9ab09b12][installation]], installation script will prompt you
if you want to install *systemd* service. If you chose *Y*, Alyverkko
: sudo systemctl disable alyverkko-cli
** The Light at the End of the Tunnel
+:PROPERTIES:
+:CUSTOM_ID: setup-benefits
+:ID: b96f4505-85a9-4347-8f50-5a83fcbd1bfb
+:END:
After setup, here's what you get:
need serious AI power without compromise.
* Usage
+:PROPERTIES:
+:CUSTOM_ID: usage
+:ID: ca6003eb-d478-47e9-b05e-21e3c668595b
+:END:
** Task file format
:PROPERTIES:
+:CUSTOM_ID: task-file-format
:ID: f2fd232d-5d2b-42fa-83db-76ab015d4df9
:END:
#+end_example
*** Task File Header Format
+:PROPERTIES:
+:CUSTOM_ID: task-file-header
+:ID: 8609b337-83c6-425b-8f6c-d4bda22ebf1e
+:END:
The first line *must* begin with exactly =TOCOMPUTE:= followed by
space-separated key-value pairs:
- =priority=[integer]=: Higher integers mean higher priority (default: 0)
*** Processed File Format
+:PROPERTIES:
+:CUSTOM_ID: processed-file-format
+:ID: f9c19a99-b610-46ea-9d2e-902288995048
+:END:
After AI processing completes, a new file is created with:
1. First line: =DONE: skill=[name] model=[alias] duration=[time]=
** Task preparation
:PROPERTIES:
+:CUSTOM_ID: task-preparation
:ID: 4b7900e4-77c1-45e7-9c54-772d0d3892ea
:END:
*** "joinfiles" command
:PROPERTIES:
+:CUSTOM_ID: joinfiles-command
:ID: be907a1f-e347-48d9-ab0c-6a556912dc49
:END:
*Note:* See also alternative solution with similar goal: [[https://github.com/aerugo/prelude][prelude]].
directories or collections of text documents.
**** Usage
+:PROPERTIES:
+:CUSTOM_ID: joinfiles-usage
+:ID: d734875d-e26a-4915-9d2d-4b047f4a4d0e
+:END:
To use the *joinfiles* command, specify the source directory
containing the files you wish to join and a topic name that will be
#+end_example
**** Options
+:PROPERTIES:
+:CUSTOM_ID: joinfiles-options
+:ID: be737651-2f1b-4197-a6ad-40ffb8bc8df5
+:END:
- **-s, --src-dir**: Specifies the source directory from which to join
files.
operation is complete.
**** Example Use Case
+:PROPERTIES:
+:CUSTOM_ID: joinfiles-example
+:ID: c2e1066c-7f1b-4007-a22c-69c40a772c48
+:END:
Imagine you have a software project with various source files that you
want to analyze using AI. You can use the *joinfiles* command to
** Initiate AI processing
:PROPERTIES:
+:CUSTOM_ID: initiate-processing
:ID: 883d6e7c-60e0-422b-8c00-5cdc9dfec20d
:END:
** Helpful GNU Emacs utilities
:PROPERTIES:
+:CUSTOM_ID: emacs-utilities
:ID: 25038854-c905-4b26-9670-cca06600223e
:END:
GNU Emacs users.
*** Easily compose new problem statement for AI from emacs
+:PROPERTIES:
+:CUSTOM_ID: emacs-compose-task
+:ID: 4ae50ef2-104d-43f7-a002-ba007d99693f
+:END:
The Elisp function *ai-new-topic* facilitates the creation and opening
of a new Org-mode file dedicated to a user-defined topic within a
#+end_src
*** Easily signal to AI that problem statement is ready for solving
+:PROPERTIES:
+:CUSTOM_ID: emacs-signal-ready
+:ID: 6ef58180-ece3-4649-927a-e8465c9b4083
+:END:
The function =alyverkko-compute= is designed to enhance the workflow
of users working with the Älyverkko CLI application by automating the
#+end_src
* Getting the source code
+:PROPERTIES:
+:CUSTOM_ID: getting-source-code
+:ID: f67a3668-eca2-425f-b284-915ff3b3ed82
+:END:
- This program is free software: released under Creative Commons Zero
(CC0) license.
** Source code
:PROPERTIES:
+:CUSTOM_ID: source-code
:ID: f5740953-079b-40f4-87d8-b6d1635a8d39
:END:
- [[https://www2.svjatoslav.eu/gitweb/?p=alyverkko-cli.git;a=snapshot;h=HEAD;sf=tgz][Download latest snapshot in TAR GZ format]]
understand diagrams. Diagrams were generated using [[https://www3.svjatoslav.eu/projects/javainspect/][JavaInspect tool]].
* Feature ideas
+:PROPERTIES:
+:CUSTOM_ID: feature-ideas
+:ID: 0f597bc9-0a23-40e2-b60d-80d2394a3f85
+:END:
- Recommend some concrete AI models.