From: Svjatoslav Agejenko Date: Sat, 16 May 2026 23:27:00 +0000 (+0300) Subject: Initial commit X-Git-Url: http://www2.svjatoslav.eu/gitweb/?a=commitdiff_plain;h=refs%2Fheads%2Fmaster;p=alyverkko-cli.git Initial commit --- 52859da52ee57624c7231d2352c42046643584aa diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..80a1eb3 --- /dev/null +++ b/.gitignore @@ -0,0 +1,14 @@ +/.idea/ +/.settings/ +/target/ +/*.iml +/*.log +/test/ + +/doc/apidocs/ +/doc/graphs/ + +/.classpath +/.factorypath +/.project +*.html diff --git a/AGENTS.org b/AGENTS.org new file mode 100644 index 0000000..e94c8e3 --- /dev/null +++ b/AGENTS.org @@ -0,0 +1,95 @@ +A Java CLI tool for batch processing with large language models using +CPU-based computation via llama.cpp. Designed for "slow but smart" AI +tasks where batch processing is more valuable than real-time +interaction. + +* Commands + +- *wizard* :: Interactive configuration wizard. Validates and fixes + configuration files, discovers new models in the models directory, + and updates configuration. Run this first to set up or update your + setup. + +- *process* :: Continuous task processor. Monitors the tasks directory + for files starting with =TOCOMPUTE:=, processes them using the + configured AI model, and appends results to the same file. + +- *listmodels* :: Lists all available models defined in the configuration. + +- *joinfiles* :: Utility to concatenate multiple files together (used + for preparing task files with context). + +- *addtaskheader* :: Adds a TOCOMPUTE header to task files. + +* Task File Format + +Task files are plain text files placed in the tasks directory. Format: + +#+BEGIN_SRC text +TOCOMPUTE: skill= model= priority= + + +#+END_SRC + +- First line must start with =TOCOMPUTE:= +- Parameters are optional: =skill= (defaults to "default"), =model= + (defaults per-skill or "default"), =priority= (defaults to 0) +- Files are processed in priority order (higher priority = processed first) +- After processing, the file =TOCOMPUTE:= heading in the file is renamed + to =DONE:= and contains the AI response + +* Configuration + +Configuration is stored in YAML format at +=~/.config/alyverkko-cli/configuration.yaml=. Key settings: + +- =tasks_directory= :: Where task files are placed for processing +- =models_directory= :: Directory containing GGUF model files +- =skills_directory= :: Directory containing skill (prompt) YAML files +- =llama_cli_path= :: Path to the llama-cli executable +- =thread_count=, =batch_thread_count= :: CPU thread settings + +** Skills (Prompts) + +Skills are YAML files in the skills directory (e.g., =writer.yaml=): + +#+BEGIN_SRC yaml +prompt: | + You are a helpful assistant... +temperature: 0.7 +top_p: 0.9 +model_alias: "mistral" +timeout_millis: 300000 +#+END_SRC + +Model selection hierarchy (highest priority first): + +1. Explicit =model= parameter in TOCOMPUTE line +2. =model_alias= in skill configuration +3. Model named "default" in configuration + +** Models + +Models are defined in the configuration with aliases, filesystem paths, +and optional settings: + +#+BEGIN_SRC yaml +models: + - alias: "mistral" + filesystem_path: "mistral-7b-instruct-v0.2.Q4_K_M.gguf" + context_size: 8192 + final_answer_indicator: "" +#+END_SRC + +The =final_answer_indicator= splits the AI response into "internal +thoughts" and "final answer" sections in the output file. + +* Source Structure + +- =src/main/java/eu/svjatoslav/alyverkko_cli/= :: Main package + - =Main.java= :: Entry point, command dispatcher + - =Command.java= :: Interface for all commands + - =commands/= :: Command implementations + - =task_processor/= :: Core batch processing logic + - =configuration/= :: Configuration model classes (Configuration, Model, SkillConfig) + - =Utils.java= :: Utility functions diff --git a/COPYING b/COPYING new file mode 100644 index 0000000..0e259d4 --- /dev/null +++ b/COPYING @@ -0,0 +1,121 @@ +Creative Commons Legal Code + +CC0 1.0 Universal + + CREATIVE COMMONS CORPORATION IS NOT A LAW FIRM AND DOES NOT PROVIDE + LEGAL SERVICES. DISTRIBUTION OF THIS DOCUMENT DOES NOT CREATE AN + ATTORNEY-CLIENT RELATIONSHIP. CREATIVE COMMONS PROVIDES THIS + INFORMATION ON AN "AS-IS" BASIS. CREATIVE COMMONS MAKES NO WARRANTIES + REGARDING THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS + PROVIDED HEREUNDER, AND DISCLAIMS LIABILITY FOR DAMAGES RESULTING FROM + THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS PROVIDED + HEREUNDER. + +Statement of Purpose + +The laws of most jurisdictions throughout the world automatically confer +exclusive Copyright and Related Rights (defined below) upon the creator +and subsequent owner(s) (each and all, an "owner") of an original work of +authorship and/or a database (each, a "Work"). + +Certain owners wish to permanently relinquish those rights to a Work for +the purpose of contributing to a commons of creative, cultural and +scientific works ("Commons") that the public can reliably and without fear +of later claims of infringement build upon, modify, incorporate in other +works, reuse and redistribute as freely as possible in any form whatsoever +and for any purposes, including without limitation commercial purposes. +These owners may contribute to the Commons to promote the ideal of a free +culture and the further production of creative, cultural and scientific +works, or to gain reputation or greater distribution for their Work in +part through the use and efforts of others. + +For these and/or other purposes and motivations, and without any +expectation of additional consideration or compensation, the person +associating CC0 with a Work (the "Affirmer"), to the extent that he or she +is an owner of Copyright and Related Rights in the Work, voluntarily +elects to apply CC0 to the Work and publicly distribute the Work under its +terms, with knowledge of his or her Copyright and Related Rights in the +Work and the meaning and intended legal effect of CC0 on those rights. + +1. Copyright and Related Rights. A Work made available under CC0 may be +protected by copyright and related or neighboring rights ("Copyright and +Related Rights"). Copyright and Related Rights include, but are not +limited to, the following: + + i. the right to reproduce, adapt, distribute, perform, display, + communicate, and translate a Work; + ii. moral rights retained by the original author(s) and/or performer(s); +iii. publicity and privacy rights pertaining to a person's image or + likeness depicted in a Work; + iv. rights protecting against unfair competition in regards to a Work, + subject to the limitations in paragraph 4(a), below; + v. rights protecting the extraction, dissemination, use and reuse of data + in a Work; + vi. database rights (such as those arising under Directive 96/9/EC of the + European Parliament and of the Council of 11 March 1996 on the legal + protection of databases, and under any national implementation + thereof, including any amended or successor version of such + directive); and +vii. other similar, equivalent or corresponding rights throughout the + world based on applicable law or treaty, and any national + implementations thereof. + +2. Waiver. To the greatest extent permitted by, but not in contravention +of, applicable law, Affirmer hereby overtly, fully, permanently, +irrevocably and unconditionally waives, abandons, and surrenders all of +Affirmer's Copyright and Related Rights and associated claims and causes +of action, whether now known or unknown (including existing as well as +future claims and causes of action), in the Work (i) in all territories +worldwide, (ii) for the maximum duration provided by applicable law or +treaty (including future time extensions), (iii) in any current or future +medium and for any number of copies, and (iv) for any purpose whatsoever, +including without limitation commercial, advertising or promotional +purposes (the "Waiver"). Affirmer makes the Waiver for the benefit of each +member of the public at large and to the detriment of Affirmer's heirs and +successors, fully intending that such Waiver shall not be subject to +revocation, rescission, cancellation, termination, or any other legal or +equitable action to disrupt the quiet enjoyment of the Work by the public +as contemplated by Affirmer's express Statement of Purpose. + +3. Public License Fallback. Should any part of the Waiver for any reason +be judged legally invalid or ineffective under applicable law, then the +Waiver shall be preserved to the maximum extent permitted taking into +account Affirmer's express Statement of Purpose. In addition, to the +extent the Waiver is so judged Affirmer hereby grants to each affected +person a royalty-free, non transferable, non sublicensable, non exclusive, +irrevocable and unconditional license to exercise Affirmer's Copyright and +Related Rights in the Work (i) in all territories worldwide, (ii) for the +maximum duration provided by applicable law or treaty (including future +time extensions), (iii) in any current or future medium and for any number +of copies, and (iv) for any purpose whatsoever, including without +limitation commercial, advertising or promotional purposes (the +"License"). The License shall be deemed effective as of the date CC0 was +applied by Affirmer to the Work. Should any part of the License for any +reason be judged legally invalid or ineffective under applicable law, such +partial invalidity or ineffectiveness shall not invalidate the remainder +of the License, and in such case Affirmer hereby affirms that he or she +will not (i) exercise any of his or her remaining Copyright and Related +Rights in the Work or (ii) assert any associated claims and causes of +action with respect to the Work, in either case contrary to Affirmer's +express Statement of Purpose. + +4. Limitations and Disclaimers. + + a. No trademark or patent rights held by Affirmer are waived, abandoned, + surrendered, licensed or otherwise affected by this document. + b. Affirmer offers the Work as-is and makes no representations or + warranties of any kind concerning the Work, express, implied, + statutory or otherwise, including without limitation warranties of + title, merchantability, fitness for a particular purpose, non + infringement, or the absence of latent or other defects, accuracy, or + the present or absence of errors, whether or not discoverable, all to + the greatest extent permissible under applicable law. + c. Affirmer disclaims responsibility for clearing rights of other persons + that may apply to the Work or any use thereof, including without + limitation any person's Copyright and Related Rights in the Work. + Further, Affirmer disclaims responsibility for obtaining any necessary + consents, permissions or other rights required for any use of the + Work. + d. Affirmer understands and acknowledges that Creative Commons is not a + party to this document and has no duty or obligation with respect to + this CC0 or use of the Work. diff --git a/alyverkko-cli b/alyverkko-cli new file mode 100755 index 0000000..aea6d86 --- /dev/null +++ b/alyverkko-cli @@ -0,0 +1,6 @@ +#!/bin/bash + +set -f + +java -Xmx4500m -classpath /opt/alyverkko-cli/* eu.svjatoslav.alyverkko_cli.Main "$@" + diff --git a/doc/examples/alyverkko-cli.yaml b/doc/examples/alyverkko-cli.yaml new file mode 100644 index 0000000..cdae0b2 --- /dev/null +++ b/doc/examples/alyverkko-cli.yaml @@ -0,0 +1,33 @@ +tasks_directory: "/home/john/AI/tasks" +models_directory: "/home/john/AI/models" +skills_directory: "/home/john/.config/alyverkko-cli/skills" +llama_cli_path: "/home/john/AI/llama.cpp/build/bin/llama-completion" + +default_temperature: 0.7 + +batch_thread_count: 10 +thread_count: 6 + +models: + + - alias: "qwen3-next-80b-A3B-thinking" + filesystem_path: "Qwen3-Next-80B-A3B-Thinking-UD-Q4_K_XL.gguf" + context_size_tokens: 131072 + temperature: 0.6 + top_p: 0.95 + top_k: 20 + min_p: 0 + + - alias: "default" + filesystem_path: "Alibaba-NLP_Tongyi-DeepResearch-30B-A3B-Q8_0.gguf" + context_size_tokens: 131072 + temperature: 0.85 + repeat_penalty: 1.1 + top_p: 0.95 + + - alias: "tongyi" + filesystem_path: "Alibaba-NLP_Tongyi-DeepResearch-30B-A3B-Q8_0.gguf" + context_size_tokens: 131072 + temperature: 0.85 + repeat_penalty: 1.1 + top_p: 0.95 diff --git a/doc/examples/skills/default.yaml b/doc/examples/skills/default.yaml new file mode 100644 index 0000000..452b7f7 --- /dev/null +++ b/doc/examples/skills/default.yaml @@ -0,0 +1,12 @@ +prompt: | + <|im_start|>system + User will provide you with task that needs to be solved along with + existing relevant information. You must provide well reasoned solution. + <|im_end|> + <|im_start|>user + /think Solve following problem: + + + + <|im_end|> + <|im_start|>assistant diff --git a/doc/examples/skills/summary.yaml b/doc/examples/skills/summary.yaml new file mode 100644 index 0000000..6530a26 --- /dev/null +++ b/doc/examples/skills/summary.yaml @@ -0,0 +1,18 @@ +prompt: | + <|im_start|>system + + User is curious about the world and wants to understand how it + works. User is finding various news, blogs and stories on the internet + that look promising but user does not have time to read them in depth. + Your task is to analyze user provided story and write the summary that + preserves the most valuable events, facts, conclusions or take-aways + from it. + + <|im_end|> + <|im_start|>user + /think Summarize following: + + + + <|im_end|> + <|im_start|>assistant diff --git a/doc/examples/tasks/DONE: personality test.org b/doc/examples/tasks/DONE: personality test.org new file mode 100644 index 0000000..03981df --- /dev/null +++ b/doc/examples/tasks/DONE: personality test.org @@ -0,0 +1,42 @@ +DONE: skill=default model=default duration=2m +* USER: + +If numbers had personalities, +what kind of personality might the number 'zero' have? + +* ASSISTANT: + +1. Personality traits based on mathematical properties: + - Neutral and unchanging unless influenced + - Control over magnitude through multiplication + - Neither positive/negative (impartial) + - Foundational yet seemingly insignificant at times + +2. Symbolism and metaphors: + - Balance point between opposites + - Container for infinite possibilities (potential) + - Cycle of beginning/endpoints + - Quiet supporter behind the scenes + +3. Social dynamics: + - Values community/cooperation + - May feel undervalued but persists regardless + - Adaptable depending on situation/placement + +4. Philosophical/spiritual parallels: + - Emptiness equates to purity/clean slate + - Potential without action = dormant power + +5. Gender and self-identity: + - Ambiguous/genderless + +6. Emotional characteristics: + - Patient; allows time for events/actions to unfold + - Resilient when facing neglect or misunderstanding + - Persistent in fulfilling its purpose + +7. Behavioral tendencies: + - Fosters growth indirectly rather than directly contributing + measurable output + - Encourages exploration beyond limits via abstract reasoning + innovations diff --git a/doc/examples/tasks/personality test.org b/doc/examples/tasks/personality test.org new file mode 100644 index 0000000..bacf84c --- /dev/null +++ b/doc/examples/tasks/personality test.org @@ -0,0 +1,4 @@ +TOCOMPUTE: skill=default model=default priority=5 + +If numbers had personalities, +what kind of personality might the number 'zero' have? diff --git a/doc/index.org b/doc/index.org new file mode 100644 index 0000000..80cdd28 --- /dev/null +++ b/doc/index.org @@ -0,0 +1,1798 @@ +#+SETUPFILE: ~/.emacs.d/org-styles/html/darksun.theme +#+TITLE: Älyverkko CLI application +#+LANGUAGE: en +#+LATEX_HEADER: \usepackage[margin=1.0in]{geometry} +#+LATEX_HEADER: \usepackage{parskip} +#+LATEX_HEADER: \usepackage[none]{hyphenat} + +#+OPTIONS: H:20 num:20 +#+OPTIONS: author:nil + +* Introduction +:PROPERTIES: +:CUSTOM_ID: introduction +:ID: b4da28df-7208-4c27-a1ea-98d8692c5012 +:END: + +The *Älyverkko CLI* application is a user-friendly tool developed in +Java, specifically tailored to streamline the utilization of expansive +language models through CPU-based computation in batch processing +mode. + +To illustrate its capabilities: Imagine harnessing the power of a vast +language model, boasting approximately 100 billion parameters, solely +relying on CPU computations and leveraging the open-source software +*llama.cpp*. This setup requires a modern consumer-grade CPU and +approximately 128 GB of RAM. To put this into perspective, 128 GB of +RAM is financially comparable to purchasing a high-quality smartphone, +making it an economical option for many users. + +In contrast, executing the same model on an Nvidia GPU could result in +costs that are significantly higher, potentially at least by an order +of magnitude. + +However, there is a trade-off: CPU-based processing for such extensive +models is inherently slow. This means real-time interaction, like +chatting with the AI, wouldn't be practical or enjoyable due to the +lag in response times. Nevertheless, when deployed in a +non-interactive batch mode, this "slow but smart" AI can complete +numerous valuable tasks within a 24-hour window. For instance, it +could generate a substantial amount of code, potentially exceeding +what you could thoroughly review in the same timeframe. Additionally, +it could process more documents than most individuals would be +inclined to read manually. + +The primary objective of the *Älyverkko CLI* project is to identify +and enable applications where this "slow but smart" AI can excel. By +utilizing *llama.cpp* as its inference engine, the project aims to +unlock a variety of uses where batch processing is more advantageous +than real-time interaction. + +Here are some practical examples of tasks suited for the +*Älyverkko CLI* application: + +- [[id:e1b6aa9a-a27d-4019-8bd4-fceb0c606e23][Domain: Natural Language Processing (NLP)]] +- [[id:f4c7d3e0-8c18-4123-a797-871ca73a7580][Domain: Code Generation]] +- [[id:f38360ad-54f6-4f24-b299-f73a9faacabd][Domain: Content Creation]] + +In summary, the *Älyverkko CLI* application opens up a realm of +possibilities for leveraging powerful AI in scenarios where immediate +responses are not critical, but high-quality batch processing output +is highly beneficial. + +Note: project is still in early stage. + +** Use cases +:PROPERTIES: +:CUSTOM_ID: use-cases +:ID: 9aeba681-2d9d-442f-bc4b-3d61a294e596 +:END: +*** Domain: Natural Language Processing (NLP) +:PROPERTIES: +:CUSTOM_ID: nlp-domain +:ID: e1b6aa9a-a27d-4019-8bd4-fceb0c606e23 +:END: + +*Problem Statement:* + +Analyze a set of customer reviews to determine overall sentiment and +extract key features that customers appreciate or dislike about a +product. + +*Usage Procedure:* +1. User collects customer reviews in plain text format within the + configured tasks directory. Lets say, about 150 kilobytes of + reviews per input file (this is dictated by AI model available + context size). +2. Each review file starts with special "TOCOMPUTE:". (See: [[id:f2fd232d-5d2b-42fa-83db-76ab015d4df9][Task file + format]]) +3. The Älyverkko CLI application processes these files, generating + sentiment analysis results and feature extraction insights. +4. Results are appended to the original files in org-mode syntax, + indicating AI responses. + +*** Domain: Code Generation +:PROPERTIES: +:CUSTOM_ID: code-generation-domain +:ID: f4c7d3e0-8c18-4123-a797-871ca73a7580 +:END: + +*Problem Statement:* + +Generate code snippets for a new software module based on detailed +specifications provided by the developer. + +*Usage Procedure:* + +1. Developer writes specifications in a text file within the tasks + directory. Text file also contains relevant parts of the program + source code and documentation. Älyverkko CLI [[id:be907a1f-e347-48d9-ab0c-6a556912dc49]["joinfiles" command]] + can be used to facilitate such text file preparation. + +2. The Älyverkko CLI application processes this file and generates the + corresponding code snippets. The generated code is appended to the + original specifications file, organized using org-mode syntax. + +3. Developer can review proposed changes and then integrate them back + into original program source code. + +Note: Large part of the [[https://www3.svjatoslav.eu/projects/alyverkko-cli/][Älyverkko CLI]] program code is written in such +a way by AI. + +*** Domain: Content Creation +:PROPERTIES: +:CUSTOM_ID: content-creation-domain +:ID: f38360ad-54f6-4f24-b299-f73a9faacabd +:END: + +*Problem Statement:* + +Draft an outline for a book on science fiction or improve its plot. + +*Usage Procedure:* + +1. The book author writes a brief describing the outline of the plot + and his book main idea for the novel. + +2. *Älyverkko CLI* processes this description and generates more + detailed outline with suggested headings and suggests possible + improvements to the plot. + +Here is [[https://www.svjatoslav.eu/writing/Whispers%20in%20the%20Stream%20of%20Time.html][example sci-fi book]] that was written with the help of +*Älyverkko CLI*. + +** Why Bother With This Setup? (The Big Picture) +:PROPERTIES: +:CUSTOM_ID: why-this-setup +:ID: 5862520c-db92-4a5e-9177-717e8dea66f8 +:END: + +Before diving into steps, let's address the elephant in the room: *Why +go through this setup when ChatGPT is just a click away?* + +Because Älyverkko CLI solves a fundamentally different problem: + +- ✨ *Privacy by design*: All processing happens on your machine - no + data ever leaves your computer +- 💰 *Cost efficiency*: Run 70B+ parameter models without paying per + token (128GB RAM ≈ cost of a smartphone) +- ⚙️ *Full control*: Tweak every parameter to match your specific needs +- 📦 *Offline capability*: Works without internet connection +- 🕒 *Batch processing*: Perfect for "set it and forget it" workflows + while you sleep + +This isn't designed for real-time chatting (CPU inference is slow), +but for substantial tasks where quality matters more than speed: code +generation, document analysis, content creation, etc. + +* Concept glossary +:PROPERTIES: +:CUSTOM_ID: concept-glossary +:ID: 3bff0680-db41-4f97-a2af-f677209dd430 +:END: +** General concepts +:PROPERTIES: +:CUSTOM_ID: general-concepts +:ID: 2ae2e3ec-835f-45e1-bc15-618c8b2c3e85 +:END: +*** Task +:PROPERTIES: +:CUSTOM_ID: task-concept +:ID: 140c53cb-8032-4a04-83ed-d1818b1cfc52 +:END: + +A /task/ represents a single unit of work submitted to the Älyverkko +CLI system for AI processing. + +Task logically consists of two core components: +- a [[id:89bc60f0-89d4-4e10-ae80-8f824f2e3c55][system prompt]] (defining the AI's role/behavior) and +- a [[id:009e5410-f852-4faa-b81a-f9c98b056ae3][user prompt]] (the specific request or question). + +Tasks are implemented as plain text files that begin with a +[[id:cd4b622a-6b74-4fac-85fe-f5b056367824]["TOCOMPUTE" header]] line specifying processing parameters. When +processed, the system appends the AI's response in structured format +and renames the file with a =DONE:= prefix. Tasks represent the +fundamental interaction pattern between users and the system - you +create a task file, Älyverkko CLI processes it while you work on other +things, and later you receive the completed response. The asynchronous +nature makes this ideal for CPU-based batch processing where responses +may take minutes to hours. + +*** Skill +:PROPERTIES: +:CUSTOM_ID: skill-concept +:ID: 6579abb4-8386-418b-9457-cae6c3345dfb +:END: +- See also: + - [[id:47fd0e4e-f86e-4c94-9656-ed76c0f9c2c5][Skill File Format]] + - [[id:4fae9009-eb2e-4d3f-a43e-68ec8c43a7cd][Example Skill File]] + +A /skill/ is a predefined behavioral configuration for the AI, +implemented as a YAML file in the skills directory. Each skill +defines: + +- A [[id:89bc60f0-89d4-4e10-ae80-8f824f2e3c55][system prompt]] that establishes the AI's role and behavior. + - System prompt will have special marker == where + specific user task will be injected. + +- Optional generation parameters ([[id:24a0a54b-828b-4c78-8208-504390848fbc][temperature]], [[id:047f5bf7-e964-49ac-a666-c7ac75754e54][top-p]] , etc.) + +- Optional [[id:074af969-8a9b-407d-a605-6725fe4e8580][Model]] alias that will be selected when this skill gets + invoked. + +Skills function as specialized "personas" for different task +types. For example, a =summary.yaml= skill might contain instructions +for concise text summarization, while a =writer.yaml= skill could +optimize for creative prose. The power of skills lies in their +reusability - once defined, you can apply the same behavioral +configuration across countless tasks by simply referencing the skill +name in your task [[id:cd4b622a-6b74-4fac-85fe-f5b056367824][TOCOMPUTE:]] header. Skills abstract away repetitive +instructions, letting you focus on the actual content of your request +rather than constantly redefining how the AI should behave. + +*** Model +:PROPERTIES: +:CUSTOM_ID: model-concept +:ID: 074af969-8a9b-407d-a605-6725fe4e8580 +:END: + +A /model/ refers to a specific AI language model implementation in +GGUF format, capable of processing tasks. Each model is configured +with: +- An alias (e.g., "default", "mistral") +- File name of the GGUF model file +- Context size (maximum tokens processable) +- Optional generation parameters +- Optional end-of-text marker + +Models represent the underlying neural network "brains" of the system. +While skills define /how/ the AI should behave, models determine +/what/ the AI is capable of. Larger models (e.g., 70B+ parameters) +generally produce higher quality outputs but require more RAM and +process slower. The system supports multiple registered models, +allowing you to select the appropriate capability/performance tradeoff +for each task via the =model== parameter in your task file. Models are +typically stored in the =models_directory= and must be compatible with +llama.cpp for CPU-based inference. + +*** "TOCOMPUTE" Marker +:PROPERTIES: +:CUSTOM_ID: tocompute-marker +:ID: cd4b622a-6b74-4fac-85fe-f5b056367824 +:END: + +The =TOCOMPUTE:= marker is a special header line that /must/ appear as +the first line of any task file to trigger processing. + +Example: +#+begin_example +TOCOMPUTE: skill=default model=default priority=5 +#+end_example + +This line specifies three critical parameters: +- =skill==: Which behavioral configuration to use (default: "default") +- =model==: Which AI model to execute the task (default: "default") +- =priority==: Integer determining processing order (higher = sooner) + +The presence of this marker transforms an ordinary text file into an +executable task. Älyverkko CLI ignores files without this header, +allowing you to safely save draft versions. When you're ready for +processing, simply add this line and save the file - the daemon will +detect the change within seconds and queue the task. This marker-based +system enables asynchronous workflow: prepare your task at your pace, +then signal completion with this single line. + +*** "DONE" Marker +:PROPERTIES: +:CUSTOM_ID: done-marker +:ID: aa69e23a-248a-4459-a36e-74b43948dba9 +:END: + +The =DONE:= marker appears as the first line of processed task files, +replacing the original =TOCOMPUTE:= line. Its format documents exactly +how the task was processed: + +#+begin_example +DONE: skill=default model=default duration=2m +#+end_example + +This line records: +- Which skill was used +- Which model processed the task +- How long processing took (in seconds/minutes/hours) + +The DONE marker serves multiple critical functions: it prevents +reprocessing of completed tasks, provides an audit trail of processing +parameters, and gives immediate visual feedback about the task's +execution environment. Combined with the structured =* USER:= and =* +ASSISTANT:= sections that follow, it creates a self-documenting +conversation history that preserves both the original request and AI +response in context. This format enables iterative refinement - you +can review the AI's response, add follow-up questions, and re-add a +=TOCOMPUTE:= line to continue the conversation. + +*** Priority +:PROPERTIES: +:CUSTOM_ID: priority-concept +:ID: fc7d5e69-a2ef-4afb-b4c3-a549d0792373 +:END: + +/Priority/ is an integer value specified in the =TOCOMPUTE:= header +(e.g., =priority=10=) that determines task processing order. Higher +integer values indicate higher priority - a task with =priority=10= +will process before one with =priority=5=. The system uses a priority +queue that processes tasks in descending priority order, with random +tiebreakers for equal priorities. + +This feature is essential for managing multiple concurrent tasks. For +example: +- Urgent tasks: =priority=100= +- Normal tasks: =priority=0= (default) +- Low priority background tasks: =priority=-10= + +When you have many tasks queued (e.g., overnight processing), priority +ensures critical work gets attention first. The flexible integer +system allows fine-grained control - you're not limited to just +"high/medium/low" but can create nuanced priority tiers matching your +workflow. Note that extremely high priorities won't make processing +faster (that depends on model/hardware), but will ensure those tasks +jump the queue. + +*** System Prompt +:PROPERTIES: +:CUSTOM_ID: system-prompt +:ID: 89bc60f0-89d4-4e10-ae80-8f824f2e3c55 +:END: + +The /system prompt/ is the foundational instruction set that defines +the AI's role, behavior, and constraints for a task. It's implemented +through skills as the =prompt= field in YAML files, containing the +special == placeholder where user input gets injected. + +Characteristics of effective system prompts: +- Establish clear role ("You are an expert Python developer...") +- Define output format requirements +- Set behavioral boundaries +- Include domain-specific knowledge + +For example, a code review skill's system prompt might: +1. Instruct the AI to analyze for security vulnerabilities +2. Require responses in markdown with specific sections +3. Specify ignoring certain file types +4. Define severity classification standards + +The system prompt operates "behind the scenes" - users never see it +directly in task files, only its influence on the AI's responses. +Well-crafted prompts dramatically improve output quality by providing +consistent context across all tasks using that skill. They represent +the primary mechanism for customizing AI behavior without retraining +models. + +*** User Prompt +:PROPERTIES: +:CUSTOM_ID: user-prompt +:ID: 009e5410-f852-4faa-b81a-f9c98b056ae3 +:END: + +The /user prompt/ is the specific request, question, or content you +provide as input to the AI within a task file. It appears after the +=TOCOMPUTE:= header and forms the substantive content the AI will +process. + +Effective user prompts typically: +- Clearly state the desired outcome +- Provide sufficient context +- Specify any constraints or requirements +- Reference relevant materials when needed + +For example, a good user prompt for code generation might: + +#+begin_example +Generate a Python function that processes CSV files, handling: +- Missing values by interpolation +- Date formatting in ISO 8601 +- Memory efficiency for large files + +Include docstrings and type hints. Target Python 3.10+. +#+end_example + +Unlike the [[id:89bc60f0-89d4-4e10-ae80-8f824f2e3c55][system prompt]] (which defines /how/ the AI behaves), the +user prompt defines /what/ specific work should be done. It's where +you bring your domain knowledge and task requirements to the +interaction. Well-structured prompts yield significantly better +results - the AI can only work with what you provide. + +*** Model Library +:PROPERTIES: +:CUSTOM_ID: model-library +:ID: 5ec0618d-90bb-4b7e-913a-477da45f406f +:END: + +The /model library/ is the internal registry of all available AI +models configured in the system. It's constructed during startup +from: +- The =models= list in the [[id:fd687508-0a76-4fee-9a1c-4031cb403c60][configuration file]] +- Verified model files in the models directory + +Key functions of the model library: +- Validates model file existence +- Resolves relative/absolute paths +- Provides model lookup by alias +- Manages default model selection + +When you run =alyverkko-cli listmodels=, it queries this library to +show available models (marking missing files with "-missing"). The +library ensures that when a task specifies =model=mistral=, the system +can locate the correct GGUF file and its associated parameters. It +serves as the critical bridge between your configuration and the +actual model files on disk, handling all path resolution and +validation so your tasks can reference models by simple aliases. + +*** GGUF Format +:PROPERTIES: +:CUSTOM_ID: gguf-format +:ID: 7d8ae98c-5d47-422e-8d18-00496774cfd9 +:END: + +/GGUF/ is the binary model format used by llama.cpp for AI inference. + +Key advantages for Älyverkko CLI users: +- Enables CPU-only operation (no GPU required) +- Multiple quantization levels (Q4_K, Q8_0, etc.) +- Active development community + +When downloading models, you'll typically see filenames like +=model-Q4_K_M.gguf= where the suffix indicates quantization +level. Lower quantization (Q4) uses less RAM but sacrifices some +quality; higher (Q8) preserves more accuracy at greater memory +cost. The format's efficiency is why you can run 70B+ parameter models +on consumer hardware - a 4-bit quantized 70B model requires "only" +~40GB RAM versus hundreds of GB for full precision. + +*** llama.cpp +:PROPERTIES: +:CUSTOM_ID: llama-cpp +:ID: 01b0d389-75d4-420f-8d5c-cae29900301f +:END: + +/llama.cpp/ is the open-source inference engine that powers Älyverkko +CLI's CPU-based AI processing. It's a critical dependency, in +particular a standalone executable (=llama-completion=) that handles: + +- Loading GGUF format models +- Tokenization and detokenization +- Core neural network computations +- Generation parameter application + +Key features enabling Älyverkko CLI's functionality: +- Optimized CPU kernels for AVX2/AVX512 +- Quantization support for memory efficiency +- Batched/unattended processing capabilities +- Cross-platform compatibility + +Älyverkko CLI acts as a sophisticated wrapper around llama.cpp +*llama-completion* executable binary, managing the complex workflow of +task processing while leveraging llama.cpp's efficient inference +capabilities. The =llama_cli_path= configuration specifies where to +find this executable, which must be built separately from source to +optimize for your specific CPU. Without llama.cpp, Älyverkko CLI +couldn't execute any AI tasks - it's the actual "brain" behind the +system. + +** Important files and directories +:PROPERTIES: +:CUSTOM_ID: important-files-directories +:ID: ca3dce55-566b-427f-b6d1-630fcd245f76 +:END: +*** Configuration File +:PROPERTIES: +:CUSTOM_ID: configuration-file +:ID: fd687508-0a76-4fee-9a1c-4031cb403c60 +:END: + +The /configuration file/ (default =~/.config/alyverkko-cli.yaml=) is +the central YAML file defining all system parameters. It contains four +critical sections: + +1. *Core Paths*: + - =tasks_directory=: Where task files live + - =models_directory=: Location of GGUF model files + - =skills_directory=: Directory for skill YAML files + - =llama_cli_path=: Path to the llama.cpp executable +2. *Generation Parameters*: + - Global defaults for temperature, top_p, etc. + - Affects all tasks unless overridden +3. *Performance Tuning*: + - =thread_count= and =batch_thread_count= optimized for your + specific hardware +4. *Model Definitions*: + - Aliases, paths, and parameters for each registered model + +This file serves as the system's blueprint - without it, Älyverkko CLI +doesn't know where to find models, tasks, or how to process them. The +configuration wizard simplifies initial setup, but advanced users +often edit this file directly for fine-grained control. Parameter +precedence follows *skill* > *model* > *global* rules, creating a +flexible hierarchy for managing complex workflows. + +*** Skill Directory +:PROPERTIES: +:CUSTOM_ID: skill-directory +:ID: c62dcf42-5017-4ba4-9e4d-af5517d4f968 +:END: + +The /skill directory/ (configured via =skills_directory=) is the +filesystem location where YAML files defining AI behaviors are stored. +Each file in this directory represents a distinct skill (e.g., +=default.yaml=, =summary.yaml=), with the filename (minus extension) +serving as the skill's alias. + +This directory enables: +- Organization of different AI personas +- Easy addition/removal of capabilities +- Version control of prompt engineering +- Sharing of skill configurations + +When setting up Älyverkko CLI, you typically start with sample skills +from the documentation, then gradually customize them to match your +needs. The directory structure keeps your behavioral configurations +separate from model files and task data, creating clean separation of +concerns. Skills are reloadable at runtime - modifying a skill YAML +file automatically affects subsequent tasks using that skill, without +requiring Alyverkko CLI restart. + +*** Task Directory +:PROPERTIES: +:CUSTOM_ID: task-directory +:ID: c59976d5-cb22-44b2-a86c-7b39c40cedee +:END: + +The /task directory/ is the designated filesystem location where users +place task files for processing, configured via =tasks_directory= in +the [[id:fd687508-0a76-4fee-9a1c-4031cb403c60][YAML configuration file]]. Älyverkko CLI continuously monitors this +directory using filesystem watchers for new or modified files. When a +file with a =TOCOMPUTE:= header is detected, it's added to the +processing queue according to its priority. After completion, the +original file is renamed with a =DONE:= prefix. This directory serves +as the central hub for user-AI interaction - users create and edit +task files here using their preferred text editor, and completed +results appear in the same location. + +You might wonder: /Why deal with text files when everything has +beautiful interfaces these days?/ + +Because *this is designed for productivity, not conversation*: + +1. *No waiting around*: With CPU inference, responses take + minutes/hours. File-based workflow lets you queue tasks and get + back to work. + +2. *Natural integration*: Works with your existing text editor (VS + Code, Emacs, etc.) rather than forcing a new interface. + +3. *Version control friendly*: You can track changes to + prompts/responses with Git. + +4. *Scriptable*: Easily integrate with other tools in your workflow. + +5. Tasks directory can be synchronized with Dropbox/Syncthing or + similar tools between multiple computers or users. This way, travel + laptop can utilize processing capability or more powerful computer + at home while being connected to internet at irregular intervals. + + +Think of it like email versus phone calls - sometimes asynchronous +communication is actually /more/ productive. + +** Generation parameters +:PROPERTIES: +:CUSTOM_ID: generation-parameters +:ID: 5a253c87-2ae4-4335-a1a2-c9e07a208f7f +:END: +*** Temperature +:PROPERTIES: +:CUSTOM_ID: temperature +:ID: 24a0a54b-828b-4c78-8208-504390848fbc +:END: + +/Temperature/ is a generation parameter controlling the randomness and +creativity of AI responses, typically ranging from 0.0 (completely +deterministic) to 2.0+ (highly creative). Lower values produce more +focused, predictable outputs ideal for factual tasks, while higher +values encourage diverse, unexpected responses better for +brainstorming. + +The parameter operates through a sophisticated probability +distribution: +- Temperature = 0.0: Always selects highest-probability token (repetitive but reliable) +- Temperature = 0.7: Balanced exploration (common default) +- Temperature = 1.5+: Significant randomness (may produce nonsensical outputs) + +Älyverkko CLI implements a three-tier hierarchy for temperature +settings: [[id:456dd42e-a474-4464-a14e-384c68713537][*skill-specific* > *model-specific* > *global default*]]. This +allows precise control +- your =creative-writing.yaml= skill might use temperature=0.9 +- while your =code-review.yaml= skill uses 0.2. + +The system automatically selects the most specific applicable value, +giving you surgical control over response characteristics without +modifying model files. + +*** Top-p (Nucleus Sampling) +:PROPERTIES: +:CUSTOM_ID: top-p +:ID: 047f5bf7-e964-49ac-a666-c7ac75754e54 +:END: + +/Top-p/ (or nucleus sampling) is a generation parameter (range 0.0-1.0) +that dynamically selects the smallest set of highest-probability tokens +whose cumulative probability exceeds the p-value. For example, with +top_p=0.9, the model considers only tokens comprising the top 90% of the +probability distribution. + +- Low values (0.3-0.6): Focused, conservative responses +- Medium values (0.7-0.9): Balanced exploration (common default) +- High values (0.95+): Maximum diversity within coherence + +Unlike temperature which affects all tokens uniformly, top-p +dynamically adjusts the token selection pool based on the current +context's probability distribution. This often produces more natural +variation in responses. Like other parameters, top-p follows the +*skill* > *model* > *global* hierarchy, allowing context-specific +tuning. The default setting (typically 0.9-0.95) works well for most +general-purpose tasks while preventing extremely low-probability +("nonsense") outputs. + +*** Repeat Penalty +:PROPERTIES: +:CUSTOM_ID: repeat-penalty +:ID: 728f6daf-7a75-4e09-832b-5d449f0b4cae +:END: + +/Repeat penalty/ is a parameter (>0.0) that discourages the AI from +repeating identical phrases or tokens. A value of 1.0 means no +penalty, while values >1.0 increasingly penalize repetitions. For +example, repeat_penalty=1.2 applies a 20% reduction to the probability +of tokens that have recently appeared. + +This parameter is crucial for maintaining response quality in longer +outputs: +- Values 1.0-1.1: Mild repetition control (good for most tasks) +- Values 1.1-1.3: Stronger anti-repetition (helpful for verbose outputs) +- Values >1.5: May produce unnatural phrasing + +The parameter operates by modifying the token probability distribution +during generation - tokens that have appeared in the recent context +have their probabilities reduced by the penalty factor. This happens +dynamically throughout generation, making it more effective than +simple post-processing filters. Like other generation parameters, +repeat penalty follows the *skill* > *model* > *global* hierarchy, +allowing you to configure strict anti-repetition for technical writing +while allowing more repetition in poetic outputs. + +*** Top-k +:PROPERTIES: +:CUSTOM_ID: top-k +:ID: 2c8eb415-509c-4269-8a65-48b2e6662290 +:END: + +/Top-k/ is a generation parameter that restricts token selection to +the K most probable tokens at each step, regardless of their actual +probability values. For example, with top_k=40, the model only +considers the 40 highest-probability tokens when generating each new +token. + +Usage considerations: +- Lower values (20-40): More focused, conservative outputs +- Higher values (50-100): Greater diversity within coherence +- Value of 0: Disables top-k filtering (uses full vocabulary) + +Unlike temperature which affects probability distribution shape, top-k +creates a hard cutoff - tokens outside the top K have zero chance of +selection. This provides more deterministic control over output +diversity. The parameter follows the standard *skill* > *model* > +*global* hierarchy, allowing context-specific tuning. While less +commonly adjusted than temperature or top-p, top-k offers valuable +fine control for specialized tasks where you want to strictly limit +the token selection pool. + +*** Min-p +:PROPERTIES: +:CUSTOM_ID: min-p +:ID: 9ee338ae-bb79-4c3e-b262-928fe237cb17 +:END: + +/Min-p/ (minimum probability threshold) is an advanced generation +parameter that filters tokens whose probability falls below a +specified fraction of the highest-probability token's probability. For +example, with min_p=0.05, only tokens with probability ≥5% of the top +token's probability are considered. + +Key characteristics: +- Range 0.0-1.0 (0.0 disables the filter) +- Complements rather than replaces top-p +- More adaptive than fixed top-k + +This parameter helps eliminate extremely low-probability "tail" tokens +that might produce nonsensical outputs, while maintaining more +flexibility than strict top-k filtering. It's particularly useful +for: +- Reducing rare factual errors +- Preventing improbable word combinations +- Maintaining response coherence in long outputs + +Like other generation parameters, min_p follows the *skill* > *model* +> *global* hierarchy, though it's typically left at default (0.0) +unless addressing specific output quality issues. Advanced users might +experiment with min_p=0.03-0.07 for critical applications requiring +maximum response reliability. + +*** Thread Count +:PROPERTIES: +:CUSTOM_ID: thread-count +:ID: 727a72a7-44d1-419d-8c3b-5b2fe224b933 +:END: + +/Thread count/ specifies the number of CPU threads dedicated to the +core AI inference process (configured via =thread_count= in +YAML). This parameter primarily affects how efficiently the system +utilizes your CPU's computational resources during token +generation. Token generation is typically bound by RAM speed and not +by CPU compute. + +The parameter targets the phase or transforming tokens through the +neural network layers. Since this phase is often limited by memory +bandwidth rather than pure compute, increasing threads beyond your +RAM's capability won't improve speed but will keep your CPU cores +uselessly busy-waiting for data. + +For instance on AMD Ryzen 5 5600G I observed that AI throughput gains +start diminishing fast after about 3 threads have been utilized. And +there is almost no performance difference between 5 and 6 threads +despite CPU claiming to have 12 threads. Reason is that RAM bandwidth +gets fully utilized already very fast with just few threads. + +*** Batch Thread Count +:PROPERTIES: +:CUSTOM_ID: batch-thread-count +:ID: 133c0c3b-f1f0-454c-a159-cca5266d728a +:END: + +/Batch thread count/ specifies threads used for prompt preprocessing +(configured via =batch_thread_count=). This parameter affects how +quickly the system parses your input text for the AI model. + +Unlike *thread_count* which handles token generation, this phase is +typically compute-bound rather than RAM-bound, so higher values often +help up to your CPU's logical core count. + +*** Context Size Tokens +:PROPERTIES: +:CUSTOM_ID: context-size-tokens +:ID: d6b978f2-1db0-4b14-93f3-30ed63d97e59 +:END: + +/Context size tokens/ defines the maximum number of tokens +(word-pieces) a model can process, configured per-model via +=context_size_tokens=. This parameter represents the AI's "working +memory" capacity for any given task. + +Critical implications: +- Determines maximum input+output length. +- Larger contexts require significantly more RAM. +- Most models support 4K-128K tokens. + +This parameter fundamentally shapes what tasks a model can handle - +code analysis of large files, book chapter processing, or +multi-document summarization all require sufficient context +size. Always verify your model's actual supported context - exceeding +it causes unpredictable or significantly degraded model output. + +*** Timeout +:PROPERTIES: +:CUSTOM_ID: timeout +:ID: 85ab8c92-9439-44bc-b8d5-cb64f1fd9270 +:END: + +/Timeout/ is a parameter that specifies the maximum time (in +milliseconds) that the AI is allowed to run for a task. If this time +is exceeded, the process is terminated, and the response is marked +with "TERMINATED BY TIMEOUT". + +The timeout parameter can be set at three levels: +1. *Skill-specific*: Defined in the skill YAML file. +2. *Model-specific*: Defined in the model configuration. +3. *Global default*: Set in the main configuration file. + +The priority hierarchy is: *skill* > *model* > *global default*. + +For example, to set a 5-minute timeout (300,000 milliseconds) for a specific skill, add: +: timeout_millis: 300000 +in the skill's YAML file. + +In the model configuration: +#+begin_src yaml +models: + - alias: "mistral" + timeout_millis: 600000 + # ... other parameters +#+end_src + +In the main configuration: +#+begin_src yaml +default_timeout_millis: 120000 +#+end_src + +Setting a timeout of 0 means no timeout. + +This feature helps prevent stuck AI processes and ensures predictable +task completion times. + +*** Parameter Precedence Hierarchy +:PROPERTIES: +:CUSTOM_ID: parameter-precedence +:ID: 456dd42e-a474-4464-a14e-384c68713537 +:END: + +Älyverkko CLI implements a sophisticated three-tier /parameter +precedence hierarchy/ for generation settings (temperature, top_p, +etc.): + +1. *Skill-specific values* (highest priority) + - Defined in skill YAML files + - Example: =temperature: 0.3= in =summary.yaml= + +2. *Model-specific values* (middle priority) + - Defined in model configuration + - Example: =temperature: 0.6= for "mistral" model + +3. *Global defaults* (lowest priority) + - Set in main configuration + - Example: =default_temperature: 0.7= + +The system automatically selects the most specific applicable value, +creating a flexible "rule cascade" where specialized configurations +override broader ones. + +** AI response post processing parameters +:PROPERTIES: +:CUSTOM_ID: response-post-processing +:ID: 2096333d-1b08-40fb-b04a-94b739cb1c4b +:END: +*** Final answer indicator +:PROPERTIES: +:CUSTOM_ID: final-answer-indicator +:ID: 8da2522b-d34f-4632-9fdd-603c1a64febb +:END: + +The *final_answer_indicator* is a optional, [[id:4206c6e9-d116-4030-94a4-87bf6f82043f][model-specific]] +configuration parameter that enables automatic separation of an AI's +response into two distinct sections: +- INTERNAL THOUGHTS :: The AI's reasoning process, step-by-step + analysis, and intermediate calculations +- ASSISTANT :: The concise final answer or conclusion + +This way, user can easily skip to final answer without reading through +lengthy reasoning. It is good for scenario where the reasoning process +is detailed but the conclusion is what matters most. + +This is only useful for *thinking* LLMs that produce internal thought +monologue before producing final response. Parameter is model specific +because different thinking models can use different ways to signal end +of internal thought and transition to final response mode. + +*How It Works:* +1. You define a unique string marker (e.g., =""=) in + your model's configuration: + + #+begin_src yaml + models: + - alias: "mistral" + filesystem_path: "Mistral-Large-Instruct-2407.Q8_0.gguf" + context_size_tokens: 32768 + final_answer_indicator: "" + #+end_src + + +2. When the AI generates a response: + - The system searches for the exact marker string in the response + - If found, it splits the response into two parts: + - Everything *before* the marker → =INTERNAL THOUGHTS= + - Everything *after* the marker → =ASSISTANT= + - If the marker is *not found*, the entire response appears in + =ASSISTANT= (fallback behavior) + +3. *Output Formatting*: The processed response is saved in your task + file with this structure: + + #+begin_example + DONE: skill=default model=mistral duration=38m + ,* USER: + [Original question] + + ,* INTERNAL THOUGHTS: + [AI's reasoning process] + + ,* ASSISTANT: + [Final answer] + #+end_example + +*** End of text marker +:PROPERTIES: +:CUSTOM_ID: end-of-text-marker +:ID: a554f321-b387-4666-b7c4-768f0e7a4e96 +:END: + +An /end of text marker/ is an optional string (e.g., "###", ”[end of +text]“) specified per-model that signals the AI has completed its +response. When configured, Älyverkko CLI automatically truncates +output at this marker, removing any trailing artifacts. + +This parameter is useful with models that use specific termination +sequences so that they will not be shown to the AI user. + +For example, if a model typically ends responses with "###", setting +=end_of_text_marker: "###"= ensures the system removes "###" at the +end of AI response. + +* Installation +:PROPERTIES: +:CUSTOM_ID: installation +:ID: d5ca2e96-2063-4236-94c8-d962d13ea484 +:END: + +When you first encounter Älyverkko CLI, the setup process might seem +involved compared to cloud-based AI services. That's completely +understandable! Let me walk you through why each step exists and how +it ultimately creates a powerful, private, and cost-effective AI +assistant that works /for you/. + +** Requirements +:PROPERTIES: +:CUSTOM_ID: requirements +:ID: c0e57874-93c1-40fe-a129-e43e7e6dc810 +:END: +*Operating System:* + +Älyverkko CLI is developed and tested on Debian 13 "Bookworm". It +should work on any modern Linux distribution with minimal adjustments +to the installation process. + +*Dependencies:* +- Java Development Kit (JDK) 17 or higher +- Apache Maven for building the project + +*Hardware Requirements:* +- Modern multi-core CPU. +- The more RAM you have, the smarter AI model you can use. For + example, at least 64 GB of RAM is needed to run decent AI models + with sufficiently large context. +- Sufficient disk space to store large language models and + input/output data. + +** Your Setup Journey - What to Expect +:PROPERTIES: +:CUSTOM_ID: setup-journey +:ID: 6a2f2fa2-6d40-46c6-bf45-69a261bd69ea +:END: + +Before we start actual setup, here's brief overview of what you'll be +doing: + +*Installing Java & Maven (The Foundation)* +- *What*: Install JDK 21+ and Apache Maven +- *Why*: Älyverkko CLI is written in Java - these tools let you build + and run the application. *Key insight*: Java was chosen because it's + cross-platform, memory-safe, and perfect for long-running background + processes like our AI task processor. + + +*Building llama.cpp (Your AI Engine):* +- *What*: Download and compile the [[https://github.com/ggml-org/llama.cpp][llama.cpp]] project from GitHub. +- *Why*: This is the [[id:01b0d389-75d4-420f-8d5c-cae29900301f][actual "brain" that runs large language models]] on + *your CPU*. We build from source (rather than using rebuilt + binaries) so it can optimize for /your specific CPU/ - squeezing out + maximum performance from your hardware. + +*Adding AI Models (The Brains):* +- *What*: Download GGUF format model files (typically 4-30GB each) +- *Where*: From Hugging Face Hub ([[https://huggingface.co/models?search=gguf][search "GGUF"]]). +- *Why*: These contain the actual neural networks that power the AI. + +*Running the Interactive Wizard setup wizard:* +- *What*: Launch the configuration wizard that asks simple questions. + You'll answer questions like "Where did you put your AI models?" + with easy prompts. *Key insight*: This creates your personal + =~/.config/alyverkko-cli.yaml= file. Note: The wizard automatically + detects your models and suggests reasonable defaults - you're not + starting from scratch. + +*Setting Up "Skills" (Your Custom Instructions)* +- *What*: You will create simple YAML files defining how the AI should + behave for different tasks. +- *Why*: So you don't have to rewrite instructions every time ("be a + coding assistant" vs "be a writing editor"). *Don't worry*: You can + start with sample skills and you can modify them gradually. + +*Preparing Your First Task (The Magic Moment)* +- *What*: Create [[id:140c53cb-8032-4a04-83ed-d1818b1cfc52][task]] text file with your request, prefixed with + *TOCOMPUTE:* +- *Why*: This triggers the background processing system and verifies + that everything is working correctly. + +** Installation +:PROPERTIES: +:CUSTOM_ID: installation-steps +:ID: 0b705a37-9b84-4cd5-878a-fedc9ab09b12 +:END: + +At the moment, to use Älyverkko CLI, you need to: +- Download sources and build [[https://github.com/ggerganov/llama.cpp][llama.cpp]] project. +- Download [[id:f5740953-079b-40f4-87d8-b6d1635a8d39][sources]] and build Älyverkko CLI project. +- Download one or more pre-trained large language models in GGUF + format. Hugging Face repository [[https://huggingface.co/models?search=GGUF][has lot of them]]. + +Follow instructions for obtaining and building Älyverkko CLI on your +computer that runs Debian 13 operating system: + +1. Ensure that you have Java Development Kit (JDK) installed on your + system. + : sudo apt-get install openjdk-21-jdk + +2. Ensure that you have Apache Maven installed: + : sudo apt-get install maven + +3. Clone the [[id:f5740953-079b-40f4-87d8-b6d1635a8d39][code repository]] or download the [[id:f5740953-079b-40f4-87d8-b6d1635a8d39][source code]] for the + `alyverkko-cli` application to your local machine. + +4. Navigate to the root directory of the cloned/downloaded project in + your terminal. + +5. Execute the installation script by running + : ./install + + This script will compile the application and install it to + directory + : /opt/alyverkko-cli + + To facilitate usage from command-line, it will also define + system-wide command *alyverkko-cli* as well as "Älyverkko CLI" + launcher in desktop applications menu. + +6. Prepare Älyverkko CLI [[id:0fcdae48-81c5-4ae1-bdb9-64ae74e87c45][configuration]] file. + +7. Verify that the application has been installed correctly by running + *alyverkko-cli* in your terminal. + +** Alyverkko CLI daemon configuration +:PROPERTIES: +:CUSTOM_ID: daemon-configuration +:ID: 0fcdae48-81c5-4ae1-bdb9-64ae74e87c45 +:END: +Älyverkko CLI application configuration is done by editing YAML +formatted configuration file. + +Configuration file should be placed under current user home directory: +: ~/.config/alyverkko-cli.yaml + +*** Key Parameters Explained +:PROPERTIES: +:CUSTOM_ID: key-parameters +:ID: bd4c9716-5f04-4e94-b713-ad0ff078f8a6 +:END: +**** Core Directories +:PROPERTIES: +:CUSTOM_ID: core-directories +:ID: 75c76c6b-637c-42a7-a31a-6f6be9570747 +:END: + +- =tasks_directory=: Where task files are placed for processing. + +- =models_directory=: Contains GGUF model files. + +- =skills_directory=: Contains YAML skill definition files. + +- =llama_cli_path=: Path to llama.cpp's *llama-completion* executable. + +**** Generation Parameters +:PROPERTIES: +:CUSTOM_ID: generation-parameters-config +:ID: 40ec81ad-cea1-4d41-a007-1f334f9b4117 +:END: + +- =default_temperature=: (Optional) Creativity control (0-3, higher = + more creative). + +- =default_top_p=: (Optional) Nucleus sampling threshold (0.0-1., + higher = more diverse). + +- =default_top_k=: (Optional) Restricts token selection to the K + tokens with the highest probabilities, regardless of their actual + probability values or the shape of the distribution. + +- =default_min_p=: (Optional) Filters the vocabulary to include only + tokens whose probability is at least a certain fraction (Min P) of + the probability of the most likely token. + +- =default_repeat_penalty=: (Optional) Penalty for repetition (>0.0, + 1.0 = no penalty) + +**** Performance Tuning +:PROPERTIES: +:CUSTOM_ID: performance-tuning +:ID: 18acf2d6-7118-4f18-9a86-ac48a33c761b +:END: + +- =thread_count=: Sets the number of threads to be used by the AI + during response generation. RAM data transfer speed is usually + bottleneck here. When RAM bandwidth is saturated, increasing thread + count will no longer increase processing speed, but it will still + keep CPU cores unnecessarily busy. + +- =batch_thread_count=: Specifies the number of threads to use for + input prompt processing. CPU computing power is usually the + bottleneck here. + +**** Model-Specific Settings +:PROPERTIES: +:CUSTOM_ID: model-specific-settings +:ID: 4206c6e9-d116-4030-94a4-87bf6f82043f +:END: + +Each model in the =models= list can have: + +- =alias=: Short model alias. Model with alias "default" would be used + by default. + +- =temperature=: (Optional) See: [[id:24a0a54b-828b-4c78-8208-504390848fbc][Temperature]] + +- =top_p=: (Optional) See: [[id:047f5bf7-e964-49ac-a666-c7ac75754e54][Top-p (Nucleus Sampling)]] + +- =min_p=: (Optional) See: [[id:9ee338ae-bb79-4c3e-b262-928fe237cb17][Min-p]] + +- =top_k=: (Optional) See: [[id:2c8eb415-509c-4269-8a65-48b2e6662290][Top-k]] + +- =repeat_penalty=: (Optional) See: [[id:728f6daf-7a75-4e09-832b-5d449f0b4cae][Repeat Penalty]] + +- =filesystem_path=: File name of the model as located within + *models_directory* + +- =context_size_tokens=: Context size in tokens that model was trained + on. + +- =end_of_text_marker=: (Optional) Some models produce certain markers + to indicate end of their output. If specified here, Älyverkko CLI + can identify and remove them so that they don't leak into + conversation. Default value is: *null*. + +- =final_answer_indicator=: (Optional) Marker that allows to separate + thinking LLM internal thought from final response. Read more: [[id:8da2522b-d34f-4632-9fdd-603c1a64febb][Final + answer indicator]]. + +*** Configuration file example +:PROPERTIES: +:CUSTOM_ID: configuration-example +:ID: 0c577b27-df6f-429f-9ac2-f89105a73544 +:END: + +The application is configured using a YAML-formatted configuration +file. Below is an example of how the configuration file might look: + +#+begin_src yaml + tasks_directory: "/home/john/AI/tasks" + models_directory: "/home/john/AI/models" + skills_directory: "/home/john/AI/skills" + llama_cli_path: "/home/john/AI/llama.cpp/build/bin/llama-completion" + + # Generation parameters + default_temperature: 0.7 + default_top_p: 0.9 + default_repeat_penalty: 1.0 + + # Performance tuning + thread_count: 6 + batch_thread_count: 10 + + # Model definitions + models: + - alias: "default" + filesystem_path: "model.gguf" + context_size_tokens: 64000 + temperature: 0.6 + top_p: 0.95 + top_k: 20 + min_p: 0 + repeat_penalty: 1.1 + + - alias: "mistral" + filesystem_path: "Mistral-Large-Instruct-2407.Q8_0.gguf" + context_size_tokens: 32768 +#+end_src + +*** Enlisting available models +:PROPERTIES: +:CUSTOM_ID: enlisting-models +:ID: 47bb1776-2824-425f-a13b-c0a863c48c23 +:END: +Once Älyverkko CLI is installed and properly configured, you can run +following command at commandline to see what models are available to +it: + +: alyverkko-cli listmodels + +Note: Models that reference missing files will be automatically marked +with "-missing" suffix in their alias by configuration wizard. You can +manually remove this suffix after fixing the model file path. + +*** Self test +:PROPERTIES: +:CUSTOM_ID: self-test +:ID: 781917b5-3d46-4368-9473-8a379088c91f +:END: +The *selftest* command performs a series of checks to ensure the +system is configured correctly: + +: alyverkko-cli selftest + +It verifies: +- Configuration file integrity. +- Model directory existence. +- The presence of the *llama.cpp* executable. + +** Skill concept and configuration +:PROPERTIES: +:CUSTOM_ID: skill-configuration +:ID: 1592510e-5a8c-4cf3-9851-4a96fae6434e +:END: ++ See also: [[id:6579abb4-8386-418b-9457-cae6c3345dfb][Skill]] concept explanation. +*** Skill File Format +:PROPERTIES: +:CUSTOM_ID: skill-file-format +:ID: 47fd0e4e-f86e-4c94-9656-ed76c0f9c2c5 +:END: + +Skills are defined in YAML files stored in the *skills_directory*. + +Each skill file contains: + +#+begin_src yaml + prompt: "Full system prompt text here" + model_alias: mistral # Optional + temperature: 0.8 # Optional + top_p: 0.95 # Optional + top_k: 20 # Optional + min_p: 0 # Optional + repeat_penalty: 1.1 # Optional +#+end_src + +The system prompt must contain ** which gets replaced with +the actual user prompt during execution. + +*** Example Skill File +:PROPERTIES: +:CUSTOM_ID: example-skill-file +:ID: 4fae9009-eb2e-4d3f-a43e-68ec8c43a7cd +:END: +: writer.yaml + +#+begin_src yaml + temperature: 0.9 + top_p: 0.95 + model_alias: mistral + prompt: | + <|im_start|>system + User will provide you with task that needs to be solved along with + existing relevant information. + + You are artificial general intelligence system that always provides well reasoned responses. + <|im_end|> + <|im_start|>user + /think Solve following problem: + + + + <|im_end|> + <|im_start|>assistant +#+end_src + +See more example skills: [[https://www3.svjatoslav.eu/projects/alyverkko-cli/examples/skills/default.yaml][default.yaml]], [[https://www3.svjatoslav.eu/projects/alyverkko-cli/examples/skills/summary.yaml][summary.yaml]] + +** Starting process daemon +:PROPERTIES: +:CUSTOM_ID: starting-daemon +:ID: ab98110c-4603-40b1-a69c-72ddc13524e6 +:END: + +Älyverkko CLI keeps continuously listening for and processing tasks +from a specified mail directory. + +There are multiple alternative ways to start Älyverkko CLI in mail +processing mode: + +**** Start via command line interface +:PROPERTIES: +:CUSTOM_ID: start-cli +:ID: bea72be2-994b-479c-bd7d-56300518fb5a +:END: + +1. Open your terminal. + +2. Run the command: + : alyverkko-cli process + +3. The application will start monitoring the configured mail directory + for incoming messages and process them accordingly in endless loop. + +4. To terminate Älyverkko CLI, just hit *CTRL+c* on the keyboard, or + close terminal window. + +**** Start using your desktop environment application launcher +:PROPERTIES: +:CUSTOM_ID: start-desktop +:ID: eeb89786-3b9a-4e3d-9699-75fd4119ccee +:END: + +1. Access the application launcher or application menu on your desktop + environment. + +2. Search for "Älyverkko CLI". + +3. Click on the icon to start the application. It will open its own + terminal. + +4. If you want to stop Älyverkko CLI, just close terminal window. + +**** Start in the background as systemd system service +:PROPERTIES: +:CUSTOM_ID: start-systemd +:ID: 85a79517-dd51-466b-8b12-3907c133e75a +:END: + +During Älyverkko CLI [[id:0b705a37-9b84-4cd5-878a-fedc9ab09b12][installation]], installation script will prompt you +if you want to install *systemd* service. If you chose *Y*, Alyverkko +CLI would be immediately started in the background as a system +service. Also it will be automatically started on every system reboot. + +To view service status, use: +: systemctl -l status alyverkko-cli + +If you want to stop or disable service, you can do so using systemd +facilities: + +: sudo systemctl stop alyverkko-cli +: sudo systemctl disable alyverkko-cli + +** The Light at the End of the Tunnel +:PROPERTIES: +:CUSTOM_ID: setup-benefits +:ID: b96f4505-85a9-4347-8f50-5a83fcbd1bfb +:END: + +After setup, here's what you get: + +- ✅ A silent background process that automatically processes tasks +- ✅ Complete privacy - no data ever leaves your machine (if you don't + synchronize tasks directory) +- ✅ The ability to run state-of-the-art models without overly + expensive hardware. +- ✅ A system that keeps working while you sleep - queue up 10 tasks + before bed, get results in the morning. + +You fill find that after the first few processed tasks, the initial +setup effort feels worthwhile. You're not just getting another chat +bot - you're building a personal AI workstation tailored to your +specific needs. The initial investment pays dividends every time you +need serious AI power without compromise. + +* Usage +:PROPERTIES: +:CUSTOM_ID: usage +:ID: ca6003eb-d478-47e9-b05e-21e3c668595b +:END: +** Task file format +:PROPERTIES: +:CUSTOM_ID: task-file-format +:ID: f2fd232d-5d2b-42fa-83db-76ab015d4df9 +:END: + +Task files follow a specific structure that begins with a header line: + +#+begin_example +TOCOMPUTE: [parameters] +[User prompt content] +#+end_example + +[[https://www3.svjatoslav.eu/projects/alyverkko-cli/examples/tasks/personality%20test.org][Example complete task file]]: +#+begin_example +TOCOMPUTE: skill=default model=default priority=5 + +If numbers had personalities, +what kind of personality might the number 'zero' have? +#+end_example + +*** Task File Header Format +:PROPERTIES: +:CUSTOM_ID: task-file-header +:ID: 8609b337-83c6-425b-8f6c-d4bda22ebf1e +:END: + +The first line *must* begin with exactly =TOCOMPUTE:= followed by +space-separated key-value pairs: + +#+begin_example +TOCOMPUTE: skill=default model=mistral priority=10 +#+end_example + +Valid parameters in the header: +- =skill=[name]=: Specifies which skill to use (defaults to "default") +- =model=[alias]=: Specifies which AI model to use (defaults to "default") +- =priority=[integer]=: Higher integers mean higher priority (default: 0) + +*** Processed File Format +:PROPERTIES: +:CUSTOM_ID: processed-file-format +:ID: f9c19a99-b610-46ea-9d2e-902288995048 +:END: + +After AI processing completes, a new file is created with: +1. First line: =DONE: skill=[name] model=[alias] duration=[time]= +2. =* USER:= section containing original user prompt +3. =* ASSISTANT:= section containing AI response + +: DONE: skill=writer model=default duration=5m +: ... + +[[https://www3.svjatoslav.eu/projects/alyverkko-cli/examples/tasks/DONE:%20personality%20test.org][Example DONE file]]: + +#+begin_example + DONE: skill=default model=default duration=2m + ,* USER: + + If numbers had personalities, + what kind of personality might the number 'zero' have? + + ,* ASSISTANT: + + 1. Personality traits based on mathematical properties: + - Neutral and unchanging unless influenced + - Control over magnitude through multiplication + - Neither positive/negative (impartial) + - Foundational yet seemingly insignificant at times + + 2. Symbolism and metaphors: + - Balance point between opposites + - Container for infinite possibilities (potential) + - Cycle of beginning/endpoints + - Quiet supporter behind the scenes + + 3. Social dynamics: + - Values community/cooperation + - May feel undervalued but persists regardless + - Adaptable depending on situation/placement + + 4. Philosophical/spiritual parallels: + - Emptiness equates to purity/clean slate + - Potential without action = dormant power + + 5. Gender and self-identity: + - Ambiguous/genderless + + 6. Emotional characteristics: + - Patient; allows time for events/actions to unfold + - Resilient when facing neglect or misunderstanding + - Persistent in fulfilling its purpose + + 7. Behavioral tendencies: + - Fosters growth indirectly rather than directly contributing + measurable output + - Encourages exploration beyond limits via abstract reasoning + innovations +#+end_example + +** Task preparation +:PROPERTIES: +:CUSTOM_ID: task-preparation +:ID: 4b7900e4-77c1-45e7-9c54-772d0d3892ea +:END: + +The Älyverkko CLI application expects input files for processing in +the form of plain text files within the specified tasks directory +(configured in the [[id:0fcdae48-81c5-4ae1-bdb9-64ae74e87c45][YAML configuration file]]). + +Suggested usage flow is to prepare AI assignments within the Älyverkko +CLI mail directory using normal text editor. Once AI assignment is +ready for processing, you should [[id:883d6e7c-60e0-422b-8c00-5cdc9dfec20d][initiate AI processing]] on that file. + +*** "joinfiles" command +:PROPERTIES: +:CUSTOM_ID: joinfiles-command +:ID: be907a1f-e347-48d9-ab0c-6a556912dc49 +:END: +*Note:* See also alternative solution with similar goal: [[https://github.com/aerugo/prelude][prelude]]. + +The *joinfiles* command is a utility for aggregating the contents of +multiple files into a single document, which can then be processed by +AI. This is particularly useful for preparing comprehensive problem +statements from various source files, such as software project +directories or collections of text documents. + +**** Usage +:PROPERTIES: +:CUSTOM_ID: joinfiles-usage +:ID: d734875d-e26a-4915-9d2d-4b047f4a4d0e +:END: + +To use the *joinfiles* command, specify the source directory +containing the files you wish to join and a topic name that will be +used to generate the output file name: + +#+begin_example +alyverkko-cli joinfiles -s /path/to/source/directory -t "my_topic" +#+end_example + +If desired, you can also specify a glob pattern to match only certain files within the directory: + +#+begin_example +alyverkko-cli joinfiles -s /path/to/source/directory -p "*.java" -t "my_topic" +#+end_example + +After joining the files, you can choose to open the resulting document +in text editor for further editing or review: + +#+begin_example +alyverkko-cli joinfiles -t "my_topic" --edit +#+end_example + +**** Options +:PROPERTIES: +:CUSTOM_ID: joinfiles-options +:ID: be737651-2f1b-4197-a6ad-40ffb8bc8df5 +:END: + +- **-s, --src-dir**: Specifies the source directory from which to join + files. + +- **-p, --pattern**: An optional glob pattern to match specific files + within the source directory. + +- **-t, --topic**: The topic name that will be used as a basis for the + output file name and should reflect the subject matter of the joined + content. + +- **-e, --edit**: Opens the joined file in text editor after the join + operation is complete. + +**** Example Use Case +:PROPERTIES: +:CUSTOM_ID: joinfiles-example +:ID: c2e1066c-7f1b-4007-a22c-69c40a772c48 +:END: + +Imagine you have a software project with various source files that you +want to analyze using AI. You can use the *joinfiles* command to +create a single document for processing: + +#+begin_example +alyverkko-cli joinfiles -s /path/to/project -p "*.java" -t "software_analysis" --edit +#+end_example + +This will recursively search the project directory for Java source +files, aggregate their contents into a file named +*software_analysis.org* (within AI processor input files directory), +and open text editor on the file, so that you can add your analysis +instructions or problem statement. Finally you [[id:883d6e7c-60e0-422b-8c00-5cdc9dfec20d][Initiate AI processing]] +and after some time, you will get results and the end of the file. + +** Initiate AI processing +:PROPERTIES: +:CUSTOM_ID: initiate-processing +:ID: 883d6e7c-60e0-422b-8c00-5cdc9dfec20d +:END: + +Once your task file is prepared, you should place *TOCOMPUTE:* marker +on the first line of that file, so that it will be considered for +processing. + +When the Älyverkko CLI detects a new or modified file in the mail +directory: + +1. It checks if file has "TOCOMPUTE:" on the first line. If no, file + is ignored. Otherwise Älyverkko CLI continues processing the file. + +2. It reads the content of the file and feeds it as an input for an AI + model to generate a response. + +4. Once the AI has generated a response, the application appends this + response to the original mail contents within the same file, using + org-mode syntax to distinguish between the user's query and the + assistant's reply. The updated file will contain both the original + query (prefixed with: "* USER:*") and the AI's response (prefixed + with "* ASSISTANT:"), ensuring a clear and organized conversation + thread. "TOCOMPUTE:" is removed from the beginning of the file to + avoid processing same file again. + +Note: During AI task file preparation, feel free to save intermediary +states as often as needed because AI engine will keep ignoring the +file until *TOCOMPUTE:* line appears. Once AI assignment is ready, add +: TOCOMPUTE: +to the beginning of the file and save one last time. Älyverkko CLI +will detect new task approximately within one second after file is +saved and will start processing it. + +If your text editor automatically reloads file when it was changed by +other process in the filesystem, AI response will appear within text +editor as soon as AI response is ready. If needed, you can add further +queries at the end of the file and re-add "TOCOMPUTE:" at the +beginning of the file. This way AI will process file again and file +becomes stateful conversation. If you use GNU Emacs text editor, you +can benefit from [[id:25038854-c905-4b26-9670-cca06600223e][purpose-built GNU Emacs utilities]]. + +** Helpful GNU Emacs utilities +:PROPERTIES: +:CUSTOM_ID: emacs-utilities +:ID: 25038854-c905-4b26-9670-cca06600223e +:END: + +Note: GNU Emacs and following Emacs Lisp utilities are not required to +use Älyverkko CLI. Their purpose is to increase comfort for existing +GNU Emacs users. + +*** Easily compose new problem statement for AI from emacs +:PROPERTIES: +:CUSTOM_ID: emacs-compose-task +:ID: 4ae50ef2-104d-43f7-a002-ba007d99693f +:END: + +The Elisp function *ai-new-topic* facilitates the creation and opening +of a new Org-mode file dedicated to a user-defined topic within a +specified directory. Now you can use this file within emacs to compose +you problem statement to AI. + +When *ai-new-topic* function triggered, it first prompts the user to +input a topic name. This name will serve as the basis for the filename +and the title within the document. + +The function then constructs a file path by concatenating the +pre-defined =alyverkko-topic-files-directory= (which should be set to +your topics directory), the topic name, and the =.org= extension. If a +file with this path does not already exist, the function will create a +new file and open it for editing. + +#+begin_src elisp :results none + + (defvar alyverkko-task-files-directory "/home/user/my-ai-tasks-directory/" + "Directory where task files are stored.") + + (defun alyverkko-new-task () + "Create and open a task file in the specified directory." + (interactive) + (let ((task (read-string "Enter task name: "))) + (let ((file-path (concat alyverkko-task-files-directory task ".org"))) + (if (not (file-exists-p file-path)) + (with-temp-file file-path + )) + (find-file file-path) + (goto-char (point-max)) + (org-mode)))) + +#+end_src + +*** Easily signal to AI that problem statement is ready for solving +:PROPERTIES: +:CUSTOM_ID: emacs-signal-ready +:ID: 6ef58180-ece3-4649-927a-e8465c9b4083 +:END: + +The function =alyverkko-compute= is designed to enhance the workflow +of users working with the Älyverkko CLI application by automating the +process of marking text files for computation with a specific AI model +and prompt. + +When function is invoked, it detects available prompts and allows user +to select one. + +Thereafter function detects available models from Älyverkko CLI +configuration file and allows user to select one. + +Finally function inserts at the beginning of currently opened file +something like this: + +: TOCOMPUTE: prompt= model= priority= + +- Adjust *prompt-dir* variable to point to your prompts directory. +- Adjust *config-file* variable to point to your Älyverkko CLI + configuration file path. + +#+begin_src emacs-lisp + +(defun alyverkko-compute () + "Interactively pick a skill, model, and priority, then insert a +TOCOMPUTE line at the top of the current buffer. + +Adjust `skill-dir` and `config-file` to match your setup." + (interactive) + (let ((skill-dir "~/.config/alyverkko-cli/skills/") + (config-file "~/.config/alyverkko-cli/alyverkko-cli.yaml") + models) + + ;; Harvest model aliases from the Älyverkko CLI config + (with-temp-buffer + (insert-file-contents config-file) + (goto-char (point-min)) + (when (search-forward-regexp "^models:" nil t) + (while (search-forward-regexp "^\\s-+- alias: \"\\([^\"]+\\)\"" nil t) + (push (match-string 1) models)))) + + (if (file-exists-p skill-dir) + (let* ((files (directory-files skill-dir t "\\`[^.].*\\.yaml\\'")) + (aliases (mapcar #'file-name-base files))) + (if aliases + (let* ((selected-alias (completing-read "Select skill: " aliases)) + (model (completing-read "Select AI model: " models)) + (priority (number-to-string + (read-number "Priority (integer, default 0): " 0)))) + (alyverkko-insert-tocompute-line selected-alias model priority)) + (message "No skill files found."))) + (message "Skill directory not found.")))) + +(defun alyverkko-insert-tocompute-line (skill-alias model &optional priority) + "Insert a TOCOMPUTE line with SKILL-ALIAS, MODEL, and PRIORITY at +the top of the current buffer." + (save-excursion + (goto-char (point-min)) + (insert (format "TOCOMPUTE: skill=%s model=%s priority=%s\n" + skill-alias model (or priority "0"))) + (save-buffer))) + +#+end_src + +* Getting the source code +:PROPERTIES: +:CUSTOM_ID: getting-source-code +:ID: f67a3668-eca2-425f-b284-915ff3b3ed82 +:END: +- This program is free software: released under Creative Commons Zero + (CC0) license. + +- Program author: + - Svjatoslav Agejenko + - Homepage: https://svjatoslav.eu + - Email: mailto://svjatoslav@svjatoslav.eu + +- [[https://www.svjatoslav.eu/projects/][Other software projects hosted at svjatoslav.eu]] + +** Source code +:PROPERTIES: +:CUSTOM_ID: source-code +:ID: f5740953-079b-40f4-87d8-b6d1635a8d39 +:END: +- [[https://www2.svjatoslav.eu/gitweb/?p=alyverkko-cli.git;a=snapshot;h=HEAD;sf=tgz][Download latest snapshot in TAR GZ format]] + +- [[https://www2.svjatoslav.eu/gitweb/?p=alyverkko-cli.git;a=summary][Browse Git repository online]] + +- Clone Git repository using command: + : git clone https://www3.svjatoslav.eu/git/alyverkko-cli.git + +- See [[https://www3.svjatoslav.eu/projects/alyverkko-cli/apidocs/][JavaDoc]]. + +- See [[https://www3.svjatoslav.eu/projects/alyverkko-cli/graphs/][automatically generated class diagrams]]. Here is [[https://www3.svjatoslav.eu/projects/javainspect/legend.png][legend]] to help + understand diagrams. Diagrams were generated using [[https://www3.svjatoslav.eu/projects/javainspect/][JavaInspect tool]]. + +* Feature ideas +:PROPERTIES: +:CUSTOM_ID: feature-ideas +:ID: 0f597bc9-0a23-40e2-b60d-80d2394a3f85 +:END: + +- Recommend some concrete AI models. + +- In task directory ignore binary files, use joinfiles command as + example how to ignore binary files. Perhaps extract plain text file + detection into some utility class. + +- Make text editor configurable in application properties file. + +- Extend the application's capabilities to include voice capture and + processing. + +- Implement image capture and processing features. diff --git a/doc/research/QwQ-32B Termination issue.pdf b/doc/research/QwQ-32B Termination issue.pdf new file mode 100644 index 0000000..827eb5d Binary files /dev/null and b/doc/research/QwQ-32B Termination issue.pdf differ diff --git a/doc/research/pausing and resuming.pdf b/doc/research/pausing and resuming.pdf new file mode 100644 index 0000000..a1a2be2 Binary files /dev/null and b/doc/research/pausing and resuming.pdf differ diff --git a/install b/install new file mode 100755 index 0000000..9439a1e --- /dev/null +++ b/install @@ -0,0 +1,96 @@ +#!/bin/bash + +SYSTEMD_SERVICE_FILE="/etc/systemd/system/alyverkko-cli.service" + + +# Function to install binary and jar +install_to_opt() { + sudo rm -rf /opt/alyverkko-cli/ + sudo mkdir -p /opt/alyverkko-cli/ + sudo chmod 755 /opt/alyverkko-cli/ + + sudo cp target/alyverkko-cli.jar "/opt/alyverkko-cli/alyverkko-cli.jar" + sudo cp "alyverkko-cli" "/opt/alyverkko-cli/alyverkko-cli" + sudo chmod +x "/opt/alyverkko-cli/alyverkko-cli" + sudo cp logo.png "/opt/alyverkko-cli/logo.png" + + sudo ln -sf "/opt/alyverkko-cli/alyverkko-cli" /usr/bin/alyverkko-cli +} + +# Function to install the desktop launcher +install_desktop_entry() { + sudo cp launchers/alyverkko-cli*.desktop /usr/share/applications/ + sudo chmod 644 /usr/share/applications/alyverkko-cli*.desktop +} + +# Function to install systemd service +install_systemd_service() { + + cat < /dev/null +[Unit] +Description=Älyverkko CLI daemon in task processor mode +After=network.target + +[Service] +User=$USER +ExecStart=/opt/alyverkko-cli/alyverkko-cli process +WorkingDirectory=/opt/alyverkko-cli +Nice=10 +Restart=always +RestartSec=10 + +[Install] +WantedBy=multi-user.target +EOF + + sudo systemctl daemon-reload + sudo systemctl enable alyverkko-cli + sudo systemctl start alyverkko-cli + sleep 1 + echo "Systemd service installed, enabled and started. Service status is:" + systemctl --no-pager -l status alyverkko-cli +} + + +# Function to pre-deploy example configuration YAML file +install_config_file() { + local alyverkko_config_dir="${HOME}/.config/alyverkko-cli" + + if [ ! -d "$alyverkko_config_dir" ]; then + mkdir -p "$alyverkko_config_dir" + cp alyverkko-cli.yaml "$alyverkko_config_dir/" + else + echo "Configuration directory already exists: $alyverkko_config_dir" + fi +} + +# Main installation function +main() { + # Build the application + mvn --settings maven.xml clean package + + install_to_opt + install_desktop_entry + install_config_file + + # Check if systemd service already exists + if [ -f "$SYSTEMD_SERVICE_FILE" ]; then + echo "Systemd service is already installed." + # Display the status without hanging + echo "Service status is:" + systemctl --no-pager -l status alyverkko-cli + else + # Install systemd service if requested + echo "Do you want to install Älyverkko CLI as a systemd service? (y/N)" + read install_service + + if [[ $install_service == [Yy] ]]; then + install_systemd_service + fi + fi + + echo "Installation complete." +} + +# Call the main installation function +main diff --git a/launchers/alyverkko-cli-pause.desktop b/launchers/alyverkko-cli-pause.desktop new file mode 100644 index 0000000..ae51de9 --- /dev/null +++ b/launchers/alyverkko-cli-pause.desktop @@ -0,0 +1,8 @@ +[Desktop Entry] +Type=Application +Name=Pause Älyverkko CLI +Comment=Freeze the current llama-cli job +Icon=media-playback-pause +Exec=/usr/bin/killall -STOP llama-cli +Terminal=false +Categories=Development; diff --git a/launchers/alyverkko-cli-resume.desktop b/launchers/alyverkko-cli-resume.desktop new file mode 100644 index 0000000..029fad7 --- /dev/null +++ b/launchers/alyverkko-cli-resume.desktop @@ -0,0 +1,8 @@ +[Desktop Entry] +Type=Application +Name=Resume Älyverkko CLI +Comment=Unfreeze the current llama-cli job +Icon=media-playback-pause +Exec=/usr/bin/killall -CONT llama-cli +Terminal=false +Categories=Development; diff --git a/launchers/alyverkko-cli.desktop b/launchers/alyverkko-cli.desktop new file mode 100644 index 0000000..54378ee --- /dev/null +++ b/launchers/alyverkko-cli.desktop @@ -0,0 +1,8 @@ +[Desktop Entry] +Type=Application +Terminal=true +Name=Älyverkko CLI +Comment=Runner for artificial neural network service +Icon=/opt/alyverkko-cli/logo.png +Exec=/opt/alyverkko-cli/alyverkko-cli process +Categories=Development; diff --git a/logo.png b/logo.png new file mode 100644 index 0000000..468758e Binary files /dev/null and b/logo.png differ diff --git a/maven.xml b/maven.xml new file mode 100644 index 0000000..505327a --- /dev/null +++ b/maven.xml @@ -0,0 +1,15 @@ + + + + + + + + + + + + \ No newline at end of file diff --git a/pom.xml b/pom.xml new file mode 100644 index 0000000..624e71c --- /dev/null +++ b/pom.xml @@ -0,0 +1,209 @@ + + 4.0.0 + eu.svjatoslav + alyverkko-cli + 1.0-SNAPSHOT + Älyverkko CLI + AI engine wrapper + + + UTF-8 + UTF-8 + + + + svjatoslav.eu + https://svjatoslav.eu + + + + + eu.svjatoslav + svjatoslavcommons + 1.8 + + + eu.svjatoslav + cli-helper + 1.3 + + + org.testng + testng + 7.7.0 + test + + + org.junit.jupiter + junit-jupiter + RELEASE + test + + + com.fasterxml.jackson.core + jackson-databind + 2.13.4.1 + + + com.fasterxml.jackson.dataformat + jackson-dataformat-yaml + 2.13.0 + + + org.apache.commons + commons-lang3 + 3.12.0 + + + org.apache.commons + commons-io + 1.3.2 + + + org.projectlombok + lombok + 1.18.32 + provided + + + + + + + org.apache.maven.plugins + maven-compiler-plugin + 3.8.1 + + 11 + 11 + true + UTF-8 + + + + + org.apache.maven.plugins + maven-source-plugin + 2.2.1 + + + attach-sources + + jar + + + + + + + org.apache.maven.plugins + maven-javadoc-plugin + 2.10.4 + + + attach-javadocs + + jar + + + + + + + + foo + bar + + + + ${java.home}/bin/javadoc + + + + + org.apache.maven.plugins + maven-resources-plugin + 2.4.3 + + UTF-8 + + + + + maven-assembly-plugin + + + + + eu.svjatoslav.alyverkko_cli.Main + + + + jar-with-dependencies + + alyverkko-cli + false + + + + + package-jar-with-dependencies + package + + single + + + + jar-with-dependencies + + + + eu.svjatoslav.alyverkko_cli.Main + + + + + + + + + + + org.apache.maven.wagon + wagon-ssh-external + 2.6 + + + + + + + + svjatoslav.eu + svjatoslav.eu + scpexe://svjatoslav.eu:10006/srv/maven + + + svjatoslav.eu + svjatoslav.eu + scpexe://svjatoslav.eu:10006/srv/maven + + + + + + svjatoslav.eu + Svjatoslav repository + https://www3.svjatoslav.eu/maven/ + + + + + scm:git:ssh://n0@svjatoslav.eu/home/git/repositories/alyverkko-cli.git + scm:git:ssh://n0@svjatoslav.eu/home/git/repositories/alyverkko-cli.git + + + + + diff --git a/src/main/java/eu/svjatoslav/alyverkko_cli/Command.java b/src/main/java/eu/svjatoslav/alyverkko_cli/Command.java new file mode 100644 index 0000000..01b48b3 --- /dev/null +++ b/src/main/java/eu/svjatoslav/alyverkko_cli/Command.java @@ -0,0 +1,32 @@ +package eu.svjatoslav.alyverkko_cli; + +import java.io.IOException; + +/** + *

Base interface for all subcommands in the Älyverkko CLI. Each command must define its name and execution logic. + *

Commands typically: + *

    + *
  • Parse their own specific arguments
    • + *
  • Access the global configuration
    • + *
  • Handle I/O operations
    • + *
+ * + *

Commands should be stateless and self-contained, using the configuration object for persistent data when needed. + */ +public interface Command { + + /** + * @return the subcommand's name. + */ + String getCommandName(); + + /** + * Called to carry out the specific subcommand. Typically, reads + * command-line arguments and performs the desired action. + * + * @param args arguments passed after the subcommand name. + * @throws IOException if I/O operations fail. + * @throws InterruptedException if the operation is interrupted. + */ + void executeCommand(String[] args) throws IOException, InterruptedException; +} diff --git a/src/main/java/eu/svjatoslav/alyverkko_cli/Main.java b/src/main/java/eu/svjatoslav/alyverkko_cli/Main.java new file mode 100644 index 0000000..daea211 --- /dev/null +++ b/src/main/java/eu/svjatoslav/alyverkko_cli/Main.java @@ -0,0 +1,84 @@ +package eu.svjatoslav.alyverkko_cli; + +import eu.svjatoslav.alyverkko_cli.commands.*; +import eu.svjatoslav.alyverkko_cli.commands.task_processor.TaskProcessorCommand; +import eu.svjatoslav.alyverkko_cli.configuration.Configuration; + +import java.io.IOException; +import java.util.Optional; + +import static java.util.Arrays.copyOfRange; + +/** + * The main entry point for the Älyverkko CLI application. + * It processes subcommands such as "wizard", "selftest", "joinfiles", + * "process", etc... + */ +public class Main { + + /** + * The list of all supported subcommands. + */ + private final java.util.List commands = java.util.Arrays.asList( + new ListModelsCommand(), + new TaskProcessorCommand(), + new JoinFilesCommand(), + new WizardCommand(), + new AddTaskHeaderCommand() + ); + + /** + * The active, loaded configuration for the entire application. + * May be null if the configuration is not loaded properly. + */ + public static Configuration configuration; + + /** + * Application entry point. Dispatches to a subcommand if one is + * specified; otherwise shows usage help. + * + * @param args command-line arguments; the first is the subcommand name. + */ + public static void main(final String[] args) throws IOException, InterruptedException { + new Main().handleCommand(args); + } + + /** + * Attempts to find and execute the subcommand specified in the given arguments, + * or prints a help message if no command is found. + * + * @param args the command-line arguments. + * @throws IOException if an I/O error occurs during command execution. + * @throws InterruptedException if the command is interrupted. + */ + public void handleCommand(String[] args) throws IOException, InterruptedException { + if (args.length == 0) { + showHelp(); + return; + } + + String commandName = args[0].toLowerCase(); + Optional commandOptional = commands.stream() + .filter(cmd -> cmd.getCommandName().equals(commandName)) + .findFirst(); + + if (!commandOptional.isPresent()) { + System.out.println("Unknown command: " + commandName); + showHelp(); + return; + } + + Command command = commandOptional.get(); + String[] remainingArgs = copyOfRange(args, 1, args.length); + command.executeCommand(remainingArgs); + } + + /** + * Displays a basic help message, listing available commands. + */ + private void showHelp() { + System.out.println("Älyverkko CLI\n"); + System.out.println("Available commands:"); + commands.forEach(cmd -> System.out.println(" " + cmd.getCommandName())); + } +} diff --git a/src/main/java/eu/svjatoslav/alyverkko_cli/Utils.java b/src/main/java/eu/svjatoslav/alyverkko_cli/Utils.java new file mode 100644 index 0000000..c9a7ffa --- /dev/null +++ b/src/main/java/eu/svjatoslav/alyverkko_cli/Utils.java @@ -0,0 +1,100 @@ +package eu.svjatoslav.alyverkko_cli; + +import com.fasterxml.jackson.databind.ObjectMapper; +import com.fasterxml.jackson.dataformat.yaml.YAMLFactory; +import eu.svjatoslav.alyverkko_cli.configuration.Configuration; +import eu.svjatoslav.commons.cli_helper.parameter_parser.parameter.FileOption; + +import java.io.BufferedReader; +import java.io.File; +import java.io.FileReader; +import java.io.IOException; + +/** + *

General utility functions for the Älyverkko CLI application. Currently provides ANSI color output capabilities for + * console messages. + *

Color formatting follows standard ANSI escape sequences, with specific methods for common message types like errors. + *

For future extensions, this class could include additional helper functions for file operations or string processing. + */ +public class Utils { + + /** + * Prints a message in red text to the console. + * + * @param message the text to print in red. + */ + public static void printRedMessageToConsole(String message) { + // set output color to red + System.out.print("\033[0;31m"); + System.out.print(message + "\n"); + // reset output color + System.out.print("\033[0m"); + } + + /** + * Reads the first line of a file. + * + * @param file File to read + * @return First line of file or null if empty + * @throws IOException if file reading fails + */ + public static String getFirstLine(File file) throws IOException { + try (BufferedReader reader = new BufferedReader(new FileReader(file))) { + String line = reader.readLine(); + line = line.replace("\uFEFF", ""); // Remove BOM if present + return line; + } + } + + /** + * Inspects the first line of the file to see if it starts with "TOCOMPUTE:". + * + * @param file the file to read. + * @return true if the file's first line starts with "TOCOMPUTE:". + * @throws IOException if file reading fails. + */ + public static boolean fileHasToComputeMarker(File file) throws IOException { + String firstLine = getFirstLine(file); + firstLine = firstLine.replace("\uFEFF", ""); // Remove BOM if present + return firstLine != null && firstLine.startsWith("TOCOMPUTE:"); + } + + /** + * The default path for the YAML config file, typically under the user's home directory. + */ + public static final String DEFAULT_CONFIG_FILE_PATH = "~/.config/alyverkko-cli/alyverkko-cli.yaml".replaceFirst("^~", System.getProperty("user.home")); + + /** + * Loads the configuration from a given file, or from the default + * path if {@code configFile} is null. + * + * @param configFile the file containing the YAML config; may be null. + * @return the {@link Configuration} object, or null if not found/invalid. + * @throws IOException if file I/O fails during reading. + */ + public static Configuration loadConfiguration(File configFile) throws IOException { + + if (!configFile.exists()) { + System.err.println("Configuration file not found: " + configFile); + return null; + } + + ObjectMapper mapper = new ObjectMapper(new YAMLFactory()); + return mapper.readValue(configFile, Configuration.class); + } + + /** + * Returns the configuration file from the given option, or the default path if not present. + * @param configFileOption the CLI option for the config file. + * @return the configuration file to load. + */ + public static File getConfigurationFile(FileOption configFileOption) { + if (configFileOption != null) + if (configFileOption.isPresent()) + return configFileOption.getValue(); + + return new File(DEFAULT_CONFIG_FILE_PATH); + } + + +} diff --git a/src/main/java/eu/svjatoslav/alyverkko_cli/commands/AddTaskHeaderCommand.java b/src/main/java/eu/svjatoslav/alyverkko_cli/commands/AddTaskHeaderCommand.java new file mode 100644 index 0000000..e1cbce4 --- /dev/null +++ b/src/main/java/eu/svjatoslav/alyverkko_cli/commands/AddTaskHeaderCommand.java @@ -0,0 +1,201 @@ +package eu.svjatoslav.alyverkko_cli.commands; + +import eu.svjatoslav.alyverkko_cli.Command; +import eu.svjatoslav.alyverkko_cli.configuration.Configuration; +import eu.svjatoslav.alyverkko_cli.configuration.Model; + +import java.io.*; +import java.nio.charset.StandardCharsets; +import java.nio.file.*; +import java.util.*; + +import static eu.svjatoslav.alyverkko_cli.Utils.*; + +/** + * This command recursively adds a TOCOMPUTE header to all non-hidden files in the current directory + * that do not already have one. It prompts the user for skill, model, priority values, + * and optional custom processing instructions, validates their existence in the configuration, + * and then processes the files accordingly. + *

+ * Usage: + * 

+ *   alyverkko-cli addheader
+ *
+ * The command will interactively prompt for: + * - Skill name (must exist in skills directory) + * - Model alias (must exist in configuration models) + * - Priority value (integer, defaults to 0) + * - Custom processing instructions (optional, multi-line input; press Enter twice to finish) + *

+ * After validation, it will process all non-hidden files in the current directory and subdirectories, + * adding the TOCOMPUTE header followed by custom instructions (if provided) and the original content. + */ +public class AddTaskHeaderCommand implements Command { + + @Override + public String getCommandName() { + return "addheader"; + } + + /** + * Executes the addheader command. Loads configuration, prompts user for skill, model, priority, + * and optional custom instructions, validates them, and processes all files in the current directory recursively. + * + * @param cliArguments command-line arguments (unused in this command) + * @throws IOException if file operations fail + * @throws InterruptedException if interrupted during processing + */ + @Override + public void executeCommand(String[] cliArguments) throws IOException, InterruptedException { + // Load configuration to validate skills and models + Configuration config = loadConfiguration(getConfigurationFile(null)); + if (config == null) { + System.err.println("ERROR: Failed to load configuration file"); + return; + } + + Scanner scanner = new Scanner(System.in); + String skill = promptForSkill(scanner, config); + String model = promptForModel(scanner, config); + int priority = promptForPriority(scanner); + + // Prompt for custom processing instructions + System.out.println("\nEnter custom processing instructions (press Enter twice to finish):"); + StringBuilder customInstructions = new StringBuilder(); + String line; + while (true) { + line = scanner.nextLine(); + // Break on empty line or EOF + if (line == null || line.trim().isEmpty()) { + break; + } + customInstructions.append(line).append("\n"); + } + + System.out.println("\nProcessing files in current directory..."); + processDirectory(new File("."), skill, model, priority, customInstructions.toString()); + System.out.println("\nProcessing complete!"); + } + + /** + * Prompts user for skill name and validates it exists in skills directory. + * + * @param scanner Scanner for user input + * @param config Current configuration + * @return Validated skill name + */ + private String promptForSkill(Scanner scanner, Configuration config) { + while (true) { + System.out.print("Enter skill name: "); + String skill = scanner.nextLine().trim(); + File skillFile = new File(config.getSkillsDirectory(), skill + ".yaml"); + if (skillFile.exists() && skillFile.isFile()) { + return skill; + } + System.out.println("ERROR: Skill '" + skill + "' not found in " + config.getSkillsDirectory()); + System.out.println("Available skills: " + Arrays.toString(config.getSkillsDirectory().list((dir, name) -> name.endsWith(".yaml")))); + } + } + + /** + * Prompts user for model alias and validates it exists in configuration. + * + * @param scanner Scanner for user input + * @param config Current configuration + * @return Validated model alias + */ + private String promptForModel(Scanner scanner, Configuration config) { + List models = config.getModels(); + if (models == null || models.isEmpty()) { + System.err.println("ERROR: No models configured. Please check your configuration file."); + System.exit(1); + } + + Map modelMap = new HashMap<>(); + for (Model model : models) { + modelMap.put(model.getAlias(), model); + } + + while (true) { + System.out.print("Enter model alias: "); + String alias = scanner.nextLine().trim(); + if (modelMap.containsKey(alias)) { + return alias; + } + System.out.println("ERROR: Model '" + alias + "' not found. Available models: " + String.join(", ", modelMap.keySet())); + } + } + + /** + * Prompts user for priority value with validation. + * + * @param scanner Scanner for user input + * @return Validated priority integer + */ + private int promptForPriority(Scanner scanner) { + while (true) { + System.out.print("Enter priority (integer, default 0): "); + String input = scanner.nextLine().trim(); + if (input.isEmpty()) { + return 0; + } + try { + int priority = Integer.parseInt(input); + if (priority < 0) { + System.out.println("Priority must be non-negative. Using 0 as default."); + return 0; + } + return priority; + } catch (NumberFormatException e) { + System.out.println("ERROR: Invalid priority. Please enter an integer."); + } + } + } + + /** + * Recursively processes all files in a directory, skipping hidden files. + * + * @param dir Directory to process + * @param skill Skill name to include in TOCOMPUTE header + * @param model Model alias to include in TOCOMPUTE header + * @param priority Priority value to include in TOCOMPUTE header + * @param customInstructions Optional custom processing instructions to prepend after header + * @throws IOException if file operations fail + */ + private void processDirectory(File dir, String skill, String model, int priority, String customInstructions) throws IOException { + File[] files = dir.listFiles(); + if (files == null) return; + + for (File file : files) { + if (file.isDirectory()) { + processDirectory(file, skill, model, priority, customInstructions); + } else if (file.isFile() && !file.getName().startsWith(".")) { + processFile(file, skill, model, priority, customInstructions); + } + } + } + + /** + * Processes a single file by adding TOCOMPUTE header with optional custom instructions. + * + * @param file File to process + * @param skill Skill name for TOCOMPUTE header + * @param model Model alias for TOCOMPUTE header + * @param priority Priority value for TOCOMPUTE header + * @param customInstructions Optional custom processing instructions to insert after header + * @throws IOException if file operations fail + */ + private void processFile(File file, String skill, String model, int priority, String customInstructions) throws IOException { + if (fileHasToComputeMarker(file)) { + System.out.println("Skipped (already has header): " + file.getAbsolutePath()); + return; + } + + String content = new String(Files.readAllBytes(file.toPath()), StandardCharsets.UTF_8); + String header = String.format("TOCOMPUTE: skill=%s model=%s priority=%d\n", skill, model, priority); + String newContent = header + customInstructions + content; + + Files.write(file.toPath(), newContent.getBytes(StandardCharsets.UTF_8)); + System.out.println("Added header: " + file.getAbsolutePath()); + } +} diff --git a/src/main/java/eu/svjatoslav/alyverkko_cli/commands/JoinFilesCommand.java b/src/main/java/eu/svjatoslav/alyverkko_cli/commands/JoinFilesCommand.java new file mode 100644 index 0000000..7f30f84 --- /dev/null +++ b/src/main/java/eu/svjatoslav/alyverkko_cli/commands/JoinFilesCommand.java @@ -0,0 +1,215 @@ +package eu.svjatoslav.alyverkko_cli.commands; + +import eu.svjatoslav.alyverkko_cli.Command; +import eu.svjatoslav.commons.cli_helper.parameter_parser.Parser; +import eu.svjatoslav.commons.cli_helper.parameter_parser.parameter.DirectoryOption; +import eu.svjatoslav.commons.cli_helper.parameter_parser.parameter.NullOption; +import eu.svjatoslav.commons.cli_helper.parameter_parser.parameter.StringOption; +import eu.svjatoslav.commons.string.GlobMatcher; + +import java.io.BufferedWriter; +import java.io.File; +import java.io.IOException; +import java.nio.charset.StandardCharsets; +import java.nio.file.*; + +import static eu.svjatoslav.alyverkko_cli.Main.configuration; +import static eu.svjatoslav.alyverkko_cli.Utils.getConfigurationFile; +import static eu.svjatoslav.alyverkko_cli.Utils.loadConfiguration; + +/** + * The JoinFilesCommand aggregates multiple files (optionally matching + * a specific pattern) into a single file for AI processing, typically + * in the mail directory. + * + * Usage Example: + * 

+ *   alyverkko-cli joinfiles -s /path/to/source -p "*.java" -t "my_topic" --edit
+ *
+ */ + +public class JoinFilesCommand implements Command { + + /** + * A command-line parser to handle joinfiles arguments. + */ + final Parser parser = new Parser(); + + /** + * Directory from which files will be joined. + */ + public DirectoryOption sourceDirectoryOption = parser.add(new DirectoryOption("Directory to join files from")) + .addAliases("--src-dir", "-s") + .mustExist(); + + /** + * Pattern for matching files, such as "*.java". + */ + public StringOption patternOption = parser.add(new StringOption("Pattern to match files")) + .addAliases("--pattern", "-p"); + + /** + * Topic name, used as the basis for the output file name. + */ + public StringOption topic = parser.add(new StringOption("Topic of the joined files")) + .addAliases("--topic", "-t") + .setMandatory(); + + /** + * If present, open the joined file using a text editor afterward. + */ + public NullOption editOption = parser.add(new NullOption("Edit the joined file using text editor")) + .addAliases("--edit", "-e"); + + /** + * The base directory for recursion when joining files. + */ + public Path sourceBaseDirectory; + + /** + * The pattern used to filter files for joining, e.g. "*.java". + */ + public String fileNamePattern = null; + + /** + * The resulting output file that aggregates all matched files. + */ + File outputFile; + + /** + * @return the name of this command, i.e., "joinfiles". + */ + @Override + public String getCommandName() { + return "joinfiles"; + } + + /** + * Executes the command that joins files from a specified directory + * (matching an optional pattern) into one output file in the mail + * directory. Optionally, it can open the output file in an editor. + * + * @param cliArguments the command-line arguments after "joinfiles". + * @throws IOException if any IO operations fail. + */ + @Override + public void executeCommand(String[] cliArguments) throws IOException { + configuration = loadConfiguration(getConfigurationFile(null)); + if (configuration == null){ + System.out.println("Failed to load configuration file"); + return; + } + + if (!parser.parse(cliArguments)) { + System.out.println("Failed to parse command-line arguments"); + parser.showHelp(); + return; + } + + // Build the path to the target file that is relative to the mail directory + outputFile = configuration.getTasksDirectory().toPath().resolve(topic.getValue() + ".org").toFile(); + + if (patternOption.isPresent()) { + fileNamePattern = patternOption.getValue(); + joinFiles(); + } + + if (editOption.isPresent()) { + openFileWithEditor(); + } + } + + /** + * Opens the joined file with a text editor. Currently uses a + * command "emc" as an example—adapt as needed. + * + * @throws IOException if the launch of the editor fails. + */ + private void openFileWithEditor() throws IOException { + String[] cmd = {"emc", outputFile.getAbsolutePath()}; + Runtime.getRuntime().exec(cmd); + } + + /** + * Joins the matching files from the configured source directory + * into a single file named {@code .org} in the mail directory. + * + * @throws IOException if reading or writing files fails. + */ + private void joinFiles() throws IOException { + boolean appendToFile = outputFile.exists(); + + if (sourceDirectoryOption.isPresent()) { + sourceBaseDirectory = sourceDirectoryOption.getValue().toPath(); + } else { + sourceBaseDirectory = Paths.get("."); + } + + try (BufferedWriter writer = Files.newBufferedWriter( + outputFile.toPath(), StandardCharsets.UTF_8, + appendToFile ? StandardOpenOption.APPEND : StandardOpenOption.CREATE)) { + + // Recursively join files that match the pattern + joinFilesRecursively(sourceBaseDirectory, writer); + } + + System.out.println("Files have been joined into: " + outputFile.getAbsolutePath()); + } + + /** + * Recursively traverses the specified directory and writes the contents + * of files that match the specified {@link #fileNamePattern}. + * + * @param directoryToIndex the directory to be searched recursively. + * @param writer the writer to which file contents are appended. + * @throws IOException if file reading fails. + */ + private void joinFilesRecursively(Path directoryToIndex, BufferedWriter writer) throws IOException { + try (DirectoryStream stream = Files.newDirectoryStream(directoryToIndex)) { + for (Path entry : stream) { + if (Files.isDirectory(entry)) { + joinFilesRecursively(entry, writer); + } else if (Files.isRegularFile(entry)) { + String fileName = entry.getFileName().toString(); + + boolean match = GlobMatcher.match(fileName, fileNamePattern); + if (match) { + System.out.println("Joining file: " + fileName); + writeFile(writer, entry); + } + } + } + } + } + + /** + * Writes the contents of a single file to the specified writer, + * including a small header containing the file path. + * + * @param writer the writer to which file contents are appended. + * @param entry the file to read and write. + * @throws IOException if file reading or writing fails. + */ + private void writeFile(BufferedWriter writer, Path entry) throws IOException { + writeFileHeader(writer, entry); + + String fileContent = new String(Files.readAllBytes(entry), StandardCharsets.UTF_8); + + // remove empty lines from the beginning and end of the file + fileContent = fileContent.replaceAll("(?m)^\\s*$", ""); + + writer.write(fileContent + "\n"); + } + + /** + * Writes a small header line to indicate which file is being appended. + * + * @param writer the writer to which the header is appended. + * @param entry the path of the current file. + * @throws IOException if writing fails. + */ + private void writeFileHeader(BufferedWriter writer, Path entry) throws IOException { + String relativePath = sourceBaseDirectory.relativize(entry).toString(); + writer.write("* file: " + relativePath + "\n\n"); + } +} diff --git a/src/main/java/eu/svjatoslav/alyverkko_cli/commands/ListModelsCommand.java b/src/main/java/eu/svjatoslav/alyverkko_cli/commands/ListModelsCommand.java new file mode 100644 index 0000000..63b6cab --- /dev/null +++ b/src/main/java/eu/svjatoslav/alyverkko_cli/commands/ListModelsCommand.java @@ -0,0 +1,50 @@ +package eu.svjatoslav.alyverkko_cli.commands; + +import eu.svjatoslav.alyverkko_cli.Command; +import eu.svjatoslav.alyverkko_cli.Utils; + +import java.io.IOException; + +import static eu.svjatoslav.alyverkko_cli.Main.configuration; + +/** + *

Displays all available AI models in the configured models directory. This command provides a quick overview of + * currently available models and their metadata. + *

The implementation: + *

    + *
  • Loads the configuration
    • + *
  • Instantiates ModelLibrary
    • + *
  • Prints model details using ModelLibrary's printModels()
    • + *
+ * + *

This command is primarily intended for administrative use to verify model availability before running tasks. + */ +public class ListModelsCommand implements Command { + + /** + * @return the name of this command, i.e., "listmodels". + */ + @Override + public String getCommandName() { + return "listmodels"; + } + + /** + * Executes the command to load the user's configuration and list + * all known AI models, printing them to stdout. + * + * @param cliArguments the command-line arguments after "listmodels". + * @throws IOException if loading configuration fails. + */ + @Override + public void executeCommand(String[] cliArguments) throws IOException { + configuration = Utils.loadConfiguration(Utils.getConfigurationFile(null)); + if (configuration == null){ + System.out.println("Failed to load configuration file"); + return; + } + + System.out.println("Listing models in directory: " + configuration.getModelsDirectory()); + configuration.printModels(); + } +} diff --git a/src/main/java/eu/svjatoslav/alyverkko_cli/commands/WizardCommand.java b/src/main/java/eu/svjatoslav/alyverkko_cli/commands/WizardCommand.java new file mode 100644 index 0000000..0d8c2b2 --- /dev/null +++ b/src/main/java/eu/svjatoslav/alyverkko_cli/commands/WizardCommand.java @@ -0,0 +1,481 @@ +package eu.svjatoslav.alyverkko_cli.commands; + +import com.fasterxml.jackson.databind.ObjectMapper; +import com.fasterxml.jackson.dataformat.yaml.YAMLFactory; +import eu.svjatoslav.alyverkko_cli.Command; +import eu.svjatoslav.alyverkko_cli.Utils; +import eu.svjatoslav.alyverkko_cli.configuration.Configuration; +import eu.svjatoslav.alyverkko_cli.configuration.Model; +import eu.svjatoslav.commons.cli_helper.parameter_parser.Parser; +import eu.svjatoslav.commons.cli_helper.parameter_parser.parameter.FileOption; + +import java.io.*; +import java.nio.file.*; +import java.util.ArrayList; +import java.util.List; + +import static eu.svjatoslav.alyverkko_cli.Utils.printRedMessageToConsole; +import static eu.svjatoslav.commons.cli_helper.CLIHelper.*; + +/** + *

Interactive configuration wizard that helps users validate and fix their configuration files. + * It performs system checks and offers to fix any missing or invalid paths, discovers new models, + * and updates the configuration accordingly. + *

Key workflow steps: + *

    + *
  1. Load or create configuration
    2. + *
  2. Validate core directory paths
    3. + *
  3. Discover and annotate new models
    4. + *
  4. Save updated configuration
    5. + *
+ *

When handling split models (.gguf files with part numbering), the wizard automatically + * detects base models and only adds part-1 files to the configuration. + */ +public class WizardCommand implements Command { + + // Command-line parser to handle wizard arguments + private final Parser cliParser = new Parser(); + + /** + * Optional CLI argument for specifying a configuration file path. + */ + public FileOption configFileOption = cliParser.add(new FileOption("Configuration file path")) + .addAliases("--config", "-c"); + + /** + * The configuration object (loaded or newly created) + */ + private Configuration configuration; + + private File configurationFile; + + private boolean configurationUpdated = false; + private boolean modelsUpdated = false; + + @Override + public String getCommandName() { + return "wizard"; + } + + @Override + public void executeCommand(String[] cliArguments) throws IOException { + // Parse command-line arguments + if (!cliParser.parse(cliArguments)) { + System.out.println("Failed to parse command-line arguments"); + cliParser.showHelp(); + return; + } + + configurationFile = Utils.getConfigurationFile(configFileOption); + loadOrCreateConfiguration(); + + checkAndFixGeneralParameters(); + + fixModelEntries(); + + trySaveConfiguration(); + + if (modelsUpdated) { + System.out.println("Configuration has been updated. Please open the configuration file in a text editor to review and adjust model settings as needed."); + } + } + + private void loadOrCreateConfiguration() throws IOException { + validateConfigurationFile(); + + if (configurationFile.exists()) { + System.out.println("Found existing configuration at: \"" + configurationFile.getAbsolutePath() + "\""); + configuration = Utils.loadConfiguration(configurationFile); + } else { + // If no config found, create a fresh one + System.out.println("Existing configuration not found at \"" + + configurationFile.getAbsolutePath() + "\". Initializing new blank configuration."); + configuration = new Configuration(); + configurationUpdated = true; + } + } + + /** + * Validates the configuration file path and checks if it is a valid file. + * If not, prints an error message and exits the program. + */ + private void validateConfigurationFile() { + if (!configurationFile.exists()) return; // No need to check further if it doesn't exist + + // Check if the file is a directory + if (configurationFile.isDirectory()) { + System.err.println("ERROR: Configuration file path incorrectly points to a directory: \"" + configurationFile.getAbsolutePath() + + "\". Please specify a file instead."); + System.exit(1); + } + + // Check if the file is readable + if (!configurationFile.canRead()) { + System.err.println("ERROR: Cannot read configuration file: \"" + configurationFile.getAbsolutePath() + + "\". Please check permissions."); + System.exit(1); + } + + // Check if the file is writable + if (!configurationFile.canWrite()) { + System.err.println("ERROR: Cannot write to configuration file: \"" + configurationFile.getAbsolutePath() + + "\". Please check file permissions."); + System.exit(1); + } + } + + /** + * Step-by-step checking (and possibly fixing) of each main config parameter. + */ + private void checkAndFixGeneralParameters() { + configuration.setTasksDirectory( + checkDirectory( + configuration.getTasksDirectory(), + "Mail directory", + true, + "The mail directory is where the AI will look for tasks to solve. " + + "It should be a directory that you can write to. Please specify new mail directory path.", + true) + ); + + configuration.setModelsDirectory( + checkDirectory( + configuration.getModelsDirectory(), + "Models directory", + null, + "The models directory is where the AI model files (*.gguf) are stored. " + + "Please specify new models directory path.", + true) + ); + + configuration.setSkillsDirectory( + checkDirectory( + configuration.getSkillsDirectory(), + "Prompts directory", + null, + "The prompts directory is where the AI prompt files (*.yaml) are stored. " + + "Please specify new prompts directory path.", + true) + ); + + configuration.setLlamaCliPath( + checkFile( + configuration.getLlamaCliPath(), + "llama.cpp project llama-cli executable file path", + "The llama-cli is commandline engine that runs GGUF language models. " + + "Usually it is located under build/bin/ directory within llama.cpp project.", + true, + true) + ); + + // Default_temperature + Float temperature = configuration.getDefaultTemperature(); + if (temperature == null || temperature < 0f || temperature > 3f) { + configuration.setDefaultTemperature(askFloat( + "Enter default temperature (0-3). Lower => more deterministic, higher => more creative.", + temperature, + 0f, 3f, false + )); + } + // Thread_count + Integer threadCount = configuration.getThreadCount(); + if (threadCount == null || threadCount < 1) { + int defaultThreadCount = Runtime.getRuntime().availableProcessors() / 2; + if (defaultThreadCount < 1) defaultThreadCount = 1; + configuration.setThreadCount(askInteger( + "Enter number of CPU threads for AI generation. Typically RAM bandwidth gets saturated " + + "first and becomes bottleneck before all CPU cores can get fully utilized. So for 12 core CPU" + + " it might be enough to set 6 threads. Increasing this number higher yields diminishing returns.", + defaultThreadCount, + 1, null, false + )); + } + + // Batch thread count + Integer batchThreadCount = configuration.getBatchThreadCount(); + if (batchThreadCount == null || batchThreadCount < 1) { + int defaultThreadCount = Runtime.getRuntime().availableProcessors(); + configuration.setBatchThreadCount( + askInteger( + "\nEnter number of CPU threads for input prompt processing (all cores is often fine).", + defaultThreadCount, + 1, null, false + )); + } + } + + /** + * Validates a directory path and prompts user to fix if needed. + * @param directory current directory value + * @param directoryName name to display to user + * @param writable if directory must be writable (null = no check) + * @param explanation message to show user when prompting + * @param offerToCreate if true, offers to create directory if missing + * @return validated directory path + */ + private File checkDirectory( + File directory, + String directoryName, + Boolean writable, + String explanation, + boolean offerToCreate) { + + while (true) { + // Check if the directory is null + boolean allOk = true; + if (directory == null) { + System.out.println(directoryName + " is not defined."); + allOk = false; + directory = getNewDirectoryPath(explanation, writable); + } + + if (!directory.exists()) { + System.out.println(directoryName + " does not exist: " + directory.getAbsolutePath()); + allOk = false; + // Offer to create it + if (offerToCreate) { + if (askBoolean("Create " + directoryName + " ?", true)) { + boolean created = directory.mkdirs(); + if (!created) { + printRedMessageToConsole("Failed to create \"" + directory + "\". Check permissions"); + } else { + System.out.println(directoryName + " created at: " + directory.getAbsolutePath()); + } + } + } + } + + if (!directory.isDirectory()) { + System.out.println(directoryName + " is not a directory: " + directory.getAbsolutePath()); + allOk = false; + directory = getNewDirectoryPath(explanation, writable); + } + + if (writable != null && writable && !directory.canWrite()) { + System.out.println(directoryName + " is not writable: " + directory.getAbsolutePath()); + allOk = false; + directory = getNewDirectoryPath(explanation, writable); + } + + if (allOk) { + System.out.println(directoryName + " is: " + directory.getAbsolutePath()); + return directory; + } + + configurationUpdated = true; + } + } + + /** + * Validates a file path and prompts user to fix if needed. + * @param file current file value + * @param fileName name to display to user + * @param mustExist if file must exist (null = no check) + * @param explanation message to show user when prompting + * @param executable if file must be executable (null = no check) + * @return validated file path + */ + private File checkFile( + File file, + String fileName, + String explanation, + Boolean mustExist, + Boolean executable) { + + while (true) { + boolean allOk = true; + if (file == null) { + System.out.println(fileName + " is not defined."); + allOk = false; + file = askFile(explanation, null, mustExist, null, null, executable, false); + } + + if (mustExist != null && mustExist && !file.exists()) { + System.out.println(fileName + " does not exist: " + file.getAbsolutePath()); + allOk = false; + file = askFile(explanation, null, mustExist, null, null, executable, false); + } + + if (!file.isFile()) { + System.out.println(fileName + " is not a file: " + file.getAbsolutePath()); + allOk = false; + file = askFile(explanation, null, null, null, null, executable, false); + } + + if (executable != null && executable && !file.canExecute()) { + System.out.println(fileName + " is not executable: " + file.getAbsolutePath()); + allOk = false; + file = askFile(explanation, null, true, null, null, executable, false); + } + + if (allOk) { + System.out.println(fileName + " is: " + file.getAbsolutePath()); + return file; + } + + configurationUpdated = true; + } + } + + private static File getNewDirectoryPath(String directoryName, Boolean writable) { + return askDirectory(directoryName, null, null, null, writable, null, false); + } + + /** + * Saves the config to the default path, verifying if user wants to + * overwrite if it already exists, etc. + */ + private void trySaveConfiguration() { + if (!configurationUpdated) { + System.out.println("No changes made to the configuration. Not saving."); + return; + } + + // ask user if user wants to save configuration + if (!askBoolean("Save configuration to: " + configurationFile, true, false)) + return; + + boolean fileExisted = configurationFile.exists(); + + try { + Files.createDirectories(configurationFile.toPath().getParent()); + try (BufferedWriter writer = Files.newBufferedWriter( + configurationFile.toPath(), + StandardOpenOption.CREATE, + StandardOpenOption.TRUNCATE_EXISTING + )) { + new ObjectMapper(new YAMLFactory()).writeValue(writer, configuration); + } + if (fileExisted) { + System.out.println("Existing configuration updated at: " + configurationFile.toPath()); + } else { + System.out.println("New configuration created at: " + configurationFile.toPath()); + } + } catch (IOException e) { + printRedMessageToConsole("Error saving configuration: " + e.getMessage()); + } + } + + /** + * Generates an alias from a .gguf filename by removing non-alphanumeric chars. + */ + private String suggestAlias(String filePath) { + String fileName = new File(filePath).getName(); + // Remove .gguf extension + fileName = fileName.replaceFirst("\\.gguf$", ""); + // Check if it's a split model + if (fileName.matches(".*-\\d{5}-of-\\d{5}")) { + // Extract base name by removing the part numbers + fileName = fileName.replaceFirst("-\\d{5}-of-\\d{5}", ""); + } + // Replace non-alphanumeric characters with hyphens + String alias = fileName.replaceAll("[^a-zA-Z0-9]", "-").toLowerCase(); + // Normalize hyphens and trim leading/trailing hyphens + return alias.replaceAll("-+", "-").replaceAll("^-|-$", ""); + } + + private void fixModelEntries() { + File modelsDir = configuration.getModelsDirectory(); + if (modelsDir == null || !modelsDir.exists() || !modelsDir.isDirectory()) { + System.out.println("Models directory is not properly configured. Skipping model checks."); + return; + } + + List existingModels = configuration.getModels(); + if (existingModels == null) { + existingModels = new ArrayList<>(); + configuration.setModels(existingModels); + configurationUpdated = true; + } + + annotateMissingModels(); + + discoverNewModels(); + } + + private void discoverNewModels() { + // List all .gguf files in models directory + File[] files = configuration.getModelsDirectory().listFiles((dir, name) -> name.endsWith(".gguf")); + if (files == null) return; + + for (File file : files) { + String relativePath = configuration.getModelsDirectory().toPath().relativize(file.toPath()).toString(); + if (isExistingModel(relativePath)) continue; + + processPotentialNewModelFile(file, relativePath); + } + } + + private boolean isExistingModel(String relativePath) { + return configuration.getModels().stream() + .anyMatch(m -> m.getFilesystemPath().equals(relativePath)); + } + + private void processPotentialNewModelFile(File file, String relativePath) { + // Check if it's a split model + if (isSplitModel(file.getName())) { + handleSplitModel(file, relativePath); + } else { + addNewModel(relativePath); + } + } + + private boolean isSplitModel(String fileName) { + return fileName.matches(".*-\\d{5}-of-\\d{5}\\.gguf"); + } + + private void handleSplitModel(File file, String relativePath) { + String baseName = relativePath.replaceFirst("-\\d{5}-of-\\d{5}\\.gguf", ""); + if (configuration.getModels().stream().anyMatch(m -> m.getAlias().startsWith(baseName))) { + return; + } + + // Extract part number + String partNumberStr = relativePath.replaceAll(".*-(\\d{5}-of-\\d{5}\\.gguf)", "$1"); + int partNumber = Integer.parseInt(partNumberStr.split("-of-")[0]); + if (partNumber == 1) { + addNewModel(relativePath); + } + } + + private void addNewModel(String relativePath) { + Model newModel = getNewModel(relativePath); + configuration.getModels().add(newModel); + System.out.println("Added new model: " + newModel.getAlias() + " (" + newModel.getFilesystemPath() + ")"); + configurationUpdated = true; + modelsUpdated = true; + } + + private Model getNewModel(String relativePath) { + String suggestedAlias = suggestAlias(relativePath); + Model newModel = new Model(); + newModel.setAlias(suggestedAlias + "-new"); + newModel.setFilesystemPath(relativePath); + newModel.setContextSizeTokens(32768); // Default context size + newModel.setEndOfTextMarker(null); // Default end marker + return newModel; + } + + private void annotateMissingModels() { + // Process existing models to add/remove -missing suffix + for (Model model : configuration.getModels()) { + File modelFile = new File(configuration.getModelsDirectory(), model.getFilesystemPath()); + if (!modelFile.exists()) { + if (!model.getAlias().endsWith("-missing")) { + model.setAlias(model.getAlias() + "-missing"); + System.out.println("Marked model as missing: " + model.getAlias()); + configurationUpdated = true; + modelsUpdated = true; + } + } else { + if (model.getAlias().endsWith("-missing")) { + model.setAlias(model.getAlias().replaceFirst("-missing$", "")); + System.out.println("Removed -missing suffix from model: " + model.getAlias()); + configurationUpdated = true; + modelsUpdated = true; + } + } + } + } + +} diff --git a/src/main/java/eu/svjatoslav/alyverkko_cli/commands/package-info.java b/src/main/java/eu/svjatoslav/alyverkko_cli/commands/package-info.java new file mode 100644 index 0000000..906a20e --- /dev/null +++ b/src/main/java/eu/svjatoslav/alyverkko_cli/commands/package-info.java @@ -0,0 +1,14 @@ +/** + *

This package implements all subcommands available in the Älyverkko CLI application. Each command class provides a + * specific functionality through the Command interface. + *

Available commands include: + *

    + *
  • Wizard-style configuration builder
    • + *
  • Model listing and management
    • + *
  • File joining for multi-file processing
    • + *
  • Mail-based AI task processing
    • + *
  • Adding TOCOMPUTE headers to files
    • + *
+ */ + +package eu.svjatoslav.alyverkko_cli.commands; \ No newline at end of file diff --git a/src/main/java/eu/svjatoslav/alyverkko_cli/commands/task_processor/Task.java b/src/main/java/eu/svjatoslav/alyverkko_cli/commands/task_processor/Task.java new file mode 100644 index 0000000..1cd2865 --- /dev/null +++ b/src/main/java/eu/svjatoslav/alyverkko_cli/commands/task_processor/Task.java @@ -0,0 +1,157 @@ +package eu.svjatoslav.alyverkko_cli.commands.task_processor; + +import eu.svjatoslav.alyverkko_cli.configuration.Model; +import eu.svjatoslav.alyverkko_cli.configuration.SkillConfig; + +import static eu.svjatoslav.alyverkko_cli.Main.configuration; + +/** + * Represents the data needed to perform a single task based AI query, + * containing prompts and the specific AI model to use. + */ +public class Task { + + /** + * The system prompt text that sets the context or role for the AI. + * This is often used to establish rules or background instructions + * for how the assistant should behave. + */ + public String systemPrompt; + + /** + * The name of the specific skill or capability that the AI should utilize + * when processing the query. This can help determine the focus or purpose + * of the response generated by the AI. + */ + public String skillName; + + public SkillConfig skill; + + /** + * The user's prompt text (the main request or query). + */ + public String userPrompt; + + /** + * The AI model to be used for processing this query. + */ + public Model model; + + /** + * The start time of the query (milliseconds since epoch). + */ + public long startTimeMillis; + + /** + * The end time of the query (milliseconds since epoch). + */ + public long endTimeMillis; + + /** + * Task priority. Bigger integer has higher priority. + */ + public int priority; + + + /** + * Returns a string containing a summary of the {@link Task} object. + * + * @return a string with the system prompt, user prompt, and model info. + */ + @Override + public String toString() { + return "MailQuery{" + + "systemPrompt='" + systemPrompt + '\'' + + ", userPrompt='" + userPrompt + '\'' + + ", model=" + model + + '}'; + } + + /** + * Calculate effective temperature based on override hierarchy. + */ + public Float getEffectiveTemperature() { + // 1. Skill-specific temperature has priority + if (skill != null && skill.getTemperature() != null) return skill.getTemperature(); + + // 2. If not in skill, check if model specifies it + if (model.getTemperature() != null) return model.getTemperature(); + + // 3. Fall back to global default + return configuration.getDefaultTemperature(); + } + + /** + * Calculates effective top-p value using hierarchy: + * skill-specific → model-specific → global default + * + * @return the applicable top-p value for this query + */ + public Float getEffectiveTopP() { + // Skill-specific has the highest priority + if (skill != null && skill.getTopP() != null) return skill.getTopP(); + + // Model-specific next + if (model.getTopP() != null) return model.getTopP(); + + // Global default as fallback + return configuration.getDefaultTopP(); + } + + /** + * Calculates effective repeat penalty using hierarchy: + * skill-specific → model-specific → global default + * + * @return the applicable repeat penalty for this query + */ + public Float getEffectiveRepeatPenalty() { + // Skill-specific has the highest priority + if (skill != null && skill.getRepeatPenalty() != null) return skill.getRepeatPenalty(); + + // Model-specific next + if (model.getRepeatPenalty() != null) return model.getRepeatPenalty(); + + // Global default as fallback + return configuration.getDefaultRepeatPenalty(); + } + + + public Float getEffectiveTopK() { + if (skill != null && skill.getTopK() != null) return skill.getTopK(); + else if (model.getTopK() != null) return model.getTopK(); + else return configuration.getDefaultTopK(); + } + + public Float getEffectiveMinP() { + if (skill != null && skill.getMinP() != null) return skill.getMinP(); + else if (model.getMinP() != null) return model.getMinP(); + else return configuration.getDefaultMinP(); + } + + /** + * Calculates the effective timeout in milliseconds using the following hierarchy: + *
    + *
  1. Skill-specific timeout (highest priority)
    2. + *
  2. Model-specific timeout
    3. + *
  3. Global default timeout (lowest priority)
    4. + *
+ * + * @return the effective timeout in milliseconds, or null if no timeout is configured. + */ + public Long getEffectiveTimeoutMillis() { + // Skill-specific has the highest priority + if (skill != null && skill.getTimeoutMillis() != null) { + return skill.getTimeoutMillis(); + } + + // Model-specific next + if (model.getTimeoutMillis() != null) { + return model.getTimeoutMillis(); + } + + // Global default as fallback + return configuration.getDefaultTimeoutMillis(); + } + + +} diff --git a/src/main/java/eu/svjatoslav/alyverkko_cli/commands/task_processor/TaskPriorityQueue.java b/src/main/java/eu/svjatoslav/alyverkko_cli/commands/task_processor/TaskPriorityQueue.java new file mode 100644 index 0000000..e64bdb1 --- /dev/null +++ b/src/main/java/eu/svjatoslav/alyverkko_cli/commands/task_processor/TaskPriorityQueue.java @@ -0,0 +1,38 @@ +package eu.svjatoslav.alyverkko_cli.commands.task_processor; + +import java.nio.file.Path; +import java.util.*; + + +/** + * A custom priority queue implementation for TaskQueueEntry that maintains tasks in priority order. + * Uses a TreeSet for efficient insertion, polling, and iteration. + */ +public class TaskPriorityQueue { + // Path to priority + private final Map tasks = new HashMap<>(); + + public Path poll() { + if (tasks.isEmpty()) return null; + + Integer highestPriority = null; + Path filePath = null; + for (Map.Entry entry : tasks.entrySet()) { + if (highestPriority == null || highestPriority < (Integer) entry.getValue()) { + highestPriority = (Integer) entry.getValue(); + filePath = (Path) entry.getKey(); + } + } + + tasks.remove(filePath); + return filePath; + } + + public void add(Path filePath, int priority) { + tasks.put(filePath, priority); + } + + public void remove(Path filePath) { + tasks.remove(filePath); + } +} diff --git a/src/main/java/eu/svjatoslav/alyverkko_cli/commands/task_processor/TaskProcess.java b/src/main/java/eu/svjatoslav/alyverkko_cli/commands/task_processor/TaskProcess.java new file mode 100644 index 0000000..9235d3e --- /dev/null +++ b/src/main/java/eu/svjatoslav/alyverkko_cli/commands/task_processor/TaskProcess.java @@ -0,0 +1,317 @@ +package eu.svjatoslav.alyverkko_cli.commands.task_processor; + +import eu.svjatoslav.alyverkko_cli.Utils; + +import java.io.*; +import java.nio.file.Files; +import java.util.ArrayList; +import java.util.concurrent.TimeUnit; + +import static eu.svjatoslav.alyverkko_cli.Main.configuration; +import static java.lang.String.join; + +/** + * + * TODO: what if directory disappeared that contained original input file ? Response cannot be written back anymore. + * + *

Executes AI inference tasks through llama.cpp CLI. This class handles the complete workflow + * from prompt construction to response formatting, including temporary file management and process execution. + *

Key processing steps: + *

    + *
  1. Build standardized input prompt
    2. + *
  2. Create a temporary input file
    3. + *
  3. Execute llama.cpp with appropriate parameters
    4. + *
  4. Capture and filter output
    5. + *
  5. Perform cleanup operations
    6. + *
+ * + *

Temperature settings, context size, and thread counts are all derived from the current configuration. + * The response is formatted to match org-mode conventions while preserving the original conversation structure. + */ +public class TaskProcess { + + /** + * Marker used by llama.cpp to print metadata. We monitor and display these lines. + */ + private static final String LLAMA_CPP_META_INFO_MARKER = "llm_load_print_meta: "; + + /** + * The mail query defining system prompt, user prompt, and which model to use. + */ + private final Task task; + + /** + * Temporary file used as input to the llama.cpp CLI. + */ + private File inputFile; + + /** + * Creates a new AI task with a given mail query. + * + * @param task the mail query containing model and prompts. + */ + public TaskProcess(Task task) { + this.task = task; + } + + /** + * Builds the prompt text that is fed to llama.cpp, including the system prompt, + * the user prompt, and an "ASSISTANT:" marker signifying where the AI response begins. + * + * @return a string containing the fully prepared query prompt. + */ + private String buildAiQuery() { + return task.systemPrompt.replace("", task.userPrompt); + } + + + /** + * Runs the AI query by constructing the prompt, writing it to a temp file, + * invoking llama.cpp, collecting output, and performing any final cleanup. + * + * @return the AI's response in a format suitable for appending back into + * the conversation file. + * @throws InterruptedException if the process is interrupted. + * @throws IOException if reading/writing the file fails or the process fails to start. + */ + public String runAiQuery() throws InterruptedException, IOException { + try { + // Record the start time of the query + task.startTimeMillis = System.currentTimeMillis(); + + // Build input prompt + initializeInputFile(buildAiQuery()); + + // Prepare a process builder + ProcessBuilder processBuilder = new ProcessBuilder(); + processBuilder.command(getCliCommand().split("\\s+")); // Splitting the command string into tokens + + // Start process + Process process = processBuilder.start(); + + // Handle process's error stream + handleErrorThread(process); + + // Handle the process's output stream + StringBuilder result = new StringBuilder(); + Thread outputThread = handleResultThread(process, result); + + // Get effective timeout value + Long timeoutMillis = task.getEffectiveTimeoutMillis(); + boolean isTimedOut = false; + + if (timeoutMillis != null && timeoutMillis > 0) { + try { + if (!process.waitFor(timeoutMillis, TimeUnit.MILLISECONDS)) { + process.destroyForcibly(); + isTimedOut = true; + } + } catch (InterruptedException e) { + Thread.currentThread().interrupt(); + process.destroyForcibly(); + isTimedOut = true; + } + } else { + process.waitFor(); + } + + // Wait for the output thread to finish reading + outputThread.join(); + + // Record the end time of the query + task.endTimeMillis = System.currentTimeMillis(); + + // Clean up the AI response: remove partial prompt text, end-of-text marker, etc. + String cleanedResponse = cleanupAiResponse(result.toString()); + if (isTimedOut) { + cleanedResponse += "\nTERMINATED BY TIMEOUT"; + } + + return cleanedResponse; + } finally { + deleteTemporaryFile(); + } + } + + /** + * Creates a temporary file for the AI input and writes the prompt to it. + * + * @param aiQuery the final prompt string for the AI to process. + * @throws IOException if file creation or writing fails. + */ + private void initializeInputFile(String aiQuery) throws IOException { + inputFile = createTemporaryFile(); + Files.write(inputFile.toPath(), aiQuery.getBytes()); + } + + /** + * Creates a temporary file that will be used for the AI prompt input. + * + * @return a new {@link File} referencing the created temporary file. + * @throws IOException if the file could not be created. + */ + private File createTemporaryFile() throws IOException { + File file = Files.createTempFile("ai-inference", ".tmp").toFile(); + file.deleteOnExit(); + return file; + } + + /** + * Cleans up the AI response by removing the partial text before the + * AI response marker and after the end-of-text marker, if specified. + * + * @param result the raw output from llama.cpp. + * @return the cleaned AI response. + */ + private String cleanupAiResponse(String result) { + + // remove text after the end of text marker if it exists + if (task.model.getEndOfTextMarker() != null) { + int endOfTextMarkerIndex = result.indexOf(task.model.getEndOfTextMarker()); + if (endOfTextMarkerIndex != -1) { + result = result.substring(0, endOfTextMarkerIndex); + } + } + + return result; + } + + /** + * Returns the full command string used to run the AI inference via llama.cpp. + * + * @return a string representing the command and all arguments. + */ + private String getCliCommand() { + int niceValue = 10; // niceness level for background tasks + String executablePath = configuration.getLlamaCliPath().getAbsolutePath(); + + ArrayList args = new ArrayList<>(); + args.add("nice -n " + niceValue); + args.add(executablePath); + args.add("--model " + configuration.getModelFullFilesystemPath(task.model)); + args.add("--threads " + configuration.getThreadCount()); + args.add("--threads-batch " + configuration.getBatchThreadCount()); + + Float topP = task.getEffectiveTopP(); + if (topP != null) args.add("--top-p " + topP); + + Float topK = task.getEffectiveTopK(); + if (topK != null) args.add("--top-k " + topK); + + Float minP = task.getEffectiveMinP(); + if (minP != null) args.add("--min-p " + minP); + + Float repetitionPenalty = task.getEffectiveRepeatPenalty(); + if (repetitionPenalty != null) args.add("--repeat-penalty " + repetitionPenalty); + + args.add("--repeat-last-n 512"); + + args.add("--mirostat 0"); + args.add("--no-display-prompt"); + args.add("--no-warmup"); + args.add("--flash-attn on"); + + // By default, llama.cpp converts escape sequence like "\n" into newline before feeding it to AI. + // This causes issues if your input to AI is a computer program that has those escape codes within strings. + // So escaping must be disabled. + args.add("--no-escape"); + + Float temperature = task.getEffectiveTemperature(); + if (temperature != null) args.add("--temp " + task.getEffectiveTemperature()); + + args.add("--ctx-size " + task.model.getContextSizeTokens()); + args.add("--batch-size 512"); + + // Maps AI model from filesystem to RAM without preloading it in advance. + // Reduces RAM usage and speeds up startup. + args.add("--mmap"); + + args.add("--single-turn"); + args.add("-n -1"); + args.add("--file " + inputFile); + + return join(" ", args); + + // "--cache-type-k q8_0", + // might save RAM, need to test if precision loss is acceptable + + // might save RAM, need to test it if precision loss is acceptable + // "--cache-type-v q8_0", + + } + + /** + * Spawns a new Thread to handle the error stream from llama.cpp, + * printing lines that contain metadata or errors to the console. + * + * @param process the process whose error stream is consumed. + */ + private static void handleErrorThread(Process process) { + Thread errorThread = new Thread(() -> { + try (BufferedReader reader = new BufferedReader(new InputStreamReader(process.getErrorStream()))) { + String line; + while ((line = reader.readLine()) != null) { + handleErrorStreamLine(line); + } + } catch (IOException e) { + System.err.println("Error reading error stream: " + e.getMessage()); + } + }); + errorThread.start(); + } + + /** + * Decides what to do with each line from the error stream: + * if it matches the llama.cpp meta-info marker, print it normally; + * otherwise print as an error. + * + * @param line a line from the llama.cpp error stream. + */ + private static void handleErrorStreamLine(String line) { + if (line.startsWith(LLAMA_CPP_META_INFO_MARKER)) { + // Print the meta-info to the console in normal color + System.out.println(line.substring(LLAMA_CPP_META_INFO_MARKER.length())); + } else { + // Print actual error lines in red + Utils.printRedMessageToConsole(line); + } + } + + /** + * Consumes the standard output (inference result) from the + * llama.cpp process, storing it into a result buffer for further + * cleanup, while simultaneously printing it to the console. + * + * @param process the AI inference process. + * @param result a string builder to accumulate the final result. + * @return the thread that is reading the output stream. + */ + private static Thread handleResultThread(Process process, StringBuilder result) { + Thread outputThread = new Thread(() -> { + try (BufferedReader reader = new BufferedReader(new InputStreamReader(process.getInputStream()))) { + String aiResultLine; + while ((aiResultLine = reader.readLine()) != null) { + System.out.print("AI: " + aiResultLine + "\n"); // Show each line in real-time + result.append(aiResultLine).append("\n"); + } + } catch (IOException e) { + throw new RuntimeException(e); + } + }); + outputThread.start(); + return outputThread; + } + + /** + * Deletes the temporary input file once processing is complete. + */ + private void deleteTemporaryFile() { + if (inputFile != null && inputFile.exists()) { + try { + Files.delete(inputFile.toPath()); + } catch (IOException e) { + System.err.println("Failed to delete temporary file: " + e.getMessage()); + } + } + } +} diff --git a/src/main/java/eu/svjatoslav/alyverkko_cli/commands/task_processor/TaskProcessorCommand.java b/src/main/java/eu/svjatoslav/alyverkko_cli/commands/task_processor/TaskProcessorCommand.java new file mode 100644 index 0000000..3789b6f --- /dev/null +++ b/src/main/java/eu/svjatoslav/alyverkko_cli/commands/task_processor/TaskProcessorCommand.java @@ -0,0 +1,483 @@ +package eu.svjatoslav.alyverkko_cli.commands.task_processor; + +import eu.svjatoslav.alyverkko_cli.*; +import eu.svjatoslav.alyverkko_cli.configuration.Model; +import eu.svjatoslav.alyverkko_cli.configuration.SkillConfig; +import eu.svjatoslav.commons.cli_helper.parameter_parser.Parser; +import eu.svjatoslav.commons.cli_helper.parameter_parser.parameter.FileOption; + +import java.io.File; +import java.io.IOException; +import java.nio.file.*; +import java.util.*; + +import static eu.svjatoslav.alyverkko_cli.Main.configuration; +import static eu.svjatoslav.alyverkko_cli.Utils.fileHasToComputeMarker; +import static eu.svjatoslav.alyverkko_cli.Utils.getFirstLine; +import static eu.svjatoslav.commons.file.IOHelper.getFileContentsAsString; +import static eu.svjatoslav.commons.file.IOHelper.saveToFile; +import static java.nio.file.StandardWatchEventKinds.*; + + +/** + * TODO: What happens when directory gets renamed ? Will event listener reindex all files inside it for processing ? + * + * The TaskProcessorCommand continuously monitors a specified tasks + * directory for new or modified text files, checks if they have a + * "TOCOMPUTE:" marker, and if so, adds them to a priority queue to be + * processed in priority order. Once processed, results are appended to + * the same file. + *

+ * Usage: + * 

+ *   alyverkko-cli process
+ *
+ */ +public class TaskProcessorCommand implements Command { + + /** + * A custom priority queue implementation that maintains tasks in priority order. + */ + private final TaskPriorityQueue taskQueue = new TaskPriorityQueue(); + + /** + * A command-line parser to handle "mail" command arguments. + */ + final Parser parser = new Parser(); + + /** + * Optional CLI argument for specifying a configuration file path. + */ + public FileOption configFileOption = parser.add(new FileOption("Configuration file path")) + .addAliases("--config", "-c") + .mustExist(); + + /** + * The WatchService instance for monitoring file system changes in + * the mail directory. + */ + private WatchService directoryWatcher; + + /** + * The directory that we continuously watch for new tasks. + */ + File taskDirectory; + + + /** + * @return the name of this command, i.e., "process". + */ + @Override + public String getCommandName() { + return "process"; + } + + /** + * Executes the "mail" command, loading configuration, starting a + * WatchService on the mail directory, adding existing files to the + * task queue, and processing tasks in priority order. + * + * @param cliArguments the command-line arguments following the "mail" subcommand. + * @throws IOException if reading/writing tasks fails. + * @throws InterruptedException if the WatchService is interrupted. + */ + @Override + public void executeCommand(String[] cliArguments) throws IOException, InterruptedException { + if (!parser.parse(cliArguments)) { + System.out.println("Failed to parse commandline arguments"); + parser.showHelp(); + return; + } + + configuration = Utils.loadConfiguration(Utils.getConfigurationFile(configFileOption)); + if (configuration == null) { + System.out.println("Failed to load configuration file"); + return; + } + + taskDirectory = configuration.getTasksDirectory(); + + // Set up a directory watch service + initializeFileWatcher(); + + // Add all existing mail files to the queue + initialTaskScanAndReply(); + + System.out.println("Mail correspondent running. Press CTRL+c to terminate."); + + // Main loop: process tasks from the queue in priority order + while (true) { + // Process the highest priority task if available + Path nextTask = taskQueue.poll(); + if (nextTask != null) processTask(nextTask); + + // Check for filesystem events + WatchKey key = directoryWatcher.poll(); + + // Sleep briefly to allow the file to be fully written + Thread.sleep(1000); + + if (key != null) { + System.out.println("Detected filesystem events in mail directory. Processing ... "); + processDetectedFilesystemEvents(key); + + if (!key.reset()) { + break; + } + } + + } + + directoryWatcher.close(); + } + + private void initialTaskScanAndReply() throws IOException { + Files.walk(taskDirectory.toPath()) + .filter(path -> Files.isRegularFile(path)) + .forEach(path -> { + try { + considerFileForQueuing(path); + } catch (IOException e) { + e.printStackTrace(); + } + }); + } + + /** + * Checks if a file needs to be processed by verifying that it: + * 1) is not hidden, + * 2) is a regular file, + * 3) starts with "TOCOMPUTE:" in the first line. + * + * @param file the file to inspect. + * @return true if the file meets the criteria for AI processing. + * @throws IOException if reading the file fails. + */ + private boolean isMailProcessingNeeded(File file) throws IOException { + // ignore hidden files + if (file.getName().startsWith(".")) { + return false; + } + + // Ensure the file exists + if (!file.exists()) { + return false; + } + + // Check if it's a regular file + if (!file.isFile()) { + return false; + } + + // Ensure the first line says "TOCOMPUTE:" + return fileHasToComputeMarker(file); + } + + /** + * Persists AI response to the file; deletes the original file + */ + private static void saveAiResponseToFile(File file, Task task, String aiResponse) throws IOException { + StringBuilder resultFileContent = new StringBuilder(); + + // The First line should be "DONE:" with settings used to process this file + resultFileContent.append(getDoneLine(task)); + + resultFileContent.append("* USER:\n"); + + // Append the original user prompt (after the first line) + resultFileContent.append(task.userPrompt).append("\n"); + + // Check for the final answer indicator in the model's configuration + String finalAnswerIndicator = task.model.getFinalAnswerIndicator(); + if (finalAnswerIndicator != null && !finalAnswerIndicator.isEmpty()) { + int index = aiResponse.indexOf(finalAnswerIndicator); + if (index != -1) { + String internalThoughts = aiResponse.substring(0, index); + String finalAnswerPart = aiResponse.substring(index + finalAnswerIndicator.length()); + + resultFileContent.append("* INTERNAL THOUGHTS:\n"); + resultFileContent.append(internalThoughts).append("\n"); + + resultFileContent.append("* ASSISTANT:\n"); + resultFileContent.append(finalAnswerPart).append("\n"); + } else { + // If the indicator is present in model config but not in response, just put entire response in ASSISTANT + resultFileContent.append("* ASSISTANT:\n"); + resultFileContent.append(aiResponse).append("\n"); + } + } else { + resultFileContent.append("* ASSISTANT:\n"); + resultFileContent.append(aiResponse).append("\n"); + } + + File newFile = new File(file.getParentFile(), "DONE: " + file.getName()); + saveToFile(newFile, resultFileContent.toString()); + + if (!file.delete()) { + System.err.println("Failed to delete original file: " + file.getAbsolutePath()); + } + } + + /** + * Processes a task by reading the file, building the MailQuery, + * running the AI query, and saving the response. + * + * @param entry the task entry containing the file path and priority. + */ + private void processTask(Path taskPath) throws IOException { + File file = taskPath.toFile(); + + if (!isMailProcessingNeeded(file)) { + System.out.println("Ignoring file: " + taskPath.getFileName() + " (does not need processing now)"); + return; + } + + try { + Task task = buildTaskFromFile(file); + TaskProcess aiTask = new TaskProcess(task); + String aiGeneratedResponse = aiTask.runAiQuery(); + + saveAiResponseToFile(file, task, aiGeneratedResponse); + } catch (IOException | InterruptedException | RuntimeException e) { + e.printStackTrace(); + } + } + + + /** + * Builds a string for the first line of the output file indicating + * that processing is done. + * + * @param task the query that was processed. + * @return a string for the first line. + */ + private static String getDoneLine(Task task) { + return "DONE: skill=" + task.skillName + + " model=" + task.model.getAlias() + + " duration=" + getDuration(task.startTimeMillis, task.endTimeMillis) + "\n"; + } + + + /** + * Returns duration string based on elapsed time. + */ + private static String getDuration(long startTimeMillis, long endTimeMillis) { + + long durationSeconds = (endTimeMillis - startTimeMillis) / 1000; + + if (durationSeconds < 180) return durationSeconds + "s"; + + long durationMinutes = durationSeconds / 60; + if (durationMinutes < 180) return durationMinutes + "m"; + + long durationHours = durationMinutes / 60; + return durationHours + "h"; + } + + /** + * Builds a Task object from the contents of a file. + *

+ * This method now implements a three-level hierarchy for model selection: + * 1. Explicit model specified in TOCOMPUTE line (the highest priority) + * 2. Model alias defined in the skill configuration (if present) + * 3. Default "default" model (the lowest priority) + * + * @param file the file to read. + * @return the constructed MailQuery. + * @throws IOException if reading the file fails. + */ + private Task buildTaskFromFile(File file) throws IOException { + Task result = new Task(); + + String inputFileContent = getFileContentsAsString(file); + int firstNewLineIndex = inputFileContent.indexOf('\n'); + if (firstNewLineIndex == -1) { + throw new IllegalArgumentException("Input file is only one line long."); + } + + String firstLine = inputFileContent.substring(0, firstNewLineIndex); + Map fileProcessingSettings = parseSettings(firstLine); + + // The rest of the file is the user prompt + result.userPrompt = inputFileContent.substring(firstNewLineIndex + 1); + + // Set system prompt + result.skillName = fileProcessingSettings.getOrDefault("skill", "default"); + SkillConfig skill = configuration.getSkillByName(result.skillName); + result.systemPrompt = skill.getPrompt(); + result.skill = skill; + + // Set AI model using hierarchy: TOCOMPUTE > skill config > default + String modelAlias = fileProcessingSettings.getOrDefault("model", + skill.getModelAlias() != null ? skill.getModelAlias() : "default"); + Optional modelOptional = configuration.findModelByAlias(modelAlias); + if (!modelOptional.isPresent()) { + throw new IllegalArgumentException("Model with alias '" + modelAlias + "' not found."); + } + result.model = modelOptional.get(); + + // Set priority + String priorityStr = fileProcessingSettings.get("priority"); + result.priority = 0; + if (priorityStr != null) { + try { + result.priority = Integer.parseInt(priorityStr); + } catch (NumberFormatException e) { + System.err.println("Invalid priority in file: " + priorityStr); + } + } + + return result; + } + + /** + * Parses the "TOCOMPUTE:" line, which should look like: + * 

TOCOMPUTE: key1=value1 key2=value2 ...
+ * + * @param toComputeLine the line beginning with "TOCOMPUTE:". + * @return a map of settings derived from that line. + */ + private Map parseSettings(String toComputeLine) { + toComputeLine = toComputeLine.replace("\uFEFF", ""); // Remove BOM if present + if (!toComputeLine.startsWith("TOCOMPUTE:")) { + throw new IllegalArgumentException("Invalid TOCOMPUTE line: " + toComputeLine); + } + + // If there's nothing beyond "TOCOMPUTE:", just return an empty map + if (toComputeLine.length() <= "TOCOMPUTE: ".length()) { + return new HashMap<>(); + } + + // Example format: "TOCOMPUTE: prompt=writer model=mistral" + String[] parts = toComputeLine.substring("TOCOMPUTE: ".length()).split("\\s+"); + Map result = new HashMap<>(); + + for (String part : parts) { + String[] keyValue = part.split("="); + if (keyValue.length == 2) result.put(keyValue[0], keyValue[1]); + } + return result; + } + + /** + * Handles the filesystem events from the WatchService (e.g., file creation, modification, or deletion), + * and updates the task queue accordingly. + * + * @param key the watch key containing the events. + * @throws IOException if file processing fails. + */ + private void processDetectedFilesystemEvents(WatchKey key) throws IOException { + Path dir = (Path) key.watchable(); + for (WatchEvent event : key.pollEvents()) { + WatchEvent.Kind kind = event.kind(); + if (kind != ENTRY_CREATE && kind != ENTRY_MODIFY && kind != ENTRY_DELETE) { + continue; + } + @SuppressWarnings("unchecked") + WatchEvent ev = (WatchEvent) event; + Path filename = ev.context(); + Path fullPath = dir.resolve(filename); + System.out.printf("Event: %s – %s%n", kind.name(), fullPath); + if (kind == ENTRY_DELETE) { + // Remove any existing tasks for this file + removeTasksForFile(fullPath); + continue; + } + // Handle directory creation + if (Files.isDirectory(fullPath)) { + if (kind == ENTRY_CREATE) { + // Register the new directory and its subdirectories for monitoring + registerAllSubdirectories(fullPath); + // Scan the new directory for existing files to process + Files.walk(fullPath) + .filter(path -> { + try { + return Files.isRegularFile(path) && !Files.isHidden(path); + } catch (IOException e) { + System.err.println("Failed to check if file is hidden: " + path + " - " + e.getMessage()); + return false; // Skip files that cause errors + } + }) + .forEach(path -> { + try { + considerFileForQueuing(path); + } catch (IOException e) { + System.err.println("Failed to process file in new directory: " + path + " - " + e.getMessage()); + } + }); + } + continue; + } + // Handle file events + if (kind == ENTRY_MODIFY) { + // Remove existing tasks for this file before adding new ones + removeTasksForFile(fullPath); + } + // Check if it's a regular, non-hidden file and needs processing + try { + if (Files.isRegularFile(fullPath) && !Files.isHidden(fullPath)) { + considerFileForQueuing(fullPath); + } + } catch (IOException e) { + System.err.println("Failed to check file: " + fullPath + " - " + e.getMessage()); + } + } + } + + private void initializeFileWatcher() throws IOException { + this.directoryWatcher = FileSystems.getDefault().newWatchService(); + registerAllSubdirectories(taskDirectory.toPath()); + } + + private void registerAllSubdirectories(Path path) { + try { + System.out.println("Registering directory for watch service: " + path); + path.register(directoryWatcher, ENTRY_CREATE, ENTRY_MODIFY); + if (Files.isDirectory(path)) { + try (DirectoryStream stream = Files.newDirectoryStream(path)) { + for (Path entry : stream) { + if (Files.isDirectory(entry)) { + registerAllSubdirectories(entry); + } + } + } + } + } catch (IOException e) { + System.err.println("Failed to register directory: " + path + " - " + e.getMessage()); + } + } + + /** + * Adds a file to the task queue if it needs processing. + * + * @param filePath the path to the file to check. + * @throws IOException if reading the first line fails. + */ + private void considerFileForQueuing(Path filePath) throws IOException { + File file = filePath.toFile(); + if (!isMailProcessingNeeded(file)) return; + + String firstLine = getFirstLine(file); + Map settings = parseSettings(firstLine); + int priority = 0; + String priorityStr = settings.get("priority"); + if (priorityStr != null) { + try { + priority = Integer.parseInt(priorityStr); + } catch (NumberFormatException e) { + System.err.println("Invalid priority in file " + filePath.getFileName() + ": " + priorityStr); + } + } + taskQueue.add(filePath, priority); + } + + /** + * Removes all tasks from the queue that match the given file path. + * + * @param filePath the file path to match and remove from the queue. + */ + private void removeTasksForFile(Path filePath) { + taskQueue.remove(filePath); + } + +} diff --git a/src/main/java/eu/svjatoslav/alyverkko_cli/commands/task_processor/package-info.java b/src/main/java/eu/svjatoslav/alyverkko_cli/commands/task_processor/package-info.java new file mode 100644 index 0000000..51e74a4 --- /dev/null +++ b/src/main/java/eu/svjatoslav/alyverkko_cli/commands/task_processor/package-info.java @@ -0,0 +1,12 @@ +/** + *

This subpackage implements the mail-based AI task processing functionality. It watches mail directories for + * new/modified files with specific markers and processes them using AI models. + *

Key components: + *

    + *
  • File monitoring with WatchService
    • + *
  • Prompt parsing and execution logic
    • + *
  • Query object for storing processing parameters
    • + *
+ */ + +package eu.svjatoslav.alyverkko_cli.commands.task_processor; \ No newline at end of file diff --git a/src/main/java/eu/svjatoslav/alyverkko_cli/configuration/Configuration.java b/src/main/java/eu/svjatoslav/alyverkko_cli/configuration/Configuration.java new file mode 100644 index 0000000..018109f --- /dev/null +++ b/src/main/java/eu/svjatoslav/alyverkko_cli/configuration/Configuration.java @@ -0,0 +1,151 @@ +package eu.svjatoslav.alyverkko_cli.configuration; + +import com.fasterxml.jackson.annotation.JsonProperty; +import lombok.Data; + +import java.io.*; +import java.util.List; +import java.util.Optional; + +import com.fasterxml.jackson.databind.ObjectMapper; +import com.fasterxml.jackson.dataformat.yaml.YAMLFactory; + + +/** + *

Central configuration class storing all application parameters. + * This class is serialized to YAML format for user editing and persistence. + *

Configuration parameters include: + *

    + *
  • Model and prompt directories
    • + *
  • Performance tuning parameters
    • + *
  • Model-specific configurations
    • + *
+ *

All paths are resolved relative to the user's home directory by default, but can be customized. The class provides + * direct access to prompt content for AI query construction. + */ +@Data +public class Configuration { + + /** + * Directory where AI tasks (mail) are placed and discovered. + */ + @JsonProperty("tasks_directory") + private File tasksDirectory; + + /** + * Directory that contains AI model files in GGUF format. + */ + @JsonProperty("models_directory") + private File modelsDirectory; + + /** + * The default "temperature" used by the AI for creative/deterministic + * tradeoff. Ranges roughly between 0 and 3. + */ + @JsonProperty("default_temperature") + private Float defaultTemperature; + + /** + * Default top-p value used when not specified elsewhere. + * Controls diversity of sampling (0.0-1.0, where 1.0 means no restriction). + */ + @JsonProperty("default_top_p") + private Float defaultTopP; + + /** + * Default global Top-K value used when not specified elsewhere. + */ + @JsonProperty("default_top_k") + private Float defaultTopK = 30f; + + /** + * Global minimum-p threshold default (applies if none set). + */ + @JsonProperty("default_min_p") + private Float defaultMinP = 0f; + + /** + * Default repeat penalty used when not specified elsewhere. + * Controls repetition reduction (typically 0.8-2.0, where 1.0 means no penalty). + */ + @JsonProperty("default_repeat_penalty") + private Float defaultRepeatPenalty; + + /** + * The filesystem path to the llama-cli executable, which processes + * AI tasks via llama.cpp. + */ + @JsonProperty("llama_cli_path") + private File llamaCliPath; + + /** + * Number of CPU threads used for input prompt processing. + */ + @JsonProperty("batch_thread_count") + private Integer batchThreadCount; + + /** + * Number of CPU threads used for AI inference. + */ + @JsonProperty("thread_count") + private Integer threadCount; + + /** + * Directory containing text prompt files. Each file is a separate + * "prompt" by alias (the filename minus ".txt"). + */ + @JsonProperty("skills_directory") + private File skillsDirectory; + + /** + * The default timeout in milliseconds for AI processing tasks. A value of 0 or null means no timeout. + * This serves as the lowest-priority fallback when no skill-specific or model-specific timeout is set. + */ + @JsonProperty("default_timeout_millis") + private Long defaultTimeoutMillis; + + + /** + * The list of models defined in this configuration. + */ + private List models; + + + /** + * Retrieves the contents of a prompt file by alias, e.g. "writer" + * maps to "writer.txt" in the prompt's directory. + * + * @param skillName the name of the prompt file (without ".txt"). + * @return the full-text content of the prompt file. + * @throws IOException if reading the prompt file fails. + */ + public SkillConfig getSkillByName(String skillName) throws IOException { + File promptFile = new File(skillsDirectory, skillName + ".yaml"); + ObjectMapper mapper = new ObjectMapper(new YAMLFactory()); + return mapper.readValue(promptFile, SkillConfig.class); + } + + + + /** + * Prints the details of each model in the library to standard output. + */ + public void printModels() { + System.out.println("Available models:\n"); + for (Model model : models) { + model.printModelDetails(); + System.out.println(); + } + } + + public String getModelFullFilesystemPath(Model model) { + return new File(modelsDirectory, model.getFilesystemPath()).getAbsolutePath(); + } + + public Optional findModelByAlias(String modelAlias) { + for (Model model : models) { + if (model.getAlias().equals(modelAlias)) return Optional.of(model); + } + return Optional.empty(); + } +} diff --git a/src/main/java/eu/svjatoslav/alyverkko_cli/configuration/Model.java b/src/main/java/eu/svjatoslav/alyverkko_cli/configuration/Model.java new file mode 100644 index 0000000..33eadfd --- /dev/null +++ b/src/main/java/eu/svjatoslav/alyverkko_cli/configuration/Model.java @@ -0,0 +1,94 @@ +package eu.svjatoslav.alyverkko_cli.configuration; + +import com.fasterxml.jackson.annotation.JsonProperty; +import lombok.Data; + + +/** + * Represents a single AI model configuration entry, including alias, + * path to the model file, token context size, and an optional + * end-of-text marker. + */ +@Data +public class Model { + + /** + * A short name for the model, e.g., "default" or "mistral". + */ + private String alias; + + /** + * Model-specific temperature value overriding global default. + */ + private Float temperature; + + /** + * Model-specific top-p value overriding global default. + */ + @JsonProperty("top_p") + private Float topP; + + @JsonProperty("min_p") + private Float minP; + + @JsonProperty("top_k") + private Float topK; + + /** + * Model-specific repeat penalty value overriding global default. + */ + @JsonProperty("repeat_penalty") + private Float repeatPenalty; + + /** + * The path to the model file (GGUF, etc.), relative to + * {@link Configuration#getModelsDirectory()} or fully qualified. + */ + @JsonProperty("filesystem_path") + private String filesystemPath; + + /** + * The maximum context size the model supports, in tokens. + */ + @JsonProperty("context_size_tokens") + private int contextSizeTokens; + + /** + * Optional text marker signifying the end of text for this model. + * If non-null, it will be used to strip trailing tokens from the AI response. + */ + @JsonProperty("end_of_text_marker") + private String endOfTextMarker; + + /** + * Optional string that indicates the start of the final answer in the model's output. + * When specified, the response is split into ASSISTANT and FINAL ANSWER sections based on this indicator. + */ + @JsonProperty("final_answer_indicator") + private String finalAnswerIndicator; + + /** + * Maximum time in milliseconds allowed for AI processing for this model. If null, no timeout is set for this model. + * This value overrides the global default timeout but is overridden by skill-specific timeouts. + */ + @JsonProperty("timeout_millis") + private Long timeoutMillis; + + /** + *

Prints the model's metadata to standard output in a consistent format. This includes the model's alias, + * filesystem path, and context token capacity. The output format is designed to be both human-readable and + * machine-parsable when needed. + *

Typical output: + * 

+     * Model: default
+     *   Path: /path/to/model.gguf
+     *   Context size: 32768
+     *
+ */ + public void printModelDetails() { + System.out.println("Model: " + alias); + System.out.println(" Path: " + filesystemPath); + System.out.println(" Context size: " + contextSizeTokens); + } + +} diff --git a/src/main/java/eu/svjatoslav/alyverkko_cli/configuration/SkillConfig.java b/src/main/java/eu/svjatoslav/alyverkko_cli/configuration/SkillConfig.java new file mode 100644 index 0000000..f260938 --- /dev/null +++ b/src/main/java/eu/svjatoslav/alyverkko_cli/configuration/SkillConfig.java @@ -0,0 +1,51 @@ +package eu.svjatoslav.alyverkko_cli.configuration; + +import com.fasterxml.jackson.annotation.JsonProperty; +import lombok.Data; + +@Data +public class SkillConfig { + + private String prompt; + + private Float temperature; + + /** + * Skill-specific top-p value overriding model/global defaults. + */ + @JsonProperty("top_p") + private Float topP; + + @JsonProperty("top_k") + private Float topK; + + @JsonProperty("min_p") + private Float minP; + + /** + * Skill-specific repeat penalty value overriding model/global defaults. + */ + @JsonProperty("repeat_penalty") + private Float repeatPenalty; + + /** + * Optional model alias to use for this skill. If specified, this model will be used by default + * when the skill is invoked, unless overridden by the task's TOCOMPUTE line. + * + * Example usage in skill YAML file: + * model_alias: "mistral" + * + * This creates a convenient way to associate specific skills with appropriate models + * without requiring users to specify the model in every task file. + */ + @JsonProperty("model_alias") + private String modelAlias; + + /** + * Maximum time in milliseconds allowed for AI processing when this skill is used. If null, no timeout is set for this skill. + * This is the highest-priority timeout value in the hierarchy, overriding model-specific and global default timeouts. + */ + @JsonProperty("timeout_millis") + private Long timeoutMillis; + +} \ No newline at end of file diff --git a/src/main/java/eu/svjatoslav/alyverkko_cli/configuration/package-info.java b/src/main/java/eu/svjatoslav/alyverkko_cli/configuration/package-info.java new file mode 100644 index 0000000..fd21d0b --- /dev/null +++ b/src/main/java/eu/svjatoslav/alyverkko_cli/configuration/package-info.java @@ -0,0 +1,13 @@ +/** + *

This package handles the configuration system for the Älyverkko CLI application. + * It provides classes for storing and retrieving application-wide settings, including + * model configurations, directory paths, and performance parameters. + *

Configuration is stored in YAML format and includes: + *

    + *
  • Model directory paths
    • + *
  • Mail task directories
    • + *
  • Performance settings (thread counts, temperature)
    • + *
+ */ + +package eu.svjatoslav.alyverkko_cli.configuration; \ No newline at end of file diff --git a/src/main/java/eu/svjatoslav/alyverkko_cli/package-info.java b/src/main/java/eu/svjatoslav/alyverkko_cli/package-info.java new file mode 100644 index 0000000..a3057d0 --- /dev/null +++ b/src/main/java/eu/svjatoslav/alyverkko_cli/package-info.java @@ -0,0 +1,14 @@ +/** + *

This package contains the core components of the Älyverkko CLI application, including the main entry point, + * command interfaces, and utility classes. It serves as the central hub for orchestrating subcommands and core + * application behavior. + *

Key responsibilities include: + *

    + *
  • Command registration and execution
    • + *
  • Configuration loading and management
    • + *
  • Basic utility functions for colored console output
    • + *
+ * + */ + +package eu.svjatoslav.alyverkko_cli; \ No newline at end of file diff --git a/tools/Implement idea b/tools/Implement idea new file mode 100755 index 0000000..6472735 --- /dev/null +++ b/tools/Implement idea @@ -0,0 +1,12 @@ +#!/bin/bash +cd "${0%/*}"; if [ "$1" != "T" ]; then gnome-terminal -e "'$0' T"; exit; fi; +cd .. + +read -p "Enter the topic name: " TOPIC_NAME + +alyverkko-cli joinfiles -t "$TOPIC_NAME" --src-dir . --pattern "*.org" +alyverkko-cli joinfiles -t "$TOPIC_NAME" --src-dir . --pattern "*.java" +alyverkko-cli joinfiles -t "$TOPIC_NAME" --src-dir . --pattern "implement*" +alyverkko-cli joinfiles -t "$TOPIC_NAME" --src-dir . --pattern "install" +alyverkko-cli joinfiles -t "$TOPIC_NAME" --src-dir . --pattern "uninstall" +alyverkko-cli joinfiles -t "$TOPIC_NAME" --edit diff --git a/tools/Open with IntelliJ IDEA b/tools/Open with IntelliJ IDEA new file mode 100755 index 0000000..304bf94 --- /dev/null +++ b/tools/Open with IntelliJ IDEA @@ -0,0 +1,54 @@ +#!/bin/bash + +# This script launches IntelliJ IDEA with the current project +# directory. The script is designed to be run by double-clicking it in +# the GNOME Nautilus file manager. + +# First, we change the current working directory to the directory of +# the script. + +# "${0%/*}" gives us the path of the script itself, without the +# script's filename. + +# This command basically tells the system "change the current +# directory to the directory containing this script". + +cd "${0%/*}" + +# Then, we move up one directory level. +# The ".." tells the system to go to the parent directory of the current directory. +# This is done because we assume that the project directory is one level up from the script. +cd .. + +# Now, we use the 'setsid' command to start a new session and run +# IntelliJ IDEA in the background. 'setsid' is a UNIX command that +# runs a program in a new session. + +# The command 'idea .' opens IntelliJ IDEA with the current directory +# as the project directory. The '&' at the end is a UNIX command that +# runs the process in the background. The '> /dev/null' part tells +# the system to redirect all output (both stdout and stderr, denoted +# by '&') that would normally go to the terminal to go to /dev/null +# instead, which is a special file that discards all data written to +# it. + +setsid idea . &>/dev/null & + +# The 'disown' command is a shell built-in that removes a shell job +# from the shell's active list. Therefore, the shell will not send a +# SIGHUP to this particular job when the shell session is terminated. + +# '-h' option specifies that if the shell receives a SIGHUP, it also +# doesn't send a SIGHUP to the job. + +# '$!' is a shell special parameter that expands to the process ID of +# the most recent background job. +disown -h $! + + +sleep 2 + +# Finally, we use the 'exit' command to terminate the shell script. +# This command tells the system to close the terminal window after +# IntelliJ IDEA has been opened. +exit diff --git a/tools/Update web site b/tools/Update web site new file mode 100755 index 0000000..ec23ca9 --- /dev/null +++ b/tools/Update web site @@ -0,0 +1,36 @@ +#!/bin/bash +cd "${0%/*}"; if [ "$1" != "T" ]; then gnome-terminal -e "'$0' T"; exit; fi; + +cd .. + +# Build the project jar file and the apidocs. +export JAVA_HOME=/usr/lib/jvm/java-21-openjdk-amd64/ +mvn clean package + +# Export org to html using emacs in batch mode +( + cd doc/ + + rm -f index.html + emacs --batch -l ~/.emacs --visit=index.org --funcall=org-html-export-to-html --kill + + #rm setup.html + #emacs --batch -l ~/.emacs --visit=setup.org --funcall=org-html-export-to-html --kill +) + +# Generate class diagrams. See: https://www3.svjatoslav.eu/projects/javainspect/ +rm -rf doc/graphs/ +mkdir -p doc/graphs/ +javainspect -j target/alyverkko-cli-*-SNAPSHOT.jar -d doc/graphs/ -n "all classes" -t png -ho +meviz index -w doc/graphs/ -t "Älyverkko CLI program classes" + +# Copy the apidocs to the doc folder so that they can be uploaded to the server. +rm -rf doc/apidocs/ +cp -r target/apidocs/ doc/ + +# Upload project homepage to the server. +rsync -avz --delete -e 'ssh -p 10006' doc/ n0@www3.svjatoslav.eu:/mnt/big/projects/alyverkko-cli/ + +echo "" +echo "Press ENTER to close this window." +read diff --git a/uninstall b/uninstall new file mode 100755 index 0000000..b4a4474 --- /dev/null +++ b/uninstall @@ -0,0 +1,13 @@ +#!/bin/bash + +SYSTEMD_SERVICE_FILE="/etc/systemd/system/alyverkko-cli.service" + +sudo systemctl stop alyverkko-cli +sudo systemctl disable alyverkko-cli +sudo rm "$SYSTEMD_SERVICE_FILE" +sudo rm -rf /opt/alyverkko-cli/ + +read -p "Do you want to remove user configuration as well? (y/N) " remove_config +if [[ $remove_config == [Yy] ]]; then + sudo rm -rf "${HOME}/.config/alyverkko-cli" +fi