From: Svjatoslav Agejenko Date: Sat, 17 May 2025 08:19:37 +0000 (+0300) Subject: initial commit X-Git-Url: http://www2.svjatoslav.eu/gitweb/?a=commitdiff_plain;h=7e66742cb72338a3bd1a3501b8d1c57cb47344ce;p=alyverkko-cli.git initial commit --- 7e66742cb72338a3bd1a3501b8d1c57cb47344ce diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..52e4c3b --- /dev/null +++ b/.gitignore @@ -0,0 +1,13 @@ +/.idea/ +/.settings/ +/target/ +/*.iml +/*.log +/test/ + +/doc/apidocs/ +/doc/graphs/ +/doc/index.html +/doc/setup.html + + diff --git a/COPYING b/COPYING new file mode 100644 index 0000000..0e259d4 --- /dev/null +++ b/COPYING @@ -0,0 +1,121 @@ +Creative Commons Legal Code + +CC0 1.0 Universal + + CREATIVE COMMONS CORPORATION IS NOT A LAW FIRM AND DOES NOT PROVIDE + LEGAL SERVICES. DISTRIBUTION OF THIS DOCUMENT DOES NOT CREATE AN + ATTORNEY-CLIENT RELATIONSHIP. CREATIVE COMMONS PROVIDES THIS + INFORMATION ON AN "AS-IS" BASIS. CREATIVE COMMONS MAKES NO WARRANTIES + REGARDING THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS + PROVIDED HEREUNDER, AND DISCLAIMS LIABILITY FOR DAMAGES RESULTING FROM + THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS PROVIDED + HEREUNDER. + +Statement of Purpose + +The laws of most jurisdictions throughout the world automatically confer +exclusive Copyright and Related Rights (defined below) upon the creator +and subsequent owner(s) (each and all, an "owner") of an original work of +authorship and/or a database (each, a "Work"). + +Certain owners wish to permanently relinquish those rights to a Work for +the purpose of contributing to a commons of creative, cultural and +scientific works ("Commons") that the public can reliably and without fear +of later claims of infringement build upon, modify, incorporate in other +works, reuse and redistribute as freely as possible in any form whatsoever +and for any purposes, including without limitation commercial purposes. +These owners may contribute to the Commons to promote the ideal of a free +culture and the further production of creative, cultural and scientific +works, or to gain reputation or greater distribution for their Work in +part through the use and efforts of others. + +For these and/or other purposes and motivations, and without any +expectation of additional consideration or compensation, the person +associating CC0 with a Work (the "Affirmer"), to the extent that he or she +is an owner of Copyright and Related Rights in the Work, voluntarily +elects to apply CC0 to the Work and publicly distribute the Work under its +terms, with knowledge of his or her Copyright and Related Rights in the +Work and the meaning and intended legal effect of CC0 on those rights. + +1. Copyright and Related Rights. A Work made available under CC0 may be +protected by copyright and related or neighboring rights ("Copyright and +Related Rights"). Copyright and Related Rights include, but are not +limited to, the following: + + i. the right to reproduce, adapt, distribute, perform, display, + communicate, and translate a Work; + ii. moral rights retained by the original author(s) and/or performer(s); +iii. publicity and privacy rights pertaining to a person's image or + likeness depicted in a Work; + iv. rights protecting against unfair competition in regards to a Work, + subject to the limitations in paragraph 4(a), below; + v. rights protecting the extraction, dissemination, use and reuse of data + in a Work; + vi. database rights (such as those arising under Directive 96/9/EC of the + European Parliament and of the Council of 11 March 1996 on the legal + protection of databases, and under any national implementation + thereof, including any amended or successor version of such + directive); and +vii. other similar, equivalent or corresponding rights throughout the + world based on applicable law or treaty, and any national + implementations thereof. + +2. Waiver. To the greatest extent permitted by, but not in contravention +of, applicable law, Affirmer hereby overtly, fully, permanently, +irrevocably and unconditionally waives, abandons, and surrenders all of +Affirmer's Copyright and Related Rights and associated claims and causes +of action, whether now known or unknown (including existing as well as +future claims and causes of action), in the Work (i) in all territories +worldwide, (ii) for the maximum duration provided by applicable law or +treaty (including future time extensions), (iii) in any current or future +medium and for any number of copies, and (iv) for any purpose whatsoever, +including without limitation commercial, advertising or promotional +purposes (the "Waiver"). Affirmer makes the Waiver for the benefit of each +member of the public at large and to the detriment of Affirmer's heirs and +successors, fully intending that such Waiver shall not be subject to +revocation, rescission, cancellation, termination, or any other legal or +equitable action to disrupt the quiet enjoyment of the Work by the public +as contemplated by Affirmer's express Statement of Purpose. + +3. Public License Fallback. Should any part of the Waiver for any reason +be judged legally invalid or ineffective under applicable law, then the +Waiver shall be preserved to the maximum extent permitted taking into +account Affirmer's express Statement of Purpose. In addition, to the +extent the Waiver is so judged Affirmer hereby grants to each affected +person a royalty-free, non transferable, non sublicensable, non exclusive, +irrevocable and unconditional license to exercise Affirmer's Copyright and +Related Rights in the Work (i) in all territories worldwide, (ii) for the +maximum duration provided by applicable law or treaty (including future +time extensions), (iii) in any current or future medium and for any number +of copies, and (iv) for any purpose whatsoever, including without +limitation commercial, advertising or promotional purposes (the +"License"). The License shall be deemed effective as of the date CC0 was +applied by Affirmer to the Work. Should any part of the License for any +reason be judged legally invalid or ineffective under applicable law, such +partial invalidity or ineffectiveness shall not invalidate the remainder +of the License, and in such case Affirmer hereby affirms that he or she +will not (i) exercise any of his or her remaining Copyright and Related +Rights in the Work or (ii) assert any associated claims and causes of +action with respect to the Work, in either case contrary to Affirmer's +express Statement of Purpose. + +4. Limitations and Disclaimers. + + a. No trademark or patent rights held by Affirmer are waived, abandoned, + surrendered, licensed or otherwise affected by this document. + b. Affirmer offers the Work as-is and makes no representations or + warranties of any kind concerning the Work, express, implied, + statutory or otherwise, including without limitation warranties of + title, merchantability, fitness for a particular purpose, non + infringement, or the absence of latent or other defects, accuracy, or + the present or absence of errors, whether or not discoverable, all to + the greatest extent permissible under applicable law. + c. Affirmer disclaims responsibility for clearing rights of other persons + that may apply to the Work or any use thereof, including without + limitation any person's Copyright and Related Rights in the Work. + Further, Affirmer disclaims responsibility for obtaining any necessary + consents, permissions or other rights required for any use of the + Work. + d. Affirmer understands and acknowledges that Creative Commons is not a + party to this document and has no duty or obligation with respect to + this CC0 or use of the Work. diff --git a/alyverkko-cli b/alyverkko-cli new file mode 100755 index 0000000..aea6d86 --- /dev/null +++ b/alyverkko-cli @@ -0,0 +1,6 @@ +#!/bin/bash + +set -f + +java -Xmx4500m -classpath /opt/alyverkko-cli/* eu.svjatoslav.alyverkko_cli.Main "$@" + diff --git a/alyverkko-cli.yaml b/alyverkko-cli.yaml new file mode 100644 index 0000000..f9dc574 --- /dev/null +++ b/alyverkko-cli.yaml @@ -0,0 +1,16 @@ +mail_directory: "/home/user/AI/mail" +models_directory: "/home/user/AI/models" +default_temperature: 0.7 +llama_cpp_dir_path: "/home/user/AI/llama.cpp/" +batch_thread_count: 10 +thread_count: 6 +prompts_directory: "/home/user/.config/alyverkko-cli/prompts" +models: + - alias: "default" + filesystem_path: "WizardLM-2-8x22B.Q5_K_M-00001-of-00005.gguf" + context_size_tokens: 64000 + end_of_text_marker: null + - alias: "mistral" + filesystem_path: "Mistral-Large-Instruct-2407.Q8_0.gguf" + context_size_tokens: 32768 + end_of_text_marker: null diff --git a/doc/QwQ-32B Termination issue.odt b/doc/QwQ-32B Termination issue.odt new file mode 100644 index 0000000..c230de4 Binary files /dev/null and b/doc/QwQ-32B Termination issue.odt differ diff --git a/doc/index.org b/doc/index.org new file mode 100644 index 0000000..15a90db --- /dev/null +++ b/doc/index.org @@ -0,0 +1,506 @@ +#+SETUPFILE: ~/.emacs.d/org-styles/html/darksun.theme +#+TITLE: Älyverkko CLI application +#+LANGUAGE: en +#+LATEX_HEADER: \usepackage[margin=1.0in]{geometry} +#+LATEX_HEADER: \usepackage{parskip} +#+LATEX_HEADER: \usepackage[none]{hyphenat} + +#+OPTIONS: H:20 num:20 +#+OPTIONS: author:nil + +* Introduction + +The *Älyverkko CLI* application is a user-friendly tool developed in +Java, specifically tailored to streamline the utilization of expansive +language models through CPU-based computation in batch processing +mode. + +To illustrate its capabilities: Imagine harnessing the power of a vast +language model, boasting approximately 100 billion parameters, solely +relying on CPU computations and leveraging the open-source software +*llama.cpp*. This setup requires a modern consumer-grade CPU and +approximately 128 GB of RAM. To put this into perspective, 128 GB of +RAM is financially comparable to purchasing a high-quality smartphone, +making it an economical option for many users. + +In contrast, executing the same model on an Nvidia GPU could result in +costs that are significantly higher, potentially by an order of +magnitude. + +However, there is a trade-off: CPU-based processing for such extensive +models is inherently slow. This means real-time interaction, like +chatting with the AI, wouldn't be practical or enjoyable due to the +lag in response times. Nevertheless, when deployed in a +non-interactive batch mode, this "slow but smart" AI can complete +numerous valuable tasks within a 24-hour window. For instance, it +could generate a substantial amount of code, potentially exceeding +what you could thoroughly review in the same timeframe. Additionally, +it could process more documents than most individuals would be +inclined to read manually. + +The primary objective of the *Älyverkko CLI* project is to identify +and enable applications where this "slow but smart" AI can excel. By +utilizing *llama.cpp* as its inference engine, the project aims to +unlock a variety of uses where batch processing is more advantageous +than real-time interaction. + +Here are some practical examples of tasks suited for the +*Älyverkko CLI* application: + +- [[id:e1b6aa9a-a27d-4019-8bd4-fceb0c606e23][Domain: Natural Language Processing (NLP)]] +- [[id:f4c7d3e0-8c18-4123-a797-871ca73a7580][Domain: Code Generation]] +- [[id:f38360ad-54f6-4f24-b299-f73a9faacabd][Domain: Content Creation]] + +In summary, the *Älyverkko CLI* application opens up a realm of +possibilities for leveraging powerful AI in scenarios where immediate +responses are not critical, but high-quality batch processing output +is highly beneficial. + +Note: project is still in early stage. + +** Use cases +*** Domain: Natural Language Processing (NLP) +:PROPERTIES: +:ID: e1b6aa9a-a27d-4019-8bd4-fceb0c606e23 +:END: + +*Problem Statement:* + +Analyze a set of customer reviews to determine overall sentiment and +extract key features that customers appreciate or dislike about a +product. + +*Usage Procedure:* +1. User collects customer reviews in plain text format within the + configured mail directory. Lets say, about 150 kilobytes of reviews + per input file (this is dictated by AI model available context + size). +2. Each review file is prefixed with "TOCOMPUTE:". +3. The Älyverkko CLI application processes these files, generating + sentiment analysis results and feature extraction insights. +4. Results are appended to the original files in org-mode syntax, + indicating AI responses. + +*** Domain: Code Generation +:PROPERTIES: +:ID: f4c7d3e0-8c18-4123-a797-871ca73a7580 +:END: + +*Problem Statement:* + +Generate code snippets for a new software module based on detailed +specifications provided by the developer. + +*Usage Procedure:* + +1. Developer writes specifications in a text file within the mail + directory, prefixed with "TOCOMPUTE:". Text file also contains + relevant parts of the program source code and + documentation. Älyverkko CLI *joinfiles* subcommand can be used to + facilitate such text file preparation. + +2. The Älyverkko CLI application processes this file and generates the + corresponding code snippets. The generated code is appended to the + original specifications file, organized using org-mode syntax. + +3. Developer can review proposed changes and then integrate them back + into original program source code. + +Note: Most of the [[https://www3.svjatoslav.eu/projects/alyverkko-cli/][Älyverkko CLI]] program code is written in such a way +by AI. [[https://www2.svjatoslav.eu/gitweb/?p=alyverkko-cli.git;a=blob;f=tools/implement+idea;h=02b0ceb260693a6c9733f221e52a0e6c5fce0a36;hb=HEAD][This script]] is used to facilitate the process. + +*** Domain: Content Creation +:PROPERTIES: +:ID: f38360ad-54f6-4f24-b299-f73a9faacabd +:END: + +*Problem Statement:* + +Draft an outline for a book on science fiction and improve its plot. + +*Usage Procedure:* + +1. The book author writes a brief describing the outline of the plot + and his book main idea for the novel. + +2. *Älyverkko CLI* processes this description and generates an outline + with suggested headings and suggests possible improvements to the + plot. + +Here is [[https://www.svjatoslav.eu/writing/Whispers%20in%20the%20Stream%20of%20Time.html][example sci-fi book]] that was written with the help of +*Älyverkko CLI*. + +* Installation + +See [[id:e469ec1e-402a-476d-a849-662a48eb4f90][Älyverkko CLI application installation tutorial]]. + +* Usage +** Task preparation +:PROPERTIES: +:ID: 4b7900e4-77c1-45e7-9c54-772d0d3892ea +:END: + +The Älyverkko CLI application expects input files for processing in +the form of plain text files within the specified mail directory +(configured in the [[id:0fcdae48-81c5-4ae1-bdb9-64ae74e87c45][YAML configuration file]]). + +Suggested usage flow is to prepare AI assignments within the Älyverkko +CLI mail directory using normal text editor. Once AI assignment is +ready for processing, you should [[id:883d6e7c-60e0-422b-8c00-5cdc9dfec20d][initiate AI processing]] on that file. + +*** "joinfiles" command +*Note:* See also alternative solution with similar goal: [[https://github.com/aerugo/prelude][prelude]]. + +The *joinfiles* command is a utility for aggregating the contents of +multiple files into a single document, which can then be processed by +AI. This is particularly useful for preparing comprehensive problem +statements from various source files, such as software project +directories or collections of text documents. + +**** Usage + +To use the *joinfiles* command, specify the source directory +containing the files you wish to join and a topic name that will be +used to generate the output file name: + +#+begin_example +alyverkko-cli joinfiles -s /path/to/source/directory -t "my_topic" +#+end_example + +If desired, you can also specify a glob pattern to match only certain files within the directory: + +#+begin_example +alyverkko-cli joinfiles -s /path/to/source/directory -p "*.java" -t "my_topic" +#+end_example + +After joining the files, you can choose to open the resulting document +in text editor for further editing or review: + +#+begin_example +alyverkko-cli joinfiles -t "my_topic" --edit +#+end_example + +**** Options + +- **-s, --src-dir**: Specifies the source directory from which to join + files. + +- **-p, --pattern**: An optional glob pattern to match specific files + within the source directory. + +- **-t, --topic**: The topic name that will be used as a basis for the + output file name and should reflect the subject matter of the joined + content. + +- **-e, --edit**: Opens the joined file in text editor after the join + operation is complete. + +**** Example Use Case + +Imagine you have a software project with various source files that you +want to analyze using AI. You can use the *joinfiles* command to +create a single document for processing: + +#+begin_example +alyverkko-cli joinfiles -s /path/to/project -p "*.java" -t "software_analysis" --edit +#+end_example + +This will recursively search the project directory for Java source +files, aggregate their contents into a file named +*software_analysis.org* (within AI processor input files directory), +and open text editor on the file, so that you can add your analysis +instructions or problem statement. Finally you [[id:883d6e7c-60e0-422b-8c00-5cdc9dfec20d][Initiate AI processing]] +and after some time, you will get results and the end of the file. + +** Initiate AI processing +:PROPERTIES: +:ID: 883d6e7c-60e0-422b-8c00-5cdc9dfec20d +:END: + +Once your task file is prepared, you should place *TOCOMPUTE:* marker +on the first line of that file, so that it will be considered for +processing. + +When the Älyverkko CLI detects a new or modified file in the mail +directory: + +1. It checks if file has "TOCOMPUTE:" on the first line. If no, file + is ignored. Otherwise Älyverkko CLI continues processing the file. + +2. It reads the content of the file and feeds it as an input for an AI + model to generate a response. + +4. Once the AI has generated a response, the application appends this + response to the original mail contents within the same file, using + org-mode syntax to distinguish between the user's query and the + assistant's reply. The updated file will contain both the original + query (prefixed with: "* USER:*") and the AI's response (prefixed + with "* ASSISTANT:"), ensuring a clear and organized conversation + thread. "TOCOMPUTE:" is removed from the beginning of the file to + avoid processing same file again. + +Note: During AI task file preparation, feel free to save intermediary +states as often as needed because AI engine will keep ignoring the +file until *TOCOMPUTE:* line appears. Once AI assignment is ready, add +: TOCOMPUTE: +to the beginning of the file and save one last time. Älyverkko CLI +will detect new task approximately within one second after file is +saved and will start processing it. + +If your text editor automatically reloads file when it was changed by +other process in the filesystem, AI response will appear within text +editor as soon as AI response is ready. If needed, you can add further +queries at the end of the file and re-add "TOCOMPUTE:" at the +beginning of the file. This way AI will process file again and file +becomes stateful conversation. If you use GNU Emacs text editor, you +can benefit from [[id:25038854-c905-4b26-9670-cca06600223e][purpose-built GNU Emacs utilities]]. + +** Helpful GNU Emacs utilities +:PROPERTIES: +:ID: 25038854-c905-4b26-9670-cca06600223e +:END: + +Note: GNU Emacs and following Emacs Lisp utilities are not required to +use Älyverkko CLI. Their purpose is to increase comfort for existing +GNU Emacs users. + +*** Easily compose new problem statement for AI from emacs + +The Elisp function *ai-new-topic* facilitates the creation and opening +of a new Org-mode file dedicated to a user-defined topic within a +specified directory. Now you can use this file within emacs to compose +you problem statement to AI. + +When *ai-new-topic* function triggered, it first prompts the user to +input a topic name. This name will serve as the basis for the filename +and the title within the document. + +The function then constructs a file path by concatenating the +pre-defined =alyverkko-topic-files-directory= (which should be set to +your topics directory), the topic name, and the =.org= extension. If a +file with this path does not already exist, the function will create a +new file and open it for editing. + +#+begin_src elisp :results none + + (defvar alyverkko-topic-files-directory "/home/user/my-ai-mail-directory/" + "Directory where topic files are stored.") + + (defun alyverkko-new-topic () + "Create and open a topic file in the specified directory." + (interactive) + (let ((topic (read-string "Enter topic name: "))) + (let ((file-path (concat alyverkko-topic-files-directory topic ".org"))) + (if (not (file-exists-p file-path)) + (with-temp-file file-path + )) + (find-file file-path) + (goto-char (point-max)) + (org-mode)))) + +#+end_src + +*** Easily signal to AI that problem statement is ready for solving + +The function =alyverkko-compute= is designed to enhance the workflow +of users working with the Älyverkko CLI application by automating the +process of marking text files for computation with a specific AI model +and prompt. + +When function is invoked, it detects available prompts and allows user +to select one. + +Thereafter function detects available models from Älyverkko CLI +configuration file and allows user to select one. + +Finally function inserts at the beginning of currently opened file +something like this: + +: TOCOMPUTE: prompt= model= + +- Adjust *prompt-dir* variable to point to your prompts directory. +- Adjust *config-file* variable to point to your Älyverkko CLI + configuration file path. + +#+begin_src emacs-lisp + + (defun alyverkko-compute () + "Select a prompt and a model interactively, then insert them at the + beginning of the current buffer." + (interactive) + (let ((prompt-dir "~/.config/alyverkko-cli/prompts/") + (config-file "~/.config/alyverkko-cli/alyverkko-cli.yaml") + (models '()) + ) + + (with-temp-buffer + (insert-file-contents config-file) + ;; Move to the beginning of the prompts section + (goto-char (point-min)) + (when (search-forward-regexp "^models:" nil t) + ;; Collect all aliases + (while (search-forward-regexp "^\\s-+- alias: \"\\([^\"]+\\)\"" nil t) + (push (match-string 1) models)))) + + (if (file-exists-p prompt-dir) + (let* ((files (directory-files prompt-dir t "\\`[^.].*\\.txt\\'")) + (aliases (mapcar (lambda (f) (file-name-base f)) files))) + (if aliases + (let ((selected-alias (completing-read "Select prompt alias: " aliases)) + (model (completing-read "Select AI model: " models))) + (alyverkko-insert-tocompute-line selected-alias model)) + (message "No prompt files found."))) + (message "Prompt directory not found.")))) + + (defun alyverkko-insert-tocompute-line (prompt-alias model) + "Inserts TOCOMPUTE line with selected PROMPT-ALIAS and MODEL at the + beginning of the buffer." + (save-excursion + (goto-char (point-min)) + (insert (format "TOCOMPUTE: prompt=%s model=%s\n" prompt-alias model)) + (save-buffer))) +#+end_src + +* Getting the source code +- This program is free software: released under Creative Commons Zero + (CC0) license. + +- Program author: + - Svjatoslav Agejenko + - Homepage: https://svjatoslav.eu + - Email: mailto://svjatoslav@svjatoslav.eu + +- [[https://www.svjatoslav.eu/projects/][Other software projects hosted at svjatoslav.eu]] + +** Source code +:PROPERTIES: +:ID: f5740953-079b-40f4-87d8-b6d1635a8d39 +:END: +- [[https://www2.svjatoslav.eu/gitweb/?p=alyverkko-cli.git;a=snapshot;h=HEAD;sf=tgz][Download latest snapshot in TAR GZ format]] + +- [[https://www2.svjatoslav.eu/gitweb/?p=alyverkko-cli.git;a=summary][Browse Git repository online]] + +- Clone Git repository using command: + : git clone https://www3.svjatoslav.eu/git/alyverkko-cli.git + +- See [[https://www3.svjatoslav.eu/projects/alyverkko-cli/apidocs/][JavaDoc]]. + +- See [[https://www3.svjatoslav.eu/projects/alyverkko-cli/graphs/][automatically generated class diagrams]]. Here is [[https://www3.svjatoslav.eu/projects/javainspect/legend.png][legend]] to help + understand diagrams. Diagrams were generated using [[https://www3.svjatoslav.eu/projects/javainspect/][JavaInspect tool]]. + +* TODO + +Ideas to be possibly implemented in the future: + +** Documentation + +- Add example problem statements and resulting solutions on various + domains. Accompany them with precise usage procedure: + - What user did, when, where and how. + - How and where did result appear. + +- Add a section detailing the architecture and design decisions behind + Älyverkko CLI. + +** System operation + +- Consider implementing a plugin architecture to allow third-party + developers to extend Älyverkko CLI's functionality with custom + modules or integrations. + +- Possibility to easily pause and resume Älyverkko CLI without loosing + in-progress computation. Unix process stop and continue signals + could possibly be used. + +- Explain how to monitor system performance and resource usage during + AI processing tasks. + +- Introduce a comprehensive suite of automated tests to ensure the + reliability and stability of new features before they are released. + +- Address potential security concerns, such as handling sensitive data + in AI processing tasks and securing communication with external + services. + +** Data management + +- in maildir ignore binary files, use joinfiles command as example how + to ignore binary files. Perhaps extract plain text file detection + into some utility class. + +- Make text editor configurable in application properties file. + +** Configuration and logging + +- Implement a fallback mechanism to use a system-wide configuration + file located in `/etc/` if no user-specific configuration is found, + enhancing out-of-the-box usability for new users. + +- Introduce optional logging of `llama.cpp` output to aid in debugging + and performance monitoring without cluttering the standard output. + +- Implement a logging framework with configurable log levels to help + users debug issues without cluttering the standard output. + +** Integration with external services + +- Add capabilities to connect with Jira, fetch content, and + potentially update issues or comments based on AI processing + results. +- Implement similar integration with Confluence for content retrieval + and updates. +- Extend the application's reach by adding the ability to interact + with arbitrary web sites, enabling information extraction and + automation of web-based tasks. + +** Tooling enhancements + +- Incorporate Python script execution capabilities directly by the AI, + expanding the range of available data manipulation and processing + tools. + +- Integrate relational database access to leverage structured data + storage and retrieval in workflows. + +- Enable web request functionality to interact with RESTful APIs or + scrape web content as part of task execution. + +- Introduce a notebook feature that allows the AI to maintain and + reference its own notes, fostering context retention across tasks. + +** Multimedia processing + +- Extend the application's capabilities to include voice capture and + processing, opening up new avenues for interaction beyond text-based + communication. + +- Implement image capture and processing features, enabling tasks that + involve image analysis or content extraction from visual data. + +** Task preparation and queue management + +- Refactor the task queue mechanism to support: + - Multiple task sources, including a REST API endpoint for + programmatic task submission. + - Load balancing across multiple executors (instances of Älyverkko + CLI) with dynamic registration and unregistration without system + interruption. + - Task priority assignments, ensuring that critical tasks are + processed in a timely manner. + +- Offer guidance on preparing input files for batch processing, + including best practices for formatting data and structuring problem + statements. + +** User interface development + +- Create a web-based UI to provide users with an interface for task + submission and result retrieval, improving accessibility and user + experience. + +- Integrate Quality of Service (QoS) concepts within the UI to ensure + equitable resource allocation among users. + +- Implement administrative features for managing user accounts and + system resources, maintaining a secure and efficient operating + environment. diff --git a/doc/pausing and resuming.odt b/doc/pausing and resuming.odt new file mode 100644 index 0000000..f519092 Binary files /dev/null and b/doc/pausing and resuming.odt differ diff --git a/doc/setup.org b/doc/setup.org new file mode 100644 index 0000000..ef1c364 --- /dev/null +++ b/doc/setup.org @@ -0,0 +1,289 @@ +:PROPERTIES: +:ID: e469ec1e-402a-476d-a849-662a48eb4f90 +:END: +#+SETUPFILE: ~/.emacs.d/org-styles/html/darksun.theme +#+TITLE: Älyverkko CLI application setup +#+LANGUAGE: en +#+LATEX_HEADER: \usepackage[margin=1.0in]{geometry} +#+LATEX_HEADER: \usepackage{parskip} +#+LATEX_HEADER: \usepackage[none]{hyphenat} + +#+OPTIONS: H:20 num:20 +#+OPTIONS: author:nil + +* Requirements +*Operating System:* + +Älyverkko CLI is developed and tested on Debian 12 "Bookworm". It +should work on any modern Linux distribution with minimal adjustments +to the installation process. + +*Dependencies:* +- Java Development Kit (JDK) 17 or higher +- Apache Maven for building the project + +*Hardware Requirements:* +- Modern multi-core CPU. +- The more RAM you have, the smarter AI model you can use. For + example, at least 64 GB of RAM is needed to run pretty decent + [[https://huggingface.co/MaziyarPanahi/WizardLM-2-8x22B-GGUF/tree/main][WizardLM-2-8x22B AI model]]. +- Sufficient disk space to store large language models and + input/output data. + +* Installation +:PROPERTIES: +:ID: 0b705a37-9b84-4cd5-878a-fedc9ab09b12 +:END: +At the moment, to use Älyverkko CLI, you need to: +- Download sources and build [[https://github.com/ggerganov/llama.cpp][llama.cpp]] project. +- Download [[id:f5740953-079b-40f4-87d8-b6d1635a8d39][sources]] and build Älyverkko CLI project. +- Download one or more pre-trained large language models in GGUF + format. Hugging Face repository [[https://huggingface.co/models?search=GGUF][has lot of them]]. My favorite is + [[https://huggingface.co/MaziyarPanahi/WizardLM-2-8x22B-GGUF/tree/main][WizardLM-2-8x22B]] for strong problem solving skills. + +Follow instructions for obtaining and building Älyverkko CLI on your +computer that runs Debian 12 operating system: + +1. Ensure that you have Java Development Kit (JDK) installed on your + system. + : sudo apt-get install openjdk-17-jdk + +2. Ensure that you have Apache Maven installed: + : sudo apt-get install maven + +3. Clone the [[id:f5740953-079b-40f4-87d8-b6d1635a8d39][code repository]] or download the [[id:f5740953-079b-40f4-87d8-b6d1635a8d39][source code]] for the + `alyverkko-cli` application to your local machine. + +4. Navigate to the root directory of the cloned/downloaded project in + your terminal. + +5. Execute the installation script by running + : ./install + + This script will compile the application and install it to + directory + : /opt/alyverkko-cli + + To facilitate usage from command-line, it will also define + system-wide command *alyverkko-cli* as well as "Älyverkko CLI" + launcher in desktop applications menu. + +6. Prepare Älyverkko CLI [[id:0fcdae48-81c5-4ae1-bdb9-64ae74e87c45][configuration]] file. + +7. Verify that the application has been installed correctly by running + *alyverkko-cli* in your terminal. + +* Configuration +:PROPERTIES: +:ID: 0fcdae48-81c5-4ae1-bdb9-64ae74e87c45 +:END: +Älyverkko CLI application configuration is done by editing YAML +formatted configuration file. + +Configuration file should be placed under current user home directory: +: ~/.config/alyverkko-cli.yaml + +** Configuration file example + +The application is configured using a YAML-formatted configuration +file. Below is an example of how the configuration file might look: + +#+begin_src yaml + mail_directory: "/home/user/AI/mail" + models_directory: "/home/user/AI/models" + default_temperature: 0.7 + llama_cli_path: "/home/user/AI/llama.cpp/build/bin/llama-cli" + batch_thread_count: 10 + thread_count: 6 + prompts_directory: "/home/user/.config/alyverkko-cli/prompts" + models: + - alias: "default" + filesystem_path: "WizardLM-2-8x22B.Q5_K_M-00001-of-00005.gguf" + context_size_tokens: 64000 + end_of_text_marker: null + - alias: "mistral" + filesystem_path: "Mistral-Large-Instruct-2407.Q8_0.gguf" + context_size_tokens: 32768 + end_of_text_marker: null +#+end_src + +** Configuration file syntax + +Here are available parameters: + +- mail_directory :: Directory where AI will look for files that + contain problems to solve. + +- models_directory :: Directory where AI models are stored. + - This option is mandatory. + +- prompts_directory :: Directory where prompts are stored. + - Each prompt is a .txt file with the same name as its alias. + - Example prompts directory content: + #+begin_verse + default.txt + writer.txt + #+end_verse + + Example content for *writer.txt*: + : You are best-selling book writer. + + See more [[id:2109b238-3f2a-4ecb-9f37-d17b52175c82][example prompts that you can try.]] + +- default_temperature :: Defines the default temperature for AI + responses, affecting randomness in the generation process. Lower + values make the AI more deterministic and higher values make it more + creative or random. + - Default value: 0.7 + +- llama_cli_path :: Specifies the filesystem path to the *llama.cpp* + project *llama-cli* executable file. + - Example Value: /home/user/AI/llama.cpp/build/bin/llama-cli + - This option is mandatory. + +- batch_thread_count :: Specifies the number of threads to use for + input prompt processing. CPU computing power is usually the + bottleneck here. + - Default value: 10 + +- thread_count :: Sets the number of threads to be used by the AI + during response generation. RAM data transfer speed is usually + bottleneck here. When RAM bandwidth is saturated, increasing thread + count will no longer increase processing speed, but it will still + keep CPU cores unnecessarily busy. + - Default value: 6 + +- models :: List of available large language models. + - alias :: Short model alias. Model with alias "default" would be used by default. + - filesystem_path :: File name of the model as located within + *models_directory* + - context_size_tokens :: Context size in tokens that model was + trained on. + - end_of_text_marker :: Some models produce certain markers to + indicate end of their output. If specified here, Älyverkko CLI can + identify and remove them so that they don't leak into + conversation. Default value is: *null*. + + +*** Example prompt files that you can try +:PROPERTIES: +:ID: 2109b238-3f2a-4ecb-9f37-d17b52175c82 +:END: + +Deliver insights: +#+begin_example +This conversation involves a user and AI assistant where the AI is +expected to provide not only immediate responses but also detailed and +well-reasoned analysis. The AI should consider all aspects of the +query and deliver insights based on logical deductions and +comprehensive understanding. +#+end_example + +Summarize text into LOLcat speak: +#+begin_example +Ur task iz to rite sumree ov wut teh hooman sez, lyk storee, blog, or +tootorial, etc. + +Make teh sumree so fun an captivatin dat teh reeder finks teh author iz +spikin directly to dem. Thiz iz why u must hide teh fact dat it iz a +sumree. No mentshun o' teh authur. Just focus on teh meowssage an sharez. + +Forbidden stuf iz like dis: + +"a thought-provoking piece titled ..." +"the author delves into the ..." +"a comment from a reader named ..." +"This observation leads the author to ..." +"Reflecting on her own experiences as ..." + +Cuz dem stuf talks 'bout da hooman or his writin an distracts teh +reeder. Rite teh sumree so it's just like a shortr version uv teh +preev-yus text, like teh authur self wrote it. + +If iz a storee, kepp teh feelz an emocshuns like teh originnal +tail wud. + +If iz a guide or tutoorial, make sure sumree haz all teh intrestin +explanashuns an steppies, so dat by readin ur version, the same quest +an target can be done just the same as teh originnal mewtoorial. + +If iz nooz, keep teh fax like who, where, when, an most impooortantly +why. Alwayz try an esplains teh whys if possible, add ur pawsome +foughts on motiveyz or why it hapened, so we can lern sumfing an +remeembr teh lesuns. + +Ensure teh key point o' teh originnal writings is kep awl shiny an +bright fer teh reedurs. Also, u gotta translate to lolcat bi talkin +funny. Rite fur teh lulz! +#+end_example + +** Enlisting available models +Once Älyverkko CLI is installed and properly configured, you can run +following command at commandline to see what models are available to +it: + +: alyverkko-cli listmodels + +Note: Models that reference missing files will be automatically marked +with "-missing" suffix in their alias by configuration wizard. You can +manually remove this suffix after fixing the model file path. + +** Self test +The *selftest* command performs a series of checks to ensure the +system is configured correctly: + +: alyverkko-cli selftest + +It verifies: +- Configuration file integrity. +- Model directory existence. +- The presence of the *llama.cpp* executable. + +* Starting daemon + +Älyverkko CLI keeps continuously listening for and processing tasks +from a specified mail directory. + +There are multiple alternative ways to start Älyverkko CLI in mail +processing mode: + +*** Start via command line interface + +1. Open your terminal. + +2. Run the command: + : alyverkko-cli mail + +3. The application will start monitoring the configured mail directory + for incoming messages and process them accordingly in endless loop. + +4. To terminate Älyverkko CLI, just hit *CTRL+c* on the keyboard, or + close terminal window. + +*** Start using your desktop environment application launcher + +1. Access the application launcher or application menu on your desktop + environment. + +2. Search for "Älyverkko CLI". + +3. Click on the icon to start the application. It will open its own + terminal. + +4. If you want to stop Älyverkko CLI, just close terminal window. + +*** Start in the background as systemd system service + +During Älyverkko CLI [[id:0b705a37-9b84-4cd5-878a-fedc9ab09b12][installation]], installation script will prompt you +if you want to install *systemd* service. If you chose *Y*, Alyverkko +CLI would be immediately started in the background as a system +service. Also it will be automatically started on every system reboot. + +To view service status, use: +: systemctl -l status alyverkko-cli + +If you want to stop or disable service, you can do so using systemd +facilities: + +: sudo systemctl stop alyverkko-cli +: sudo systemctl disable alyverkko-cli diff --git a/install b/install new file mode 100755 index 0000000..188e05b --- /dev/null +++ b/install @@ -0,0 +1,108 @@ +#!/bin/bash + +SYSTEMD_SERVICE_FILE="/etc/systemd/system/alyverkko-cli.service" + + +# Function to install binary and jar +install_to_opt() { + sudo rm -rf /opt/alyverkko-cli/ + sudo mkdir -p /opt/alyverkko-cli/ + sudo chmod 755 /opt/alyverkko-cli/ + + sudo cp target/alyverkko-cli.jar "/opt/alyverkko-cli/alyverkko-cli.jar" + sudo cp "alyverkko-cli" "/opt/alyverkko-cli/alyverkko-cli" + sudo chmod +x "/opt/alyverkko-cli/alyverkko-cli" + sudo cp logo.png "/opt/alyverkko-cli/logo.png" + + sudo ln -sf "/opt/alyverkko-cli/alyverkko-cli" /usr/bin/alyverkko-cli +} + +# Function to install the desktop launcher +install_desktop_entry() { + local desktop_entry_path="/usr/share/applications/alyverkko-cli.desktop" + + cat < /dev/null +[Desktop Entry] +Type=Application +Terminal=true +Name=Älyverkko CLI +Comment=Runner for artificial neural network service +Icon=/opt/alyverkko-cli/logo.png +Exec=/opt/alyverkko-cli/alyverkko-cli mail +Categories=Development; +EOF + + sudo chmod 644 "$desktop_entry_path" +} + +# Function to install systemd service +install_systemd_service() { + + cat < /dev/null +[Unit] +Description=Älyverkko CLI daemon in mail mode +After=network.target + +[Service] +User=$USER +ExecStart=/opt/alyverkko-cli/alyverkko-cli mail +WorkingDirectory=/opt/alyverkko-cli +Nice=10 +Restart=always +RestartSec=10 + +[Install] +WantedBy=multi-user.target +EOF + + sudo systemctl daemon-reload + sudo systemctl enable alyverkko-cli + sudo systemctl start alyverkko-cli + sleep 1 + echo "Systemd service installed, enabled and started. Service status is:" + systemctl --no-pager -l status alyverkko-cli +} + + +# Function to pre-deploy example configuration YAML file +install_config_file() { + local alyverkko_config_dir="${HOME}/.config/alyverkko-cli" + + if [ ! -d "$alyverkko_config_dir" ]; then + mkdir -p "$alyverkko_config_dir" + cp alyverkko-cli.yaml "$alyverkko_config_dir/" + else + echo "Configuration directory already exists: $alyverkko_config_dir" + fi +} + +# Main installation function +main() { + # Build the application + mvn --settings maven.xml clean package + + install_to_opt + install_desktop_entry + install_config_file + + # Check if systemd service already exists + if [ -f "$SYSTEMD_SERVICE_FILE" ]; then + echo "Systemd service is already installed." + # Display the status without hanging + echo "Service status is:" + systemctl --no-pager -l status alyverkko-cli + else + # Install systemd service if requested + echo "Do you want to install Älyverkko CLI as a systemd service? (y/N)" + read install_service + + if [[ $install_service == [Yy] ]]; then + install_systemd_service + fi + fi + + echo "Installation complete." +} + +# Call the main installation function +main diff --git a/logo.png b/logo.png new file mode 100644 index 0000000..468758e Binary files /dev/null and b/logo.png differ diff --git a/maven.xml b/maven.xml new file mode 100644 index 0000000..505327a --- /dev/null +++ b/maven.xml @@ -0,0 +1,15 @@ + + + + + + + + + + + + \ No newline at end of file diff --git a/pom.xml b/pom.xml new file mode 100644 index 0000000..624e71c --- /dev/null +++ b/pom.xml @@ -0,0 +1,209 @@ + + 4.0.0 + eu.svjatoslav + alyverkko-cli + 1.0-SNAPSHOT + Älyverkko CLI + AI engine wrapper + + + UTF-8 + UTF-8 + + + + svjatoslav.eu + https://svjatoslav.eu + + + + + eu.svjatoslav + svjatoslavcommons + 1.8 + + + eu.svjatoslav + cli-helper + 1.3 + + + org.testng + testng + 7.7.0 + test + + + org.junit.jupiter + junit-jupiter + RELEASE + test + + + com.fasterxml.jackson.core + jackson-databind + 2.13.4.1 + + + com.fasterxml.jackson.dataformat + jackson-dataformat-yaml + 2.13.0 + + + org.apache.commons + commons-lang3 + 3.12.0 + + + org.apache.commons + commons-io + 1.3.2 + + + org.projectlombok + lombok + 1.18.32 + provided + + + + + + + org.apache.maven.plugins + maven-compiler-plugin + 3.8.1 + + 11 + 11 + true + UTF-8 + + + + + org.apache.maven.plugins + maven-source-plugin + 2.2.1 + + + attach-sources + + jar + + + + + + + org.apache.maven.plugins + maven-javadoc-plugin + 2.10.4 + + + attach-javadocs + + jar + + + + + + + + foo + bar + + + + ${java.home}/bin/javadoc + + + + + org.apache.maven.plugins + maven-resources-plugin + 2.4.3 + + UTF-8 + + + + + maven-assembly-plugin + + + + + eu.svjatoslav.alyverkko_cli.Main + + + + jar-with-dependencies + + alyverkko-cli + false + + + + + package-jar-with-dependencies + package + + single + + + + jar-with-dependencies + + + + eu.svjatoslav.alyverkko_cli.Main + + + + + + + + + + + org.apache.maven.wagon + wagon-ssh-external + 2.6 + + + + + + + + svjatoslav.eu + svjatoslav.eu + scpexe://svjatoslav.eu:10006/srv/maven + + + svjatoslav.eu + svjatoslav.eu + scpexe://svjatoslav.eu:10006/srv/maven + + + + + + svjatoslav.eu + Svjatoslav repository + https://www3.svjatoslav.eu/maven/ + + + + + scm:git:ssh://n0@svjatoslav.eu/home/git/repositories/alyverkko-cli.git + scm:git:ssh://n0@svjatoslav.eu/home/git/repositories/alyverkko-cli.git + + + + + diff --git a/src/main/java/eu/svjatoslav/alyverkko_cli/AiTask.java b/src/main/java/eu/svjatoslav/alyverkko_cli/AiTask.java new file mode 100644 index 0000000..1ce9a92 --- /dev/null +++ b/src/main/java/eu/svjatoslav/alyverkko_cli/AiTask.java @@ -0,0 +1,320 @@ +package eu.svjatoslav.alyverkko_cli; + +import eu.svjatoslav.alyverkko_cli.commands.mail_correspondant.MailQuery; + +import java.io.*; +import java.nio.file.Files; + +import static eu.svjatoslav.alyverkko_cli.Main.configuration; +import static java.lang.String.join; + +/** + * Encapsulates the process of running an AI inference query via + * llama.cpp. It prepares an input file, spawns the process, collects + * output, and cleans up temporary files. + */ +public class AiTask { + + /** + * Marker for the AI's response block, used in the constructed prompt string. + */ + public static final String AI_RESPONSE_MARKER = "ASSISTANT:"; + + /** + * Marker used by llama.cpp to print metadata. We monitor and display these lines. + */ + private static final String LLAMA_CPP_META_INFO_MARKER = "llm_load_print_meta: "; + + /** + * The mail query defining system prompt, user prompt, and which model to use. + */ + private final MailQuery mailQuery; + + /** + * The temperature (creativity factor) for the AI. + */ + private final Float temperature; + + /** + * Temporary file used as input to the llama.cpp CLI. + */ + private File inputFile; + + /** + * Creates a new AI task with a given mail query. + * + * @param mailQuery the mail query containing model and prompts. + */ + public AiTask(MailQuery mailQuery) { + this.mailQuery = mailQuery; + this.temperature = configuration.getDefaultTemperature(); + } + + /** + * Builds the prompt text that is fed to llama.cpp, including the system prompt, + * the user prompt, and an "ASSISTANT:" marker signifying where the AI response begins. + * + * @return a string containing the fully prepared query prompt. + */ + private String buildAiQuery() { + StringBuilder sb = new StringBuilder(); + sb.append("SYSTEM:\n").append(mailQuery.systemPrompt).append("\n"); + + String filteredUserPrompt = filterParticipantsInUserInput(mailQuery.userPrompt); + if (!filteredUserPrompt.startsWith("USER:")) { + sb.append("USER:\n"); + } + sb.append(filteredUserPrompt).append("\n"); + + sb.append(AI_RESPONSE_MARKER); + return sb.toString(); + } + + /** + * In the user input, rewrite lines like "* USER:" or "* ASSISTANT:" + * to "USER:" or "ASSISTANT:" so that we standardize them in the final prompt. + * + * @param input the raw user input. + * @return a sanitized or standardized version of the user prompt. + */ + public static String filterParticipantsInUserInput(String input) { + StringBuilder result = new StringBuilder(); + String[] lines = input.split("\n"); + for (int i = 0; i < lines.length; i++) { + String line = lines[i]; + if (i > 0) { + result.append("\n"); + } + if ("* ASSISTANT:".equals(line)) { + line = "ASSISTANT:"; + } + if ("* USER:".equals(line)) { + line = "USER:"; + } + result.append(line); + } + return result.toString(); + } + + /** + * In the AI's response, revert lines like "ASSISTANT:" to "* ASSISTANT:" + * for easier reading in org-mode, plus append a * USER: prompt at the end + * to form the basis for a continuing conversation. + * + * @param response the raw AI response. + * @return a sanitized response for org-mode usage. + */ + public static String filterParticipantsInAiResponse(String response) { + StringBuilder result = new StringBuilder(); + String[] lines = response.split("\n"); + for (int i = 0; i < lines.length; i++) { + String line = lines[i]; + if (i > 0) { + result.append("\n"); + } + if ("ASSISTANT:".equals(line)) { + line = "* ASSISTANT:"; + } + if ("USER:".equals(line)) { + line = "* USER:"; + } + result.append(line); + } + result.append("\n* USER:\n"); + return result.toString(); + } + + /** + * Runs the AI query by constructing the prompt, writing it to a temp file, + * invoking llama.cpp, collecting output, and performing any final cleanup. + * + * @return the AI's response in a format suitable for appending back into + * the conversation file. + * @throws InterruptedException if the process is interrupted. + * @throws IOException if reading/writing the file fails or the process fails to start. + */ + public String runAiQuery() throws InterruptedException, IOException { + try { + // Build input prompt + initializeInputFile(buildAiQuery()); + + // Prepare process builder + ProcessBuilder processBuilder = new ProcessBuilder(); + processBuilder.command(getCliCommand().split("\\s+")); // Splitting the command string into tokens + + // Start process + Process process = processBuilder.start(); + + // Handle process's error stream + handleErrorThread(process); + + // Handle process's output stream + StringBuilder result = new StringBuilder(); + Thread outputThread = handleResultThread(process, result); + + // Wait for the process to finish + process.waitFor(); + + // Wait for the output thread to finish reading + outputThread.join(); + + // Clean up the AI response: remove partial prompt text, end-of-text marker, etc. + return filterParticipantsInAiResponse(cleanupAiResponse(result.toString())); + } finally { + deleteTemporaryFile(); + } + } + + /** + * Creates a temporary file for the AI input and writes the prompt to it. + * + * @param aiQuery the final prompt string for the AI to process. + * @throws IOException if file creation or writing fails. + */ + private void initializeInputFile(String aiQuery) throws IOException { + inputFile = createTemporaryFile(); + Files.write(inputFile.toPath(), aiQuery.getBytes()); + } + + /** + * Creates a temporary file that will be used for the AI prompt input. + * + * @return a new {@link File} referencing the created temporary file. + * @throws IOException if the file could not be created. + */ + private File createTemporaryFile() throws IOException { + File file = Files.createTempFile("ai-inference", ".tmp").toFile(); + file.deleteOnExit(); + return file; + } + + /** + * Cleans up the AI response by removing the partial text before the + * AI response marker and after the end-of-text marker, if specified. + * + * @param result the raw output from llama.cpp. + * @return the cleaned AI response. + */ + private String cleanupAiResponse(String result) { + // remove text before AI response marker + int aIResponseIndex = result.lastIndexOf(AI_RESPONSE_MARKER); + if (aIResponseIndex != -1) { + result = result.substring(aIResponseIndex + AI_RESPONSE_MARKER.length()); + } + + // remove text after end of text marker, if it exists + if (mailQuery.model.endOfTextMarker != null) { + int endOfTextMarkerIndex = result.indexOf(mailQuery.model.endOfTextMarker); + if (endOfTextMarkerIndex != -1) { + result = result.substring(0, endOfTextMarkerIndex); + } + } + + return result + "\n"; + } + + /** + * Returns the full command string used to run the AI inference via llama.cpp. + * + * @return a string representing the command and all arguments. + */ + private String getCliCommand() { + int niceValue = 10; // niceness level for background tasks + String executablePath = configuration.getLlamaCliPath().getAbsolutePath(); + + return join(" ", + "nice", "-n", Integer.toString(niceValue), + executablePath, + "--model " + mailQuery.model.filesystemPath, + "--threads " + configuration.getThreadCount(), + "--threads-batch " + configuration.getBatchThreadCount(), + "--mirostat 2", + "--flash-attn", + "--cache-type-k q8_0", + "--cache-type-v q8_0", + "--no-warmup", + "--temp " + temperature, + "--ctx-size " + mailQuery.model.contextSizeTokens, + "--batch-size 8", + "--no-conversation", + "-n -1", + "--repeat_penalty 1.1", + "--file " + inputFile + ); + } + + /** + * Spawns a new Thread to handle the error stream from llama.cpp, + * printing lines that contain metadata or errors to the console. + * + * @param process the process whose error stream is consumed. + */ + private static void handleErrorThread(Process process) { + Thread errorThread = new Thread(() -> { + try (BufferedReader reader = new BufferedReader(new InputStreamReader(process.getErrorStream()))) { + String line; + while ((line = reader.readLine()) != null) { + handleErrorStreamLine(line); + } + } catch (IOException e) { + System.err.println("Error reading error stream: " + e.getMessage()); + } + }); + errorThread.start(); + } + + /** + * Decides what to do with each line from the error stream: + * if it matches the llama.cpp meta-info marker, print it normally; + * otherwise print as an error. + * + * @param line a line from the llama.cpp error stream. + */ + private static void handleErrorStreamLine(String line) { + if (line.startsWith(LLAMA_CPP_META_INFO_MARKER)) { + // Print the meta-info to the console in normal color + System.out.println(line.substring(LLAMA_CPP_META_INFO_MARKER.length())); + } else { + // Print actual error lines in red + Utils.printRedMessageToConsole(line); + } + } + + /** + * Consumes the standard output (inference result) from the + * llama.cpp process, storing it into a result buffer for further + * cleanup, while simultaneously printing it to the console. + * + * @param process the AI inference process. + * @param result a string builder to accumulate the final result. + * @return the thread that is reading the output stream. + */ + private static Thread handleResultThread(Process process, StringBuilder result) { + Thread outputThread = new Thread(() -> { + try (BufferedReader reader = new BufferedReader(new InputStreamReader(process.getInputStream()))) { + String aiResultLine; + while ((aiResultLine = reader.readLine()) != null) { + System.out.print("AI: " + aiResultLine + "\n"); // Show each line in real-time + result.append(aiResultLine).append("\n"); + } + } catch (IOException e) { + throw new RuntimeException(e); + } + }); + outputThread.start(); + return outputThread; + } + + /** + * Deletes the temporary input file once processing is complete. + */ + private void deleteTemporaryFile() { + if (inputFile != null && inputFile.exists()) { + try { + Files.delete(inputFile.toPath()); + } catch (IOException e) { + System.err.println("Failed to delete temporary file: " + e.getMessage()); + } + } + } +} diff --git a/src/main/java/eu/svjatoslav/alyverkko_cli/Command.java b/src/main/java/eu/svjatoslav/alyverkko_cli/Command.java new file mode 100644 index 0000000..dfd9b14 --- /dev/null +++ b/src/main/java/eu/svjatoslav/alyverkko_cli/Command.java @@ -0,0 +1,26 @@ +package eu.svjatoslav.alyverkko_cli; + +import java.io.IOException; + +/** + * A simple interface for all subcommands used by the Älyverkko CLI. + * Implementing classes define a unique name (e.g., "wizard") and an + * {@code execute} method for the command's logic. + */ +public interface Command { + + /** + * @return the subcommand's name. + */ + String getCommandName(); + + /** + * Called to carry out the specific subcommand. Typically, reads + * command-line arguments and performs the desired action. + * + * @param args arguments passed after the subcommand name. + * @throws IOException if I/O operations fail. + * @throws InterruptedException if the operation is interrupted. + */ + void executeCommand(String[] args) throws IOException, InterruptedException; +} diff --git a/src/main/java/eu/svjatoslav/alyverkko_cli/Main.java b/src/main/java/eu/svjatoslav/alyverkko_cli/Main.java new file mode 100644 index 0000000..f7d36a9 --- /dev/null +++ b/src/main/java/eu/svjatoslav/alyverkko_cli/Main.java @@ -0,0 +1,83 @@ +package eu.svjatoslav.alyverkko_cli; + +import eu.svjatoslav.alyverkko_cli.commands.*; +import eu.svjatoslav.alyverkko_cli.commands.mail_correspondant.MailCorrespondentCommand; +import eu.svjatoslav.alyverkko_cli.configuration.Configuration; + +import java.io.IOException; +import java.util.Optional; + +import static java.util.Arrays.copyOfRange; + +/** + * The main entry point for the Älyverkko CLI application. + * It processes subcommands such as "wizard", "selftest", "joinfiles", + * "mail", and "listmodels". + */ +public class Main { + + /** + * The list of all supported subcommands. + */ + private final java.util.List commands = java.util.Arrays.asList( + new ListModelsCommand(), + new MailCorrespondentCommand(), + new JoinFilesCommand(), + new WizardCommand() + ); + + /** + * The active, loaded configuration for the entire application. + * May be null if the configuration is not loaded properly. + */ + public static Configuration configuration; + + /** + * Application entry point. Dispatches to a subcommand if one is + * specified; otherwise shows usage help. + * + * @param args command-line arguments; the first is the subcommand name. + */ + public static void main(final String[] args) throws IOException, InterruptedException { + new Main().handleCommand(args); + } + + /** + * Attempts to find and execute the subcommand specified in the given arguments, + * or prints a help message if no command is found. + * + * @param args the command-line arguments. + * @throws IOException if an I/O error occurs during command execution. + * @throws InterruptedException if the command is interrupted. + */ + public void handleCommand(String[] args) throws IOException, InterruptedException { + if (args.length == 0) { + showHelp(); + return; + } + + String commandName = args[0].toLowerCase(); + Optional commandOptional = commands.stream() + .filter(cmd -> cmd.getCommandName().equals(commandName)) + .findFirst(); + + if (!commandOptional.isPresent()) { + System.out.println("Unknown command: " + commandName); + showHelp(); + return; + } + + Command command = commandOptional.get(); + String[] remainingArgs = copyOfRange(args, 1, args.length); + command.executeCommand(remainingArgs); + } + + /** + * Displays a basic help message, listing available commands. + */ + private void showHelp() { + System.out.println("Älyverkko CLI\n"); + System.out.println("Available commands:"); + commands.forEach(cmd -> System.out.println(" " + cmd.getCommandName())); + } +} diff --git a/src/main/java/eu/svjatoslav/alyverkko_cli/Utils.java b/src/main/java/eu/svjatoslav/alyverkko_cli/Utils.java new file mode 100644 index 0000000..8f42cde --- /dev/null +++ b/src/main/java/eu/svjatoslav/alyverkko_cli/Utils.java @@ -0,0 +1,20 @@ +package eu.svjatoslav.alyverkko_cli; + +/** + * Utility functions for miscellaneous tasks such as colored console output. + */ +public class Utils { + + /** + * Prints a message in red text to the console. + * + * @param message the text to print in red. + */ + public static void printRedMessageToConsole(String message) { + // set output color to red + System.out.print("\033[0;31m"); + System.out.print(message + "\n"); + // reset output color + System.out.print("\033[0m"); + } +} diff --git a/src/main/java/eu/svjatoslav/alyverkko_cli/commands/JoinFilesCommand.java b/src/main/java/eu/svjatoslav/alyverkko_cli/commands/JoinFilesCommand.java new file mode 100644 index 0000000..072bb3a --- /dev/null +++ b/src/main/java/eu/svjatoslav/alyverkko_cli/commands/JoinFilesCommand.java @@ -0,0 +1,216 @@ +package eu.svjatoslav.alyverkko_cli.commands; + +import eu.svjatoslav.alyverkko_cli.Command; +import eu.svjatoslav.alyverkko_cli.configuration.ConfigurationHelper; +import eu.svjatoslav.commons.cli_helper.parameter_parser.Parser; +import eu.svjatoslav.commons.cli_helper.parameter_parser.parameter.DirectoryOption; +import eu.svjatoslav.commons.cli_helper.parameter_parser.parameter.NullOption; +import eu.svjatoslav.commons.cli_helper.parameter_parser.parameter.StringOption; +import eu.svjatoslav.commons.string.GlobMatcher; +import org.apache.commons.lang3.StringUtils; + +import java.io.BufferedWriter; +import java.io.File; +import java.io.IOException; +import java.nio.charset.StandardCharsets; +import java.nio.file.*; + +import static eu.svjatoslav.alyverkko_cli.Main.configuration; +import static eu.svjatoslav.alyverkko_cli.configuration.ConfigurationHelper.getConfigurationFile; +import static eu.svjatoslav.alyverkko_cli.configuration.ConfigurationHelper.loadConfiguration; + +/** + * The JoinFilesCommand aggregates multiple files (optionally matching + * a specific pattern) into a single file for AI processing, typically + * in the mail directory. + * + * Usage Example: + * 
+ *   alyverkko-cli joinfiles -s /path/to/source -p "*.java" -t "my_topic" --edit
+ *
+ */ +public class JoinFilesCommand implements Command { + + /** + * A command-line parser to handle joinfiles arguments. + */ + final Parser parser = new Parser(); + + /** + * Directory from which files will be joined. + */ + public DirectoryOption sourceDirectoryOption = parser.add(new DirectoryOption("Directory to join files from")) + .addAliases("--src-dir", "-s") + .mustExist(); + + /** + * Pattern for matching files, such as "*.java". + */ + public StringOption patternOption = parser.add(new StringOption("Pattern to match files")) + .addAliases("--pattern", "-p"); + + /** + * Topic name, used as the basis for the output file name. + */ + public StringOption topic = parser.add(new StringOption("Topic of the joined files")) + .addAliases("--topic", "-t") + .setMandatory(); + + /** + * If present, open the joined file using a text editor afterward. + */ + public NullOption editOption = parser.add(new NullOption("Edit the joined file using text editor")) + .addAliases("--edit", "-e"); + + /** + * The base directory for recursion when joining files. + */ + public Path sourceBaseDirectory; + + /** + * The pattern used to filter files for joining, e.g. "*.java". + */ + public String fileNamePattern = null; + + /** + * The resulting output file that aggregates all matched files. + */ + File outputFile; + + /** + * @return the name of this command, i.e., "joinfiles". + */ + @Override + public String getCommandName() { + return "joinfiles"; + } + + /** + * Executes the command that joins files from a specified directory + * (matching an optional pattern) into one output file in the mail + * directory. Optionally, it can open the output file in an editor. + * + * @param cliArguments the command-line arguments after "joinfiles". + * @throws IOException if any IO operations fail. + */ + @Override + public void executeCommand(String[] cliArguments) throws IOException { + configuration = loadConfiguration(getConfigurationFile(null)); + if (configuration == null){ + System.out.println("Failed to load configuration file"); + return; + } + + if (!parser.parse(cliArguments)) { + System.out.println("Failed to parse command-line arguments"); + parser.showHelp(); + return; + } + + // Build the path to the target file that is relative to the mail directory + outputFile = configuration.getMailDirectory().toPath().resolve(topic.getValue() + ".org").toFile(); + + if (patternOption.isPresent()) { + fileNamePattern = patternOption.getValue(); + joinFiles(); + } + + if (editOption.isPresent()) { + openFileWithEditor(); + } + } + + /** + * Opens the joined file with a text editor. Currently uses a + * command "emc" as an example—adapt as needed. + * + * @throws IOException if the launch of the editor fails. + */ + private void openFileWithEditor() throws IOException { + String[] cmd = {"emc", outputFile.getAbsolutePath()}; + Runtime.getRuntime().exec(cmd); + } + + /** + * Joins the matching files from the configured source directory + * into a single file named {@code .org} in the mail directory. + * + * @throws IOException if reading or writing files fails. + */ + private void joinFiles() throws IOException { + boolean appendToFile = outputFile.exists(); + + if (sourceDirectoryOption.isPresent()) { + sourceBaseDirectory = sourceDirectoryOption.getValue().toPath(); + } else { + sourceBaseDirectory = Paths.get("."); + } + + try (BufferedWriter writer = Files.newBufferedWriter( + outputFile.toPath(), StandardCharsets.UTF_8, + appendToFile ? StandardOpenOption.APPEND : StandardOpenOption.CREATE)) { + + // Recursively join files that match the pattern + joinFilesRecursively(sourceBaseDirectory, writer); + } + + System.out.println("Files have been joined into: " + outputFile.getAbsolutePath()); + } + + /** + * Recursively traverses the specified directory and writes the contents + * of files that match the specified {@link #fileNamePattern}. + * + * @param directoryToIndex the directory to be searched recursively. + * @param writer the writer to which file contents are appended. + * @throws IOException if file reading fails. + */ + private void joinFilesRecursively(Path directoryToIndex, BufferedWriter writer) throws IOException { + try (DirectoryStream stream = Files.newDirectoryStream(directoryToIndex)) { + for (Path entry : stream) { + if (Files.isDirectory(entry)) { + joinFilesRecursively(entry, writer); + } else if (Files.isRegularFile(entry)) { + String fileName = entry.getFileName().toString(); + + boolean match = GlobMatcher.match(fileName, fileNamePattern); + if (match) { + System.out.println("Joining file: " + fileName); + writeFile(writer, entry); + } + } + } + } + } + + /** + * Writes the contents of a single file to the specified writer, + * including a small header containing the file path. + * + * @param writer the writer to which file contents are appended. + * @param entry the file to read and write. + * @throws IOException if file reading or writing fails. + */ + private void writeFile(BufferedWriter writer, Path entry) throws IOException { + writeFileHeader(writer, entry); + + String fileContent = new String(Files.readAllBytes(entry), StandardCharsets.UTF_8); + + // remove empty lines from the beginning and end of the file + fileContent = fileContent.replaceAll("(?m)^\\s*$", ""); + + writer.write(fileContent + "\n"); + } + + /** + * Writes a small header line to indicate which file is being appended. + * + * @param writer the writer to which the header is appended. + * @param entry the path of the current file. + * @throws IOException if writing fails. + */ + private void writeFileHeader(BufferedWriter writer, Path entry) throws IOException { + String relativePath = sourceBaseDirectory.relativize(entry).toString(); + writer.write("* file: " + relativePath + "\n\n"); + } +} diff --git a/src/main/java/eu/svjatoslav/alyverkko_cli/commands/ListModelsCommand.java b/src/main/java/eu/svjatoslav/alyverkko_cli/commands/ListModelsCommand.java new file mode 100644 index 0000000..5078177 --- /dev/null +++ b/src/main/java/eu/svjatoslav/alyverkko_cli/commands/ListModelsCommand.java @@ -0,0 +1,45 @@ +package eu.svjatoslav.alyverkko_cli.commands; + +import eu.svjatoslav.alyverkko_cli.Command; +import eu.svjatoslav.alyverkko_cli.model.ModelLibrary; + +import java.io.IOException; + +import static eu.svjatoslav.alyverkko_cli.Main.configuration; +import static eu.svjatoslav.alyverkko_cli.configuration.ConfigurationHelper.getConfigurationFile; +import static eu.svjatoslav.alyverkko_cli.configuration.ConfigurationHelper.loadConfiguration; + +/** + * Lists all configured models in the system, loading them from the + * user’s configuration and printing them to the console. + */ +public class ListModelsCommand implements Command { + + /** + * @return the name of this command, i.e., "listmodels". + */ + @Override + public String getCommandName() { + return "listmodels"; + } + + /** + * Executes the command to load the user's configuration and list + * all known AI models, printing them to stdout. + * + * @param cliArguments the command-line arguments after "listmodels". + * @throws IOException if loading configuration fails. + */ + @Override + public void executeCommand(String[] cliArguments) throws IOException { + configuration = loadConfiguration(getConfigurationFile(null)); + if (configuration == null){ + System.out.println("Failed to load configuration file"); + return; + } + + System.out.println("Listing models in directory: " + configuration.getModelsDirectory()); + ModelLibrary modelLibrary = new ModelLibrary(configuration.getModelsDirectory(), configuration.getModels()); + modelLibrary.printModels(); + } +} diff --git a/src/main/java/eu/svjatoslav/alyverkko_cli/commands/WizardCommand.java b/src/main/java/eu/svjatoslav/alyverkko_cli/commands/WizardCommand.java new file mode 100644 index 0000000..36616d6 --- /dev/null +++ b/src/main/java/eu/svjatoslav/alyverkko_cli/commands/WizardCommand.java @@ -0,0 +1,480 @@ +package eu.svjatoslav.alyverkko_cli.commands; + +import com.fasterxml.jackson.databind.ObjectMapper; +import com.fasterxml.jackson.dataformat.yaml.YAMLFactory; +import eu.svjatoslav.alyverkko_cli.Command; +import eu.svjatoslav.alyverkko_cli.configuration.Configuration; +import eu.svjatoslav.alyverkko_cli.configuration.ConfigurationHelper; +import eu.svjatoslav.alyverkko_cli.configuration.ConfigurationModel; +import eu.svjatoslav.commons.cli_helper.parameter_parser.Parser; +import eu.svjatoslav.commons.cli_helper.parameter_parser.parameter.FileOption; + +import java.io.*; +import java.nio.file.*; +import java.util.ArrayList; +import java.util.List; + +import static eu.svjatoslav.alyverkko_cli.Utils.printRedMessageToConsole; +import static eu.svjatoslav.alyverkko_cli.configuration.ConfigurationHelper.getConfigurationFile; +import static eu.svjatoslav.commons.cli_helper.CLIHelper.*; + +/** + * A single WizardCommand that: + * 1. Loads existing configuration (if any). + * 2. Performs "selftest" style validation checks interactively, + * prompting the user to fix invalid or missing items. + * 3. If no config file exists, it goes through all config parameters + * from scratch. + * 4. Offers to remove model entries that reference missing files. + * 5. Autodiscovers new .gguf files and lets the user add them to the config. + * 6. Saves the resulting (fixed) config file. + * 7. Prints a final pass/fail summary for the user. + */ +public class WizardCommand implements Command { + + // Command-line parser to handle wizard arguments + private final Parser cliParser = new Parser(); + + /** + * Optional CLI argument for specifying a configuration file path. + */ + public FileOption configFileOption = cliParser.add(new FileOption("Configuration file path")) + .addAliases("--config", "-c"); + + /** + * The configuration object (loaded or newly created) + */ + private Configuration configuration; + + private File configurationFile; + + private boolean configurationUpdated = false; + private boolean modelsUpdated = false; + + @Override + public String getCommandName() { + return "wizard"; + } + + @Override + public void executeCommand(String[] cliArguments) throws IOException { + // Parse command-line arguments + if (!cliParser.parse(cliArguments)) { + System.out.println("Failed to parse command-line arguments"); + cliParser.showHelp(); + return; + } + + configurationFile = getConfigurationFile(configFileOption); + loadOrCreateConfiguration(); + + checkAndFixGeneralParameters(); + + fixModelEntries(); + + trySaveConfiguration(); + + if (modelsUpdated) { + System.out.println("Configuration has been updated. Please open the configuration file in a text editor to review and adjust model settings as needed."); + } + } + + private void loadOrCreateConfiguration() throws IOException { + validateConfigurationFile(); + + if (configurationFile.exists()) { + System.out.println("Found existing configuration at: \"" + configurationFile.getAbsolutePath() + "\""); + configuration = ConfigurationHelper.loadConfiguration(configurationFile); + } else { + // If no config found, create a fresh one + System.out.println("Existing configuration not found at \"" + + configurationFile.getAbsolutePath() + "\". Initializing new blank configuration."); + configuration = new Configuration(); + configurationUpdated = true; + } + } + + /** + * Validates the configuration file path and checks if it is a valid file. + * If not, prints an error message and exits the program. + */ + private void validateConfigurationFile() { + if (!configurationFile.exists()) return; // No need to check further if it doesn't exist + + // Check if the file is a directory + if (configurationFile.isDirectory()) { + System.err.println("ERROR: Configuration file path incorrectly points to a directory: \"" + configurationFile.getAbsolutePath() + + "\". Please specify a file instead."); + System.exit(1); + } + + // Check if the file is readable + if (!configurationFile.canRead()) { + System.err.println("ERROR: Cannot read configuration file: \"" + configurationFile.getAbsolutePath() + + "\". Please check permissions."); + System.exit(1); + } + + // Check if the file is writable + if (!configurationFile.canWrite()) { + System.err.println("ERROR: Cannot write to configuration file: \"" + configurationFile.getAbsolutePath() + + "\". Please check file permissions."); + System.exit(1); + } + } + + /** + * Step-by-step checking (and possibly fixing) of each main config parameter. + */ + private void checkAndFixGeneralParameters() { + configuration.setMailDirectory( + checkDirectory( + configuration.getMailDirectory(), + "Mail directory", + true, + "The mail directory is where the AI will look for tasks to solve. " + + "It should be a directory that you can write to. Please specify new mail directory path.", + true) + ); + + configuration.setModelsDirectory( + checkDirectory( + configuration.getModelsDirectory(), + "Models directory", + null, + "The models directory is where the AI model files (*.gguf) are stored. " + + "Please specify new models directory path.", + true) + ); + + configuration.setPromptsDirectory( + checkDirectory( + configuration.getPromptsDirectory(), + "Prompts directory", + null, + "The prompts directory is where the AI prompt files (*.txt) are stored. " + + "Please specify new prompts directory path.", + true) + ); + + configuration.setLlamaCliPath( + checkFile( + configuration.getLlamaCliPath(), + "llama.cpp project llama-cli executable file path", + "The llama-cli is commandline engine that runs GGUF language models. " + + "Usually it is located under build/bin/ directory within llama.cpp project.", + true, + true) + ); + + // Default_temperature + Float temperature = configuration.getDefaultTemperature(); + if (temperature == null || temperature < 0f || temperature > 3f) { + configuration.setDefaultTemperature(askFloat( + "Enter default temperature (0-3). Lower => more deterministic, higher => more creative.", + temperature, + 0f, 3f, false + )); + } + // Thread_count + Integer threadCount = configuration.getThreadCount(); + if (threadCount == null || threadCount < 1) { + int defaultThreadCount = Runtime.getRuntime().availableProcessors() / 2; + if (defaultThreadCount < 1) defaultThreadCount = 1; + configuration.setThreadCount(askInteger( + "Enter number of CPU threads for AI generation. Typically RAM bandwidth gets saturated " + + "first and becomes bottleneck before all CPU cores can get fully utilized. So for 12 core CPU" + + " it might be enough to set 6 threads. Increasing this number higher yields diminishing returns.", + defaultThreadCount, + 1, null, false + )); + } + + // Batch thread count + Integer batchThreadCount = configuration.getBatchThreadCount(); + if (batchThreadCount == null || batchThreadCount < 1) { + int defaultThreadCount = Runtime.getRuntime().availableProcessors(); + configuration.setBatchThreadCount( + askInteger( + "\nEnter number of CPU threads for input prompt processing (all cores is often fine).", + defaultThreadCount, + 1, null, false + )); + } + } + + /** + * Validates a directory path and prompts user to fix if needed. + * @param directory current directory value + * @param directoryName name to display to user + * @param writable if directory must be writable (null = no check) + * @param explanation message to show user when prompting + * @param offerToCreate if true, offers to create directory if missing + * @return validated directory path + */ + private File checkDirectory( + File directory, + String directoryName, + Boolean writable, + String explanation, + boolean offerToCreate) { + + while (true) { + // Check if the directory is null + boolean allOk = true; + if (directory == null) { + System.out.println(directoryName + " is not defined."); + allOk = false; + directory = getNewDirectoryPath(explanation, writable); + } + + if (!directory.exists()) { + System.out.println(directoryName + " does not exist: " + directory.getAbsolutePath()); + allOk = false; + // Offer to create it + if (offerToCreate) { + if (askBoolean("Create " + directoryName + " ?", true)) { + boolean created = directory.mkdirs(); + if (!created) { + printRedMessageToConsole("Failed to create \"" + directory + "\". Check permissions"); + } else { + System.out.println(directoryName + " created at: " + directory.getAbsolutePath()); + } + } + } + } + + if (!directory.isDirectory()) { + System.out.println(directoryName + " is not a directory: " + directory.getAbsolutePath()); + allOk = false; + directory = getNewDirectoryPath(explanation, writable); + } + + if (writable != null && writable && !directory.canWrite()) { + System.out.println(directoryName + " is not writable: " + directory.getAbsolutePath()); + allOk = false; + directory = getNewDirectoryPath(explanation, writable); + } + + if (allOk) { + System.out.println(directoryName + " is: " + directory.getAbsolutePath()); + return directory; + } + + configurationUpdated = true; + } + } + + /** + * Validates a file path and prompts user to fix if needed. + * @param file current file value + * @param fileName name to display to user + * @param mustExist if file must exist (null = no check) + * @param explanation message to show user when prompting + * @param executable if file must be executable (null = no check) + * @return validated file path + */ + private File checkFile( + File file, + String fileName, + String explanation, + Boolean mustExist, + Boolean executable) { + + while (true) { + boolean allOk = true; + if (file == null) { + System.out.println(fileName + " is not defined."); + allOk = false; + file = askFile(explanation, null, mustExist, null, null, executable, false); + } + + if (mustExist != null && mustExist && !file.exists()) { + System.out.println(fileName + " does not exist: " + file.getAbsolutePath()); + allOk = false; + file = askFile(explanation, null, mustExist, null, null, executable, false); + } + + if (!file.isFile()) { + System.out.println(fileName + " is not a file: " + file.getAbsolutePath()); + allOk = false; + file = askFile(explanation, null, null, null, null, executable, false); + } + + if (executable != null && executable && !file.canExecute()) { + System.out.println(fileName + " is not executable: " + file.getAbsolutePath()); + allOk = false; + file = askFile(explanation, null, true, null, null, executable, false); + } + + if (allOk) { + System.out.println(fileName + " is: " + file.getAbsolutePath()); + return file; + } + + configurationUpdated = true; + } + } + + private static File getNewDirectoryPath(String directoryName, Boolean writable) { + return askDirectory(directoryName, null, null, null, writable, null, false); + } + + /** + * Saves the config to the default path, verifying if user wants to + * overwrite if it already exists, etc. + */ + private void trySaveConfiguration() { + if (!configurationUpdated) { + System.out.println("No changes made to the configuration. Not saving."); + return; + } + + // ask user if user wants to save configuration + if (!askBoolean("Save configuration to: " + configurationFile, true, false)) + return; + + boolean fileExisted = configurationFile.exists(); + + try { + Files.createDirectories(configurationFile.toPath().getParent()); + try (BufferedWriter writer = Files.newBufferedWriter( + configurationFile.toPath(), + StandardOpenOption.CREATE, + StandardOpenOption.TRUNCATE_EXISTING + )) { + new ObjectMapper(new YAMLFactory()).writeValue(writer, configuration); + } + if (fileExisted) { + System.out.println("Existing configuration updated at: " + configurationFile.toPath()); + } else { + System.out.println("New configuration created at: " + configurationFile.toPath()); + } + } catch (IOException e) { + printRedMessageToConsole("Error saving configuration: " + e.getMessage()); + } + } + + /** + * Generates an alias from a .gguf filename by removing non-alphanumeric chars. + */ + private String suggestAlias(String filePath) { + String fileName = new File(filePath).getName(); + // Remove .gguf extension + fileName = fileName.replaceFirst("\\.gguf$", ""); + // Check if it's a split model + if (fileName.matches(".*-\\d{5}-of-\\d{5}")) { + // Extract base name by removing the part numbers + fileName = fileName.replaceFirst("-\\d{5}-of-\\d{5}", ""); + } + // Replace non-alphanumeric characters with hyphens + String alias = fileName.replaceAll("[^a-zA-Z0-9]", "-").toLowerCase(); + // Normalize hyphens and trim leading/trailing hyphens + return alias.replaceAll("-+", "-").replaceAll("^-|-$", ""); + } + + private void fixModelEntries() { + File modelsDir = configuration.getModelsDirectory(); + if (modelsDir == null || !modelsDir.exists() || !modelsDir.isDirectory()) { + System.out.println("Models directory is not properly configured. Skipping model checks."); + return; + } + + List existingModels = configuration.getModels(); + if (existingModels == null) { + existingModels = new ArrayList<>(); + configuration.setModels(existingModels); + configurationUpdated = true; + } + + annotateMissingModels(); + + discoverNewModels(); + } + + private void discoverNewModels() { + // List all .gguf files in models directory + File[] files = configuration.getModelsDirectory().listFiles((dir, name) -> name.endsWith(".gguf")); + if (files == null) return; + + for (File file : files) { + String relativePath = configuration.getModelsDirectory().toPath().relativize(file.toPath()).toString(); + if (isExistingModel(relativePath)) continue; + + processPotentialNewModelFile(file, relativePath); + } + } + + private boolean isExistingModel(String relativePath) { + return configuration.getModels().stream() + .anyMatch(m -> m.getFilesystemPath().equals(relativePath)); + } + + private void processPotentialNewModelFile(File file, String relativePath) { + // Check if it's a split model + if (isSplitModel(file.getName())) { + handleSplitModel(file, relativePath); + } else { + addNewModel(relativePath); + } + } + + private boolean isSplitModel(String fileName) { + return fileName.matches(".*-\\d{5}-of-\\d{5}\\.gguf"); + } + + private void handleSplitModel(File file, String relativePath) { + String baseName = relativePath.replaceFirst("-\\d{5}-of-\\d{5}\\.gguf", ""); + if (configuration.getModels().stream().anyMatch(m -> m.getAlias().startsWith(baseName))) { + return; + } + + // Extract part number + String partNumberStr = relativePath.replaceAll(".*-(\\d{5}-of-\\d{5}\\.gguf)", "$1"); + int partNumber = Integer.parseInt(partNumberStr.split("-of-")[0]); + if (partNumber == 1) { + addNewModel(relativePath); + } + } + + private void addNewModel(String relativePath) { + ConfigurationModel newModel = getNewModel(relativePath); + configuration.getModels().add(newModel); + System.out.println("Added new model: " + newModel.getAlias() + " (" + newModel.getFilesystemPath() + ")"); + configurationUpdated = true; + modelsUpdated = true; + } + + private ConfigurationModel getNewModel(String relativePath) { + String suggestedAlias = suggestAlias(relativePath); + ConfigurationModel newModel = new ConfigurationModel(); + newModel.setAlias(suggestedAlias + "-new"); + newModel.setFilesystemPath(relativePath); + newModel.setContextSizeTokens(32768); // Default context size + newModel.setEndOfTextMarker(null); // Default end marker + return newModel; + } + + private void annotateMissingModels() { + // Process existing models to add/remove -missing suffix + for (ConfigurationModel model : configuration.getModels()) { + File modelFile = new File(configuration.getModelsDirectory(), model.getFilesystemPath()); + if (!modelFile.exists()) { + if (!model.getAlias().endsWith("-missing")) { + model.setAlias(model.getAlias() + "-missing"); + System.out.println("Marked model as missing: " + model.getAlias()); + configurationUpdated = true; + modelsUpdated = true; + } + } else { + if (model.getAlias().endsWith("-missing")) { + model.setAlias(model.getAlias().replaceFirst("-missing$", "")); + System.out.println("Removed -missing suffix from model: " + model.getAlias()); + configurationUpdated = true; + modelsUpdated = true; + } + } + } + } + +} diff --git a/src/main/java/eu/svjatoslav/alyverkko_cli/commands/mail_correspondant/MailCorrespondentCommand.java b/src/main/java/eu/svjatoslav/alyverkko_cli/commands/mail_correspondant/MailCorrespondentCommand.java new file mode 100644 index 0000000..469f626 --- /dev/null +++ b/src/main/java/eu/svjatoslav/alyverkko_cli/commands/mail_correspondant/MailCorrespondentCommand.java @@ -0,0 +1,346 @@ +package eu.svjatoslav.alyverkko_cli.commands.mail_correspondant; + +import eu.svjatoslav.alyverkko_cli.*; +import eu.svjatoslav.alyverkko_cli.configuration.ConfigurationHelper; +import eu.svjatoslav.alyverkko_cli.model.Model; +import eu.svjatoslav.alyverkko_cli.model.ModelLibrary; +import eu.svjatoslav.commons.cli_helper.parameter_parser.Parser; +import eu.svjatoslav.commons.cli_helper.parameter_parser.parameter.FileOption; + +import java.io.BufferedReader; +import java.io.File; +import java.io.FileReader; +import java.io.IOException; +import java.nio.file.*; +import java.util.HashMap; +import java.util.Map; +import java.util.Optional; + +import static eu.svjatoslav.alyverkko_cli.Main.configuration; +import static eu.svjatoslav.alyverkko_cli.configuration.ConfigurationHelper.getConfigurationFile; +import static eu.svjatoslav.alyverkko_cli.configuration.ConfigurationHelper.loadConfiguration; +import static eu.svjatoslav.commons.file.IOHelper.getFileContentsAsString; +import static eu.svjatoslav.commons.file.IOHelper.saveToFile; +import static java.nio.file.StandardWatchEventKinds.ENTRY_CREATE; +import static java.nio.file.StandardWatchEventKinds.ENTRY_MODIFY; + +/** + * The MailCorrespondentCommand continuously monitors a specified mail + * directory for new or modified text files, checks if they have a + * "TOCOMPUTE:" marker, and if so, processes them with an AI model. + * Once processed, results are appended to the same file. + *

+ * Usage: + * 

+ *   alyverkko-cli mail
+ *
+ */ +public class MailCorrespondentCommand implements Command { + + /** + * A command-line parser to handle "mail" command arguments. + */ + final Parser parser = new Parser(); + + /** + * Optional CLI argument for specifying a configuration file path. + */ + public FileOption configFileOption = parser.add(new FileOption("Configuration file path")) + .addAliases("--config", "-c") + .mustExist(); + + /** + * The library of available models, constructed from configuration. + */ + ModelLibrary modelLibrary; + + /** + * The WatchService instance for monitoring file system changes in + * the mail directory. + */ + private WatchService directoryWatcher; + + /** + * The directory that we continuously watch for new tasks. + */ + File mailDir; + + /** + * @return the name of this command, i.e., "mail". + */ + @Override + public String getCommandName() { + return "mail"; + } + + /** + * Executes the "mail" command, loading configuration, starting a + * WatchService on the mail directory, and running an infinite loop + * that processes newly discovered tasks. + * + * @param cliArguments the command-line arguments following the "mail" subcommand. + * @throws IOException if reading/writing tasks fails. + * @throws InterruptedException if the WatchService is interrupted. + */ + @Override + public void executeCommand(String[] cliArguments) throws IOException, InterruptedException { + if (!parser.parse(cliArguments)) { + System.out.println("Failed to parse commandline arguments"); + parser.showHelp(); + return; + } + + configuration = loadConfiguration(getConfigurationFile(configFileOption)); + if (configuration == null) { + System.out.println("Failed to load configuration file"); + return; + } + + modelLibrary = new ModelLibrary(configuration.getModelsDirectory(), configuration.getModels()); + mailDir = configuration.getMailDirectory(); + + // Set up directory watch service + initializeFileWatcher(); + + // Process any existing files that might already be in the directory + initialMailScanAndReply(); + + System.out.println("Mail correspondent running. Press CTRL+c to terminate."); + + // Main loop: watch for file events + while (true) { + WatchKey key; + try { + key = directoryWatcher.take(); + } catch (InterruptedException e) { + System.out.println("Interrupted while waiting for file system events. Exiting."); + break; + } + + System.out.println("Detected filesystem event."); + + // Sleep briefly to allow the file to be fully written + Thread.sleep(1000); + + processDetectedFilesystemEvents(key); + + if (!key.reset()) { + break; + } + } + + directoryWatcher.close(); + } + + /** + * Performs an initial scan of existing files in the mail directory, + * processing those that need AI inference (i.e., that start with "TOCOMPUTE:"). + * + * @throws IOException if reading files fails. + * @throws InterruptedException if the thread is interrupted. + */ + private void initialMailScanAndReply() throws IOException, InterruptedException { + File[] files = mailDir.listFiles(); + if (files == null) return; + + for (File file : files) { + processMailIfNeeded(file); + } + } + + /** + * Checks if a file needs to be processed by verifying that it: + * 1) is not hidden, + * 2) is a regular file, + * 3) starts with "TOCOMPUTE:" in the first line. + * + * @param file the file to inspect. + * @return true if the file meets the criteria for AI processing. + * @throws IOException if reading the file fails. + */ + private boolean isMailProcessingNeeded(File file) throws IOException { + // ignore hidden files + if (file.getName().startsWith(".")) { + return false; + } + + // Check if it's a regular file + if (!file.isFile()) { + return false; + } + + // Ensure the first line says "TOCOMPUTE:" + return fileHasToComputeMarker(file); + } + + /** + * Inspects the first line of the file to see if it starts with "TOCOMPUTE:". + * + * @param file the file to read. + * @return true if the file's first line starts with "TOCOMPUTE:". + * @throws IOException if file reading fails. + */ + private static boolean fileHasToComputeMarker(File file) throws IOException { + try (BufferedReader reader = new BufferedReader(new FileReader(file))) { + String firstLine = reader.readLine(); + return firstLine != null && firstLine.startsWith("TOCOMPUTE:"); + } + } + + /** + * Processes a file if it has the "TOCOMPUTE:" marker, running an AI + * query and appending the result to the file. Otherwise logs that + * it's being ignored. + * + * @param file the file to possibly process. + * @throws IOException if reading/writing the file fails. + * @throws InterruptedException if the AI query is interrupted. + */ + private void processMailIfNeeded(File file) throws IOException, InterruptedException { + if (!isMailProcessingNeeded(file)) { + System.out.println("Ignoring file: " + file.getName() + " (does not need processing for now)"); + return; + } + + System.out.println("\nReplying to mail: " + file.getName()); + + // Read the mail content + String inputFileContent = getFileContentsAsString(file); + + // Parse the relevant data into a MailQuery object + MailQuery mailQuery = parseInputFileContent(inputFileContent); + + // Create an AiTask and run the query + AiTask aiTask = new AiTask(mailQuery); + String aiGeneratedResponse = aiTask.runAiQuery(); + + // Build new content + StringBuilder resultFileContent = new StringBuilder(); + + // Ensure the user prompt block is labeled if it isn't already + if (!mailQuery.userPrompt.startsWith("* USER:\n")) { + resultFileContent.append("* USER:\n"); + } + resultFileContent.append(mailQuery.userPrompt).append("\n"); + + // Append the AI response block + resultFileContent + .append("* ASSISTANT:\n") + .append(aiGeneratedResponse) + .append("\n"); + + // Write the combined result back to the same file + saveToFile(file, resultFileContent.toString()); + } + + /** + * Converts the raw file content (including the line beginning with "TOCOMPUTE:") + * into a {@link MailQuery} object that the AI can process. + * + * @param inputFileContent the raw contents of the mail file. + * @return a {@link MailQuery} containing the system prompt, user prompt, and the selected model. + * @throws IOException if reading prompt files fails. + */ + private MailQuery parseInputFileContent(String inputFileContent) throws IOException { + MailQuery mailQuery = new MailQuery(); + + // Find the newline that separates "TOCOMPUTE: ..." from the rest + int firstNewLineIndex = inputFileContent.indexOf('\n'); + if (firstNewLineIndex == -1) { + throw new IllegalArgumentException("Input file is only one line long. Content: " + inputFileContent); + } else { + // The user prompt is everything after the first line + mailQuery.userPrompt = inputFileContent.substring(firstNewLineIndex + 1); + } + + // The first line will look like "TOCOMPUTE: model=... prompt=... etc." + String firstLine = inputFileContent.substring(0, firstNewLineIndex); + + // Parse out the key/value pairs + Map settings = parseSettings(firstLine); + + // Look up system prompt from the "prompt" alias + String promptAlias = settings.getOrDefault("prompt", "default"); + mailQuery.systemPrompt = configuration.getPromptByAlias(promptAlias); + + // Resolve model from the "model" alias + String modelAlias = settings.getOrDefault("model", "default"); + Optional modelOptional = modelLibrary.findModelByAlias(modelAlias); + if (!modelOptional.isPresent()) { + throw new IllegalArgumentException("Model with alias '" + modelAlias + "' not found."); + } + mailQuery.model = modelOptional.get(); + + return mailQuery; + } + + /** + * Parses the "TOCOMPUTE:" line, which should look like: + * 
TOCOMPUTE: key1=value1 key2=value2 ...
+ * + * @param toComputeLine the line beginning with "TOCOMPUTE:". + * @return a map of settings derived from that line. + */ + private Map parseSettings(String toComputeLine) { + if (!toComputeLine.startsWith("TOCOMPUTE:")) { + throw new IllegalArgumentException("Invalid TOCOMPUTE line: " + toComputeLine); + } + + // If there's nothing beyond "TOCOMPUTE:", just return an empty map + if (toComputeLine.length() <= "TOCOMPUTE: ".length()) { + return new HashMap<>(); + } + + // Example format: "TOCOMPUTE: prompt=writer model=mistral" + String[] parts = toComputeLine.substring("TOCOMPUTE: ".length()).split("\\s+"); + Map settings = new HashMap<>(); + + for (String part : parts) { + String[] keyValue = part.split("="); + if (keyValue.length == 2) { + settings.put(keyValue[0], keyValue[1]); + } + } + return settings; + } + + /** + * Handles the filesystem events from the WatchService (e.g. file creation + * or modification), then processes those files if necessary. + * + * @param key the watch key containing the events. + * @throws IOException if file reading/writing fails. + * @throws InterruptedException if the AI process is interrupted. + */ + private void processDetectedFilesystemEvents(WatchKey key) throws IOException, InterruptedException { + for (WatchEvent event : key.pollEvents()) { + WatchEvent.Kind kind = event.kind(); + + // Skip OVERFLOW event + if (kind == StandardWatchEventKinds.OVERFLOW) { + continue; + } + + // The filename for the event + Path filename = ((WatchEvent) event).context(); + System.out.println("Event: " + kind + " for file: " + filename); + + // Process the file + if (kind == ENTRY_CREATE || kind == ENTRY_MODIFY) { + File file = mailDir.toPath().resolve(filename).toFile(); + processMailIfNeeded(file); + } + } + } + + /** + * Registers the mail directory with a WatchService for ENTRY_CREATE + * and ENTRY_MODIFY events. + * + * @throws IOException if registration fails. + */ + private void initializeFileWatcher() throws IOException { + this.directoryWatcher = FileSystems.getDefault().newWatchService(); + Paths.get(mailDir.getAbsolutePath()).register(directoryWatcher, ENTRY_CREATE, ENTRY_MODIFY); + } +} diff --git a/src/main/java/eu/svjatoslav/alyverkko_cli/commands/mail_correspondant/MailQuery.java b/src/main/java/eu/svjatoslav/alyverkko_cli/commands/mail_correspondant/MailQuery.java new file mode 100644 index 0000000..a33584f --- /dev/null +++ b/src/main/java/eu/svjatoslav/alyverkko_cli/commands/mail_correspondant/MailQuery.java @@ -0,0 +1,41 @@ +package eu.svjatoslav.alyverkko_cli.commands.mail_correspondant; + +import eu.svjatoslav.alyverkko_cli.model.Model; + +/** + * Represents the data needed to perform a single mail-based AI query, + * containing prompts and the specific AI model to use. + */ +public class MailQuery { + + /** + * The system prompt text that sets the context or role for the AI. + * This is often used to establish rules or background instructions + * for how the assistant should behave. + */ + public String systemPrompt; + + /** + * The user's prompt text (the main request or query). + */ + public String userPrompt; + + /** + * The AI model to be used for processing this query. + */ + public Model model; + + /** + * Returns a string containing a summary of the {@link MailQuery} object. + * + * @return a string with the system prompt, user prompt, and model info. + */ + @Override + public String toString() { + return "MailQuery{" + + "systemPrompt='" + systemPrompt + '\'' + + ", userPrompt='" + userPrompt + '\'' + + ", model=" + model + + '}'; + } +} diff --git a/src/main/java/eu/svjatoslav/alyverkko_cli/configuration/Configuration.java b/src/main/java/eu/svjatoslav/alyverkko_cli/configuration/Configuration.java new file mode 100644 index 0000000..4e0c52e --- /dev/null +++ b/src/main/java/eu/svjatoslav/alyverkko_cli/configuration/Configuration.java @@ -0,0 +1,82 @@ +package eu.svjatoslav.alyverkko_cli.configuration; + +import com.fasterxml.jackson.annotation.JsonProperty; +import lombok.Data; + +import java.io.*; +import java.util.List; + +import static eu.svjatoslav.commons.file.IOHelper.getFileContentsAsString; + +/** + * Encapsulates all user configuration for the Älyverkko CLI application, + * such as model directories, mail directory, default temperature, + * llama-cli path, etc. + */ +@Data +public class Configuration { + + /** + * Directory where AI tasks (mail) are placed and discovered. + */ + @JsonProperty("mail_directory") + private File mailDirectory; + + /** + * Directory that contains AI model files in GGUF format. + */ + @JsonProperty("models_directory") + private File modelsDirectory; + + /** + * The default "temperature" used by the AI for creative/deterministic + * tradeoff. Ranges roughly between 0 and 3. + */ + @JsonProperty("default_temperature") + private Float defaultTemperature; + + /** + * The filesystem path to the llama-cli executable, which processes + * AI tasks via llama.cpp. + */ + @JsonProperty("llama_cli_path") + private File llamaCliPath; + + /** + * Number of CPU threads used for input prompt processing. + */ + @JsonProperty("batch_thread_count") + private Integer batchThreadCount; + + /** + * Number of CPU threads used for AI inference. + */ + @JsonProperty("thread_count") + private Integer threadCount; + + /** + * Directory containing text prompt files. Each file is a separate + * "prompt" by alias (the filename minus ".txt"). + */ + @JsonProperty("prompts_directory") + private File promptsDirectory; + + /** + * The list of models defined in this configuration. + */ + private List models; + + + /** + * Retrieves the contents of a prompt file by alias, e.g. "writer" + * maps to "writer.txt" in the prompts directory. + * + * @param alias the name of the prompt file (without ".txt"). + * @return the full text content of the prompt file. + * @throws IOException if reading the prompt file fails. + */ + public String getPromptByAlias(String alias) throws IOException { + File promptFile = new File(promptsDirectory, alias + ".txt"); + return getFileContentsAsString(promptFile); + } +} diff --git a/src/main/java/eu/svjatoslav/alyverkko_cli/configuration/ConfigurationHelper.java b/src/main/java/eu/svjatoslav/alyverkko_cli/configuration/ConfigurationHelper.java new file mode 100644 index 0000000..45891db --- /dev/null +++ b/src/main/java/eu/svjatoslav/alyverkko_cli/configuration/ConfigurationHelper.java @@ -0,0 +1,48 @@ +package eu.svjatoslav.alyverkko_cli.configuration; + +import com.fasterxml.jackson.databind.ObjectMapper; +import com.fasterxml.jackson.dataformat.yaml.YAMLFactory; +import eu.svjatoslav.commons.cli_helper.parameter_parser.parameter.FileOption; + +import java.io.File; +import java.io.IOException; + +public class ConfigurationHelper { + + /** + * The default path for the YAML config file, typically under the user's home directory. + */ + public static final String DEFAULT_CONFIG_FILE_PATH = "~/.config/alyverkko-cli/alyverkko-cli.yaml".replaceFirst("^~", System.getProperty("user.home")); + + /** + * Loads the configuration from a given file, or from the default + * path if {@code configFile} is null. + * + * @param configFile the file containing the YAML config; may be null. + * @return the {@link Configuration} object, or null if not found/invalid. + * @throws IOException if file I/O fails during reading. + */ + public static Configuration loadConfiguration(File configFile) throws IOException { + + if (!configFile.exists()) { + System.err.println("Configuration file not found: " + configFile); + return null; + } + + ObjectMapper mapper = new ObjectMapper(new YAMLFactory()); + return mapper.readValue(configFile, Configuration.class); + } + + /** + * Returns the configuration file from the given option, or the default path if not present. + * @param configFileOption the CLI option for the config file. + * @return the configuration file to load. + */ + public static File getConfigurationFile(FileOption configFileOption) { + if (configFileOption != null) + if (configFileOption.isPresent()) + return configFileOption.getValue(); + + return new File(DEFAULT_CONFIG_FILE_PATH); + } +} diff --git a/src/main/java/eu/svjatoslav/alyverkko_cli/configuration/ConfigurationModel.java b/src/main/java/eu/svjatoslav/alyverkko_cli/configuration/ConfigurationModel.java new file mode 100644 index 0000000..604f327 --- /dev/null +++ b/src/main/java/eu/svjatoslav/alyverkko_cli/configuration/ConfigurationModel.java @@ -0,0 +1,38 @@ +package eu.svjatoslav.alyverkko_cli.configuration; + +import com.fasterxml.jackson.annotation.JsonProperty; +import lombok.Data; + +/** + * Represents a single AI model configuration entry, including alias, + * path to the model file, token context size, and an optional + * end-of-text marker. + */ +@Data +public class ConfigurationModel { + + /** + * A short name for the model, e.g., "default" or "mistral". + */ + private String alias; + + /** + * The path to the model file (GGUF, etc.), relative to + * {@link Configuration#getModelsDirectory()} or fully qualified. + */ + @JsonProperty("filesystem_path") + private String filesystemPath; + + /** + * The maximum context size the model supports, in tokens. + */ + @JsonProperty("context_size_tokens") + private int contextSizeTokens; + + /** + * Optional text marker signifying the end of text for this model. + * If non-null, it will be used to strip trailing tokens from the AI response. + */ + @JsonProperty("end_of_text_marker") + private String endOfTextMarker; +} diff --git a/src/main/java/eu/svjatoslav/alyverkko_cli/model/Model.java b/src/main/java/eu/svjatoslav/alyverkko_cli/model/Model.java new file mode 100644 index 0000000..e985ac1 --- /dev/null +++ b/src/main/java/eu/svjatoslav/alyverkko_cli/model/Model.java @@ -0,0 +1,55 @@ +package eu.svjatoslav.alyverkko_cli.model; + +import java.io.File; + +/** + * Represents an AI model stored on the filesystem, including details such + * as path, context size, alias, and an optional end-of-text marker. + */ +public class Model { + + /** + * The path to the model file on the filesystem. + */ + public final File filesystemPath; + + /** + * The size of the context (in tokens) that this model is able to handle. + */ + public final int contextSizeTokens; + + /** + * A user-friendly alias for the model, e.g. "default" or "mistral". + */ + public final String alias; + + /** + * An optional marker indicating end of the AI-generated text (e.g., "###"). + * If non-null, it can be used to detect where the model has finished answering. + */ + public final String endOfTextMarker; + + /** + * Constructs a {@link Model} instance. + * + * @param filesystemPath The path to the model file on the filesystem. + * @param contextSizeTokens The size of the context in tokens. + * @param modelAlias A short alias by which the model is referenced. + * @param endOfTextMarker Optional text that signifies the end of the AI's output. + */ + public Model(File filesystemPath, int contextSizeTokens, String modelAlias, String endOfTextMarker) { + this.filesystemPath = filesystemPath; + this.contextSizeTokens = contextSizeTokens; + this.alias = modelAlias; + this.endOfTextMarker = endOfTextMarker; + } + + /** + * Prints the model's alias, path, and context size to standard output. + */ + public void printModelDetails() { + System.out.println("Model: " + alias); + System.out.println(" Path: " + filesystemPath); + System.out.println(" Context size: " + contextSizeTokens); + } +} diff --git a/src/main/java/eu/svjatoslav/alyverkko_cli/model/ModelLibrary.java b/src/main/java/eu/svjatoslav/alyverkko_cli/model/ModelLibrary.java new file mode 100644 index 0000000..b53f9e4 --- /dev/null +++ b/src/main/java/eu/svjatoslav/alyverkko_cli/model/ModelLibrary.java @@ -0,0 +1,126 @@ +package eu.svjatoslav.alyverkko_cli.model; + +import eu.svjatoslav.alyverkko_cli.Utils; +import eu.svjatoslav.alyverkko_cli.configuration.ConfigurationModel; + +import java.io.File; +import java.util.ArrayList; +import java.util.List; +import java.util.Optional; + +/** + * A container (library) for multiple AI models, providing + * functionality for adding and retrieving models by alias. + */ +public class ModelLibrary { + + /** + * The list of all successfully loaded models in this library. + */ + private final List models; + + /** + * The default model for this library (e.g., the first successfully + * loaded model in the list). + */ + private static Model defaultModel; + + /** + * Base directory containing the model files. + */ + private final File modelsBaseDirectory; + + /** + * Constructs a library of AI models from the provided list of + * {@link ConfigurationModel}s, ignoring those whose paths do not exist. + * + * @param modelsBaseDirectory the root directory where model files are stored. + * @param configModels a list of model configurations. + */ + public ModelLibrary(File modelsBaseDirectory, List configModels) { + this.modelsBaseDirectory = modelsBaseDirectory; + this.models = new ArrayList<>(); + + for (ConfigurationModel configModel : configModels) { + addModelFromConfig(configModel); + } + + if (models.isEmpty()) { + throw new RuntimeException("No models are defined!"); + } + + defaultModel = models.get(0); + } + + /** + * Attempts to construct a {@link Model} from the given + * {@link ConfigurationModel}, verifying that the file actually exists. + * + * @param configModel the configuration describing the model. + */ + private void addModelFromConfig(ConfigurationModel configModel) { + File modelFile = new File(modelsBaseDirectory, configModel.getFilesystemPath()); + if (!modelFile.exists()) { + Utils.printRedMessageToConsole("WARN: Model file not found: " + modelFile.getAbsolutePath() + " . Skipping model."); + return; + } + + addModel(new Model( + modelFile, + configModel.getContextSizeTokens(), + configModel.getAlias(), + configModel.getEndOfTextMarker() + )); + } + + /** + * Adds a model to the library if no model with the same alias + * already exists. + * + * @param model the model to add. + * @throws RuntimeException if a model with the same alias already exists. + */ + public void addModel(Model model) { + if (findModelByAlias(model.alias).isPresent()) { + throw new RuntimeException("Model with alias \"" + model.alias + "\" already exists!"); + } + models.add(model); + } + + /** + * @return the list of loaded models in this library. + */ + public List getModels() { + return models; + } + + /** + * Finds a model by its alias in this library. + * + * @param alias the model alias to look for. + * @return an {@link Optional} describing the found model, or empty if none match. + */ + public Optional findModelByAlias(String alias) { + return models.stream() + .filter(model -> model.alias.equals(alias)) + .findFirst(); + } + + /** + * @return the default model (first loaded model). + */ + public Model getDefaultModel() { + return defaultModel; + } + + /** + * Prints the details of each model in the library to standard output. + */ + public void printModels() { + System.out.println("Available models:\n"); + for (Model model : models) { + model.printModelDetails(); + System.out.println(); + } + } +} \ No newline at end of file diff --git a/src/test/java/eu/svjatoslav/alyverkko_cli/AiTaskTest.java b/src/test/java/eu/svjatoslav/alyverkko_cli/AiTaskTest.java new file mode 100644 index 0000000..f8aaa1f --- /dev/null +++ b/src/test/java/eu/svjatoslav/alyverkko_cli/AiTaskTest.java @@ -0,0 +1,24 @@ +package eu.svjatoslav.alyverkko_cli; + +import org.junit.jupiter.api.Test; + +import static org.testng.AssertJUnit.assertEquals; + +public class AiTaskTest { + + @Test + void testFilterUserInput() { + String input = "* ASSISTANT:\nHello\n* USER:\nHi!"; + String expectedOutput = "ASSISTANT:\nHello\nUSER:\nHi!"; + String output = AiTask.filterParticipantsInUserInput(input); + assertEquals(expectedOutput, output); + } + + @Test + void testFilterUserInputAnyNumberOfSpacesAtTheEnd() { + String input = "* ASSISTANT: \nHello"; + String expectedOutput = "ASSISTANT:\nHello"; + String output = AiTask.filterParticipantsInUserInput(input); + assertEquals(expectedOutput, output); + } +} \ No newline at end of file diff --git a/tools/implement idea b/tools/implement idea new file mode 100755 index 0000000..6472735 --- /dev/null +++ b/tools/implement idea @@ -0,0 +1,12 @@ +#!/bin/bash +cd "${0%/*}"; if [ "$1" != "T" ]; then gnome-terminal -e "'$0' T"; exit; fi; +cd .. + +read -p "Enter the topic name: " TOPIC_NAME + +alyverkko-cli joinfiles -t "$TOPIC_NAME" --src-dir . --pattern "*.org" +alyverkko-cli joinfiles -t "$TOPIC_NAME" --src-dir . --pattern "*.java" +alyverkko-cli joinfiles -t "$TOPIC_NAME" --src-dir . --pattern "implement*" +alyverkko-cli joinfiles -t "$TOPIC_NAME" --src-dir . --pattern "install" +alyverkko-cli joinfiles -t "$TOPIC_NAME" --src-dir . --pattern "uninstall" +alyverkko-cli joinfiles -t "$TOPIC_NAME" --edit diff --git a/tools/open with IntelliJ IDEA b/tools/open with IntelliJ IDEA new file mode 100755 index 0000000..304bf94 --- /dev/null +++ b/tools/open with IntelliJ IDEA @@ -0,0 +1,54 @@ +#!/bin/bash + +# This script launches IntelliJ IDEA with the current project +# directory. The script is designed to be run by double-clicking it in +# the GNOME Nautilus file manager. + +# First, we change the current working directory to the directory of +# the script. + +# "${0%/*}" gives us the path of the script itself, without the +# script's filename. + +# This command basically tells the system "change the current +# directory to the directory containing this script". + +cd "${0%/*}" + +# Then, we move up one directory level. +# The ".." tells the system to go to the parent directory of the current directory. +# This is done because we assume that the project directory is one level up from the script. +cd .. + +# Now, we use the 'setsid' command to start a new session and run +# IntelliJ IDEA in the background. 'setsid' is a UNIX command that +# runs a program in a new session. + +# The command 'idea .' opens IntelliJ IDEA with the current directory +# as the project directory. The '&' at the end is a UNIX command that +# runs the process in the background. The '> /dev/null' part tells +# the system to redirect all output (both stdout and stderr, denoted +# by '&') that would normally go to the terminal to go to /dev/null +# instead, which is a special file that discards all data written to +# it. + +setsid idea . &>/dev/null & + +# The 'disown' command is a shell built-in that removes a shell job +# from the shell's active list. Therefore, the shell will not send a +# SIGHUP to this particular job when the shell session is terminated. + +# '-h' option specifies that if the shell receives a SIGHUP, it also +# doesn't send a SIGHUP to the job. + +# '$!' is a shell special parameter that expands to the process ID of +# the most recent background job. +disown -h $! + + +sleep 2 + +# Finally, we use the 'exit' command to terminate the shell script. +# This command tells the system to close the terminal window after +# IntelliJ IDEA has been opened. +exit diff --git a/tools/update web site b/tools/update web site new file mode 100755 index 0000000..7507bff --- /dev/null +++ b/tools/update web site @@ -0,0 +1,35 @@ +#!/bin/bash +cd "${0%/*}"; if [ "$1" != "T" ]; then gnome-terminal -e "'$0' T"; exit; fi; + +cd .. + +# Build the project jar file and the apidocs. +mvn clean package + +# Export org to html using emacs in batch mode +( + cd doc/ + + rm -f index.html + emacs --batch -l ~/.emacs --visit=index.org --funcall=org-html-export-to-html --kill + + rm setup.html + emacs --batch -l ~/.emacs --visit=setup.org --funcall=org-html-export-to-html --kill +) + +# Generate class diagrams. See: https://www3.svjatoslav.eu/projects/javainspect/ +rm -rf doc/graphs/ +mkdir -p doc/graphs/ +javainspect -j target/alyverkko-cli-*-SNAPSHOT.jar -d doc/graphs/ -n "all classes" -t png -ho +meviz index -w doc/graphs/ -t "Älyverkko CLI program classes" + +# Copy the apidocs to the doc folder so that they can be uploaded to the server. +rm -rf doc/apidocs/ +cp -r target/apidocs/ doc/ + +# Upload project homepage to the server. +rsync -avz --delete -e 'ssh -p 10006' doc/ n0@www3.svjatoslav.eu:/mnt/big/projects/alyverkko-cli/ + +echo "" +echo "Press ENTER to close this window." +read diff --git a/uninstall b/uninstall new file mode 100755 index 0000000..b4a4474 --- /dev/null +++ b/uninstall @@ -0,0 +1,13 @@ +#!/bin/bash + +SYSTEMD_SERVICE_FILE="/etc/systemd/system/alyverkko-cli.service" + +sudo systemctl stop alyverkko-cli +sudo systemctl disable alyverkko-cli +sudo rm "$SYSTEMD_SERVICE_FILE" +sudo rm -rf /opt/alyverkko-cli/ + +read -p "Do you want to remove user configuration as well? (y/N) " remove_config +if [[ $remove_config == [Yy] ]]; then + sudo rm -rf "${HOME}/.config/alyverkko-cli" +fi