--- /dev/null
+/.idea/
+/.settings/
+/target/
+/*.iml
+/*.log
+/test/
+
+/doc/apidocs/
+/doc/graphs/
+/doc/index.html
+
+
--- /dev/null
+Creative Commons Legal Code
+
+CC0 1.0 Universal
+
+ CREATIVE COMMONS CORPORATION IS NOT A LAW FIRM AND DOES NOT PROVIDE
+ LEGAL SERVICES. DISTRIBUTION OF THIS DOCUMENT DOES NOT CREATE AN
+ ATTORNEY-CLIENT RELATIONSHIP. CREATIVE COMMONS PROVIDES THIS
+ INFORMATION ON AN "AS-IS" BASIS. CREATIVE COMMONS MAKES NO WARRANTIES
+ REGARDING THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS
+ PROVIDED HEREUNDER, AND DISCLAIMS LIABILITY FOR DAMAGES RESULTING FROM
+ THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS PROVIDED
+ HEREUNDER.
+
+Statement of Purpose
+
+The laws of most jurisdictions throughout the world automatically confer
+exclusive Copyright and Related Rights (defined below) upon the creator
+and subsequent owner(s) (each and all, an "owner") of an original work of
+authorship and/or a database (each, a "Work").
+
+Certain owners wish to permanently relinquish those rights to a Work for
+the purpose of contributing to a commons of creative, cultural and
+scientific works ("Commons") that the public can reliably and without fear
+of later claims of infringement build upon, modify, incorporate in other
+works, reuse and redistribute as freely as possible in any form whatsoever
+and for any purposes, including without limitation commercial purposes.
+These owners may contribute to the Commons to promote the ideal of a free
+culture and the further production of creative, cultural and scientific
+works, or to gain reputation or greater distribution for their Work in
+part through the use and efforts of others.
+
+For these and/or other purposes and motivations, and without any
+expectation of additional consideration or compensation, the person
+associating CC0 with a Work (the "Affirmer"), to the extent that he or she
+is an owner of Copyright and Related Rights in the Work, voluntarily
+elects to apply CC0 to the Work and publicly distribute the Work under its
+terms, with knowledge of his or her Copyright and Related Rights in the
+Work and the meaning and intended legal effect of CC0 on those rights.
+
+1. Copyright and Related Rights. A Work made available under CC0 may be
+protected by copyright and related or neighboring rights ("Copyright and
+Related Rights"). Copyright and Related Rights include, but are not
+limited to, the following:
+
+ i. the right to reproduce, adapt, distribute, perform, display,
+ communicate, and translate a Work;
+ ii. moral rights retained by the original author(s) and/or performer(s);
+iii. publicity and privacy rights pertaining to a person's image or
+ likeness depicted in a Work;
+ iv. rights protecting against unfair competition in regards to a Work,
+ subject to the limitations in paragraph 4(a), below;
+ v. rights protecting the extraction, dissemination, use and reuse of data
+ in a Work;
+ vi. database rights (such as those arising under Directive 96/9/EC of the
+ European Parliament and of the Council of 11 March 1996 on the legal
+ protection of databases, and under any national implementation
+ thereof, including any amended or successor version of such
+ directive); and
+vii. other similar, equivalent or corresponding rights throughout the
+ world based on applicable law or treaty, and any national
+ implementations thereof.
+
+2. Waiver. To the greatest extent permitted by, but not in contravention
+of, applicable law, Affirmer hereby overtly, fully, permanently,
+irrevocably and unconditionally waives, abandons, and surrenders all of
+Affirmer's Copyright and Related Rights and associated claims and causes
+of action, whether now known or unknown (including existing as well as
+future claims and causes of action), in the Work (i) in all territories
+worldwide, (ii) for the maximum duration provided by applicable law or
+treaty (including future time extensions), (iii) in any current or future
+medium and for any number of copies, and (iv) for any purpose whatsoever,
+including without limitation commercial, advertising or promotional
+purposes (the "Waiver"). Affirmer makes the Waiver for the benefit of each
+member of the public at large and to the detriment of Affirmer's heirs and
+successors, fully intending that such Waiver shall not be subject to
+revocation, rescission, cancellation, termination, or any other legal or
+equitable action to disrupt the quiet enjoyment of the Work by the public
+as contemplated by Affirmer's express Statement of Purpose.
+
+3. Public License Fallback. Should any part of the Waiver for any reason
+be judged legally invalid or ineffective under applicable law, then the
+Waiver shall be preserved to the maximum extent permitted taking into
+account Affirmer's express Statement of Purpose. In addition, to the
+extent the Waiver is so judged Affirmer hereby grants to each affected
+person a royalty-free, non transferable, non sublicensable, non exclusive,
+irrevocable and unconditional license to exercise Affirmer's Copyright and
+Related Rights in the Work (i) in all territories worldwide, (ii) for the
+maximum duration provided by applicable law or treaty (including future
+time extensions), (iii) in any current or future medium and for any number
+of copies, and (iv) for any purpose whatsoever, including without
+limitation commercial, advertising or promotional purposes (the
+"License"). The License shall be deemed effective as of the date CC0 was
+applied by Affirmer to the Work. Should any part of the License for any
+reason be judged legally invalid or ineffective under applicable law, such
+partial invalidity or ineffectiveness shall not invalidate the remainder
+of the License, and in such case Affirmer hereby affirms that he or she
+will not (i) exercise any of his or her remaining Copyright and Related
+Rights in the Work or (ii) assert any associated claims and causes of
+action with respect to the Work, in either case contrary to Affirmer's
+express Statement of Purpose.
+
+4. Limitations and Disclaimers.
+
+ a. No trademark or patent rights held by Affirmer are waived, abandoned,
+ surrendered, licensed or otherwise affected by this document.
+ b. Affirmer offers the Work as-is and makes no representations or
+ warranties of any kind concerning the Work, express, implied,
+ statutory or otherwise, including without limitation warranties of
+ title, merchantability, fitness for a particular purpose, non
+ infringement, or the absence of latent or other defects, accuracy, or
+ the present or absence of errors, whether or not discoverable, all to
+ the greatest extent permissible under applicable law.
+ c. Affirmer disclaims responsibility for clearing rights of other persons
+ that may apply to the Work or any use thereof, including without
+ limitation any person's Copyright and Related Rights in the Work.
+ Further, Affirmer disclaims responsibility for obtaining any necessary
+ consents, permissions or other rights required for any use of the
+ Work.
+ d. Affirmer understands and acknowledges that Creative Commons is not a
+ party to this document and has no duty or obligation with respect to
+ this CC0 or use of the Work.
--- /dev/null
+#!/bin/bash
+
+set -f
+
+java -Xmx4500m -classpath /opt/alyverkko-cli/* eu.svjatoslav.alyverkko_cli.Main "$@"
+
--- /dev/null
+mail_directory: "/home/user/AI/mail"
+models_directory: "/home/user/AI/models"
+default_temperature: 0.7
+llama_cpp_executable_path: "/home/user/AI/llama.cpp/main"
+batch_thread_count: 10
+thread_count: 6
+models:
+ - alias: "wizard"
+ filesystem_path: "WizardLM-2-8x22B.Q5_K_M-00001-of-00005.gguf"
+ context_size_tokens: 64000
+ end_of_text_marker: null
+
--- /dev/null
+#+SETUPFILE: ~/.emacs.d/org-styles/html/darksun.theme
+#+TITLE: Älyverkko CLI application
+#+LANGUAGE: en
+#+LATEX_HEADER: \usepackage[margin=1.0in]{geometry}
+#+LATEX_HEADER: \usepackage{parskip}
+#+LATEX_HEADER: \usepackage[none]{hyphenat}
+
+#+OPTIONS: H:20 num:20
+#+OPTIONS: author:nil
+
+* General
+- This program is free software: released under Creative Commons Zero
+ (CC0) license
+
+- Program author:
+ - Svjatoslav Agejenko
+ - Homepage: https://svjatoslav.eu
+ - Email: mailto://svjatoslav@svjatoslav.eu
+
+- [[https://www.svjatoslav.eu/projects/][Other software projects hosted at svjatoslav.eu]]
+
+** Source code
+:PROPERTIES:
+:ID: f5740953-079b-40f4-87d8-b6d1635a8d39
+:END:
+- [[https://www2.svjatoslav.eu/gitweb/?p=alyverkko-cli.git;a=snapshot;h=HEAD;sf=tgz][Download latest snapshot in TAR GZ format]]
+
+- [[https://www2.svjatoslav.eu/gitweb/?p=alyverkko-cli.git;a=summary][Browse Git repository online]]
+
+- Clone Git repository using command:
+ : git clone https://www3.svjatoslav.eu/git/alyverkko-cli.git
+
+- See [[https://www3.svjatoslav.eu/projects/alyverkko-cli/apidocs/][JavaDoc]].
+
+- See [[https://www3.svjatoslav.eu/projects/alyverkko-cli/graphs/][automatically generated class diagrams]]. Here is [[https://www3.svjatoslav.eu/projects/javainspect/legend.png][legend]] to help
+ understand diagrams. Diagrams were generated using [[https://www3.svjatoslav.eu/projects/javainspect/][JavaInspect tool]].
+
+* Introduction
+:PROPERTIES:
+:GPTEL_TOPIC: introduction
+:END:
+
+The *Älyverkko CLI* application is a user-friendly tool developed in
+Java, specifically tailored to streamline the utilization of expansive
+language models through CPU-based computation in batch processing
+mode.
+
+To illustrate its capabilities: Imagine harnessing the power of a vast
+language model, boasting around 100 billion parameters, solely relying
+on CPU computations and leveraging the open-source software
+llama.cpp. This setup requires a modern consumer-grade CPU and
+approximately 128 GB of RAM. To put this into perspective, 128 GB of
+RAM is financially comparable to purchasing a high-quality smartphone,
+making it an economical option for many users.
+
+In contrast, executing the same model on an Nvidia GPU could result in
+costs that are significantly higher, potentially by an order of
+magnitude.
+
+However, there is a trade-off: CPU-based processing for such extensive
+models is inherently slow. This means real-time interaction, like
+chatting with the AI, wouldn't be practical or enjoyable due to the
+lag in response times. Nevertheless, when deployed in a
+non-interactive batch mode, this "slow but smart" AI can complete
+numerous valuable tasks within a 24-hour window. For instance, it
+could generate a substantial amount of code, potentially exceeding
+what you could thoroughly review in the same timeframe. Additionally,
+it could process more documents than most individuals would be
+inclined to read manually.
+
+The primary objective of the *Älyverkko CLI* project is to identify
+and enable applications where this "slow but smart" AI can excel. By
+utilizing llama.cpp as its inference engine, the project aims to
+unlock a variety of uses where batch processing is more advantageous
+than real-time interaction.
+
+Here are some practical examples of tasks suited for the
+*Älyverkko CLI* application:
+
+- *Automated Report Generation*: The AI can analyze large datasets
+ overnight and produce comprehensive reports by morning, saving
+ countless hours of manual analysis.
+- *Code Development Support*: Developers could use the AI to generate
+ code snippets or even entire modules based on specifications
+ provided, which they can then integrate and refine during their
+ workday.
+- *Content Creation*: The AI can draft articles, create outlines for
+ books, or compile research notes, providing a solid foundation for
+ content creators to edit and finalize.
+- *Data Processing*: It can process and organize large volumes of
+ unstructured data, such as customer feedback or scientific
+ observations, into structured formats ready for human review.
+
+In summary, the *Älyverkko CLI* application opens up a realm of
+possibilities for leveraging powerful AI in scenarios where immediate
+responses are not critical, but high-quality batch processing output
+is highly beneficial.
+
+Note: project is still in early stage.
+
+* Setup
+** Installation
+:PROPERTIES:
+:ID: 0b705a37-9b84-4cd5-878a-fedc9ab09b12
+:END:
+At the moment, to use Älyverkko CLI, you need to:
+- Download sources and build [[https://github.com/ggerganov/llama.cpp][llama.cpp]] project.
+- Download [[id:f5740953-079b-40f4-87d8-b6d1635a8d39][sources]] and build Älyverkko CLI project.
+- Download one or more pre-trained large language models in GGUF
+ format. Hugging Face repository [[https://huggingface.co/models?search=GGUF][has lot of them]]. My favorite is
+ [[https://huggingface.co/MaziyarPanahi/WizardLM-2-8x22B-GGUF/tree/main][WizardLM-2-8x22B]] for strong problem solving skills.
+
+Follow instructions for obtaining and building Älyverkko CLI on your
+computer that runs Debian 12 operating system:
+
+1. Ensure that you have Java Development Kit (JDK) installed on your
+ system.
+ : sudo apt-get install openjdk-17-jdk
+
+2. Ensure that you have Apache Maven installed:
+ : sudo apt-get install maven
+
+3. Clone the [[id:f5740953-079b-40f4-87d8-b6d1635a8d39][code repository]] or download the [[id:f5740953-079b-40f4-87d8-b6d1635a8d39][source code]] for the
+ `alyverkko-cli` application to your local machine.
+
+4. Navigate to the root directory of the cloned/downloaded project in
+ your terminal.
+
+5. Execute the installation script by running
+ : ./install
+
+ This script will compile the application and install it to
+ directory
+ : /opt/alyverkko-cli
+
+ To facilitate usage from command-line, it will also define
+ system-wide command *alyverkko-cli* as well as "Älyverkko CLI"
+ launcher in desktop applications menu.
+
+6. Prepare Älyverkko CLI [[id:0fcdae48-81c5-4ae1-bdb9-64ae74e87c45][configuration]] file.
+
+7. Verify that the application has been installed correctly by running
+ *alyverkko-cli* in your terminal.
+
+** Configuration
+:PROPERTIES:
+:ID: 0fcdae48-81c5-4ae1-bdb9-64ae74e87c45
+:END:
+Älyverkko CLI application configuration is done by editing YAML
+formatted configuration file.
+
+Configuration file should be placed under current user home directory:
+: ~/.config/alyverkko-cli.yaml
+
+*** Configuration file example
+
+The application is configured using a YAML-formatted configuration
+file. Below is an example of how the configuration file might look:
+
+#+begin_src yaml
+ mail_directory: "/home/user/AI/mail"
+ models_directory: "/home/user/AI/models"
+ default_temperature: 0.7
+ llama_cpp_executable_path: "/home/user/AI/llama.cpp/main"
+ batch_thread_count: 10
+ thread_count: 6
+ models:
+ - alias: "wizard"
+ filesystem_path: "WizardLM-2-8x22B.Q5_K_M-00001-of-00005.gguf"
+ context_size_tokens: 64000
+ end_of_text_marker: null
+ - alias: "maid"
+ filesystem_path: "daringmaid-20b.Q4_K_M.gguf"
+ context_size_tokens: 4096
+ end_of_text_marker: null
+
+#+end_src
+
+*** Configuration file syntax
+
+Here are available parameters:
+
+- mail_directory :: Directory where AI will look for files that
+ contain problems to solve.
+
+- models_directory :: Directory where AI models are stored.
+ - This option is mandatory.
+
+- default_temperature :: Defines the default temperature for AI
+ responses, affecting randomness in the generation process. Lower
+ values make the AI more deterministic and higher values make it more
+ creative or random.
+ - Default value: 0.7
+
+- llama_cpp_executable_path :: Specifies the file path to the
+ *llama.cpp* *main* executable.
+ - Example Value: /home/user/AI/llama.cpp/main
+ - This option is mandatory.
+
+- batch_thread_count :: Specifies the number of threads to use for
+ input prompt processing. CPU computing power is usually the
+ bottleneck here.
+ - Default value: 10
+
+- thread_count :: Sets the number of threads to be used by the AI
+ during response generation. RAM data transfer speed is usually
+ bottleneck here. When RAM bandwidth is saturated, increasing thread
+ count will no longer increase processing speed, but it will still
+ keep CPU cores unnecessarily busy.
+ - Default value: 6
+
+- models :: List of available large language models. First model in
+ the list would be used by default.
+
+ - alias :: Short model alias.
+ - filesystem_path :: File name of the model as located within
+ *models_directory*
+ - context_size_tokens :: Context size in tokens that model was
+ trained on.
+ - end_of_text_marker :: Some models produce certain markers to
+ indicate end of their output. If specified here, Älyverkko CLI can
+ identify and remove them so that they don't leak into
+ conversation. Default value is: *null*.
+
+*** Enlisting available models
+Once Älyverkko CLI is installed and properly configured, you can run
+following command at commandline to see what models are available to
+it:
+
+: alyverkko-cli listmodels
+
+** Starting daemon
+
+Älyverkko CLI keeps continuously listening for and processing tasks
+from a specified mail directory.
+
+There are multiple alternative ways to start Älyverkko CLI in mail
+processing mode:
+
+**** Start via command line interface
+
+1. Open your terminal.
+
+2. Run the command:
+ : alyverkko-cli mail
+
+3. The application will start monitoring the configured mail directory
+ for incoming messages and process them accordingly in endless loop.
+
+4. To terminate Älyverkko CLI, just hit *CTRL+c* on the keyboard, or
+ close terminal window.
+
+**** Start using your desktop environment application launcher
+
+1. Access the application launcher or application menu on your desktop
+ environment.
+
+2. Search for "Älyverkko CLI".
+
+3. Click on the icon to start the application. It will open its own
+ terminal.
+
+4. If you want to stop Älyverkko CLI, just close terminal window.
+
+**** Start in the background as systemd system service
+
+During Älyverkko CLI [[id:0b705a37-9b84-4cd5-878a-fedc9ab09b12][installation]], installation script will prompt you
+if you want to install *systemd* service. If you chose *Y*, Alyverkko
+CLI would be immediately started in the background as a system
+service. Also it will be automatically started on every system reboot.
+
+To view service status, use:
+: systemctl -l status alyverkko-cli
+
+If you want to stop or disable service, you can do so using systemd
+facilities:
+
+: sudo systemctl stop alyverkko-cli
+: sudo systemctl disable alyverkko-cli
+
+* Usage
+:PROPERTIES:
+:ID: 883d6e7c-60e0-422b-8c00-5cdc9dfec20d
+:END:
+
+The Älyverkko CLI application expects input files for processing in
+the form of plain text files within the specified mail directory
+(configured in the [[id:0fcdae48-81c5-4ae1-bdb9-64ae74e87c45][YAML configuration file]]). Each file should begin
+with a `TOCOMPUTE:` marker on the first line to be considered for
+processing.
+
+When the application detects a new or modified file in the mail
+directory:
+
+1. It checks if file has "TOCOMPUTE:" on the first line. If no, file
+ is ignored. Otherwise Älyverkko CLI continues processing the file.
+
+2. It reads the content of the file and feeds it as an input for an AI
+ model to generate a response.
+
+4. Once the AI has generated a response, the application appends this
+ response to the original mail contents within the same file, using
+ org-mode syntax to distinguish between the user's query and the
+ assistant's reply. The updated file will contain both the original
+ query (prefixed with: "* USER:*") and the AI's response (prefixed
+ with "* ASSISTANT:"), ensuring a clear and organized conversation
+ thread. "TOCOMPUTE:" is removed from the beginning of the file to
+ avoid processing same file again.
+
+Suggested way to use mail processing mode is to prepare assignments
+within the Älyverkko CLI mail directory using normal text editor. Feel
+free to save intermediary states. Once AI assignment is ready, add
+: TOCOMPUTE:
+to the beginning of the file and save one last time. Älyverkko CLI
+will detect new task within one second and will start processing it.
+
+If your text editor automatically reloads file when it was changed by
+other process in the filesystem, AI response will appear within text
+editor as soon as AI response is ready. If needed, you can add further
+queries at the end of the file and re-add "TOCOMPUTE:" at the
+beginning of the file. This way AI will process file again and file
+becomes stateful conversation. If you use GNU Emacs text editor, you
+can benefit from [[id:25038854-c905-4b26-9670-cca06600223e][purpose-built GNU Emacs utilities]].
+
+**** Helpful GNU Emacs utilities
+:PROPERTIES:
+:ID: 25038854-c905-4b26-9670-cca06600223e
+:END:
+
+Note: GNU Emacs and following Emacs Lisp utilities are not required to
+use Älyverkko CLI. Their purpose is to increase comfort for existing
+GNU Emacs users.
+
+***** Easily compose new problem statement for AI from emacs
+:PROPERTIES:
+:GPTEL_TOPIC: create-new-topic-dedicated-file
+:END:
+
+The Elisp function *ai-new-topic* facilitates the creation and opening
+of a new Org-mode file dedicated to a user-defined topic within a
+specified directory. Now you can use this file within emacs to compose
+you problem statement to AI.
+
+When *ai-new-topic* function triggered, it first prompts the user to
+input a topic name. This name will serve as the basis for the filename
+and the title within the document.
+
+The function then constructs a file path by concatenating the
+pre-defined =ai-topic-files-directory= (which should be set to your
+topics directory), the topic name, and the =.org= extension. If a file
+with this path does not already exist, the function will create a new
+file and open it for editing.
+
+#+begin_src elisp :results none
+(defvar ai-topic-files-directory "/home/user/my-ai-mail-directory/"
+ "Directory where topic files are stored. Set it to directory you want to use.")
+
+(defun ai-new-topic ()
+ "Create and open a topic file in the specified directory."
+ (interactive)
+ (let ((topic (read-string "Enter topic name: ")))
+ (let ((file-path (concat ai-topic-files-directory topic ".org")))
+ (if (not (file-exists-p file-path))
+ (with-temp-file file-path
+ (insert "#+TITLE: " topic "\n\n")))
+ (find-file file-path)
+ (goto-char (point-max))
+ (org-mode))))
+#+end_src
+
+***** Easily signal to AI that problem statement is ready for solving
+When *ai-compute* function is triggered, it inserts "TOCOMPUTE:" line
+at the beginning of file and saves it. Marking it for processing by
+AI.
+
+#+begin_src elisp :results none
+ (defun ai-compute ()
+ "Inserts 'TOCOMPUTE:' at the beginning of the buffer."
+ (interactive)
+ (goto-char (point-min)) ; Move to the beginning of the buffer
+ (insert "TOCOMPUTE:\n") ; Insert the string followed by a new line
+ (save-buffer) ; Save the buffer
+ )
+#+end_src
+
+* TODO
+
+Ideas to be possibly implemented in the future:
+
+** System operation
+- Implement CPU nice priority for inference processes to minimize the
+ impact on system responsiveness during heavy computations.
+- Enable model selection per individual inference task, allowing for
+ dynamic adjustment based on task requirements.
+- Allow specification of custom prompts for each inference task to
+ tailor interactions precisely.
+- Introduce an aliasing system for frequently used prompts,
+ streamlining the prompt selection process.
+- Consider implementing a plugin architecture to allow third-party
+ developers to extend Älyverkko CLI's functionality with custom
+ modules or integrations.
+- Possibility to easily pause and resume Älyverkko CLI without loosing
+ in-progress computation. Unix process stop and continue signals
+ could possibly be used.
+** Data management
+- Develop a feature to recursively aggregate files into a single
+ document using Emacs org-mode syntax, facilitating the preparation
+ of comprehensive problem statements for AI processing.
+ - Ensure that binary files are excluded from this aggregation
+ process to maintain text readability and compatibility.
+
+** Configuration and logging
+- Implement a fallback mechanism to use a system-wide configuration
+ file located in `/etc/` if no user-specific configuration is found,
+ enhancing out-of-the-box usability for new users.
+- Introduce optional logging of `llama.cpp` output to aid in debugging
+ and performance monitoring without cluttering the standard output.
+
+** Integration with external services
+- Add capabilities to connect with Jira, fetch content, and
+ potentially update issues or comments based on AI processing
+ results.
+- Implement similar integration with Confluence for content retrieval
+ and updates.
+- Extend the application's reach by adding the ability to interact
+ with arbitrary web sites, enabling information extraction and
+ automation of web-based tasks.
+
+** Tooling enhancements
+- Incorporate Python script execution capabilities directly by the AI,
+ expanding the range of available data manipulation and processing
+ tools.
+- Integrate relational database access to leverage structured data
+ storage and retrieval in workflows.
+- Enable web request functionality to interact with RESTful APIs or
+ scrape web content as part of task execution.
+- Introduce a notebook feature that allows the AI to maintain and
+ reference its own notes, fostering context retention across tasks.
+
+** Multimedia processing
+- Extend the application's capabilities to include voice capture and
+ processing, opening up new avenues for interaction beyond text-based
+ communication.
+- Implement image capture and processing features, enabling tasks that
+ involve image analysis or content extraction from visual data.
+
+** Task queue management
+- Refactor the task queue mechanism to support:
+ - Multiple task sources, including a REST API endpoint for
+ programmatic task submission.
+ - Load balancing across multiple executors (instances of Älyverkko
+ CLI) with dynamic registration and unregistration without system
+ interruption.
+ - Task priority assignments, ensuring that critical tasks are
+ processed in a timely manner.
+
+** User interface development
+- Create a web-based UI to provide users with an interface for task
+ submission and result retrieval, improving accessibility and user
+ experience.
+- Integrate Quality of Service (QoS) concepts within the UI to ensure
+ equitable resource allocation among users.
+- Implement administrative features for managing user accounts and
+ system resources, maintaining a secure and efficient operating
+ environment.
--- /dev/null
+#!/bin/bash
+
+# Define constants for paths and filenames
+INSTALL_DIR="/opt/alyverkko-cli"
+BINARY="alyverkko-cli"
+DESKTOP_FILE="alyverkko-cli.desktop"
+ICON_FILE="$INSTALL_DIR/logo.png"
+APPLICATIONS_DIR="/usr/share/applications"
+SYSTEM_SERVICE_DIR="/etc/systemd/system"
+CONFIG_DIR="${HOME}/.config/"
+
+# Ask user if they want to install systemd service
+echo "Do you want to install Älyverkko CLI as a systemd service? (y/N)"
+read install_service
+
+# Build the application
+mvn --settings maven.xml clean package
+
+# Install the binary and jar
+sudo mkdir -p "$INSTALL_DIR"
+sudo rm -f "$INSTALL_DIR/$BINARY.jar"
+sudo cp target/"$BINARY".jar "$INSTALL_DIR/$BINARY.jar"
+sudo cp "$BINARY" "$INSTALL_DIR/$BINARY"
+
+# Create a symbolic link to run the program from CLI
+sudo rm -f /usr/bin/"$BINARY"
+sudo ln -s "$INSTALL_DIR/$BINARY" /usr/bin/"$BINARY"
+
+# Install the desktop launcher
+cat <<EOF | sudo tee "$APPLICATIONS_DIR/$DESKTOP_FILE" > /dev/null
+[Desktop Entry]
+Type=Application
+Terminal=true
+Name=Älyverkko CLI
+Comment=Runner for artificial neural network service
+Icon=$ICON_FILE
+Exec=/opt/alyverkko-cli/$BINARY mail
+Categories=Development;
+EOF
+
+# Make sure the desktop file is executable
+sudo chmod 644 "$APPLICATIONS_DIR/$DESKTOP_FILE"
+
+# Copy the icon (assuming you have an icon named alyverkko-cli.svg)
+sudo cp logo.png "$ICON_FILE"
+
+# Pre-deploy example configuration YAML file
+sudo mkdir -p "$CONFIG_DIR"
+
+if [ -f "$CONFIG_DIR/alyverkko-cli.yaml" ]; then
+ echo "Configuration file already exists. Skipping pre-deployment."
+else
+ sudo cp alyverkko-cli.yaml "$CONFIG_DIR"
+fi
+
+# Install systemd service if requested
+if [[ $install_service == [Yy] ]]; then
+
+cat <<EOF | sudo tee "$SYSTEM_SERVICE_DIR/alyverkko-cli.service" > /dev/null
+[Unit]
+Description=Älyverkko CLI daemon in mail mode
+After=network.target
+
+[Service]
+User=$USER
+ExecStart=/opt/alyverkko-cli/alyverkko-cli mail
+WorkingDirectory=/opt/alyverkko-cli
+Nice=10
+Restart=always
+RestartSec=10
+
+[Install]
+WantedBy=multi-user.target
+EOF
+ sudo systemctl daemon-reload
+ sudo systemctl enable alyverkko-cli
+ sudo systemctl start alyverkko-cli
+ echo "Systemd service installed, enabled and started. Service status is:"
+ systemctl -l status alyverkko-cli
+fi
+
+echo "Installation complete."
--- /dev/null
+<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0
+ http://maven.apache.org/xsd/settings-1.0.0.xsd">
+ <localRepository/>
+ <interactiveMode/>
+ <usePluginRegistry/>
+ <offline/>
+ <pluginGroups/>
+ <servers/>
+ <mirrors/>
+ <proxies/>
+ <profiles/>
+ <activeProfiles/>
+</settings>
\ No newline at end of file
--- /dev/null
+<project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://maven.apache.org/POM/4.0.0"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+ <groupId>eu.svjatoslav</groupId>
+ <artifactId>alyverkko-cli</artifactId>
+ <version>1.0-SNAPSHOT</version>
+ <name>Älyverkko CLI</name>
+ <description>AI engine wrapper</description>
+
+ <properties>
+ <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
+ <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
+ </properties>
+
+ <organization>
+ <name>svjatoslav.eu</name>
+ <url>https://svjatoslav.eu</url>
+ </organization>
+
+ <dependencies>
+ <dependency>
+ <groupId>eu.svjatoslav</groupId>
+ <artifactId>svjatoslavcommons</artifactId>
+ <version>1.8</version>
+ </dependency>
+ <dependency>
+ <groupId>eu.svjatoslav</groupId>
+ <artifactId>cli-helper</artifactId>
+ <version>1.2</version>
+ </dependency>
+ <dependency>
+ <groupId>org.testng</groupId>
+ <artifactId>testng</artifactId>
+ <version>7.7.0</version>
+ <scope>test</scope>
+ </dependency>
+ <dependency>
+ <groupId>org.junit.jupiter</groupId>
+ <artifactId>junit-jupiter</artifactId>
+ <version>RELEASE</version>
+ <scope>test</scope>
+ </dependency>
+ <dependency>
+ <groupId>com.fasterxml.jackson.core</groupId>
+ <artifactId>jackson-databind</artifactId>
+ <version>2.13.4.1</version>
+ </dependency>
+ <dependency>
+ <groupId>com.fasterxml.jackson.dataformat</groupId>
+ <artifactId>jackson-dataformat-yaml</artifactId>
+ <version>2.13.0</version>
+ </dependency>
+ </dependencies>
+
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-compiler-plugin</artifactId>
+ <version>3.8.1</version>
+ <configuration>
+ <source>11</source>
+ <target>11</target>
+ <optimize>true</optimize>
+ <encoding>UTF-8</encoding>
+ </configuration>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-source-plugin</artifactId>
+ <version>2.2.1</version>
+ <executions>
+ <execution>
+ <id>attach-sources</id>
+ <goals>
+ <goal>jar</goal>
+ </goals>
+ </execution>
+ </executions>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-javadoc-plugin</artifactId>
+ <version>2.10.4</version>
+ <executions>
+ <execution>
+ <id>attach-javadocs</id>
+ <goals>
+ <goal>jar</goal>
+ </goals>
+ </execution>
+ </executions>
+ <configuration>
+ <!-- workaround for https://bugs.openjdk.java.net/browse/JDK-8212233 -->
+ <javaApiLinks>
+ <property>
+ <name>foo</name>
+ <value>bar</value>
+ </property>
+ </javaApiLinks>
+ <!-- Workaround for https://stackoverflow.com/questions/49472783/maven-is-unable-to-find-javadoc-command -->
+ <javadocExecutable>${java.home}/bin/javadoc</javadocExecutable>
+ </configuration>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-resources-plugin</artifactId>
+ <version>2.4.3</version>
+ <configuration>
+ <encoding>UTF-8</encoding>
+ </configuration>
+ </plugin>
+
+ <plugin>
+ <artifactId>maven-assembly-plugin</artifactId>
+
+ <configuration>
+ <archive>
+ <manifest>
+ <mainClass>eu.svjatoslav.alyverkko_cli.Main</mainClass>
+ </manifest>
+ </archive>
+ <descriptorRefs>
+ <descriptorRef>jar-with-dependencies</descriptorRef>
+ </descriptorRefs>
+ <finalName>alyverkko-cli</finalName>
+ <appendAssemblyId>false</appendAssemblyId>
+ </configuration>
+
+ <executions>
+ <execution>
+ <id>package-jar-with-dependencies</id>
+ <phase>package</phase>
+ <goals>
+ <goal>single</goal>
+ </goals>
+ <configuration>
+ <descriptorRefs>
+ <descriptorRef>jar-with-dependencies</descriptorRef>
+ </descriptorRefs>
+ <archive>
+ <manifest>
+ <mainClass>eu.svjatoslav.alyverkko_cli.Main</mainClass>
+ </manifest>
+ </archive>
+ </configuration>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+
+ <extensions>
+ <extension>
+ <groupId>org.apache.maven.wagon</groupId>
+ <artifactId>wagon-ssh-external</artifactId>
+ <version>2.6</version>
+ </extension>
+ </extensions>
+ </build>
+
+
+ <distributionManagement>
+ <snapshotRepository>
+ <id>svjatoslav.eu</id>
+ <name>svjatoslav.eu</name>
+ <url>scpexe://svjatoslav.eu:10006/srv/maven</url>
+ </snapshotRepository>
+ <repository>
+ <id>svjatoslav.eu</id>
+ <name>svjatoslav.eu</name>
+ <url>scpexe://svjatoslav.eu:10006/srv/maven</url>
+ </repository>
+ </distributionManagement>
+
+ <repositories>
+ <repository>
+ <id>svjatoslav.eu</id>
+ <name>Svjatoslav repository</name>
+ <url>https://www3.svjatoslav.eu/maven/</url>
+ </repository>
+ </repositories>
+
+ <scm>
+ <connection>scm:git:ssh://n0@svjatoslav.eu/home/git/repositories/alyverkko-cli.git</connection>
+ <developerConnection>scm:git:ssh://n0@svjatoslav.eu/home/git/repositories/alyverkko-cli.git
+ </developerConnection>
+ </scm>
+
+
+</project>
--- /dev/null
+package eu.svjatoslav.alyverkko_cli;
+
+import eu.svjatoslav.alyverkko_cli.model.Model;
+
+import java.io.*;
+import java.nio.file.Files;
+
+import static eu.svjatoslav.alyverkko_cli.Main.configuration;
+import static java.lang.String.join;
+
+public class AiTask {
+ public static final String AI_RESPONSE_MARKER = "ASSISTANT:";
+ private static final String LLAMA_CPP_META_INFO_MARKER = "llm_load_print_meta: ";
+
+ private final String aiQuery;
+ private final Model model;
+ private final Float temperature;
+ File inputFile;
+
+ /**
+ * Creates a new AI task.
+ *
+ * @param input Problem statement to be used for the AI task.
+ * @param model The model to be used for the AI task.
+ * @param temperature The temperature to be used for the AI inference process.
+ */
+ public AiTask(String input, Model model, Float temperature) {
+ this.aiQuery = buildAiQuery(input);
+ this.model = model;
+ this.temperature = temperature == null ? configuration.getDefaultTemperature() : temperature;
+ }
+
+ private String buildAiQuery(String input) {
+ StringBuilder sb = new StringBuilder();
+
+ sb.append("SYSTEM:\nThis conversation involves a user and AI assistant where the AI " +
+ "is expected to provide not only immediate responses but also detailed and " +
+ "well-reasoned analysis. The AI should consider all aspects of the query " +
+ "and deliver insights based on logical deductions and comprehensive understanding." +
+ "AI assistant should reply using emacs org-mode syntax.\n" +
+ "Quick recap: *this is bold* [[http://domain.org][This is link]]\n" +
+ "* Heading level 1\n" +
+ "** Heading level 2\n" +
+ "| Col 1 Row 1 | Col 2 Row 1 |\n" +
+ "| Col 1 Row 2 | Col 2 Row 2 |\n" +
+ "#+BEGIN_SRC python\n" +
+ " print ('Hello, world!')\n" +
+ "#+END_SRC\n\n");
+
+
+ String filteredInput = filterParticipantsInUserInput(input);
+
+ // if filtered input does not start with "USER:", add it
+ if (!filteredInput.startsWith("USER:")) {
+ filteredInput = "USER:\n" + filteredInput;
+ }
+
+ sb.append(filteredInput).append("\n").append(AI_RESPONSE_MARKER);
+ return sb.toString();
+ }
+
+ public static String filterParticipantsInUserInput(String input) {
+ StringBuilder result = new StringBuilder();
+ String[] lines = input.split("\n");
+ for (int i = 0; i < lines.length; i++) {
+ String line = lines[i];
+ if (i > 0) result.append("\n");
+ if ("* ASSISTANT:".equals(line)) line = "ASSISTANT:";
+ if ("* USER:".equals(line)) line = "USER:";
+ result.append(line);
+ }
+ return result.toString();
+ }
+
+ public static String filterParticipantsInAiResponse(String response) {
+ StringBuilder result = new StringBuilder();
+ String[] lines = response.split("\n");
+ for (int i = 0; i < lines.length; i++) {
+ String line = lines[i];
+ if (i > 0) result.append("\n");
+ if ("ASSISTANT:".equals(line)) line = "* ASSISTANT:";
+ if ("USER:".equals(line)) line = "* USER:";
+ result.append(line);
+ }
+ result.append("\n* USER:\n");
+ return result.toString();
+ }
+
+ /**
+ * Compute the AI task.
+ * @return The result of the AI task.
+ */
+ public String runAiQuery() throws InterruptedException, IOException {
+ try {
+ initializeInputFile();
+
+ ProcessBuilder processBuilder = new ProcessBuilder();
+ processBuilder.command(getCliCommand().split("\\s+")); // Splitting the command string into parts
+
+ Process process = processBuilder.start();
+ handleErrorThread(process);
+ StringBuilder result = new StringBuilder();
+ Thread outputThread = handleResultThread(process, result);
+ process.waitFor(); // Wait for the main AI computing process to finish
+ outputThread.join(); // Wait for the output thread to finish
+ return filterParticipantsInAiResponse(cleanupAiResponse(result.toString()));
+ } finally {
+ deleteTemporaryFile();
+ }
+ }
+
+ /**
+ * Initializes the input file for the AI task.
+ */
+ private void initializeInputFile() throws IOException {
+ // write AI input to file
+ inputFile = createTemporaryFile();
+ Files.write(inputFile.toPath(), aiQuery.getBytes());
+ }
+
+ /**
+ * Creates and starts a thread to handle the error stream of an AI inference process.
+ *
+ * @param process the process to read the error stream from.
+ */
+ private static void handleErrorThread(Process process) {
+ Thread errorThread = new Thread(() -> {
+ try (BufferedReader reader = new BufferedReader(new InputStreamReader(process.getErrorStream()))) {
+ String line;
+ while ((line = reader.readLine()) != null) handleErrorStreamLine(line);
+ } catch (IOException e) {
+ System.err.println("Error reading error stream: " + e.getMessage());
+ }
+ });
+ errorThread.start();
+ }
+
+
+ /**
+ * Handles a single line from the error stream of an AI inference process.
+ * If the line contains meta-info, it is printed to the console.
+ *
+ * @param line the line to be handled.
+ */
+ private static void handleErrorStreamLine(String line) {
+ if (line.startsWith(LLAMA_CPP_META_INFO_MARKER)) {
+ // Print the meta-info to console
+ System.out.println(line.substring(LLAMA_CPP_META_INFO_MARKER.length()));
+ return;
+ }
+
+ // Print the error to console
+ Utils.printRedMessageToConsole(line);
+ }
+
+ /**
+ * Gets the full command to be executed by the AI inference process.
+ *
+ * @return the full command to be executed by the AI inference process.
+ */
+ private String getCliCommand() {
+
+ return join(" ",
+ configuration.getLlamaCppExecutablePath().getAbsolutePath(),
+ "--model " + model.filesystemPath,
+ "--threads " + configuration.getThreadCount(),
+ "--threads-batch " + configuration.getBatchThreadCount(),
+ "--mirostat 2",
+ "--log-disable",
+ "--temp " + temperature,
+ "--ctx-size " + model.contextSizeTokens,
+ "--batch-size 8",
+ "-n -1",
+ "--repeat_penalty 1.1",
+ "--file " + inputFile);
+
+ }
+
+
+ /**
+ * Creates and starts a thread to handle the result of the AI inference process.
+ * The result is read from the process's input stream and saved in a StringBuilder.
+ *
+ * @param process the process to read the result from.
+ * @param result the StringBuilder to save the result in.
+ * @return the thread that handles the result.
+ */
+ private static Thread handleResultThread(Process process, StringBuilder result) {
+ Thread outputThread = new Thread(() -> {
+ try (BufferedReader reader = new BufferedReader(new InputStreamReader(process.getInputStream()))) {
+ String aiResultLine;
+ while ((aiResultLine = reader.readLine()) != null) {
+ System.out.print("AI: " + aiResultLine + "\n"); // Display each line as it's being read
+ result.append(aiResultLine).append("\n"); // Save the result
+ }
+ } catch (IOException e) {
+ throw new RuntimeException(e);
+ }
+ });
+ outputThread.start();
+ return outputThread;
+ }
+
+ /**
+ * Returns the temporary file for the AI to work with.
+ */
+ private File createTemporaryFile() throws IOException {
+ File file = Files.createTempFile("ai-inference", ".tmp").toFile();
+ file.deleteOnExit();
+ return file;
+ }
+
+ /**
+ * Cleans up the AI response by removing unnecessary text.
+ * @param result the AI response string to be cleaned up.
+ * @return the cleaned-up AI response.k
+ */
+ private String cleanupAiResponse(String result) {
+
+ // remove text before AI response marker
+ int aIResponseIndex = result.lastIndexOf(AI_RESPONSE_MARKER);
+ if (aIResponseIndex != -1) {
+ result = result.substring(aIResponseIndex + AI_RESPONSE_MARKER.length());
+ }
+
+ // remove text after end of text marker, if it exists
+ if (model.endOfTextMarker != null) {
+ int endOfTextMarkerIndex = result.indexOf(model.endOfTextMarker);
+ if (endOfTextMarkerIndex != -1) {
+ result = result.substring(0, endOfTextMarkerIndex);
+ }
+ }
+ return result + "\n";
+ }
+
+ private void deleteTemporaryFile() {
+ if (inputFile != null && inputFile.exists()) {
+ try {
+ Files.delete(inputFile.toPath());
+ } catch (IOException e) {
+ System.err.println("Failed to delete temporary file: " + e.getMessage());
+ }
+ }
+ }
+
+ public static String runAiQuery(String problemStatement, Model model, Float temperature) throws IOException, InterruptedException {
+ AiTask ai = new AiTask(problemStatement, model, temperature);
+ return ai.runAiQuery();
+ }
+
+}
--- /dev/null
+package eu.svjatoslav.alyverkko_cli;
+
+import java.io.IOException;
+
+public interface Command {
+
+ String getName();
+
+ void execute(String[] args) throws IOException, InterruptedException;
+}
--- /dev/null
+package eu.svjatoslav.alyverkko_cli;
+
+import eu.svjatoslav.alyverkko_cli.commands.ListModelsCommand;
+import eu.svjatoslav.alyverkko_cli.configuration.Configuration;
+import eu.svjatoslav.alyverkko_cli.commands.MailCorrespondentCommand;
+
+import java.io.IOException;
+import java.util.Optional;
+
+import static java.util.Arrays.copyOfRange;
+
+public class Main {
+
+ private final java.util.List<Command> commands = java.util.Arrays.asList(
+ new ListModelsCommand(),
+ new MailCorrespondentCommand()
+ );
+
+ public static void main(final String[] args) throws IOException, InterruptedException {
+ new Main().handleCommand(args);
+ }
+
+ public static Configuration configuration;
+
+ public void handleCommand(String[] args) throws IOException, InterruptedException {
+ if (args.length == 0) {
+ showHelp();
+ return;
+ }
+
+ String commandName = args[0].toLowerCase();
+ Optional<Command> commandOptional = commands.stream()
+ .filter(cmd -> cmd.getName().equals(commandName))
+ .findFirst();
+
+ if (!commandOptional.isPresent()) {
+ System.out.println("Unknown command: " + commandName);
+ showHelp();
+ return;
+ }
+
+ Command command = commandOptional.get();
+ String[] remainingArgs = copyOfRange(args, 1, args.length);
+ command.execute(remainingArgs);
+ }
+
+ private void showHelp() {
+ System.out.println("Älyverkko CLI\n");
+ System.out.println("Available commands:");
+ commands.forEach(cmd -> System.out.println(" " + cmd.getName()));
+ }
+
+}
\ No newline at end of file
--- /dev/null
+package eu.svjatoslav.alyverkko_cli;
+
+public class Utils {
+
+ public static void printRedMessageToConsole(String message) {
+ // set output color to red
+ System.out.print("\033[0;31m");
+ System.out.print(message + "\n");
+ // reset output color
+ System.out.print("\033[0m");
+ }
+
+}
--- /dev/null
+package eu.svjatoslav.alyverkko_cli.commands;
+
+import eu.svjatoslav.alyverkko_cli.Command;
+import eu.svjatoslav.alyverkko_cli.model.ModelLibrary;
+import eu.svjatoslav.commons.cli_helper.parameter_parser.Parser;
+import eu.svjatoslav.commons.cli_helper.parameter_parser.parameter.*;
+
+import java.io.IOException;
+
+import static eu.svjatoslav.alyverkko_cli.Main.configuration;
+import static eu.svjatoslav.alyverkko_cli.configuration.Configuration.loadConfiguration;
+
+public class ListModelsCommand implements Command {
+
+ @Override
+ public String getName() {
+ return "listmodels";
+ }
+
+ @Override
+ public void execute(String[] cliArguments) throws IOException {
+ configuration = loadConfiguration();
+ if (configuration == null){
+ System.out.println("Failed to load configuration file");
+ return;
+ }
+
+ System.out.println("Listing models in directory: " + configuration.getModelsDirectory());
+ ModelLibrary modelLibrary = new ModelLibrary(configuration.getModelsDirectory(), configuration.getModels());
+ modelLibrary.printModels();
+
+ }
+}
--- /dev/null
+package eu.svjatoslav.alyverkko_cli.commands;
+
+import eu.svjatoslav.alyverkko_cli.*;
+import eu.svjatoslav.alyverkko_cli.model.Model;
+import eu.svjatoslav.alyverkko_cli.model.ModelLibrary;
+import eu.svjatoslav.commons.cli_helper.parameter_parser.Parser;
+import eu.svjatoslav.commons.cli_helper.parameter_parser.parameter.DirectoryOption;
+import eu.svjatoslav.commons.cli_helper.parameter_parser.parameter.FileOption;
+
+import java.io.BufferedReader;
+import java.io.File;
+import java.io.FileReader;
+import java.io.IOException;
+import java.nio.file.*;
+
+import static eu.svjatoslav.alyverkko_cli.configuration.Configuration.loadConfiguration;
+import static eu.svjatoslav.alyverkko_cli.Main.configuration;
+import static eu.svjatoslav.commons.file.IOHelper.getFileContentsAsString;
+import static eu.svjatoslav.commons.file.IOHelper.saveToFile;
+import static java.nio.file.StandardWatchEventKinds.ENTRY_CREATE;
+import static java.nio.file.StandardWatchEventKinds.ENTRY_MODIFY;
+
+public class MailCorrespondentCommand implements Command {
+
+ final Parser parser = new Parser();
+
+ ModelLibrary modelLibrary;
+
+ private WatchService watcher;
+
+ /**
+ * The directory containing mail files.
+ */
+ DirectoryOption mailDirectoryOption = parser.add(new DirectoryOption("Directory containing mail files"))
+ .addAliases("--mail", "-m").mustExist();
+
+ File mailDirectory;
+
+ /**
+ * Configuration file location.
+ */
+ public FileOption configFileOption = parser.add(new FileOption("Configuration file path"))
+ .addAliases("--config", "-c").mustExist();
+
+ private void initialMailScanAndReply() throws IOException, InterruptedException {
+ File[] files = mailDirectory.listFiles();
+ if (files == null) return;
+
+ for (File file : files)
+ processMailIfNeeded(file);
+ }
+
+ private boolean isMailProcessingNeeded(File file) throws IOException {
+ // ignore hidden files
+ if (file.getName().startsWith("."))
+ return false;
+
+ // Check if the file is a mail file (not a directory
+ if (!file.isFile()) return false;
+
+ return fileHasToComputeMarker(file);
+ }
+
+ private static boolean fileHasToComputeMarker(File file) throws IOException {
+ try (BufferedReader reader = new BufferedReader(new FileReader(file))) {
+ String firstLine = reader.readLine();
+ return firstLine != null && firstLine.startsWith("TOCOMPUTE:");
+ }
+ }
+
+ private void processMailIfNeeded(File file) throws IOException, InterruptedException {
+ if (!isMailProcessingNeeded(file)) {
+ System.out.println("Ignoring file: " + file.getName() + " (does not need processing for now)");
+ return;
+ }
+
+ System.out.println("\nReplying to mail: " + file.getName());
+
+ // Read the mail contents, and remove the TOCOMPUTE: prefix from the first line
+ String mailContents = getFileContentsAsString(file);
+ mailContents = removeToComputePrefixFile(mailContents);
+
+ // faster model for testing for development time testing
+ // String modelAlias = "maid";
+ // TODO: make model CLI argument
+ String modelAlias = "wizard";
+
+ Model model = modelLibrary.findModelByAlias(modelAlias).get();
+ String aiGeneratedResponse = AiTask.runAiQuery(mailContents, model, null);
+
+ // Append the AI response to the mail contents
+ if (!mailContents.startsWith("* USER:\n")) {
+ mailContents = "* USER:\n" + mailContents;
+ }
+
+ String newMailContents = mailContents + "\n* ASSISTANT:\n" + aiGeneratedResponse;
+
+ // Write the result to the file
+ saveToFile(file, newMailContents);
+ }
+
+ private String removeToComputePrefixFile(String mailContents) {
+ // Remove the first line from the mail contents
+ int firstNewLineIndex = mailContents.indexOf('\n');
+ if (firstNewLineIndex != -1) {
+ mailContents = mailContents.substring(firstNewLineIndex + 1);
+ }
+ return mailContents;
+ }
+
+
+ @Override
+ public String getName() {
+ return "mail";
+ }
+
+ @Override
+ public void execute(String[] cliArguments) throws IOException, InterruptedException {
+ if (!parser.parse(cliArguments)) {
+ System.out.println("Failed to parse commandline arguments");
+ parser.showHelp();
+ return;
+ }
+
+ configuration = loadConfiguration(configFileOption.isPresent() ? configFileOption.getValue() : null);
+ if (configuration == null){
+ System.out.println("Failed to load configuration file");
+ return;
+ }
+
+ modelLibrary = new ModelLibrary(configuration.getModelsDirectory(), configuration.getModels());
+ mailDirectory = mailDirectoryOption.isPresent() ? mailDirectoryOption.getValue() : configuration.getMailDirectory();
+
+ initializeFileWatcher();
+
+ // before we start processing incremental changes in directory, we need to process all the existing files
+ initialMailScanAndReply();
+
+ System.out.println("Mail correspondent running. Press CTRL+c to terminate.");
+
+ while (true) {
+ WatchKey key;
+ try {
+ key = watcher.take();
+ } catch (InterruptedException e) {
+ System.out.println("Interrupted while waiting for file system events. Exiting.");
+ break;
+ }
+
+ System.out.println("Detected filesystem event.");
+
+ // sleep for a while to allow the file to be fully written
+ Thread.sleep(1000);
+
+ processDetectedFilesystemEvents(key);
+
+ if (!key.reset()) break;
+ }
+
+ watcher.close();
+ }
+
+ private void processDetectedFilesystemEvents(WatchKey key) throws IOException, InterruptedException {
+ for (WatchEvent<?> event : key.pollEvents()) {
+ WatchEvent.Kind<?> kind = event.kind();
+
+ // Skip OVERFLOW event
+ if (kind == StandardWatchEventKinds.OVERFLOW) continue;
+
+ // Retrieve the file name associated with the event
+ Path filename = ((WatchEvent<Path>) event).context();
+ System.out.println("Event: " + kind + " for file: " + filename);
+
+ // Process the event
+ processFileSystemEvent(kind, filename);
+ }
+ }
+
+ private void initializeFileWatcher() throws IOException {
+ this.watcher = FileSystems.getDefault().newWatchService();
+ Paths.get(mailDirectory.getAbsolutePath()).register(watcher, ENTRY_CREATE, ENTRY_MODIFY);
+ }
+
+ private void processFileSystemEvent(WatchEvent.Kind<?> kind, Path filename) throws IOException, InterruptedException {
+ if (kind != ENTRY_CREATE && kind != ENTRY_MODIFY) return;
+
+ File file = mailDirectory.toPath().resolve(filename).toFile();
+ processMailIfNeeded(file);
+ }
+
+
+}
--- /dev/null
+package eu.svjatoslav.alyverkko_cli.configuration;
+
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.dataformat.yaml.YAMLFactory;
+import com.fasterxml.jackson.annotation.JsonProperty;
+
+
+import java.io.*;
+import java.util.List;
+
+public class Configuration {
+
+ private static final String DEFAULT_CONFIG_FILE_PATH = "~/.config/alyverkko-cli.yaml".replaceFirst("^~", System.getProperty("user.home"));;
+
+ @JsonProperty("mail_directory")
+ private File mailDirectory;
+
+ @JsonProperty("models_directory")
+ private File modelsDirectory;
+
+ @JsonProperty("default_temperature")
+ private float defaultTemperature;
+
+ @JsonProperty("llama_cpp_executable_path")
+ private File llamaCppExecutablePath;
+
+ @JsonProperty("batch_thread_count")
+ private int batchThreadCount;
+
+ @JsonProperty("thread_count")
+ private int threadCount;
+
+ public List<ConfigurationModel> getModels() {
+ return models;
+ }
+
+ public void setModels(List<ConfigurationModel> models) {
+ this.models = models;
+ }
+
+ private List<ConfigurationModel> models;
+
+ public Configuration() {
+ }
+
+ public static Configuration loadConfiguration() throws IOException {
+ return loadConfiguration(null);
+ }
+
+ public static Configuration loadConfiguration(File configFile) throws IOException {
+
+ if (configFile == null) {
+ // Load configuration from the default path
+ configFile = new File(DEFAULT_CONFIG_FILE_PATH);
+ }
+
+ if (!configFile.exists()) {
+ System.err.println("Configuration file not found: " + configFile);
+ return null;
+ }
+
+ ObjectMapper mapper = new ObjectMapper(new YAMLFactory());
+ return mapper.readValue(configFile, Configuration.class);
+ }
+
+
+ public File getMailDirectory() {
+ return mailDirectory;
+ }
+
+ public void setMailDirectory(File mailDirectory) {
+ this.mailDirectory = mailDirectory;
+ }
+
+ public File getModelsDirectory() {
+ return modelsDirectory;
+ }
+
+ public void setModelsDirectory(File modelsDirectory) {
+ this.modelsDirectory = modelsDirectory;
+ }
+
+ public float getDefaultTemperature() {
+ return defaultTemperature;
+ }
+
+ public void setDefaultTemperature(float defaultTemperature) {
+ this.defaultTemperature = defaultTemperature;
+ }
+
+ public File getLlamaCppExecutablePath() {
+ return llamaCppExecutablePath;
+ }
+
+ public void setLlamaCppExecutablePath(File llamaCppExecutablePath) {
+ this.llamaCppExecutablePath = llamaCppExecutablePath;
+ }
+
+ public int getBatchThreadCount() {
+ return batchThreadCount;
+ }
+
+ public void setBatchThreadCount(int batchThreadCount) {
+ this.batchThreadCount = batchThreadCount;
+ }
+
+ public int getThreadCount() {
+ return threadCount;
+ }
+
+ public void setThreadCount(int threadCount) {
+ this.threadCount = threadCount;
+ }
+}
--- /dev/null
+package eu.svjatoslav.alyverkko_cli.configuration;
+
+import com.fasterxml.jackson.annotation.JsonProperty;
+
+public class ConfigurationModel {
+
+ private String alias;
+
+ /**
+ * Absolute path in the file system where the model is stored.
+ */
+ @JsonProperty("filesystem_path")
+ private String filesystemPath;
+
+ @JsonProperty("context_size_tokens")
+ private int contextSizeTokens;
+
+ @JsonProperty("end_of_text_marker")
+ private String endOfTextMarker;
+
+ public String getAlias() {
+ return alias;
+ }
+
+ public void setAlias(String alias) {
+ this.alias = alias;
+ }
+
+ public String getFilesystemPath() {
+ return filesystemPath;
+ }
+
+ public void setFilesystemPath(String filesystemPath) {
+ this.filesystemPath = filesystemPath;
+ }
+
+ public int getContextSizeTokens() {
+ return contextSizeTokens;
+ }
+
+ public void setContextSizeTokens(int contextSizeTokens) {
+ this.contextSizeTokens = contextSizeTokens;
+ }
+
+ public String getEndOfTextMarker() {
+ return endOfTextMarker;
+ }
+
+ public void setEndOfTextMarker(String endOfTextMarker) {
+ this.endOfTextMarker = endOfTextMarker;
+ }
+}
--- /dev/null
+package eu.svjatoslav.alyverkko_cli.model;
+
+import java.io.File;
+
+public class Model {
+ /**
+ * The path to the file system where the model will be stored.
+ */
+ public final File filesystemPath;
+
+ /**
+ * The size of the context in terms of tokens.
+ */
+ public final int contextSizeTokens;
+
+ /**
+ * The alias for the model.
+ */
+ public final String alias;
+
+ /**
+ * The marker used to signify the end of AI generated text.
+ */
+ public final String endOfTextMarker;
+
+
+ /**
+ * The constructor for the Model class.
+ * @param filesystemPath The path to the file system where the model will be stored.
+ * @param contextSizeTokens The size of the context in terms of tokens.
+ * @param modelAlias The alias for the model.
+ * @param endOfTextMarker The marker used to signify the end of AI generated text.
+ */
+ public Model(File filesystemPath, int contextSizeTokens, String modelAlias, String endOfTextMarker) {
+ this.filesystemPath = filesystemPath;
+ this.contextSizeTokens = contextSizeTokens;
+ this.alias = modelAlias;
+ this.endOfTextMarker = endOfTextMarker;
+ }
+
+ /**
+ * Prints the details of the model.
+ */
+ public void printModelDetails(){
+ System.out.println("Model: " + alias);
+ System.out.println(" Path: " + filesystemPath);
+ System.out.println(" Context size: " + contextSizeTokens);
+ }
+
+}
--- /dev/null
+package eu.svjatoslav.alyverkko_cli.model;
+
+
+import eu.svjatoslav.alyverkko_cli.Utils;
+import eu.svjatoslav.alyverkko_cli.configuration.ConfigurationModel;
+
+import java.io.File;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Optional;
+
+public class ModelLibrary {
+
+ /**
+ * List of all models available in the library.
+ */
+ private final List<Model> models;
+
+ private static Model defaultModel;
+ private final File modelsBaseDirectory;
+
+ /**
+ * Represents a library of AI models.
+ *
+ * @param modelsBaseDirectory the directory containing the models
+ */
+ public ModelLibrary(File modelsBaseDirectory, List<ConfigurationModel> configModels) {
+ this.modelsBaseDirectory = modelsBaseDirectory;
+ models = new ArrayList<>();
+
+
+ for (ConfigurationModel configModel : configModels) {
+ addModelFromConfig(configModel);
+ }
+
+ if (models.isEmpty())
+ throw new RuntimeException("No models are defined!");
+
+ defaultModel = models.get(0);
+ }
+
+ private void addModelFromConfig(ConfigurationModel configModel) {
+
+ // validate that model actually exists in the filesystem
+ File modelFile = new File(modelsBaseDirectory, configModel.getFilesystemPath());
+ if (!modelFile.exists()){
+ Utils.printRedMessageToConsole("WARN: Model file not found: " + modelFile.getAbsolutePath() + " . Skipping model.");
+ return;
+ }
+
+ addModel(new Model(
+ modelFile,
+ configModel.getContextSizeTokens(),
+ configModel.getAlias(),
+ configModel.getEndOfTextMarker()
+ ));
+ }
+
+
+ /**
+ * Adds a given model to the existing models list if it does not already exist.
+ *
+ * @param model the model to be added
+ * @throws RuntimeException if a model with the same alias already exists in the models list
+ */
+ public void addModel(Model model){
+ if (findModelByAlias(model.alias).isPresent())
+ throw new RuntimeException("Model with alias \"" + model.alias + "\" already exists!");
+
+ models.add(model);
+ }
+
+ /**
+ * Returns a list of all the models.
+ *
+ * @return a list of {@link Model} objects representing the models
+ */
+ public List<Model> getModels() {
+ return models;
+ }
+
+ /**
+ * Finds a model by its alias.
+ *
+ * @param alias the alias of the model to be found
+ * @return an {@link Optional} containing the model if it was found, or an empty {@link Optional} otherwise
+ */
+ public Optional<Model> findModelByAlias(String alias){
+ return models.stream()
+ .filter(model -> model.alias.equals(alias))
+ .findFirst();
+ }
+
+ public Model getDefaultModel() {
+ return defaultModel;
+ }
+
+ /**
+ * Prints the details of each model in the list.
+ */
+ public void printModels(){
+ System.out.println("Available models:\n");
+ for (Model model : models) {
+ model.printModelDetails();
+ System.out.println();
+ }
+ }
+}
--- /dev/null
+package eu.svjatoslav.alyverkko_cli;
+
+import org.junit.jupiter.api.Test;
+
+import static org.testng.AssertJUnit.assertEquals;
+
+public class AiTaskTest {
+
+ @Test
+ void testFilterUserInput() {
+ String input = "* ASSISTANT:\nHello\n* USER:\nHi!";
+ String expectedOutput = "ASSISTANT:\nHello\nUSER:\nHi!";
+ String output = AiTask.filterParticipantsInUserInput(input);
+ assertEquals(expectedOutput, output);
+ }
+
+ @Test
+ void testFilterUserInputAnyNumberOfSpacesAtTheEnd() {
+ String input = "* ASSISTANT: \nHello";
+ String expectedOutput = "ASSISTANT:\nHello";
+ String output = AiTask.filterParticipantsInUserInput(input);
+ assertEquals(expectedOutput, output);
+ }
+}
\ No newline at end of file
--- /dev/null
+#!/bin/bash
+
+# This script launches IntelliJ IDEA with the current project
+# directory. The script is designed to be run by double-clicking it in
+# the GNOME Nautilus file manager.
+
+# First, we change the current working directory to the directory of
+# the script.
+
+# "${0%/*}" gives us the path of the script itself, without the
+# script's filename.
+
+# This command basically tells the system "change the current
+# directory to the directory containing this script".
+
+cd "${0%/*}"
+
+# Then, we move up one directory level.
+# The ".." tells the system to go to the parent directory of the current directory.
+# This is done because we assume that the project directory is one level up from the script.
+cd ..
+
+# Now, we use the 'setsid' command to start a new session and run
+# IntelliJ IDEA in the background. 'setsid' is a UNIX command that
+# runs a program in a new session.
+
+# The command 'idea .' opens IntelliJ IDEA with the current directory
+# as the project directory. The '&' at the end is a UNIX command that
+# runs the process in the background. The '> /dev/null' part tells
+# the system to redirect all output (both stdout and stderr, denoted
+# by '&') that would normally go to the terminal to go to /dev/null
+# instead, which is a special file that discards all data written to
+# it.
+
+setsid idea . &>/dev/null &
+
+# The 'disown' command is a shell built-in that removes a shell job
+# from the shell's active list. Therefore, the shell will not send a
+# SIGHUP to this particular job when the shell session is terminated.
+
+# '-h' option specifies that if the shell receives a SIGHUP, it also
+# doesn't send a SIGHUP to the job.
+
+# '$!' is a shell special parameter that expands to the process ID of
+# the most recent background job.
+disown -h $!
+
+
+sleep 2
+
+# Finally, we use the 'exit' command to terminate the shell script.
+# This command tells the system to close the terminal window after
+# IntelliJ IDEA has been opened.
+exit
--- /dev/null
+#!/bin/bash
+cd "${0%/*}"; if [ "$1" != "T" ]; then gnome-terminal -e "'$0' T"; exit; fi;
+
+cd ..
+
+# Build the project jar file and the apidocs.
+mvn clean package
+
+# Export org to html using emacs in batch mode
+(
+ cd doc/
+ rm -f index.html
+ emacs --batch -l ~/.emacs --visit=index.org --funcall=org-html-export-to-html --kill
+)
+
+# Generate class diagrams. See: https://www3.svjatoslav.eu/projects/javainspect/
+rm -rf doc/graphs/
+mkdir -p doc/graphs/
+javainspect -j target/alyverkko-cli-*-SNAPSHOT.jar -d doc/graphs/ -n "all classes" -t png -ho
+meviz index -w doc/graphs/ -t "Älyverkko CLI program classes"
+
+# Copy the apidocs to the doc folder so that they can be uploaded to the server.
+rm -rf doc/apidocs/
+cp -r target/apidocs/ doc/
+
+# Upload project homepage to the server.
+rsync -avz --delete -e 'ssh -p 10006' doc/ n0@www3.svjatoslav.eu:/mnt/big/projects/alyverkko-cli/
+
+echo ""
+echo "Press ENTER to close this window."
+read
\ No newline at end of file
projects / alyverkko-cli.git / commitdiff
