**** Generation Parameters
-- =default_temperature=: Creativity control (0-3, higher = more
- creative).
+- =default_temperature=: (Optional) Creativity control (0-3, higher =
+ more creative).
-- =default_top_p=: Nucleus sampling threshold (0.0-1., higher = more
- diverse).
+- =default_top_p=: (Optional) Nucleus sampling threshold (0.0-1.,
+ higher = more diverse).
-- =default_top_k=: Restricts token selection to the K tokens with the
- highest probabilities, regardless of their actual probability values
- or the shape of the distribution.
+- =default_top_k=: (Optional) Restricts token selection to the K
+ tokens with the highest probabilities, regardless of their actual
+ probability values or the shape of the distribution.
-- =default_min_p=: Filters the vocabulary to include only tokens whose
- probability is at least a certain fraction (Min P) of the
- probability of the most likely token.
+- =default_min_p=: (Optional) Filters the vocabulary to include only
+ tokens whose probability is at least a certain fraction (Min P) of
+ the probability of the most likely token.
-- =default_repeat_penalty=: Penalty for repetition (>0.0, 1.0 = no
- penalty)
+- =default_repeat_penalty=: (Optional) Penalty for repetition (>0.0,
+ 1.0 = no penalty)
**** Performance Tuning
- =alias=: Short model alias. Model with alias "default" would be used
by default.
+- =temperature=: (Optional) (TODO: document)
+
+- =top_p=: (Optional) (TODO: document)
+
+- =min_p=: (Optional) (TODO: document)
+
+- =top_k=: (Optional) (TODO: document)
+
+- =repeat_penalty=: (Optional) (TODO: document)
+
- =filesystem_path=: File name of the model as located within
*models_directory*
- =context_size_tokens=: Context size in tokens that model was trained
on.
-- =end_of_text_marker=: Some models produce certain markers to
- indicate end of their output. If specified here, Älyverkko CLI can
- identify and remove them so that they don't leak into
+- =end_of_text_marker=: (Optional) Some models produce certain markers
+ to indicate end of their output. If specified here, Älyverkko CLI
+ can identify and remove them so that they don't leak into
conversation. Default value is: *null*.
-- =temperature=, =top_p=, =repeat_penalty=: Model-specific overrides
*** Configuration file example
skills_directory: "/home/user/AI/skills"
llama_cli_path: "/home/user/AI/llama.cpp/build/bin/llama-cli"
- # Processing parameters
+ # Generation parameters
default_temperature: 0.7
default_top_p: 0.9
default_repeat_penalty: 1.0
+
+ # Performance tuning
thread_count: 6
batch_thread_count: 10
- alias: "default"
filesystem_path: "model.gguf"
context_size_tokens: 64000
- end_of_text_marker: null
- temperature: 0.8 # Optional model-specific parameter
- top_p: 0.9 # Optional
- repeat_penalty: 1.1 # Optional
+ temperature: 0.8
+ top_p: 0.9
+ repeat_penalty: 1.1
+
- alias: "mistral"
filesystem_path: "Mistral-Large-Instruct-2407.Q8_0.gguf"
context_size_tokens: 32768
- end_of_text_marker: null
#+end_src
*** Parameter Precedence Hierarchy
-For *temperature*, *top_p*, and *repeat_penalty* parameters, values
-are determined using this priority order (highest to lowest):
+For *temperature*, *top_p*, *top_k*, *min_p* and *repeat_penalty*
+parameters, values are determined using this priority order (highest
+to lowest):
1. *Skill-specific value* (from skill's YAML file)
2. *Model-specific value* (from model configuration)