Change load mode to auto, benchmarks from RPi5.

4 files changed, 311 insertions, 2 deletions

diff --git a/bench/results_qwen2.5-3b-instruct-q4_k_m.gguf.pi5.mlock.txt b/bench/results_qwen2.5-3b-instruct-q4_k_m.gguf.pi5.mlock.txt
new file mode 100644
index 0000000..e316eea
--- /dev/null
+++ b/bench/results_qwen2.5-3b-instruct-q4_k_m.gguf.pi5.mlock.txt
@@ -0,0 +1,102 @@
+PROMPT: Define imperious in 'an imperious necessity drove him forward on his quest'
+
+imperious (adj) - Formal
+
+DEFINITION:
+Having a commanding or domineering attitude; asserting one's will over others.
+
+GENERAL TONE:
+The necessity was so strong that it commanded him to move forward on his journey.
+SYNONYMS:
+urgent, pressing, compelling, urgent.
+
+EXAMPLES:
+1. The imperious necessity pushed him towards the goal.
+2. His imperious need compelled him to continue his research.
+
+METRICS: ELAPSED: 193.49s | DAEMON RSS: 1.32 MB
+
+------------------------------------------------------------------------
+
+PROMPT: Define despondency in 'he fell into a state of deep, uninterrupted despondency'
+
+despondency (noun) - Formal
+
+DEFINITION:
+deep and prolonged state of depression or hopelessness.
+
+GENERAL TONE:
+The tone is somber, reflecting the heavy emotional state described.
+
+SYNONYMS:
+dejection, melancholy, gloominess
+
+EXAMPLES:
+1. He fell into a state of deep, uninterrupted despondency.
+2. After the loss of his job, he sank into a long period of despondency.
+
+METRICS: ELAPSED: 37.90s | DAEMON RSS: 2626.21 MB
+
+------------------------------------------------------------------------
+
+PROMPT: Define arduous in 'the long and arduous journey through the frozen mountains'
+
+arduous (adjective) - Formal
+
+DEFINITION:
+Challenging or difficult to accomplish, especially over a long period of time.
+
+GENERAL TONE:
+The journey through the frozen mountains was tough and took much effort.
+
+SYNONYMS:
+Difficult, strenuous, laborious, taxing
+
+EXAMPLES:
+1. The expedition through the dense jungle was an arduous task.
+2. Climbing Mount Everest is an arduous challenge for mountaineers.
+
+METRICS: ELAPSED: 37.48s | DAEMON RSS: 2626.79 MB
+
+------------------------------------------------------------------------
+
+PROMPT: Define ephemeral in 'human history appeared as a brief and ephemeral flash in cosmic time'
+
+ephemeral (adj) - Formal
+
+DEFINITION:
+lasting only a very short time; fleeting.
+
+GENERAL TONE:
+The phrase suggests that human history is insignificant compared to the vastness of cosmic time.
+
+SYNONYMS:
+transient, fleeting, momentary.
+
+EXAMPLES:
+1. The sunset was an ephemeral display in the sky.
+2. His fame is just an ephemeral glow on the internet.
+
+METRICS: ELAPSED: 35.36s | DAEMON RSS: 2627.35 MB
+
+------------------------------------------------------------------------
+
+PROMPT: Define austere in 'they lived under the austere discipline of their new order'
+
+austere (adj) - Formal
+
+DEFINITION:
+strictly simple or plain in style, manner, or appearance; severe and unadorned.
+
+GENERAL TONE:
+The tone is formal, emphasizing the strictness and simplicity of their lifestyle under a new order.
+
+SYNONYMS:
+rigid, severe, plain, stark
+
+EXAMPLES:
+1. The monastery was built in austere simplicity.
+2. His speech was delivered with an austere dignity.
+
+METRICS: ELAPSED: 38.72s | DAEMON RSS: 2627.89 MB
+
diff --git a/bench/results_qwen2.5-3b-instruct-q4_k_m.gguf.pi5.mmap.txt b/bench/results_qwen2.5-3b-instruct-q4_k_m.gguf.pi5.mmap.txt
new file mode 100644
index 0000000..03ff19b
--- /dev/null
+++ b/bench/results_qwen2.5-3b-instruct-q4_k_m.gguf.pi5.mmap.txt
@@ -0,0 +1,102 @@
+PROMPT: Define imperious in 'an imperious necessity drove him forward on his quest'
+
+imperious (adj) - Formal
+
+DEFINITION:
+Having a commanding or domineering attitude; asserting one's will over others.
+
+GENERAL TONE:
+The necessity was so strong that it commanded him to move forward on his journey.
+SYNONYMS:
+urgent, pressing, compelling, urgent.
+
+EXAMPLES:
+1. The imperious necessity pushed him towards the goal.
+2. His imperious need compelled him to continue his research.
+
+METRICS: ELAPSED: 172.11s | DAEMON RSS: 1.30 MB
+
+------------------------------------------------------------------------
+
+PROMPT: Define despondency in 'he fell into a state of deep, uninterrupted despondency'
+
+despondency (noun) - Formal
+
+DEFINITION:
+deep and prolonged state of depression or hopelessness.
+
+GENERAL TONE:
+The tone is somber, reflecting the heavy emotional state described.
+
+SYNONYMS:
+dejection, melancholy, gloominess
+
+EXAMPLES:
+1. He fell into a state of deep, uninterrupted despondency.
+2. After the loss of his job, he sank into a long period of despondency.
+
+METRICS: ELAPSED: 37.49s | DAEMON RSS: 2868.65 MB
+
+------------------------------------------------------------------------
+
+PROMPT: Define arduous in 'the long and arduous journey through the frozen mountains'
+
+arduous (adjective) - Formal
+
+DEFINITION:
+Challenging or difficult to accomplish, especially over a long period of time.
+
+GENERAL TONE:
+The journey through the frozen mountains was tough and took much effort.
+
+SYNONYMS:
+Difficult, strenuous, laborious, taxing
+
+EXAMPLES:
+1. The expedition through the dense jungle was an arduous task.
+2. Climbing Mount Everest is an arduous challenge for mountaineers.
+
+METRICS: ELAPSED: 40.15s | DAEMON RSS: 2869.39 MB
+
+------------------------------------------------------------------------
+
+PROMPT: Define ephemeral in 'human history appeared as a brief and ephemeral flash in cosmic time'
+
+ephemeral (adj) - Formal
+
+DEFINITION:
+lasting only a very short time; fleeting.
+
+GENERAL TONE:
+The phrase suggests that human history is insignificant compared to the vastness of cosmic time.
+
+SYNONYMS:
+transient, fleeting, momentary.
+
+EXAMPLES:
+1. The sunset was an ephemeral display in the sky.
+2. His fame is just an ephemeral glow on the internet.
+
+METRICS: ELAPSED: 37.26s | DAEMON RSS: 2870.13 MB
+
+------------------------------------------------------------------------
+
+PROMPT: Define austere in 'they lived under the austere discipline of their new order'
+
+austere (adj) - Formal
+
+DEFINITION:
+strictly simple or plain in style, manner, or appearance; severe and unadorned.
+
+GENERAL TONE:
+The tone is formal, emphasizing the strictness and simplicity of their lifestyle under a new order.
+
+SYNONYMS:
+rigid, severe, plain, stark
+
+EXAMPLES:
+1. The monastery was built in austere simplicity.
+2. His speech was delivered with an austere dignity.
+
+METRICS: ELAPSED: 37.48s | DAEMON RSS: 2870.81 MB
+
diff --git a/bench/results_qwen2.5-3b-instruct-q4_k_m.gguf.pi5.none.txt b/bench/results_qwen2.5-3b-instruct-q4_k_m.gguf.pi5.none.txt
new file mode 100644
index 0000000..e51b40b
--- /dev/null
+++ b/bench/results_qwen2.5-3b-instruct-q4_k_m.gguf.pi5.none.txt
@@ -0,0 +1,102 @@
+PROMPT: Define imperious in 'an imperious necessity drove him forward on his quest'
+
+imperious (adj) - Formal
+
+DEFINITION:
+Having a commanding or domineering attitude; asserting one's will over others.
+
+GENERAL TONE:
+The necessity was so strong that it commanded him to move forward on his journey.
+SYNONYMS:
+urgent, pressing, compelling, urgent.
+
+EXAMPLES:
+1. The imperious necessity pushed him towards the goal.
+2. His imperious need compelled him to continue his research.
+
+METRICS: ELAPSED: 178.32s | DAEMON RSS: 1.30 MB
+
+------------------------------------------------------------------------
+
+PROMPT: Define despondency in 'he fell into a state of deep, uninterrupted despondency'
+
+despondency (noun) - Formal
+
+DEFINITION:
+deep and prolonged state of depression or hopelessness.
+
+GENERAL TONE:
+The tone is somber, reflecting the heavy emotional state described.
+
+SYNONYMS:
+dejection, melancholy, gloominess
+
+EXAMPLES:
+1. He fell into a state of deep, uninterrupted despondency.
+2. After the loss of his job, he sank into a long period of despondency.
+
+METRICS: ELAPSED: 35.89s | DAEMON RSS: 1928.74 MB
+
+------------------------------------------------------------------------
+
+PROMPT: Define arduous in 'the long and arduous journey through the frozen mountains'
+
+arduous (adjective) - Formal
+
+DEFINITION:
+Challenging or difficult to accomplish, especially over a long period of time.
+
+GENERAL TONE:
+The journey through the frozen mountains was tough and took much effort.
+
+SYNONYMS:
+Difficult, strenuous, laborious, taxing
+
+EXAMPLES:
+1. The expedition through the dense jungle was an arduous task.
+2. Climbing Mount Everest is an arduous challenge for mountaineers.
+
+METRICS: ELAPSED: 37.04s | DAEMON RSS: 1930.48 MB
+
+------------------------------------------------------------------------
+
+PROMPT: Define ephemeral in 'human history appeared as a brief and ephemeral flash in cosmic time'
+
+ephemeral (adj) - Formal
+
+DEFINITION:
+lasting only a very short time; fleeting.
+
+GENERAL TONE:
+The phrase suggests that human history is insignificant compared to the vastness of cosmic time.
+
+SYNONYMS:
+transient, fleeting, momentary.
+
+EXAMPLES:
+1. The sunset was an ephemeral display in the sky.
+2. His fame is just an ephemeral glow on the internet.
+
+METRICS: ELAPSED: 33.40s | DAEMON RSS: 1932.18 MB
+
+------------------------------------------------------------------------
+
+PROMPT: Define austere in 'they lived under the austere discipline of their new order'
+
+austere (adj) - Formal
+
+DEFINITION:
+strictly simple or plain in style, manner, or appearance; severe and unadorned.
+
+GENERAL TONE:
+The tone is formal, emphasizing the strictness and simplicity of their lifestyle under a new order.
+
+SYNONYMS:
+rigid, severe, plain, stark
+
+EXAMPLES:
+1. The monastery was built in austere simplicity.
+2. His speech was delivered with an austere dignity.
+
+METRICS: ELAPSED: 36.38s | DAEMON RSS: 1933.52 MB
+
diff --git a/llm.c b/llm.c
index 23264db..4d3dc58 100644
--- a/llm.c
+++ b/llm.c
@@ -91,8 +91,11 @@ struct llm_ctx *llm_init(const char *model_path)
 
 	mparams = llama_model_default_params();
 	mparams.n_gpu_layers = 0;   /* force all layers onto CPU */
-	mparams.load_mode = LLAMA_LOAD_MODE_MLOCK;  /* Force model to RAM, 
-	                                             * don't swap/compress */ 
+
+	/* LLAMA_LOAD_MODE_MLOCK (force model to RAM, no swap/compression)
+	 * would have been preferrable. But my Raspberry Pi5 doesn't have
+	 * enough memory. */
+	mparams.load_mode = LLAMA_LOAD_MODE_NONE;
 
 	llm = MALLOC(sizeof(*llm));
 	llm->model = llama_model_load_from_file(model_path, mparams);