Add swap occupancy to test script.

2 files changed, 41 insertions, 292 deletions

diff --git a/results_qwen2.5-3b-instruct-q4_k_m.gguf.txt b/results_qwen2.5-3b-instruct-q4_k_m.gguf.txt
deleted file mode 100644
index 45773b6..0000000
--- a/results_qwen2.5-3b-instruct-q4_k_m.gguf.txt
+++ /dev/null
@@ -1,269 +0,0 @@
-PROMPT: Define imperious in 'an imperious necessity drove him forward on his quest'
-
-imperious (adj) - Formal
-
-DEFINITION:
-Having a commanding or domineering character; asserting one's will with authority and arrogance.
-
-GENERAL TONE:
-The necessity was so strong that it pushed him forward, leaving no room for resistance.
-SYNONYMS:
-urgent, pressing, compelling, urgent.
-
-EXAMPLES:
-1. The imperious necessity compelled him to finish the project on time.
-2. His imperious attitude made it hard for others to disagree with him.
-
-METRICS: ELAPSED: 19.96s | DAEMON RSS: 3.07 MB
-
-------------------------------------------------------------------------
-
-PROMPT: Define despondency in 'he fell into a state of deep, uninterrupted despondency'
-
-despondency (noun) - Formal
-
-DEFINITION:
-a state of deep and persistent depression or hopelessness.
-
-GENERAL TONE:
-The tone is somber, reflecting the heavy emotional state described.
-
-SYNONYMS:
-melancholy, despair, gloom
-
-EXAMPLES:
-1. He fell into a state of deep, uninterrupted despondency.
-2. Her friends noticed her mood had become increasingly despondent over the past few weeks.
-
-METRICS: ELAPSED: 13.46s | DAEMON RSS: 3388.88 MB
-
-------------------------------------------------------------------------
-
-PROMPT: Define arduous in 'the long and arduous journey through the frozen mountains'
-
-arduous (adjective) - Formality: Literary
-
-DEFINITION:
-Challenging or difficult, especially in a way that requires great effort and perseverance.
-
-GENERAL TONE:
-The journey through the frozen mountains was fraught with difficulties requiring immense determination to overcome.
-
-SYNONYMS:
-Challenging, difficult, strenuous, taxing
-
-EXAMPLES:
-1. The expedition through the dense jungle was an arduous journey.
-2. Climbing Mount Everest is an arduous undertaking for even the most experienced climbers.
-
-METRICS: ELAPSED: 15.35s | DAEMON RSS: 3388.91 MB
-
-------------------------------------------------------------------------
-
-PROMPT: Define ephemeral in 'human history appeared as a brief and ephemeral flash in cosmic time'
-
-ephemeral (adj) - Formal
-
-DEFINITION:
-lasting only a very short time; brief.
-
-GENERAL TONE:
-The concept of human history being fleeting in the vastness of cosmic time is often used to emphasize how insignificant our existence can seem from a cosmic perspective.
-
-SYNONYMS:
-transient, fleeting, momentary.
-
-EXAMPLES:
-1. The success of the project was ephemeral; it didn't last long.
-2. In the grand scheme of things, life is an ephemeral experience.
-
-METRICS: ELAPSED: 15.77s | DAEMON RSS: 3388.91 MB
-
-------------------------------------------------------------------------
-
-PROMPT: Define austere in 'they lived under the austere discipline of their new order'
-
-austere (adjective) - Formal
-
-DEFINITION:
-strict and plain in behavior or lifestyle; austere rules, regulations.
-
-GENERAL TONE:
-The tone is formal, emphasizing the strictness of the discipline imposed by the new order.
-
-SYNONYMS:
-rigid, severe, austonian
-
-EXAMPLES:
-1. The monastery lived under austere rules.
-2. His lifestyle was austere and simple.
-
-METRICS: ELAPSED: 12.76s | DAEMON RSS: 3388.91 MB
-
-------------------------------------------------------------------------
-
-PROMPT: Define venerable in 'the venerable monuments of a long-forgotten civilization'
-
-venerable (adjective) - Formal
-
-DEFINITION:
-Respected and held in high esteem due to age or historical significance.
-
-GENERAL TONE:
-The monuments are ancient structures that have stood the test of time, earning them respect from modern visitors.
-
-SYNONYMS:
-Ancient, revered, esteemed, venerable
-
-EXAMPLES:
-1. The venerable monuments of a long-forgotten civilization are now preserved as historical sites.
-2. She felt a sense of reverence for the venerable old church in her hometown.
-
-METRICS: ELAPSED: 15.89s | DAEMON RSS: 3388.91 MB
-
-------------------------------------------------------------------------
-
-PROMPT: Define tumultuous in 'the species endured a tumultuous period of war and renewal'
-
-tumultuous (adj) - Formality: Literary
-
-DEFINITION:
-Marked by violent disturbance or disorder; stormy.
-
-GENERAL TONE:
-The passage conveys a sense of chaos and upheaval during the period described.
-
-SYNONYMS:
-Stormy, turbulent, chaotic, stormlike.
-
-EXAMPLES:
-1. The tumultuous weather led to widespread power outages.
-2. His life was marked by a tumultuous period of political upheaval.
-
-METRICS: ELAPSED: 14.44s | DAEMON RSS: 3388.91 MB
-
-------------------------------------------------------------------------
-
-PROMPT: Define imperturbable in 'he maintained an imperturbable calm despite the surrounding chaos'
-
-imperturbable (adjective) - Formal
-
-DEFINITION:
-Unflappable; able to remain calm and composed under stress or in difficult circumstances.
-
-GENERAL TONE:
-The tone is formal, emphasizing the ability of someone to maintain composure even when faced with chaos.
-
-SYNONYMS:
-Unflappable, composed, unruffled, steady
-
-EXAMPLES:
-1. Despite the stormy weather, he maintained an imperturbable calm.
-2. The actor's performance was so imperturbable that it seemed scripted.
-
-METRICS: ELAPSED: 16.26s | DAEMON RSS: 3388.91 MB
-
-------------------------------------------------------------------------
-
-PROMPT: Define singular in 'he was a man of a singular and unpredictable temperament'
-
-singular (adjective) - Formal
-
-DEFINITION:
-single or unique in nature, not typical.
-
-GENERAL TONE:
-The term suggests a person with an unusual and unpredictable personality.
-
-SYNONYMS:
-unusual, eccentric, peculiar, strange.
-
-EXAMPLES:
-1. He was a man of a singular and unpredictable temperament.
-2. Her approach to problem-solving is singularly innovative.
-
-METRICS: ELAPSED: 11.76s | DAEMON RSS: 3388.91 MB
-
-------------------------------------------------------------------------
-
-PROMPT: Define enigmatic in 'he left behind an enigmatic smile and no further explanation'
-
-enigmatic (adjective) - Formal
-
-DEFINITION:
-Mysterious and difficult to understand or interpret.
-
-GENERAL TONE:
-The smile was a puzzle, its meaning unclear.
-SYNONYMS:
-mystifying, puzzling, enigmatical
-
-EXAMPLES:
-1. The riddle left me feeling enigmatic.
-2. His smile was so enigmatic it made everyone curious.
-
-METRICS: ELAPSED: 11.82s | DAEMON RSS: 3388.91 MB
-
-------------------------------------------------------------------------
-
-PROMPT: Define bestial in 'You submitted to be the slave of bestial men'
-
-bestial (adj) - Formal
-
-DEFINITION:
-Of or relating to animals; brutish.
-
-GENERAL TONE:
-The term suggests a brutal and uncivilized state of being, often associated with violence and degradation.
-
-SYNONYMS:
-brutal, savage, uncivilized
-
-EXAMPLES:
-1. You submitted to be the slave of bestial men.
-2. The treatment of animals by these bestial owners was inhumane.
-
-METRICS: ELAPSED: 14.01s | DAEMON RSS: 3388.91 MB
-
-------------------------------------------------------------------------
-
-PROMPT: Define flung in 'As if plunged in a stormy water, I was flung about wildly, and felt myself sinking.'
-
-flung (verb) - Formal
-
-DEFINITION:
-To throw or hurl with force.
-
-GENERAL TONE:
-The verb conveys a sense of sudden and violent action, often used in dramatic contexts.
-
-SYNONYMS:
-Threw, tossed, hurled
-
-EXAMPLES:
-1. As if plunged in a stormy water, I was flung about wildly, and felt myself sinking.
-2. The ball was flung across the field by the pitcher.
-
-METRICS: ELAPSED: 14.77s | DAEMON RSS: 3388.91 MB
-
-------------------------------------------------------------------------
-
-PROMPT: Define hewers in 'You might by this time have made the Bags hewers of wood and drawers of water'
-
-hewers (noun) - Formal
-
-DEFINITION:
-In the context of colonial America, hewers referred to those who cut wood.
-
-GENERAL TONE:
-The phrase suggests a laborious and manual work environment typical in early American settlements.
-
-SYNONYMS:
-Cutters, Sawyers, Woodcutters
-
-EXAMPLES:
-1. The hewers of wood were the backbone of the community.
-2. By the time you mentioned, they had already become hewers and drawers of water.
-
-METRICS: ELAPSED: 15.79s | DAEMON RSS: 3388.91 MB
-
diff --git a/test_prompts.sh b/test_prompts.sh
index 61fb69f..0424e62 100755
--- a/test_prompts.sh
+++ b/test_prompts.sh
@@ -19,26 +19,39 @@ if [ ! -f "$MODEL_PATH" ]; then
 fi
 
 # Derive base model name and output result filename
-# e.g., models/qwen2.5-7b-q4.gguf -> results_qwen2.5-7b-q4.gguf.txt
 MODEL_NAME=$(basename "$MODEL_PATH")
 RESULT_FILE="results_${MODEL_NAME}.txt"
 
 # Default prompt filename (optional 2nd argument)
 PROMPT_FILE="${2:-test_prompts.txt}"
 
-# Ensure the executable exists and is runnable
+# Ensure executable exists
 if [ ! -x "$EXECUTABLE" ]; then
     echo "Error: Executable '$EXECUTABLE' not found or not executable." >&2
     exit 1
 fi
 
-# Ensure the prompt file exists
+# Ensure prompt file exists
 if [ ! -f "$PROMPT_FILE" ]; then
     echo "Error: Prompt file '$PROMPT_FILE' not found." >&2
     exit 1
 fi
 
-# Count valid prompts using POSIX awk to avoid ksh $(...) parenthetical syntax errors
+# Helper function to get current swap occupancy in MB by parsing swapctl -l
+# OpenBSD swapctl -l outputs 512-byte block counts by default
+get_swap_occupancy_mb() {
+    swapctl -l | awk '
+        NR > 1 {
+            # Column 3 is Used blocks (512-byte blocks)
+            used_blocks += $3
+        }
+        END {
+            # Convert 512-byte blocks to MB (blocks * 512 / 1024 / 1024)
+            printf "%.2f", used_blocks * 512 / 1048576
+        }'
+}
+
+# Count valid prompts using POSIX awk
 total_prompts=$(awk '/^[[:space:]]*($|#)/ {next} {count++} END {print count+0}' "$PROMPT_FILE")
 
 if [ "$total_prompts" -eq 0 ]; then
@@ -46,10 +59,10 @@ if [ "$total_prompts" -eq 0 ]; then
     exit 0
 fi
 
-# Create (or truncate) the result file at the start of the run
+# Create (or truncate) result file
 > "$RESULT_FILE"
 
-# Create horizontal rule.
+# Create horizontal rule
 RULE=""
 i=0
 while [ $i -lt 72 ]; do
@@ -57,7 +70,7 @@ while [ $i -lt 72 ]; do
     i=$((i + 1))
 done
 
-# --- Start the daemon once, wait for it to come up, capture its baseline RSS ---
+# --- Start daemon ---
 
 echo "Starting daemon with model: '$MODEL_PATH'"
 "$EXECUTABLE" -m "$MODEL_PATH" &
@@ -70,9 +83,6 @@ cleanup() {
 }
 trap cleanup EXIT INT TERM
 
-# Wait for the socket to appear (covers model load time) and for the
-# process to still be alive (covers early startup failure, e.g. bad
-# model path -> daemon exits immediately).
 waited=0
 while [ ! -S "$SOCK_PATH" ]; do
     if ! kill -0 "$DAEMON_PID" 2>/dev/null; then
@@ -98,44 +108,51 @@ first=1
 # Temporary file to capture time / rusage output
 TIME_LOG=$(mktemp /tmp/lex_time.XXXXXX)
 
-# Process each prompt line by line
+# Process each prompt
 while IFS= read -r prompt || [ -n "$prompt" ]; do
-    # Skip empty lines or comment lines starting with #
     case "$prompt" in
         ""|\#*) continue ;;
     esac
 
     current=$((current + 1))
 
-    # Add delimiter before every prompt except the first in the results file
     if [ "$first" -eq 1 ]; then
         first=0
     else
         printf "%s\n\n" "$RULE" >> "$RESULT_FILE"
     fi
 
-    # Log prompt header to the result file
     printf "PROMPT: %s\n\n" "$prompt" >> "$RESULT_FILE"
 
-    # Sample the daemon's RSS right before firing the request, so we have
-    # a rough steady-state figure (won't catch a mid-request peak, but the
-    # model's footprint is dominated by the loaded weights, not per-request
-    # allocation, so this is a reasonable proxy).
+    # Sample daemon RSS prior to request
     daemon_rss_kb=$(ps -o rss= -p "$DAEMON_PID" 2>/dev/null | tr -d ' ')
 
-    # Send the prompt to the daemon over the socket, timing the round trip.
-    # This now measures request latency only (no model-load cost), which is
-    # the number that actually matters day to day with the daemon design.
+    # Capture baseline swap occupancy before sending the prompt
+    swap_start_mb=$(get_swap_occupancy_mb)
+
+    # Send prompt to daemon over socket
     /usr/bin/time -lp sh -c "printf '%s\n' \"\$1\" | nc -U \"\$2\"" _ "$prompt" "$SOCK_PATH" \
         >> "$RESULT_FILE" 2> "$TIME_LOG"
 
+    # Capture ending swap occupancy after prompt completes
+    swap_end_mb=$(get_swap_occupancy_mb)
+
     real_time=$(awk '/^real/ {print $2}' "$TIME_LOG")
 
+    # Calculate net change in swap occupancy during request execution
+    if [ -n "$swap_start_mb" ] && [ -n "$swap_end_mb" ]; then
+        swap_delta_mb=$(awk -v start="$swap_start_mb" -v end="$swap_end_mb" \
+            'BEGIN { printf "%+.2f", end - start }')
+    else
+        swap_end_mb="unavailable"
+        swap_delta_mb="+0.00"
+    fi
+
     if [ -n "$daemon_rss_kb" ]; then
         daemon_rss_mb=$(awk -v kb="$daemon_rss_kb" 'BEGIN{printf "%.2f", kb/1024}')
-        metrics_summary="ELAPSED: ${real_time}s | DAEMON RSS: ${daemon_rss_mb} MB"
+        metrics_summary="ELAPSED: ${real_time}s | DAEMON RSS: ${daemon_rss_mb} MB | SWAP: ${swap_end_mb} MB (${swap_delta_mb} MB)"
     else
-        metrics_summary="ELAPSED: ${real_time}s | DAEMON RSS: unavailable"
+        metrics_summary="ELAPSED: ${real_time}s | DAEMON RSS: unavailable | SWAP OCCUPIED: ${swap_end_mb} MB (${swap_delta_mb} MB)"
     fi
 
     printf "METRICS: %s\n\n" "$metrics_summary" >> "$RESULT_FILE"
@@ -146,3 +163,4 @@ done < "$PROMPT_FILE"
 
 echo "---------------------------------------------------"
 echo "Done! Processed $total_prompts prompt(s)."
+