krisclarkdev/edge-command-model

Edge Command Model — EVE-OS & Linux Terminal Assistant

A fine-tuned Qwen3-0.6B model trained to act as a lightweight command assistant for EVE-OS edge devices and Linux systems. It accepts natural language requests and responds exclusively with structured JSON tool calls.

Intended Use

This model runs on edge hardware (ARM or x86 CPU, no GPU required) and serves as an on-device command assistant for operators managing EVE-OS edge nodes. It is designed for offline, air-gapped, or bandwidth-constrained environments where cloud-based LLMs are not available.

Example interaction:

User: Show memory usage
Model: {"tool": "terminal", "command": "free -h"}

User: What is zedagent?
Model: {"tool": "explain", "text": "zedagent is the main EVE-OS orchestration agent. It processes configurations from ZedCloud, manages application deployment, handles device attestation, and coordinates all other EVE services."}

Output Format

The model always responds with a single JSON object in one of two formats:

Terminal commands (for actions to execute):

{"tool": "terminal", "command": "<shell command>"}

Explanations (for informational queries):

{"tool": "explain", "text": "<explanation>"}

Training Details

Parameter	Value
Base model	Qwen/Qwen3-0.6B
Method	QLoRA (4-bit quantization during training)
LoRA rank (r)	32
LoRA alpha	64
Target modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Learning rate	1e-4
Scheduler	Cosine
Epochs	15
Max sequence length	512
Training examples	1,715
Training hardware	Single consumer GPU

Performance

Tested against 100 randomly sampled prompts from the training set.

JSON Validity Rate: 99.3% Tool Routing Accuracy: 98.6% Exact Match Accuracy: 20.0% Fuzzy Match Accuracy: 27.6% Average Inference Time: 0.692s per query Peak Memory Usage: 736.0 MB

Training Data

The model was trained on 1,715 instruction-output pairs covering:

• ~350 unique commands with 4-5 phrasing variations each
• Linux commands: file operations, text processing, networking, process management, disk/storage, kernel modules, containers (containerd/runc), ZFS, LVM, security, namespaces, cgroups
• EVE-OS commands and concepts: all pillar microservices (zedagent, nim, domainmgr, zedrouter, volumemgr, etc.), device filesystem paths (/persist, /config, /run), ZedCloud connectivity, EdgeView, TPM management, containerd operations
• Explanations: EVE-OS architecture, Linux subsystems, file paths, configuration files

All training data was human-curated and reviewed for accuracy.

Quantization

The model is provided in GGUF format quantized to Q4_K_M for efficient CPU-only inference.

Format	File Size	RAM Required	Use Case
Q4_K_M (recommended)	~450 MB	2-4 GB	Edge deployment, CPU inference
Q8_0	~700 MB	4-6 GB	Higher accuracy, more RAM available
F16	~1.2 GB	6-8 GB	Maximum accuracy, development/testing

Hardware Requirements

Minimum:

• CPU: Any modern ARM or x86 processor
• RAM: 2 GB
• Storage: 500 MB
• GPU: Not required

Recommended:

• CPU: ARM Cortex-A72 or better / x86-64
• RAM: 4 GB
• Storage: 1 GB

Tested on Raspberry Pi 4 (4GB) and x86 edge gateways.

How to Use

With llama.cpp

./llama-cli -m edge-command-model-Q4_K_M.gguf \
  --temp 0.1 \
  --top-p 0.9 \
  -p "<|im_start|>system
You are an edge device command assistant. You respond ONLY with valid JSON tool calls. Never respond with plain text. Available tools: terminal, explain.
<|im_end|>
<|im_start|>user
Show disk space
<|im_end|>
<|im_start|>assistant
"

With Ollama

Create a Modelfile:

FROM ./edge-command-model-Q4_K_M.gguf
SYSTEM "You are an edge device command assistant. You respond ONLY with valid JSON tool calls. Never respond with plain text. Available tools: terminal, explain."
PARAMETER temperature 0.1
PARAMETER num_ctx 512

Then:

ollama create edge-cmd -f Modelfile
ollama run edge-cmd "Show memory usage"

With llama-cpp-python

from llama_cpp import Llama

model = Llama(model_path="edge-command-model-Q4_K_M.gguf", n_ctx=512, n_threads=4)

prompt = """<|im_start|>system
You are an edge device command assistant. You respond ONLY with valid JSON tool calls. Never respond with plain text. Available tools: terminal, explain.
<|im_end|>
<|im_start|>user
Show memory usage
<|im_end|>
<|im_start|>assistant
"""

output = model(prompt, max_tokens=128, temperature=0.1, stop=["<|im_end|>"])
print(output["choices"][0]["text"])

Coverage

The model covers commands and concepts across these categories:

Linux: File operations, text processing (grep, sed, awk), networking (ip, ss, tcpdump, iptables), process management, disk/storage (lsblk, fdisk, ZFS, LVM), kernel modules, containers (containerd, runc), security (namespaces, cgroups, capabilities), compression, certificates (openssl), WireGuard

EVE-OS: All pillar microservices (zedagent, nim, domainmgr, zedrouter, volumemgr, baseosmgr, tpmmgr, vaultmgr, loguploader, ledmanager, nodeagent, and more), device filesystem layout (/persist, /config, /run), ZedCloud communication, EdgeView remote diagnostics, containerd operations on EVE, ZFS pool management, device identity and certificates

Limitations

• The model is trained on a fixed set of ~350 commands. It may hallucinate plausible but incorrect commands for requests outside its training distribution.
• Explain responses are generated, not memorized. Factual accuracy of explanations should be verified for critical operations.
• The model does not support multi-turn conversation. Each request is independent.
• Complex compound commands (multi-pipe chains) may be less accurate than single commands.
• The model was trained for EVE-OS specifically and may not generalize well to other edge operating systems.

Safety

This model is intended to be used behind an agent harness that:

• Requires user confirmation (y/n) before executing any terminal command
• Blocks dangerous commands (rm -rf /, mkfs on mounted volumes, fork bombs)
• Enforces timeouts on command execution
• Limits output capture size

Never execute model outputs directly without human review.

License

MIT