A standalone, agentic LLM shell for your terminal.
Hali is a local-first agentic coding/chat shell built on llama.cpp, rendered with notcurses. No server, no browser tab, no cloud dependency — just a fast terminal UI driving a local model with tool use, RAG, MCP, Web development server and careful context management.
Most agentic shells assume a hosted API and treat context as free. Hali assumes neither: it's built for consumer GPUs (developed against a 16GB RTX 5060 Ti) where the KV cache is a scarce resource and every tool call has to earn its place in the context window.
-
notcurses TUI — plane-based rendering, modal popups, persistent input history (Up/Down navigation), live
/setcommands for generation parameters, in-chat search, Kitty keyboard protocol support for reliable input in modern terminals, and mouse scroll-wheel support. -
Theme system — three built-in themes (Dark, Light, Navy) switchable at runtime with
/themeor F12. -
Fragmentation-aware KV cache management — a
KVCachePresetenum (F16/Balanced/Compact) coupled to flash-attention settings, with graceful recovery when sequence removal fragments the cache. -
Dynamic tool-result budgeting —
max_tool_result_size()targets ~75% of remaining context so a single large tool result can't blow the budget. -
Unambiguous tool-call protocol — an explicit
HALI_END_TOOLterminator so tool boundaries never get confused with model chatter. -
Multi-Token Prediction (MTP) — optional speculative decoding via MTP-capable GGUF models. Configurable with
mtp_enabled,mtp_n_max,mtp_n_min,mtp_p_min,mtp_adaptive, andmtp_pos_thresholdsettings to control how many draft tokens are proposed, the acceptance threshold, and adaptive self-tuning. -
Suffix drafting — a lightweight n-gram
SuffixDrafterthat matches recent tokens against earlier context and drafts continuations, helping with repetitive/structured text when MTP acceptance is low. -
GPU introspection —
gpu_infodetects VRAM via the ggml backend API and scales the KV-cache restart threshold (8 GB → 75 %, 16 GB → 85 %, 32 GB → 90 %, cap 92 %). -
Usage logging — per-turn metrics (LLM latency, completion tokens, tool calls/errors, KV usage, files read/written) tracked in a
UsageLogwith JSON export and KV-trend detection. -
Audio system — polyphonic playback via a note-script parser (
TOOL:PLAY). Supports sine tones, chords, tempo/volume/length control tokens, and optional UI sounds (tool-completion tick, user-prompt ding). Enable withaudio_enabledin config or/audio. -
Planning tool —
TOOL:PLANprovides structured goal/step tracking (set,add,advance,skip) so the model can maintain a visible work plan across turns. Plans are referenced inSESSION.mdfor session handoff. -
Codebase audit —
/auditscans C/C++ source files for issues that make the codebase difficult for an LLM to handle (e.g. missing includes, implicit declarations, overly long functions). Results are scored 0–100, formatted as a table, and fed back to the model as a remediation prompt. Confirmed false positives are persisted toaudit_feedback.jsonand filtered from future scans. -
Sampling profiles — after model load, Hali prompts for a sampling profile (General, Programming, Creative, Concise) that sets temperature, top_p, top_k, min_p, and penalty parameters in one step. Smart defaults are also applied based on detected VRAM and GPU layer count.
-
Task modes —
/taskswitches the agent's behavioral focus betweenweb,music,audio,knowledge, ordefault. Each mode adjusts the system prompt, enables relevant tools, and can start/stop subsystems (e.g. the web server forwebmode). -
Multi-track music —
TOOL:MUSICmanages a persistent multi-track project with per-track volume, pan, and note scripts. Supports pitch notes, drum tokens (kick/snare/hi-hat/open-hat/tom), chords, and global tempo. Projects can be saved, loaded, listed, and exported to stereo.wav. -
Command completions — tab-completion for slash commands in the TUI input line. With multiple matches, repeated Tab presses cycle through the options (wrapping around); a single match is accepted immediately.
-
A full sandboxed tool suite — every file operation is scoped to a sandbox root (the directory Hali is launched in, or an explicit path argument):
Tool Purpose TOOL:LIST,TOOL:READ,TOOL:EXISTSInspect files inside the sandbox (read access also extends to extra_dirs)TOOL:SEARCHSearch file contents by pattern with rich flags: --recursive,--line-numbers,--count,--line-count,--files-only,--name(match filenames),--include=<glob>,--context=<n>(surrounding lines),--lines=<start>,<end>(line range)TOOL:WRITE,TOOL:APPEND,TOOL:PATCH,TOOL:MKDIRModify files inside the sandbox TOOL:REPLACEReplace text in a file with --alland--countflagsTOOL:RUNRun a program inside the sandbox, gated by an optional run_allowedallowlist and shell-metacharacter detectionTOOL:BUILDAuto-detect and run project build systems (CMake, Gradle, Maven, Python) with modes: setup,build,test,clean,lint(clang-tidy)TOOL:CURLHTTP GET via libcurl, with HTML-to-text stripping and a 32 KB response size cap TOOL:RAGQuery the local RAG index for extra context TOOL:ASKAsk the user directly, for interactive/agentic flows TOOL:GRAPHRender bar charts and tree diagrams as ASCII art from a small JSON schema TOOL:TABLEFormat pipe-delimited rows into an aligned text table TOOL:MCPInvoke a tool on a connected MCP server TOOL:PERMISSIONExplicit user confirmation before destructive or irreversible actions TOOL:RESTARTHand off to a fresh session, writing current task context to SESSION.mdfirstTOOL:PLAYPlay a melody from a note-script (sine tones, chords, tempo/volume/length control) TOOL:MUSICMulti-track music project: create,load,list,add <track> <script>,set <n> <vol|pan|script> <v>,remove <n>,tempo <bpm>,play,show,save,export,clear. Supports pitch notes, drum tokens (K/S/H/O/T), chords, per-track volume/pan.TOOL:PLANStructured goal/step planning: set <goal>,add <step>,advance,skip <idx>TOOL:CHOICEMultiple-choice popup: `TOOL:CHOICE "question" "opt1 TOOL:STATFile metadata: size, type, permissions, line/word/char count TOOL:SYSINFOSystem info: user, uid, hostname, OS, kernel, arch, cwd TOOL:GITRun whitelisted git subcommands ( status,log,diff,add,commit,branch,show) in the sandboxTOOL:HEXxxd-style hex dump with optional offset and length, capped at 256 lines TOOL:TUNEAdjust sampling settings live: temperature,top_p,min_p,top_k,penalty_repeat,penalty_freq,penalty_present,penalty_last_n(values are clamped to valid ranges)TOOL:DATE,TOOL:TIME,TOOL:RND,TOOL:INTROSPECTSmall utility/introspection tools TOOL:GUIOpen a browser tab at http://localhost:8080/ai(requires-p, --web-port <port>at startup), then just tell Hali what to show — a chart, a diagram, a form, anything. Hali renders your HTML in the browser and can read input back from it. Subcommands:<html_fragment>(display HTML),input(read pending browser input),clear(reset the display),help(show usage). -
Persistent run-allowlist — user-approved
TOOL:RUNprogram names are saved to.hali/user-approved.txtunder the sandbox with timestamps and a 30-day TTL, so approvals survive across sessions without re-prompting. Revoke individual approvals with/revoke <name>. -
MCP client — connect to external Model Context Protocol servers (e.g. JetBrains IDE built-in MCP servers) with
--mcp, filter which tools get exposed with--mcp-filter, and dry-run the resulting system context with--mcp-test. Server connection details live inmcp.json. -
Skills system — load one or more markdown skill files into the static system-prompt prefix at session start with
--skill <name>(no per-turn routing, so it doesn't disrupt the KV cache).hali.md,persona.md, andAGENTS.mdare auto-discovered from the current directory if present. Skills can also be loaded mid-conversation with/skill <name> [kickoff]. Theskills/folder ships a starter set covering debugging, TDD, code review, CMake/build troubleshooting, memory-safety review, dependency-free frontend work, SmallBASIC's raylib plugin, free JSON API sourcing, a local-info feed pattern, planning, spiking, code simplification, house style, skill import, and a few just-for-fun ones (chess, an ELIZA-style roleplay, generative interactive fiction, a self-scoring introspection game). -
Pure C++ RAG pipeline — semantic chunker, binary
.dbindex, deduplicatingRagSession, with a folder picker for building indexes on the fly. -
Web development mode — a local HTTP server with live-reload that watches the sandbox for HTML changes. The model can push messages to the browser and trigger reloads, enabling a tight edit-preview loop. Enabled with
-p, --web-port <port>. -
Persistent settings — configuration lives in
.hali/config.jsonunder the sandbox, or point at a specific file with-c/--config(defaults to.hali/config.json). Runtime data (input history, usage log, user-approved run list, plans, badges) is stored in the.hali/folder under the sandbox root. -
Test suite — unit tests under
tests/(file operations, string/Unicode utilities, SHA-1, MCP message formatting, the graph renderer), runnable via CTest.
Every tool that touches the filesystem is confined to a sandbox root — the directory Hali is launched in, or an explicit path argument. The security model is layered:
All file operations resolve their target path against the sandbox root using std::filesystem::canonical() (which resolves symlinks and .. components) and verify the result is a prefix of the sandbox. This prevents directory traversal via ../../etc/passwd, symlink escapes, and absolute-path injection.
Two access tiers are enforced:
| Tier | Function | Applies to |
|---|---|---|
| Write (strict) | path_in_sandbox() |
TOOL:WRITE, TOOL:PATCH, TOOL:APPEND, TOOL:MKDIR, TOOL:REPLACE |
| Read (extended) | path_allowed() |
TOOL:LIST, TOOL:READ, TOOL:EXISTS, TOOL:SEARCH, TOOL:STAT, TOOL:HEX |
Read access extends to a configurable list of extra directories (extra_dirs in config, or added at runtime with /allow <path>). Write access is always sandbox-only.
TOOL:RUN is gated by three mechanisms:
- Allowlist — if
run_allowedis set (comma-separated basenames), only those programs may execute. An empty allowlist permits any program inside the sandbox. - Shell-metacharacter detection —
has_shell_metachars()rejects arguments containing;,|,&,>,<,$, backticks, or newlines that could enable command injection. - Dangerous-pattern check —
hasDangerousPatterns()flags known-destructive commands (e.g.rm -rf /,dd,mkfs) even when they pass the allowlist.
User-approved programs are persisted to .hali/user-approved.txt under the sandbox with a 30-day TTL.
Before any write operation, Hali applies a series of validation gates:
- Brace/paren balance — for C-family and curly-brace languages, unbalanced content is flagged.
- Size-shrink warning — if new content is less than 50% of the original file size, a confirmation dialog appears.
- Large-file warning — files exceeding
val_write_size_warnbytes (default 1000) trigger a warning. - Conflict-marker detection — patch delimiters (
<<<<<<<,=======,>>>>>>>) in the payload are flagged. - Timestamped backup — if
--backup-pathis set, the original file is copied to a timestamped backup before the write.
These gates can be individually disabled via val_write_validate and val_patch_validate config keys for models that are known to produce well-formed output.
With -n, --prompt-permission, destructive tool calls (TOOL:WRITE, TOOL:REPLACE) require explicit user confirmation via a TUI dialog before executing. TOOL:PERMISSION provides a general-purpose confirmation mechanism for any action the model deems risky.
TOOL:READis capped at 32 KB per call; larger files require--lines=<start>,<end>to read specific sections.TOOL:CURLresponses are capped at 32 KB.TOOL:HEXoutput is capped at 256 lines.
TOOL:GIT only permits a fixed set of subcommands: status, log, diff, add, commit, branch, show. Arbitrary git commands (e.g. push, reset --hard, config) are rejected.
/security-audit marks the sandbox as security-audited by writing a marker file (.hali/.sec_audit) whose contents include a SHA-1 hash incorporating a compile-time salt that is not exposed to the model or user. This prevents the model from forging a valid audit marker. The marker is checked before certain sensitive operations and can be cleared with clear_audited() (e.g. after sandbox contents change).
The log_erase config option (or --log-erase) truncates the log file at startup, preventing unbounded log growth across sessions.
Inside the TUI, type a leading / to run a built-in command. Tab-completion is available for command names.
| Command | Description |
|---|---|
/model [path] |
Load or hot-reload a GGUF model (opens a file picker if no path given) |
/embed [path] |
Load an embedding model for RAG (file picker if no path) |
/rag [path] |
Index a file or directory for RAG, or load an existing .bin index (picker if no path) |
/memory |
Show KV cache, VRAM, and layer offload stats |
/clear |
Reset the conversation (rebuilds the system prompt) |
/theme |
Cycle through the Dark / Light / Navy themes (F12 also works) |
/audio [test|volume <v>] |
Enable/disable audio, play a test tone, or set volume |
/task [name] |
Direct agent behavior toward a mode: web, music, audio, knowledge, or default |
/save [file] |
Save the current chat transcript to a file |
/session [file] |
Save a structured session snapshot (task, pending actions, errors, last output) |
/settings |
Display the current runtime settings |
/set <key> <value> |
Change a setting live (e.g. /set temperature 0.7) |
/allow [path] |
Grant read access to a directory outside the sandbox |
/allow-list |
List all directories with read access |
/allow-remove [path] |
Remove read access to a directory |
/revoke <name> |
Remove an approved TOOL:RUN program from the allowlist |
/audit [path] |
Scan the codebase for issues that are hard for an LLM to handle |
/security-audit |
Mark the sandbox as security-audited (compile-time salt prevents model forgery) |
/lint |
List source files and run clang-tidy lint checks |
/skill [name] [kickoff] |
Load a skill file into the conversation (optionally with a kickoff prompt) |
/tool [TOOL:NAME [args]] |
Invoke a tool directly (no args lists all available tools) |
/plan [text] |
Open a multiline text editor for a plan |
/index [name|status|remove|load|unload] |
Manage knowledge sources (ZIM/Wikipedia) |
/help |
Show this command list |
exit / quit |
Leave Hali |
Settable keys via /set: temperature, top_p, top_k, min_p, penalty_repeat, penalty_last_n, rag_top_k, n_gpu_layers, offload_kqv, run_allowed (comma-separated list of allowed program basenames, or none to clear), mtp_enabled, mtp_n_max, mtp_n_min, mtp_p_min, mtp_adaptive, mtp_pos_threshold, rope_freq_scale, rope_scaling_type, audio_enabled, audio_volume, audio_tool_tick, audio_user_prompt, val_write_validate, val_patch_validate, val_write_size_warn, log_erase, disable_run, disable_save_on_exit.
Hali can build offline knowledge bases from ZIM archives (Wikipedia, DevDocs, Stack Exchange, etc.) and query them via RAG. The workflow is: pick a .zim file → index → query.
- An embedding model loaded (
/embedor-eat startup) — required for indexing and retrieval. - zim-tools (
zimdump) installed for ZIM text extraction:sudo apt install zim-tools. - A
.zimfile on disk (download separately from Kiwix or any other source).
All subcommands that accept a source name will show an interactive picker (TUI multi-choice popup) when no name is given. Press Esc to cancel.
| Subcommand | Description |
|---|---|
/index |
Picker: select an indexed source to load, "Pick a .zim file…", or "status" |
/index status |
Show status: indexed sources, loaded sources, disk usage |
/index <path> |
Index a .zim file from a path and load it (one step) |
/index load [id] |
Load a .rag index into the session (picker if no id) |
/index unload [id] |
Remove a loaded index from the session (picker if no id) |
/index remove [id] |
Delete the .rag and .zim files for a source (picker if no id) |
# 1. Pick a .zim file from the interactive picker
/index
# 2. (or target a specific file directly)
/index /path/to/devdocs_en_python.zim
# 3. Query it — the model uses TOOL:RAG automatically
Files are stored under the sandbox's .hali/ directory:
| Path | Contents |
|---|---|
.hali/rag/<id>.rag |
Binary RAG index (embeddings + chunks) |
.hali/kiwix-index.json |
Local tracking index of indexed sources (id, paths, chunk count, timestamp) |
At startup, after the embedding model is loaded, Hali automatically scans .hali/rag/ and loads any .rag files found. This means previously indexed sources are available immediately without manual /index load.
Indexed sources are merged into the same RagLibrary used by TOOL:RAG. When the model calls TOOL:RAG, it searches across all loaded sources (project code + knowledge base) and returns the most relevant chunks. The model's system prompt is informed of available knowledge sources so it knows when to query them.
Hali vendors llama.cpp as a submodule and links everything statically.
git clone --recurse-submodules https://github.com/chrisws/hali.git
cd hali
./setup.sh # submodules, deps, build, smoke test — one shot
./build/bin/hali
setup.sh is a one-shot bootstrap: it checks for required dependencies (cmake, g++, make, notcurses), initialises submodules, builds the project, runs a --help smoke test, detects GPU VRAM (via a compiled CUDA probe or nvidia-smi), downloads an appropriate Qwen3 GGUF model from HuggingFace into models/, and generates a ready-to-use .hali/config.json tuned to your hardware. Pass --test to also build and run the unit test suite.
cmake -B build -DLLAMA_BACKEND=AUTO
cmake --build build -j
./build/bin/hali
./maintain.sh # build + test (default)
./maintain.sh build # build only
./maintain.sh test # run tests only
./maintain.sh clean # clean build artifacts
cmake -B build-tests -S tests
cmake --build build-tests -j
ctest --test-dir build-tests
-DLLAMA_BACKEND= |
Behavior |
|---|---|
AUTO (default) |
Probes for nvcc; falls back to CPU + OpenMP if CUDA isn't found |
CPU |
Forces CPU with native SIMD optimizations |
GPU |
Non-CUDA GPU path, CPU/OpenMP fallback |
CUDA |
Forces CUDA; fails the configure step if nvcc isn't found |
| Library | Required | Notes |
|---|---|---|
| notcurses | Yes | apt install libnotcurses-dev, or -DNOTCURSES_DIR=<prefix> |
| libcurl | No | Enables TOOL:CURL; apt install libcurl4-openssl-dev, or -DCURL_DIR=<prefix> |
| CUDA toolkit | No | Only for -DLLAMA_BACKEND=CUDA/AUTO with an NVIDIA GPU: apt install nvidia-open cuda-toolkit |
| yyjson | Vendored | JSON parsing for MCP and tool payloads; bundled under yyjson/, no system package needed |
hali [sandbox-dir] [options]
-m, --model <path> path to a GGUF model
-e, --embed <path> path to an embedding model (for RAG)
-g, --gpu-layers <n> number of layers to offload to GPU
-s, --skill <name> load a skill file into the system prompt (repeatable)
--mcp enable the MCP client
--mcp-filter <name> only expose this MCP tool (repeatable)
--mcp-test print the resolved MCP system context and exit
-c, --config <path> load settings from a specific JSON file
-l, --log <path> write logs to a file
-t, --think disable model "thinking" output
-n, --prompt-permission require explicit confirmation before destructive tool calls
-p, --web-port <port> enable web development mode with live-reload on the given port
-b, --backup-path <path> create file backups prior to invoking TOOL:WRITE
-h, --help show this help
A positional argument sets the sandbox root — the directory all file tools (TOOL:LIST/TOOL:READ/TOOL:WRITE/etc.) are confined to. It defaults to the current directory.
hali/
├── CMakeLists.txt
├── mcp.json # MCP server connection config
├── .hali/ # runtime data (config, history, plans, badges)
│ └── config.json # example runtime settings
├── setup.sh # one-shot setup: submodules, deps, build, smoke test
├── maintain.sh # day-to-day build / test / clean runner
├── apps/ # standalone HTML games + AI stub servers
│ ├── chess.html # chess (browser)
│ ├── checkers.html # checkers (browser)
│ ├── battleships.html # battleships (browser)
│ ├── space.html # space shooter (browser)
│ └── ai-stub/ # Python WebSocket AI opponents (stdlib only)
│ ├── chess.py
│ ├── battleship.py
│ └── space.py
├── scripts/ # utility scripts
│ └── mtp_log_summary.py # MTP performance log summariser
├── tools/ # build-time utilities
│ └── gpu_probe.c # CUDA VRAM probe (compiled by setup.sh)
├── skills/ # markdown skill files, loaded via --skill
├── src/
│ ├── main.cpp # entry point, CLI parsing
│ ├── core/ # agent loop, config, dispatch
│ │ ├── agent.cpp/.h # agent loop, system prompt assembly, think parsing
│ │ ├── agent_dispatch.cpp # tool dispatch table (all TOOL:* handlers)
│ │ ├── agent_utils.cpp/.h # path helpers, tool arg parsing, SEARCH/REPLACE/TABLE, ThinkParser
│ │ ├── config.cpp/.h # settings, persistence, help text, prompt assembly, modes
│ │ ├── knowledge.cpp/.h # KnowledgeManager: ZIM index build/load, local tracking
│ │ ├── run_allowed.h # persistent user-approved TOOL:RUN allowlist
│ │ └── shell_util.cpp/.h # shell command execution, metacharacter detection
│ ├── model/ # llama.cpp integration
│ │ ├── llama_sb.cpp/.h # llama.cpp wrapper
│ │ ├── llama_sb_mtp.cpp # MTP speculative decoding
│ │ ├── llama_sb_suffix.cpp/.h # n-gram SuffixDrafter
│ │ └── llama_sb_rag.cpp/.h # RAG session, chunker, indexer
│ ├── tools/ # tool implementations
│ │ ├── audit.cpp/.h # /audit codebase scanner, false-positive tracking
│ │ ├── build.cpp/.h # TOOL:BUILD (CMake/Gradle/Maven/Python auto-detect)
│ │ ├── curl.cpp/.h # TOOL:CURL via libcurl
│ │ ├── file.cpp/.h # sandboxed file tool implementations, path validation
│ │ ├── git_tool.cpp/.h # TOOL:GIT whitelisted subcommands
│ │ ├── mcp_client.cpp/.h # MCP protocol client
│ │ ├── mcp_format.cpp/.h # MCP message formatting
│ │ ├── patch.h # patch delimiters (PATCH_BEGIN/BOUNDARY/END)
│ │ ├── planner.cpp/.h # TOOL:PLAN structured goal/step tracking
│ │ ├── security_audit.cpp/.h # sandbox security-audit marker (compile-time salt)
│ │ ├── slash_commands.cpp/.h # slash command registry and completions
│ │ ├── sysinfo.cpp/.h # TOOL:STAT, TOOL:SYSINFO, TOOL:HEX
│ │ └── usage_log.cpp/.h # per-turn metrics tracking, JSON export
│ ├── ui/ # TUI, audio, web, rendering
│ │ ├── audio.cpp/.h # audio subsystem (init, tones, chords, scripts)
│ │ ├── audio_impl.cpp # audio backend implementation
│ │ ├── gpu_info.cpp/.h # VRAM detection, KV restart threshold
│ │ ├── graph.cpp/.h # TOOL:GRAPH ASCII chart/tree renderer
│ │ ├── icon.h # TUI icon constants
│ │ ├── input.cpp/.h # keyboard input handling
│ │ ├── input_event.h # input event types
│ │ ├── input_history.h # persistent input history
│ │ ├── music_project.cpp/.h # TOOL:MUSIC multi-track project (save/load/list)
│ │ ├── tui.cpp/.h # notcurses UI, input, rendering, themes, search, mouse
│ │ ├── tui_context.h # TUI context interface (events, rendering)
│ │ ├── ui_text.cpp/.h # in-app help, settings display, welcome banner
│ │ └── webview.cpp/.h # web development mode (HTTP server, live-reload)
│ └── util/ # shared utilities
│ ├── json.cpp/.h # yyjson wrapper
│ ├── line_list.h # line-list file I/O utility
│ ├── logging.cpp/.h # log file handling
│ ├── sha1.cpp/.h # SHA-1 hashing (RAG dedup)
│ ├── string_utils.cpp/.h # Unicode-aware string helpers
│ └── utf8.h # UTF-8 utilities
├── tests/ # unit tests, CTest-driven
├── yyjson/ # vendored JSON library
├── llama.cpp/ # submodule
└── forks/ # forked upstream projects (e.g. Strata)
The apps/ directory contains a collection of standalone, dependency-free HTML games that run entirely in the browser. Each game is a
single self-contained .html file (no build step, no npm, no CDN) designed to be served via Hali's web development mode (-p, --web-port <port>) for a tight edit-preview loop.
| Game | File | Description |
|---|---|---|
| Chess | chess.html |
Full chess with drag-and-drop, move validation, and a WebSocket AI opponent |
| Checkers | checkers.html |
American checkers with jump captures and king promotion |
| Battleships | battleships.html |
Classic naval combat with random or AI-placed fleets |
| Space | space.html |
Arcade space shooter with waves, power-ups, and high scores |
GPL2 — see LICENSE.