Skip to content
ThejavaZ
Liz-AI
Public

About

No description, website, or topics provided.

Resources

Readme
MIT license
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository
BranchesTags

Repository files navigation

Liz-AI

Personal AI assistant for Linux, built with Python.

Liz-AI is a personal assistant designed to help with programming, system tasks, organization, and everyday computer interactions.

Built with modularity, security, and practical AI tooling in mind.

Features

  • Multi-provider AI support (Gemini, OpenAI, Claude, DeepSeek, Ollama)
  • Model fallback chain — a bad model name degrades instead of failing
  • Local model support via Ollama (no API key, no quota)
  • Tool calling with permission control
  • Persistent conversation memory (SQLite)
  • Fact storage and session restore
  • Persistent notes with search
  • Web search without an API key
  • Safe calculator (AST-based, never executes code)
  • Streaming responses (token by token, all providers)
  • Markdown rendering in the terminal (headings, bold, lists, quotes, links, aligned tables, code fences) with ANSI and zero dependencies
  • Remember / forget facts (/remember, /forget, plus a memory tool Liz can use herself)
  • Multiple named sessions (/session) persisted across runs
  • Memory search over past messages and facts (/recall, also a memory tool action)
  • Export / import conversation history as JSON (/export, /import)
  • Context summarization when the conversation grows too long
  • Health diagnostics (/doctor) and config inspection (/config)
  • Voice interaction: speech-to-text (/talk), text-to-speech, wake words and a background listener (/voice on|off)
  • Permission policy with real confirmation prompts for sensitive tools (run_command, note deletion, fact removal)
  • Path sandbox for filesystem access and terminal arguments (FS_ROOTS, home + work dir by default)
  • Internal CLI commands (/help, /tools, /models, /memory, /notes, /config, /doctor, /clear, /status)
  • Docker support
  • 584+ automated tests

Quick Start

Installation

git clone git@github.com:ThejavaZ/Liz-AI.git
cd Liz-AI
python -m venv .venv
source .venv/bin/activate
pip install -e .

Optional extras:

pip install -e ".[web]"   # optional second web-search backend
pip install -e ".[dev]"   # test tooling

Configuration

cp .env.example .env

Edit .env with your API keys:

AI_PROVIDER=gemini
AI_MODEL=gemini-2.0-flash
GEMINI_API_KEY=your-key-here

Run

python main.py

Multiline input

  • Paste a paragraph — it arrives as a single message you can edit before sending with Enter (bracketed paste; terminals without it are coalesced automatically).
  • End a line with \ to keep writing on the next line (... continuation prompt).
  • Ctrl+C cancels the draft in progress; at a clean prompt it leaves the CLI. Ctrl+D or /exit also quits.

Docker

docker compose up --build

CLI Commands

Internal commands are available without calling the AI provider:

Command Description
/help Show available commands
/tools List registered tools
/models List models for the current provider
/memory Show stored facts and session info
/remember Store a fact: /remember editor = neovim
/forget Remove a stored fact
/session List sessions, or /session <name> to switch
/recall Search past messages and facts
/export Save history to JSON: /export [file.json]
/import Load history from JSON: /import <file.json>
/notes Manage notes: list / add / del / find
/talk One voice turn: record, transcribe, answer aloud
/voice Voice status, or /voice on / off for the listener
/config Show effective configuration (keys masked)
/doctor Run health diagnostics
/clear Clear conversation history
/status Show provider, model, and state
/exit Exit Liz AI
/quit Exit Liz AI

Any input without / is sent to the AI provider.

Supported Providers

Provider Variable Default Model API Key
Gemini AI_PROVIDER=gemini gemini-2.0-flash GEMINI_API_KEY
OpenAI AI_PROVIDER=openai gpt-4o-mini OPENAI_API_KEY
Claude AI_PROVIDER=claude claude-sonnet-4-20250514 ANTHROPIC_API_KEY
DeepSeek AI_PROVIDER=deepseek deepseek-chat DEEPSEEK_API_KEY
Ollama AI_PROVIDER=ollama llama3.1 none (local)

Set the provider and model in .env:

AI_PROVIDER=openai
AI_MODEL=gpt-4o-mini
OPENAI_API_KEY=sk-...

AI_MODEL is optional. If omitted, Liz falls back to the provider default and then through a fallback chain, so a stale or mistyped model name degrades instead of failing.

Local models (Ollama)

Run models locally with no API key and no quota:

ollama pull llama3.1
AI_PROVIDER=ollama
AI_MODEL=llama3.1

Tools

Liz has access to the following tools:

Tool Description
list_directory List files in a directory
read_file Read file contents
file_exists Check if a file exists
git_status Show git status
git_log Show git log
git_branch List git branches
run_command Execute a shell command
calculator Evaluate math safely (AST-based, no eval)
web_search Search the web (DuckDuckGo, no API key)
notes Create, list, search and delete persistent notes
memory Remember, forget or search facts and past conversation

Tools are controlled by a permission layer (ALLOW/DENY/ASK).

web_search queries DuckDuckGo's HTML endpoint directly and uses the optional ddgs package when installed (formerly duckduckgo-search) as a backup, so it works either way.

Voice

Voice runs entirely locally through external binaries — no Python ML dependency. Every piece is optional: missing engines degrade to clear messages instead of failing.

Piece Engine How Liz finds it
Capture arecord (ALSA) PATH
Speech-to-text whisper.cpp WHISPER_BIN / WHISPER_MODEL, or auto-detected under whisper.cpp/
Text-to-speech spd-say → espeak-ng → espeak first one installed wins

Install the pieces you want:

sudo apt install espeak-ng            # TTS (dependable fallback)
git clone https://github.com/ggml-org/whisper.cpp
cmake -B whisper.cpp/build -S whisper.cpp
cmake --build whisper.cpp/build -j    # STT binary
bash whisper.cpp/models/download-ggml-model.sh base   # ~74 MB model

Then, inside Liz:

  • /talk — one turn: record a window, transcribe it, answer aloud
  • /voice on — background listener; say a wake word ("hola liz") first
  • /voice off — stop listening
  • /voice — status of engines, language, wake words and listener

Tuning in .env:

VOICE_LANGUAGE=es            # STT/TTS language
VOICE_RECORD_SECONDS=5       # length of each listening window
VOICE_WAKE_WORDS=hola liz, hey liz
WHISPER_BIN=whisper.cpp/build/bin/whisper-cli
WHISPER_MODEL=whisper.cpp/models/ggml-base.bin

Voice activity detection is an energy threshold over the recorded PCM (core/voice.py), so silence never reaches the agent.

Security

Liz runs tools under an explicit policy (see src/liz/tools/):

  • Permission policy (permissions.py): every tool has a rule — read-only tools are allow; run_command, deleting a note and forgetting a fact are ask; anything unregistered defaults to ask. Rules can be narrowed per action (notes:delete).
  • Confirmation is real (registry.py): ask shows an interactive Allow? [y/N] prompt with a preview of the request. No prompt available (tests, pipes) means deny.
  • Path sandbox (sandbox.py): filesystem tools and run_command arguments are resolved (symlinks and .. included) and must stay inside the allowed roots — home and the working directory by default, configurable with FS_ROOTS.
  • Terminal allowlist (terminal.py): only a fixed set of safe commands, no shell, 30s timeout, and find -exec / -delete style escape hatches rejected.

/tools shows the effective permission of each tool.

Architecture

CLI → CommandHandler (internal commands)
       ↓
     Agent → AIProvider (fallback chain) → Tool Calling → ToolRegistry → PermissionLayer → Tool
       ↓
   ConversationMemory → MemoryStore (SQLite)

Project Structure

src/liz/
├── cli/
│   ├── app.py          # CLI entry point
│   ├── markdown.py     # ANSI renderer for streamed replies
│   ├── voice.py        # VoiceLoop: turns + background listener
│   └── commands.py     # Internal commands handler
├── config/
│   └── settings.py     # Pydantic settings
├── core/
│   ├── agent.py        # Agent with tool calling loop
│   ├── context.py      # Conversation context
│   ├── memory.py       # Memory port + ConversationMemory
│   ├── notes.py        # Notes port + Note
│   ├── voice.py        # STT/TTS ports + VAD and wake word policies
│   └── models.py       # Domain models
├── infrastructure/
│   ├── ai/
│   │   ├── base.py           # AIProvider interface
│   │   ├── factory.py        # Provider factory
│   │   ├── fallback.py       # Model fallback chain
│   │   ├── models_registry.py# Provider → models map
│   │   ├── gemini.py         # Google Gemini
│   │   ├── openai.py         # OpenAI
│   │   ├── claude.py         # Anthropic Claude
│   │   ├── deepseek.py       # DeepSeek
│   │   └── ollama.py         # Local models via Ollama
│   ├── persistence/
│   │   ├── memory_store.py   # SQLiteMemoryStore
│   │   └── notes_store.py    # SQLiteNotesStore
│   └── voice/
│       ├── audio.py          # arecord capture + WAV reading
│       ├── speech_to_text.py # whisper.cpp adapter + factory
│       └── text_to_speech.py # spd-say / espeak adapters + factory
└── tools/
    ├── base.py         # Tool interface
    ├── models.py       # ToolResult, ToolStatus
    ├── registry.py     # ToolRegistry + ask confirmation
    ├── permissions.py  # PermissionLayer (per tool and per action)
    ├── sandbox.py      # Path roots shared by fs and terminal tools
    ├── filesystem.py   # File operations (sandboxed)
    ├── git.py          # Git operations
    ├── terminal.py     # Allowlisted commands (sandboxed args)
    ├── calculator.py   # Safe math evaluation
    ├── web.py          # Web search
    └── notes.py        # Notes tool

Development

Run Tests

python -m pytest tests/ -v

Test Coverage

tests/
├── memory_helpers.py           # Shared in-memory MemoryStore double
├── test_context.py            # Context and conversation history
├── test_models.py             # Domain models
├── test_agent.py              # Agent chat loop and tool rounds
├── test_factory.py            # Provider selection
├── test_providers.py          # Provider integrations
├── test_model_flexibility.py  # Model registry and fallback chain
├── test_memory.py             # Memory persistence
├── test_memory_tool.py        # Memory tool (remember/forget/list/recall)
├── test_phase6.py             # Sessions, recall, export/import, summarizer
├── test_markdown.py           # Terminal Markdown renderer
├── test_notes.py              # Notes persistence and tool
├── test_calculator.py         # Safe math evaluation
├── test_web.py                # Web search tool
├── test_streaming.py          # Streaming responses and fallback
├── test_openai_tools.py       # Tool calling for OpenAI-compatible providers
├── test_filesystem.py         # File operations
├── test_git.py                # Git operations
├── test_terminal.py           # Terminal commands
├── test_permissions.py        # Permission layer
├── test_registry.py           # Tool registry
├── test_tool_models.py        # Tool result models
├── test_tool_calling.py       # Agent-tool integration
├── test_voice.py              # Voice: VAD, wake word, engines, loop
├── test_security.py           # Confirmations, path sandbox, terminal args
└── test_commands.py           # CLI commands

Roadmap

Phase 1 — Foundation ✅

  • Project structure
  • Configuration layer
  • Agent structure
  • Prompt system
  • Basic CLI conversation
  • AI provider integration

Phase 2 — Tools ✅

  • Filesystem tools
  • Git tools
  • Terminal tools
  • Permission system
  • Tool error handling
  • Multi-provider support
  • Tool calling integration

Phase 3 — CLI ✅

  • Internal commands
  • Docker support
  • Model registry and fallback chain
  • Local model support (Ollama)
  • /models command

Phase 4 — Memory ✅

  • Persistent conversation history (SQLite)
  • Fact storage
  • Session restore on startup
  • /memory command
  • Persistent notes with search
  • /notes command

Phase 5 — Practical Tools ✅

  • Safe calculator (AST-based, no eval)
  • Web search (DuckDuckGo, no API key)
  • /config effective settings view
  • /doctor health diagnostics
  • Model fallback chain and /models

Phase 6 — Conversation Quality ✅

  • Streaming responses (token by token)
  • Conversation summarization when context grows
  • Multiple named sessions (/session)
  • Memory search (retrieve relevant past facts with /recall)
  • /remember and /forget commands
  • Export / import conversation history (/export, /import)
  • Markdown rendering in terminal output (cli/markdown.py, stdlib ANSI)

Phase 7 — Voice ✅

  • Speech-to-text (whisper.cpp by subprocess, infrastructure/voice/speech_to_text.py)
  • Text-to-speech (spd-say → espeak-ng autodetect, infrastructure/voice/text_to_speech.py)
  • Voice interaction loop (/talk, cli/voice.py)
  • Wake word / activation ("Hey Liz", configurable via VOICE_WAKE_WORDS)
  • Voice activity detection (energy VAD, stdlib, core/voice.py)
  • Background listening daemon (/voice on|off)

Phase 8 — Proactivity

  • Event monitors (battery, network, builds)
  • Scheduled tasks / cron-like routines
  • Proactive notifications
  • Rule engine ("when X happens, do Y")
  • Priority and quiet hours

Phase 9 — System Integration

  • Application launching
  • Process management
  • System information
  • Desktop notifications
  • Clipboard access
  • Screenshot capture
  • Window management

Phase 10 — Multimodal

  • Screen reading / vision
  • Image understanding
  • Document parsing (PDF, Office)
  • Web page reading

Phase 11 — Automation & Workflows

  • Multi-step workflow engine
  • Scriptable routines
  • Task chaining (tool → tool)
  • Undo / rollback for actions
  • Action history and audit log

Phase 12 — Personalization

  • User preference learning
  • Custom system prompt per user
  • Tone and style adaptation
  • Frequently used tool shortcuts
  • Per-project context profiles

Phase 13 — Ecosystem

  • Plugin / extension architecture
  • Third-party tool registry
  • MCP (Model Context Protocol) support
  • REST API for external control
  • Web dashboard

Phase 14 — Reliability & Security

  • Rate limit handling and retry policies
  • Sandboxed command execution
  • Secret redaction in logs
  • Encrypted memory at rest
  • Audit trail for tool calls
  • Graceful offline mode

Security

  • AI commands require explicit permission
  • Tool execution controlled by PermissionLayer
  • No unrestricted system access
  • User confirmation for sensitive operations

License

See LICENSE for details.

About

No description, website, or topics provided.

Resources

Readme
MIT license
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages