---
language:
- en
- id
license: mit
library_name: custom
tags:
- coding-agent
- llm-agents
- local-ai
- llama-cpp
- hybrid-inference
- python
- fastapi
- ai-agent
- qwen3.8-flash-next
pipeline_tag: text-generation
base_model: unsloth/Qwen3.8-Flash-Next-GGUF
extra:
  model_repository: unsloth/Qwen3.8-Flash-Next-GGUF
  recommended_quantization: UD-Q4_K_XL
  runtime: llama.cpp
---

# ZEVORA

**Zero-External Vendor Oriented Reasoning Agent** — a local-first hybrid AI coding
workspace with a private on-device GGUF model and optional cloud providers.
Memory, cache, experience, knowledge, and project context stay **local** by
default.

> This is a **source distribution** of the ZEVORA application on Hugging Face.
> It mirrors the GitHub repository for discoverability and distribution.
> **GitHub is the primary development source:**
> <https://github.com/dani12po/zevora>
>
> This repository does **not** contain model weights — see
> [MODELS.md](MODELS.md) for how ZEVORA uses Qwen3.8-Flash-Next through llama.cpp.

## Overview

ZEVORA is a FastAPI-based hybrid AI agent gateway. It runs a local GGUF model via
llama.cpp (no API key, no internet required) and optionally falls back to cloud
providers for complex, multimodal, or long-context work. It provides a Web UI,
an SSE/streaming chat API, an adaptive hybrid router, local memory and cache,
approval-gated project tools (MCP), and a CLI controller.

## Why ZEVORA

- **Local-first**: on-device inference keeps your data on your machine.
- **Hybrid**: cloud capacity when you want or need it; `LOCAL_ONLY` / `CLOUD_ONLY` modes.
- **Private**: prompts, memory, cache, experience, and project context remain local.
- **Controlled**: all filesystem, terminal, and git actions are approval-gated and workspace-scoped.
- **Reproducible**: deterministic configuration, verified updates, and SHA-256 checks.

## Architecture

```text
User
 │
 ▼
ZEVORA Gateway (FastAPI + Web UI)
 │
 ▼
Agent Core
 │
 ├── Cache
 ├── Memory
 ├── Experience
 ├── Knowledge
 └── Project Context
 │
 ▼
Adaptive Hybrid Router
 │
 ├── ZEVORA Local AI
 │     └── llama.cpp
 │          └── Qwen3.8-Flash-Next GGUF
 │
 └── Cloud Providers
       ├── OpenAI
       ├── Anthropic
       ├── Gemini
       ├── DeepSeek
       ├── xAI
       └── NVIDIA
 │
 ▼
Tools / MCP (approval-gated, workspace-scoped)
 │
 ▼
Verification
 │
 ▼
Final Result
```

See [ARCHITECTURE.md](ARCHITECTURE.md) and [docs/](docs/) for details.

## Local AI

`ZEVORA Local AI` is ZEVORA's on-device inference layer. It loads a single
selected GGUF model through llama.cpp lazily, on first local generation. The
local model and cloud providers share the same `AIProvider` abstract contract,
registry, discovery, router, and fallback path.

The local data layer keeps the following on your machine:

| Component | Purpose |
|-----------|---------|
| **Exact Cache** | Return previous responses to identical prompts without inference |
| **Memory** | Conversation and project records (SQLite) |
| **Experience** | Per-provider routing history that improves model selection |
| **Knowledge Engine** | Extracts reusable solution patterns from responses |
| **Project Context** | Indexed project metadata for scoped workspace operations |
| **MCP Tools** | Filesystem, Git, and terminal access scoped to the selected project |

## Qwen3.8-Flash-Next

ZEVORA is configured to use the **Qwen3.8-Flash-Next** model family from the
upstream repository:

- Repository: `unsloth/Qwen3.8-Flash-Next-GGUF`
- Recommended quantization: `UD-Q4_K_XL`
- Runtime: llama.cpp (via `llama-cpp-python`)

**ZEVORA does not bundle, modify, train, or claim ownership of Qwen weights.**
See [MODELS.md](MODELS.md) for download instructions and
[Third-Party Models](#third-party-models--attribution).

## llama.cpp

Local inference uses **llama.cpp** through the `llama-cpp-python` binding. On
Windows, install the prebuilt CPU wheel inside the project virtual environment:

```powershell
.venv\Scripts\python.exe -m pip install --prefer-binary "llama-cpp-python>=0.3.14,<0.4" --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cpu
```

For GPU acceleration, install the matching CUDA wheel for your hardware and set
`LOCAL_MODEL_GPU_LAYERS` accordingly. If no prebuilt wheel exists for your
platform, install Visual C++ Build Tools and follow the upstream llama.cpp
Windows source-build instructions.

## Installation

Requirements: **Python 3.11–3.13**

```powershell
git clone https://github.com/dani12po/zevora
cd ZEVORA
python bootstrap.py
zevora
```

Open `http://127.0.0.1:7432` in your browser. If the `zevora` command is not on
`PATH`, use `python launcher.py`.

## Quick Start

1. Install Python 3.11+.
2. Clone and run `python bootstrap.py`.
3. Open the Web UI at `http://127.0.0.1:7432`.
4. (Optional) Place your chosen GGUF at `models/` and configure `LOCAL_MODEL_PATH`.
5. (Optional) Add a cloud API key in the Providers page for cloud-first complex work.

## Configuration

All settings live in `.env`. Copy `.env.example` to `.env` to start:

```powershell
Copy-Item .env.example .env
```

API keys are **never** stored in the database or logs.

## Local Model Configuration

Relevant environment variables (see `.env.example` for the full list):

| Variable | Default | Purpose |
|----------|---------|---------|
| `LOCAL_MODEL_ENABLED` | `true` | Enable local inference |
| `LOCAL_MODEL_RUNTIME` | `llamacpp` | `llamacpp`, `ollama`, or `openai-compatible` |
| `LOCAL_MODEL_REPOSITORY` | `unsloth/Qwen3.8-Flash-Next-GGUF` | Upstream GGUF repository reference |
| `LOCAL_MODEL_QUANT` | `UD-Q4_K_XL` | Preferred quantization |
| `LOCAL_MODEL_PATH` | `models/zevora-4b-thinking.gguf` | Path to the GGUF file |
| `LOCAL_MODEL_NAME` | `qwen3.8-flash-next` | Model identifier shown in the UI/routing |
| `LOCAL_MODEL_DISPLAY_NAME` | `Qwen3.8-Flash-Next` | Human-readable name |
| `LOCAL_MODEL_CONTEXT_LENGTH` | `8192` | Context window |
| `LOCAL_MODEL_MAX_TOKENS` | `2048` | Max output tokens |
| `LOCAL_MODEL_THREADS` | `0` | CPU threads (0 = auto) |
| `LOCAL_MODEL_GPU_LAYERS` | `0` | GPU offload layers |
| `LOCAL_MODEL_BATCH_SIZE` | `512` | Prompt processing batch size |
| `LOCAL_MODEL_TEMPERATURE` | `0.4` | Sampling temperature |

## Cloud Providers

OpenAI, Anthropic, Gemini, DeepSeek, xAI, and NVIDIA are supported out of the
box; custom OpenAI-compatible providers can be registered via the Providers page
or `config/providers.json`. Keys are resolved only at request time and never
stored in manifests, logs, or the database.

## MCP / Tools

Filesystem, Git, and terminal operations are routed through a constrained local
MCP gateway scoped to the selected workspace. Read-only operations follow
workspace preferences; mutations and risky commands require explicit approval.
Paths outside the selected workspace are blocked even when approval is granted.
See [docs/MCP_TOOLS.md](docs/MCP_TOOLS.md).

## Security Model

- Local GGUF files are verified by **SHA-256** before loading when a reference exists.
- Cloud provider base URLs are validated to block SSRF to loopback/link-local/metadata hosts.
- Workspace boundaries are enforced even for approved actions.
- Telemetry and logs are redacted of credentials.
- See [SECURITY.md](SECURITY.md).

## Memory / Cache / Experience

These stay local by default (SQLite under `data/`). `EXPERIENCE_LOGGING`,
`MEMORY_ENABLED`, and `CACHE_ENABLED` control them. The cache is keyed with a
model signature so responses for one model/quantization are never replayed for
another.

## Project Context

`project_discovery` reports frameworks, languages, package manager, and a bounded
file tree for the selected workspace. Local knowledge, memory, and project index
enrich context before inference; an exact prompt+project match can answer from
cache without inference.

## Adaptive Routing

`AdaptiveHybridRouter` picks candidates by capability, cost, history, health,
context window, required tools, and task complexity. In `AUTO` mode, routine
coding/debugging work is local-first; complex, architectural, migration, vision,
and long-context work is cloud-first. `LOCAL_ONLY` and `CLOUD_ONLY` constrain the
pool explicitly.

## Verification

Approved project actions produce authoritative observations. Failed verification
is recorded; a new, explicitly approved action is required before code can
change again.

## Hardware Requirements

Local inference runs on CPU; GPU offload is optional. The quantization you
select should match your hardware:

| Quantization class | Typical use |
|--------------------|-------------|
| Smaller (`Q4_K_M`-class) | Low RAM/VRAM, CPU-only, or constrained devices |
| Larger (`UD-Q4_K_XL`/higher) | More RAM/VRAM, higher quality at slower speed |

Choose the quantization that fits your RAM, VRAM, GPU, CPU, target context length,
quality, and speed trade-offs.

## Model Selection

Different quantizations exist for the same model. There is no single "best"
quantization — it depends on your hardware and goals. Use the model card on
`unsloth/Qwen3.8-Flash-Next-GGUF` to pick the right one, and set
`LOCAL_MODEL_QUANT` and `LOCAL_MODEL_PATH` in `.env`.

## Development

```powershell
python -m pip install -r requirements.txt
python -m pytest -q
```

## Testing

The test suite (`tests/`) covers the gateway, providers, routing,
local intelligence, MCP tools, memory, storage, CLI, and security. Run with:

```powershell
python -m pytest
```

## Hugging Face Distribution

This repository is a sanitized **source distribution** of ZEVORA for
discoverability on Hugging Face. It does **not** include:

- model weights (GGUF files),
- runtime databases (`data/`),
- local caches, logs, or user workspaces,
- `.env` files or any credentials,
- virtual environments or build artifacts.

Only files appropriate for public distribution are published.

## GitHub Repository

Primary development happens on GitHub:

- **Repository:** <https://github.com/dani12po/zevora>
- **Issues / PRs:** please open them on GitHub.

## License

ZEVORA source code is licensed under the **MIT License** — see [LICENSE](LICENSE).
This repository mirrors the GitHub source of truth.

## Third-Party Models / Attribution

ZEVORA **references**, but does not redistribute or claim ownership of, the
following third-party assets:

- **Qwen3.8-Flash-Next** — model weights and its license remain with their
  respective owners (Qwen / Alibaba Cloud, and the upstream model card).
- **Unsloth GGUF distribution** — `unsloth/Qwen3.8-Flash-Next-GGUF` is owned by
  its author and redistributed by them under their posted license; we reference
  it, we do not mirror it.

Please refer to the upstream model cards for the exact model licensing terms.
We do not invent or re-license these models.

## Limitations

- Local inference quality and speed depend on your hardware and chosen quantization.
- Local context is bounded (`LOCAL_MODEL_CONTEXT_LENGTH`).
- Vision and very large/long-context or complex-architecture work is
  cloud-first and therefore requires a configured cloud provider.
- Custom runtime providers are **not** an OS sandbox: approved code retains the
  host user's filesystem and network privileges. Trust only code from known sources.