--- library_name: onnx pipeline_tag: text-classification license: apache-2.0 language: - en tags: - intentguard - guardrails - llm-safety - content-moderation - finance - deberta-v2 - onnx-runtime - intent-classification - chatbot-security model-index: - name: intentguard-finance results: - task: type: text-classification name: Intent Classification metrics: - name: Accuracy type: accuracy value: 99.6 - name: Legitimate Block Rate type: accuracy value: 0.0 - name: Off-Topic Pass Rate type: accuracy value: 0.0 --- # IntentGuard — Financial Services [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) [![Accuracy](https://img.shields.io/badge/accuracy-99.6%25-brightgreen.svg)](#performance) [![Size](https://img.shields.io/badge/model_size-2.5MB-orange.svg)](#model-details) [![Latency](https://img.shields.io/badge/p99_latency-<30ms_CPU-green.svg)](#performance) [![Format](https://img.shields.io/badge/format-ONNX_INT8-purple.svg)](#model-details) **Production-ready vertical intent classifier for LLM chatbot guardrails. Classifies user messages as `allow`, `deny`, or `abstain` to keep financial services chatbots on-topic and secure.** [Research Article](https://perfecxion.ai/articles/intentguard-vertical-intent-classifier-llm-guardrails.html) | [perfecXion.ai](https://perfecxion.ai) | [Finance Model](https://huggingface.co/perfecXion/intentguard-finance) | [Healthcare Model](https://huggingface.co/perfecXion/intentguard-healthcare) | [Legal Model](https://huggingface.co/perfecXion/intentguard-legal) --- ## IntentGuard Model Family IntentGuard provides specialized intent classifiers for high-stakes verticals where chatbot misuse carries regulatory, legal, or safety risk: | Model | Vertical | Accuracy | Off-Topic Pass Rate | Link | |-------|----------|----------|---------------------|------| | **intentguard-finance** | Financial Services | **99.6%** | 0.00% | This model | | **intentguard-healthcare** | Healthcare & Clinical | 98.9% | 0.98% | [perfecXion/intentguard-healthcare](https://huggingface.co/perfecXion/intentguard-healthcare) | | **intentguard-legal** | Legal & Compliance | 97.9% | 0.50% | [perfecXion/intentguard-legal](https://huggingface.co/perfecXion/intentguard-legal) | --- ## Overview ### The Problem Enterprise chatbots in regulated industries face a critical challenge: users inevitably ask off-topic questions (sports, entertainment, relationship advice) that the underlying LLM will happily answer — exposing the organization to compliance risk, brand damage, and potential liability. Traditional keyword filters miss nuanced off-topic queries, while LLM-based guardrails are too slow and expensive for real-time inference. ### The Solution IntentGuard uses a tiny, purpose-trained DeBERTa-v3-xsmall model (22M parameters, 2.5MB quantized) to classify user intent in <30ms on CPU. The three-way classification (`allow`/`deny`/`abstain`) enables precise control: - **Allow** — On-topic for the vertical, pass to the LLM - **Deny** — Clearly off-topic, block with a polite redirect - **Abstain** — Ambiguous, escalate to secondary classifier or human review --- ## Performance | Metric | Value | |--------|-------| | **Overall Accuracy** | 99.6% | | **Legitimate Block Rate** | 0.00% (no false positives) | | **Off-Topic Pass Rate** | 0.00% (no false negatives) | | **p99 Latency (CPU)** | <30ms | | **Model Size (ONNX INT8)** | 2.5MB | | **Base Parameters** | 22M (DeBERTa-v3-xsmall) | | **Expected Calibration Error** | <0.03 | ### Classification Decision Framework ``` User Message → Tokenize → DeBERTa Inference → Softmax ↓ ┌──────────────┼──────────────┐ │ │ │ ALLOW DENY ABSTAIN (on-topic) (off-topic) (uncertain) │ │ │ Pass to LLM Block + Redirect Escalate ``` --- ## Model Details | Property | Value | |----------|-------| | **Architecture** | DeBERTa-v3-xsmall (fine-tuned for 3-way classification) | | **Format** | ONNX (INT8 quantized) | | **Version** | 1.0 | | **Vertical** | Finance (Financial Services) | | **Training** | Supervised fine-tuning on curated intent datasets | | **Quantization** | INT8 via ONNX Runtime | | **GPU Required** | No — runs on CPU | | **Publisher** | [perfecXion.ai](https://perfecxion.ai) | ### Core Topics (Allow) Banking, lending, credit, payments, investing, insurance, tax, personal finance, retirement, mortgages, financial planning, budgeting ### Hard Exclusions (Deny) Sports, entertainment, cooking, gaming, celebrity gossip, fashion, travel/leisure, fiction writing, relationship advice --- ## Usage ### Python (ONNX Runtime) ```python import onnxruntime as ort from transformers import AutoTokenizer import numpy as np # Load model and tokenizer tokenizer = AutoTokenizer.from_pretrained("perfecXion/intentguard-finance") session = ort.InferenceSession("model.onnx") # Classify a user message text = "What are the current mortgage rates for a 30-year fixed loan?" inputs = tokenizer(text, return_tensors="np", max_length=128, truncation=True, padding="max_length") logits = session.run(None, { "input_ids": inputs["input_ids"], "attention_mask": inputs["attention_mask"] })[0] labels = ["allow", "deny", "abstain"] prediction = labels[np.argmax(logits)] confidence = float(np.max(np.exp(logits) / np.sum(np.exp(logits)))) print(f"Intent: {prediction} (confidence: {confidence:.3f})") # Output: Intent: allow (confidence: 0.998) ``` ### Docker ```bash # Pull and run the container docker pull ghcr.io/perfecxion/intentguard:finance-1.0 docker run -p 8080:8080 ghcr.io/perfecxion/intentguard:finance-1.0 # Classify a message curl -X POST http://localhost:8080/v1/classify \ -H "Content-Type: application/json" \ -d '{"messages": [{"role": "user", "content": "What are the current mortgage rates?"}]}' # Response: {"intent": "allow", "confidence": 0.998} ``` ### pip ```bash pip install intentguard # Python usage from intentguard import IntentGuard guard = IntentGuard.load("finance") result = guard.classify("What are the current mortgage rates?") print(result) # Intent(label='allow', confidence=0.998) ``` --- ## Example Classifications | User Message | Predicted | Confidence | Correct? | |-------------|-----------|------------|----------| | "What are mortgage rates for a 30-year fixed?" | allow | 0.998 | ✅ | | "How do I open a Roth IRA?" | allow | 0.997 | ✅ | | "Who won the Super Bowl?" | deny | 0.999 | ✅ | | "Tell me a joke" | deny | 0.996 | ✅ | | "Is my health insurance FSA-eligible?" | allow | 0.942 | ✅ (financial context) | | "What's the weather today?" | deny | 0.998 | ✅ | --- ## Citation ```bibtex @misc{thornton2025intentguard, title={IntentGuard: A Production-Grade Vertical Intent Classifier for LLM Guardrails}, author={Thornton, Scott}, year={2025}, publisher={perfecXion.ai}, url={https://perfecxion.ai/articles/intentguard-vertical-intent-classifier-llm-guardrails.html}, note={Model: https://huggingface.co/perfecXion/intentguard-finance} } ``` --- ## Quality Metrics | Metric | Result | |--------|--------| | Accuracy (Finance vertical) | 99.6% | | Legitimate Block Rate | 0.00% | | Off-Topic Pass Rate | 0.00% | | Expected Calibration Error | <0.03 | | ONNX INT8 Quantization | Validated | | CPU Inference (p99) | <30ms | | Docker Container | Available | --- ## License Apache 2.0 --- ## Links - **Research Article**: [IntentGuard: A Production-Grade Vertical Intent Classifier for LLM Guardrails](https://perfecxion.ai/articles/intentguard-vertical-intent-classifier-llm-guardrails.html) - **Publisher**: [perfecXion.ai](https://perfecxion.ai) - **Healthcare Model**: [perfecXion/intentguard-healthcare](https://huggingface.co/perfecXion/intentguard-healthcare) - **Legal Model**: [perfecXion/intentguard-legal](https://huggingface.co/perfecXion/intentguard-legal) - **Docker Image**: `ghcr.io/perfecxion/intentguard:finance-1.0`