---
license: apache-2.0
pipeline_tag: text-generation
language: en
tags:
  - lfm2
  - agent
  - fine-tune
  - tool-use
  - liquid-ai
base_model: LiquidAI/LFM2.5-1.2B-Thinking
datasets:
  - TeichAI/Ox-Alpha-Pi-Traces
metrics:
  - loss
---

# LiquidAgent-1.2B

A supervised fine-tune of [LiquidAI/LFM2.5-1.2B-Thinking](https://huggingface.co/LiquidAI/LFM2.5-1.2B-Thinking) on agent traces from [TeichAI/Ox-Alpha-Pi-Traces](https://huggingface.co/datasets/TeichAI/Ox-Alpha-Pi-Traces), with tool results properly included in the training data.

## What this is

The base LFM2.5-1.2B-Thinking model is a 1.17B-param hybrid architecture (10 LIV-conv + 6 GQA blocks, 32K context, 65536 vocab) with built-in thinking. This fine-tune adapts it for agentic tool-use: the model learns to emit tool calls, see the tool results, and reason about them in subsequent turns.

**Key fix over prior attempts:** The original parser for the TeichAI data silently dropped all `toolResult` messages (the `type` field is always `"message"` in the raw data, and tool results arrive as `message.role == "toolResult"` - a branch that did not exist). This fine-tune uses a fixed parser that recovers 100% of tool turns (verified: 562/562 on a 50-file sample).

## Training details

- **Base:** LiquidAI/LFM2.5-1.2B-Thinking (bfloat16)
- **Data:** 379 examples, ~1.34M assistant tokens (from Ox-Alpha-Pi-Traces, 400 session files, 397 usable)
- **Optimizer:** AdamW 8-bit, lr 2e-6 to 8e-6 (cosine warmup over 47 steps)
- **Hardware:** NVIDIA RTX 5090 (32 GB), gradient checkpointing, batch size 1
- **Duration:** 81 seconds (47 steps)
- **Final loss:** 0.5965 (step 40) - training completed at step 47

## Architecture

| Parameter | Value |
|-----------|-------|
| Params | 1,170,340,608 (tied embedding) |
| Layers | 16 (10 conv + 6 full attention) |
| Hidden size | 2048 |
| FF dim | 12288 |
| Attention heads | 32 Q / 8 KV |
| Vocab | 65,536 |
| Context | 32,768 (max_position_embeddings: 128,000) |
| Dtype | bfloat16 |

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "Compactbot/liquidagent-1.2b",
    torch_dtype="bfloat16",
    device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained("Compactbot/liquidagent-1.2b")

messages = [{"role": "user", "content": "Write a Python function to parse a JSON config file."}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=512, temperature=0.7, top_p=0.9)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
```

## Honest limitations

- Trained on only 379 examples (~1.3M tokens) - this is a very light adaptation, not a full agent training run.
- The model retains the base's thinking pattern (emits thinking blocks before responding).
- No independent benchmark evaluation was run; quality was verified by generation samples (coherent, on-topic, no degenerate repetition).
- The 32k sequence length requested by the requester was not used in this run (data was packed at shorter lengths to fit the 32GB GPU with 8-bit AdamW).