---
tags:
- multimodal
- NPU
- On-device
- Snapdragon PC
- Android
license: cc-by-4.0
license_name: nexa-research
license_link: LICENSE
pipeline_tag: any-to-any
base_model:
- NexaAI/OmniNeural-4B
---
# **OmniNeural** — World’s First NPU-aware Multimodal Model (Mobile Version)
## **Overview**
**OmniNeural** is the first fully multimodal model designed specifically for Neural Processing Units (NPUs). It natively understands **text, images, and audio**, and runs across PCs, mobile devices, automobile, IoT, and robotics.
## Demos
### 📱 Mobile Phone NPU - Demo on Samsung S25 Ultra
The first-ever fully local, multimodal, and conversational AI assistant that hears you and sees what you see, running **natively on Snapdragon NPU** for long battery life and low latency.
---
## **Key Features**
- **Multimodal Intelligence** – Processes **text, image, and audio** in a unified model for richer reasoning and perception.
- **NPU-Optimized Architecture** – Uses ReLU ops, sparse tensors, convolutional layers, and static graph execution for maximum throughput — **20% faster than non-NPU-aware models** .
- **Hardware-Aware Attention** – Attention patterns tuned for NPU, lowering compute and memory demand .
- **Native Static Graph** – Supports variable-length multimodal inputs with stable, predictable latency .
- **Performance Gains** – **9× faster audio processing** and **3.5× faster image processing** on NPUs compared to baseline encoders .
- **Privacy-First Inference** – All computation stays local: private, offline-capable, and cost-efficient.
---
## **Performance / Benchmarks**
### Human Evaluation (vs baselines)
- **Vision**: Wins/ties in ~75% of prompts against Apple Foundation, Gemma-3n-E4B, Qwen2.5-Omni-3B.
- **Audio**: Clear lead over baselines, much better than Gemma3n and Apple foundation model.
- **Text**: Matches or outperforms leading multimodal baselines.
### Nexa Attention Speedups
- **9× faster** audio encoding (vs Whisper encoder).
- **3.5× faster** image encoding (vs SigLIP encoder).
---
## **Architecture Overview**
OmniNeural’s design is tightly coupled with NPU hardware:
- **NPU-friendly ops** (ReLU > GELU/SILU).
- **Sparse + small tensor multiplications** for efficiency.
- **Convolutional layers** favored over linear for better NPU parallelization.
- **Hardware-aware attention** patterns to cut compute cost.
- **Static graph execution** for predictable latency.
---
## **Production Use Cases**
- **PC & Mobile** – On-device AI agents combine **voice, vision, and text** for natural, accurate responses.
- Examples: Summarize slides into an email (PC)*, *extract action items from chat (mobile).
- Benefits: Private, offline, battery-efficient.
- **Automotive** – In-car assistants handle **voice control, cabin safety, and environment awareness**.
- Examples: Detects risks (child unbuckled, pet left, loose objects) and road conditions (fog, construction).
- Benefits: Decisions run locally in milliseconds.
- **IoT & Robotics** – Multimodal sensing for **factories, AR/VR, drones, and robots**.
- Examples: Defect detection, technician overlays, hazard spotting mid-flight, natural robot interaction.
- Benefits: Works without network connectivity.
---
## How to use
Note this version is for mobile only (Android). See documentation for how to use:
[Quickstart](https://docs.nexa.ai/nexa-sdk-android/quickstart#run-your-first-model)
---
## Links & Community
[](https://discord.com/invite/nexa-ai)
[](https://x.com/nexa_ai)
[](https://nexa.ai)
- **Issues / Feedback:** Use the **HF Discussions** tab or submit an issue in our discord or nexa-sdk github.
- **Roadmap & updates:** Follow us on X and Discord.
> If you want to see more **NPU-first, multimodal** releases on HF, please give our model a like ❤️.
## Limitation
The current model is mainly optimized for English. We will optimize other language as the next step.
---
## **Citation**
```bibtex
@misc{
title={OmniNeural: World’s First NPU-aware Multimodal Model},
author={Nexa AI},
year={2025},
url={https://huggingface.co/NexaAI/OmniNeural-4B},
}
```
## License
This model is released under the **Creative Commons Attribution–NonCommercial 4.0 (CC BY-NC 4.0)** license.
Non-commercial use, modification, and redistribution are permitted with attribution.
For commercial licensing, please contact **dev@nexa.ai**.