Memories-ai/security_model

Model Card for Memories-S0

Memories-S0 is a highly efficient, 3-billion-parameter video understanding model designed specifically for the security and surveillance domain. It leverages synthetic data generation (via Veo 3) and extreme optimization strategies to achieve state-of-the-art performance on edge devices.

Model Details

• Model Name: Memories-S0
• Organization: Memories.ai Research
• Model Architecture: 3B Parameter VideoLLM
• Release Date: Jan 2026
• License: Apache 2.0
• Paper: Memories-SO: An Efficient and Accurate Framework for Security Video Understanding
• Code Repository: https://github.com/Memories-ai-labs/memories-s0

Model Description

Memories-S0 is designed to address two key challenges in security video understanding: data scarcity and deployment efficiency on resource-constrained devices.

• Data Innovation: The model is pre-trained on a massive, diverse set of synthetic surveillance videos generated by advanced video generation models (like Veo 3). This allows for pixel-perfect annotations and covers diverse scenarios (e.g., dimly lit hallways, unattended packages).
• Extreme Efficiency: It utilizes an innovative input token compression algorithm that dynamically prunes redundant background tokens, focusing computation on foreground objects and motion. This allows the 3B model to run efficiently on mobile/edge hardware.
• Post-Training: The model employs a unique post-training strategy using Reinforcement Learning (RL) and event-based temporal shuffling to enhance sequential understanding without expensive full fine-tuning.

Installation

conda create -n memories-s0 python=3.10 -y
conda activate memories-s0

# Install PyTorch with CUDA support
pip install torch torchvision torchaudio --index-url <https://download.pytorch.org/whl/cu121>

# Install dependencies for Qwen2.5-VL architecture and Flash Attention
pip install transformers>=4.37.0 accelerate qwen_vl_utils
pip install flash-attn --no-build-isolation

Inference

The following script demonstrates how to run the Memories-S0 model. It automatically handles the loading of weights from the official Hugging Face repository.

import torch
import argparse
from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor
from qwen_vl_utils import process_vision_info

# Official Model Repository
MODEL_ID = "Memories-ai/security_model"

def run_inference(video_path, model_id=MODEL_ID):
    # Load Model with Flash Attention 2 for efficiency
    model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
        model_id,
        torch_dtype=torch.bfloat16,
        attn_implementation="flash_attention_2",
        device_map="auto",
    )

    processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)

    # Define Security Analysis Prompt
    prompt_text = """YOUR_PROMPT"""

    messages = [
        {
            "role": "user",
            "content": [
                {"type": "video", "video": video_path},
                {"type": "text", "text": prompt_text},
            ],
        }
    ]

    # Preprocessing
    text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
    image_inputs, video_inputs, video_kwargs = process_vision_info(messages, return_video_kwargs=True)

    inputs = processor(
        text=[text],
        images=image_inputs,
        videos=video_inputs,
        padding=True,
        return_tensors="pt",
        **video_kwargs,
    )
    inputs = inputs.to("cuda")

    # Generate
    generated_ids = model.generate(**inputs, max_new_tokens=768)
    generated_ids_trimmed = [
        out_ids[len(in_ids):] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
    ]
    output_text = processor.batch_decode(
        generated_ids_trimmed, skip_special_tokens=True, clean_up_tokenization_spaces=False
    )

    print(output_text[0])

if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument("--video_path", type=str, required=True, help="Path to input video")
    args = parser.parse_args()
    run_inference(args.video_path)

Intended Use

Primary Use Cases

• Security & Surveillance: Detecting anomalies, tracking suspicious activities, and monitoring public safety.
• Smart Home Monitoring: Analyzing video feeds for unusual events (e.g., falls, intruders) as benchmarked on SmartHomeBench.
• Edge Computing: Deploying high-performance video analysis directly on cameras or local gateways with limited memory and compute power.

Out-of-Scope Use Cases

• General open-domain video understanding (e.g., movie classification) may not be optimal as the model is specialized for surveillance angles and events.
• Biometric identification (Face Recognition) is not the primary design goal; the focus is on action and event understanding.

Performance (SmartHomeBench)

We evaluated Memories-S0(3B) on the SmartHomeBench dataset, a recognized benchmark for smart home video anomaly detection.

Despite having only 3B parameters, our model achieves an F1-score of 79.21 using a simple Zero-shot prompt, surpassing larger models like VILA-13b and performing competitively against GPT-4o and Claude-3.5-Sonnet (which require complex Chain-of-Thought prompting).

Model	Params	Prompting Method	Accuracy	Precision	Recall	F1-score
Memories-S0 (Ours)	3B	Zero-shot	71.33	73.04	86.51	79.21
VILA-13b	13B	Few-shot CoT	67.17	69.18	70.57	69.87
GPT-4o	Closed	Zero-shot	68.41	80.09	55.16	65.33
Gemini-1.5-Pro	Closed	Zero-shot	57.36	84.34	25.73	39.43

Citation

If you use this model or framework in your research, please cite our technical report:

@techreport{memories_s0_2025,
  title       = {{Memories-S0}: An Efficient and Accurate Framework for Security Video Understanding},
  author      = {{Memories.ai Research}},
  institution = {Memories.ai},
  year        = {2025},
  month       = oct,
  url         = {https://huggingface.co/Memories-ai/security_model},
  note        = {Accessed: 2025-11-20}
}