Miril-Drone-2B-1

An open-weight aerial vision-language model for civilian drones

**Drones can talk! **

Enabling drones to reason and talk about their environment.

Miril-Drone-2B-1 is a 2B-class aerial VLM by Miril.ai for drone-view imagery. Give it an overhead frame and a clear instruction, and it returns a structured JSON response: a scene caption, a direct visual answer, or a rough operational point for review.

Civilian drones are becoming everyday infrastructure: delivery, first response, agriculture, construction, inspection, mapping, disaster recovery, and infrastructure monitoring. Those workflows need more than video. They need models that can explain what is in the frame, answer questions about the ground below, and produce simple machine-readable outputs that operators and downstream software can inspect.

This is V1 of Miril-Drone. It is built to show that small edge-oriented VLMs can make aerial systems easier to supervise, scale, and make safer.

This is what it does!

If the embedded player does not appear, open the demo compilation video directly.

Disclaimer: this is not (yet) a safety system!

This is mostly just a demo for now: it's a very small model trained on a sub-sampling of the WALDO dataset. It's interesting to show because it points in the direction of what drones will be able to do in the future, but this should by no means be considered a production-ready safety system. We will get there, but this is just pointing in that direction ;) It's also still plain wrong quite a lot! The model is based on Gemma4-2B and while the vision backbone is surprisingly strong for a model so small it's far from the perception capabilities we would need to make this perform perfectly on generalized use cases. Right now for instance small things in drone-imagery like people seen from a distance are lost most of the time, and the VLM only recovers them at lower altitudes.

TL;DR

Miril-Drone-2B-1 is an open-weight aerial VLM for:

• drone-view scene descriptions,
• aerial visual question answering,
• visible object and ground-context review,
• delivery-zone and landing-area screening,
• first-response and inspection triage,
• structured JSON outputs for operator tools and autonomy workflows.

The model accepts an image plus a visible text prompt. It does not receive hidden object counts, bounding boxes, or schema metadata at inference time. If you want JSON, put the JSON requirement in the prompt.

Architecture note: the exported checkpoint inherits Gemma4 multimodal components, including audio and video tokens/processors/towers. They are kept so the checkpoint remains a faithful merged Gemma4 model. This Miril-Drone release, however, was trained and validated for drone-view image-plus-text use. Audio input and native temporal video understanding are not part of the current release contract; the demo video path samples still frames and runs image inference on those frames.

Try the demo Space:

https://huggingface.co/spaces/MirilAI/mirilai-miril-drone-2b-1

The V1 Pointing Contract

V1 supports operational pointing with rough representative grid cues.

That wording matters. Coordinates are image-relative values in a 0..1000 frame, and many V1 pointing outputs should be read as a coarse review cue, not as a certified landing point, delivery command, or closed-loop control target.

Use V1 pointing for:

• asking where a human should look first,
• highlighting a candidate open area,
• screening an image before another planner or operator takes over,
• feeding a safety-aware review interface.

Do not use V1 pointing as:

• an autonomous flight command,
• a guaranteed safe landing point,
• a centimeter-level or pixel-perfect localization system.

Miril-DroneVLM-2B-2 is the successor. It adds plain-English routing across four typed JSON responses, separates landing and delivery selection from object pointing, and makes precise, coarse-direction, and no-target outcomes explicit.

What It Can Do

Use Miril-Drone-2B-1 when you want a small aerial model to turn a drone frame into a usable answer:

• "Describe the drone image."
• "How many cars are visible?"
• "Are people clearly visible?"
• "What areas should be avoided for landing or dropping something?"
• "What would be the safest place to land a drone in this image?"

The answer is returned as bare JSON so it can be consumed by dashboards, review tools, notebooks, routing systems, or VLA-style autonomy stacks.

WALDO Lineage

Miril-Drone-2B-1 is not named WALDO, but it inherits aerial vocabulary from the WALDO perception line.

WALDO30 is an overhead object detector for drone-relevant imagery. Its class vocabulary shaped the way this model talks about aerial scenes:

WALDO class	What it covers in plain English
LightVehicle	Cars, sedans, rickshaws, small personal vehicles, vans, and similar light road vehicles.
Person	People and human figures visible from above.
Building	Buildings, sheds, shacks, pergolas, canopies, tents, and other roofed structures.
UPole	Utility poles, telegraph poles, power-line fixtures, lamp posts, street fixtures, satellite dishes, antennas, and thin truss-like structures.
Boat	Boats, ships, canoes, kayaks, surfboards, and watercraft-like objects.
Bike	Bicycles, motorbikes, mopeds, scooters, and other two-wheeled vehicles.
Container	Storage containers, including shipping containers.
Truck	Trucks, transport vehicles, articulated trucks, semi trucks, flatbeds, and larger industrial road vehicles.
Gastank	Gas tanks, storage tanks, liquid tanks, cylindrical storage containers, and silo-like tanks.
Digger	Diggers, tractors, excavators, and construction vehicles.
Solarpanels	Solar panel arrays and individual panel groups.
Bus	Buses and bus-like passenger vehicles.

The VLM does not output detector boxes by default. It uses this aerial vocabulary to answer natural drone-view questions.

Prompting

The model was trained for image-plus-instruction calls. Keep prompts plain and close to the task. Do not add brand language or lineage language to the prompt; keep that context in documentation.

Use this system prompt for JSON-producing calls:

You are a careful visual assistant for drone imagery. Answer only with valid bare JSON. Do not use Markdown, prose outside JSON, or unrequested keys.

Then send the image with one of these user prompt families.

caption_v1

Use this for a compact whole-frame description.

Describe the drone image.

Required response format: return only valid bare JSON with exactly one key, {"caption": string}. Do not use Markdown or prose outside the JSON object.

Expected output:

{"caption": "Overhead view of a residential block with roads, parked vehicles, buildings, trees, and open ground areas."}

simple_answer_v1

Use this for visible objects, counts, scene type, locations, hazards, and operator questions.

How many light vehicles are visible in the image?

Required response format: return only valid bare JSON with exactly one key, {"answer": string}. Do not use Markdown or prose outside the JSON object.

Expected output:

{"answer": "Several light vehicles are visible along the road and in parking areas."}

operational_coordinate_v2

Use this for rough landing, delivery, inspection, or review cues. Bear in mind the output is rough coordinates and is only meant as an indication for now. The next generation will be able to point more accurately, and we are training larger versions of this model too which will be more accurate! This is foremost meant to showcase how a drone could reason by itself about things like landing spots for commercial drone-delivery applications!

What would be the safest place to land a drone in this image?

Required response format: return only valid bare JSON with exactly these keys: "coordinate_system", "description", "point_2d", "point_semantics", "status", "x", and "y". "description", "point_semantics", and "status" must be strings. "coordinate_system" must be "gemma_relative_0_1000_yx" or null. "point_2d" must be [y, x] with numbers from 0 to 1000, or null. "x" and "y" must be numbers from 0 to 1000, or null. "point_semantics" must be one of "specific_point", "representative_point", or "not_applicable". "status" must be one of "target_found", "conditional_candidate", or "no_safe_area". If you describe any candidate landing, delivery, or review area, you must point to it with approximate coordinates: use "coordinate_system": "gemma_relative_0_1000_yx", numeric "x" and "y", "point_2d": [y, x], and "point_semantics": "representative_point" unless the prompt asks for a specific object point. Use "target_found" for a clearly usable target and "conditional_candidate" for a plausible but uncertain target. Use "no_safe_area" only when no meaningful target area can be pointed to; then set "coordinate_system", "point_2d", "x", and "y" to null and set "point_semantics" to "not_applicable". Never omit required keys. Candidate skeleton: {"coordinate_system": "gemma_relative_0_1000_yx", "description": string, "point_2d": [500, 500], "point_semantics": "representative_point", "status": "conditional_candidate", "x": 500, "y": 500}. No-safe skeleton: {"coordinate_system": null, "description": string, "point_2d": null, "point_semantics": "not_applicable", "status": "no_safe_area", "x": null, "y": null}. Do not use Markdown or prose outside the JSON object.

Coordinate interpretation:

• x=0 is the left edge of the image.
• x=1000 is the right edge.
• y=0 is the top edge.
• y=1000 is the bottom edge.
• point_2d is [y, x].
• x and y duplicate the coordinate for easier downstream parsing.

Drawable V1 example:

{
  "coordinate_system": "gemma_relative_0_1000_yx",
  "description": "Open grass near the right side of the frame, away from visible roofs, trees, vehicles, and roads.",
  "point_2d": [500, 750],
  "point_semantics": "representative_point",
  "status": "target_found",
  "x": 750,
  "y": 500
}

No-area example:

{
  "coordinate_system": null,
  "description": "No clearly safe landing area is visible because roads, roofs, trees, and constrained spaces dominate the frame.",
  "point_2d": null,
  "point_semantics": "not_applicable",
  "status": "no_safe_area",
  "x": null,
  "y": null
}

Example Outputs

Scene Explanation

Prompt:

Describe the drone image.

Output:

{"caption": "Overhead view of a residential area with pitched-roof buildings, parked cars, narrow roads, garden plots, trees, and several open grassy sections."}

Visual Question Answering

Prompt:

What areas should be avoided for landing or dropping something in this scene?

Output:

{"answer": "Avoid rooftops, roads, parked vehicles, tree cover, utility fixtures, narrow yards, and areas close to buildings or fences."}

Operational Pointing

Prompt:

What would be the safest place to land a drone in this image?

Output:

{
  "coordinate_system": "gemma_relative_0_1000_yx",
  "description": "A relatively open grassy area away from the main road, buildings, parked cars, and dense tree cover.",
  "point_2d": [750, 500],
  "point_semantics": "representative_point",
  "status": "conditional_candidate",
  "x": 500,
  "y": 750
}

Read this as a rough review cue. A downstream UI may draw a marker there, but a real drone system still needs separate safety checks, geofencing, obstacle sensing, map context, and human or certified autonomy approval.

Loader Compatibility

Use transformers>=5.12.1. Gemma 4 E2B declares 35 language layers and 20 shared-KV layers. Layers 15 through 34 reuse key/value states from the non-shared stack, so they intentionally do not serialize separate k_proj, v_proj, or k_norm tensors. The checkpoint contains k_norm for language layers 0 through 14 and all 16 vision layers, exactly as its configuration requires.

We verified the published export with output_loading_info=True under Transformers 5.12.1: it reports zero missing, unexpected, or mismatched keys. A loader that reports missing k_norm tensors for the shared layers is not applying Gemma 4's shared-KV configuration. Update the loader rather than creating or copying tensors.

Quickstart

python -m pip install "transformers>=5.12.1" accelerate pillow torch

from PIL import Image
import torch
from transformers import AutoProcessor, AutoModelForImageTextToText

model_id = "MirilAI/Miril-Drone-2B-1"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

image = Image.open("drone_frame.jpg").convert("RGB")
prompt = '''Describe the drone image.

Required response format: return only valid bare JSON with exactly one key, {"caption": string}. Do not use Markdown or prose outside the JSON object.'''

messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "image": image},
            {"type": "text", "text": prompt},
        ],
    }
]

rendered = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=[rendered], images=[[image]], return_tensors="pt", padding=True).to(model.device)

with torch.no_grad():
    output = model.generate(**inputs, max_new_tokens=192, do_sample=False)

generated = output[:, inputs["input_ids"].shape[-1]:]
print(processor.batch_decode(generated, skip_special_tokens=True)[0])

Current-Contract Evaluation

Miril-Drone-2B-1 is evaluated on the same held-out aerial benchmark as Miril-DroneVLM-2B-2 and every deployment variant. The benchmark covers whole-scene captioning, factual visual answers, landing selection, delivery selection, object location, visual pointing, and tracker initialization. It measures strict JSON, schema validity, current-contract routing, text overlap, spatial status and mode agreement, coordinate recovery, coarse-direction agreement, and no-target coordinate discipline.

V1 predates the four-type plain-English router used by V2. Its native prompting contract remains the explicit schema-guided interface documented above. The shared benchmark intentionally measures how V1 behaves when asked to satisfy the newer contract, so route and schema scores should be read as cross-generation compatibility metrics rather than a rewrite of V1's documented interface.

Private staging notice: complete same-case results are still being generated. Publication tooling must reject this card while MIRIL_RELEASE_METRICS_PENDING remains.

MIRIL_RELEASE_METRICS_PENDING

Automated scores are regression signals, not physical-world safety evidence.

Variants

Related deployment variants (derived from this model):

• Miril-Drone-2B-1-bnb4: CUDA 4-bit bitsandbytes path.
• Miril-Drone-2B-1-bnb8: CUDA 8-bit bitsandbytes path.
• Miril-Drone-2B-1-MLX-4bit: Apple Silicon MLX 4-bit path.
• Miril-Drone-2B-1-MLX-8bit: Apple Silicon MLX 8-bit path.

Safety Notes

Miril-Drone-2B-1 is a perception and reasoning model. It is not a flight controller, autopilot, certified safety system, or substitute for operational validation.

Use it as one component in a larger stack:

• keep a human or validated planner in the loop,
• independently check obstacles, terrain, people, vehicles, trees, wires, and weather,
• enforce geofencing and vehicle limits,
• log model outputs and failures,
• treat invalid JSON, null coordinates, and no_safe_area as no-target outcomes.

The model can be wrong. It can miss small objects, confuse scale, overstate open areas, or produce a coordinate that only makes sense as a rough review marker.

Intended Use

Good fits:

• aerial image review,
• drone operator assistance,
• field inspection triage,
• first-response situational awareness,
• delivery or landing-area pre-screening,
• research on aerial VLMs and edge-oriented perception,
• structured-output demos for physical AI workflows.

Not good fits:

• direct closed-loop flight control,
• safety-critical operation without independent validation,
• surveillance or identification of people,
• claims of certified landing-zone selection.

License

Apache License 2.0. See LICENSE and NOTICE.

This model is derived from google/gemma-4-E2B-it; users should also review the upstream base model terms.