What a vision-language model is.
A vision-language model, or VLM, is an AI model that takes images and text as input and answers in text. Show it a few frames from a camera and ask “what is happening here?” and it writes a description: two people standing by a car at the edge of the lot, one of them looking into the driver’s window, the lot otherwise empty. It is the same family of technology as the chat assistants people use every day, trained to reason about pictures as well as words.
That is a different job from object detection. A detector is fast and narrow: it finds people and vehicles and draws boxes. A VLM is slower and broad: it reads the whole scene, including things nobody trained a detector class for.
What VLMs add to security monitoring.
Scene reports
The biggest win is a short written report attached to each alert. Instead of “person detected, camera 7,” the guard reads what is happening, what the model noticed and what it recommends. That report is the difference between opening a clip cold and opening it already knowing where to look.
Context detectors cannot see
A detector sees a person. A VLM can notice that the person is carrying a ladder, standing at a closed gate, wearing a hi-vis vest, or lying on the floor rather than sitting. It can tell a delivery driver at a dock from someone pulling on a locked door, at least some of the time. Site instructions can steer it: “people near the service entrance after closing should be flagged.”
A recommendation, not a verdict
A good VLM output ends with a suggestion such as “guard review suggested” or “no escalation,” and a plain statement of what remains uncertain. That feeds triage and priority levels so the most concerning events rise to the top of the queue.
Where VLMs go wrong.
VLMs are impressive and fallible, and anyone selling them for security should say so plainly.
- Confident mistakes. A model can describe something that is not there, or miss something that is, in fluent and convincing language.
- Intent is a guess. “Appears to be trying the door handle” may be right, or may be someone reaching for a phone. Pictures do not show intent.
- Few frames, limited view. A model reviewing a handful of stills sees less than a guard scrubbing through the clip.
- Night and distance. Grainy IR footage and tiny figures degrade the description just as they degrade detection.
- Latency. Reasoning over images takes seconds, not milliseconds. A system that waits for the VLM before alerting makes guards wait too.
Why the evidence stays beside the assessment.
Because VLMs can be wrong in convincing ways, the model’s words must never replace the pictures. The right design puts the original frames right next to the AI report, so the guard can check the claim against the evidence in one glance. If the report says “person at the gate” and the frame shows a shadow, the guard sees that instantly and closes it.
That same principle protects investigations. When a clip ends up in an incident report, a client email or a police request, what matters is what the camera actually recorded, not what a model said about it. Semaphor keeps the original image for every processed frame and crop, with an exact SHA-256 hash and perceptual hashes, so evidence can be shown to be unaltered. See incident report video evidence.
The AI writes the first draft of what happened. The guard, looking at the original evidence, writes the decision.
Alert first, reasoning second.
Semaphor separates the fast part from the slow part. When local detection finds activity, a preliminary alert publishes immediately, measured in our backend tests at well under a second. The VLM review then updates the same card when it is done. Guards never wait on the AI to see that something moved, and the scene report arrives as added context rather than a gate. That is the alert-first review design, and it is why VLM latency does not become response latency.
Where VLMs run.
VLMs are larger than detectors and need a capable GPU. They can run locally on a workstation at the monitoring station or on hosted inference. Semaphor uses models from the Qwen VL family and supports both: local, or hosted over verified HTTPS with authentication and optional mutual TLS. Selected camera images only go to a remote model endpoint if one is explicitly configured. The trade-offs are covered in edge vs cloud video analytics.
Where Semaphor fits.
Semaphor uses a vision-language model to write AI scene reports for each alert: what is happening, observations, a recommendation and what remains uncertain, beside the original evidence on the same card. It runs from the monitoring screen your guards already watch, with no camera replacement and no VMS integration. The guard stays in control, as described in human-in-the-loop security AI.
Read what Semaphor is, watch the recorded demo with saved scene reports on the home page, then book a live demo on your own camera screens or start a pilot. VLM and related terms are in the glossary.
Frequently asked questions.
What is a vision-language model?
An AI model that reads images and text together and answers in text. On security cameras it describes what is happening in a scene, beyond just locating people and vehicles.
Can a vision-language model decide whether to call the police?
It should not. It can recommend review or escalation, but it can be wrong about intent. The guard makes the call, with the original evidence in front of them.
Are vision-language models too slow for live monitoring?
On their own they take seconds per review. That is why Semaphor alerts first from fast local detection and adds the VLM report to the same card when it is ready.
Do vision-language models replace object detection?
No. Detection is fast and runs on every frame; VLMs are slower and add context to selected events. The two work together.