Why most AI monitoring pilots fail.
They fail for the same reasons every time. Nobody wrote down what success looks like. There is no baseline, so there is nothing to compare against. The pilot runs on a quiet site where nothing happens. Guards are not told what the tool is for, so they ignore it. And at the end, the decision comes down to a vendor slide deck and a vague feeling.
A good pilot fixes all of that before the first shift. Here is the playbook.
Step 1: Define success metrics.
Pick a small set of metrics that match what your desk and your clients care about. Four cover most monitoring operations:
- Missed events. Real incidents or reportable activity the desk did not catch live. This is the metric clients care about most.
- Nuisance alerts. Alerts that needed no action: weather, animals, shadows, known vehicles, staff. Count them per camera per shift. Our guide on reducing nuisance alerts lists the usual causes.
- Response delay. Time from activity starting on camera to a guard acknowledging it, and to the right action being taken.
- Evidence review time. How long it takes to assemble the clip, stills and timeline for an incident report or client request.
Add one or two of your own if they matter, such as client complaints, patrol dispatches or guard workload at peak hours. Write down in advance what result would make you expand, and what would make you stop.
Step 2: Measure a baseline.
Before the AI goes live, run a week or two of normal shifts and measure the same metrics the same way. Use your existing incident log, alarm history and guard reports. For missed events, sample recorded video from a few cameras and check what happened versus what was logged; it is tedious, and it is the most revealing number you will get.
Keep the baseline honest. If your current desk misses things, that is exactly what the pilot needs to show.
Step 3: Pick one station.
Start with one monitoring station, not the whole operation. Choose the desk with:
- The most cameras per guard. That is where attention runs out first and where improvement shows fastest.
- Real activity. A site with nights, parking, perimeter and after-hours access gives the pilot something to measure. A sleepy lobby does not.
- Representative camera software. If your clients run a mix of platforms, include more than one. See adding AI to existing security cameras.
- Willing guards. Brief them. Explain that the tool assists and they still decide.
Before the first shift, confirm capture quality on the actual screens, the dashboard layouts the guards use, and the events your post orders care about.
Step 4: Run side by side.
Run the AI alongside normal operations, not instead of them. The guard keeps doing the job as usual, with the AI inbox as an additional input. That keeps the site safe and gives you a clean comparison.
Cover real shifts.
Include nights and weekends. Quiet daytime shifts tell you little about nuisance alerts or missed events.
Log every alert.
For each AI alert, record the guard’s outcome: acted on, reviewed and closed, or nuisance. The inbox should capture this as part of the workflow.
Note what the AI missed.
When a guard catches something the AI did not flag, log it. Misses are as important as hits.
Step 5: Review with evidence.
Hold a short weekly review with the guard lead and a supervisor. Walk through a sample of alerts with the original frames: the useful ones, the noisy ones and the misses. Tune what can be tuned, like camera masking or layouts, and note what cannot. Compare each metric against the baseline.
Ask the guards directly. Did the inbox pull their eyes to the right camera? Did the explanations help or get in the way? Were there cameras it covered well and cameras it struggled with? Guard feedback often explains the numbers.
Step 6: Decide.
At the end, go back to what you wrote in step 1 and make the call:
- Expand to more stations and sites if missed events, nuisance alerts, response delay or review time improved enough to matter.
- Extend briefly if results are promising but a specific issue, like one camera platform or night views, needs more data.
- Stop if it did not help. A clean no is a good pilot outcome too.
A pilot is a measurement, not a demo. Set the metrics first, measure the baseline, and let the numbers from your own desk make the decision.
Where Semaphor fits.
Semaphor is built to pilot this way. Because it works from the monitoring screen your guards already use, there is no camera replacement, no new VMS and no integration project to wait on. We set it up on one station, confirm capture quality and dashboard layouts, and it starts watching the same camera views your guards do.
Every alert lands in the guard inbox where guards acknowledge, review and close it with an outcome, which gives you the pilot log as a by-product of normal work. The evidence and audit trail keeps original frames, hashes and exportable manifests for every alert, so weekly reviews start from real evidence. Coverage gaps are explicit: a camera that cycled off screen is marked unobserved, so you know exactly what was and was not analyzed.
Ready to measure it on your own desk? Read the pilot program details, or book a live demo on your own camera screens first.
Pilot questions.
How long should an AI video monitoring pilot last?
Long enough to cover a representative range of shifts, including nights and weekends, after a baseline period. Several weeks is typical; the exact length depends on how much activity the site sees.
What metrics should we measure in a security AI pilot?
Missed events, nuisance alerts, response delay and evidence review time, compared against a baseline measured the same way before the pilot.
Do we need to change cameras or VMS for a pilot?
Not with Semaphor. It runs from the existing monitoring screen on one station.
Should the AI replace guard decisions during the pilot?
No. Run side by side. The guard keeps full control, and the pilot measures whether the AI helps them do the job better.