By Rolf Versluis · Published September 1, 2026 on Uncommon Knowledge in Business and Life

My farm is a business and I'm not always on it. I had trail cameras before this. Every time a branch blew in the breeze, I got a picture. After a few months of alerts, I stopped looking. That's what off-the-shelf products do for you.

I needed something that would actually tell me what was on the property, not just that something moved. The trail cameras would show me a deer at midnight or a shadow from a passing cloud. They couldn't tell me whether the motion was my truck or a stranger's truck, a deer passing through or a coyote testing the fence. Signal-to-noise too low. You start ignoring the alerts. The system becomes useless.

The project is open source at github.com/blockops1/ai_camera_monitor. MIT-licensed, runs on a Mac Mini or an old PC, the README has the install instructions. This post is about how the four-stage pipeline works and what I learned running vision models on motion events at volume.

I put a Reolink camera on the property. When motion triggers, it fires a webhook to the listener. The listener grabs four images from the camera's RTSP ring buffer. Those four images are the input.

The first thing I do is run YOLO on each image. YOLO is a small convolutional neural network that runs locally on the listener machine. I run it against every frame and ask one question: is there a vehicle, a person, or an animal in this picture, or is it just motion with nothing there? On a typical day the camera fires about 3000 motion events. YOLO drops the false positives, which means only a handful of events make it past the gate. The expensive model never gets called for events that don't matter. That keeps the system cheap to run 24 hours a day.

When YOLO says yes, I run a pairwise differential across the four images to figure out exactly where the motion is. That gives me two tight cropped boxes around the moving object. The crops, not the full frame, go forward.

Stage three is Qwen3-VL. Local, on port 8093. It gets the two cropped boxes and a prompt that depends on what YOLO classified the object as. If YOLO flagged vehicle, the prompt asks for make, model, color, body style. If YOLO said person, it asks for clothing, posture, distinguishing features. Animal: species, size, behavior. Structured output comes back.

What the LLM sees that YOLO cannot is the part that makes the matcher work. YOLO recognizes seven animal classes. The LLM recognizes hundreds, and it picks up individual patterns: the shape of an ear, the length of a snout, the color and pattern of fur. For vehicles it reads make, model, and color. The bent mirror on the passenger side. For people, YOLO sees "person." I see "adult in a green jacket, male, mid-50s, walking toward the gate." That detail is what makes the matcher work downstream.

I made the LLM authoritative. YOLO says vehicle, LLM says person, alert goes out as a person. YOLO says person, LLM says nothing visible, alert gets dropped. I run both models locally, and the hardware sits on the property. Qwen3-VL on the Mac Mini for vision, a small chat model on port 8081 for text formatting. The cheap classifier is fast and usually right, but I don't trust it when it matters. False positives don't get through because of that.

Stage four is a small text model on port 8081, same machine. It takes the structured output and writes the human-readable alert body. This is what makes the alert read like a person wrote it instead of "VEHICLE DETECTED AT 10:22." After that, the alert goes to Telegram through the bot API. The whole pipeline runs in about two seconds.

My first instinct is always to go find some open-source software and implement that. Agents are good at that.

For this surveillance problem, the leading open-source project out there is called Frigate. It does motion detection plus object classification well, and it'll tell you a vehicle is on the property. It won't tell you whether the vehicle is yours. It's a complete system with its own UI, its own dashboard, its own assumptions about what you want. Good product. Not built for the question I needed answered: is this mine, or not mine?

The missing piece was the matcher. For vehicles, the matcher scores the LLM output against enrolled vehicle signatures (make, model, color, distinguishing features). For people, it scores against enrolled clothing patterns and posture. For animals, it scores against species and individual markings. Three stores, three kinds of match logic.

That's the 5% that decides if this is mine: the prompts, the local run, the matcher. No vendor sells you that part. Too specific to a property, its cameras, its lighting, its definition of "known."

This works because I have the hardware, and the hardware's a Mac Mini with enough memory to run both models. The same architecture works with cloud models too; point the vision endpoint at a hosted API and the text endpoint at a hosted chat API. Prompts are written so they work either way. The README has the configuration. Trade-off is privacy and cost. Local means the property images stay on the property. Cloud means a hosted provider sees every frame. For mine it doesn't work.

It took a few months of iteration. At first, the LLM was getting slammed with motion alerts. It mistook shadows for trucks. The text formatter wrote "VEHICLE DETECTED" no matter what the LLM said.

I kept a log of the wrong cases. Each one pointed at a small fix. The YOLO confidence threshold needed adjusting. The differential crop had to hug the motion tighter. The enrolled store needed the truck added. The text formatter needed to pull the actual classification from the LLM output instead of using a hardcoded template. Each fix taught the system something the next alert would benefit from.

After a few months of operating, the false positives were gone. The system identified what was at the property, took a picture, and sent it to me.

Pattern is the same one I keep finding with AI agents in general. The 95% is the open-source pieces. The 5% is the design calls that turn the pieces into your system. Design calls only get made by someone who knows what they want the system to do.

It does what a security guard does. Watches everything, only calls when something matters. I can't afford to hire a person for that. I've got an agent that wrote a program to do it.

But now when something moves out there, I know what it is. That's the difference.

The code is at github.com/blockops1/ai_camera_monitor. My address is localhermesagent.com.