A camera archive you can talk to.
Finding the right footage across hundreds of cameras and months of retention is a large manual task. Surveillance Broker turns the archive into something you can browse on a true wall-clock timeline, search by what actually happened, and simply ask — and it opens the exact camera and time in one step. And because it ties the cameras to your access control, inventory, and production systems, you can ask questions about your operation, not just your footage.
Surveillance Broker sits on top of Frigate and adds the local-AI retrieval, memory, and conversational control that camera systems normally lack. It runs in production today across a live archive of thousands of recordings from hundreds of cameras.
Once the local AI understands what is in the footage — and the footage is tied to the other systems in the building — the archive stops being a pile of video and becomes a way to answer operational questions.
Surveillance Broker integrates with the access control system. When someone swipes a card on camera, the system ties that swipe to the AI-recognized person — turning a visual match into a positive, confirmed identity. Whole-person recognition then carries that identity across cameras, even where there is no reader.
Because the local models can watch continuously, they can observe specific processes and behaviors, not just who was where: whether products are being packaged correctly, how much time goes into maintaining a machine, whether product is being taken, how inventory went missing. For finer, facility-specific activities that need to be understood precisely, we fine-tune the models with LoRAs.
Surveillance Broker also connects to operational systems — inventory, manufacturing, and the like. A single question can cross from the video into the data those systems hold and back, so the answer draws on everything the facility recorded — not just the cameras.
The result is questions no camera system could answer on its own:
Surveillance Broker is two cooperating systems — a local system at the site and a cloud system for centralized access — sharing one domain implementation across every interface.
We install Frigate — the widely used, open-source, industry-standard NVR — to handle live cameras, recording, review, and timeline. Frigate brings its full standard toolkit, including object detection, facial recognition, and license-plate reading. Surveillance Broker takes it much further: the goal is to let a user find nearly anything happening in their facility that the cameras captured, whether or not it fits a predefined detector.
Everything runs on locally installed AI models. Running the vision-language models on-site is what makes that possible economically — we can sample tens of thousands of frames a day without paying per-token cloud inference costs, the difference between analyzing a facility continuously and analyzing it occasionally. Recordings are decoded once into a clean archive and sampled for analysis in the same pass; lightweight detection runs first, then the local VLM describes the people, activity, objects, and visible text in each scene.
Person tracking is central, and built for real footage. Facial recognition needs a clear, well-angled shot of a face — often impossible on real security video. Our AI-based person recognition uses the whole person instead: build, clothing, what they are carrying, and what they are doing. That lets Surveillance Broker follow an individual through a space and across cameras even when a face is never clearly visible. Every observation becomes a searchable record, embedding, and knowledge-graph entry, kept alongside the camera, time, and source frames it came from.
The cloud system receives authenticated archive uploads and catalogs them immutably. It serves the timeline, playback, and export experience, and runs hybrid search — relational, full-text, vector, and knowledge-graph retrieval combined and reranked — behind the conversational agent. Every answer carries stable references that become viewer deep links and export boundaries.
Long-running jobs — import, processing, enrichment, upload, reconciliation — use durable checkpoints and idempotent operations, so nothing is silently lost and a repeated upload is always safe. A single source of truth holds the authoritative catalog; the knowledge graph is a rebuildable projection on top of it. And there is one implementation behind many front doors: the same operations serve the web UI, scheduled jobs, and other authorized agents over a common tool interface.
