By Peter Evans, CEO, Xtract One Technologies
I came across an article recently about a weapons detection deployment at a hospital. The vendor was discussing their results, and the headline claim was this: across a single year, the system stopped more than 20 firearms and nearly 180 knives from entering the facility.
I understand why that number gets published. It sounds significant. It’s concrete. It gives a procurement team something to point to in a board presentation. But as someone who studied applied mathematics, I found myself doing what I always do when I see a number presented without context: I started looking for the denominator.
The physical security industry has a vanity metrics problem. The number of people per hour a system processes and the number of weapons detected are the numbers vendors lead with, and on the surface they seem meaningful. Another vendor story I came across recently claims to detect and stop 62 firearms daily. While an arresting figure, it’s also almost entirely without context, which makes it close to meaningless as a performance indicator.
Any mathematician will tell you that a single data point doesn’t define a trend. A single number without context doesn’t define an outcome. In basic fractions or complex quadratic equations, you need a series of numbers to determine anything useful. For a ratio to carry meaning, you need the numerator and the denominator. In this case, the numerator is the number of weapons detected. The denominator is the number of weapons that actually entered the facility, or perhaps the weapons demographic statistics of the population.
Without both, you have a marketing claim, not a performance measurement.
Let’s do some basic critical thinking with publicly available data.
In the first example, the deployment was in Illinois. Approximately 27% of people in Illinois live in a household with a firearm. Around 4% of the population holds a license to carry. Approximately 1.3% of the population carries that firearm with them at any given time.
If the average hospital entrance processes 500 people per day, that’s 182,500 entrants per year. At 1.3%, that’s approximately 2,372 firearms potentially entering that facility over the course of a year. The system detected 20. That’s less than 1% of all weapons that passed through.
I’m not sure that’s something worth bragging about.
If we apply a generous assumption, say 90% of individuals were sensible enough not to carry a firearm into a hospital, the denominator drops significantly. Even at that reduced figure, the capture rate sits below 10%. The headline number looks impressive until you ask the question it’s designed to avoid: out of how many?
The second example is even more striking. The claim of 62 firearms detected per day is presented as evidence of an effective security program. Let’s extend some grace and assume underreporting is a factor, and increase that figure to 600 firearms per day. The vendor in question reports scanning over 900 million people per year, which works out to approximately 2.5 million people per day. At the same 1.3% carry rate, 600 detected firearms against a population of 2.5 million represents a capture rate of approximately 0.02%.
At that rate, the security program is stopping fewer than one in five thousand of the weapons that statistically should be present in the screened population. And the marketing material is leading with the 600 claim.
This matters for a straightforward reason. The physical security industry is young. The buyers making procurement decisions are, through no fault of their own, working with limited frameworks for evaluating vendor claims. When a vendor publishes a detection number without a denominator, most buyers don’t have the reference point to know what question they should be asking next. That information asymmetry is convenient for vendors and dangerous for the facilities relying on the technology.
Vanity metrics have their place. Reduction in fine lines on a skincare product, Rotten Tomatoes scores, and box office revenue. In those contexts, the number creates a useful impression of performance and the stakes of misunderstanding it are low.
Physical security is not that context. When a hospital, a school, a sports venue, or a corporate facility deploys a weapons detection program, the number of weapons detected is only half the equation. The other half, the number of weapons that entered despite the program, is the one that determines whether the program is actually doing its job.
The industry has been comfortable leading with numerators because denominators are harder to calculate, harder to explain, and considerably less flattering. That comfort serves vendors. It doesn’t serve the security directors who are responsible for the safety of the people inside their facilities.
The right question to ask any vendor isn’t how many weapons their system detected but what percentage of the weapons present in a statistically representative population their system caught, at what sensitivity setting, under what operational conditions, with what secondary screening infrastructure in place.
If the answer comes back as another numerator without a denominator, you have what you need to know about how seriously that vendor takes the question of actual performance.
Measure what matters.