A crowd is not a data point

When artificial intelligence enters a control room, the challenge is no longer seeing. It is understanding.

And this, in my view, is where it becomes interesting.

Every new technology in security brings the same temptation: if we can collect more data, we can make better decisions.

It sounds logical.

But it does not necessarily follow.

At a major event today, we can have cameras, access control, sensors, drones, communications systems, transport data, information about flows, emergency systems and algorithms capable of analysing a vast amount of this in real time.

For anyone working in a control room, the ambition is clear: a complete picture of what is happening.

The problem is that a picture, however detailed, is not yet understanding.

Above all, an algorithm flagging an anomaly does not automatically mean it has understood what is happening.

The paradigm shift

For a long time, technology in security followed a fairly simple logic:

observe → record → report → intervene.

Video surveillance multiplied the eyes available on the ground. But someone still had to watch the screens and decide whether what they were seeing mattered.

We are now entering a different model:

observe → analyse → interpret → predict → suggest an intervention.

This is an enormous change.

Computer vision, machine learning and flow analysis systems can identify sudden changes in density, congestion, changes in crowd movement and other anomalies.

That is an extraordinary opportunity.

But there is one detail we risk forgetting.

Between the data and the decision, there is still a person.

That person needs to understand whether the data actually mean something.

The problem of a false emergency

Imagine a concert attended by 50,000 people.

An algorithm detects a rapid increase in density in a particular area.

Alert.

The control room sees it.

But what is happening?

It could be the beginning of a dangerous situation.

Or it could simply be thousands of people arriving at the same time when the gates open.

It could result from a change of direction after a rumour spreads that Belén is at Entrance 2.

It could be the end of the show, with thousands of people heading for the exits.

Or, indeed, it could be the beginning of a serious problem.

The same data can describe completely different situations.

This is precisely where, in my view, we need to be careful.

Artificial intelligence can be very good at recognising changes, correlations and anomalies.

Security needs to understand the context.

These are different tasks.

A crowd is not a motorway

Road traffic can be described reasonably well using a few parameters: speed, density, direction and flow.

People are much more complicated.

People are not particles.

They feel fear, curiosity and urgency.

They have friends to meet, children to protect and someone they are waiting for.

They receive information from their phones, loudspeakers, security personnel and, above all, the people around them.

And they react to our decisions.

This point is fundamental.

If I close an exit because an algorithm has detected congestion, I may solve the problem.

But I may also direct thousands of people towards another exit and create an even worse bottleneck.

This is a classic security problem: an intervention changes the system we are trying to control.

That is why crowd management cannot be reduced to finding the familiar magic number: “people per square metre”.

The number matters.

But the number alone does not tell the story.

The real revolution is decision quality

This also changes how we should assess security technology.

The question should not be:

“How intelligent is this system?”

We should ask:

“How much does it improve the quality of our decisions?”

These are very different questions.

A system can be technically extraordinary yet have limited operational value.

Conversely, a relatively simple system can be extremely effective if it gives the operator the right information, at the right time and in the right way.

A control room does not necessarily need to know everything.

It needs to know what matters.

The temptation of the dashboard

There is another risk, perhaps even more subtle.

I would call it the dashboard illusion.

A large wall of monitors, maps, charts, percentages, flows and camera counts.

At some point, we feel that everything is under control.

But having a lot of information does not mean having situational awareness.

A control room can be rich in technology and poor in awareness of the situation at the same time.

Situational awareness means more than:

“I know what is happening.”

It means:

“I know what is happening, I understand why it is happening, I can imagine what might happen next, and I know what I can do to influence how it develops.”

This is where, paradoxically, the Security Manager’s role becomes even more important.

AI should advise, not command

This does not mean we should fear artificial intelligence.

Quite the opposite.

At major events, it can greatly increase operators’ capabilities.

It can filter thousands of pieces of information.

It can identify anomalies.

It can connect apparently separate events.

It can highlight a trend before it becomes visible to the human eye.

It can help develop scenarios.

But there is a fundamental difference between supporting a decision and making one.

The first is intelligence.

The second is command.

Confusing them would be an organisational error before it was a technological one.

The legislation also has something to tell us

The issue is, of course, more than operational.

In Europe, it has also become a legal issue.

The AI Act treats real-time remote biometric identification in publicly accessible spaces for law enforcement purposes with particular caution. The general principle is prohibition, with limited exceptions subject to specific conditions.

This is more than bureaucracy.

It recognises a practical problem: the more powerful a technology becomes at recognising and classifying people, the greater the risk of disproportionate use.

At a major event, this becomes even more sensitive.

For a few hours or days, public space becomes an environment with an enormous concentration of people, information, technology and decisions.

The Security Manager must also become an interpreter

Perhaps this is one of the most interesting consequences.

The Security Manager of the future will not necessarily need to understand every algorithm in detail.

They do not need to become a data scientist.

They do, however, need to know what they can ask of an algorithm and what they cannot.

They must understand its margins of error.

They must understand data quality.

They must know when an alert is meaningful and when it may simply be noise.

Above all, they must build procedures that prevent technology from becoming a shortcut to decisions.

Technology needs to become part of the security process.

It must not replace that process.

A new control room

Perhaps we should imagine the control room of the future as a system in which people and machines do different things, rather than a place where artificial intelligence “sees on our behalf”.

The machine:

• observes;
• compares;
• calculates;
• detects anomalies;
• highlights correlations;
• proposes scenarios.

The person:

• interprets;
• places information in context;
• evaluates;
• decides;
• takes responsibility;
• changes the strategy.

This division of work is far more interesting than simple automation.

And, above all, far more realistic.

The real risk is not that AI makes a mistake

There is another question I consider even more important.

What happens if the algorithm makes a mistake and we notice it?

It is a problem.

But we can correct it.

The truly dangerous situation is when the algorithm makes a mistake and we do NOT notice, because we have stopped questioning what it tells us.

That is where error becomes automated.

It is one of the worst things that can happen in security.

A human error can be challenged by another person.

An error turned into a number, chart, heat map or apparently scientific alert can acquire an authority it does not deserve.

The problem is more than being wrong.

It is not knowing that we are wrong.

A crowd is not an algorithm

The future of security will not be a choice between people and technology.

It will depend on our ability to build a system in which technology increases human capabilities without dulling judgement.

The question will no longer be:

“Can we see everything?”

Probably.

The truly important question will be:

“Can we distinguish what we see from what is actually happening?”

In security, there is an enormous difference between having one more camera and having one more piece of information.

The difference between one more piece of information and a better decision is greater still.

Ultimately, in front of a crowd of 50,000 people, no algorithm will be responsible for the decision.

Someone will.

That is precisely why, even in the age of artificial intelligence, the human factor remains the most important security system we have.

Essential sources

• European Commission, AI Act — Article 5: Prohibited AI practices.
• European Commission, Navigating the AI Act — biometric identification.
• ISO, ISO 31000:2018 — Risk management — Guidelines.

All articles →

The Strathego Method →

Author: Francesco di Castri

Topics: AI · Data Management · Security

Nota di adattamento: il riferimento a Belén è conservato dall’originale. Per un pubblico internazionale può essere reso con “a celebrity”, previa scelta editoriale, senza cambiare l’esempio.