1. Short Description
The Face Detection and Recognition
(FDR) module is a physical detector designed to assist in the detection of
potentially unauthorized access to secure areas via the analysis of live video
footage from CCTV cameras. It can process video files or video streams, in
order to export the detected and recognized faces that may belong to unknown
individuals. The module is linked with an authorized person database which contains
the list of authorized persons including a set of face images for each one which
are used to match the detected faces. The main expected result is the
production of alarms whenever a person is found to be unknown or not one of the
authorized.
Figure 1 - Face Detection and Recognition (FDR)
2. Main Purpose and Benefits
Figure 2 - FDR module
architecture
The FDR module architecture is
shown in Figure 2. The
system is initialized with a video stream, but it also can support on-demand
analysis of video files in an offline processing scenario. By providing an
authorized person database, the module can be tuned to recognize the included
individuals as they move through a monitored area. As soon as an unknown person
is detected the FDR module produces alerts that can be used to enhance situational
awareness and provide better decision support in physical attacks. The module
works on the background as a backend service and upon successful installation its
operation is completely automatic. In scenarios where multiple cameras with
different sets of authorized persons are required for the monitoring of a very
large area, multiple instances of the FDR module can be installed and linked
with the respective databases which can run independently.
3. Main FunctionsThe FDR module has two main components.
Processing begins on the Face Detection (FD) component. FD is responsible to
detect patches inside the input frame where faces are tightly enclosed. The
acquired face patches are instantly characterized as unknown and are
immediately provided to the Face Recognition (FR) component for further processing.
The detected faces can be either (a) authorized (known) personnel
expected to appear in the area covered by the CCTV camera, or (b) unauthorized
individuals which are not allowed to be in this location.
FR cooperates with a
pre-existing gallery of known faces and is responsible to decide whether
all or some of the unknown faces can be matched with any known face
from the gallery. . Each face is analysed and transformed in order to extract appropriate
facial features that uniquely characterize it. After feature extraction, the unknown query faces go through the feature
matching process, which tries to match their facial features to the authorized
identities in the gallery. Depending on whether a match can or cannot be
confirmed at this stage, the output may contain recognized and unrecognized
faces. Whenever a detected face cannot be validated as authorized, the FDR
module produces an alert indicating a potential unwanted access from an unknown
individual.
Figure 3 - FDR Module Function
4. Integration with other Tools
After the recognition process, a
detailed report can be produced with the detection and recognition details using
the 7SHIELD United Alert Format (UAF) format. The output of FDR is initially stored
locally in the same machine as the deployed module and is then forwarded
through the 7SHIELD message bus to the Geospatial Complex Event Processing
Engine (GCEP) which is responsible to combine physical and cyber threats. The
FDR module itself does not provide any user interface since it only runs as a
background process.
One of the main benefits of the
approach is that the initial gallery can be expanded dynamically with
additional persons, without the need to retrain the deployed detection and
recognition models.
Additionally, in a large-scale
environment with multiple cameras, a separate gallery can be created per camera
in order to support customized monitoring for each specific area.
5. Infrastructure Requirements
The recommended way to operate FDR
is to install it locally in a machine connected to the same local network with
the streaming cameras in order to guarantee fast and smooth video streaming.
In order for the host machine to
support the operation of FDR, a modern GPU graphics card is required, with minimum
available video memory (VRAM) of 4GB (6GB recommended).
6. Operation Manual
6.1 Set-up
In order
to operate the FDR module on the host machine, a python virtual environment has
to be created. Specifically, FDR requires Python 3.8 and several python
libraries need to be installed beforehand. The list of requirements is given in
Table 6.1.
For configuration, an RTSP link is
required and the full path of the person database which has to be locally
stored within the same machine.
Table 6.1 - FDR Python Requirements
|
confluent-kafka==1.7.0
|
pandas==1.4.1
|
scipy==1.8.0
|
|
jsonschema==3.2.0
|
Pillow==8.3.1
|
tensorflow-gpu==2.5.0
|
|
Keras-Preprocessing==1.1.2
|
protobuf==3.17.3
|
torch==1.10.2+cu113
|
|
matplotlib==3.4.2
|
pytz==2021.1
|
torchaudio==0.10.2+cu113
|
|
numpy==1.19.5
|
requests==2.25.1
|
torchvision==0.11.3+cu113
|
|
opencv-python==4.5.2.54
|
scikit-learn==1.0.2
|
tqdm==4.62.3
|
|
|
|
urllib3==1.26.6
|