阅读说明：本技术 一种带有人脸识别分析功能的网络视频存储装置及方法 (network video storage device with face recognition and analysis function and method ) 是由 傅剑辉 王东 于 2018-06-07 设计创作，主要内容包括：本发明公开了一种带有人脸识别分析功能的网络视频存储装置,实现视频接入、存储、显示、计算加速和人脸识别、特征获取与比对,以及信息生成和传输的功能,本发明能够用于各种新建和已有网络视频监控场合,能够大幅度提高监控视频中最重要的人脸信息的保存时间,并实时对风险情况提出警示,及时显示相应人脸图片,有利于满足治安防控的多种需求；采用在时间窗中多次采样取优的方式,解决因工况环境带来的人脸特征分析技术不足的问题；利用人脸特征稳定性的特征,解决了各种中小安防监控系统中无法及时进行人员身份辨识与警示的问题。(The invention discloses a network video storage device with a face recognition and analysis function, which realizes the functions of video access, storage, display, calculation acceleration, face recognition, feature acquisition and comparison, and information generation and transmission, can be used in various newly-built and existing network video monitoring occasions, can greatly improve the storage time of the most important face information in a monitoring video, provides real-time warning for the risk condition, displays corresponding face pictures in time, and is favorable for meeting various requirements of public security and defense control; the method of sampling and optimizing for multiple times in a time window is adopted, so that the problem of insufficient human face feature analysis technology caused by a working condition environment is solved; by utilizing the characteristic of the stability of the human face characteristic, the problem that personnel identity identification and warning cannot be timely performed in various small and medium security monitoring systems is solved.)

1. The utility model provides a network video storage device with face identification analysis function which characterized in that: the functions of video access, storage, display, calculation acceleration, face recognition, feature acquisition and comparison, information generation and transmission are realized, and the functions comprise

01 video access port: receiving external multi-channel video image code streams according to an instruction of a central control module (08) to obtain externally provided video images;

02 image reproduction module: copying the received video according to the instruction of the central control module (08) and sending the copied video to the image storage module (04);

03 image decoding conversion module: decoding and YUV-RGB conversion are carried out on the corresponding video according to the instruction of the central control module (08), and image down-sampling is carried out according to the instruction of the central control module (08);

04 image storage module: storing the received images in a corresponding directory of a hard disk according to an instruction of a central control module (08), and covering and deleting overdue video records exceeding the video time limit;

05 face identification module: according to the instruction of the central control module (08), carrying out face recognition by using an artificial intelligent network model through convolution calculation and loading a pruning dynamic face recognition algorithm to obtain a face image in the picture;

06 face comparison module: according to the instruction of the central control module (08), extracting the feature code of the face picture obtained by the face recognition module (05), comparing the feature code with the face data stored in the database for a certain time, screening out the face image needing prompting, and providing the face image to the image display module (07) for displaying;

07 image display module: according to the instruction of a central control module (08), displaying the received video in real time, displaying a picture prompting personnel, and displaying playback video and personnel information retrieval;

08 central control module: realize the comprehensive control and management of other modules of the device.

2. A face target recognition method is characterized in that: the method comprises the following steps of adopting a time window as a sampling period, setting the window size as required, carrying out multiple sampling in the sampling period, selecting optimal sampling as final sampling information, traversing all faces when a plurality of faces appear in a sampling frame, adopting a rejection strategy for the faces which cannot be identified, and carrying out rejection compensation by multiple sampling in the period, wherein the specific method comprises the following steps:

when the time window is triggered, the device starts to decode the video, firstly carries out video frame lifting, carries out frame image optimization after frame lifting, carries out multi-face positioning after optimization, detects a human face image after positioning, then verifies the human face image to see whether the human face image meets the requirements, carries out a non-conforming return time window triggering step, carries out human face correction and duplication removal meeting the requirements, carries out feature code extraction after duplication removal, carries out a human face comparison and alarm judgment step after extracting the feature code, carries out human face comparison and output alarm type and human face image meeting the requirements, then carries out the next time window, continues to carry out new time window triggering, and carries out a non-conforming return time window triggering step.

Technical Field

The invention relates to the field of artificial intelligence, in particular to a network video storage device with a face recognition and analysis function and a method.

Background

At present, the main functions of a network video storage device (mainly a network video recorder NVR) comprise video real-time display, video recording and video playback, and the network video storage device mainly plays a role in displaying and recording video image information acquired by a network video camera in real time, automatically covering the earliest video after a certain time (generally one month or two months) so as to achieve the effects of circular recording and saving storage space, and plays a role in real-time monitoring and after-the-fact investigation in preventing unexpected events. The existing network video storage device can not carry out intelligent personnel analysis prompt based on face recognition.

based on the existing security network video recording system, the network video storage device and method for carrying out face recognition and personnel analysis prompting based on the AI acceleration chip are formed by combining the edge calculation technology, the AI chip acceleration technology, the convolutional neural network technology, the deep learning algorithm and the face tracking snapshot technology

The network video storage device with the face recognition and analysis function and the method thereof not only carry out general real-time display and video recording on monitoring videos of various places, but also carry out face acquisition and recognition on persons shot in the monitoring videos, and give corresponding prompts to operators according to the frequency of appearance of the persons, whether the persons appear for the first time, whether the persons are provided with masks/sunglasses, whether the persons always face sideways or back to a camera and other conditions and behaviors. Meanwhile, the data volume of the face information is usually the most important information in the video image, and the data volume is far smaller than that of the video image, so that the important face data can be stored for a longer time (for example, 6 or 12 months) on the basis of keeping the original video recording time. The invention provides an intelligent new method with larger time span for improving the prevention and control level of a monitoring system and the joint security and prevention control management level.

disclosure of Invention

the invention aims to solve the defects and provides a network video storage device with a face recognition and analysis function and a method.

the purpose of the invention is realized by the following technical scheme:

A network video storage device with face recognition and analysis functions for realizing video access, storage, display, calculation acceleration, face recognition, feature acquisition and comparison, and information generation and transmission functions comprises

01 video access port: receiving external multi-channel video image code streams according to an instruction of a central control module (08) to obtain externally provided video images;

02 image reproduction module: copying the received video according to the instruction of the central control module (08) and sending the copied video to the image storage module (04);

03 image decoding conversion module: decoding and YUV-RGB conversion are carried out on the corresponding video according to the instruction of the central control module (08), and image down-sampling is carried out according to the instruction of the central control module (08);

04 image storage module: storing the received images in a corresponding directory of a hard disk according to an instruction of a central control module (08), and covering and deleting overdue video records exceeding the video time limit;

05 face identification module: according to the instruction of the central control module (08), carrying out face recognition by using an artificial intelligent network model through convolution calculation and loading a pruning dynamic face recognition algorithm to obtain a face image in the picture;

06 face comparison module: according to the instruction of the central control module (08), extracting the feature code of the face picture obtained by the face recognition module (05), comparing the feature code with the face data stored in the database for a certain time, screening out the face image needing prompting, and providing the face image to the image display module (07) for displaying;

07 image display module: according to the instruction of a central control module (08), displaying the received video in real time, displaying a picture prompting personnel, and displaying playback video and personnel information retrieval;

08 central control module: realize the comprehensive control and management of other modules of the device.

A human face target recognition method adopts a time window as a sampling period, the window size can be set according to needs, multiple sampling is carried out in one sampling period, the optimal sampling is selected as the final sampling information, for a plurality of human faces appearing in one sampling frame, all human faces need to be traversed, a rejection strategy is adopted for the human faces which cannot be recognized, and the rejection compensation is carried out by multiple sampling in the period, and the specific method is as follows:

when the time window is triggered, the device starts to decode the video, firstly carries out video frame lifting, carries out frame image optimization after frame lifting, carries out multi-face positioning after optimization, detects a human face image after positioning, then verifies the human face image to see whether the human face image meets the requirements, carries out a non-conforming return time window triggering step, carries out human face correction and duplication removal meeting the requirements, carries out feature code extraction after duplication removal, carries out a human face comparison and alarm judgment step after extracting the feature code, carries out human face comparison and output alarm type and human face image meeting the requirements, then carries out the next time window, continues to carry out new time window triggering, and carries out a non-conforming return time window triggering step.

the invention has the following beneficial effects:

the invention can be used in various newly-built and existing network video monitoring occasions, can greatly improve the storage time of the most important face information in the monitoring video, provides warning for the risk condition in real time, displays corresponding face pictures in time, and is beneficial to meeting various requirements of public security prevention and control; the method of sampling and optimizing for multiple times in a time window is adopted, so that the problem of insufficient human face feature analysis technology caused by a working condition environment is solved; by utilizing the characteristic of the stability of the human face characteristic, the problem that personnel identity identification and warning cannot be timely performed in various small and medium security monitoring systems is solved.

Drawings

FIG. 1 is a schematic structural diagram of a memory device according to the present invention;

fig. 2 is a flow chart of the face recognition method of the present invention.

Detailed Description

the invention is further described with reference to the accompanying drawings in which:

as shown in fig. 1 and 2, a network video storage device with a face recognition and analysis function, which implements the functions of video access, storage, display, computation acceleration, face recognition, feature acquisition and comparison, and information generation and transmission, includes

01 video access port: receiving external multi-channel video image code streams according to an instruction of a central control module (08) to obtain externally provided video images;

02 image reproduction module: copying the received video according to the instruction of the central control module (08) and sending the copied video to the image storage module (04);

03 image decoding conversion module: decoding and YUV-RGB conversion are carried out on the corresponding video according to the instruction of the central control module (08), and image down-sampling is carried out according to the instruction of the central control module (08);

04 image storage module: storing the received images in a corresponding directory of a hard disk according to an instruction of a central control module (08), and covering and deleting overdue video records exceeding the video time limit;

05 face identification module: according to the instruction of the central control module (08), carrying out face recognition by using an artificial intelligent network model through convolution calculation and loading a pruning dynamic face recognition algorithm to obtain a face image in the picture;

06 face comparison module: according to the instruction of the central control module (08), extracting the feature code of the face picture obtained by the face recognition module (05), comparing the feature code with the face data stored in the database for a certain time, screening out the face image needing prompting, and providing the face image to the image display module (07) for displaying;

07 image display module: according to the instruction of a central control module (08), displaying the received video in real time, displaying a picture prompting personnel, and displaying playback video and personnel information retrieval;

08 central control module: realize the comprehensive control and management of other modules of the device.

a human face target recognition method adopts a time window as a sampling period, the window size can be set according to needs, multiple sampling is carried out in one sampling period, the optimal sampling is selected as the final sampling information, for a plurality of human faces appearing in one sampling frame, all human faces need to be traversed, a rejection strategy is adopted for the human faces which cannot be recognized, and the rejection compensation is carried out by multiple sampling in the period, and the specific method is as follows:

When the time window is triggered, the device starts to decode the video, firstly carries out video frame lifting, carries out frame image optimization after frame lifting, carries out multi-face positioning after optimization, detects a human face image after positioning, then verifies the human face image to see whether the human face image meets the requirements, carries out a non-conforming return time window triggering step, carries out human face correction and duplication removal meeting the requirements, carries out feature code extraction after duplication removal, carries out a human face comparison and alarm judgment step after extracting the feature code, carries out human face comparison and output alarm type and human face image meeting the requirements, then carries out the next time window, continues to carry out new time window triggering, and carries out a non-conforming return time window triggering step.

The working principle is as follows: the video code stream signal received by the network port is copied, one path is directly written into a specified directory in the hard disk according to a storage rule, the second path is used for carrying out video decoding in a display cache, the video is spliced with other paths of videos and then sent to a display output port (VGA or HDMI) for real-time display, simultaneously, the decoded video is subjected to YUV-RGB conversion and down-sampling, according to the artificial intelligent network model, the pruning dynamic face recognition algorithm is calculated and loaded through convolution, detecting, tracking, snapshotting and removing duplication of human faces in the frame pictures, constructing detection personnel information by using the characteristic values as information identifiers, completing personnel information acquisition and recording, performing data operations such as comparison with human faces in a database, warehousing of newly added human faces and the like, and for the human face meeting the detection rules of frequent visits, first visits, sunglasses/masks wearing and the like, putting the human face into a display output port, and giving visual prompts on a display picture. The core principle is that dynamic face acquisition, analysis and prompt are carried out while network video recording is finished.

The mobile form expression is adopted, and the mobile non-contact dynamic personnel snapshot acquisition device is formed by loading the mobile form expression to a vehicle-mounted terminal; the device can independently form a hardware system, and is in butt joint with a video to form a front plug-in device; the device can be directly connected behind the existing high-definition network camera in series, can directly complete work at the front end, obtains high-quality face structural data, efficiently and timely provides corresponding warning information on site, and can be widely applied to video application places of mechanisms such as transportation (airports, high-speed rails and subways), office buildings, communities, supermarkets, hospitals, banks, securities and insurance. The device can be deployed in a mobile workstation, and uses GPS/Beidou/4G to carry out positioning and wireless transmission, and uses a temporarily distributed face snapshot place; . In the implementation, optimization can be performed on a core algorithm structure, and other service functions such as key personnel tracking, personnel attribute analysis and the like are added.

The problem of applying the AI face recognition and comparison technology to a network video recorder is solved; the method can be used for various newly-built and existing network video monitoring occasions, can greatly prolong the storage time of the most important face information in the monitoring video, provides warning for the risk condition in real time, displays corresponding face pictures in time, and is favorable for meeting various requirements of public security prevention and control; the method of sampling and optimizing for multiple times in a time window is adopted, so that the problem of insufficient human face feature analysis technology caused by a working condition environment is solved; by utilizing the characteristic of the stability of the human face characteristic, the problem that personnel identity identification and warning cannot be timely performed in various small and medium security monitoring systems is solved.

the limitation of contact type personnel information acquisition by means of human authentication verification, 1: 1 human face acquisition and the like in the prior art is broken through, and a non-interference working mode is adopted, so that the behavior of monitored personnel is not disturbed; the face recognition technology and the network video recording technology are reused, the use of the face recognition technology in security service processing is enlarged, and the use area of the face recognition technology is expanded; the existing camera stock is fully utilized, and the existing unstructured video can be directly upgraded into intelligent structured data in a plug-in mode.