阅读说明：本技术 一种实时视频敏感信息保护系统及其方法 (Real-time video sensitive information protection system and method thereof ) 是由 马兆远 韩德伟 殷小雷 李康 徐健 朱善玮 董利健 毕东柱 焦孟君 于 2021-08-09 设计创作，主要内容包括：本发明提供一种实时视频敏感信息保护系统及其方法,实时视频敏感信息保护方法包括如下步骤：S1.采集实时视频流；S2.分离视频流,等间隔分隔视频流,采用检测算法提取每段视频流中预设的前景流,前景流通过可逆算法隐藏其中的隐私和敏感信息,可逆算法通过对应时间间隔内生成的公钥进行指代,检测算法包括目标检测、模板匹配算法,可逆算法包括置乱算法和密码本翻译算法,置乱算法通过图像拆分、数据扩展及图像组合进行数据加密；S3.采用不可逆算法对视频流中的敏感信息进行抹除,获取背景流,不可逆算法包括马赛克方法、色块覆盖方法和毛玻璃效果；S4.原始视频流复原。本发明可提高视频流加密程度且加密可逆,极大提高个人隐私的保护程度。(The invention provides a real-time video sensitive information protection system and a method thereof, wherein the real-time video sensitive information protection method comprises the following steps: s1, collecting a real-time video stream; s2, separating video streams, separating the video streams at equal intervals, extracting a preset foreground stream in each video stream by adopting a detection algorithm, hiding privacy and sensitive information in the foreground stream through a reversible algorithm, referring the reversible algorithm through a public key generated in a corresponding time interval, wherein the detection algorithm comprises target detection and a template matching algorithm, the reversible algorithm comprises a scrambling algorithm and a codebook translation algorithm, and the scrambling algorithm carries out data encryption through image splitting, data expansion and image combination; s3, erasing sensitive information in the video stream by adopting an irreversible algorithm to obtain a background stream, wherein the irreversible algorithm comprises a mosaic method, a color block covering method and a frosted glass effect; and S4, restoring the original video stream. The invention can improve the encryption degree of the video stream, and the encryption is reversible, thereby greatly improving the protection degree of personal privacy.)

1. A real-time video sensitive information protection method is characterized in that: the method comprises the following steps:

s1, collecting real-time video stream as original data;

s2, separating video streams, separating the video streams at equal intervals, extracting a preset foreground stream in each video stream by adopting a detection algorithm, hiding privacy and sensitive information in the foreground stream through a reversible algorithm, referring the reversible algorithm through a public key generated in a corresponding time interval, wherein the detection algorithm comprises target detection and a template matching algorithm, the reversible algorithm comprises a scrambling algorithm and a codebook translation algorithm, and the scrambling algorithm carries out data encryption through image splitting, data expansion and image combination;

s3, erasing the sensitive information in the video stream by using an irreversible algorithm to obtain a background stream, wherein the irreversible algorithm comprises a mosaic method, a color block covering method and a frosted glass effect, the mosaic method recalculates the size of the sensitive information frame detected by the detection algorithm in the step S2 and the number of preset mosaic blocks, and the color block covering method adopts color transition color blocks to replace pure color blocks;

s4, original video stream is restored, the foreground stream is decrypted by adopting a reversible algorithm indicated by a secret key generated at corresponding intervals, and the original video stream is generated after the background stream is compounded.

2. The method for protecting real-time video sensitive information according to claim 1, wherein: in step S2, the h _ Arnold algorithm is used for encryption in the scrambling algorithm, and the pseudo code is as follows:

disordered_Mat＝h_Arnold(Mat)，

wherein Mat is a color sensitive information rectangular image taken out from an original video middle frame, and discrete _ Mat is an encrypted image;

the encryption method specifically comprises the following steps:

s21, splitting the color sensitive information rectangular image into three independent IxJx1 images according to an RGB channel, and processing the images respectively;

s22, converting the image of IxJx1 into an NxNx1 image with a specific shape by adopting a data expansion method, which specifically comprises the following steps: calculating a square with the minimum side length N capable of accommodating one image, wherein the calculation formula is as follows:

N＝Ceil(Sqrt(IxJx1))，

wherein, Ceil is rounded up, then NxNx1> -IxJx 1 applies for a NxN space, IxJ data are copied in, if there is a residual space, then 0 is used for filling;

s23. combine the 3 NxNx1 scrambled images into one NxNx3 image output.

3. The method for protecting real-time video sensitive information according to claim 1, wherein: in step S3, the mosaic method specifically includes: aiming at the face frame, the area of the face frame and the number of mosaic blocks preset in the face frame are detected and calculated through a target detection algorithm, and the size of a single mosaic block is calculated, wherein the specific calculation formula is as follows: and L is Sqrt (PxQ/K), wherein the privacy target size is PxQ, namely the face frame size is PxQ, the preset number of mosaic blocks is K, and Sqrt is an open root number.

4. The method for protecting real-time video sensitive information according to claim 1, wherein: in step S3, the color block covering method uses a gaussian model to generate a gaussian color block whose color decreases from the center to the edge and adapts to the face of the human body, so as to improve the appearance of the processed video without affecting privacy protection, and specifically includes: gaussian color block baseIn the Gaussian function f (x) ae^-(x-b)/2c*cAnd c, calculating according to the length of a diagonal line of the face frame, wherein the value of the central area of the generated Gaussian color block approaches to a, the surrounding values are gradually reduced, and the Gaussian color block is superposed with the sensitive area in the video stream, so that the background stream with the sensitive information erased can be obtained.

5. The method for protecting real-time video sensitive information according to claim 2, wherein: in step S4, the deh _ Arnold algorithm is used for decryption in the scrambling algorithm, and the pseudo code is as follows:

recovered_Mat＝deh_Arnold(disordered_Mat)

wherein: the recovered _ Mat is a restored sensitive information rectangular image, and the discrete _ Mat is an encrypted image;

the decryption method specifically comprises the following steps:

s41, splitting the encrypted image into three independent NxNx1 images;

s42, decrypting the NxNx1 image;

s43, copying IxJ data from the decrypted NxNx1 image to an IxJx1 image, and finally combining 3 IxJx1 images into an IxJx3 image to be output, namely the decrypted image.

6. A protection system using the real-time video sensitive information protection method according to any one of claims 1 to 5, characterized by: the method comprises the following steps:

the real-time video acquisition equipment is used for acquiring a real-time video stream;

the key pair generation equipment is used for generating a plurality of public keys or key pairs at certain time intervals through an asymmetric encryption algorithm according to preset conditions, timestamps and hardware information;

the key storage equipment is electrically connected with the key pair generation equipment, is used for storing a key file generated by the key pair generation equipment and is used for decrypting the encrypted foreground stream when the original video stream is recovered;

the video processing equipment is electrically connected with the real-time video acquisition equipment and the secret key storage equipment and is used for separating the real-time video stream into a foreground stream and a background stream, a public key generated by the secret key generation equipment is adopted for the foreground stream through a reversible algorithm, parameters and modes are selected or the public key is used for conducting codebook translation encryption on the foreground stream, privacy or sensitive information is hidden, and the privacy or sensitive information is removed through an irreversible algorithm on the background stream.

7. The protection system of the real-time video sensitive information protection method according to claim 6, wherein: the video playing client is further included, when the original video stream needs to be played, the encryption server is accessed to obtain the secret key file generated in the time corresponding to the video to be played, the encrypted foreground stream is decrypted, and the original video stream is generated and played after the background stream is compounded.

8. The protection system of the real-time video sensitive information protection method according to claim 7, wherein: the video playing system comprises a video playing client and a foreground video storage device, and is characterized by further comprising a background video storage device and a foreground video storage device which are connected with the video processing device and the video playing client, wherein the foreground video storage device is further connected with a secret key storage device, the foreground video storage device is used for storing foreground streams obtained after the video streams are processed through a reversible algorithm designated by a public key, and the background video storage device is used for storing background streams obtained after the video streams are processed through an irreversible algorithm and providing video data for the video playing client.

9. The protection system of the real-time video sensitive information protection method according to claim 6, wherein: the real-time video acquisition equipment comprises a monitoring camera and a camera module.

Technical Field

The invention belongs to the technical field of data privacy protection, and particularly relates to a real-time video sensitive information protection system and a method thereof.

Background

With the increase of cameras, information such as human faces and human bodies can be inevitably shot in real-time videos, but the information belongs to sensitive information. In a scene such as a usual road monitoring, an elevator monitoring, a kitchen monitoring, and the like, a privacy protection operation should be performed on a real-time screen in order to protect individual privacy. When an abnormal condition occurs, the original video is called again to be used as evidence.

In the prior art, sensitive information such as human faces, human bodies and windows in videos collected by a camera needs to be subjected to fuzzy processing, so that the sensitive information is prevented from being leaked. For example, patent publication No. CN111654664A discloses a method, a system, a computer device and a storage medium for detecting a high altitude parabola, and a real-time video is collected by a video collector; the image processing unit carries out semantic separation on the monitored image and carries out fuzzy processing on the separated window area; and displaying the blurred image. For another example, patent with publication number CN108040230A provides a monitoring method and apparatus for protecting privacy, which detects whether there is a target feature matching a preset reference feature in a monitoring picture; if yes, the target features in the monitoring picture are subjected to detail removal processing, and a monitoring browsing picture is obtained. By the method, after the target characteristics in the monitoring picture are subjected to detail removal processing, the effect of protecting privacy can be achieved when the monitoring picture is displayed or called in real time. In addition, three different videos can be seen through different user rights management: 1. removing the detailed video; 2. a video with details and user watermark information; 3. a video with details. Although the method plays a certain privacy protection role, when the video is leaked, technicians can easily analyze the fuzzy processing mode of the video, and find an inverse operation method to crack the video processed by the fuzzy processing mode, so that sensitive information is leaked and video files stored in the whole system are threatened.

Disclosure of Invention

The invention aims to provide a real-time video sensitive information protection system and a method thereof, which support video streams in multiple modes, improve the encryption degree, realize reversible encryption and greatly improve the protection degree of personal privacy.

The invention provides the following technical scheme:

the application provides a real-time video sensitive information protection method, which comprises the following steps:

s1, collecting real-time video stream as original data;

s2, separating video streams, separating the video streams at equal intervals, extracting a preset foreground stream in each video stream by adopting a detection algorithm, hiding privacy and sensitive information in the foreground stream through a reversible algorithm, referring the reversible algorithm through a public key generated in a corresponding time interval, wherein the detection algorithm comprises target detection and a template matching algorithm, the reversible algorithm comprises a scrambling algorithm and a codebook translation algorithm, and the scrambling algorithm carries out data encryption through image splitting, data expansion and image combination;

s3, erasing the sensitive information in the video stream by using an irreversible algorithm to obtain a background stream, wherein the irreversible algorithm comprises a mosaic method, a color block covering method and a frosted glass effect, the mosaic method recalculates the size of the sensitive information frame detected by the detection algorithm in the step S2 and the number of preset mosaic blocks, and the color block covering method adopts color transition color blocks to replace pure color blocks;

s4, original video stream is restored, the foreground stream is decrypted by adopting a reversible algorithm indicated by a secret key generated at corresponding intervals, and the original video stream is generated after the background stream is compounded.

Preferably, in step S2, the h _ Arnold algorithm is used for encryption in the scrambling algorithm, and the pseudo code is as follows:

disordered_Mat＝h_Arnold(Mat)，

wherein Mat is a color sensitive information rectangular image taken out from an original video middle frame, and discrete _ Mat is an encrypted image;

the encryption method specifically comprises the following steps:

s21, splitting the color sensitive information rectangular image into three independent IxJx1 images according to an RGB channel, and processing the images respectively;

s22, converting the image of IxJx1 into an NxNx1 image with a specific shape by adopting a data expansion method, which specifically comprises the following steps: calculating a square with the minimum side length N capable of accommodating one image, wherein the calculation formula is as follows:

N＝Ceil(Sqrt(IxJx1))，

wherein, Ceil is rounded up, then NxNx1> -IxJx 1 applies for a NxN space, IxJ data are copied in, if there is a residual space, then 0 is used for filling;

s23. combine the 3 NxNx1 scrambled images into one NxNx3 image output.

Preferably, in step S3, the mosaic method specifically includes: aiming at the face frame, the area of the face frame and the number of mosaic blocks preset in the face frame are detected and calculated through a target detection algorithm, and the size of a single mosaic block is calculated, wherein the specific calculation formula is as follows: and L is Sqrt (PxQ/K), wherein the privacy target size is PxQ, namely the face frame size is PxQ, the preset number of mosaic blocks is K, and Sqrt is an open root number.

Preferably, in step S3, the color block coverage method generates a color by using a Gaussian modelThe Gaussian color block which is gradually decreased from the center to the edge and is adaptive to the face shape of the human body is used for improving the appearance of the processed video on the basis of not influencing privacy protection, and specifically comprises the following steps: gauss color block is based on Gauss function f (x) ae^-(x-b)/2c*cAnd c, calculating according to the length of a diagonal line of the face frame, wherein the value of the central area of the generated Gaussian color block approaches to a, the surrounding values are gradually reduced, and the Gaussian color block is superposed with the sensitive area in the video stream, so that the background stream with the sensitive information erased can be obtained.

Preferably, in step S4, deh _ Arnold algorithm is used for decryption in the scrambling algorithm, and the pseudo code is as follows:

recovered_Mat＝deh_Arnold(disordered_Mat)

wherein: the recovered _ Mat is a restored sensitive information rectangular image, and the discrete _ Mat is an encrypted image;

the decryption method specifically comprises the following steps:

s41, splitting the encrypted image into three independent NxNx1 images;

s42, decrypting the NxNx1 image;

s43, copying IxJ data from the decrypted NxNx1 image to an IxJx1 image, and finally combining 3 IxJx1 images into an IxJx3 image to be output, namely the decrypted image.

Based on the real-time video sensitive information protection method, the application also provides a protection system using the real-time video sensitive information protection method, which comprises the following steps:

the real-time video acquisition equipment is used for acquiring a real-time video stream;

the key pair generation equipment is used for generating a plurality of public keys or key pairs at certain time intervals through an asymmetric encryption algorithm according to preset conditions, timestamps and hardware information;

the key storage equipment is electrically connected with the key pair generation equipment, is used for storing a key file generated by the key pair generation equipment and is used for decrypting the encrypted foreground stream when the original video stream is recovered;

the video processing equipment is electrically connected with the real-time video acquisition equipment and the secret key storage equipment and is used for separating the real-time video stream into a foreground stream and a background stream, a public key generated by the secret key generation equipment is adopted for the foreground stream through a reversible algorithm, parameters and modes are selected or the public key is used for conducting codebook translation encryption on the foreground stream, privacy or sensitive information is hidden, and the privacy or sensitive information is removed through an irreversible algorithm on the background stream.

Preferentially, the system also comprises a video playing client, when the original video stream needs to be played, the access encryption server obtains a key file generated in time corresponding to the video to be played, the encrypted foreground stream is decrypted, and the original video stream is generated and played after the background stream is compounded.

Preferably, the video playing system further comprises a background video storage device and a foreground video storage device which are both connected with the video processing device and the video playing client, the foreground video storage device is also connected with the key storage device, the foreground video storage device is used for storing a foreground stream after the video stream is processed by a reversible algorithm indicated by a public key, and the background video storage device is used for storing a background stream after the video stream is processed by an irreversible algorithm and is used for providing video data for the video playing client.

Preferentially, the real-time video acquisition equipment comprises a monitoring camera and a camera module.

The invention has the beneficial effects that:

1. extracting foreground stream from original stream by algorithms such as target detection, template matching and the like, processing the foreground stream by an irreversible algorithm to form background stream with privacy information removed, separating video stream and facilitating direct playing;

2. the foreground stream is encrypted by combining a public key generated by the secret key generating equipment with reversible encryption algorithms such as an encryption algorithm and a codebook in a scrambling algorithm, and the encryption modes at different time intervals are different, so that the encryption degree of the foreground stream is improved, the security of sensitive information in a video is enhanced, and the situation that the foreground stream is reversely cracked to cause great loss is avoided;

3. when the original video stream is restored, the secret key generated by the secret key generating equipment is accessed, the encrypted foreground stream is decrypted by combining the reversible algorithm of the foreground stream encryption algorithm, the background stream is combined, the original video is formed, data are not lost, the privacy disclosure risk is reduced, and the evidence obtaining is facilitated.

Drawings

The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention and not to limit the invention. In the drawings:

FIG. 1 is a schematic diagram of the protection system connection of the present invention;

FIG. 2 is a schematic flow diagram of the protection method of the present invention;

FIG. 3 is a schematic diagram of color block coverage of the present invention.

Detailed Description

Example one

As shown in fig. 1, the present application provides a method for protecting real-time video sensitive information, which includes the following steps:

s1, collecting real-time video stream as original data;

s2, separating video streams, separating the video streams at equal intervals, extracting a preset foreground stream in each video stream by adopting a detection algorithm, hiding privacy and sensitive information in the foreground stream through a reversible algorithm, referring the reversible algorithm through a public key generated in a corresponding time interval, namely generating a plurality of public keys or secret key pairs at a certain time interval through an asymmetric encryption algorithm according to preset conditions, timestamps, hardware information and the like aiming at the playing time of the video streams, wherein the public keys are used for encrypting the foreground streams, the secret keys are used for decrypting the foreground streams, and the reversible algorithms corresponding to the public keys in different time periods are different, namely encrypting the foreground streams extracted from the whole video stream by adopting a plurality of reversible algorithms in different time periods, so that the cracking difficulty of the foreground streams is improved.

The detection algorithm comprises target detection and a template matching algorithm, the reversible algorithm comprises a scrambling algorithm and a codebook translation algorithm, and the scrambling algorithm carries out data encryption through image splitting, data expansion and image combination.

In step S2, the h _ Arnold algorithm is used for encryption in the scrambling algorithm, and the pseudo code is as follows:

disordered_Mat＝h_Arnold(Mat)，

wherein Mat is a color sensitive information rectangular image taken out from an original video middle frame, and discrete _ Mat is an encrypted image;

the encryption method specifically comprises the following steps:

s21, splitting the color sensitive information rectangular image into three independent IxJx1 images according to an RGB channel, and processing the images respectively;

s22, converting the image of IxJx1 into an NxNx1 image with a specific shape by adopting a data expansion method, which specifically comprises the following steps: calculating a square with the minimum side length N capable of accommodating one image, wherein the calculation formula is as follows:

N＝Ceil(Sqrt(IxJx1))，

wherein, Ceil is rounded up, then NxNx1> -IxJx 1 applies for a NxN space, IxJ data are copied in, if there is a residual space, then 0 is used for filling;

s23. combine the 3 NxNx1 scrambled images into one NxNx3 image output.

And S3, erasing the sensitive information in the video stream by adopting an irreversible algorithm to obtain a background stream, wherein the irreversible algorithm comprises a mosaic method, a color block covering method and a frosted glass effect, and the mosaic method recalculates the size of the mosaic blocks according to the size of the sensitive information frame detected by the detection algorithm in the step S2 and the number of preset mosaic blocks.

The mosaic blocks in the traditional mosaic method are fixed in size, and if the mosaic blocks are too large and the privacy target is small, information outside a face frame can be covered, so that the video impression is influenced; if the mosaic blocks are too small and the face frames are too large, sensitive information is exposed too much, and the purpose of privacy protection cannot be achieved.

In step S3, the mosaic method specifically includes: aiming at the face frame, the area of the face frame and the number of mosaic blocks preset in the face frame are detected and calculated through a target detection algorithm, and the size of a single mosaic block is calculated, wherein the specific calculation formula is as follows: and L is Sqrt (PxQ/K), wherein the privacy target size is PxQ, namely the face frame size is PxQ, the preset number of mosaic blocks is K, and Sqrt is an open root number.

As shown in fig. 3, in the conventional color block covering method, a rectangular pure color block is directly used to cover a sensitive area, and the video impression is poor. The color block covering method in the application adopts a color transition color block to replace a pure color block.

In step S3, the color block covering method uses a gaussian model to generate a gaussian color block whose color decreases from the center to the edge and adapts to the face of the human body, so as to improve the appearance of the processed video without affecting privacy protection, and specifically includes: gauss color block is based on Gauss function f (x) ae^-(x-b)/2c*cAnd generating, wherein x is an independent variable, a is a fixed integer value, namely a is 255, b is a midpoint of a sensitive information region in the video stream, namely a midpoint of a face frame, c is calculated according to the length of a diagonal line of the face frame, the value of a central region of a generated Gaussian color block is close to 255, surrounding values are gradually reduced, and the Gaussian color block and the sensitive region in the video stream are superposed to obtain the background stream with the sensitive information erased.

S4, original video stream is restored, the foreground stream is decrypted by adopting a reversible algorithm indicated by a secret key generated at corresponding intervals, and the original video stream is generated after the background stream is compounded.

In step S4, the deh _ Arnold algorithm is used for decryption in the scrambling algorithm, and the pseudo code is as follows:

recovered_Mat＝deh_Arnold(disordered_Mat)

wherein: the recovered _ Mat is a restored sensitive information rectangular image, and the discrete _ Mat is an encrypted image;

the decryption method specifically comprises the following steps:

s41, splitting the encrypted image into three independent NxNx1 images;

s42, decrypting the NxNx1 image;

s43, copying IxJ data from the decrypted NxNx1 image to an IxJx1 image, and finally combining 3 IxJx1 images into an IxJx3 image to be output, namely the decrypted image.

As shown in fig. 2, based on the real-time video sensitive information protection method, the present application further provides a protection system using the real-time video sensitive information protection method, including:

the real-time video acquisition equipment is used for acquiring real-time video streams and comprises a monitoring camera and a camera module.

And the key pair generation equipment is used for generating a plurality of public keys or key pairs at certain time intervals through an asymmetric encryption algorithm according to preset conditions, time stamps and hardware information.

And the key storage equipment is electrically connected with the key pair generation equipment, is used for storing the key file generated by the key pair generation equipment and is used for decrypting the encrypted foreground stream when the original video stream is recovered.

The video processing equipment is electrically connected with the real-time video acquisition equipment and the secret key storage equipment and is used for separating the real-time video stream into a foreground stream and a background stream, performing parameter and mode selection or reversibly encrypting the foreground stream by using a public key generated by the secret key generation equipment by using a reversible algorithm on the foreground stream, hiding privacy or sensitive information and removing the privacy or sensitive information by using an irreversible algorithm on the background stream.

The video playing client is further included, when the original video stream needs to be played, the encryption server is accessed to obtain the secret key file generated in the time corresponding to the video to be played, the encrypted foreground stream is decrypted, and the original video stream is generated and played after the background stream is compounded.

The video playing system comprises a video playing client and a foreground video storage device, and is characterized by further comprising a background video storage device and a foreground video storage device which are connected with the video processing device and the video playing client, wherein the foreground video storage device is further connected with a secret key storage device, the foreground video storage device is used for storing foreground streams obtained after the video streams are processed through a reversible algorithm designated by a public key, and the background video storage device is used for storing background streams obtained after the video streams are processed through an irreversible algorithm and providing video data for the video playing client.

Example two

The present embodiment is different from the first embodiment in that: the reversible algorithm adopted by the foreground stream adopts a codebook translation algorithm, the public key is used for carrying out codebook translation encryption on the foreground stream, and the processed foreground stream is stored in the foreground video storage device. Other technical features and working principles of the present embodiment are the same as those of the first embodiment.

Although the present invention has been described in detail with reference to the foregoing embodiments, it will be apparent to those skilled in the art that changes may be made in the embodiments and/or equivalents thereof without departing from the spirit and scope of the invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.