information processing method and apparatus, image device, and storage medium

文档序号：1783057 发布日期：2019-12-06 浏览：23次中文

阅读说明：本技术 信息处理方法及装置、图像设备及存储介质 (information processing method and apparatus, image device, and storage medium ) 是由李江涛付豪彭雨佳张寅艳黄礼玮于 2019-06-28 设计创作，主要内容包括：本公开实施例公开了一种信息处理方法及装置、图像设备及存储介质。所述信息处理方法,包括：获取目标对象筛选条件,所述目标对象筛选条件至少包括：时间条件,所述时间条件包括：在第一时间段内目标对象的第一出现频次、以及在第二时间段内所述目标对象的第二出现频次共同满足预设条件；根据所述目标对象筛选条件,确定出所述目标对象的聚类档案,其中,一个所述聚类档案包括一个采集对象的档案信息,所述档案信息包括所述采集对象活动的时空范围信息；基于所述目标对象的聚类档案,执行预定操作。(The embodiment of the disclosure discloses an information processing method and device, an image device and a storage medium. The information processing method comprises the following steps: obtaining target object screening conditions, wherein the target object screening conditions at least comprise: a temporal condition, the temporal condition comprising: the method comprises the following steps that a first appearance frequency of a target object in a first time period and a second appearance frequency of the target object in a second time period meet preset conditions together; determining cluster archives of the target object according to the target object screening conditions, wherein one cluster archive comprises archive information of an acquisition object, and the archive information comprises space-time range information of the activity of the acquisition object; and executing a predetermined operation based on the clustering archive of the target object.)

1. an information processing method characterized by comprising:

Obtaining target object screening conditions, wherein the target object screening conditions at least comprise: a temporal condition, the temporal condition comprising: the method comprises the following steps that a first appearance frequency of a target object in a first time period and a second appearance frequency of the target object in a second time period meet preset conditions together;

Determining cluster archives of the target object according to the target object screening conditions, wherein one cluster archive comprises archive information of an acquisition object, and the archive information comprises space-time range information of the activity of the acquisition object;

And executing a predetermined operation based on the clustering archive of the target object.

2. The method of claim 1, wherein the target object screening condition further comprises: spatial conditions; wherein the spatial conditions comprise: the spatial range of motion of the target object, the temporal condition further comprising: and the first appearance frequency of the target object in the first time period and the second appearance frequency of the target object in the second time period in the activity space range jointly meet preset conditions.

3. The method according to claim 1 or 2, wherein the first frequency of occurrence and the second frequency of occurrence together satisfy the preset condition, including at least one of:

the ratio of the second occurrence frequency to the sum of the first occurrence frequency and the second occurrence frequency is greater than a ratio threshold;

the first frequency of occurrence is below a first frequency threshold and the second frequency of occurrence is above a second frequency threshold, wherein the second frequency threshold is greater than the first frequency threshold.

4. The method according to any one of claims 1 to 3, wherein the frequency of occurrence of the preset events of the first time period is higher than the frequency of occurrence of the preset events of the second time period.

5. the method according to any one of claims 1 to 4, wherein the performing a predetermined operation based on the cluster profile of the target object comprises:

Filtering to obtain the target object based on the filtering information;

And outputting early warning information based on the clustering archives of the target objects.

6. The method according to any one of claims 1 to 5, wherein the performing a predetermined operation based on the cluster profile of the target object further comprises:

Outputting at least part of information in the cluster profile of the target object based on the cluster profile of the target object.

7. The method of claim 6, wherein outputting at least part of the information of the target object based on the cluster archive of the target object comprises:

and if the number of the target objects is at least two, sequentially outputting at least part of information in the clustering archives of the target objects according to the frequency of occurrence from high to low.

8. An information processing apparatus characterized by comprising:

a first obtaining module, configured to obtain target object screening conditions, where the target object screening conditions at least include: a temporal condition, the temporal condition comprising: the method comprises the following steps that a first appearance frequency of a target object in a first time period and a second appearance frequency of the target object in a second time period meet preset conditions together;

The determining module is used for determining cluster archives of the target object according to the target object screening conditions, wherein one cluster archive comprises archive information of an acquisition object, and the archive information comprises space-time range information of the activity of the acquisition object;

and the execution module is used for executing a preset operation based on the clustering archives of the target objects.

9. An electronic device, the electronic device comprising:

A memory for storing computer executable instructions;

A processor coupled to the memory for implementing the method provided by any of claims 1 to 7 by executing the computer-executable instructions.

10. a computer storage medium having stored thereon computer-executable instructions; the computer executable instructions, when executed by a processor, are capable of implementing the method of any one of claims 1 to 7.

Technical Field

The present invention relates to the field of image technologies, and in particular, to an information processing method and apparatus, an image device, and a storage medium.

Background

With the development of the related technology, face retrieval is widely applied, and particularly, when a case is resolved in the public security industry, retrieval needs to be carried out in a massive portrait library according to suspect images of unidentified identities. The commonly used face retrieval method is to compare the retrieved pictures with the database pictures one by one. When a large number of pictures exist in the database, the calculation amount of face retrieval is greatly increased, so that the retrieval speed is slow and the recall rate is low.

Disclosure of Invention

in view of the above, it is desirable to provide an information processing method and apparatus, an image device, and a storage medium.

The technical scheme of the invention is realized as follows:

An information processing method comprising:

And executing a predetermined operation based on the clustering archive of the target object.

Based on the above scheme, the target object screening condition further includes: spatial conditions; wherein the spatial conditions comprise: the spatial range of motion of the target object, the temporal condition further comprising: and the first appearance frequency of the target object in the first time period and the second appearance frequency of the target object in the second time period in the activity space range jointly meet preset conditions.

Based on the above scheme, the first frequency of occurrence and the second frequency of occurrence together satisfy the preset condition, including at least one of:

The ratio of the second occurrence frequency to the sum of the first occurrence frequency and the second occurrence frequency is greater than a ratio threshold;

Based on the scheme, the occurrence frequency of the preset events in the first time period is higher than that in the second time period.

Based on the above scheme, the performing a predetermined operation based on the cluster archive of the target object includes:

Filtering to obtain the target object based on the filtering information;

And outputting early warning information based on the clustering archives of the target objects.

based on the above scheme, the executing a predetermined operation based on the cluster archive of the target object further includes:

outputting at least part of information in the cluster profile of the target object based on the cluster profile of the target object.

Based on the above scheme, the outputting at least part of information of the target object based on the cluster archive of the target object includes:

based on the above scheme, if there are at least two target objects, sequentially outputting at least part of information in the cluster archive of the target objects according to the frequency of occurrence from high to low includes:

if the number of the target objects is at least two, and the occurrence frequency of the at least two target objects is equal, sequentially outputting at least part of information in the cluster files of the at least two target objects with the equal occurrence frequency according to the occurrence time sequence of the target object appearing in the time range for the last time.

based on the above scheme, the executing a predetermined operation based on the cluster archive of the predetermined object further includes:

Determining an activity trajectory of the predetermined object based on the cluster profile of the target object.

Based on the above scheme, the method further comprises:

Selecting a second clustering archive from the first clustering archive according to the selection information;

the determining the cluster archive of the target object according to the target object screening condition includes:

And determining the clustering archive of the target object from the second clustering archive according to the target object screening condition.

based on the scheme, the selection information comprises one or more of the following:

Library information of the second hierarchical archive;

library information of a third hierarchical archive, wherein the third hierarchical archive is: a cluster archive in the first cluster archive other than the second cluster archive.

based on the above scheme, the method further comprises:

Acquiring first image information of the acquisition object;

Determining object feature classes to which the acquisition objects belong, wherein the object feature classes are included in the first image information, according to object features of the first image information, wherein each object feature class corresponds to one acquisition object;

generating the clustering archive according to the object feature class;

And clustering archives of different acquisition objects form the clustering archive library.

Based on the above scheme, the method further comprises:

Acquiring identity information including second image information;

Determining an object feature class matched with the second image information according to the object features of the second image information;

generating the cluster archive according to the object feature class includes:

and generating a cluster archive corresponding to the object feature class matched with the second image information based on the identity information comprising the second image information, the plurality of image information comprised by the object special line class and the acquisition information of the acquisition object in each image information.

an information processing apparatus comprising:

A first obtaining module, configured to obtain target object screening conditions, where the target object screening conditions at least include: a temporal condition, the temporal condition comprising: the method comprises the following steps that a first appearance frequency of a target object in a first time period and a second appearance frequency of the target object in a second time period meet preset conditions together;

the determining module is used for determining cluster archives of the target object according to the target object screening conditions, wherein one cluster archive comprises archive information of an acquisition object, and the archive information comprises space-time range information of the activity of the acquisition object;

and the execution module is used for executing a preset operation based on the clustering archives of the target objects.

based on the above scheme, the target object screening condition further includes: spatial conditions; wherein the spatial conditions comprise: the spatial range of motion of the target object, the temporal condition further comprising: and the first appearance frequency of the target object in the first time period and the second appearance frequency of the target object in the second time period in the activity space range jointly meet preset conditions.

Based on the above scheme, the first frequency of occurrence and the second frequency of occurrence together satisfy the preset condition, including at least one of:

the ratio of the second occurrence frequency to the sum of the first occurrence frequency and the second occurrence frequency is greater than a ratio threshold;

Based on the scheme, the frequency of the preset events in the first time period is higher than that in the second time period.

Based on the above scheme, the execution module is specifically configured to filter the target object to obtain a target object from the collected object based on filtering information; and outputting early warning information based on the clustering archives of the target objects.

based on the above scheme, the execution module is specifically configured to output at least part of information in the cluster archive of the target object based on the cluster archive of the target object.

Based on the above scheme, the execution module is specifically configured to, if there are at least two target objects, sequentially output at least part of information in the cluster archive of the target objects according to the frequency of occurrence from high to low.

based on the above scheme, the executing module is specifically configured to, if there are at least two target objects and there are at least two target objects whose occurrence frequencies are equal, sequentially output at least part of information in the cluster files of the at least two target objects whose occurrence frequencies are equal according to an occurrence time sequence of the last occurrence of the target object in the time range.

based on the above scheme, the executing module is specifically configured to determine an activity track of the predetermined object based on the cluster profile of the target object.

Based on the above scheme, the apparatus further comprises:

the first selection module is used for selecting a second clustering archive from the first clustering archive according to the selection information;

And the determining module is used for determining the clustering archive of the target object from the second clustering archive according to the target object screening condition.

based on the scheme, the selection information comprises one or more of the following:

Library information of the second hierarchical archive;

Library information of a third hierarchical archive, wherein the third hierarchical archive is: a cluster archive in the first cluster archive other than the second cluster archive.

Based on the above scheme, the apparatus further comprises:

the first image module is used for acquiring first image information of the acquisition object;

the first object feature class module is used for determining an object feature class to which the acquisition object belongs, wherein the object feature class is contained in the first image information, and each object feature class corresponds to one acquisition object;

The generating module is used for generating the clustering file according to the object feature class;

and the forming module is used for forming the cluster archive library of different acquisition objects.

Based on the above scheme, the apparatus further comprises:

The identity information module is used for acquiring identity information comprising second image information;

the second object feature class module is used for determining an object feature class matched with the second image information according to the object features of the second image information;

the generating module is configured to generate a cluster archive corresponding to an object feature class matched with the second image information based on the identity information including the second image information, the plurality of image information included in the object special line class, and the acquisition information of the acquisition object in each of the image information

An embodiment of the present disclosure further provides an electronic device, which includes:

A memory for storing computer executable instructions;

And the processor is connected with the memory and used for realizing the information processing method provided by any technical scheme by executing the computer executable instruction.

A computer storage medium having stored thereon computer-executable instructions; the computer-executable instructions can be used for realizing the information processing method provided by any one of the preceding items after being executed by a processor.

According to the technical scheme provided by the embodiment of the disclosure, the pre-established cluster archives are inquired and retrieved after the target object screening conditions are obtained, and the target objects which frequently meet the target object screening conditions in the time range indicated by the target time information are directly found from the cluster archives, so that the method has the characteristic of high efficiency; meanwhile, compared with manual judgment, the method reduces manual errors and error rate caused by unskilled technicians, and has the characteristics of high accuracy and high recall rate; and the clustering archives are pre-established based on image information and the like, so that the target object can be quickly found out by directly inquiring the clustering archives when retrieval is needed, and the efficiency can be greatly improved.

drawings

Fig. 1 is a schematic flowchart of an information processing method according to an embodiment of the present invention;

fig. 2 is a schematic diagram illustrating an effect of a UI for acquiring target time information according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of a process for generating a cluster file according to an embodiment of the present invention;

Fig. 4 is a schematic structural diagram of an information processing apparatus according to an embodiment of the present invention;

FIG. 5 is a schematic diagram of performing feature clustering for archiving according to an embodiment of the present invention;

FIG. 6 is a schematic flow chart of another feature classification provided in the embodiments of the present invention;

Fig. 7 is a schematic diagram illustrating an effect of feature clustering according to an embodiment of the present invention;

FIG. 8 is a schematic diagram of another process for generating a cluster file according to an embodiment of the present invention;

Fig. 9 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

Detailed Description

the technical solution of the present invention is further described in detail with reference to the drawings and the specific embodiments of the specification.

As shown in fig. 1, the present embodiment provides an information processing method including:

step S101: obtaining target object screening conditions, wherein the target object screening conditions at least comprise: a temporal condition, the temporal condition comprising: the method comprises the following steps that a first appearance frequency of a target object in a first time period and a second appearance frequency of the target object in a second time period meet preset conditions together;

step S102: obtaining target object screening conditions, wherein the target object screening conditions at least comprise: a temporal condition, the temporal condition comprising: the method comprises the following steps that a first appearance frequency of a target object in a first time period and a second appearance frequency of the target object in a second time period meet preset conditions together;

step S103: and executing a predetermined operation based on the clustering archive of the target object.

Embodiments of the present disclosure may be applied to various electronic devices including, but not limited to, stationary devices and/or mobile devices, for example, the stationary devices include, but are not limited to: personal Computers (PCs), servers, or the like. The mobile devices include, but are not limited to: a cell phone, tablet, or wearable device.

In this embodiment, the target object screening conditions at least include: target time information. The target time information may include various types: the time information of the acquisition object captured by the image acquisition device. The target time information defines a time condition. The time condition is that the time condition is combined with the occurrence frequency of the first time interval and the second time interval to know the target object from the candidate objects.

the target object screening condition may be information received by the electronic device from the human-computer interaction interface, or may be information received by the electronic device.

Fig. 2 is a schematic diagram of a User Interface (UI) for receiving the target time information according to this embodiment.

and displaying an input frame in which a time range and a place can be written in the UI, and receiving time information in the input frame corresponding to the time range by the electronic equipment to obtain the target object screening condition.

In some embodiments, the spatial information may also be obtained within a selection box corresponding to the location within the UI interface shown in fig. 2. The spatial information defines spatial conditions.

For example, the spatial conditions include: a spatial extent of motion of the target object. Thus, the number of target objects is further reduced, and accurate searching can be realized.

in some embodiments, if the spatial condition is not obtained, all cluster files may be retrieved to obtain a cluster file of a target object whose occurrence frequency only satisfies the temporal condition.

In other embodiments, if the spatial condition is not obtained, the cluster archive in the cluster archive may be retrieved conventionally to obtain the cluster archive of the target object whose occurrence frequency only satisfies the temporal condition.

for example, different cluster archives may correspond to cluster archives of candidates residing in different spatial ranges.

in some embodiments, the target object screening condition further comprises: spatial conditions; wherein the spatial conditions comprise: the spatial range of motion of the target object, the temporal condition further comprising: and the first appearance frequency of the target object in the first time period and the second appearance frequency of the target object in the second time period in the activity space range jointly meet preset conditions.

In other embodiments, the cluster archive contains all activity information for one collection object. The post-motion information includes: temporal-spatial information of the activity, type information of the activity, etc.

In some embodiments, the cluster archive further comprises identity information of a collection object.

In this embodiment, the cluster archive may be generated in advance, and if the cluster archive is retrieved according to the time condition, the cluster archive may include activity information of the acquisition object, where the activity information may be activity spatiotemporal information obtained based on acquisition time and acquisition space of the acquired image. Since the cluster archive has already been generated, after obtaining the temporal condition, it is only necessary to match the temporal condition to the active spatio-temporal information in the cluster archive to know the cluster archive of the target object that needs to be retrieved. For example, after the time condition is input in the UI interface shown in fig. 2, it is known that 50 persons have moved in the time period B of the location a through the information retrieval comparison, and the cluster file of the 50 persons is obtained. Certain operations will be performed in step S103 directly based on the cluster profiles of the 50 persons.

For example, in step S103, the cluster profile number of the cluster profile of 50 persons is output, which facilitates law enforcement personnel to retrieve the cluster profile of 50 persons.

For another example, the identity attribute information of the 50-person cluster file is directly output, so that law enforcement personnel can conveniently and quickly read the cluster file.

in this embodiment, since the clustering archive is created based on the image information filed in advance, in the security field, in the crime monitoring field, or in the road monitoring field, when some information needs to be called, the clustering archive corresponding to the target object with a high frequency of occurrence in the time range and the time range can be obtained by directly inputting the time condition.

In the embodiment of the present disclosure, a cluster archive of a collection object may be acquired, where the cluster archive includes activity information of the collection object. The activity information includes at least one of:

Acquiring specific appearance time information of an object in a specific space-time;

Acquiring specific appearance place information of an object in a specific space-time;

collecting the occurrence frequency information of an object;

Behavior feature information of the object is collected.

In some embodiments, the cluster archive further comprises identity attribute information of the collection object. The identity attribute information includes, but is not limited to, identification information of the acquisition object. The identification information includes, but is not limited to, identity information and/or biometric information of the acquisition subject, including, but not limited to: face features, iris features, voiceprint features, and the like.

in some embodiments, a cluster profile of the collection object includes, in addition to activity information of the collection object, identity attribute information describing the collection object, including but not limited to identity information of the collection object, attribute information of the collection object. The identity information includes, but is not limited to, an identification number of the collecting object, a passport number, and the like, which can uniquely identify the identity. The attribute information includes, but is not limited to: collecting gender, age, contact information and/or social relationship information of the object.

In some embodiments, the first period and the second period are different time periods within a time range indicated by the time condition.

for example, the first period of time may be night time and the second period of time may be day time.

For another example, the first period of time may be a holiday and the second period of time may be a weekday.

For another example, the first period may be a work period of a day; the second period may be a rest period of the day, for example, a period of noon lunch break or a period after work. The working time period is the morning work period in the morning, the afternoon work period in the afternoon or the night work period at night. The second period of time may be any period of time other than the operating period.

In summary, in this embodiment, the first time period may be a time period in which the occurrence frequency of the security event is higher relative to the second time period. The security event can be various security and defense related events in the field of security and protection, such as a security accident and the like. The safety accidents comprise various accidents related to personal and/or property safety in the monitoring range, for example, accidents related to personal and/or property safety in a cell; accidents of personal and/or property safety in the plant; personal and/or property security incidents in buildings. For example, searching through a cluster file to locate people or vehicles in daytime and nighttime; the first time period may be set to day and the second time period may be set to night. Specifically, the number of times of daytime appearance (snapshot), which is daytime appearance, is smaller than the number of times of nighttime appearance (snapshot). Time range of the day: 7: 00 to 19: 00, time range of night: 19: 00-8: 00(+1 day).

All the snapshot portraits in one cluster file are compared with 2 time periods in corresponding snapshot time, the statistical travel times in the daytime period are N1, the statistical travel times in the nighttime period are N2, and N2/(N1+ N2) ═ N% is the proportion of the travel times in the nighttime period to the total times, namely the night travel index.

When the night index is greater than 50%, the cluster file of the citizen or the vehicle is considered to be updated every time, and the night index is updated accordingly.

In this embodiment, the generation of the cluster archive is formed by the acquisition of image information.

Specifically, as shown in fig. 3, generating the cluster archive includes:

Step S201: acquiring first image information of an acquisition object;

step S202: determining object feature classes to which acquisition objects belong, wherein the acquisition objects belong, the object feature classes are contained in the first image information, and each object feature class corresponds to one acquisition object;

Step S203: generating the clustering archive according to the object feature class;

Step S204: and clustering archives of different acquisition objects form the clustering archive library.

the acquisition object includes but is not limited to a human being, and in other embodiments, the acquisition object may also be a vehicle, a low-altitude aircraft, or a self-moving robot, etc.

in the present embodiment, the first image information of the image-captured objects is captured by using cameras distributed in different areas. The first image information may be a single frame image, a video, a sequence frame, or the like.

after the first image information is obtained, object features of the image may be obtained through various feature recognition algorithms and the like, for example, taking the acquisition object as an example, the object features may include: the human face feature, the height feature and other external apparent features can be acquired through the image.

In other embodiments, if the first image information is information acquired by a specific device, features that can be used for identifying the type of the object within the appearance such as bone features of a human body can be acquired.

After the object features are obtained through image recognition and other modes, the collected objects collected by the first image information are classified to obtain the object feature classes of the collected objects. All object features contained in one object feature class are considered as features of the same acquisition object.

For example, a plurality of face features obtained from the plurality of pieces of first image information are classified, and the face features belonging to the same person are classified into one class, and thus, the class features are all the faces describing the same person.

And after the behavior characteristics of the object are obtained, updating the behavior characteristics to the cluster archives. Therefore, the behavior characteristics of one acquisition object can be continuously updated to the corresponding cluster archives according to the latest acquired first image information. And further classifying the behavior characteristics into the clustering archives of the objects. The behavioral characteristics may include: the type, number of times an action is performed by the object, the location and/or time at which the action occurred are collected.

In some embodiments, the cluster archive further comprises: the object features are used for archiving the first image information acquired subsequently so as to continuously and dynamically generate the cluster archive.

For example, in some embodiments, generating or updating the cluster archive may further include:

Acquiring object characteristics of a target object in a plurality of pieces of first image information;

clustering the first image information according to the object features to obtain at least one object feature class, and determining the object feature class to which each first image information belongs.

in some embodiments, said generating the cluster profile according to the behavior feature includes:

Acquiring acquisition information of the target object in each first image information in the object feature class; wherein the collecting information at least comprises: and collecting space-time information.

And generating a cluster archive corresponding to the object feature class according to the plurality of pieces of first image information corresponding to the object feature class and the acquisition information of the acquisition object in each piece of first image information.

In some embodiments, the method further comprises:

Acquiring identity information including second image information; the second image information may be information acquired by the acquisition object in advance, for example, the second image information may be identification card information of a person, vehicle license information of a vehicle, or the like;

determining an object feature class matched with the second image information according to the object features of the second image information;

In some embodiments, the method further comprises:

taking the cluster archives corresponding to the object feature classes as cluster archives of the acquisition objects;

providing the cluster profile of the collection object to a user.

In some embodiments, the determining, according to the cluster archive corresponding to the object feature class, the behavior feature of the collected object in a preset time period includes:

and connecting a plurality of acquisition places to form a track of the acquisition object according to the acquisition time of each image information in a preset time period in the cluster file.

In some embodiments, the method further comprises:

Determining a starting point and an end point of the track of the acquisition object;

And displaying the acquisition position of the acquisition object in the image information, and the starting point and the end point of the track on an electronic map.

In still other embodiments, the method further comprises:

Under the condition that any one acquisition place in the electronic map receives a viewing instruction, displaying one or more information of acquisition time, acquisition position information, a face image of an acquisition object, a plurality of image information acquired within a preset acquisition time period and a video source corresponding to the image information, wherein the acquisition time, the acquisition position information, the face image of the acquisition object correspond to the acquisition place; wherein the preset acquisition time period comprises the acquisition time of the image information of the acquisition place.

in some further embodiments, the determining, according to the cluster archive corresponding to the object feature class, the behavior feature of the collected object in a preset time period includes:

And counting the image acquisition times of the acquisition object in each acquisition place within a preset time period according to the acquisition information corresponding to each image information in the cluster file.

in still other embodiments, the method further comprises:

and determining the frequent place of the acquisition object in a preset time period according to the image acquisition times of the acquisition object in each acquisition place.

And counting the image acquisition times of the acquisition object in each preset statistical period in a preset time period according to the acquisition information corresponding to each image information recorded by the cluster archive.

in still other embodiments, the method further comprises:

determining the image acquisition times of the acquisition object in each statistical time period of each preset statistical period in a preset time period according to the acquisition time of the image information in each preset statistical period;

And determining the total image acquisition times accumulated in each statistical time period corresponding to a plurality of preset statistical periods according to the image acquisition times of the acquisition object in each statistical time period.

Acquiring a plurality of video sources of the image information in the cluster archive;

Determining a preset number of video sources with the maximum image acquisition times of the acquisition object according to the number of the image information in each video source;

and determining the frequently-occurring region of the acquisition object according to the image acquisition regions corresponding to the preset number of video sources.

the above is a specific implementation manner of generating or updating the cluster archive, but the specific implementation manner is not limited to the above.

In the embodiment of the present disclosure, the number of occurrences of the target object in the time range indicated by the time condition exceeds a first threshold.

Some illegal actors such as thieves prefer to go out at night and night, and the frequency of the people in the night is higher than that in the day. Therefore, non-thieves who normally move in the daytime can be filtered from the preset pair of objects by dividing the first time interval and the second time interval, and therefore information of the target object to be searched can be rapidly filtered.

In some embodiments, the first frequency of occurrence and the second frequency of occurrence together satisfy the preset condition, including:

and the ratio of the second occurrence frequency to the sum of the first occurrence frequency and the second occurrence frequency is greater than a ratio threshold.

the ratio threshold may be any predetermined threshold, and a specific value may be 0.5 to 1, for example, 0.6, 0.7, or 0.8.

for example, assume the first frequency of occurrence is F1; the second frequency of occurrence is F2; the ratio is: F2/(F1+ F2); if the ratio threshold is 0.6, if F2/(F1+ F2) is greater than 0.6, the first frequency of occurrence and the second frequency of occurrence are considered to jointly satisfy a preset condition.

in other embodiments, the first frequency of occurrence and the second frequency of occurrence together may satisfy the preset condition further:

The first threshold and the second threshold may be preset values, for example, default thresholds of a security application or a security system, or thresholds received from a User Interface (UI) Interface.

in this embodiment, the units of the first threshold and the second threshold may be: the number of times per unit time, for example, the first threshold value may be: one said first period of time M times; one said second period of time N times; wherein M and N are positive integers, and M is less than N.

In a word, in this embodiment, the occurrence frequency in the first time period and the occurrence frequency in the second time period in the target space-time are comprehensively considered, the collected objects in the security accident high-occurrence time end are rapidly screened out, and compared with the method for temporarily checking image information when retrieval is needed, the method has the characteristics of high efficiency and high recall rate.

For example, the first threshold and the second threshold may be both preset and may be received from a UI interface. For example, if the first threshold is 3 and the second threshold is an integer of 4 or more than 4, then in step S102, based on the search of the cluster file, it can be quickly determined that the number of times is less than 3 in the first time interval within the time range indicated by the time condition, and the number of times is less than 4 in the second time interval.

in summary, the frequency of occurrence of the preset events of the first time period is higher than the frequency of occurrence of the preset events of the second time period. The preset event includes, but is not limited to, a security event.

In this embodiment, the predetermined operation in step S103 may be various types of operations, including but not limited to at least one of the following:

A prompt operation, for example, to prompt that the target object may be an incident maker of a particular event, and/or to prompt that the target object appears, which may require security focus attention;

And an output operation of outputting at least part of information of the target object, for example, identity attribute information such as name and/or age, and historical behavior information such as a population registration place and historical criminal behaviors.

In this embodiment, the predetermined operation may be an operation performed by the electronic device according to a built-in instruction, or an operation performed according to a requirement of cluster file retrieval or an input operation of a user.

In short, in the embodiment, the cluster archive is located after the time condition is obtained, and then the predetermined operation is executed after the cluster archive is directly found, so that the retrieval efficiency, the recall rate and the like of the target object are greatly improved.

In some embodiments, the step S103 may include:

Filtering to obtain a target object from the acquisition objects based on filtering information;

And outputting early warning information based on the clustering archives of the target objects.

The filtering information may be information for further performing filtering on the target objects, and a target object that may better meet a certain query requirement is further selected from the selected target objects.

The filtering information can be received from the UI interface or other devices; or may be automatically analyzed by the electronic device from a particular event.

The early warning reminder can be warning information in various forms, such as text early warning, sound early warning and the like, and the user receiving the early warning reminder can strengthen the precaution.

further, the filtering information may be various types of information for filtering a part of the objects. The filtering information includes, but is not limited to: temporal filtering information, regional filtering information, identity attribute filtering information, and/or event attribute filtering information.

in some embodiments, the outputting early warning information based on the cluster profile of the target object includes at least one of:

Selecting, from the target objects, the target objects of the highly stationary population having the first frequency of occurrence within the target acquisition space above the first threshold based on resident demographic information;

Selecting the target objects of suspected criminal actors who have the first occurrence frequency higher than the first threshold within a predetermined time period of the target time range from the target objects based on criminal pre-record information;

selecting, from the target objects, the target objects of suspected offenders whose first frequency of occurrence is higher than the first threshold at the period and/or place of occurrence of the monitoring event pointed to by the security event information, based on security event information;

Selecting the target object with the first occurrence frequency higher than the first threshold value in a special period from the target objects based on the criminal antecedent record information, the security event information or the area information of the security deployment and control key area.

The resident population information can be obtained by inquiring from the household registration record of the public security organization, can be obtained from a registration system registered by the floating population, and can be obtained by statistics from the daily statistical information of the user.

for example, in some embodiments, the monitoring camera or other image acquisition devices are used to acquire the flow of past people in a specific area, and an acquisition object whose frequency is higher than a third threshold value in a specific time length range in the specific area is regarded as a resident population, and resident population information of the acquisition object is established; therefore, the resident population can be conveniently located without depending on an external system.

In short, in the present embodiment, the resident population in the time range corresponding to the entry and exit time condition which is often legal can be excluded as the cause of the security accident, and the like, by the resident population information.

The pre-criminal record information may be queried from a public security system, for example, including but not limited to: illegal record information, crime record information, illegal crime prosecution and non-success information, and the like. In a word, the criminal antecedent record information can be focused and screened aiming at scofflaw and the like, so that the determination rate of the target object is improved.

the security and protection distribution and control area can be set according to security and protection levels, and the area with the security and protection level larger than the preset level can be regarded as the security and protection distribution and control key area, or the security and protection distribution and control key area can be determined according to the type of security and protection distribution and control.

The security event information may be information of an occurred security event.

the special period may be an emergency period set by a state or a regional government, or a specific period such as an accident handling period when a major safety accident or a security accident occurs. Selecting a target object with a first occurrence frequency higher than a first threshold value in a special period as a target object based on the pre-criminal record information; since these target objects have past subjects, the probability of a possible culprit being a trigger event that caused a particular period of time to be initiated is high.

If the occurrence frequency of the security event pointed by the security event information in the occurrence time and space of the security event is drafted in a special period, or people with high occurrence frequency in a key control area are all possible objects needing key attention.

The security event information includes at least one of: terrorist event information; information of the medical alarm incident; information of group fighting events, information of intentional injury events and information of public safety hazards;

and/or the presence of a gas in the gas,

the security and protection deployment and control key area comprises at least one of the following parts:

A cultural relic key protection area;

A military protective area;

And reserving the engineering construction area.

The predetermined engineering construction area may be: areas currently undergoing major engineering, such as hydraulic engineering, military engineering, road engineering, bridge engineering, etc. Important protection is required because important materials or secret concerns required for construction may exist in the engineering construction areas.

Further, the step S103 may include: outputting at least part of the information of the target object based on the cluster archive of the target object.

the at least part of the information may be from within the cluster file or from outside the cluster file, distinguished by the source of the information.

The at least part of the information comprises: at least part of the activity information contained by the cluster profile and/or identity attribute information of the collection object described by the cluster profile.

in some embodiments, the at least part of the information may retrieve information of a cluster profile, such as a cluster profile number of a cluster profile of the target object.

therefore, through the output of the partial information in the clustering file, the information of the target object can be conveniently and rapidly searched by working personnel such as public security and the like.

In some embodiments, the step S103 may include:

And if the number of the target objects is at least two, sequentially outputting at least part of information of the target objects according to the frequency of occurrence from high to low.

For example, if more than one target object such as a person or a vehicle repeatedly appearing in the C-cell is present within the time range indicated by the time condition, in order to better facilitate tracking of a suspect by security personnel such as public security, the appearance frequency within the time range corresponding to the time condition is sorted, and then at least a part of information of the cluster file of the target object is output in an output order from high to low at a time. Thus, generally going from high to low, the probability of a perpetrator of a predetermined event, such as a theft event, is higher.

further, if there are at least two target objects, sequentially outputting at least part of information of the target objects according to the frequency of occurrence from high to low, including:

If the number of the target objects is at least two, and the occurrence frequency of at least two target objects is equal, sequentially outputting at least part of information in the cluster files of at least two target objects with equal occurrence frequency according to the occurrence time sequence of the last occurrence of the target object in the time range.

In this embodiment, there are at least two target objects whose occurrence frequencies are the same, and at this time, the occurrence times of the last occurrence of the target objects in the time range indicated by the time condition are time-sorted to obtain the occurrence time sequence, and then at least part of the information output of the target objects is sorted according to the occurrence time sequence.

the appearance time sequence can be from morning to evening or from evening to morning; therefore, through the appearance time sequence, the output of at least part of information of the target objects with equivalent appearance frequency is solved, and on the other hand, sequencing according to the appearance time sequence can arrange and sequence the target objects with larger suspicion in front with higher probability so as to accelerate the investigation of the suspicion objects.

in some embodiments, the method further comprises:

Selecting a second clustering archive from the first clustering archive according to the selection information;

the step S102 may include:

And determining the clustering archive of the target object from the second clustering archive according to the target object screening condition.

in some embodiments, the electronic device also obtains selection information that can be used to at least initially filter out portions of the cluster profile that do not require retrieval.

In some embodiments, the selection information comprises at least one of:

Library information of the second hierarchical archive;

Library information of a third hierarchical archive, wherein the third hierarchical archive is: a cluster archive in the first cluster archive other than the second cluster archive.

the repository information of the second category archive may be: for example, receiving library information from the UI, thus corresponds to the user directly indicating in which libraries to search for the cluster archive.

the library information of the third class archive is information of excluded libraries, for example, there are M libraries excluding one or more libraries, and the rest of all or part of the libraries is additionally the second class archive.

The second class archive can be directly selected by the selection information including the library information of the second class archive. For example, the cluster archive in which the cluster archive of the criminal is located can be selected as the second class archive by the selection information of the library information of the cluster archive containing the criminal.

The second-class archive can be obtained by excluding the third-class archive from the existing cluster archive by including the exclusion information of the library information of the third-class archive.

For example, the cluster archives of users of different cells are divided into different cluster archives according to the residences. For example, the cluster archives of resident residents in the cell a are all located in the cluster archive a; if the cluster archive a needs to be excluded when determining illegal members such as theft frequently entering or exiting the cell a by the method provided by the embodiment of the present application, the cluster archive a here is the third class archive.

For example, the selection information may include: library selection information of a database, and equipment selection information of the image acquisition equipment of the cluster archive is generated; object selection information of the acquisition object, and the like.

For example, if the cluster file needs to search for information of a car, the type of the collection object is car, but not human, and only the cluster file of the car needs to be searched at this time, but not the cluster file of human.

for another example, the library storing the cluster archive is divided into a static library and a dynamic library, and the static library can store first-type information such as static information or approximate static information of the acquisition object; and the dynamic library stores the dynamic information of the acquisition object. For example, the static information includes: the identity attribute information, gender information in the identity attribute information, etc. may be static information, while the age information or frequent-premises information may be one that is approximately static information that does not change for a relatively long period of time (e.g., for at least one year). And the activity track of the collected object in one month and the like which may be dynamically changed are stored in the dynamic library.

in short, the acquisition of the selection information may select a part of the cluster archives adapted to the selection information from a large number of selectable cluster archives as candidate cluster archives for performing the time condition search.

Finally, a target cluster profile for the target object is selected based on the temporal condition.

The target cluster profile is a cluster profile of the target object, which is a cluster profile that needs to be focused on, and is a cluster profile that may execute the predetermined operation in step S103.

by introducing the selection information, the retrieval efficiency can be improved again.

In some embodiments, the method may further comprise:

Selecting an alternative clustering archive from the second clustering archive according to the selection information;

Determining a cluster profile of target objects that at least meet the first threshold of the temporal condition from the candidate cluster profiles; or, determining a cluster archive of target objects satisfying the time condition and the first threshold of the spatial condition from the candidate cluster archives

The archive directing information of the alternative clustering archives, wherein the archive directing information comprises: at least one of a file name, a file number, and a file type;

and the alternative clustering archives describe label information of the acquired objects.

the cluster profile pointing information includes, but is not limited to: the cluster archive number, the identity attribute of the collection object contained in the cluster archive, and other indication information for excluding the cluster archive or selecting the cluster archive.

The cluster profile pointing information includes, but is not limited to: the cluster archive number, the identity attribute of the collection object contained in the cluster archive, and other indication information for excluding the cluster archive or selecting the cluster archive.

in this embodiment, the cluster profiles may belong to different cluster profiles, or the information in the cluster profiles may come from different libraries. For example, as shown in FIG. 8, a library containing cluster profiles may include: a portrait library and/or a snapshot library, etc.

In some embodiments, the step S103 further comprises:

determining an activity track of the target object based on the cluster profile of the target object.

in some embodiments, the activity track is at least used for performing security control on the collected object corresponding to the face information with the frequency higher than the second threshold

For example, the activity information of the target object is recorded in the different activity information, and the activity track of the target object can be obtained through analysis according to the geographical position information in the activity information, so that security and protection can be distributed and controlled according to the activity track when tracking of escaped people is carried out.

As shown in fig. 4, the present embodiment also provides an information processing apparatus including:

a first obtaining module 101, configured to obtain target object screening conditions, where the target object screening conditions at least include: a temporal condition, the temporal condition comprising: the method comprises the following steps that a first appearance frequency of a target object in a first time period and a second appearance frequency of the target object in a second time period meet preset conditions together;

A determining module 102, configured to determine cluster profiles of the target object according to the target object screening condition, where one cluster profile includes profile information of an acquisition object, and the profile information includes temporal-spatial range information of an activity of the acquisition object;

and the execution module 103 is used for executing a predetermined operation based on the cluster archive of the target object.

In some embodiments, the first obtaining module 101, the determining module 102 and the executing module 103 may be program modules, which, when executed by a processor, are capable of obtaining the aforementioned time condition, querying the cluster file of the target object and executing the predetermined operation.

In some further embodiments, the first obtaining module 101, the determining module 102, and the executing module 103 may be a hardware-software combination module, which may include: various programmable arrays; the programmable array includes, but is not limited to, a complex programmable array or a field programmable array.

In some embodiments, the first obtaining module 101, the determining module 102, and the executing module 103 may be pure hardware modules including, but not limited to, application specific integrated circuits.

In addition, in some embodiments, the executing module 103 is specifically configured to output at least part of the information of the target object based on the cluster profile of the target object.

In some further embodiments, the executing module 103 is specifically configured to, if there are at least two target objects, sequentially output at least part of information in the cluster archive of the target objects according to the frequency of occurrence from high to low.

In some embodiments, the first frequency of occurrence and the second frequency of occurrence together satisfy the preset condition, including at least one of:

The ratio of the second occurrence frequency to the sum of the first occurrence frequency and the second occurrence frequency is greater than a ratio threshold;

in some embodiments, the first time period is a time period in which a preset event occurs more frequently than the second time period.

In some embodiments, the execution module is specifically configured to filter the target object from the acquisition object of the target object based on filtering information to obtain the target object; and outputting early warning information based on the clustering archives of the target objects.

In some embodiments, the execution module is specifically configured to execute at least one of:

Selecting a non-resident population as a target object, wherein the first occurrence frequency and the second occurrence frequency in the target collection space jointly meet the preset condition, from the target object on the basis of resident population information;

based on the pre-criminal record information, selecting a suspected criminal activity person whose first occurrence frequency and second occurrence frequency meet the preset condition in a preset time period of the target time range from the target objects as the target object;

selecting a suspected troubled hit person, as the target object, from the target object, the first frequency of occurrence and the second frequency of occurrence of the first frequency of occurrence and/or the second frequency of occurrence of the monitoring event, to which the security event information points, together meeting the preset condition;

and selecting the target objects, of which the first occurrence frequency and the second occurrence frequency in a special period meet the preset condition, from the target objects based on the record information of the criminal antecedent, the security event information or the area information of the security deployment and control key area.

In some embodiments, the resident demographic information is: the acquisition objects whose frequency of occurrence within a human-specific time period in the space indicated by the time condition is higher than a third frequency threshold.

In some embodiments, the security event information includes at least one of: terrorist event information; information of the medical alarm incident; information of group fighting events, information of intentional injury events and information of public safety hazards;

and/or the presence of a gas in the gas,

the security and protection deployment and control key area comprises at least one of the following parts:

A cultural relic key protection area;

A military protective area;

and reserving the engineering construction area.

In some embodiments, the execution module is specifically configured to output at least part of the information of the target object based on the cluster profile of the target object.

in some embodiments, the execution module is specifically configured to, if there are at least two target objects, sequentially output at least part of information of the target objects according to the frequency of occurrence from high to low.

In some embodiments, the execution module is specifically configured to, if there are at least two target objects and there are at least two target objects whose occurrence frequencies are equal, sequentially output at least part of information in the cluster files of the at least two target objects whose occurrence frequencies are equal according to an occurrence time sequence of the last occurrence of the target object in the time range.

In some embodiments, the executing module 103 is specifically configured to, if there are at least two target objects and the occurrence frequencies of at least two target objects are equal, sequentially output at least part of information in the cluster files of at least two target objects with equal occurrence frequencies according to an occurrence time sequence of the last occurrence of the target object in the time range.

In some embodiments, the executing module 103 is specifically configured to determine an activity track of the predetermined object based on the cluster profile of the target object.

In some embodiments, the apparatus further comprises:

The first selection module is used for selecting a second clustering archive from the first clustering archive according to the selection information;

the determining module 102 is configured to determine a cluster archive of the target object from the second cluster archive according to the target object screening condition.

In some embodiments, the selection information comprises one or more of:

Library information of the second hierarchical archive;

library information of a third hierarchical archive, wherein the third hierarchical archive is: a cluster archive in the first cluster archive other than the second cluster archive.

In some embodiments, the apparatus further comprises:

The first image module is used for acquiring first image information of the acquisition object;

The first object feature class module is used for determining an object feature class to which the acquisition object belongs, wherein the object feature class is contained in the first image information, and each object feature class corresponds to one acquisition object;

the generating module is used for generating the clustering file according to the object feature class;

And the forming module is used for forming the cluster archive library of different acquisition objects.

in some embodiments, the apparatus further comprises:

The identity information module is used for acquiring identity information comprising second image information;

The second object feature class module is used for determining an object feature class matched with the second image information according to the object features of the second image information;

several specific examples are provided below in connection with any of the embodiments described above:

Example 1:

a population library (static library) with citizen identity is used as a reference library, face snapshot pictures with time-space information are captured by a snapshot machine for clustering, pairwise similarity is used as a judgment standard, information suspected of being the same person in a face recognition system is correlated, and therefore a person has a unique comprehensive clustering file. From the cluster archive, the attribute features, behavior features, etc. of the potential suspect can be obtained.

the class generated by clustering is a collection of a set of data objects that are similar to objects in the same class and different from objects in other classes.

And extracting the content with the space-time information through the details of the cluster archives formed after the cluster hits the library, and finding out the cluster archives of which the number of the snap shots exceeds a threshold value in some video sources in a specified time range. These information departments are directly used for efficiently extracting the personnel clustering archives with frequent passing-person characteristics.

and carrying out face clustering on a snapshot library formed by a figure captured by the snapshot machine and a static library formed by population information of real names of citizens, and then colliding the libraries to obtain all clustering files of one person in the system.

and (4) performing condition screening on all clustered (including real names and non-real names) clustered archives to find out a certain person clustered archive with the number of the same person snapshot exceeding a certain threshold in a specified time range from a specified video source.

after the cluster file is obtained, the user can quickly find out people with the frequency higher than the threshold value in a certain time period in a certain area so as to judge whether certain people meet the condition of passing people frequently.

In a possible implementation manner, the image to be processed may include a picture or a video frame, where the picture or the video frame includes a human face. For a plurality of images to be processed, the images can be classified into one or more clusters according to factors such as human faces, time, places and the like, wherein the clusters are preliminary classifications of the images to be processed, and one cluster can contain one or more images to be processed. For example, the same person is subjected to image acquisition for multiple times at different times, and pictures of the same person can be divided into multiple clusters according to different acquisition times.

As shown in FIG. 6, the process of establishing a cluster archive may include:

Generating a snapshot library by using pictures grabbed by the snapshot machine;

Acquiring the corresponding characteristics of each picture;

clustering the features based on the feature distance to obtain an object feature class;

The snap pictures are classified according to people.

Fig. 5 illustrates a schematic diagram of an application scenario of an information processing method according to an embodiment of the present disclosure. As shown in fig. 2, for the image to be processed 21, feature extraction may be performed first to obtain an object feature 22 thereof; the image class 24 of the image to be processed may then be determined by Faiss 29 from the object features 21 and the class-center features 23 of the plurality of reference image classes in the feature library. When the image category 24 of the image to be processed 21 is the first category 26 of the multiple reference image categories, the class center feature of the first category 26 may be updated according to the object feature 21 and multiple feature information of the first category 26 in the feature library. When the image class 24 of the image to be processed 21 is the class-free class 25, the clustering operation 27 may be performed on the processed image 21 by means of the Faiss 29, and according to the clustering result, a new image class (second class) 28 may be determined, and the class-center feature of the new image class 28 is added to the feature library, and the new image class 28 is added to the plurality of reference image classes.

According to the information processing method, the image type of the image to be processed can be determined according to the feature information of the image to be processed and the class center features in the feature library, the feature library and the image type are updated, clustering of the image to be processed is achieved, and the retrieval speed and the recall rate of image retrieval, particularly face retrieval, can be improved; and a personnel clustering archive can be automatically constructed, so that the image utilization rate is improved.

in one possible implementation, the information processing method can be used for processing the image to be processed in real time or periodically. For example, image processing may be performed once a day or a week, or may be initiated after a certain number of images to be processed have been acquired, or may be performed before picture retrieval is performed. The present disclosure does not limit the start timing of image processing.

In a possible implementation manner, feature extraction is performed on an image to be processed to obtain object features of the image to be processed. The object feature may include one or more feature information of the image to be processed, for example, the object feature includes a plurality of feature information of a human face. After the feature extraction is performed on the image to be processed, the extracted feature may be used as the object feature thereof. The present disclosure does not limit the manner of feature extraction.

in a possible implementation manner, feature extraction is performed on an image to be processed to obtain a second feature of the image to be processed; and carrying out normalization processing on the second characteristic to obtain the object characteristic of the image to be processed. The normalization process can summarize and unify the feature information, and unify the feature values in a certain range. The normalization process may, for example, comprise a regularization process, and the present disclosure does not limit the specific manner of normalization.

And normalizing the second characteristic of the image to be processed, and taking the normalized characteristic value as the object characteristic of the image to be processed, so that the characteristic values of the object characteristic are all in a certain range, thereby reducing the complexity of calculation and improving the calculation efficiency.

in a possible implementation manner, the image category of the image to be processed is determined according to the object feature and the class center feature of a plurality of reference image categories in the feature library. Where the reference image category may be a classified image category in the feature library, an image category may be a collection of images of a certain category, such as a collection of images of a person. For each reference image class, a class center feature may be determined. The image category of the image to be processed can be determined according to the object features of the image to be processed and the class center features of a plurality of reference image categories in the feature library.

in one possible implementation, before step S11, the method may further include: and respectively carrying out class center extraction on the feature information of each reference image class in the feature library to obtain the class center features of each image class. That is, for each reference image category in the feature library, class center extraction may be performed on feature information of each reference image category, and the extracted feature information may be used as a class center feature of each reference image category. The present disclosure does not limit the method of class center extraction.

In one possible implementation, step S12 may include: obtaining a plurality of first distances between the object feature and a plurality of class center features; and determining the image category of the image to be processed as the first category corresponding to the second distance when the second distance with the smallest distance value in the plurality of first distances is less than or equal to the distance threshold value. The distance threshold value can be preset, and the value of the distance threshold value is not limited by the disclosure.

In one possible implementation manner, distances between the object feature of the image to be processed and the plurality of class center features may be calculated respectively, so as to obtain a plurality of first distances. And taking the first distance with the smallest distance value as the second distance in the plurality of first distances. And then, judging the relation between the second distance and a preset distance threshold value. If the second distance is less than or equal to the distance threshold, the image category of the image to be processed may be determined as the first category corresponding to the second distance.

in one possible implementation, the class-centered feature includes N class-centered features, where obtaining a plurality of first distances between the object feature and a plurality of class-centered features includes: respectively carrying out quantization processing on the N class center features to obtain N feature vectors; respectively acquiring N third distances between the object feature and the N feature vectors; determining K class center features corresponding to the K minimum third distances of the N third distances; determining K first distances between the object feature and the K class center features, K being a positive integer and K < N.

in a possible implementation manner, quantization processing may be performed on the N class center features, respectively, to obtain N feature vectors. For example, the N class-centered features may be quantized using the IVFADC algorithm in Faiss (Facebook AI Similarity Search, which is an open source Similarity Search library provided by Facebook), wherein the IVFADC algorithm includes a coarse quantizer (e.g., a k-means algorithm) and a product quantizer. The N class-center features may be first coarsely quantized using a coarse quantizer (e.g., K-means algorithm), the N class-center features are divided into P groups (P is a positive integer and K < N), the quantization centers of each group are calculated separately, and the residual vectors of each vector and the quantization center in the group are calculated; and then, performing product quantization on each residual vector by using a product quantizer, dividing the D-dimensional residual vector into M (M < D) sub-vectors along the dimension, performing rough quantization on one sub-vector, compressing the D-dimensional residual vector to M dimensions, and thus obtaining N M-dimensional feature vectors corresponding to N class center features.

In one possible implementation, N third distances between the object feature and the N feature vectors may be obtained respectively. For example, N third distances between the object feature and the N feature vectors may be calculated using the asymmetric distances, wherein the third distances are approximate distances (e.g., approximate euclidean distances).

In one possible implementation, the third distance may be calculated using the following equation (1):

In formula (1), x represents an object feature, y represents a class center feature, q represents quantization processing, q1 represents a coarse quantizer, q1(y) represents a quantization result (quantization center) of the coarse quantizer, q2 represents a product quantizer, q2(y-q1(y)) represents a result of product quantization, and its input y-q1(y) represents a residual of y and the quantization center.

In a possible implementation manner, the smallest K third distances may be selected from the N third distances, and K class center features corresponding to the K third distances may be determined. For the K class center features, the precise distance (e.g., inner product distance) between the object feature and each class center feature may be calculated, and the calculation result may be used as K first distances between the object feature and the K class center features.

Through the method, the quantization processing can quantize and reduce the dimension of the N class center features, K class center features in the N class center features are used for calculating the first distance, the operation amount can be reduced, and the calculation speed of the plurality of first distances can be improved.

in one possible implementation, in a case that the second distance is greater than the distance threshold, the image category of the image to be processed is determined as the second category. Wherein the second category is a new image category other than the plurality of reference images. That is, in the case where the second distance is greater than the distance threshold, a new image category (second category) may be determined for the image to be processed, thereby improving the accuracy of image classification.

In a possible implementation manner, in a case that the image category of the image to be processed is a first category in the plurality of reference image categories, the class center feature of the first category may be updated according to the object feature and a plurality of feature information of the first category in the feature library. In this way, the center-like feature of the reference image class in the feature library may be updated as new images are added.

in one possible implementation, the method may further include: under the condition that the image category of the image to be processed is a second category except the multiple reference image categories, carrying out class center extraction on the object feature of the image to be processed to obtain the class center feature of the second category; adding the object features and the class-centered features of the second class to the feature library, and adding the second class to a plurality of reference image classes.

in one possible implementation, if the image class of the to-be-processed image does not belong to any of the plurality of reference image classes, the plurality of to-be-processed images without classes may be clustered into a new class (second class). In this case, class center extraction may be performed on the object feature of the image to be processed to obtain the class center feature of the second category. For example, according to the object feature of an image to be processed, searching (for example, searching by Faiss) in other multiple images to be processed can be performed to obtain the top K similarity results; for K similarity results, the clustering may be determined by plotting an affinity graph to find the joint flux, or by DFS (Deep First Search) recursive staining, where whether to stain may be determined according to a similarity threshold, e.g., if the similarity threshold is 0.7, staining is performed on similarity results greater than the similarity threshold, and skipping is performed on similarity results less than the similarity threshold. After a plurality of to-be-processed images to be clustered are determined, class center extraction can be performed on object features of the to-be-processed images, and the object features are used as class center features of a second class.

In one possible implementation, the object features of the image to be processed and the class-centered features of the second class may be added to the feature library, and the second class may be added to the plurality of reference image classes. In this way, new images and new image classes can be added, thereby updating the feature library and the reference image classes.

fig. 7 is a schematic diagram of clustering features to obtain 5 classes, which respectively obtain class 1, class 2, class 3, class 4, and class 5, where one class may correspond to one person (i.e., one collection object) and one collection object may correspond to one cluster file.

before image retrieval, the images to be processed are clustered, so that the retrieval speed and recall rate of image retrieval, particularly face retrieval, can be improved. For example, face retrieval is an important scenario for solving the case in the public security industry, and a series of information such as the identity of a suspect needs to be determined by retrieving in a massive portrait library according to a suspect picture with an undetermined identity.

Before retrieval, image processing can be carried out on the picture of the suspect to determine the image category (clustering) of the picture; during retrieval, the retrieval speed and the recall rate can be improved by comparing the suspect picture with the class center features, so that more accurate suspect information can be returned more quickly, and the police staff can be helped to quickly study and judge the suspect information and the case.

the information processing method disclosed by the embodiment of the disclosure can realize that clustering of the images to be processed can automatically construct the personnel clustering archive, thereby improving the utilization rate of the images. For example, in a public security information system, a large number of snap-shot pictures exist, after the snap-shot pictures are subjected to image processing, the snap-shot pictures can be gathered into one type by taking human dimensions, integration of the large number of snap-shot pictures and scattered snap-shot pictures is achieved, all the snap-shot pictures related to the same person can be seen in the system, personal tracks are formed, large data analysis is achieved, and case study and judgment are assisted.

the information processing method of the embodiment of the disclosure can realize automatic iteration of the image system through clustering, and for the continuously added images to be processed, the image category and the category center thereof can be updated after clustering each time, so that the system is continuously subjected to incremental training, a positive feedback cycle is formed, and the system capacity is improved.

example 2:

The first step is as follows: and (4) carrying out nearest neighbor search on the new input features and the bottom library class center, and determining whether the new input features belong to the existing bottom library or not by using a distance card threshold value, namely whether the new input features have classes or not.

the second step is that: for the classified features: clustering with the existing category and updating the base class center.

The third step: for class-free features: clustering, determining the category, and adding the new clustering center into the base center.

as shown in fig. 8, the algorithm scheme of face clustering:

Storing the portraits in a portrait library in batches, and aggregating the portraits with identity information such as identification numbers to form an initial clustering file;

in the snapshot library, the corresponding characteristics of each portrait picture are obtained through the snapshot library, the characteristic clusters are close, namely the clustering with high portrait matching degree similarity is carried out, then the snapshot pictures are classified by taking people as units, the activity information formed by the pictures stored in the snapshot library is filed, and the updated cluster file is obtained.

Subsequently, a portrait library and a snapshot are newly added according to requirements.

and clustering the snapshot libraries. The snapshot library clustering is triggered automatically and regularly by the system, for example, once an hour or a day, and the time is configurable. The initial full-scale clustering and the subsequent incremental clustering are aggregated with the existing classes. And no similar classes can be automatically aggregated into a new class.

And aggregating the portrait libraries. And the identity card numbers in the portrait base are the same and are aggregated into a cluster file.

The snapshot library and the portrait library collide with each other. The snapshot library is clustered and then divided into a plurality of classes (people), each class is provided with a class center corresponding to a class center characteristic value. And performing total quantity 1 on each class center characteristic value and the portrait library: n, taking a portrait with the highest similarity TOP1 and larger than a preset value. The identity information corresponding to the portrait of the TOP1 is given to the class of the snapshot library, so that the real name of the portrait is captured.

And (5) incrementally bumping the warehouse. Snapshot library increment: the snapshot library is subjected to daily incremental clustering, and the class center of the new class collides with the portrait library

and (4) adding the portrait library: and carrying out identity information (identification card number) correlation query with the existing portrait library, and if the identity information exists, merging the identity information and the clustering file.

The collision garage shown in fig. 8 is: and matching the information of the same person in different libraries, and associating the information of the same person from different libraries into a cluster file after the matching is successful, so that one person can obtain one file.

for example, if the portrait library has been archived and not captured, the initial cluster archive is retained.

As another example, if the person is captured and named, the captured person is associated with the identity of the person library. The real name here indicates that a cluster archive has been established in the portrait repository. And the picture is taken but is not named actually, which indicates that the portrait library is not filed. If the real name shows that the file is built (the clustering file of the corresponding person is built), filing based on the information obtained by snapshot; if the corresponding information is not stored in real name, the corresponding information is filed and filed when the identity of the person is determined.

When the clustering file of the citizens who go out at daytime and night is screened, auxiliary filtering can be carried out by adopting at least one of the following modes:

The real population management: the method overcomes the defects of missed registration, time consumption, labor consumption and the like of the traditional population registration mode, and determines the community constant population by defining that the community is considered to live in more than X times within a specified time period. Here, X may correspond to the aforementioned first threshold value.

burglary: a person or persons wander around the house all the time, possibly at a stepping point, requiring vigilance. At the moment, strangers repeatedly appear for more than X times within a specified time period, so that suspects can be found from the clustering files → tracks are found → positions are confirmed → deployment and control are implemented for catching after a theft case occurs; or prompt and early warning when a theft case possibly occurs.

anti-terrorist monitoring: in a special period, forepart related to terrorism can travel more than X days at night near some important places, need to pay attention, and can screen out the people for management and control; or after terrorism occurs in these places, the clustering file of such people can be screened out by the day and night acrobatics and tactics, and the clustering file is taken as the key suspect to pay attention.

The key points of national security are as follows: foreheads are near the national security important place to travel more than X days at night, need important attention, and can screen out the people for management and control; or after adverse events occur at these locations, the cluster files of such persons are screened out by the day-night development technique and tactics, and the clustered files are paid attention as key suspects

Medical alarm causing: before medical treatment, the medical personnel need to screen out the defense improvement and control when traveling in the hospital for more than X days; or after a medical alarm event occurs, the clustering files of the people are screened out through a day-night birth technical and tactic method and are taken into the attention of key suspects.

The method and the system automatically combine the snap pictures of the same person in video monitoring with the existing static personnel database, thereby being convenient for police to serially connect clues and improving the solution efficiency.

And when the face picture information of the suspect does not exist, frequently appearing strangers are searched in a specified time and area so as to hope to find the clustering file of the suspect.

As shown in fig. 9, an embodiment of the present application provides an image processing apparatus including:

A memory for storing information;

and the processor is connected with the display and the memory respectively, and is used for implementing the information processing method provided by one or more of the technical schemes by executing the computer executable instructions stored in the memory, for example, at least one of the information processing methods shown in fig. 1, fig. 3, fig. 6 and fig. 8.

The memory can be various types of memories, such as random access memory, read only memory, flash memory, and the like. The memory may be used for information storage, e.g., storing computer-executable instructions, etc. The computer-executable instructions may be various program instructions, such as object program instructions and/or source program instructions, and the like.

the processor may be various types of processors, such as a central processing unit, a microprocessor, a digital signal processor, a programmable array, a digital signal processor, an application specific integrated circuit, or an image processor, among others.

The processor may be connected to the memory via a bus. The bus may be an integrated circuit bus or the like.

In some embodiments, the terminal device may further include: a communication interface, which may include: a network interface, e.g., a local area network interface, a transceiver antenna, etc. The communication interface is also connected with the processor and can be used for information transceiving.

In some embodiments, the terminal device further comprises a human-computer interaction interface, for example, the human-computer interaction interface may comprise various input and output devices, such as a keyboard, a touch screen, and the like.

In some embodiments, the image processing apparatus further comprises: a display that can display various prompts, captured facial images, and/or various interfaces.

The embodiment of the application provides a computer storage medium, wherein computer executable codes are stored in the computer storage medium; after being executed, the computer executable code can implement an information processing method provided by one or more of the foregoing technical solutions, for example, at least one of the information processing methods shown in fig. 1, fig. 3, fig. 6, and fig. 8.

In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above-described device embodiments are merely illustrative, for example, the division of the unit is only a logical functional division, and there may be other division ways in actual implementation, such as: multiple units or components may be combined, or may be integrated into another system, or some features may be omitted, or not implemented. In addition, the coupling, direct coupling or communication connection between the components shown or discussed may be through some interfaces, and the indirect coupling or communication connection between the devices or units may be electrical, mechanical or other forms.

the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed on a plurality of network units; some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, all the functional units in the embodiments of the present invention may be integrated into one processing module, or each unit may be separately used as one unit, or two or more units may be integrated into one unit; the integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit. Those of ordinary skill in the art will understand that: all or part of the steps for implementing the method embodiments may be implemented by hardware related to program instructions, and the program may be stored in a computer readable storage medium, and when executed, the program performs the steps including the method embodiments; and the aforementioned storage medium includes: a mobile storage device, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.

The methods disclosed in the several method embodiments provided in the present application may be combined arbitrarily without conflict to obtain new method embodiments.

the features disclosed in several of the apparatus embodiments provided in the present application may be combined arbitrarily, without conflict, to arrive at new apparatus embodiments.

The features disclosed in the several method or apparatus embodiments provided in the present application may be combined arbitrarily, without conflict, to arrive at new method embodiments or apparatus embodiments.

the above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention, and all the changes or substitutions should be covered within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the appended claims.

29页详细技术资料下载

上一篇：一种医用注射器针头装配设备

下一篇：一种建立人脸索引的方法、装置、处理服务器及存储介质

information processing method and apparatus, image device, and storage medium

相关技术

网友询问留言