A kind of human action search method and equipment based on hierarchical model
阅读说明：本技术 一种基于层次模型的人体动作检索方法及设备 (A kind of human action search method and equipment based on hierarchical model ) 是由 黄天羽 黄晓舟 于 2019-08-28 设计创作，主要内容包括：本发明涉及一种基于层次模型的人体动作检索方法及设备,方法包括以下步骤：从人体运动文件中提取每一帧每个关节点信息,计算该关节点在世界坐标系下的位置；确定位置变化最大的关节点及该关节点的主要运动方向；对该运动文件数据按照层次模型进行编码；将文件运动数据及其层次模型编码存入运动数据库；根据数据库中每个文件对应的层次模型编码为所有文件建立索引树；使用层次模型编码对数据库进行检索。本发明使用基于层次模型的编码保留运动数据的主要几何特征,把对复杂运动数据的检索转化为对简单数字编码的检索,极大的提高了检索的时间效率。(The present invention relates to a kind of human action search method and equipment based on hierarchical model, method calculate position of the artis under world coordinate system the following steps are included: extract each artis information of each frame from human motion file；Determine the maximum artis of change in location and the main movement direction of the artis；The move file data are encoded according to hierarchical model；By file exercise data and its hierarchical model coding deposit motion database；All Files, which are encoded to, according to the corresponding hierarchical model of file each in database establishes index tree；Database is retrieved using hierarchical model coding.The present invention retains the main geometrical characteristic of exercise data using the coding based on hierarchical model, and the retrieval to compound movement data is converted into the retrieval to simple digital coding, greatly improves the time efficiency of retrieval.)
1. a kind of human action search method based on hierarchical model, which comprises the following steps:
The each artis information of each frame is extracted from human motion file, calculates position of the artis under world coordinate system It sets；
Determine the maximum artis of change in location and the main movement direction of the artis；
The move file data are encoded according to hierarchical model, one-level code corresponding position changes maximum artis, second level The main movement direction of the corresponding artis of code, three-level code correspond to motion frequency；
By file exercise data and its hierarchical model coding deposit motion database；
All Files, which are encoded to, according to the corresponding hierarchical model of file each in database establishes index tree, the non-leaf of index tree Node includes the coding of corresponding level, and leaf node contains an index structure inode, records following index information: { level Model based coding, place filename, the pointer of previous inode, the pointer of next inode }；
When retrieving motion database, hierarchical model coding is carried out to the movement of inquiry first, it is similar that coding is found in index tree Leaf node, finally search the similar move file of all codings further according to the pointer information of inode to get to a phase Like motion candidates file set.
2. human action search method according to claim 1, which is characterized in that in accordance with the following methods, determine that position becomes Change maximum artis and the main movement direction of the artis:
Concussion factor S is set separatelyx、Sy、Sz, to record in human body skeletal architecture each artis under world coordinate system X, Y, the maximum displacement of Z-direction；Find out Sx、Sy、SzMaximum value, that is, can determine the maximum artis of change in location and the artis Main movement direction.
3. human action search method according to claim 2, which is characterized in that calculate each artis of each frame and exist Behind position under world coordinate system, the differentiation and extraction of key frame are carried out, the hierarchical model of file is determined using key frame data Coding, and only by key frame exercise data and its hierarchical model coding deposit motion database.
4. human action search method according to claim 3, which is characterized in that after carrying out key-frame extraction, according to The position of each each artis of frame judges the period of motion, and the key frame chosen in a period of motion determines change in location most The main movement direction of big artis and the artis.
5. human action search method according to claim 4, which is characterized in that be filtered place to the data after dimensionality reduction Reason, the current location of more each each data frame of artis maximum change location or minimum change in entire movement with the joint The difference for changing position is considered as interference information if the dynamic change average value that its difference is greater than the artis, i.e., the artis Location information under this data frame is deleted.
6. a kind of storage equipment, which is characterized in that be wherein stored with a plurality of instruction, described instruction is suitable for being loaded and being held by processor Row:
The each artis information of each frame is extracted from human motion file, calculates position of the artis under world coordinate system It sets；
Determine the maximum artis of change in location and the main movement direction of the artis；
The move file data are encoded according to hierarchical model, one-level code corresponding position changes maximum artis, second level The main movement direction of the corresponding artis of code, three-level code correspond to motion frequency；
By file exercise data and its hierarchical model coding deposit motion database；
All Files, which are encoded to, according to the corresponding hierarchical model of file each in database establishes index tree, the non-leaf of index tree Node includes the coding of corresponding level, and leaf node contains an index structure inode, records following index information: { level Model based coding, place filename, the pointer of previous inode, the pointer of next inode }；
When retrieving motion database, hierarchical model coding is carried out to the movement of inquiry first, it is similar that coding is found in index tree Leaf node, finally search the similar move file of all codings further according to the pointer information of inode to get to a phase Like motion candidates file set.
7. storage equipment according to claim 6, which is characterized in that described instruction in accordance with the following methods, determines that position becomes Change maximum artis and the main movement direction of the artis:
Concussion factor S is set separatelyx、Sy、Sz, to record in human body skeletal architecture each artis under world coordinate system X, Y, the maximum displacement of Z-direction；Find out Sx、Sy、SzMaximum value, that is, can determine the maximum artis of change in location and the artis Main movement direction.
8. storage equipment according to claim 7, which is characterized in that described instruction calculates each artis of each frame and exists Behind position under world coordinate system, the differentiation and extraction of key frame are carried out, the hierarchical model of file is determined using key frame data Coding, and only by key frame exercise data and its hierarchical model coding deposit motion database.
9. storage equipment according to claim 8, which is characterized in that described instruction after carrying out key-frame extraction, according to The position of each each artis of frame judges the period of motion, and the key frame chosen in a period of motion determines change in location most The main movement direction of big artis and the artis.
10. storage equipment according to claim 9, which is characterized in that described instruction is filtered the data after dimensionality reduction Processing, the current location and the joint of more each each data frame of artis maximum change location or minimum in entire movement The difference of change location is considered as interference information if the dynamic change average value that its difference is greater than the artis, i.e., the joint Location information of the point under this data frame is deleted.
Technical field
The present invention relates to a kind of data retrieval method and equipment, in particular to a kind of human action inspection based on hierarchical model Rope method and apparatus.
Background technique
As the rise of capturing movement technology and all kinds of optics, work capture the progress of equipment, present people Have been able to a large amount of human action three-dimensional data files of quick obtaining.Since human action three-dimensional data file can more precisely The total movement track recorded in experimenter's each period, human body can be obtained by capturing resulting data by analysis movement The elaboration of movement greatly improves the convenience of action data related work acquisition and the reliability of data.Move number According to multiplexing and the movement capturing technology that is established as of extensive motion database provide a kind of more time saving and economic scheme, Meanwhile also to the tissue of motion database and search technique, more stringent requirements are proposed.
Motion retrieval technology is to realize a key technology of movement capturing data multiplexing.Human action sequence is a kind of allusion quotation The higher-dimension time series of type, the processing for high dimensional information, if a large amount of operation will be expended by carrying out retrieval using conventional method Time and memory headroom.Therefore, choosing suitable character representation method makes retrieval rate and retrieval quality that can receive to show It obtains extremely important.The existing retrieval to motion database is mainly realized by extracting geometrical characteristic and calculating Euclidean distance , its purpose is to realize the retrieval to motion database content, but the calculating to complex geometry feature and Euclidean distance So that needing to expend a large amount of runing time when database retrieval, it is unable to satisfy real-time demand.
Summary of the invention
In view of the above-mentioned deficiencies in the prior art, it is an object of the present invention to provide a kind of human action retrieval based on hierarchical model Method and apparatus realizes that the exercise data based on content is retrieved using new geometrical characteristic, improves the time of conventional retrieving methods Efficiency.
In order to achieve the above objectives, the human action search method based on hierarchical model that the present invention provides a kind of, including with Lower step:
The each artis information of each frame is extracted from human motion file, calculates the artis under world coordinate system Position；
Determine the maximum artis of change in location and the main movement direction of the artis；
The move file data are encoded according to hierarchical model, one-level code corresponding position changes maximum artis, Second level code corresponds to the main movement direction of the artis, and three-level code corresponds to motion frequency；
By file exercise data and its hierarchical model coding deposit motion database；
Be encoded to All Files according to the corresponding hierarchical model of file each in database and establish index tree, index tree it is non- Leaf node includes the coding of corresponding level, and leaf node contains an index structure inode, records following index information: { hierarchical model coding, place filename, the pointer of previous inode, the pointer of next inode }；
When retrieving motion database, hierarchical model coding is carried out to the movement of inquiry first, coding is found in index tree Similar leaf node finally searches the similar move file of all codings further according to the pointer information of inode to get to one A similar movement alternative file set.
Preferably, in accordance with the following methods, the maximum artis of change in location and the main movement direction of the artis are determined:
Concussion factor S is set separatelyx、Sy、Sz, to record in human body skeletal architecture each artis in world coordinate system The maximum displacement of lower X, Y, Z-direction；Find out Sx、Sy、SzMaximum value, that is, can determine the maximum artis of change in location and the pass The main movement direction of node.
Preferably, each artis of each frame is calculated behind the position under world coordinate system, carries out the differentiation of key frame With extraction, determine that the hierarchical model of file encodes using key frame data, and only by key frame exercise data and its level mould Type coding deposit motion database.
Preferably, after carrying out key-frame extraction, the period of motion is judged according to the position of each artis of each frame, and select The key frame in a period of motion is taken to determine the maximum artis of change in location and the main movement direction of the artis.
Preferably, the data after dimensionality reduction are filtered, the current location of more each each data frame of artis With the joint in entire movement the difference of maximum change location or minimum change position, if its difference is greater than the dynamic of the artis Variation average value is then considered as interference information, i.e., the location information the artis under this data frame is deleted.
The present invention also provides a kind of storage equipment for human action retrieval, wherein it is stored with a plurality of instruction, it is described Instruction is suitable for being loaded and being executed by processor:
The each artis information of each frame is extracted from human motion file, calculates the artis under world coordinate system Position；
Determine the maximum artis of change in location and the main movement direction of the artis；
The move file data are encoded according to hierarchical model, one-level code corresponding position changes maximum artis, Second level code corresponds to the main movement direction of the artis, and three-level code corresponds to motion frequency；
By file exercise data and its hierarchical model coding deposit motion database；
Be encoded to All Files according to the corresponding hierarchical model of file each in database and establish index tree, index tree it is non- Leaf node includes the coding of corresponding level, and leaf node contains an index structure inode, records following index information: { hierarchical model coding, place filename, the pointer of previous inode, the pointer of next inode }；
When retrieving motion database, hierarchical model coding is carried out to the movement of inquiry first, coding is found in index tree Similar leaf node finally searches the similar move file of all codings further according to the pointer information of inode to get to one A similar movement alternative file set.
Preferably, described instruction in accordance with the following methods, determines the main of the maximum artis of change in location and the artis The direction of motion:
Concussion factor S is set separatelyx、Sy、Sz, to record in human body skeletal architecture each artis in world coordinate system The maximum displacement of lower X, Y, Z-direction；Find out Sx、Sy、SzMaximum value, that is, can determine the maximum artis of change in location and the pass The main movement direction of node.
Preferably, described instruction calculates each artis of each frame behind the position under world coordinate system, carries out crucial The differentiation and extraction of frame determine that the hierarchical model of file encodes using key frame data, and only by key frame exercise data and Its hierarchical model coding deposit motion database.
Preferably, described instruction judges to move after carrying out key-frame extraction according to the position of each artis of each frame Period, and the key frame chosen in a period of motion determines the maximum artis of change in location and the main movement of the artis Direction.
Preferably, described instruction is filtered the data after dimensionality reduction, more each each data frame of artis Current location and the joint difference of maximum change location or minimum change position in entire movement, if its difference is greater than the joint The dynamic change average value of point is then considered as interference information, i.e., the location information the artis under this data frame is deleted.
Beneficial effect
Human action search method and equipment proposed by the present invention based on hierarchical model, uses the volume based on hierarchical model Code retains the main geometrical characteristic of exercise data, and the retrieval to compound movement data is converted into the inspection to simple digital coding Rope greatly improves the time efficiency of retrieval.
Detailed description of the invention
Fig. 1 is the flow chart of the method for the invention；
Fig. 2 is the file structure of BVH file header；
Fig. 3 is the file structure of BVH file data section.
Fig. 4 is using the joint Hips as the skeleton hierarchical model of root node.
Fig. 5 is the substantially planar schematic diagram of human motion.
Fig. 6 is human body motion layer time model.
Specific embodiment
In order to make the objectives, technical solutions, and advantages of the present invention clearer, with reference to embodiments, to the present invention It is further elaborated.It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention, it is not used to Limit the present invention.
When traditional Motion Capture equipment acquires human body movement data, it will usually be stored as the original motion data BVH file, the thought followed are the data conversion based on father and son's node relationships.Since BVH file is there are father and son's node relationships, Therefore file generally can be divided into two parts, and one is the file header for storing skeleton hierarchical structure, as shown in Fig. 2, logical It, can be clearly often using HIERARCHY as file origin identification, using ROOT as the mark of the starting point of human body skeletal layer time structure Internal connection present in skeleton hierarchical structure is showed by succinct form, it also can be skeleton node Required data class and data area are demarcated；Secondly to store the data segment of specific exercise data, such as Fig. 3 institute Show, usually using MOTION as the origin identification of data segment, with Frames for the length mark comprising how many motion sequence, can incite somebody to action Distinctive skeleton hierarchical structure is distinguished with specific exercise data.
It is found that human body bone level is using Hips as root node, as shown in Figure 4 from the example of Fig. 2 and Fig. 3.Each Relative to father node, in the translational movement under local coordinate system, the skeleton framework in BVH file defines in part skeletal joint point It obtains, the rotational component relative to father node obtains in each data frame.Root node data in data frame includes six dimensions The information of degree, rotary angle information under the displacement of root node and local coordinate system respectively under world coordinate system.Data in Fig. 3 (- 195.76,92.90,76.64) are the displacement informations of root node start frame, and data (6.83, -1.04,91.45) are root sections The rotary angle information of point start frame.In addition to root node data, the data of any bone node are all only by three rotational angle datas It constitutes, respectively each skeletal joint point is relative to the pitch angle of father node, roll angle, course deviation angular data.By these data, Position of the available each artis of each frame under world coordinate system.
Common movement either in daily life or the sports type games on stadium, skeleton it is each important Artis regular space-time characteristic and motion frequency is all presented, constitute the semantic feature of human motion.Human motion One important space-time characteristic is exactly the maximum artis of change in location and the main movement direction of the artis, passes through semanteme spy Sign can distinguish most of type of sports.Thought of the invention is exactly the semantic feature using human motion, is realized The efficiently human action retrieval based on content.
In human body movement data processing, usually the substantially planar of human body can be defined as horizontal plane, frontal plane, sagittal Three, face plane, wherein horizontal plane is the section that crosscutting body is parallel to the ground under upright state, and body is divided into upper and lower two Point；Frontal plane is that human body is divided into forward and backward two parts using body bilateral diameter as longitudal section made by tangent line；Sagittal plane is with body Body anteroposterior diameter is longitudal section made by tangent line, and human body is divided into left and right two parts；By the basic axis of human body be defined as frontal axis, Vertical axis, sagittal axis, wherein frontal axis is X-axis, and perpendicular to sagittal plane, it is frontal plane and horizontal plane that direction, which is left and right direction, Intersection；Vertical axis is Y-axis, and perpendicular to horizontal plane, it is the intersection of frontal plane and sagittal plane that direction, which is upper and lower direction,；Sagittal axis is Z Axis, perpendicular to frontal plane, it is the intersection of sagittal plane and horizontal plane that direction, which is front and back direction,.
By taking the movement of most common human body running as an example, clearly it is displaced as shown in figure 5, human body has in sagittal plane.Wherein, trunk Part is in horizontal plane, frontal plane without obviously significantly shaking；Four limbs have obvious and regular swing in sagittal plane, and under Limb amplitude of fluctuation is much larger than upper limb, thus judges lower limb for main powered member；Simultaneously according to four limbs hunting frequency and place-exchange Speed can determine whether specific exercise intensity.Therefore, retain each artis of human skeleton model in the location information of sagittal plane, so that it may Determine its main semantic feature.
Based on this thought, the embodiment of the present invention 1 realizes a kind of human action search method based on hierarchical model, including Following steps:
S1: each artis information of each frame is extracted from human motion file, calculates the artis in world coordinate system Under position；
S2: the maximum artis of change in location and the main movement direction of the artis are determined；
S3: encoding the move file data according to hierarchical model, and one-level code corresponding position changes maximum joint Point, second level code correspond to the main movement direction of the artis, and three-level code corresponds to motion frequency；
S4: by file exercise data and its hierarchical model coding deposit motion database；
S5: All Files are encoded to according to the corresponding hierarchical model of file each in database and establish index tree, index tree N omicronn-leaf child node include corresponding level coding, leaf node contains an index structure inode, records following index letter Breath: { hierarchical model coding, place filename, the pointer of previous inode, the pointer of next inode }；
S6: when retrieval motion database, hierarchical model coding is carried out to the movement of inquiry first, finds and compiles in index tree The similar leaf node of code finally searches the similar move file of all codings further according to the pointer information of inode to get arriving One similar movement alternative file set.
In embodiment 1, step S2 in accordance with the following methods, determines the maximum artis of change in location and the master of the artis Want the direction of motion:
Concussion factor S is set separatelyx、Sy、Sz, to record in human body skeletal architecture each artis in world coordinate system The maximum displacement of lower X, Y, Z-direction；Find out Sx、Sy、SzMaximum value, that is, can determine the maximum artis of change in location and the pass The main movement direction of node.
Step S3 is the core procedure of the present embodiment search method, embodies core of the invention thought.Human motion One important space-time characteristic is exactly the maximum artis of change in location and the main movement direction of the artis, passes through semanteme spy Sign can distinguish most of type of sports.The semantic feature based on human motion, the present invention construct human motion Hierarchical model, as shown in fig. 6, and the efficiently human action retrieval based on content based on this model realization.In human motion In hierarchical model, the first level is using main position of having an effect as motor area minute mark standard, by human motion according to the artis of manikin It distinguishes；Second level is divided into and generating in X-axis using the main movement direction of main movable joint point of having an effect as motor area minute mark standard The movement of displacement generates the movement of displacement, in the movement of Z axis generation displacement in Y-axis；Third level is using motion frequency as motor area Minute mark is quasi-, user by customized threshold value, by motion frequency be defined as it is fast, in, the different sizes such as slow move with distinguishing definition.
According to the hierarchical model defined, all movements can be encoded.Table 1 is that the hierarchical model of embodiment 1 is compiled Code example, the motion encoded number for artis of the 1st level, the motion encoded value of the second level are { 0,1,2 }, respectively indicate master Wanting the direction of motion is the exercise data of X-axis, Y-axis and Z axis, and the motion encoded value of third level is { 0,1,2 }, respectively indicates movement Frequency be it is fast, in, slow exercise data.
1 human action hierarchical model of table coding
Based on thought of the invention, the definition of hierarchical model can have many variations.For example, group can be carried out to artis It closes, is divided into upper limb master have an effect movement, the trunk master of movement, lower limb master that have an effect and has an effect movement, comprehensive movement of having an effect.It can thus incite somebody to action Main position of having an effect is determined as 4 kinds of situations.Hierarchical model is encoded, it can be according to shown in table 1, with specific decimal number table Show each classification, such as the artis that number is 12 is expressed as 12, binary system also can be used, with two binary digit tables Show corresponding three reference axis of the direction of motion, if main position of having an effect is determined as 4 kinds of situations, so that it may with two binary digit tables Show.
In step s 4, by file exercise data and its hierarchical model coding deposit motion database.
In step s 5, All Files are encoded to according to the corresponding hierarchical model of file each in database and establish index Tree, the n omicronn-leaf child node of index tree include the coding of corresponding level, and leaf node contains an index structure inode, are recorded Following index information: { hierarchical model coding, place filename, the pointer of previous inode, the pointer of next inode }；
In the normal retrieval of motion database, if necessary to carry out content-based retrieval, it usually needs extract geometry Feature, and Euclidean distance is calculated, in the building of index and during retrieval each time, need to be related to complicated geometry meter It calculates, to take a substantial amount of time.According to the present invention, the main geometry of exercise data is retained using the coding based on hierarchical model Retrieval to compound movement data is converted into the retrieval to simple digital coding, can greatly improve database by feature Recall precision, the motion retrieval application scenarios for the coarseness that is highly suitable for carrying out classifying etc. to exercise data, is also applied for file The retrieval of grade.
Step S6 is the step of retrieval using the database for establishing index: right first when retrieval motion database The movement of inquiry carries out hierarchical model coding, finds in index tree and encodes similar leaf node, finally further according to inode's Pointer information searches the similar move file of all codings to get to a similar movement alternative file set.