阅读说明：本技术 一种基于互联网点对点技术的服装商品基础数据同步方法 (Clothing commodity basic data synchronization method based on internet point-to-point technology ) 是由 宋翀 管睿 于 2020-04-14 设计创作，主要内容包括：本发明公开了一种基于互联网点对点技术的服装商品基础数据同步方法,包括分布式存储系统的实现与查询,使用结构化点对点覆盖网络,将信息以哈希表条目形式加以存储,所述哈希表条目被分散地存储在各个节点上,然后以特定的方式进行有效的查询,分布式存储系统的实现需要两张分布式哈希表即服装信息字典和图片文件索引字典；利用基于点对点技术的分布式存储方案,可以大幅度减少对中心服务器和带宽资源的依赖,从而缩减成本,减少对服务器的依赖,并保证网络通畅响应及时,从而实现服装行业基础数据的同步,其要旨在于利用分布式存储去中心化,在有限的服务器资源和网络带宽条件下,最大化实现流畅的数据同步和资源共享。(The invention discloses a clothing commodity basic data synchronization method based on internet point-to-point technology, which comprises the steps of realizing and inquiring a distributed storage system, storing information in a hash table entry form by using a structured point-to-point overlay network, wherein the hash table entries are dispersedly stored on each node and then effectively inquired in a specific mode, and the realization of the distributed storage system needs two distributed hash tables, namely a clothing information dictionary and a picture file index dictionary; the distributed storage scheme based on the point-to-point technology can greatly reduce the dependence on a central server and bandwidth resources, thereby reducing the cost, reducing the dependence on the server, and ensuring smooth response of a network in time, thereby realizing the synchronization of basic data in the clothing industry.)

1. A clothing commodity basic data synchronization method based on internet point-to-point technology is characterized by comprising the steps of realizing and inquiring a distributed storage system, storing information in a hash table entry form by using a structured point-to-point overlay network, wherein the hash table entries are dispersedly stored on each node, and then effectively inquiring in a specific mode;

the implementation of the distributed storage system needs two distributed hash tables, namely a clothing information dictionary and a picture file index dictionary;

the clothing information dictionary is used for inquiring the picture name and the related file information according to the keyword, wherein the key value is the SHA1 hash corresponding to the keyword, the value of the response is a list, and the picture name and the file information corresponding to the keyword are given;

the picture file index dictionary is used for inquiring the target node according to the given file information, wherein the key value is equal to the SHA1 check value of the required picture, and the corresponding value is also a list and gives the network information of all the nodes which currently possess the file;

based on the two DHTs, the query and synchronization of the pictures only need the client to input keywords, then the garment information dictionary queries the SHA1 check value of the picture file, and then the picture file index dictionary is queried according to the value, so that the target node information is obtained, and the downloading or synchronization link is established.

2. The clothing commodity basic data synchronization method based on the internet point-to-point technology as claimed in claim 1, wherein the distributed hash table adopts a Kademlia network technology, the distance of nodes is introduced for directional query, firstly, each node has an exclusive ID which is a randomly generated 160-bit integer, and the distance between any two nodes is defined as a bitwise binary sum of the ID values of the two nodes, so that each node can judge the distance between other nodes and itself according to the distance concept, and a storage, maintenance and query scheme is adopted by using the distance concept.

3. The clothing commodity basic data synchronization method based on the internet point-to-point technology, according to claim 1, characterized in that the storage scheme comprises item storage, any item is according to the specific value of its key, the item is copied and stored in k nodes with the node ID nearest to the key value (namely nearest to the target node N at present), the typical value of k is 20, in order to realize shorter query response delay, any item can be cached onto any node in the process of item query; meanwhile, in order to prevent excessive cache and ensure that the information is fresh enough, the timeliness of the storage of the entries on the nodes must be considered: the closer to the target node N, the longer the entry will be saved, and conversely, the shorter the timeout time is; the entry saved on top of the target node can be retained for 24 hours, during which time the entry is additionally republished by its publishing source, the save time of which can be further extended.

4. The method for synchronizing the basic data of clothing items based on the Internet peer-to-peer technology as claimed in claim 1, wherein the maintenance scheme comprises node maintenance, each node maintains 160 lists, wherein the ith list records the network information (NodeID, IP address, UDP port) of a plurality of k opposite end nodes with the distance of 2^ i to 2^ (i +1) known by the current node; the opposite end node information in each list is sorted according to the access time, the earliest access is at the head of the list, the latest access is at the tail of the list, and the node information is updated according to the principle of Least-recently Seen evolution: when the capacity of the list is not full and the newly accessed opposite end node information is not in the current list, the information is directly added into the tail of the list queue; if its information is already in the current list, it will be moved to the end of the queue; under the condition that the capacity of the list is full, whether the head node of the queue accessed earliest still has response is checked firstly, if yes, the head node of the queue is moved to the tail of the queue, the information of the new access node is discarded, if not, the head node of the queue is discarded, and the information of the node accessed latest is inserted into the tail of the queue.

5. The clothing item basic data synchronization method based on the internet point-to-point technology as claimed in claim 1, wherein the query scheme includes a node query process, wherein when a certain node ID is known, a process of obtaining network information corresponding to k nodes with shortest distance to the node in the current network is searched, that is, a primary node query process;

(a) the query initiator selects a plurality of nodes closest to the target ID from the list of the query initiator and simultaneously sends asynchronous query requests to the nodes;

(b) after receiving the request, the queried node finds out a plurality of nodes which are known by the queried node and are closest to the query target ID from the list of the queried node, and returns the nodes to the initiator;

(c) after receiving the return information, the initiator picks out a plurality of nodes which are not requested from all known nodes which are closer to the target again, and repeats the step (a);

(d) the steps are repeated continuously until the active nodes which are closer to the target than the k nodes currently known by the inquirer cannot be obtained;

(e) in the inquiry process, the nodes which do not respond in time are immediately eliminated; the querier must ensure that the k-most nodes ultimately obtained are all active.

6. The method for synchronizing basic data of clothing items based on the internet point-to-point technology as claimed in claim 1, wherein the query plan includes a new node, and when a new node first attempts to join the network, the following three operations are performed:

(a) acquiring information of a node which has been added into the network through a server, and adding the information into a list of the server;

(b) initiating a node query request aiming at the ID of the node to the node, thereby acquiring a series of information of other nodes which are close to the node;

(c) and refreshing all lists to ensure that all the node information obtained by the self is fresh.

Technical Field

The invention relates to a clothing commodity basic data synchronization method based on an interconnection network point-to-point technology, in particular to a clothing commodity basic data synchronization method based on the interconnection network point-to-point technology.

Background

Under the background of the current information age, the integration of the clothing industry and the Internet of things is imperative. The storage and transmission of massive clothing pictures need to consume a large number of computer and network resources, and the traditional central storage mode cannot cope with the increase of data scale, and the problems of network congestion, server resource exhaustion, difficult database maintenance and the like are faced.

Disclosure of Invention

The invention provides a clothing commodity basic data synchronization method based on an internet point-to-point technology, which is used for overcoming the defects of data maintenance and synchronization cost rise caused by explosive growth of big data in the clothing industry in the prior art.

In order to solve the technical problems, the invention provides the following technical scheme:

the invention discloses a clothing commodity basic data synchronization method based on internet point-to-point technology, which comprises the steps of realizing and inquiring a distributed storage system, storing information in a hash table entry form by using a structured point-to-point overlay network, dispersedly storing the hash table entries on each node, and then effectively inquiring in a specific mode;

the implementation of the distributed storage system needs two distributed hash tables, namely a clothing information dictionary and a picture file index dictionary;

the clothing information dictionary is used for inquiring the picture name and the related file information according to the keyword, wherein the key value is the SHA1 hash corresponding to the keyword, the value of the response is a list, and the picture name and the file information corresponding to the keyword are given;

the picture file index dictionary is used for inquiring the target node according to the given file information, wherein the key value is equal to the SHA1 check value of the required picture, and the corresponding value is also a list and gives the network information of all the nodes which currently possess the file;

based on the two DHTs, the query and synchronization of the pictures only need the client to input keywords, then the garment information dictionary queries the SHA1 check value of the picture file, and then the picture file index dictionary is queried according to the value, so that the target node information is obtained, and the downloading or synchronization link is established.

Furthermore, the distributed hash table adopts Kademlia network technology, the distance of nodes is introduced for directional query, each node has an exclusive ID which is a randomly generated integer of 160 bits, the distance between any two nodes is defined as the bitwise binary sum of the ID values of the two nodes, so that each node can judge the distance between other nodes and the node according to the distance concept, and a storage, maintenance and query scheme is adopted by utilizing the distance concept.

Further, the storage scheme includes entry storage, according to the specific value of the key of any entry, the entry is copied and stored in k nodes with the node ID closest to the key value (namely, closest to the target node N at present), the typical value of k is 20, and in order to realize shorter query response delay, any entry can be cached to any node in the process of entry query; meanwhile, in order to prevent excessive cache and ensure that the information is fresh enough, the timeliness of the storage of the entries on the nodes must be considered: the closer to the target node N, the longer the entry will be saved, and conversely, the shorter the timeout time is; the entry saved on top of the target node can be retained for 24 hours, during which time the entry is additionally republished by its publishing source, the save time of which can be further extended.

Furthermore, the maintenance scheme comprises node maintenance, each node maintains 160 lists, wherein in the ith list, network information (NodeID, IP address and UDP port) of a plurality of k opposite end nodes with the known distance from the current node to 2^ i-2 ^ (i +1) is recorded; the opposite end node information in each list is sorted according to the access time, the earliest access is at the head of the list, the latest access is at the tail of the list, and the node information is updated according to the principle of Least-recently Seen evolution: when the capacity of the list is not full and the newly accessed opposite end node information is not in the current list, the information is directly added into the tail of the list queue; if its information is already in the current list, it will be moved to the end of the queue; under the condition that the capacity of the list is full, whether the head node of the queue accessed earliest still has response is checked firstly, if yes, the head node of the queue is moved to the tail of the queue, the information of the new access node is discarded, if not, the head node of the queue is discarded, and the information of the node accessed latest is inserted into the tail of the queue.

Further, the query scheme includes a process of querying a node, knowing an ID of the node, and searching for network information corresponding to k nodes having the shortest distance to the node in the current network, which is a primary node query process.

(a) The query initiator selects a plurality of nodes closest to the target ID from the list of the query initiator and simultaneously sends asynchronous query requests to the nodes;

(b) after receiving the request, the queried node finds out a plurality of nodes which are known by the queried node and are closest to the query target ID from the list of the queried node, and returns the nodes to the initiator;

(c) after receiving the return information, the initiator picks out a plurality of nodes which are not requested from all known nodes which are closer to the target again, and repeats the step (a);

(d) the above steps are repeated until no active nodes closer to the target than the k nodes currently known by the querier are available.

(e) In the inquiry process, the nodes which do not respond in time are immediately eliminated; the querier must ensure that the k-most nodes ultimately obtained are all active.

Further, the query plan includes a new node, and when a new node attempts to join the network for the first time, the following three operations are performed:

(a) acquiring information of a node which has been added into the network through a server, and adding the information into a list of the server;

(b) initiating a node query request aiming at the ID of the node to the node, thereby acquiring a series of information of other nodes which are close to the node;

(c) and refreshing all lists to ensure that all the node information obtained by the self is fresh.

The invention has the following beneficial effects: the distributed storage scheme based on the point-to-point technology can greatly reduce the dependence on a central server and bandwidth resources, thereby reducing the cost, reducing the dependence on the server, and ensuring smooth response of a network in time, thereby realizing the synchronization of basic data in the clothing industry.

Drawings

The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention and not to limit the invention. In the drawings:

FIG. 1 is a system flow diagram of the present invention.

Detailed Description

The preferred embodiments of the present invention will be described in conjunction with the accompanying drawings, and it will be understood that they are described herein for the purpose of illustration and explanation and not limitation.

Example 1

As shown in fig. 1, a clothing item basic data synchronization method based on internet point-to-point technology includes the implementation and query of a distributed storage system, using a structured point-to-point overlay network to store information in the form of hash table entries, where the hash table entries are dispersedly stored on each node, and then performing effective query in a specific manner;

the implementation of the distributed storage system needs two distributed hash tables, namely a clothing information dictionary and a picture file index dictionary;

the clothing information dictionary is used for inquiring the picture name and the related file information according to the keyword, wherein the key value is the SHA1 hash corresponding to the keyword, the value of the response is a list, and the picture name and the file information corresponding to the keyword are given;

the picture file index dictionary is used for inquiring the target node according to the given file information, wherein the key value is equal to the SHA1 check value of the required picture, and the corresponding value is also a list and gives the network information of all the nodes which currently possess the file;

based on the two DHTs, the query and synchronization of the pictures only need the client to input keywords, then the garment information dictionary queries the SHA1 check value of the picture file, and then the picture file index dictionary is queried according to the value, so that the target node information is obtained, and the downloading or synchronization link is established.

The distributed hash table adopts Kademlia network technology, introduces the distance of nodes for directional query, firstly, each node has an exclusive ID which is a randomly generated integer of 160 bits, and defines the distance between any two nodes as the binary bit-by-bit sum of the ID values of the two nodes, so that each node can judge the distance between other nodes and the node according to the distance concept, and the storage, maintenance and query schemes are adopted by utilizing the distance concept.

The storage scheme comprises entry storage, wherein according to the specific value of the key of any entry, the entry is copied and stored in k nodes with the node ID closest to the key value (namely closest to a target node N at present), the typical value of k is 20, and in order to realize shorter query response delay, any entry can be cached to any node in the process of entry query; meanwhile, in order to prevent excessive cache and ensure that the information is fresh enough, the timeliness of the storage of the entries on the nodes must be considered: the closer to the target node N, the longer the entry will be saved, and conversely, the shorter the timeout time is; the entry saved on top of the target node can be retained for 24 hours, during which time the entry is additionally republished by its publishing source, the save time of which can be further extended.

The maintenance scheme comprises node maintenance, each node maintains 160 lists, wherein in the ith list, network information (NodeID, IP address and UDP port) of a plurality of k opposite end nodes with the known distance of 2^ i-2 ^ (i +1) from the current node is recorded; the opposite end node information in each list is sorted according to the access time, the earliest access is at the head of the list, the latest access is at the tail of the list, and the node information is updated according to the principle of Least-recently Seen evolution: when the capacity of the list is not full and the newly accessed opposite end node information is not in the current list, the information is directly added into the tail of the list queue; if its information is already in the current list, it will be moved to the end of the queue; under the condition that the capacity of the list is full, whether the head node of the queue accessed earliest still has response is checked firstly, if yes, the head node of the queue is moved to the tail of the queue, the information of the new access node is discarded, if not, the head node of the queue is discarded, and the information of the node accessed latest is inserted into the tail of the queue.

The query scheme comprises a process of querying nodes, knowing the ID of a certain node, and searching and obtaining network information corresponding to k nodes with the shortest distance to the node in the current network, namely a primary node query process.

(a) The query initiator selects a plurality of nodes closest to the target ID from the list of the query initiator and simultaneously sends asynchronous query requests to the nodes;

(b) after receiving the request, the queried node finds out a plurality of nodes which are known by the queried node and are closest to the query target ID from the list of the queried node, and returns the nodes to the initiator;

(c) after receiving the return information, the initiator picks out a plurality of nodes which are not requested from all known nodes which are closer to the target again, and repeats the step (a);

(d) the above steps are repeated until no active nodes closer to the target than the k nodes currently known by the querier are available.

(e) In the inquiry process, the nodes which do not respond in time are immediately eliminated; the querier must ensure that the k-most nodes ultimately obtained are all active.

The query plan includes a newly added node, and when a new node attempts to join the network for the first time, the following three operations are performed:

(a) acquiring information of a node which has been added into the network through a server, and adding the information into a list of the server;

(b) initiating a node query request aiming at the ID of the node to the node, thereby acquiring a series of information of other nodes which are close to the node;

(c) and refreshing all lists to ensure that all the node information obtained by the self is fresh.

Finally, it should be noted that: although the present invention has been described in detail with reference to the foregoing embodiments, it will be apparent to those skilled in the art that changes may be made in the embodiments and/or equivalents thereof without departing from the spirit and scope of the invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.