Content identification generation method, device, equipment and storage medium

文档序号：912928 发布日期：2021-02-26 浏览：8次中文

阅读说明：本技术 内容标识的生成方法、装置、设备及存储介质 (Content identification generation method, device, equipment and storage medium ) 是由李绍华封伟城于 2020-11-19 设计创作，主要内容包括：本申请公开了一种内容标识的生成方法、装置、设备及存储介质,属于计算机技术领域。所述方法包括：获取目标内容对应的时间戳；获取目标内容在时间戳下唯一的自增编码；基于时间戳和自增编码,生成目标内容的标识编码。本申请实施例提供的技术方案中,通过获取时间戳以区分不同时刻的内容,通过生成时间戳下唯一的自增编码以区分在同一时刻请求生成标识编码的各个内容,基于时间戳和时间戳下唯一的自增编码,可以生成目标内容的唯一的标识编码,作为目标内容在全球范围内真正全局唯一的数值标识,并且扩大了标识编码的应用范围。(The application discloses a method, a device, equipment and a storage medium for generating content identification, and belongs to the technical field of computers. The method comprises the following steps: acquiring a timestamp corresponding to target content; acquiring a unique self-increment code of the target content under a timestamp; and generating an identification code of the target content based on the time stamp and the incremental code. According to the technical scheme, the time stamp is acquired to distinguish the contents at different moments, the unique self-increment code under the time stamp is generated to distinguish each content requesting to generate the identification code at the same moment, the unique identification code of the target content can be generated based on the time stamp and the unique self-increment code under the time stamp, and is used as the real globally unique numerical identification of the target content in the global range, and the application range of the identification code is expanded.)

1. A method for generating a content identifier, the method comprising:

acquiring a timestamp corresponding to target content;

acquiring a unique self-increment code of the target content under the timestamp, wherein the self-increment code is used for distinguishing each content which requests to generate an identification code at the same moment;

generating an identification code for the target content based on the timestamp and the incremental code.

2. The method of claim 1, wherein obtaining the unique self-increment code of the target content under the timestamp comprises:

generating a temporary self-increment code of the target content under the time stamp based on the time stamp;

performing consistency detection on the temporary self-increment codes, wherein the consistency detection is used for detecting the uniqueness of the temporary self-increment codes;

determining the temporary autogain encoding to be the autogain encoding of the target content if the temporary autogain encoding passes the consistency detection.

3. The method of claim 2, wherein the performing consistency detection on the temporal autoincrement code comprises:

acquiring proposal information carrying the temporary self-increment code;

and under the condition that the proposal information conforms to a preset rule, determining that the temporary self-increment code passes the consistency detection, wherein the preset rule means that a plurality of nodes with the number meeting a preset proportion condition in self-increment code generating service all agree with the proposal information, and the self-increment code generating service comprises a plurality of nodes used for generating the self-increment code for the target content.

4. The method of claim 2, wherein after the consistency detection of the temporal self-increment encoding, further comprising:

under the condition that the temporary self-increment code does not pass the consistency detection, updating the numerical value of the temporary self-increment code to obtain an updated temporary self-increment code;

carrying out consistency detection on the updated temporary self-increment codes;

determining the updated temporary autogain encoding to be the autogain encoding of the target content if the updated temporary autogain encoding passes the consistency detection.

5. The method of claim 1, wherein obtaining the unique self-increment code of the target content under the timestamp comprises:

distributing target incremental coding in an incremental coding library to the target content based on the timestamp to serve as the incremental coding of the target content, wherein the incremental coding library comprises a preset number of unassigned incremental coding, and the target incremental coding refers to any unassigned incremental coding in the incremental coding library;

removing the target surcharge code from the surcharge code library.

6. The method according to claim 5, wherein after removing the target surrogated code from the surrogated code library, further comprising:

acquiring coding margin of the self-increment coding library, wherein the coding margin refers to the number of the self-increment codes remained in the self-increment coding library;

and generating a preset number of self-increment codes to fill the self-increment code library under the condition that the coding margin of the self-increment code library is lower than a threshold value.

7. The method of any of claims 1 to 6, wherein generating the identification code of the target content based on the timestamp and the incremental code comprises:

performing data splicing processing on the timestamp, the self-increment code, the application identification code and the area code to obtain an identification code of the target content;

the application identification content code refers to an identification code of an application service for processing the target content, and the region code refers to an identification code of a region where the application service is located.

8. The method of claim 7, wherein the identification code of the target content is a 64-bit binary code; the time stamp occupies 32 bits, the area code occupies 4 bits, the application identification code occupies 8 bits, and the incremental code occupies 20 bits.

9. An apparatus for generating a content id, the apparatus comprising:

the time stamp obtaining module is used for obtaining a time stamp corresponding to the target content;

the self-increment code acquisition module is used for acquiring a unique self-increment code of the target content under the timestamp, and the self-increment code is used for distinguishing each content which requests to generate an identification code at the same moment;

and the identification code generating module is used for generating the identification code of the target content based on the timestamp and the incremental code.

10. A computer arrangement, characterized in that the arrangement comprises a processor and a memory, in which a computer program is stored, which computer program is loaded and executed by the processor to implement the method according to any of claims 1-8.

11. A computer-readable storage medium, in which a computer program is stored which is loaded and executed by a processor to implement the method according to any one of claims 1 to 8.

Technical Field

The present application relates to the field of computer technologies, and in particular, to a method, an apparatus, a device, and a storage medium for generating a content identifier.

Background

With the development of internet technology, short video service platforms are more and more popular with users, and the number of videos generated in the short video service platforms is increased. Therefore, a method for generating a unique ID (Identity Document) for mass videos in a short video service platform is needed.

In the related art, a manually labeled machine ID, a data format ID, and a time ID are usually used to combine them to generate a final ID value.

The related art has a certain limitation when applied to a distributed system.

Disclosure of Invention

The embodiment of the application provides a content identifier generation method, a content identifier generation device, content identifier generation equipment and a content identifier storage medium, which can truly and globally identify a unique numerical value of target content in a global range, and enlarge the application range of identifier codes. The technical scheme is as follows:

according to an aspect of an embodiment of the present application, there is provided a method for generating a content identifier, the method including:

acquiring a timestamp corresponding to target content;

generating an identification code for the target content based on the timestamp and the incremental code.

According to an aspect of the embodiments of the present application, there is provided an apparatus for generating a content identifier, the apparatus including:

the time stamp obtaining module is used for obtaining a time stamp corresponding to the target content;

and the identification code generating module is used for generating the identification code of the target content based on the timestamp and the incremental code.

According to an aspect of the embodiments of the present application, there is provided a computer device, the device includes a processor and a memory, the memory stores a computer program, and the computer program is loaded and executed by the processor to implement the content identification generation method.

According to an aspect of the embodiments of the present application, there is provided a computer-readable storage medium, in which a computer program is stored, the computer program being loaded and executed by a processor to implement the above-mentioned content identification generation method.

According to an aspect of embodiments herein, there is provided a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to make the computer device execute the content identification generation method.

The technical scheme provided by the embodiment of the application can bring the following beneficial effects:

the unique identification code of the target content can be generated based on the timestamp and the unique self-increment code under the timestamp, and can be used as a real globally unique numerical identification of the target content in a global range, so that the application range of the identification code is expanded.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

FIG. 1 is a schematic diagram of an application execution environment provided by one embodiment of the present application;

fig. 2 is a flowchart of a method for generating a content identifier according to an embodiment of the present application;

FIG. 3 illustrates a diagram of a data structure for an identification code;

FIG. 4 illustrates a schematic diagram of a video ID generation flow in a distributed system;

FIG. 5 illustrates a flow diagram for generating a content identifier;

fig. 6 is a block diagram of a device for generating a content identifier according to an embodiment of the present application.

Detailed Description

To make the objects, technical solutions and advantages of the present application more clear, embodiments of the present application will be described in further detail below with reference to the accompanying drawings.

The background of the application of the present application is first described to facilitate the reader's understanding of the problems addressed by the present application.

At present, authoring of content on the internet can be divided into three types: UGC (User-Generated Content), PGC (professional-Generated Content), and OGC (professional-Generated Content). The UGC and PGC are distinguished by the fact that PGC has professional knowledge, qualification, and certain knowledge background and work qualification in the field of shared content.

Compared with a PGC video platform, the UGC video platform has the advantages that no professional learning and background is needed for registering an account or uploading videos, so that the UGC video platform has various video sources and various video contents. Among UGC video platforms, there are high-quality videos shot by professional content producers, videos shot at will by general content producers, popular videos relating to professional knowledge such as astronomy, geography, history, and the like, and candidate videos relating to interests of daily life. For UGC video platforms, the level of video volume uploaded by content producers per day can amount to tens of millions.

Therefore, in a distributed system, in the face of the situation that the traffic of service messages is huge and the deployment area of service machines is wide, how to efficiently generate a theoretically truly globally unique numerical ID in a global scope is a relatively challenging problem, and the problem is also solved by the content identification generation method provided by the application. This is explained below.

Besides, the scheme of the present application can generate IDs for videos, and also can give other data contents, such as orders, information, dynamic messages, audio, and the like, where the videos are merely exemplary, and the form of the data contents is not limited in the embodiments of the present application.

Referring to fig. 1, a schematic diagram of an application execution environment according to an embodiment of the present application is shown. The application execution environment may include: a terminal 10 and a server 20.

The terminal 10 may be an electronic device such as a mobile phone, a tablet Computer, a game console, a multimedia playing device, a wearable device, a PC (Personal Computer), and the like. A client of the application may be installed in the terminal 10.

In the embodiment of the present application, the application may be any application capable of uploading or publishing content. Typically, the application is a video playback type application, such as a short video application or the like. Of course, besides game applications, videos can also be played in user interfaces in other types of applications, and after a user registers an account in a client of an application, the user can upload and watch the videos, and also can watch videos uploaded by other users. For example, the application may be a Virtual Reality (VR) application, an Augmented Reality (AR) application, a social interaction application, an interactive entertainment application, and the like, which is not limited in this embodiment.

Optionally, a client of the above application program runs in the terminal 10. Generally, one client is installed in one terminal 10, but the present embodiment does not exclude a case where a plurality of clients are installed in one terminal 10. The number of clients, that is, the number of terminals 10 is not limited in the embodiment of the present application.

The server 20 is used to provide background services for clients of applications in the terminal 10. For example, the server 20 may be a backend server for the application described above. The server 20 may be a server, a server cluster composed of a plurality of servers, or a cloud computing service center. Optionally, the server 20 provides background services for applications in multiple terminals 10 simultaneously.

Alternatively, the terminal 10 and the server 20 may communicate with each other through the network 30.

Referring to fig. 2, a flow chart of a method provided by an embodiment of the present application is shown. The method comprises the following steps (201-203):

step 201, obtaining a timestamp corresponding to the target content.

In an exemplary embodiment, the target content is any item of data content in the client, such as a video, a piece of comment data, a piece of like data. The Client (Client) is also called a Client and corresponds to the server, and in the embodiment of the present application, the Client may be an application program providing a video playing service, that is, the Client may provide the video playing service for the user. Alternatively, the client corresponds to a user interface, and the application program can play the video in the user interface of the client to provide the video playing service. Optionally, the user interface of the client includes some operation controls, such as buttons, sliders, icons, and the like, for the user to operate.

In response to receiving the identification generation request, a timestamp of the reception time is acquired. And the time stamp of the receiving time is the time stamp corresponding to the target content. The identification generation request refers to an instruction for requesting generation of an identification code of the target content, and may be intuitively understood as an ID acquisition request.

Optionally, the timestamp corresponding to the target content is a timestamp of a time when the server receives the ID obtaining request for the target content.

The user can upload or publish target content in the client, the client sends data of the target content to an application server providing background service for the client after responding to the uploading or publishing operation of the user, the application server receives and stores the data of the target content, and then sends an ID acquisition request to a management server deployed with an ID generation service for resource management so as to obtain an identification code (content ID) of the target content, the ID generation service deployed in the management server records time information of the received request as a time stamp of the target content after receiving the ID acquisition request, and finally the management server can generate a unique content ID, such as a video ID, a comment ID and a praise ID, for the target content. Optionally, in some scenarios, the ID generation service may also be deployed in an application server or a client.

The embodiment of the application does not limit the triggering mode of the ID obtaining request, and in one example, a video uploading control is displayed in a user interface of the client, and a user can trigger the client to generate the ID obtaining request by clicking the video uploading control; in another example, the user may trigger the client to generate the ID retrieval request by sliding, voice, gesture, or the like, in addition to triggering the client to generate the ID retrieval request by touch operation. The content included in the ID obtaining request is not limited in the embodiment of the present application, and optionally, the content included in the video obtaining request includes, but is not limited to, at least one of the following content: an application identification code of the client, a user account logged in by the client, characteristics of the user account logged in by the client, and a Uniform Resource Locator (URL) of the target content.

And step 202, acquiring a unique self-increment code of the target content under the time stamp.

In a possible implementation manner, after receiving the ID acquisition request, the ID generation service in the management server may call the self-increment code generation service also deployed in the management server to generate a self-increment code unique to the timestamp for the target content by using the timestamp as a data key (key) in addition to recording the timestamp. The incremental code is used to distinguish between the respective contents that are requested to generate the identification code at the same time. Alternatively, the self-incrementing encoding of the content at different time stamps may be the same or different. Optionally, in some scenarios, the self-increment code generation service may be deployed in an application server. Optionally, the self-increment code generation service is provided by a Paxos KV cluster consisting of a plurality of Paxos KV nodes. In distributed storage systems, the storage of critical data is typically accomplished using a consistency algorithm, such as the Paxos algorithm. The consistency algorithm allows multiple nodes to work together as a cluster, and the cluster can still work properly when several of the nodes fail. For example, the Paxos algorithm can provide consistent, self-maintaining, peer-to-peer copy based Key-Value (KV) database services and can tolerate failure of no more than half of the nodes in a cluster. Alternatively, similar effects can be achieved in Paxos KV clusters if other KV-type storage is used, such as Remote Dictionary service (Redis), etcd systems, and so forth.

In an exemplary embodiment, the above step 202 can be implemented in two ways, which are described below to facilitate understanding by those skilled in the art.

Mode one (step 2021-step 2026):

step 2021, based on the timestamp, generates a temporary self-increment code of the target content under the timestamp.

The temporary incremental code is generated for the target content by a certain node in the incremental code generation service.

Step 2022, performing consistency check on the temporary self-increment codes.

Consistency detection is used to detect uniqueness of a temporal self-increment code, consistency (consensus) refers to multiple nodes reaching consistency in state. Optionally, the consistency detection is used to detect uniqueness of the temporary incremental code at each node in the incremental code generation service.

In an exemplary embodiment, proposal information carrying a temporary self-increment code is acquired; and under the condition that the proposal information accords with a preset rule, determining that the temporary self-increment codes pass consistency detection, wherein the preset rule refers to that a plurality of nodes meeting a preset proportion condition exist in self-increment code generation service and agree with the proposal information, and the self-increment code generation service comprises a plurality of nodes used for generating the self-increment codes for target contents.

It can be simply understood that a certain node in the self-increment code generation service sends the temporary self-increment code generated for the target content as a proposal to other nodes in the self-increment code generation service, and the other nodes compare the value of the temporary self-increment code in the proposal with the current code value, and then give a feedback result to indicate whether to agree with the proposal.

Step 2023, determining the temporary self-increment code as the self-increment code of the target content when the temporary self-increment code passes consistency detection.

And a certain node in the self-increment code generation service sends the proposal information carrying the temporary self-increment codes to other nodes in the self-increment code generation service. And under the condition that a plurality of nodes meeting the preset proportion condition in the self-increasing code generation service agree with the proposal information, determining that the temporary self-increasing code passes through consistency detection, and further determining that the temporary self-increasing code is the self-increasing code of the target content.

Step 2024, updating the numerical value of the temporary self-increment code to obtain the updated temporary self-increment code when the temporary self-increment code fails the consistency detection.

Alternatively, the value of the temporary self-increment code is updated by an accumulation operation, for example, the current temporary self-increment code is 01 (binary), and then the self-increment code after one update can be 10 (binary).

Step 2025, performing consistency check on the updated temporary self-increment codes.

The process of performing consistency detection on the updated temporary self-increment codes is the same as the process of performing consistency detection on the non-updated temporary self-increment codes, and only changes in the detection values are detected, which is not described herein again.

Step 2026, determining the updated temporary self-increment code as the self-increment code of the target content if the updated temporary self-increment code passes the consistency detection.

The above way is to guarantee that the generated incremental code (incremental ID) is unique under a certain timestamp by executing Paxos algorithm in each node in the incremental code generation service.

Mode two (step 202 a-step 202 d):

step 202a, distributing the target self-increment codes in the self-increment code library to the target content based on the time stamp to serve as the self-increment codes of the target content.

The incremental code library comprises a preset number of unassigned incremental codes, and the target incremental code refers to any unassigned incremental code in the incremental code library. The selfjoin code library is generated in advance for use in allocation calling.

Optionally, the incremental codes in the incremental code library are sequentially distributed from small to large according to the size of the incremental codes.

In step 202b, the target surcharge code is removed from the surcharge code library.

And removing the self-increment codes which are allocated to be used from the self-increment code library.

Step 202c, obtaining the coding margin of the self-increment coding library.

The coding margin refers to the number of the self-increment codes remaining in the self-increment code library.

Step 202d, generating a preset number of incremental codes to fill the incremental code library under the condition that the coding margin of the incremental code library is lower than the threshold value.

By monitoring the coding margin in the self-increment coding library, the self-increment codes are supplemented to the self-increment coding library in time, and the distribution process of the self-increment codes is ensured not to be interrupted.

Alternatively, the target number of the incremental codes are generated according to the coding size sequence to fill the incremental code library, and the target number of the incremental codes with continuous codes can be generated after the maximum code value in the incremental codes generated last time. For example, the last generated incrementation code is 0-100, then the incrementation code generated this time can be generated from 101.

Optionally, the target number of the incremental codes and the remaining incremental codes in the incremental code library are a plurality of codes in succession. For example, the remaining turbo codes in the turbo code library are 80-100, and the target number of turbo codes generated this time may be 101-.

And step 203, generating an identification code of the target content based on the time stamp and the incremental code.

The time stamp corresponding to the target content and the unique self-increment code under the time stamp are combined to obtain the identification code of the target content, and the time stamp can represent the time information of the target content, so that the target content can be distinguished from the data content at other moments. For the data content with the same time stamp as the target content, the identification code of the target content has the unique self-increment code under the time stamp, so that the target content and the data content with the same time stamp as the target content can be distinguished, and the uniqueness of the identification code is ensured.

In an exemplary embodiment, the timestamp, the incremental coding, the application identification coding and the region coding are subjected to data splicing processing to obtain the identification coding of the target content.

The application identification code refers to an identification code of an application service that processes the target content. Optionally, the application identification code is obtained at the same time or after the time stamp corresponding to the target data content is obtained. The application identification code can be carried by the ID acquisition request, and further the application identification code can be acquired from the ID acquisition request. Optionally, the application identification code is set for the application service in advance.

The region code is an identification code of a region where the application service is located. Alternatively, the region is divided into continents on a global basis, and each continent on the earth is assigned with a region code. The ID generation service in the management server may obtain the corresponding region code according to the region where the ID generation service is located, and finally add the region code to the identification code of the target content.

In an exemplary embodiment, the identification encoding of the target content is a 64-bit binary encoding; the time stamp occupies 32 bits, the area code occupies 4 bits, the application identification code occupies 8 bits, and the self-increment code occupies 20 bits. In one example, as shown in FIG. 3, a diagram illustrating a data structure for an identification code is shown. Wherein, the identification code is 64-bit binary code; the second-level time stamp occupies 32 bits, the area code occupies 4 bits after the time stamp, the application identification code occupies 8 bits after the area code, and the self-increment code occupies 20 bits after the application identification code, and the total number of the self-increment code is 64 bits.

Optionally, if the incremental coding takes the second-level timestamp as the key, each second of incremental coding is sequentially incremented from 0, and the incremental coding occupies 20 bits, so the scheme can support each large area to generate 100w content identifiers per second, which is sufficient to meet the service requirement in the context of content identifier generation.

And the timestamp does not depend on clock synchronization among machines, the requirement on the system is low, even if the condition of server clock callback occurs, as long as the key in the Paxos KV cluster exists, the generation of the content identification is not influenced, and the uniqueness of the generated content identification can still be ensured.

The content identification generation method provided by the embodiment of the present application is briefly described here by way of an example from the perspective of a distributed system. As shown in fig. 4, a schematic diagram of a video ID generation flow in a distributed system is exemplarily shown. In the figure, a client of an application program installed in a terminal 41, a short video service platform 42 is an application server providing a background service for the client of the application program installed in the terminal 41, and an ID generation service and a self-increment code generation service are deployed in a management server 43 for providing a unique video ID for a video stored in the application server.

The user may execute a corresponding operation on the terminal 41, and after responding to the video uploading or publishing operation of the user, the client uploads the video to the short video service platform 42, and after receiving and storing the data of the video in the short video service platform 42, in order to facilitate resource management, the client sends an ID acquisition request, which carries an application identification code, to the management server 43 with the ID generation service deployed. After receiving the ID acquisition request, the ID generation service deployed in the management server 43 records time information of the received request as a timestamp of the target content, acquires an application identification code carried in the ID acquisition request, and uses the timestamp as a data key (key), invokes a self-increment code generation service deployed in the management server 43 as well to generate a unique self-increment code under the timestamp for the target content, and a plurality of Paxos KV nodes in the self-increment code generation service generate a unique self-increment ID of the video under the timestamp through a Paxos algorithm and return the generated self-increment ID to the ID generation service. After receiving the self-increment ID, the ID generation service concatenates the timestamp, the self-increment code, the application identification code, and the area code to obtain the video ID of the video and sends the video ID to the short video service platform 42. The short video service platform 42 assigns the video ID to the video via the URL of the video. Optionally, a set of management servers 43 is deployed in each continent (continent), where the management servers include a group of Paxos KV clusters and a group of ID generation service clusters, and video IDs between different continents are distinguished by a continent ID in the video IDs.

The generation flow of the content identifier is briefly described here by another example, as shown in fig. 5, which schematically illustrates a flow chart of generating the content identifier. Firstly, step 51 is executed to obtain a second-level time stamp, an application identification code and a region code; then, executing step 52, requesting the Paxos KV cluster to acquire the self-increment code; finally, step 53 is performed to assemble the data in order to generate the content id. The above flow is only a brief description for facilitating understanding of the technical solutions provided in the present application, and does not represent an actual processing procedure.

In summary, according to the embodiments provided by the present application, the time stamp is obtained to distinguish the contents at different times, the unique self-increment code under the time stamp is generated to distinguish the contents that request to generate the identification code at the same time, based on the unique self-increment code under the time stamp and the time stamp, the unique identification code of the target content can be generated, and the unique identification code is used as the real globally unique numerical identifier of the target content in the global range, so that time consistency among machines is not required, and the application range of the identification code is expanded.

In addition, the video ID is generated based on the Paxos KV cluster, and the generation of the global unique ID which does not depend on time synchronization among machines can be conveniently realized. Because Paxos KV has very high write performance (10 ten thousand times/second), large-scale identification code generation can be supported without difficulty, and by using some pre-allocation methods, very high identification code generation performance can be achieved. The method also has the characteristic of high availability due to the realization of service clustering distributed deployment.

The following are embodiments of the apparatus of the present application that may be used to perform embodiments of the method of the present application. For details which are not disclosed in the embodiments of the apparatus of the present application, reference is made to the embodiments of the method of the present application.

Referring to fig. 6, a block diagram of a device for generating a content identifier according to an embodiment of the present application is shown. The device has the function of realizing the method for generating the content identification. The apparatus 600 may include: a timestamp acquisition module 601, an incremental code acquisition module 602, and an identification code generation module 603.

The timestamp obtaining module 601 is configured to obtain a timestamp corresponding to the target content.

A self-increment code obtaining module 602, configured to obtain a unique self-increment code of the target content under the timestamp, where the self-increment code is used to distinguish each content that is requested to generate an identification code at the same time.

An identification code generating module 603, configured to generate an identification code of the target content based on the timestamp and the incremental code.

In an exemplary embodiment, the incremental code acquisition module 602 is configured to: