阅读说明：本技术 写入操作的控制方法、装置和设备 (Control method, device and the equipment of write operation ) 是由 周文静 周飞龙 冯小珂 于 2019-08-28 设计创作，主要内容包括：本申请公开了一种写入操作的控制方法、装置和设备,涉及大数据技术领域。具体实现方案为：通过设置第一阈值和第二阈值,并将SSTable文件的当前数量分别与第一阈值和第二阈值进行比较,若当前数量大于或等于第一阈值,且小于第二阈值,说明继续写入的话,SSTable文件的数量会存在大于第二阈值的风险,则在单位时间内的前预设时间段内拒绝所有数据写入操作,待预设时间段结束后,再在除预设时间段之外的时间段内允许所有数据写入操作,从而通过前该预设时间段控制拒绝继续写入操作,仅执行SSTable文件的合并,以减少底层的SSTable文件的数量,从而提高了控制的灵活性,使得控制效果较好。(This application discloses a kind of control method of write operation, device and equipment, are related to big data technical field.Specific implementation are as follows: pass through setting first threshold and second threshold, and the current quantity of SSTable file is compared with first threshold and second threshold respectively, if current quantity is greater than or equal to first threshold, and it is less than second threshold, illustrate if continuing to write to, the quantity of SSTable file can have the risk greater than second threshold, then refuse all data write operations in the preceding preset time period within the unit time, after preset time period, allow all data write operations within the period in addition to preset time period again, to continue to write to operation by preceding preset time period control refusal, only execute the merging of SSTable file, to reduce the quantity of the SSTable file of bottom, to improve the flexibility of control, so that control effect is preferable.)

1. a kind of control method of data write operation characterized by comprising

Obtain the current quantity of the string table SSTable file of sequence；

If the current quantity be greater than or equal to first threshold, and be less than second threshold, then within the unit time before preset when Between all data write operations of refusal in section, to the preset time period after, then in addition to the preset time period Allow all data write operations in period.

2. the method according to claim 1, wherein refusing in the preceding preset time period within the unit time Before all data write operations, further includes:

It obtains in the unit time, refuses the refusal ratio of all data write operations；

It according to the refusal ratio, determines in the unit time, refuses the preset time period of all data write operations.

3. according to the method described in claim 2, it is characterized in that, refusing all data in the acquisition unit time The refusal ratio of write operation, comprising:

Calculate separately the first difference and the current quantity and the first threshold of the second threshold and the first threshold The second difference；

It according to first difference and second difference, obtains in the unit time, refuses all data write operations Refusal ratio.

4. according to the method described in claim 3, it is characterized in that, described according to first difference and second difference, It obtains in the unit time, refuses the refusal ratio of all data write operations, comprising:

Calculate the ratio of second difference Yu first difference；

It according to the ratio of second difference and first difference, obtains in the unit time, refuses all data write-ins The refusal ratio of operation.

5. according to the method described in claim 4, it is characterized in that, described according to second difference and first difference Ratio obtained in the unit time, refused the refusal ratio of all data write operations, comprising:

According toIt obtains in the unit time, refuses the refusal ratio of all data write operations；

Wherein, P indicates to refuse the refusal ratio of all data write operations, and n is an integer greater than 0, and v2 indicates that second is poor Value, v1 indicate the first difference.

6. method according to claim 1-5, which is characterized in that the method also includes:

If the current quantity is less than the first threshold, allow to execute all data write operations, until the current number Amount is greater than or equal to the first threshold.

7. method according to claim 1-5, which is characterized in that the method also includes:

If the current quantity is greater than or equal to the second threshold, all data do not allow to execute write operation, until The quantity of the SSTable file is less than the second threshold.

8. a kind of control device of data write operation characterized by comprising

Module is obtained, the current quantity of the string table SSTable file for obtaining sequence；

Control module if being greater than or equal to first threshold for the current quantity, and is less than second threshold, then in the unit time All data write operations of refusal in interior preceding preset time period, to the preset time period after, then except described default Allow all data write operations in period except period.

9. device according to claim 8, which is characterized in that

The acquisition module is also used to obtain the refusal ratio for refusing all data write operations in the unit time；

The control module is also used to be determined in the unit time according to the refusal ratio, refuses all data write-ins and grasp The preset time period of work.

10. device according to claim 9, which is characterized in that

The acquisition module, specifically for calculating separately the first difference of the second threshold and the first threshold and described Second difference of current quantity and the first threshold；And according to first difference and second difference, the list is obtained In the time of position, refuse the refusal ratio of all data write operations.

11. device according to claim 10, which is characterized in that

The acquisition module, specifically for calculating the ratio of second difference and first difference；And according to described second The ratio of difference and first difference, obtained in the unit time, refused the refusal ratio of all data write operations.

12. device according to claim 11, which is characterized in that

The acquisition module is specifically used for basisIt obtains in the unit time, refuses all data write-ins The refusal ratio of operation；

Wherein, P indicates to refuse the refusal ratio of all data write operations, and n is an integer greater than 0, and v2 indicates that second is poor Value, v1 indicate the first difference.

13. according to the described in any item devices of claim 8-12, which is characterized in that

The control module allows to execute all data write-ins if being also used to the current quantity less than the first threshold Operation, until the current quantity is greater than or equal to the first threshold.

14. according to the described in any item devices of claim 8-12, which is characterized in that

The control module, if being also used to the current quantity more than or equal to the second threshold, all data do not permit Perhaps write operation is executed, until the quantity of the SSTable file is less than the second threshold.

15. a kind of electronic equipment characterized by comprising

At least one processor；And

The memory being connect at least one described processor communication；Wherein,

The memory is stored with the instruction that can be executed by least one described processor, and described instruction is by described at least one It manages device to execute, so that at least one described processor is able to carry out data write operation of any of claims 1-7 Control method.

16. a kind of non-transitory computer-readable storage medium for being stored with computer instruction, which is characterized in that the computer refers to Enable the control method for making the computer perform claim require data write operation described in any one of 1-7.

17. a kind of control method of data write operation characterized by comprising

Obtain the current quantity of the string table SSTable file of sequence；

If the current quantity is greater than or equal to first threshold, and is less than second threshold, then refuse institute in preceding preset time period There is data write operation, then allows all data write operations within the period in addition to the preset time period.

Technical field

The application design data processing technology field more particularly to big data technical field.

Background technique

LSM tree is a kind of data structure based on disk comprising two kinds of nodes, MemoryTable node and SSTable (subsequent abbreviation SST) node.In memory, when there is data write-in, data can first be written into more MemoryTable A MemoryTable；After the quantity of MemoryTable reaches threshold value, the data in MemoryTable can be written to more In a SSTable.It can be triggered again after the quantity of SSTable reaches threshold value merging (that is, Compaction), multiple SSTable is merged into a SSTable.When writing speed is excessive, merges Compaction speed and does not catch up with writing speed, The quantity of the SSTable of bottom can be more and more.Due to needing to inquire each SSTable when reading data, SSTable is got over More, the delay for reading data is bigger.

In order to control the quantity of bottom SSTable, in the prior art, by the way that the threshold for limiting SSTable quantity is arranged Value only executes SSTable and merges so that just no longer allowing any write-in when SSTable quantity reaches threshold value, only waits and closes The quantity of bottom SSTable after and could be written again after being less than threshold value.But existing method is used, in bottom Before the quantity of SSTable reaches threshold value, write-in is not affected completely, can be normally written；Once reaching threshold value, write Inbound traffics will appear the drop of cliff of displacement formula, therefore, using existing method, when controlling the quantity of bottom SSTable, control Flexibility is not high, and control effect is bad.

Summary of the invention

The embodiment of the present application provides control method, device and the equipment of a kind of write operation, in control bottom SSTable Quantity when, the flexibility of control is improved, so that control effect is preferable.

In a first aspect, the embodiment of the present application provides a kind of control method of data write operation, this method may include:

Obtain the current quantity of SSTable file.

If the current quantity is greater than or equal to first threshold, and be less than second threshold, then within the unit time before it is pre- If all data write operations of refusal in the period, to the preset time period after, then except the preset time period it Allow all data write operations in the outer period.

It can be seen that in the embodiment of the present application, when controlling the quantity of bottom SSTable, current quantity is greater than or waits In first threshold, and it is less than second threshold, illustrates if continuing to write to, the quantity of SSTable file, which can exist, is greater than the second threshold The risk of value then refuses all data write operations in the preceding preset time period within the unit time, i.e., in the preceding preset time In section, the merging of SSTable file is only executed, to reduce the quantity of the SSTable file of bottom, to the end of preset time period Afterwards, then within the period in addition to preset time period allow all data write operations, to pass through the preceding preset time Section control refusal continues to write to operation, executes the merging of SSTable file, only to reduce the number of the SSTable file of bottom Amount, so that the flexibility of control is improved, so that control effect is preferable.

In one possible implementation, refuse all data in the preceding preset time period within the unit time Before write operation, further includes:

It obtains in the unit time, refuses the refusal ratio of all data write operations；And according to the refusal ratio Example, determines in the unit time, refuses the preset time period of all data write operations.

In this way after preset time period is calculated, it can refuse to own in the preceding preset time period within the unit time Data write operation, to preset time period after, then allow all data to write within the period in addition to preset time period Enter operation, to continue to write to operation by preceding preset time period control refusal, executes the merging of SSTable file, only to subtract The quantity of the SSTable file of few bottom, so that the flexibility of control is improved, so that control effect is preferable.

In one possible implementation, described to obtain in the unit time, refuse all data write operations Refusal ratio, comprising:

Calculate separately the first difference and the current quantity and described the of the second threshold and the first threshold Second difference of one threshold value；And according to first difference and second difference, obtain in the unit time, refuses institute There is the refusal ratio of data write operation.

By refusing the refusal of all data write operations according in the first difference and the second difference unit of account time Ratio, to improve the accuracy of preset time period calculating according to refusal ratio-dependent preset time period.

In one possible implementation, described according to first difference and second difference, obtain the list In the time of position, refuse the refusal ratio of all data write operations, comprising:

Calculate the ratio of second difference Yu first difference；And it is poor according to second difference and described first The ratio of value obtained in the unit time, refuses the refusal ratio of all data write operations, equally can be improved default The accuracy that period calculates.

In one possible implementation, the ratio according to second difference and first difference obtains In the unit time, refuse the refusal ratio of all data write operations, comprising:

According toIt obtains in the unit time, refuses the refusal ratio of all data write operations.

Wherein, P indicates to refuse the refusal ratio of all data write operations, n is one be greater than 0 integer, v2 indicates the Two differences, v1 indicate the first difference.

In one possible implementation, the method also includes:

If the current quantity is less than the first threshold, allow to execute all data write operations, until described work as Preceding quantity is greater than or equal to the first threshold.

It can be seen that illustrating the current of SSTable file when the current quantity of SSTable file is less than first threshold Negligible amounts, the quantity that continues to execute write operation not and will lead to SSTable file is excessive, at this point it is possible to continue to execute all Data write operation allows to execute all data write operations.It should be noted that not being unconfined to allow to hold herein All data write operations of row, but when the current quantity of SSTable file is greater than or equal to first threshold, Ji Keting Only continue to execute data write operation.

In one possible implementation, the method also includes:

If the current quantity is greater than or equal to the second threshold, all data do not allow to execute write operation, Until the quantity of the SSTable file is less than the second threshold.

It can be seen that illustrating SSTable file when the current quantity of SSTable file is greater than or equal to first threshold Current quantity have been subjected to more, then do not allow to execute any write operation, due to needing to inquire each SSTable when reading data, Therefore, the current quantity of SSTable file is more, and the delay that data are read from SSTable file is bigger, at this point it is possible to refuse Exhausted all data write operations, i.e., all data do not allow to execute write operation, until the quantity of SSTable file is less than Second threshold, so as to avoid because the quantity of SSTable file is excessive and the time delay for reading data of exceeding the time limit is larger, thus Reduce the time delay of reading data.

Second aspect, the embodiment of the present application also provide a kind of control device of data write operation, the data write operation Control device may include:

Module is obtained, for obtaining the current quantity of SSTable file；

Control module if being greater than or equal to first threshold for the current quantity, and is less than second threshold, then in list All data write operations of refusal in preceding preset time period in the time of position, to the preset time period after, then removing institute Stating allows all data write operations in the period except preset time period.

In one possible implementation, the acquisition module was also used to obtain in the unit time, refused institute There is the refusal ratio of data write operation.

The control module is also used to determine in the unit time according to the refusal ratio, refuse all data The preset time period of write operation.

In one possible implementation, the acquisition module, specifically for calculating separately the second threshold and institute State the first difference of first threshold and the second difference of the current quantity and the first threshold；And it is poor according to described first Value and second difference, obtain in the unit time, refuse the refusal ratio of all data write operations.

In one possible implementation, the acquisition module is specifically used for calculating second difference and described the The ratio of one difference；And according to the ratio of second difference and first difference, obtain in the unit time, refuses The refusal ratio of all data write operations.

In one possible implementation, the acquisition module is specifically used for basisDescribed in acquisition In unit time, refuse the refusal ratio of all data write operations；

Wherein, P indicates to refuse the refusal ratio of all data write operations, n is one be greater than 0 integer, v2 indicates the Two differences, v1 indicate the first difference.

In one possible implementation, the control module, if being also used to the current quantity less than described first Threshold value then allows to execute all data write operations, until the current quantity is greater than or equal to the first threshold.

In one possible implementation, the control module, if being also used to the current quantity more than or equal to institute Second threshold is stated, then all data do not allow to execute write operation, until the quantity of the SSTable file is less than described Second threshold.

The third aspect, the embodiment of the present application also provide a kind of electronic equipment, which may include:

At least one processor；And

The memory being connect at least one described processor communication；Wherein,

The memory be stored with can by least one described processor execute instruction, described instruction by it is described at least One processor executes, so that at least one described processor is able to carry out the described in any item data of above-mentioned first aspect and writes Enter the control method of operation.

Fourth aspect, the embodiment of the present application, which also provides, a kind of to be stored with the non-instantaneous of computer instruction and computer-readable deposits Storage media, the computer instruction is for making the computer execute the described in any item data write-in behaviour of above-mentioned first aspect The control method of work.

5th aspect, the embodiment of the present application also provide a kind of control method of data write operation, the data write operation Control method may include:

Obtain the current quantity of SSTable file.

If the current quantity is greater than or equal to first threshold, and is less than second threshold, then refused in preceding preset time period Exhausted all data write operations, then allow all data write operations within the period in addition to the preset time period.

One embodiment in above-mentioned application has the following advantages that or the utility model has the advantages that in the quantity for controlling bottom SSTable When, by setting first threshold and second threshold, so that when getting the current quantity of SSTable file；By SSTable The current quantity of file is compared with first threshold and second threshold respectively, if current quantity is greater than or equal to first threshold, And it is less than second threshold, illustrate if continuing to write to, the quantity of SSTable file can have the risk greater than second threshold, then Refuse all data write operations in preceding preset time period within the unit time, i.e., in the preceding preset time period, only holds The merging of row SSTable file, to reduce the quantity of the SSTable file of bottom, to preset time period after, then removing Allow all data write operations in period except preset time period, to pass through preceding preset time period control refusal Operation is continued to write to, the merging of SSTable file is only executed, to reduce the quantity of the SSTable file of bottom, to improve The flexibility of control, so that control effect is preferable.

Other effects possessed by above-mentioned optional way are illustrated hereinafter in conjunction with specific embodiment.

Detailed description of the invention

Attached drawing does not constitute the restriction to the application for more fully understanding this programme.Wherein:

Fig. 1 can be achieved on a kind of scene figure of the control method of the write operation of the embodiment of the present application；

Fig. 2 is the flow diagram of the control method of the data write operation provided according to the application first embodiment；

Fig. 3 is the flow diagram of the control method of the data write operation provided according to the application second embodiment；

Fig. 4 is between the current quantity and refusal ratio of the SSTable file provided according to the application second embodiment The schematic diagram of relationship；

Fig. 5 is the structural schematic diagram of the control device of the data write operation provided according to the application 3rd embodiment；

Fig. 6 is the block diagram according to the electronic equipment of the control method of the data write operation of the embodiment of the present application.

Specific embodiment

It explains below in conjunction with exemplary embodiment of the attached drawing to the application, including each of the embodiment of the present application Kind details should think them only exemplary to help understanding.Therefore, those of ordinary skill in the art should recognize Know, it can be with various changes and modifications are made to the embodiments described herein, without departing from the scope and spirit of the present application. Equally, for clarity and conciseness, descriptions of well-known functions and structures are omitted from the following description.

In embodiments herein, "at least one" refers to one or more, " multiple " refer to two or two with On."and/or" describes the incidence relation of affiliated partner, indicates may exist three kinds of relationships, for example, A and/or B, it can be with table Show: individualism A exists simultaneously A and B, these three situations of individualism B, wherein A, B can be odd number or plural number.At this In the verbal description of application, character "/" typicallys represent the relationship that forward-backward correlation object is a kind of "or".

Fig. 1 can be achieved on a kind of scene figure of the control method of the write operation of the embodiment of the present application, exemplary, ask As shown in connection with fig. 1, the control method of the write operation can be applied in the data structure of LSM tree (LSM Tree).When there is number When according to write-in, data can first be written into multiple memory tables (MemoryTable)；When the quantity of MemoryTable reaches threshold value Afterwards, the data in MemoryTable can be written to multiple sequences string table (Sorted String Table, SSTable in).After the quantity of SSTable reaches threshold value, multiple SSTable are merged into a SSTable.When write-in speed When spending big, aggregate speed does not catch up with writing speed, and the quantity that will lead to the SSTable of bottom can be more and more.

In order to control the quantity of bottom SSTable, in the prior art, by the way that the threshold for limiting SSTable quantity is arranged Value only executes SSTable and merges so that just no longer allowing any write-in when SSTable quantity reaches threshold value, only waits and closes The quantity of bottom SSTable after and could be written again after being less than threshold value.But existing method is used, in bottom Before the quantity of SSTable reaches threshold value, write-in is not affected completely, can be normally written；Once reaching threshold value, write Inbound traffics will appear the drop of cliff of displacement formula, therefore, using existing method, when controlling the quantity of bottom SSTable, control Flexibility is not high, and control effect is bad.

In order to improve the flexibility of control, so that control effect is preferable, the embodiment of the present application provides a kind of write operation Control method, by setting first threshold and second threshold, so that when getting the current quantity of SSTable file；It will The current quantity of SSTable file is compared with first threshold and second threshold respectively, if current quantity is greater than or equal to the One threshold value, and it is less than second threshold, illustrate if continuing to write to, the quantity of SSTable file, which can exist, is greater than second threshold Risk then refuses all data write operations in the preceding preset time period within the unit time, i.e., in the preceding preset time period It is interior, only execute SSTable file merging, to reduce the quantity of the SSTable file of bottom, to preset time period after, Allow all data write operations within the period in addition to preset time period again, to pass through the preceding preset time period control System refusal continues to write to operation, only executes the merging of SSTable file, to reduce the quantity of the SSTable file of bottom, from And the flexibility of control is improved, so that control effect is preferable.

It should be noted that in the embodiment of the present application, refusing all numbers in the preceding preset time period within the unit time According to write operation, i.e., in the preceding preset time period, the merging of SSTable file is executed, only to reduce the SSTable of bottom The quantity of file.And allow all data write operations within the period in addition to preset time period, i.e., when this is except presetting Between in period except section, that is, execute the write operation of all data, while can also execute the merging of SSTable file, with Reduce the quantity of the SSTable file of bottom.

It is understood that for all data write operations refused in the preceding preset time period, when it is re-initiated When data write operation is requested, then it can allow to execute these data write operations, therefore, for executing data write operation For user, these data write operations are only time-consuming larger in writing process, but not will lead to service disconnection.

It is described in detail below with technical solution of the specific embodiment to the application.These are specific real below Applying example can be combined with each other, and the same or similar concept or process may be repeated no more in some embodiments.