阅读说明：本技术 具有集成麦克风的便携式设备的实验室系统及相关方法 (Laboratory system with portable device integrated microphone and related method ) 是由 奥利弗·克罗尔 盖塔诺·布兰达 S·西尔伯 乌尔夫·施内伯格 因加·胡森 迈克尔·巴达斯 于 2020-03-13 设计创作，主要内容包括：本发明涉及一种实验室系统,包括：至少一个实验室设备(252；254；406；408；412；414；416),其设计用于分析和/或合成化学物质；至少一个实验室软件模块(248),其设计用于处理由该至少一个实验室设备获得的数据；具有控制软件(222)的数据处理设备(213),该控制软件提供用于操作该至少一个实验室设备和/或该至少一个实验室软件模块的接口；以及具有麦克风(214)的便携式设备(212),其中,该设备通过网络(236)以可交互操作的方式被连接至该控制软件,其中,该设备设计用于在与控制软件交互操作的情况下允许用户(202)通过对麦克风的语音输入(204)来无介入地操作该至少一个实验室设备和/或至少一个实验室软件模块。(The invention relates to a laboratory system comprising: at least one laboratory device (252; 254; 406; 408; 412; 414; 416) designed for analyzing and/or synthesizing a chemical substance; at least one laboratory software module (248) designed for processing data obtained by the at least one laboratory device; a data processing device (213) with control software (222) providing an interface for operating the at least one laboratory device and/or the at least one laboratory software module; and a portable device (212) having a microphone (214), wherein the device is interactively connected to the control software via a network (236), wherein the device is designed to allow a user (202) to operate the at least one laboratory device and/or the at least one laboratory software module without intervention by means of a voice input (204) to the microphone in the case of interaction with the control software.)

1. A laboratory system comprising:

at least one laboratory device (252; 254; 406; 408; 412; 414; 416) designed for analyzing and/or synthesizing chemical substances,

-at least one laboratory software module (248) designed for processing data obtained by the at least one laboratory device;

-a data processing device (213) with control software (222) providing an interface for operating the at least one laboratory device and/or the at least one laboratory software module; and

-a portable device (212) with a microphone (214), wherein the device is interactively connected to the control software via a network (236), wherein the device is designed to allow a user (202) to operate the at least one laboratory device and/or the at least one laboratory software module in an intervention-free manner via a speech input (204) to the microphone in case of interaction with the control software.

2. The laboratory system according to claim 1, wherein the control software is designed as a virtual laboratory assistant for operating the at least one laboratory device and the at least one laboratory software module.

3. Laboratory system according to one of the preceding claims, wherein the portable device is configured for:

-receiving a speech signal (206) of the user through the microphone;

-forwarding the received speech signal to the control software for converting the speech signal into text (208) by the control software or by a speech to text conversion system (226) operatively connected to the control software.

4. The laboratory system according to claim 3, wherein the portable device further comprises a speaker (218), wherein the portable device is configured for:

-receiving a function execution result (242) from the control software in response to said speech signal transmission, wherein the function is executed by the at least one laboratory device and/or the at least one laboratory software module according to the text; and is

-outputting the result to the user via the speaker.

5. Laboratory system according to one of the preceding claims, wherein the portable device is located in a room (404) common to the at least one laboratory device and/or in a room common to a host computer (246) of a laboratory facility comprising the at least one laboratory device.

6. Laboratory system according to one of the preceding claims, wherein the portable device is a single board computer and in particular a raspberry pie.

7. Laboratory system according to one of the preceding claims, wherein the control software is configured for:

-receiving from the portable device a speech signal (206) picked up by the microphone based on said speech input, wherein the speech signal comprises common and term words spoken by the user;

-inputting the received speech signal into a speech to text conversion system, wherein the speech to text conversion system only supports converting the speech signal into a target vocabulary (234) not comprising said term;

-receiving from the speech to text conversion system text (208) generated by the speech to text conversion system in accordance with the speech signal;

-generating a corrected text (210) on the basis of the received text, wherein the corrected text is generated by the control software alone or by the control software with interpretation by means of text correction software operatively connected to the control software, wherein the corrected text is generated by automatically replacing words and phrases of a target vocabulary in the received text by term words according to an assignment table (238), wherein the assignment table assigns each of a plurality of term words at least one word from the target vocabulary, wherein the at least one word of the target vocabulary assigned to a term word is a word or phrase which the speech-to-text conversion system misrecognizes when the term word is entered in the form of a speech signal; and is

-using the correction text to cause the at least one laboratory facility and/or the at least one laboratory software module to analyze, synthesize and/or said software function according to information in said correction text.

8. The laboratory system according to claim 7, wherein the term word is a word from one of the following categories:

the names of chemical substances, in particular paints and varnishes;

-physical, chemical, mechanical, optical or tactile properties of chemical substances;

-names of laboratory and chemical industrial equipment;

-names of laboratory consumables and laboratory requisites;

trade names in the paint and varnish field.

9. Laboratory system according to one of the preceding claims 7 to 8, wherein the words and phrases of the target language stored in the allocation table represent erroneous text output of the speech to text conversion system, said text output being generated on the basis of a number of different human speech input terms.

10. Laboratory system according to one of the preceding claims, wherein the at least one laboratory device is a plurality of laboratory devices, wherein the laboratory devices are each selected from the group comprising:

-an analysis station (406; 408) designed for analyzing a chemical substance or mixture; and/or

-a synthesis station (412; 414) designed to carry out the synthesis of a chemical substance or mixture; and/or

-a pre-or post-treatment station (410) designed to modify the chemical substances before or after the analysis or synthesis step, so as to enable the subsequent working or transport step.

11. Laboratory system according to one of the preceding claims, wherein the at least one laboratory device is a plurality of laboratory devices, which laboratory devices all serve as respective chemical processing stations, further comprising:

-a transport unit (256) designed for transporting substances for synthesis or analysis to the processing stations, to allow the processing stations to perform at least one processing step on the respective substances, respectively;

-a master computer of a laboratory facility (244) comprising a plurality of laboratory devices, wherein the master computer comprises master software configured for arranging the processing steps and the transportation of the chemical substance based on instructions in the form of structured text.

12. Laboratory system according to one of the preceding claims, wherein the control software comprises or interoperates with an NLP processor (258), wherein the control software is configured for:

-converting, using the NLP processor, the corrected text into structured text (211) that can be interpreted by a target system, wherein the target system is the master software of the at least one laboratory software module, the at least one laboratory device and/or a master computer of a laboratory facility that includes the at least one laboratory device; and is

-entering the structured text into the target system to cause the target system to perform software functions and/or hardware functions related to a laboratory.

13. Laboratory system according to one of the preceding claims, wherein the non-invasive operation comprises: converting the speech input into text (210; 211), wherein,

-said at least one laboratory software module is a chemical substance database designed to interpret said input text as a search input and to determine information about said search input within the database and to return it as a result to the control software; and/or

-said at least one laboratory software module is simulation software designed for simulating the properties of chemical products, in particular paints and varnishes, based on predefined mixture specifications, wherein the simulation software is designed for interpreting said input text as a specification of the product of which the properties should be simulated;

-said at least one laboratory software module is a master control software (248) of a master control computer of a laboratory facility (244) containing the at least one laboratory device, wherein the master control software is designed to cause said at least one laboratory device and transport unit to perform said plurality of process steps and transport of chemicals between said laboratory devices according to the information contained in said input text.

14. Laboratory system according to one of the preceding claims, wherein the control software comprises a register with a plurality of user accounts registered with the control software, wherein the control software comprises a speech recognition function for recognizing each of the registered users by voice, wherein the control software is designed for allowing the user to operate without intervention only the laboratory devices and/or laboratory software modules which the user has the right to operate by voice input to the microphone.

15. A method for controlling laboratory equipment and/or laboratory software modules, comprising:

-providing at least one laboratory device (252; 254; 406; 408; 412; 414; 416) and at least one laboratory software module (248), wherein the at least one laboratory device is designed for analyzing and/or synthesizing chemicals, wherein the at least one laboratory software module is a laboratory software module for processing data obtained by the at least one laboratory device (252; 254; 406; 408; 412; 414; 416);

-providing a data processing device (213) with control software (222), wherein the control software provides a user with an interface for operating the at least one laboratory device and/or the at least one laboratory software module; and is

-providing a portable device (212) with a microphone (214), wherein the device is interactively connected to the control software via a network (236), wherein the device is designed to allow the user to operate without intervention said at least one laboratory device and/or said at least one laboratory software module by voice input to the microphone in case of interaction with the control software.

16. The method of claim 15, wherein the providing of the device comprises: the equipment is placed in a common room (404) with the at least one laboratory equipment and/or in a common room with a host computer of a laboratory facility including the at least one laboratory equipment.

Technical Field

The present invention relates to the control of laboratory equipment, in particular of complex laboratory systems.

Prior Art

In chemical laboratories, because of the various material-derived and equipment-derived hazards, many rules apply to ensure safe operation there. Depending on the type of laboratory, the activities performed there and the substances used, the following safety regulations may therefore exist in particular: personal protective equipment should be worn which may include protective glasses or face shields and protective gloves in addition to a lab coat. Typically, food and beverages are not allowed to be carried and consumed, and to avoid contamination, office and laboratory work areas, including desks, brochures, paper product documentation, computer workstations, and internet access, are spatially separated from one another. The spatial separation can be defined such that a changeover between the office area and the laboratory area can only be effected via a security gate. Provision may also be made for the safety suit to be taken off when leaving the laboratory area.

Safety regulations can sometimes make the work process quite difficult: laboratory devices and in particular complex laboratory systems with a plurality of laboratory devices provide the user with a separate operating surface, which is usually formed by a screen, a mouse and a keyboard and/or a touch screen. In general, it is a standard instrument component here, which cannot or is difficult to handle with laboratory gloves. In turn, erroneous inputs occur frequently, which slow down the workflow and may also lead to erroneous control and operation of the laboratory equipment. Removing gloves before operating the laboratory system poses a contamination risk, delays work and is sometimes even impossible from a safety standpoint.

In individual cases there are laboratory devices with large-sized keyboards, for example in the form of large touch screens, to facilitate gloved input. However, this special hardware is expensive and not all laboratory equipment is suitable. In particular, standard computers and standard notebook computers do not have such "glove-ready" keyboards. However, the possible way to also operate a standard computer with a traditional internet browser is becoming more and more important in a laboratory situation, e.g. in order to be able to retrieve an online available chemical database.

In the case of chemical or biological laboratories, the possible ways currently available for controlling or interacting with laboratory equipment and laboratory systems are very limited and inefficient.

SUMMARY

It is an object of the present invention to provide an improved laboratory system and method for controlling at least one laboratory device or laboratory software module, which allow an improved control of laboratory-related hardware functions and software functions, in particular in a laboratory environment, according to the independent claims. Embodiments of the invention are specified in the dependent claims. The embodiments of the present invention can be freely combined with each other unless they are mutually exclusive.

In one aspect, the invention relates to a laboratory system. The laboratory system comprises at least one laboratory device designed for analyzing and/or synthesizing chemical substances. The laboratory system also comprises at least one laboratory software module, which is designed to process data obtained from the at least one laboratory device. The laboratory system further comprises a data processing device with control software which provides an interface for operating the at least one laboratory device and/or the at least one laboratory software module. The laboratory system also includes a portable device with a microphone. The device is interactively connected to the control software via a network and is designed to allow a user to operate the at least one laboratory device and/or the at least one laboratory software module without intervention by voice input to the microphone if the device is interactively operated with the control software.

Embodiments of the present invention are advantageous in that they allow easier, better inspection and control of laboratory equipment, for example in biochemical laboratories, by one or more people. Since the monitoring of the laboratory equipment and the laboratory software modules associated therewith is performed by voice input, it is not necessary to take off gloves or take other time-consuming measures to input control commands. The speech-based input allows information to be input anywhere within the acoustic range of the portable device, particularly within the laboratory zone. The person making the speech input therefore does not have to leave his laboratory workstation or he can move freely between different workstations in the laboratory zone, provided that he does not leave the reception range of the microphone of the portable device at the time.

"controlling a target system by means of speech input" is known per se. Therefore, there are inexpensive terminals on the market with microphones and powerful applications for voice-based input commands, such as Echo Dot with alexa (amazon), smart speakers of various manufacturers with cortana (microsoft), Google Home Max/Min with Google assistant (Google), and HomePod with siri (apple). They are designed to support end-user daily activities such as shopping, selecting radio programs or booking hotels. The terminal and the application are thus designed for everyday situations and also only support common words. Existing terminals are neither designed for, nor suitable for use in laboratories or for term identification. The terminals that are commercially available today are not designed for post-modification by the user, making it impossible for the user to use the smart speaker system with voice recognition functionality that is currently available on the market in the laboratory area. According to the embodiments of the present invention, a control software capable of correctly converting a voice input with a term into a text alone or in a case of interacting with other constituent elements as well has been developed, and a portable device having a microphone, which is interoperable with the control software, has been developed. A laboratory system is thus provided which enables a simple and non-intrusive voice-based control of laboratory equipment and associated laboratory software modules, as is today achieved in the smart home field on the basis of, for example, amazon Echo Dot.

According to an embodiment, the control software is designed as a virtual laboratory assistant for operating the at least one laboratory facility and the at least one laboratory software module.

This may be advantageous because the user input of control commands and the interaction between the user and the laboratory device or the laboratory software module is further simplified. For example, a virtual laboratory assistant may implement one or more sound generators that each generate computer-generated, but natural-sounding sounds and read the results of performing a particular function by a laboratory device or laboratory software module to a user in natural sound. It is also possible that the virtual laboratory assistant can also "tell" a certain name, i.e. check for this whether the speech input or the text generated thereby mentions his name, and in the case mentioned perform or cause the target system to perform certain functions by itself. Thus, a laboratory worker may interact with the control software and virtual laboratory assistants implemented thereby in a similar manner as when interacting with human colleagues.

According to an embodiment, the portable device is configured to receive a voice signal of a user through the microphone and to forward the received voice signal to the control software. The speech signal is forwarded to enable conversion of the speech signal to text by the control software or by a speech to text conversion system operatively connected to the control software.

According to an embodiment of the invention, the portable device further comprises a speaker. The portable device is configured to receive the function execution result from the control software as a response to the voice signal transmission. The functions are performed by at least one laboratory device and/or by at least one laboratory software module according to the text. The portable device is also configured to output the reception result to the user through the speaker. The received result may be, for example, text that is converted into an audio signal using text-to-speech conversion software of the device. According to an alternative embodiment, the control software includes a text-to-speech conversion function, and the result has been transmitted to the portable device in the form of an electronic audio signal, so that in some embodiments a terminal with only low computing power no longer has to achieve a text-to-speech conversion.

It may be advantageous to output the result as an audio signal through a loudspeaker, since this does not restrict the freedom of movement of the user, which is achieved by inputting commands based on speech. Thus, the user can move freely from any point of the laboratory that is within the microphone detection range and within the audible range of the loudspeaker sound output, while outputting control commands and retrieval requests through voice input, and receiving results of execution of these commands and/or information retrieval. This is particularly useful in laboratory work environments, as laboratory workers often need to walk between different equipment and laboratory zones, for example, to reload or configure machines, check the progress of work steps, release blocked components or transport boxes, and the like.

According to some embodiments, the results of the software functions or hardware functions performed by the laboratory software module or the laboratory device according to the speech input are displayed to the user through a screen. This may be done through a speaker, in addition to or instead of such output. Especially in the case of complex and extensive results, such as documents determined during a database or internet query, output on a screen may be advantageous because a user may not be able to receive and understand all of his information if the long text is output through a speaker only.

According to an embodiment, the portable device is located in a common room with the at least one laboratory device and/or with a host computer of a laboratory facility containing the at least one laboratory device.

This may be advantageous because most microphone types, and in particular the microphone of portable devices, are able to pick up speech input in the room, and/or because the volume of the most common speaker, and in particular the speaker of the portable device, is sufficient for a person in the same room to hear and understand.

According to some embodiments, the laboratory system comprises two or more of the portable devices described herein with a microphone and optionally a speaker. The apparatus is preferably placed in a laboratory dispersed uniformly. This may be advantageous, since thereby a particularly good coverage may be achieved both when receiving the speech input and when outputting the result by the loudspeaker.

According to an embodiment of the invention, the portable device is a single board computer, in particular a Raspberry Pi computer.

This may be advantageous because such a single board computer is an inexpensive component designed for adjustment and modification by end users, such as by installing client side software that interacts over a network with freely configurable server side software, such as control software, so that the user can use the functions of the control software through voice input.

Such as "google assistant SDK," allow for the integration of google voice recognition services into a terminal, such as a raspberry pie. A large number of relevant video courses can be found on YouTube.

However, according to embodiments of the present invention, the portable device is not directly connected to google services or other speech to text conversion services, but rather only client software is installed that enables the portable device to interoperate with control software that coordinates the exchange of data with a speech to text conversion system connected over a network and the correction of text received from the conversion system.

For example, "raspberry pi google assistant" software is installed on the raspberry pi and is configured to send voice signals received by the portable device to the control software. The address of the control software is thus set and stored in the portable device. This may be advantageous in that for simple interaction with the laboratory data processing devices and services, a very cheap portable terminal is provided, which may be configured for interoperation with any software application, such as control software implementing a virtual laboratory assistant.

It may be advantageous to outsource this coordination function to control software because the raspberry pi computer has only low computational power, so that the conversion of speech signals to text and perhaps subsequent text correction can be performed more quickly by other more powerful devices.

According to an embodiment of the invention, the control software is configured for receiving from the portable device a speech signal picked up by the microphone based on the speech input. The speech signal includes the general and the term spoken by the user. The control software is also configured to input the received speech signal into a speech-to-text conversion system, where the speech-to-text conversion system supports only conversion of the speech signal into a target vocabulary that does not contain the term. The control software then receives text from the speech to text conversion system that was generated by the speech to text conversion system based on the speech signal. The control software is further configured to generate correction text in accordance with the received text, wherein the correction text is generated by the control software alone or by the control software in interpretation via the text correction software, the control software being operatively connected to the text correction software in a local manner or via a network. The corrected text is generated by automatically replacing words and phrases of the target vocabulary in the received text by term words according to a word allocation table in text form. The assignment table assigns each of the plurality of term words at least one word from the target vocabulary. At least one word of the target vocabulary assigned to a term word is a word or phrase that the speech-to-text conversion system misrecognizes when the term word is input in the form of a speech signal. The control software is configured to use the calibration text to cause the at least one laboratory device and/or at least one laboratory software module to perform analysis, synthesis and/or software functions based on information in the calibration text.

It may be advantageous to use a speech to text conversion system that does not support terminology for a variety of reasons, while subsequently correcting the text based on the assignment table.

On the one hand, there have been a range of generic speech to text conversion systems, such as google's "speech to text" service, which have a high degree of accuracy and sometimes free use of at least generic words and phrases, and which can be integrated into proprietary software programs through open APIs. Speech input in a laboratory environment typically includes terms and common words mixed with each other, thus at least ensuring that the common words are correctly converted to text. As google and other cloud service providers' speech-to-text conversion systems continue to evolve, laboratory equipment and laboratory management system manufacturers are generally unable to provide the same quality of service as large cloud service providers such as google, amazon, and apple can achieve.

However, the generic speech-to-text conversion system is not suitable for use in a laboratory environment by itself, since the language-to-text conversion system does not support the term ("terminology"). In biology and especially in the chemical industry, a large number of terms are used in a laboratory environment that do not appear in the common language. They cannot be recognized by the speech-to-text conversion system, and the similarly pronounced generic words are therefore misrecognized and converted to text. High accuracy of speech recognition is also particularly important, especially in chemical laboratory environments. Small errors are usually recognizable in everyday language and can be easily corrected or compensated by the user or receiving system (e.g. single/complex forms of misidentification do not result in significantly different results being returned by the corresponding input in an internet search engine), whereas in the context of chemical synthesis, small deviations (e.g. "double" instead of "triple") may already result in "identifying" a substance that is distinct from the speaker's true meaning, and the resulting product is unusable. Therefore, the speech-to-text conversion system designed for daily use is not suitable for use in biochemical laboratories with corresponding risks.

There are sometimes also speech-to-text conversion systems that are specifically designed for the relevant objects and words of a certain profession. Thus, for example, new ons corporation provides "Dragon Legal" software to law owners that includes Legal terms in addition to everyday vocabulary. A disadvantage is, however, that the vocabulary required in certain laboratories, for example in the field of the production and analysis of paints and varnishes, is so professional and dynamically variable that speech recognition software using chemical terms, which may be available, for example, from chemical standards textbooks, is generally not suitable for practice in particular companies or in particular branches of the chemical industry, since laboratories are also often trademarked with substances. These trade names may change or a large number of new trade names for related products are added each year. In particular, a large number of other products and product variants are marketed each year which can be used for the manufacture of paints and varnishes under new trade names. Even if the accuracy of the speech to text conversion system is at the level of the google or apple everyday speech system and would contain the most important chemical terms (but this is not the case), the system is less suitable for practical applications due to the dynamics and large number of names that are crucial in chemical laboratories, especially in the manufacture of paints and varnishes, since most of the practically relevant words would not be supported, or at least after a few years the words would be completely obsolete.

According to an embodiment of the invention, this problem is solved as described above by resorting to speech to text conversion systems which are known not to support the relevant terms. Therefore, no attempt was made at the outset to implement an expensive and complex special development here which serves only a small market segment and would therefore hardly achieve the recognition accuracy of the famous large scale conversion systems of amazon, google or apple, since this involves a general-language concept which should usually be taken into account and correctly recognized in addition to chemical terms in speech input. In contrast, embodiments of the present invention take advantage of the already very good recognition accuracy of existing service providers for the common language concept and correct it before outputting the recognized text. In the correction process, the recognized wrong word is replaced by a term according to the allocation table, thereby generating a corrected text that is finally output. Due to the dynamic nature of the domain and the large number of market participants, products and corresponding product names, the highly specialized terminology vocabulary should be constantly updated to keep the software in close proximity to utility, and thus is ultimately placed in the distribution table. This makes it possible to keep the state up to date with little effort. Thus, according to embodiments of the present invention, it is sufficient to modify the program source code and statistical language model, and not to recompile and/or retrain the machine learning program, modify or supplement the tables.

The new term may be supplemented simply by supplementing the new term into the allocation table along with one or more corresponding target vocabulary words that were misrecognized for the term. Thus, in terms of technology, the storage and updating of terms is completely decoupled from the real speech recognition logic. This also has the advantage of avoiding reliance on a particular provider of speech recognition services. The field of speech recognition has started and it has not been foreseen which of the multitude of parallel solutions will be the best choice in terms of recognition accuracy and/or price for a long time. According to an embodiment of the invention, the association with a particular speech to text conversion system is only done by first sending the received speech signal to the conversion system and receiving the (wrong) text. Furthermore, the allocation table contains the erroneously recognized words of the target vocabulary, which have been returned by the special conversion system for a particular term (error). Both can however easily be changed by using a different speech to text conversion system to generate the (wrong) text and to do so also by means of the different conversion system to recreate the allocation table. No complex changes to the logic of the parser and/or the neural network, for example, are required.

The combination of a speech to text conversion system in everyday language for field service employees in chemical industry or chemical production may also be advantageous according to embodiments of the present invention, since employees already often use computers or at least one smart phone during their work activities and, compared to text input by means of a keyboard, the input by speech to correction software, for example in the form of an application or a browser plug-in, makes them more attentive to the client or its activities. These users can also access and control the functionality of the laboratory devices and/or the laboratory software modules of the laboratory system by means of client software on their terminal, which interacts with the control software in order to forward the user's speech signals via the control software to the speech-to-text conversion system.

Another advantage may be that according to an embodiment, the portable device only collects the speech signal, forwards it to the control software, and optionally receives and outputs the function execution result according to the information in the speech input. The actual speech-to-text conversion of a speech signal into text, i.e. a step which is significantly computationally intensive, is performed by the speech-to-text conversion system. The speech to text conversion system may be, for example, a server connected to the control software via a network, such as the internet. Thus, long and complex speech inputs can also be entered using portable devices with low processor performance, such as smart phones or single board computers, and the computer resources on which the control software is installed are safeguarded.

According to an embodiment of the invention, the term word is a word from one of the following categories:

names of chemical substances, in particular paints and varnishes, in particular also referring to chemical names according to the chemical naming convention, for example according to the IUPAC nomenclature;

-physical, chemical, mechanical, optical or tactile properties of chemical substances;

names of laboratory and chemical industrial plants (for example trade names or proprietary names specified by the user for laboratory equipment of a laboratory);

-names of laboratory consumables and laboratory requisites;

trade names in the paint and varnish field.

According to an embodiment of the invention, the term word is a word from the chemical field, in particular the chemical industry, and in particular the paint and varnish chemical field.

According to an embodiment of the present invention, the target vocabulary is composed of a collection of generic words.

According to other embodiments of the present invention, the target vocabulary is comprised of a collection of generic terms and their derivatives. For example, the derivatives may be a dynamically created concatenated combination of two or more generic terms. For example, in german, many words, and particularly nouns, are combined from a plurality of other nouns. For example, the term "ship propeller" (schiffsshurbe) is so common that it generally appears in most general dictionaries. Whereas most commonly used dictionaries do not contain considerably less of the term used, such as "screw-on" (Befestigungsschriube). Some speech-to-text conversion systems may also recognize words such as "fastening screws" (Befestigung) by heuristics and/or neural networks, provided that the separate words "fastening" (Befestigung) and "screws" (Schraube) are part of the target word. In this sense, the term "fastening screw" (befestigungsschube) therefore also belongs to the target vocabulary of such speech-to-text conversion systems.

According to other embodiments of the present invention, the target vocabulary is composed of a collection of generic words supplemented with words formed by combining the recognized syllables. Thus, the speech to text conversion system is more flexible as to which words can be recognized, since recognition can also be done at least at the level of individual syllables and not just individual words. Syllable-based recognition is also particularly error-prone because the risk of misrecognizing words that are not present in the known vocabulary is particularly great. Because of the limited number of syllables supported or known and the limitation of typical word length on the number of combined syllables, the number of target words that can be generated based on syllables is also limited. Thus, speech to text conversion systems that support syllable-based word-making have limited target vocabulary, despite greater flexibility. Even if such a system is theoretically capable, due to its flexibility, of dynamically recognizing a number of chemical terms not contained in a previously known dictionary, in practice the recognition accuracy is so low that such a system will eventually have a target vocabulary that does not contain or support these chemical terms in practice.

In some embodiments of the invention, the target vocabulary is composed of a set of generic words, supplemented with their derivatives, and supplemented with words formed by identifying syllable combinations. These conversion systems are also based on a target vocabulary which does not contain terms or which in actual use does not recognize terms accurately enough, but rather misrecognizes other words, usually common words, and converts them into text.

Thus, a number of different speech-to-text conversion systems that are available today may be used in embodiments of the present invention, even though these systems essentially only "support" the everyday words (i.e., can correctly recognize and convert to text with sufficient accuracy).

According to an embodiment of the invention, a computer system performing text correction, i.e. for example a computer with control software or a correction computer, receives or calculates frequency information of at least some of the words in a text generated by a speech to text conversion system in accordance with a speech signal. The frequency information indicates, for each word in the text, a statistically expected frequency of occurrence of the word.

In the generation of the corrected text, only the terms of the target vocabulary in the received text are selectively replaced by the term words according to the assignment table, the statistically expected frequency of occurrence of the words being below a defined threshold value according to the received frequency information.

This may be advantageous because the user's speech input typically contains a mix of common words and terms. It may therefore also happen that the text received by the conversion system contains words of the target vocabulary, which words are assigned to the respective term in the assignment table and will usually be replaced. For example, the returned text may contain the phrase "Polymer Innovation" (Polymer Innovation). Since the phrase "polymer innovation" is assigned to the term "polymerization" in the assignment table, the phrase is typically replaced by "polymerization" in the text correction process. However, if the frequency information assigned to the phrase "polymer innovation" indicates a high probability of occurrence, the correction software will assume that the phrase "polymer innovation" is correct based on this frequency of occurrence, even though it is assigned to the corresponding term in the assignment table and thus remains the same in the text for the sake of brevity. For example, contextual analysis of words in sentences or in the entire speech input may indicate that the word "Innovation" (Innovation) is frequently present in the text alone, for example because the text is from an outsider describing the advantages of a certain polymer product. In this regard, the phrase "polymer innovation" can also refer to a properly recognized phrase. In case both "polymer" and "innovation" are not mentioned separately, this probability is reduced. Regardless of the context, words that can be derived, for example, from large text corpora also already have different probabilities of occurrence from one another themselves.

It may be advantageous to replace words according to the assignment table according to the word occurrence probability in the received text, because it is avoided that in few individual cases a word of the target language is replaced by a term error, either by itself or with a high occurrence probability in the context of the corresponding text, which replacement results in an error instead of an error correction.

According to one embodiment, the frequency of occurrence of words of the text is calculated by the speech to text conversion system and returned to the control software by the speech to text conversion system together with the text. For example, a speech to text conversion system may use Hidden Markov Models (HMMs) to calculate the probability of a word occurring in the context of a sentence. In addition to, or instead of, this, the speech-to-text conversion system may equate the frequency of occurrence of a word with the frequency of occurrence of the word in a large reference corpus. For example, all text in a newspaper over several years or another large text dataset can be used as a reference corpus. The ratio of the word count in the corpus to the total number of words in the corpus is the frequency of occurrence of the word observed in this reference corpus. The frequency information may be sent to the control software by the speech to text conversion system along with the text.

According to another embodiment, the word occurrence frequency of the text is calculated by the control software or correction software after the text is received. As described above, the probability of occurrence of each word or phrase can be calculated by means of HMMs, taking into account the text context of the word or depending on the frequency of the word in the reference corpus. For example, all text previously transmitted from the speech to text conversion system to the control software or correction software may be used as reference corpora.

Thus, according to an embodiment, the frequency information is calculated by means of a hidden markov model (e.g. by control software or correction software). For example, the expected frequency of occurrence, i.e. the probability of occurrence, may be calculated as the product of the emission probabilities of the individual words of a single word sequence, like for example estimating the probability in b.cestnik ": the key task in machine learning "(" the ninth european artificial intelligence conference proceedings ", pages 147 and 150, stockholm sweden, 1990) is described.

According to an embodiment of the invention, the program performing the text correction receives not only the text, which is generated from the speech signal by the speech to text conversion system, but also part of speech tags (POS tags) for at least some words in the text. Part-of-speech tags are received by the control software directly from the speech to text conversion system or by the correction software bypassing the control software and contain at least tags for nouns, adjectives and verbs. It is also possible that part-of-speech tags contain additional types of syntactic or semantic tags. The exact composition of the POS tag considered may also depend on the corresponding language. In the assignment table, the term words are stored in an associated manner along with their POS tags. In generating the corrected text, words in the received text whose part-of-speech tags match the target vocabulary are replaced by only the term words according to the assignment table.

This may be advantageous because the accuracy of the text correction step will thereby be improved. It may be assumed that the POS tag in the assignment table is correct, because the entries in the table are created semi-automatically in such a way that one or more speakers input a term word or term phrase into the microphone, the audio signal resulting therefrom is converted by the speech-to-text conversion system into a (wrong) word or a (wrong) phrase of the target vocabulary, and the wrong word or the wrong phrase is stored in the assignment table in an associated manner together with the term phrase. Because it is known what the term represents and whether it is, for example, a noun, verb, or adjective, the term phrase can also be stored in an associated manner with the correct POS tag when the table is created or updated. Therefore, if a word and a phrase in a text should be replaced with a term word according to the assignment table, but the part-of-speech tag of the text to be replaced does not coincide with the part-of-speech tag of the term word, this indicates that the corresponding word in the textMay still be correct. The recognition rate of the POS tag is high, so that the quality of the correction step can be improved by this measure. It is for example possible that the term word is for example the trade nameIt refers to a thermoplastic polyurethane film from scientific inc. In the table, the part-of-speech tag "noun" is assigned to this term. It is known from speech-to-text conversion systems that it often erroneously converts the spoken word "plation" into the target vocabulary word "Platin" (platinum), so that the word "Platin" of the target vocabulary is assigned to the term "plation" in an assignment table. However, in the case of the user's current speech input, this word is used as an adjective: "adding platinum-based or zinc-based catalyst.]". From the part-of-speech tag of "Platin" in the text returned by the conversion system, it is possible to recognize here: the term "Platin" is used herein correctly and should not be replaced by "Platilon".

According to an embodiment, the control software is provided in the form of a cloud service by a cloud computer system or a separate server computer system. A cloud computer system or server system may be, for example, a system operated by a laboratory operator such as a university or a corporation.

According to one embodiment, the control software performs the text correction itself or by interoperating with a program locally installed on a computer common to the control software.

According to another embodiment, the correction of the text received by the speech to text conversion system is performed by another computer, which is connected to the computer controlling the software via a network. The further computer may be, for example, a cloud system or a separate server, which is also run by the laboratory operator, for example. The control software sends the received text to the software and computer that performs the correction according to the allocation table and receives the corrected text from the software or computer ("correction software" or "correction computer"). The network connecting the control software and the remote calibration software may be, for example, the internet or an organization intranet. This embodiment may be advantageous because the access rights to functions and data for both the control software and the correction software may be better separated. If the text correction is performed on a separate correction computer or correction cloud system, the user can be given selective access to the correction computer or correction cloud system to update the tables here, without or without thus also being given access to sensitive data of the control software which can check critical functions of, for example, a laboratory device.

According to embodiments of the present invention, coordinating data exchange with the speech to text conversion system, text correction, and forwarding corrected text to the execution system is thus performed entirely by the control software or organized and coordinated by the control software. According to some embodiments of the method, the portable device is thus essentially a device with a microphone and an output interface, optionally for correcting text execution results.

According to an embodiment of the present invention, a speech-to-text conversion system is implemented as a service provided to a plurality of terminals via the internet. For example, the speech to text conversion system may be a cloud computer system with google's "speech to text" cloud service. This may be advantageous because for this there is a powerful API client database, e.g. for the. NET.

The portable device may contain client software that is pre-configured for data exchange with the control software. This means that the client software on the portable device is configured to send the speech signal over the network to the control software, so that the control software can cause the conversion of the speech signal into text, the correction of the terms of the text and the execution of functions according to the corrected text. The execution result of the corrected text is received by the portable device and output through the speaker as a response to the voice signal transmission.

According to an embodiment of the present invention, the words and phrases in the target language stored in the assignment table represent erroneous text output of the speech to text conversion system, which is generated based on the term speech input of a large number of different people.

According to an embodiment of the invention, the method comprises an allocation table generation step. For each of a plurality of terms, at least one reference speech signal is recorded which selectively reproduces the term. The reference speech signal is from at least one speaker. Also for the term phrase, at least one reference speech signal selectively reproducing the term phrase can be spoken and recorded, respectively, by at least one speaker. Other steps are substantially the same for words and phrases, and thus the following also includes the term phrase when referring to the term word. Each recorded reference speech signal is input into a speech to text conversion system. The input can in particular be made via a network, such as the internet. For each input reference speech signal, the device that has input the reference signal receives at least one word of the target vocabulary generated by the speech to text conversion system from the input reference speech signal. The device may be, for example, any data processing system with a microphone and connected to a speech to text conversion system. The input of the reference speech signal is preferably done via a device that is as similar as possible to the portable device in terms of microphone type and/or positioning with respect to noise sources to ensure that the same errors are reproducibly produced. Because the target vocabulary of the speech to text conversion system does not support the term words, at least one word (which may also be a phrase) of the target vocabulary received for each term represents an erroneous conversion. Finally, an assignment table is generated as a table which assigns to each term word for which at least one reference speech signal has been correspondingly captured at least one word of the target vocabulary in text form, which has been returned by the speech to text conversion system for the respective term word.

This may be advantageous because the table may be easily modified and supplemented without having to alter the source code, recompile the program, or retrain the neural network. Even if different speech to text conversion systems are used, only the corresponding client interface needs to be adapted and the term phrases of the form re-entered by one or more speakers with a microphone and transmitted to the new speech to text conversion system. The wrong words and phrases returned by the new system for the target language form the basis of the new allocation table. It is thus possible to functionally extend the language-to-text conversion system for any commonly used term without the need for thorough or complex changes and without the need for retraining the language software, so that spoken text with the term words and term phrases is correctly converted into words.

The allocation table may be stored, for example, as a table of a relational database or as a tab delimited text file or other functionally similar data structure.

According to an embodiment of the invention, for each of at least several of the term words (or term phrases) a plurality of reference speech signals of respective different speakers is recorded. The plurality of reference language signals reproduce the term word (or term phrase). The assignment table assigns each of at least several of the term words (or phrases) a corresponding plurality of words (or phrases) of the target vocabulary in textual form. The multiple words (or phrases) of the target vocabulary represent erroneous conversions by the speech to text conversion system upon their voices for different speakers.

For example, a particular term such as "1, 2-methylenedioxybenzene" may be spoken by 100 different people and recorded accordingly using a microphone as a reference speech signal. These persons are preferably those familiar with the pronunciation of chemical phrases. Thus, there are 100 reference speech signals for this substance name. Each of these 100 reference speech signals is sent to the speech to text conversion system and in response 100 words or phrases of the target vocabulary are returned, none of which correctly reproduces the true term name. Typically, the 100 words returned are identical to each other, but this is not always the case. Different people have different voices, that is, speech input differs in tone, volume, pitch, and clarity. Thus, a language-to-text conversion system may return a number of mutually different, incorrect words or phrases for a term word (or term phrase), all of which are incorporated into the assignment table.

It may be advantageous to take the speech input of many different persons into account for creating the allocation table, since thereby the diversity of the human voice is better taken into account, and thus a higher error correction rate can be obtained.

According to an embodiment of the invention, the at least one laboratory device is a plurality of laboratory devices. The laboratory devices are each selected from the group consisting of:

-an analysis station designed for analyzing a chemical substance or mixture; and/or

-a synthesis station designed for synthesizing a chemical substance or mixture; and/or

A pre-or post-treatment station designed to purify the chemical substances, to combine them with other substances, to dilute, concentrate or otherwise modify them, before or after the analysis or synthesis step, so that the subsequent working or transport step can be carried out.

According to an embodiment of the present invention, the at least one laboratory device is a plurality of laboratory devices, each laboratory device serving as a respective chemical substance processing station. The laboratory device and optionally also the at least one laboratory software module may be part of a laboratory facility which additionally also comprises a transport unit. The transport unit is designed to transport the substance for synthesis or analysis to the processing stations in order to allow the processing stations to carry out at least one processing step on the substance, respectively.

According to an embodiment of the invention, the laboratory system further comprises a master control computer of the laboratory facility comprising at least one laboratory device. The master computer contains master software configured to schedule process steps and chemical transports through the laboratory equipment of the laboratory facility based on instructions in the form of structured text.

According to an embodiment of the present invention, the control software includes an NLP processor (NLP: natural language processing) or is interoperable with an NLP processor connected through a network. By "one software application is interoperable with another software application" is meant herein that the interfaces and routines of the two interoperable software applications are coordinated and adapted with each other so that the two can exchange requests, control commands and data with each other in a coordinated manner, thereby performing a certain software-based function by coordinating the exchange and returning the result to the requesting software application. The control software is configured to transform the text-corrected text into structured text interpretable by the target system using the NLP processor. The target system is the master software of at least one laboratory software module and/or of at least one laboratory device and/or of a master control computer of a laboratory facility comprising at least one laboratory device. The hosting software is configured to receive from the NLP processor structured text formed by the NLP processor based on the corrected text and input it into the target system. The structured text is caused to perform laboratory-related software functions and/or hardware functions by entering the text into the target system.

For example, the article by m.hummel, d.porcinula and e.sapper in the european paint journal (2019/2/1) "natural language processing: the semantic framework of paint science-robotic reading recipes "-the conversion of natural language text input into structured text by means of grammatical parsers, part-of-speech tags and other techniques is described.

The target system may be, for example, at least one of a laboratory device and a transport unit, the input of structured text causing them to perform a number of processing steps and chemical transport between the laboratory devices according to the information contained in the structured text.

According to another example, the target system is control software of a laboratory system, and the input of the structured text causes it to perform software functions according to information contained in the structured text. The software function may be, for example, the calculation of certain concentrations or other parameters which are used, for example, to configure or calibrate the laboratory equipment or to set the pH and viscosity accordingly during the performance of certain analyses and/or syntheses as defined herein.

According to another example, the target system is a chemical database of a laboratory system that contains the results of the analysis and synthesis that have been performed by the system. The entry of structured text causes it to perform database searches according to the information and terms contained in the structured text. The retrieved results may be returned as results directly to the user by the control software or contain advantageous recipes based on the retrieved substances as input for further data processing steps, such as simulation and prediction.

According to an embodiment, the laboratory system comprises a chemical plant for analyzing and/or synthesizing chemicals and/or a plant for producing mixtures, in particular paints and varnishes. The system also includes a master control computer with master control software. The at least one laboratory device and the at least one laboratory software module are components of a high-throughput device and can be checked and controlled by the checking software by the control software. The apparatus is designed to read a text as a specification on the synthesis or on the components of the mixture or on the analysis to be carried out.

The device may be a high throughput device (HTE device), for example a high throughput device for analyzing and producing paints and varnishes. The HTE equipment may be, for example, a system for automated testing and automated production of chemicals as described in WO 2017/072351 a 2.

The control software of the device is prompted to control one or more laboratory devices and/or one or more laboratory software modules of the device as specified in the correction text by outputting the correction text or the structured text converted from the correction text to the control software of the device. This may be particularly advantageous in a biological or chemical laboratory environment, since the speech input is processed in such a way that it can be directly transmitted to the technical system and interpreted correctly by it, without the user having to remove gloves for this purpose, for example. For example, the laboratory equipment may be equipment for determining the oil absorption value of a pigment or equipment for determining the viscosity of a paint.

The device may, for example, be designed to automatically perform one or more of the following work steps in fully automatic response to corrected text entered via the machine-to-machine interface or structured text obtained therefrom:

-rheological analysis of substances and mixtures;

measuring the storage stability of the substances and mixtures, in particular in terms of inhomogeneity and precipitation tendency of the liquid mixtures, for example the analysis can be carried out after sampling according to optical measurements in cuvettes;

-determining the pH of the substance and the mixture;

-foam testing of substances and mixtures, in particular measurement of the defoaming effect and measurement of the foam breaking kinetics;

viscosity measurements of substances and mixtures, which may include an automatic dilution step, especially in the case of high-viscosity substances or mixtures, since the viscosity in the diluted solution can be determined more easily; the viscosity of the starting material or mixture is calculated based on the viscosity of the diluted solution;

measuring the kneading behaviour of the substance or mixture and in particular of the finished product (abrasion test);

measuring the colour values (so-called L-a-B values), haze and gloss of substances and mixtures according to a spectrophotometer operating for example with light scattering;

layer thickness measurements of substances and mixtures applied to a plane according to various specified parameters (temperature, humidity, surface properties of the plane, etc.);

image analysis processing of images of substances and mixtures, in particular for the characterization of the surface of substances, such as the number, size and distribution of bubbles or scratches in paints and varnishes.

The substances and mixtures can be, in particular, substances and mixtures for producing paints and varnishes. Furthermore, the substances and mixtures may be end products such as paints and varnishes in liquid or dry form, as well as intermediate products and solvents used.

According to an embodiment of the invention, the non-intrusive operation comprises: the speech input is converted to text. Especially into natural language text.

The at least one laboratory software module can be designed, for example, as a chemical substance database, which is designed to interpret input text as search input and to determine information about the search input within the database and to return it as a result to the control software. The database may be, inter alia, a database management system (DBMS) such as MySQL or PostgreSQL.

Additionally or alternatively thereto, the laboratory system may comprise a laboratory software module as simulation software. The simulation software is designed to simulate the properties of chemical products, particularly paints and varnishes, according to a predefined specification of the mixture. The simulation software is designed to interpret the input text as a specification of the product whose characteristics are to be simulated.

In addition or alternatively thereto, the laboratory system may comprise a laboratory software module which is designed as control software for the laboratory system comprising at least one laboratory device. The control software is installed on the main control computer. For example, it may be the hosting software already described for the embodiments. The master control software is designed to cause the laboratory devices and transport units of the laboratory system to perform and orchestrate the transport of chemicals between the large number of process steps and the laboratory devices according to the information contained in the input text. The laboratory facilities can be designed as HTE devices for the controlled chemical analysis and/or synthesis and/or generation of mixtures, in particular paints and varnishes. The control software is designed to interpret the corrected text or the structured text extracted therefrom as a specification about the composition or analysis or synthesis of the mixture.

In addition or alternatively thereto, the laboratory facility may contain laboratory software modules designed to perform simple data processing steps on the data, interacting between the component parts of the laboratory system in the case of analysis or synthesis. In this case, it can be, for example, the conversion, checking and/or supplementation of control commands from control software to laboratory equipment, the storage, checking or standardization of measurement data or similar data processing steps.

The corresponding text input into the laboratory device or the laboratory software module may be corrected text or structured text generated from the corrected text by the NLP processor.

According to an embodiment, the control software is configured to use at least part of the corrected text, e.g. text containing a certain keyword such as "internet search", as input to an internet search engine. The control software receives the search results, such as by parsing a browser window, and transmits output results to a target system, such as a portable device with a speaker, in the vicinity of the user.

According to an embodiment of the invention, the control software contains a register with a number of user accounts registered with the control software. The control software includes a voice recognition function for recognizing a corresponding registered user based on the voice. The control software is designed to allow the user to operate without intervention, by voice input to the microphone, only the laboratory device and/or the laboratory software module to which the user has the operating authority.

This may be advantageous because access control and regulation of laboratory equipment and laboratory software modules is seamless, as both rely on user voice. There is no need to remember or reliably keep passwords or special access tokens.

According to an embodiment of the present invention, the control software allows a user to select a corresponding laboratory device from a large number of laboratory devices and to selectively control its functions by voice input by inputting key terms, in particular laboratory device names and/or room names, as part of the voice input. For example, the control software is configured to analyze the corrected text according to specified key terms. If a key term is recognized, the control software forwards the corrected text in the form of natural language text or structured text to the respective laboratory device or its control software recognized accordingly. This may be advantageous because a large number of different laboratory devices may be controlled by means of a common speech interface.

According to an embodiment of the present invention, the control software is configured to automatically convert the corrected text created based on the user's voice input into a structure that can be correctly interpreted by the target systems set as the target systems for text input, respectively. For example, the control software dynamically recognizes a target system that the user selects from a large number of available target systems by referring to the name of a certain laboratory device or facility in the speech input to perform a particular function. The control system maintains for each target system a format that is the input format required by the corresponding target system and automatically converts the corrected text or structured text obtained therefrom to the format required by the dynamically identified target system. For example, a first laboratory device may require as input certain parameters in the form of an XML file. Another device has an NLP processor and can use the proofreading text directly as input. Another laboratory facility requires certain parameter values or recipes in tabular form. This may be advantageous because the user can input speech input in the natural language sentence in the normal syntax. Thus, the user does not have to laboriously remember the syntax required for the various laboratory devices, nor have to speak "differently" to each individual instrument. Therefore, a special match of the user's language structure to the vocabulary or syntax of each laboratory device is not necessary. The user can input his input in the syntax of his customary language, which he will also use when talking to colleagues. The special requirements of the interface of the respective laboratory device are fulfilled by the control software.

In another aspect, the invention relates to a method for controlling laboratory equipment and/or laboratory software modules. The method comprises the following steps:

-providing at least one laboratory device and at least one laboratory software module, wherein the at least one laboratory device is designed for analysis and/or synthesis of chemical substances, wherein the at least one laboratory software module is designed for processing data obtained from the at least one laboratory device;

-providing a data processing device with control software providing a user with an interface for operating at least one laboratory device and/or at least one laboratory software module; and is

Providing a portable device with a microphone, wherein the device is connected to the control software in an interoperable manner via a network, wherein the device is designed to allow a user to operate the at least one laboratory device and/or the at least one laboratory software module without intervention by voice input to the microphone in the event of an interaction with the control software.

According to an embodiment, the providing of the device comprises: the portable device is placed in a common room with at least one laboratory device and/or the portable device is placed in a common room with a host computer of a laboratory facility containing at least one laboratory device.

"structured text" herein refers to text, particularly ASCII text, that contains words and phrases having a particular meaning at a specified location. For example, the structured text may be in the form of a comma separated file, a table, an XML document, and the like. Structured text can be more easily interpreted and processed by machines and computers than natural language text.

"natural language text" here means text, in particular ASCII text, which is expressed in the language spoken by humans and is usually composed of complete sentences.

An "NLP processor" is a software program or software function that is capable of understanding, processing "natural language" text in such a way that relevant information contained in the text is extracted from the text and can optionally be converted into other formats, such as a structured text format. Thus, the NLP tool can process both text composed of whole sentences and extract information from unique sentences.

A "transport unit" is understood here to mean an automated system consisting of one or more components which can transport chemical substances in liquid and/or solid form and optionally also other objects such as consumables or containers from one processing station to the next. The transport unit may be, for example, a robotic arm, a collection of robotic arms, a conveyor belt, a collection of conveyor belts, or a combination thereof. The transport unit may be an integral part of a high throughput system and may be controlled by control software. The processing unit may be a laboratory device for analyzing or synthesizing chemical substances, or a device for pre-and post-treatment (dilution, concentration, staining, mixing, etc.) of said substances.

A "speech to text conversion system" is a data processing system that is used to convert a person's speech signal into text. The data processing system may be, for example, a combination of hardware and software, and is, for example, designed as a cloud computer system or a server computer system that provides speech-to-text conversion services to one or more client devices over a network.

"non-intrusive operation" of a device refers to an operation that does not involve any manual interaction of a user with the device or the device human-machine interface. Thus, it is a "touchless" or "touchless" operation.

A "single board computer" or "single board computer" is a computer system where all the electronic components required for operation are integrated on a single circuit board. Typically, the power supply is installed separately as the only component.

A "voice signal" is an electronic signal picked up by a microphone when a person inputs voice content to the microphone.

A "virtual laboratory assistant" is software or software routine that is operatively connected to one or more laboratory devices and/or software programs within a laboratory so that information from these laboratory devices and laboratory software programs can be received and commands to perform functions sent by the laboratory assistant to the laboratory devices and laboratory software programs. The laboratory assistant therefore has an interface for data exchange with and control of one or more laboratory devices and laboratory software programs. The laboratory assistant also has a user interface and is configured to enable a user to more easily use, monitor and/or control the laboratory equipment and laboratory software programs through the interface. For example, the user interface may be designed as an acoustic interface or as a natural language text interface.

The "virtual laboratory assistant" preferably includes a number of functions to achieve interaction between the virtual assistant and the person in a manner that is as similar as possible to human-to-human interaction. For example, the virtual laboratory assistant may have software functionality for reading out text with as natural a sound as possible. The read text may for example contain the result of the execution of the function, which is output via a loudspeaker of the portable device. Additionally or alternatively thereto, the virtual laboratory assistant may be configured for searching received speech signals or text generated therefrom according to keywords and for invoking certain hardware functions or software functions according to keywords. For example, one of the keywords may be a name, such as a common human name, that the virtual laboratory assistant interprets as its own name and as a command for it to perform or invoke a particular function.

"laboratory apparatus" here means an electronic instrument used in a laboratory for carrying out chemical processing steps, such as synthesis or analysis or other processing steps, for the pretreatment or post-treatment of substances in connection with the next/previous processing step.

A "laboratory facility" here refers to a system consisting of a plurality of laboratory devices, a transport unit and one or more software modules, which system is able to jointly control the laboratory devices and the laboratory software modules in a coordinated manner for the automatic or semi-automatic execution of a chemical workflow. For example, the workflow may be a composite workflow or an analytical workflow or a combination of both workflows.

A "laboratory software module" here refers to a software application, a sub-module of a software application or a software routine, which is configured to process data provided by a laboratory device as measurement data or used as input parameters. The laboratory software module is preferably a physical component and/or a functional component of the laboratory device or of the laboratory facility.

"vocabulary" as used herein refers to a region of language, i.e., a collection of words that are available to an entity such as a speech to text conversion system.

"word" as used herein refers to a coherent string of characters that appears in a particular vocabulary and represents an independent linguistic unit. In natural language, words have inherent meanings, unlike phonemes or syllables.

The phrase "herein refers to a linguistic unit consisting of two or more words.

The term "or" term "is used herein as a word of the term vocabulary. The term does not belong to the target vocabulary and is not usually a component of the generic vocabulary.

The expression "the speech to text conversion system supports only the conversion of speech signals into the target vocabulary" means that words of another vocabulary can either not be converted into text at all or can only be converted into text with a high error rate, which is above the error rate limit for each word or phrase to be converted, which limit should be regarded as the maximum tolerance for the conversion of the speech to text function. The limit value may be more than 50%, preferably already more than 10%, in terms of the error probability of each word or phrase, for example.

Part-of-speech tagging (POS tag) herein refers to a special label ("tag") assigned to each word in a corpus of text to specify the portion of language that the word represents in its respective context of text and also other grammatical categories such as tense, number (plural/singular), upper/lower case, etc. The set of all POS tags used by the corpus is called a tag set. The sets of tags in different languages are typically different from each other. The base tag set contains tags for the most common language components (e.g., N for nouns, V for verbs, A for adjectives, etc.).

By "portable device" is here meant a portable, preferably battery-operated data processing device which can be freely placed and which can be connected to a network, such as the internet or an organizational intranet, preferably a wireless connection. For example, "smart speakers" with microphones and/or single board systems (such as raspberry-pie computers) are used as "portable devices". It is also possible that the user uses his smartphone as a portable device, wherein the smartphone has a microphone and client software interoperable with the control software.

Brief description of the drawings

Embodiments of the invention will be illustrated in detail in the following figures:

FIG. 1 shows a flow chart of a method for controlling laboratory equipment and laboratory software modules;

FIG. 2 shows a block diagram of a distributed system for voice control of laboratory equipment and laboratory software modules;

FIG. 3 shows a block diagram of an alternative distributed system for voice control of laboratory equipment and laboratory software modules;

fig. 4 shows a block diagram of another alternative distributed system for voice control of laboratory equipment and laboratory software modules.

Detailed Description

FIG. 1 shows a flow diagram of a computer-implemented method for voice control of laboratory equipment and laboratory software modules.

In a first step 102 one or more laboratory devices and one or more laboratory software modules are provided.

The laboratory equipment may be stand alone equipment or a processing unit within a complex laboratory facility. The laboratory device may be configured and equipped for performing one or more work steps in a complex analysis or synthesis workflow, respectively, in an analysis and/or synthesis environment of a substance or a mixture, or with respect to pre-and post-treatment of substances and mixtures.

Similarly, a laboratory software module may be a separately installable software application, such as simulation software, that accounts for or predicts a chemical, physical, tactile, optical, or other property based on data obtained from a laboratory device. It may also be a DBMS with a database that has stored the results of previous analyses and syntheses of laboratory equipment, the contents of which may be used, for example, to perform database searches for substances (e.g. paints and varnishes) that have been manufactured and whose properties are known. For example, the search request can be to search for all varnishes with viscosities below the limit specified in the search request. The software modules may also be modules and functions of a larger software application, such as the control software of a complex laboratory facility.

In a further step 104, a data processing device with control software is provided. The control software provides an interface for operating at least one laboratory device and/or at least one laboratory software module to one or more users. The data processing device with the control software may be located in a common room or laboratory area with the laboratory device or the laboratory facility containing the laboratory device. The data processing device may be, for example, a desktop computer or a server. The data processing device is preferably set up in another room, for example in the data center of the organization operating the laboratory or in the data center of an external cloud storage provider. But the data processing device and the control software can be accessed by a computer in the laboratory area, for example for maintenance purposes.

In step 106, a portable device with a microphone is provided and set up so that it can be connected and interoperated with control software over a network, such as the internet or the institutional intranet. The device is designed to allow a user to operate at least one laboratory device and/or at least one laboratory software module in a non-intrusive manner by voice input to a microphone, interoperable with control software. The portable device is preferably provided such that the device is disposed in the vicinity of a laboratory working person's workspace from which the laboratory working person wants to control the laboratory device or where the laboratory working person is otherwise engaged in his laboratory work. It is also possible to place the device with the microphone centrally in the laboratory area, from which speech input from the entire extent of the room or laboratory area can be received and captured substantially always well.

According to an embodiment, the providing of the device comprises: the portable device, such as a single board computer with a microphone, is placed in a room where at least one laboratory device is located and/or the portable device is placed in a room where a host computer of a laboratory facility that includes at least one laboratory device is located.

The steps can be performed in any order. They allow one or more laboratory devices and the laboratory software modules associated therewith to be operated in a non-intrusive manner by voice input from any area of the laboratory without the need to remove gloves for this purpose. Since the control software can cause the speech-to-text conversion system to convert the speech signals picked up by the microphone into natural-language text, the text is then corrected by means of special text correction steps, so that the term words in the speech input, for example from the chemical industry or from special branches such as the paint and varnish production field, are reproduced correctly in the text.

In different embodiments of the laboratory system, the steps performed by the control software, the laboratory devices and the laboratory software modules may differ slightly, and the laboratory system may comprise other components, such as an external text correction system and/or an NLP processor for converting natural language corrected text into structured text. In some embodiments, however, the laboratory facility itself contains such an NLP processor that performs this conversion, so the native language corrected text can be directly input into the control software of the laboratory facility.

Fig. 2 shows a block diagram of a distributed system 200 for voice control of laboratory equipment 252, 254 and laboratory software modules 248, 250. The system 200 comprises laboratory equipment 252, 254 and laboratory software modules 248, 250, a control computer 213 with control software and a portable device 212 with a microphone. Alternatively, the system may contain other portable devices 212 that are substantially identical in construction and function. Optionally, the system may also include a master control computer with master control software for controlling the laboratory equipment. Optionally, the system 200 may also contain correction software 225, which may be implemented as a function of the control software or as a separate locally or remotely installed software application. Optionally, the system 200 may also include an NLP processor 258. Optionally, the system 200 may also include a speech to text conversion system 226.

The laboratory equipment and laboratory software modules contained by the system 200 are controlled by a user 202, such as a laboratory worker, who is spatially close to a portable device 212 with a microphone 214, through voice input 204. For example, the user may enter the following commands in the microphone: "Synthesis of a coating named < varnish name > and determination of the rheological Properties of the coating after Synthesis! ". The speech signal 206 collected by the microphone for speech input is forwarded by the portable device 212 via a network 236, such as the internet or an intranet, to the control software 222 installed on the control computer 213. The control software forwards the speech signal 206 via the network to a bilingual speech-to-text conversion system 226, which converts the received speech signal 206 into text 208, the words and phrases of which are taken only from the bilingual target vocabulary 234, which does not contain the term words, such as the paint name or the "rheology". For example, the speech to text conversion system 226 may be designed as a cloud computer system that provides conversion in the form of a software-based service, referred to herein as a "speech recognition processor" 232, to a plurality of clients. The speech to text conversion system 226 may be, for example, a google cloud computer system that provides google "speech to text" cloud services over the internet. Thus, in this case, interface 224 is a cloud-based API from google. The speech to text conversion system returns the recognized text 208 to the control software 222.

According to the embodiment shown in fig. 2, the control software contains a correction function 225. It may access an assignment table 238 in which a plurality of term words and term phrases are correspondingly assigned one or more words or phrases of the target vocabulary.

In the allocation table 238, words are corresponded to each other in text form. Specifically, the allocation table allocates at least one word from the target vocabulary to each of the plurality of term words or term phrases. At least one word of the target vocabulary assigned to a term word (or term phrase) is a word or phrase that the speech-to-text conversion system misrecognizes (and has been misrecognized beforehand at the time of creating the table) when the term word is input into the speech-to-text conversion system in the form of an audio signal.

Alternatively, the term phrase may also be assigned a POS tag, which is optionally evaluated and considered by the correction function 225 during the text correction process. The calibration function 225 may also be designed as a separate calibration software application 225 that is installed on the control computer 213 and interoperable with the control software 222.

Control software 222 passes the received text 208 to correction function or correction software 225 and causes it to correct the text 208 using allocation table 238. During the correction process, the erroneously recognized words and phrases of the target language are replaced by the respectively associated term words and term phrases according to the table. The generated correction text 210 is transmitted from the control software 222 directly to some target system, such as the chemical plant 244, or is first forwarded ahead to the NLP processor 258.

The NLP processor shown in fig. 2 is mounted on the control computer 213. But according to an alternative embodiment, for example as shown in fig. 3, the NLP processor may also be provided as a service over a network. The NLP processor analyzes the syntactic structure of the corrected text 210 to understand the semantic content of the corrected text and extract relevant information therefrom and convert it into structured text 211. The step of converting corrected text 210 into corrected structured text 211 may be advantageous because many laboratory systems and laboratory software modules available on the market today do not yet interpret natural language text correctly. Many target systems are provided with an input interface for structured text, i.e. for example for tables or XML files having a specific defined structure. In other embodiments, the target system contains its own NLP processor, so the corrected text 210 in natural language can be entered into the target system. In this case, the control software sends the corrected text 210 directly to the target system.

The target system may be, for example, a single laboratory device or a laboratory facility containing a plurality of laboratory devices. The laboratory facility 244 shown in fig. 2 may be a high-throughput system (high-throughput environment/HTE device) that includes a plurality of laboratory devices 252, 254 and one or more laboratory software modules 250, 248.

For example, the laboratory equipment 252, 254 may be analytical instruments configured to perform various chemical analyses (e.g., viscosity, pH, color, surface structure, layer thickness). For example, the software module 250 may be a chemical database that interprets the input text 211 or 210 as a search request and performs a corresponding search within the chemical database. The laboratory software module 248 is control software that is installed on the host computer 246 of the chemical plant 244 and is configured to coordinate the work steps performed by each of the laboratory devices in the plant. For example, the control software 248 not only controls the analysis steps performed by the analysis equipment, but also controls the operation of the transport unit 256, which transports the substances and intermediate products from one processing station to the next within the chemical plant 244.

The control of chemical plants by means of speech input may be particularly advantageous, since these plants are now capable of substantially automatically producing certain chemical products, such as certain paints or certain varnishes, in accordance with certain recipes. It is thus possible to input a recipe for producing a certain chemical substance or mixture by means of speech recognition, thereby prompting the chemical plant 244 to automatically produce the corresponding substance or mixture. It is likewise possible to initiate a corresponding substance analysis or control, possibly to interrupt, modify or stop its execution, by means of a speech input.

The internal database 250 can be used not only to store chemicals and mixtures and their characteristics, but also to store corresponding recipes. Thus, the database 250 may contain, for example, formulations of paints and varnishes and raw materials thereof, as well as their corresponding physical, chemical, optical and other characteristics. In addition, other relevant data may be stored in the database, such as product data sheets from material manufacturers, safety data sheets, configuration parameters for modules of the HTE device used to analyze or synthesize certain materials or products, and the like. The HTE devices are designed to analyze and synthesize from formulations and specifications entered in textual form.

In some embodiments, one or more of the chemical plant devices have their own interface by which they can receive natural language or structured correction text 210, 211 directly from the control software. In some other embodiments, the laboratory facility of the chemical plant may only receive the correction text 210, 211 indirectly from the control software 222 through the master software 248.

According to an embodiment, the speech input contains key terms from which the control software 222 determines the number and identity of the corresponding desired target systems. The corrected text is then sent by the control software to each identified target system in natural language or structured form. For example, by entering the keyword "target system" along with the name of each target system (laboratory device or software module), the user can determine to which target device the control software sends the correction text. In addition or as an alternative thereto, the control software may also search for key terms on its own and autonomously determine the appropriate target system in dependence on the terms. For example, speech input can only determine a certain type of synthesis, and the control software dynamically decides which instrument of a large number of laboratory devices available for synthesis will be used for the requested synthesis. For example, the current load condition can be used for judging, so that the laboratory equipment which is not loaded currently or has a low load rate can be selected.

The results 242 of the execution of a particular hardware or software function are preferably returned from the target system to the control software 222. The control software can now return the result of the function execution to the user 202. Various channels may be used for this purpose. For example, an email, SMS, or a message in a different format containing the results of the function execution may be sent to the user. According to the embodiment shown in fig. 2, the result 242 is at least also (or only) returned to the portable device 212 and output via its speaker 218. This is particularly advantageous because laboratory workers 202 have maximum freedom of movement, both in terms of entering control commands, chemical recipes and search requests, and in terms of receiving results of execution of corresponding hardware or software functions. For example, a user may be located within a particular work area of a laboratory room and facilitate analysis of a substance through voice commands in a manner that is parallel to their manual activities. The analysis results are acoustically conveyed through a loudspeaker. Thus eliminating the need for the user to interrupt their current manual activity in order to query the results. Instead, the user can speak normally with the control software and, for example, after having obtained the result that the specific synthesis has ended, also initiate a corresponding analysis step for the product substance by means of a speech input. Thus, laboratory work efficiency may be significantly improved, as thereby enabling users to work more in parallel, and to facilitate and control complex synthesis and analysis, for example, while other laboratories work.

Common activities that can be controlled by means of speech input according to embodiments of the present invention in a laboratory involve, for example, the following activities and exemplified associated speech inputs:

the previous day, laboratory staff has started to analyze the rheological properties of a certain coating and now wants to query the results stored in the HTE equipment database. Possible speech inputs are: "control computer, give me a look at the rheological analysis results of room 22 HTE equipment in 2019 on day 2, month 24".

The laboratory staff needs to save costs and consider replacing some solvent < < solvent _ expensive > with a cheaper solvent < < solvent _ cheap > >. The name < < solvent _ cheap > is the trade name of the manufacturer. He is not sure whether the cheaper solvents are suitable for the varnish to be produced and wants to look at the product data sheet where further information about the chemical and physical properties of the cheap solvents is specified. Possible speech inputs are: "control computer, give me a look at the product data sheet of < < solvent _ cheap >", or "control computer, give me a look at the product data sheet of < < solvent _ cheap > > stored in the HTE database in room No. 22".

After looking at the product data sheet for solvent < < solvent _ cheap >, laboratory staff think that this solvent is expected to be a replacement for the more expensive solvent for producing certain varnishes. It is assumed that some adjustments should be made to the formulation because various parameters such as pH, rheology, polarity, etc. are different from those of the more expensive solvent. Because these attributes interact, laboratory workers cannot determine the necessary adjustments to a recipe based on theoretical considerations. Performing a series of tests is both laborious and time consuming. But the software available in the laboratory can predict (simulate) the properties of chemical products such as paints and varnishes based on a certain recipe. The simulation may for example be based on CNN (convolutional neural network). Laboratory workers would like to use the simulation software to simulate possible characteristics of varnishes based on known formulations, in which expensive solvents have been replaced by inexpensive ones. Possible speech inputs are: "control computer, let HTE simulation software calculate varnish properties according to the following recipe: 70.2 g naphthenic oil, 4 g methyl-n-amyl ketone, 1.5 g n-amyl propionate, 1 g superabsorber, 50 g < < solvent _ cheap > >.

Simulations show that inexpensive solvents are not suitable for producing varnishes. Laboratory workers now want to search the internet for other solvents that can replace the expensive solvents without affecting product quality to reduce costs. Possible speech inputs are: "control computer, search < < high viscosity solvent for varnish production > on the Internet".

According to embodiments of the present invention, all of these inputs and commands to the respective execution system can be accomplished without the user having to leave the laboratory and/or remove gloves.

The speech inputs exemplified above mostly contain words and phrases in the form of common languages and terms. Thus, for example, the words or phrases "rheological", "naphthenic oil", "methyl-n-amyl ketone", "n-amyl propionate" are chemical terms and < < solvent _ cheap > > are trade names for chemical products. The word or phrase is typically not contained in a vocabulary ("target vocabulary") supported by a common, generic speech-to-text conversion system. Since the post-correction is performed by means of the correction software, it is ensured that the language input is still correctly converted into text in the case of a speech-to-text conversion system 226 using a common language. The control software performs a number of coordination and control activities with respect to the management and processing of speech signals and text generated thereby, which differ slightly depending on the system architecture.

Fig. 3 shows a block diagram of an alternative distributed system 300 for voice control of laboratory equipment and laboratory software modules. The system 300 may contain substantially the same components and functions as shown in fig. 2. Unlike the system shown in fig. 2, the calibration software 255 is not an integral part of the control software and is also not installed locally on the control computer 213 as a separate software application, but is provided as a cloud service by the cloud computer system 302 through the internet. The control software 222 is configured to correct the text 208 received from the speech to text conversion system via the respective interface 304 by means of the allocation table now stored on the cloud computer system 302 and to receive the corrected text 210.

Fig. 4 shows a block diagram of an alternative distributed system 400 for voice control of laboratory equipment and laboratory software modules. The data exchange of the various laboratory devices 406 and 414 for controlling the HTE devices 244 is shown in more detail in fig. 4.

The laboratory includes a laboratory area 404 in which there are various separately established laboratory equipment 416, such as centrifuges, and HTE equipment 418.

The HTE device contains a number of modules and hardware units 406 and 414 that are managed and controlled by the host software on the host computer 246. The master software of the present embodiment serves as an external central interface for monitoring and controlling the instruments contained in the HTE device. Control software 222 on control computer 213 has access to the master software and is configured to send corrected text 210, 211 as input to the master software, and optionally also to other target systems such as laboratory equipment 416 and/or internet search engine 402, in natural language or structured form. The corrected text 210, 211 is generated in accordance with the speech input 204 of the user 202, as described in accordance with embodiments of the present invention.

The control software 222 may be implemented as a "virtual laboratory assistant" 223. For example, the virtual laboratory assistant may recognize names such as "control computer," LUISA, "or" EVA. The control software is configured to search the corrected text 210 by keywords and to invoke specific hardware functions or software functions of the target system based on the keywords. For example, the control software searches for keywords such as "control computer" or "EVA" in the text 210. If the corrected text contains the keyword, virtual laboratory assistant 223 is prompted to subsequently further analyze the corrected text to determine whether the corrected text includes instructions for performing the hardware function or the software function, and if so, by which hardware or software the commands should be performed under the control of the controlling software or laboratory assistant. For example, the calibration text may contain the name of the device or laboratory area that specifies to which device and which software the command should be forwarded.

Thus, the word "control computer" or other names of virtual assistants in speech input can be used in the chemical laboratory field in a manner similar to the names "Siri" and "Alexa" of virtual assistants known for "everyday problems" for more efficient and safe control of hardware and software functions.

In one possible embodiment, the virtual laboratory assistant's evaluation of corrected text 210 indicates that Internet search engine 402 should search for something specified in corrected text 210 as a term word or term phrase. The corrected text, or some portion thereof, is input by the virtual assistant to the search engine via the internet. The internet retrieval results 424 are returned to the control software or an assistant, which forwards them to a suitable output device in the vicinity of the user 202, such as the portable device 212 with a microphone and speaker 214. The forwarding is preferably performed such that the virtual laboratory assistant, i.e. the control software, reads the results 424 to the user in natural sound through the loudspeaker 214.

In another possible embodiment, the virtual laboratory assistant's evaluation of the calibration text 210 indicates that the individual laboratory equipment 416 (i.e., centrifuge) should granulate a substance at a particular speed. The names of the centrifuge and the substance are specified in the correction text 210 as terms or phrases, which is sufficient since the centrifuge automatically reads the centrifuge parameters to be used, such as the duration and the rotational speed, from the internal database on the basis of the substance name. The correction text, or some portion thereof, is sent by the virtual assistant to the centrifuge 416 over the internet. The centrifuge initiates a centrifugation program pertaining to the substance and returns a message in the form of a text message 422 as to whether the centrifugation was successful. The results 422 are returned to the assistant 223 and output as described for results 424 through the speaker 214 and optionally also through other output interfaces as screens.

In another possible embodiment, an evaluation of the corrected text 210 by the virtual laboratory assistant indicates that the HTE device 244 should synthesize a particular varnish. The paint ingredients are also specified in the calibration text and consist of a mixture of the trade names of chemicals and the IUPAC materials name. The HTE device receives the corrected text 210 and autonomously decides to synthesize in the synthesizing unit 414. A message or error notification about the success of the synthesis is returned as a result 426 from the synthesis unit 414 to the master control software of the HTE device, and the control software in turn returns the result 426 to the control software or virtual laboratory assistant, which forwards it to a suitable output device, for example the terminal 312, which will output the result 426 via the loudspeaker 214 and optionally also via other output interfaces, as described for the result 424, as a screen.

The target software and/or hardware in outputting the corrected text 210, 211 is preferably software and hardware within the laboratory 404. It is possible, however, that the text or a part thereof is also sent to a target system outside the laboratory, the results of which are designed or adapted for chemical analysis or synthesis.

The voice control of the laboratory device and the laboratory software modules associated therewith can be used for retrieving and outputting in the respective database of the laboratory the results of the analyses and syntheses performed in the laboratory, the laboratory protocols and the product data tables, and for performing supplementary retrieval also on the internet and in public or proprietary databases accessible via the internet in a voice-controlled manner. The voice command containing the name and modifier of the specific trade name and/or chemical term of the chemical or laboratory facility or laboratory consumable is also correctly converted into text and thus correctly interpreted by the target system. According to embodiments of the present invention, a highly integrated operation of a chemical or biological laboratory or laboratory HTE device, substantially in the form of voice control, can be achieved.

The hardware (smart speakers) of known voice-to-text cloud providers pursues the goal of directly controlling and using services developed by the cloud providers themselves. Applications in the field of professional vocabulary are currently not developed or are developed only to a very limited extent. All system architectures 200, 300, 400 shown herein allow the use of existing speech-to-text APIs of various cloud providers with independent hardware not limited by the cloud provider to enable professional specific speech recognition and control laboratory facilities and electronic search services within the laboratory on this basis.

Efficient collection of chemical substance information and voice-based control of laboratory facilities and HTE equipment is particularly advantageous in terms of chemical synthesis of paints and varnishes, as the manufacture of paints and varnishes requires large amounts of raw materials, the properties of which interact in a complex manner and significantly influence the product properties. There is thus a large number of analytical, control steps and series of tests in terms of paint and varnish production. Paints and varnishes are highly complex mixtures of up to 20 or more raw materials, such as solvents, resins, hardeners, pigments, fillers and a large number of additives (dispersants, wetting agents, adhesion promoters, defoamers, biocides, flame retardants, etc.). The effective collection of information about individual components and information for controlling the corresponding analytical and synthetic equipment can significantly speed up the production process and improve product quality assurance.

List of reference numerals

102 step 106

200 distributed system

202 users

204 speech input

206 speech signal

208 recognize text

210 corrected text (Natural language)

211 corrected text (structured text)

212 portable device

213 control computer

214 microphone

218 loudspeaker

222 control software

223 virtual laboratory assistant

224 speech to text interface (client side)

224' speech-to-text interface (server side)

225 correction software

226 speech to text conversion system/cloud system

232 speech recognition processor

234 target vocabulary

236 network

238 distribution table

242 correcting the execution result of the text

246 Master control computer

248 main control software

250 chemical Database (DBMS)

252 analysis instrument

254 analysis instrument

256 transport unit

300 distributed system

302 server with text correction service

304 correction service interface (client side)

304' correction service interface (Server end)

308 NLP interface

310 control interface for laboratory facility

400 distributed system

402 internet search engine

404 laboratory area/laboratory room

406 analytical instrument

408 analytical instrument

410 mixer

412 synthesis unit

414 Synthesis Unit

414 control computer

416 independent laboratory equipment

422 correction of the execution result of the text (text form)

424 execution result of correction text (text form)

426 correction of the execution result of the text (text form)