System for assisting team work by means of augmented reality
阅读说明：本技术 用于借助于增强现实辅助团队工作的系统 (System for assisting team work by means of augmented reality ) 是由 罗伯特·霍夫迈斯特 于 2018-04-20 设计创作，主要内容包括：描述了一种用于辅助团队工作的计算机辅助系统,其借助于增强现实(AR)。根据一实施例,该系统包括：生产服务器,其被构造成存储以下各项：机器的至少一部分的三维几何形状以及多个任务,其中,给每个任务配属一在完成对应任务的空间中的位置以及涉及完成该任务的位置相关信息。该系统还包括通过计算机网络连接生产服务器的多个用户终端,其中,给每个用户终端配属团队的人员。此外,所述生产服务器被构造用于,给团队的每个人员分配一特定任务,其中,被配属给对应任务的位置相关信息被显示在所涉及的人员的用户终端上。(A computer-assisted system for assisting team work by means of Augmented Reality (AR) is described. According to one embodiment, the system comprises: a production server configured to store: the three-dimensional geometry of at least one part of the machine and a plurality of tasks, wherein each task is associated with a position in space for completing the respective task and position-related information relating to the completion of the task. The system further comprises a plurality of user terminals connected to the production server via a computer network, wherein each user terminal is assigned to a team person. Furthermore, the production server is designed to assign a specific task to each person of the team, wherein the position-related information associated with the respective task is displayed on the user terminals of the persons concerned.)
1. A computer-assisted system for assisting team work, having the following means:
a production server (11) configured for storing the following: a three-dimensional geometry of at least a part of the machine and a plurality of tasks (T12, T22..) wherein each task is assigned a position (P11, P12..) in space that completes the corresponding task (T12, T22.) and position-related information relating to completion of the task;
a plurality of user terminals (41) connected to the production server (11) via a computer network, wherein each user terminal (41) is assigned to a person (O3, O4, O5.),
wherein the production server (11) is further configured to assign a task (T11, T12,..) to each person (O3, O4, O5,.) of the team, wherein the position-related information (410) assigned to the respective task (T11, T12,..) is displayed on the user terminal (41) of the person (O3, O4, O5,.).
2. The system of claim 1, wherein the first and second sensors are disposed in a common housing,
wherein the user terminal (41) is configured to display position-related information (410) assigned to the respective task (T11, T12. -) at a position (P11, P12. -) assigned to the task in the space.
3. The system according to claim 1 or 2,
wherein the production server (11) is also designed to assign each person to a single task or a group of tasks, wherein the skill level of the respective person is taken into account during the assignment.
4. The system of any one of claims 1 to 3,
wherein the production server (11) is further configured for establishing a task list (L1) for the processes to be handled by the team, and assigning each person (O3, O4, O5..) a single task or a set of tasks from the list,
wherein the assignment of the task is dynamically made according to the number of the persons (O3, O4, O5.)) and is updated when the number of the persons (O3, O4, O5.)) changes.
5. The system of any one of claims 1 to 4,
wherein the production server (11) is further configured for marking each person (O5..) of the team assigned a task (T41) as busy and marking the assigned task as active.
6. The system of claim 5, wherein the first and second sensors are arranged in a single unit,
wherein the production server (11) is further configured for marking a task (T41) as completed if a person (O5..) assigned to the task (41) confirms completion of the task (T41) by means of its user terminal (41).
7. The system of claim 6, wherein the first and second sensors are arranged in a single package,
wherein the production server (11) is further configured to delete a task as an active flag without marking it as completed again if the user terminal (41) of the person (O5) assigned the task (T41) logs off from the production server (11) without the person (O5) confirming the completion of the task (T41).
8. The system according to claim 6 or 7,
wherein the production server (11) is further configured for assigning a new task from the plurality of tasks to a person (O2, O4..) not currently assigned a task, as long as the plurality of tasks (T11, T12..) contains a task that is marked as neither active nor complete.
9. The system of any one of claims 1 to 8,
wherein the production server (11) is further configured for adding additional people (O5) to the team with an additional user terminal (41) by registering the additional user terminal (41) by the production server.
10. The system of any one of claims 1 to 9,
wherein the production server (11) is also designed to identify the personnel (O3, O4, O5.) of the team on the basis of the identifier of the user terminal (41) associated therewith.
11. The system of any one of claims 1 to 10,
wherein each of the tasks (T11, T12..) is assigned to a cluster, and each task of the cluster is marked as incomplete, active, or complete.
12. The system as set forth in claim 11, wherein,
wherein the production server (11) is also designed to initially consider the tasks in a cluster comprising the most unfinished tasks and the least active tasks when assigning tasks to persons.
13. The system of any one of claims 1 to 12,
wherein the user terminals (41) are Augmented Reality (AR) terminals, each having a stereoscopic heads-up display and one or more sensors that three-dimensionally detect the geometry of the environment.
14. The system of claim 13, wherein the first and second light sources are,
wherein at least a part of the AR terminal (41) and/or the production server (11) is configured to, in an authoring mode, build a 3D model of the environment from the sensor data, the 3D model also containing locations (P11, P12..) assigned to the tasks (T11, T12.).
15. The system of any one of claims 1 to 14,
wherein the production server (11) is further configured for initially establishing, for a process to be handled by the team, a list of tasks required for performing the process (L1),
wherein process parameters of previously performed processes are taken into account when establishing the list (L1).
16. The system of claim 15, wherein the first and second sensors are configured to sense the temperature of the fluid,
wherein the task list (L1) is constantly dynamically adapted taking into account the number of tasks (T11, T12..) and personnel (O3, O4, O5.). that have already been completed and the amount of possible changes in their skill level.
17. A computer-assisted method for assisting team work, the method comprising the steps of:
storing on the production server (11) the following data: a three-dimensional geometry of at least a part of the machine and a plurality of tasks (T11, T12,..) wherein each task is assigned a position (P11, P12,..) in space that completes the corresponding task (T12, T22,.) and position-related information relating to completion of the task;
registering a plurality of user terminals (41) on the production server (11), the user terminals being connected to the production server (11) via a computer network, wherein each user terminal (41) is assigned to a person (O3, O4, O5.);
assigning a task (T11, T12.) of the plurality of tasks to each person (O3, O4, O5..) of the team, wherein the position-related information (410) assigned to the respective task (T11, T12..) is displayed on the user terminal (41) of the person (O3, O4, O5.) concerned.
18. The method of claim 17, wherein the first and second light sources are selected from the group consisting of,
wherein the corresponding user terminal (41) displays position-related information (410) assigned to the corresponding task (T11, T12. -) at a position (P11, P12. -) assigned to the task in the space.
19. The method according to claim 17 or 18,
wherein each user terminal (41) displays a task assigned to a person (O3, O4, O5.) belonging to that user terminal (41), and when the user confirms the task, position-related information (410) assigned to the corresponding task (T11, T12.)) is displayed at a position (P11, P12.)) assigned to the corresponding task in the space.
20. The method of any of claims 17-19, wherein distributing tasks from the plurality of tasks (T11, T12.) comprises:
assigning a set of tasks (T11, T12,. said.) from the plurality of tasks to one or more persons (O3, O4, O5,. said.) of the team, wherein a user terminal (41) of the person (O3, O4, O5,. said.) is enabled to select a task from the set of tasks (T11, T12,. said.); and
-displaying position-related information (410) attached to a task (T11, T12..) selected by a person (O3, O4, O5.)) involved on a user terminal (41) of said person (O3, O4, O5.)).
21. The method of any one of claims 17 to 20,
wherein the tasks (T11, T12.), the positions in the space assigned to the tasks and the associated position-related information (410) are stored in a database of the production server (11).
Technical Field
The present invention relates to systems and methods for dynamically assisting a team of people with an Augmented Reality (AR) system in operating (e.g., fitting or retrofitting) a machine.
Background
Augmented reality (also referred to as mixed reality or holographic computing) may be understood as an overlay of a real environment and virtual elements. Such a superimposition can be realized, for example, by means of a stereoscopic head-up display (HUD). Such displays can be worn like glasses, wherein two glasses lenses serve as a transparent heads-up display, which enables stereoscopic (3D) display elements in the field of view of the display, and are therefore also referred to as AR glasses. Commercial systems with a heads-up display and an integrated processor (CPU) and Graphics Processor (GPU) and sensors for determining the pose of the AR glasses in space are for example from Microsoft
The use of AR systems in production and service-for example in the automotive industry-is known per se (see for example EP 1295099B 1). The AR system may assist inspectors, for example, in quality control. Here, the AR system can fade in information in its field of view to the user (e.g., examiner in the case of quality control) and thereby expand his perception. Image data, video data or text data can be faded in as information. The user has the possibility of simply performing a nominal-actual balance by superimposing the virtual environment and the real environment.
Disclosure of Invention
The aim of the invention is to better utilize the potential of modern AR systems in production, for example in situations where a plurality of persons together (as a team) are to carry out a process on a machine, and to this end work orders are dynamically generated and steps are maintained and adapted dynamically.
The mentioned task is solved by a system according to claim 1 and a method according to claim 17. Various embodiments and further developments are the subject matter of the dependent claims.
A computer-assisted system for assisting team work with Augmented Reality (AR) is described. According to one embodiment, the system includes a production server configured to store: a three-dimensional geometry of at least a part of the machine and a plurality of tasks, wherein each task is assigned a position in space for completing the corresponding task and position-related information relating to the completion of said task. The system further comprises a plurality of user terminals connected to the production server via a computer network, wherein each user terminal is assigned to a team person. The production server is also designed to assign a task to each person of the team, wherein the position-related information associated with the respective task is displayed on the user terminals of the persons concerned.
Furthermore, a computer-assisted method for assisting team work is described. According to one embodiment, the method includes storing, on a production server: a three-dimensional geometry of at least a part of the machine and a plurality of tasks, wherein each task is assigned a position in space for completing the corresponding task and position-related information relating to the completion of said task. The method further includes registering a plurality of user terminals on the production server, the user terminals being connected to the production server via a computer network. Each user terminal is assigned to a person of the team. The method further comprises assigning a task of the plurality of tasks to each person of the team, wherein the position-related information assigned to the respective task is displayed on the user terminal of the person concerned.
In one embodiment, the respective user terminal can display (e.g., by means of augmented reality technology) the location-related information associated with the respective task at the location in space associated with the task. A person may also be assigned a set of tasks (from a plurality of tasks), wherein the user terminal of the person is able to effect a selection of a task from the set of tasks. The user terminal of the person concerned can then display the position-related information, which is associated with the task selected by the person.
The assignment of a task or a group of tasks to the persons (to which the user terminal is assigned) can be carried out dynamically and adapted to the processes respectively carried out by the team. Here, the assignment of tasks may be dynamically updated as the number of participating user terminals changes, and the assignment of tasks may depend on a person skill level representing the authority or capability of the corresponding person.
Drawings
The invention is explained in detail below with the aid of examples shown in the drawings. The drawings are not necessarily to scale and the invention is not limited to the illustrated aspects. Rather, emphasis is placed upon illustrating the principles of the invention. In the drawings:
fig. 1 schematically shows an example of a system for assisting a team of (any number of) operators (operators) by means of networked AR terminals.
FIG. 2 is an exemplary view of a machine with a view of information relating to the location and context of the operating or display elements of the machine.
Fig. 3 shows, by means of a schematic sketch, the division of a production line into a plurality of equipment modules and a plurality of positions, on which tasks (jobs) can be performed.
Fig. 4 shows, according to a schematic sketch, the sum of all possible tasks (tasks T11, T21) which are required for the dynamic generation of a process, wherein individual tasks are assigned to associated positions on a production line.
Fig. 5 shows an assignment of tasks to different task clusters according to a schematic sketch.
Fig. 6 is a diagram for illustrating automated organization of operations of a team of operators (operators) while processing a job.
Fig. 7 shows a procedure for assigning a task to an operator of a newly joined team according to three schematical diagrams (a-c).
Fig. 8 illustrates the procedure when a specific task is completed by the corresponding operator, according to a schematic diagram.
Fig. 9 illustrates the procedure when the operator leaves the team according to a schematic diagram.
Detailed Description
Fig. 1 schematically shows a first example of a system for assisting a team of operators (operators) when operating a production machine by means of a networked terminal set (terminal), such as an augmented reality terminal device (also referred to as AR terminal). The process performed by the team may be, for example, the assembly or retrofitting of one or more machines of the production line. In many industrial fields, for example, packaging machines must often be retrofitted in order to produce packaged products having different package sizes or having different package variants. Merely as an illustrative example, mention may be made of automatically packaging pharmaceutical products, in which case many different countries and products require many different packages. Regular retrofitting of the packaging machine is required and the efficiency of such a retrofitting process can be an important factor.
In the example shown, the system can be operated in two modes of operation. The first mode of operation is denoted as authoring mode and the second mode of operation is denoted as customer mode. The system comprises one or more production servers 11, 12 etc. (e.g. one server for each production line) which may be connected to a central server 10 in a computer network. In smaller production facilities, the functions of the central server and the production server 11 may be centralized in one (production) server. In the present example, the production server 11 is assigned to a production line 31. Via a computer network, a plurality of user terminals, such as AR terminals 41, and optionally an administrator terminal 40, such as a personal computer, tablet PC or other AR terminal, may be connected to the production server 11, for example via a Wireless Local Area Network (WLAN). The production servers 11, 12 do not have to run on separate (computer) hardware, but the server services involved may also be implemented on a user terminal (e.g. an AR terminal or other mobile device) or an administrator terminal 40.
The AR terminal may be a stereoscopic heads-up display (HUD, or HMD, i.e. head mounted display) with or without sensing means for spatially detecting the environment. A commercially available system is for example Google
The local production server 11 for the production line 31 stores production-specific data (fig. 1, reference numeral 15) relating to the production line 31, and can dynamically provide the AR terminal 41 and the administrator terminal 40-as needed. This does not occur once at the beginning of the process, but can be adapted to the current situation at all times as the process remains stepwise during the process. For example, the adaptation may or must be performed when one or more operators (operators) leave (check out) or re-enter (check in) the system during the procedure. A current list of tasks (jobs) must then be established and provided to the corresponding operator dynamically for each operator (using computer algorithms), for example taking into account the "skill level" of the operator. This also occurs-for each job-after the job is completed by the operator. This ensures that all jobs are processed, that no jobs are assigned multiple times, and that each operator provides or assigns permitted jobs only in terms of his operator status. "providing" means here that, depending on the skill level of the operator (for example, skill level "professional"), for example, in the case of consideration and localization of other operators, not only the specific task but also the selection of the cluster and task (task group) is provided to the operator. Thus, the operator (operator) can select (within certain limits) which specific task he will perform next.
These skill levels and the assignment of the corresponding operators can be stored, for example, in a database of the production server 11 or of a central server. For example, the operator may authenticate himself to the user terminal by means of a code or password, and the system may thus identify the user from the networked user terminal on which the user "logs in".
The central server 10 has access to a number (or all) of the production lines to production specific data and can update these data periodically. It may also be configured to perform (e.g., compare) data analysis (fig. 1, reference numeral 101). The central server 10 may be connected to the local production server 11 via a local computer network. The central server 10 may also be connected to the local production servers 11 via the public internet (for example by means of a virtual private network VPN) if it is likely to be located at a different site. The mentioned data analysis may for example comprise real-time analysis (point-of-care analysis), benchmarking, etc.
The production line 31 comprises at least one machine 310 with a plurality of operating elements 312 and/or display devices 311 (see also fig. 2). In the present example, the operation element 312 has a display device 311. As the AR terminal, an AR terminal having a sensing device for spatially detecting an environment may be used. This enables marker-less AR (marker-less augmented reality) and accurate tracking of the location of the corresponding AR terminal. In the authoring mode, the geometry of the environment (and therefore also of the machine 310) is detected three-dimensionally by means of at least one AR terminal 41 and the sensors contained therein (and the software listed on the terminal) and a corresponding 3D model is built and stored in the local production server 11. Using different sensors in the process and e.g. for simultaneous localization and (3D) mapping (simultaneous localization and mapping, SLAM)) The algorithm of (1). Suitable sensors are, for example, cameras, TOF cameras (time of flight cameras), Inertial Measurement Units (IMU), etc. Methods and algorithms for data detection and building of 3D models are known per se and are assisted by commercially available devices, such as microsoft' s
Fig. 2 schematically illustrates a part of a 3D model of a machine 310, wherein two operating elements 312 with a display device 311 are exemplarily shown. The operating element may be generally referred to as a Human-machine interface (HMI). In the authoring mode, other locations (e.g., in the form of coordinates) may be defined in space, where location-related information 410 relating to tasks (jobs) to be completed at the corresponding locations may be dynamically allocated and displayed later in the client mode. Thus, the location-related information 410 may represent the task settings in a human-perceptible form (e.g., text, numerical values, graphics, etc.). For example, in addition to each human-machine interface (operating element 312/display device 311), a location can be "marked" on which information about the corresponding human-machine interface can be displayed in the client mode. This location related information may be superimposed by the AR terminal 41 to the real environment. In this example, no specific marking of the location (e.g., with a QR code) is required. In so-called marker-less AR systems, the desired location is again identified based on current sensor information provided by the AR terminal and a previously established (or otherwise provided) 3D model of the environment. However, markers may also be used in simpler systems. In another example, both label-free and label-based AR terminals and AR-incapable terminals may be combined in a system. It should be noted that the operating element 312 with the display device 311 is only one example for locations on the machine where tasks may be completed. Other examples include the installation or attachment of mechanical parts, replacement of worn parts, and the like.
In the example shown in fig. 2, in the client mode, an operator (operator) carrying the AR terminal shows the position-related information 410 at a point in space previously defined in the authoring mode. In this example, the position-related information 410 is shown at a point that is directly spatially adjacent to the operational element 312 (noted as "C1.1"). Once the point is in the space within the field of view of the AR terminal 41, the information 410 is displayed by the AR terminal of the corresponding operator, which in turn is related to the pose (location and orientation) of the AR terminal 41 in space. The information 410 may relate to a task (work) to be performed by the operator at a corresponding location (e.g. at the location of the operating element C1.1, see fig. 2). That is, each task is assigned a specific location in space.
The information 410 shown at a specific location may be, for example, a numerical value which the operator should adjust on the operating element 312 and/or which should be checked on the associated display device 311. When the operator has completed the task, i.e. set the desired value on the operating element 312 (or checked the set value), the operator can confirm completion of the task and end the current working step, e.g. with the gesture in the field of view of the AR terminal 41 (fig. 2, step C1.1). The AR terminal 41 then provides the operator with multiple locations (positions in space) for selection or a specific location (if only one is selected) and then directs the operator to the next task, i.e. where the next work step should be completed (e.g. in case of the next operating element/next HMI). At the end of the task, its successful completion can also be verified and recorded, for example by taking a picture of the operating element 312 or the display element 311 by means of the AR terminal. Other data, such as the name, hour or duration of a person required for a specific task, can also be detected for analysis purposes (analyewecke) (see fig. 1, analysis 101). These times may be stored in the production server in relation to the person or anonymously. The detection of the person-related data can be, for example, stipulated in the pharmaceutical field with regard to safety certification or even (GMP, good manufacturing practice). In this step, the system may detect tasks that have been completed and, when the task list is dynamically built, no longer provide or assign these tasks to other operators.
Fig. 1 and 2 show the structure of the entire system and the view of an operator (operator) through an AR terminal 41. In the following fig. 3-9, the functional manner of the system is illustrated, wherein the respective functions may be partly implemented in the production server 11 and partly in the AR terminal 41 or the administrator terminal 40. As already explained above, in the authoring mode, one or more equipment modules of the production line 31 (see fig. 1) are detected geometrically and define a plurality of positions or areas (blobs) on which a large number of tasks are performed by a team of operators (operators). In the schematic according to fig. 3, these positions are denoted by P11, P12, P13, P21, etc., which form a large number of all possible positions. In this example, particular work areas in the production line (e.g., different portions of the machine 310) are defined as equipment modules, and each defined location may be assigned to a module. The (equipment) module corresponds to a machine or a part of a machine in the production line concerned. In the view according to fig. 3, the production line 31 (or a part thereof) is divided into three modules M1, M2 and M3. Module M1 includes positions P11, P12 and P13, module M2 includes positions P21 and P22, and module M3 includes positions P31, P32 and P33. If needed in the process under consideration, each of the positions P11, P12, P13, P21, P22, P31, P32, and P33 may be dynamically assigned a particular task when establishing a work document for the corresponding process (e.g., retrofitting a machine), which must be completed by a team operator. The location and module assignment is not mandatory and may depend on the particular circumstances of the production line and the precise requirements of the products being processed (e.g. packaged) in a particular process on the production line. Each of the positions P11, P12, P13, P21, P22, P31, P32 and P33 may be defined, for example, by three spatial coordinates. Each position can also be assigned an orientation (for the purpose of 3D representing position-related information assigned to a position with respect to the task to be completed at the corresponding position).
The assignment of tasks (jobs) to the positions P11, P12, P13, P21, P22, P31, P32 and P33 is shown in the diagram a of fig. 4. The individual tasks can be combined into clusters C1, C2, etc. (groups), wherein the tasks assigned to a particular cluster can be, for example, correlated with one another (and therefore, for example, have to be processed one after the other). In addition, tasks that have to be performed in positions that are very close to one another in space can be assigned to a cluster. The cluster may, but need not, include those jobs assigned to the module locations. In the example of fig. 4 (diagram a), jobs T11, T12, and T13 are assigned to positions P11, P12, and P13 in module M1; jobs T14 and T21 are assigned to positions P21 and P22 in module M2. It can be seen that the workload of the module is not equal to the workload of the cluster. In this example, jobs T11, T12, T13, and T14 are assigned to cluster CL1, and job T21 (among other jobs) is assigned to cluster CL 2. As already mentioned, a module represents a specific area or section (three-dimensional geometry) of a machine in a production line, as opposed to clusters that group jobs in terms of their logical order or relevance. Graph B in fig. 4 shows a detail in fig. 2. The position P21 represents the location of the operating element and the information 410 shown relates to the work assigned to the position P21. As mentioned, there is no one-to-one assignment between device modules and clusters. For example, as shown in diagram C of fig. 4, two device modules M1 and M2 may also be partially arranged one above the other. It may already be meaningful due to the spatial proximity to have the same person (or the same person) perform tasks at positions P21 and P13, P22 and P14, and P23 in sequence. In this case, the right part of the module Ml, which is above the module M2, is assigned to the same cluster C2 as the module M2. Only the left part of module Ml is assigned to cluster C1. The assignments between the jobs T11, T12, T13 and T14 and the positions P11, P12, P13, P21 and the associated position-related information may be stored, for example, in a database of the production server 11 of the production line concerned.
FIG. 5 shows an example of a system description by means of clusters and the distribution of jobs over a plurality of clusters. In the present example, four jobs (T11, T12, T13, T14) are assigned to the first cluster CL1, two jobs (T21, T22) are assigned to the second cluster, and three jobs (T31, T32, T33) are assigned to the third cluster. The overall system (with respect to production line 31) may be basically defined by three dynamically managed lists (see fig. 6), namely a list L1 of unfinished jobs that have not yet been assigned to operators of the team, a list L2 of "active" jobs that are respectively assigned to operators, and a list L3 of finished jobs. The list L1 is a task list that is generated at the start of the process dynamically as a function of the process results to be obtained (the process parameters required to achieve the desired results) and variable framework conditions (e.g. the number of operators and the skill level), which task list may also be adapted automatically during the process. The list L1 may also depend on process parameters of a previously performed process. Thus, for example, a specific job can be cancelled when the settings/configurations involved on the production line have been made for the previous process and do not have to be changed for the current process.
The lists L1, L2, L3 may likewise be stored in the database of the production server 11 and dynamically managed by it. In this respect, dynamic means that the list of tasks to be completed and the assignment of tasks to the team operators are not established statically at the start of the process (i.e. before the operator starts working), but can be changed/adapted during the execution of the work. The number of team operators may change during the procedure (as the team performs the procedure), the skill level of the operators in the team may change (e.g., when an operator with skill level "practice" is replaced by an operator with skill level "enter"). This dynamics, which is not possible with "manual" planning, can significantly improve the efficiency of the process execution.
Fig. 6 exemplarily illustrates a description of a production line with eleven unfinished jobs (list L1: jobs T12, T13, and T14 from cluster C1, jobs T24 and T25 from cluster C2, jobs T32, T33, and T34 from cluster C3, jobs T41, T42, and T43 from cluster C4), three active jobs (list L2: jobs/operators T11/O1, T23/O2, and jobs T31/O3), and five finished jobs (list L3: jobs T11, T21, T22, T23, and T31). Operators O1, O2, and O3 can be distinguished according to AR terminals used thereby. It is also to be mentioned again here that the list L1 of unfinished jobs is not a fixedly defined list, but it can be established dynamically at the start of the process. In this case, a number of factors influence the establishment of the list. At the start of the process, the operator (for example with skill level "supervision") can decide, for example, whether the settings known from the previous process on the production line should be taken into account when composing the list of tasks to be completed L1. Based on the list L1 thus generated, a list L2 (active job) for each operator O1, O2, or the like is dynamically established. It is considered whether jobs T1, T2, etc., can be assigned to operators O1, O2, etc., that is to say whether the execution of specific jobs is authorized and whether these jobs are assigned to them as a list L1 for selection or in a fixedly predefined, dynamically generated order.
It is to be noted here that, for example, when a production line is converted (conversion process), it is not always necessary to carry out/check all possible settings/adaptations on the production line. This may be desirable in some cases. However, it may be more efficient to perform/check the settings/adaptations on the production line during the retrofitting process only for each position of the production line where the desired settings/adaptations are distinguished compared to the final retrofitting process.
Each AR terminal has an explicit identifier, e.g. the MAC address (medium access control address) of the corresponding AR terminal in the computer network. However, any other unambiguous identifier may be used to identify the operator's AR terminal. Associated with this identifier may be personal data and/or data about the completion (determined time, documentation, etc.) of the task of the operator concerned.
In many production facilities, the machine must be operated by a plurality of persons (referred to herein as operators) who must work as a team (e.g., when retrofitting a machine for producing or packaging or processing other products). Team members must be well balanced on top of each other in order to cooperate effectively. In practice, a flow plan (e.g., in paper form) is established for each machine (or set of machines of a production line) for the personnel of the team, and each team member processes its flow plan. Team size is fixed for a particular process (e.g., machine retrofit) and cannot be easily changed after work begins. The flow plan is static. If a team member exits (or is just taking a shift, for example), this may greatly delay the entire process. Even by adding another person to the team, the entire process (the sum of jobs) cannot be significantly accelerated as this can confuse the balanced flow plan.
The system described herein enables flexible adaptation of team sizes by means of AR terminals (see fig. 1) used by team members (operators). A method is described below which enables, by means of the networked AR terminal 41, the dynamic adaptation of the size of a team of operators, for example differently structured (i.e. with different skill levels, such as "practice", "entry", "senior", "professional", "supervising", etc.) and equipped with different terminals, and also of the assignment of the tasks belonging to the team members. The example of fig. 7 shows a situation (person is new to team) in which the team is expanded by an operator (operator O5). For this purpose, the AR terminal of the operator O5 is registered in the production server 11. This registration may be done automatically (e.g. after connecting the AR terminal of operator O5 with a particular WLAN access point) or triggered manually (e.g. by entering a password) by operator O5 (e.g. with its AR terminal 41 or administrator terminal 40). After registration, operator O5, which is the free operator, is received into the list of free operators L0 (and skill level is considered if necessary) (i.e., those operators/team members who have not yet been assigned or provided with a task). This situation is shown in graph a of fig. 7.
Diagram B in fig. 7 illustrates, by way of example, how a decision may be made automatically, which job (from one of the clusters) that has not yet completed is assigned which free operator. An operator with a low skill level (e.g., "entry") may be dynamically assigned by the system a list of tasks that he must process in a predetermined order. The system does not distribute jobs that have already been completed, nor jobs that are not appropriate for the skill level of the operator. Furthermore, the system can assign to ("entry") operators preferably jobs which can be carried out on as free a part of the line as possible. For this purpose, the system can confirm, by means of suitable algorithms, the optimal starting position of the "entry" operator and start generating the job list supplied to it (keeping step-wise the conditions that change constantly during the process). An operator with a higher level of skill (e.g., "professional") may choose from a plurality of clusters and jobs provided to it by the system to preempt and "block" (which prevents the "entry" operator from being able to perform a particular job) an operator who, in the case of an operator with that higher level of skill, may dynamically generate a new (e.g., optimized according to certain criteria) task list (auto-navigation mode) for the "entry" operator. The process runs constantly and in steps with the process maintenance in the background and provides an optimized distribution of the operators on the production line. Depending on the skill level of the operator, only one job may be assigned to the operator at a time, respectively, or it is also possible to assign to the operator a job group that can be selected by the operator (by means of his user terminal). When the operator confirms a specific task (by means of his user terminal), the user terminal can virtually display the associated (location-dependent) information at the location in space assigned to the corresponding task, which information is intended to assist or guide the operator in the execution of the task.
In the example shown in graph B of FIG. 7, the clusters are destaged (first criteria) corresponding to the number of outstanding jobs. As a second criterion for the dynamic generation, the number of active jobs can be used (in the case of an upgrade classification). That is, jobs are assigned to a free operator (here operator O5) (e.g., having a skill level "entry") from the cluster that includes the most outstanding jobs. If the selection criteria does not result in an explicit result (e.g., because the number of outstanding jobs is the same in the case of two clusters), then the jobs from the cluster that includes the most outstanding jobs and the least active jobs are assigned to operator O5. Alternatively or additionally, other selection criteria may be considered. For example, all operator current positions known to the system (and periodically updated) may be taken into account as criteria on the extent to which one or more jobs are assigned or provided to the (idle) operator closest to the position assigned to the job. This approach allows shortening the operator path. It is also possible to avoid specific attachments between work and operator, for example when a specific attachment would result in too many operators staying in a specific spatial area at the same time. As mentioned, the location of a single operator may be tracked, for example, by means of the AR terminal 41, and this information is also taken into account when dynamically adapting the task lists L1-L3. It should be appreciated that for small production lines (e.g., only a single machine), there is no need to group jobs into clusters. As criteria for assigning tasks to the operator, for example, the result path of the operator or the like can be considered. Even random attachment is possible if the application allows it.
In this example, cluster C4 includes three outstanding jobs and no active jobs (because, for example, a job has just been completed, see also fig. 8), cluster C2 includes outstanding jobs and active jobs, cluster C3 includes no outstanding jobs and includes active jobs, and cluster C1 includes no outstanding jobs and includes two active jobs. Thus, jobs from cluster C4 (e.g., job T41), among others, are provided to idle operator 05. This allocation is schematically shown in graph C of fig. 7. This job is cleared in the uncompleted job list L1 and received into the active job list L2. Operator O5 is scratched out of the free operator list L0 (see fig. 6). The order of the jobs can be fixedly predefined within the cluster. If the necessary order exists (e.g., first removing a part and then loading a new part), the order is confirmed in the authoring mode. If the fixed sequence of jobs (or a subset of jobs) is not predefined by the production line, the jobs can be selected according to specific selection criteria (for example, so that spatial overlapping (i.e., collision) of the work areas of the operators at the positions assigned to the jobs is avoided).
Fig. 8 schematically illustrates a situation in which the operator has confirmed that a job (task) is completed or the last job in the job group is completed if it has been allocated to the operators of a plurality of jobs. This confirmation is usually done manually by the operator at his AR terminal (e.g. by operating virtual buttons in the field of view of the AR terminal, by operating real buttons at the AR terminal, by means of a bluetooth controller, a voice controller, etc.). In this case, the active job (e.g., job 41) is removed from active job list L2 and received into completed job list L3, and the operator concerned (e.g., operator O5) is re-received into free operator list L0. There are situations where an active operator leaves (or must leave) his workstation without terminating (or being able to terminate) the job assigned to it. When leaving the workstation, the AR terminal of the operator concerned logs off (unregisters) from the server 11. This log-off may be done automatically (e.g., when the WLAN connection between the AR terminal and the access point is broken) or manually by an operator or another person (e.g., via the administrator terminal 40 or the AR terminal). This situation is shown in graph a of fig. 9. In the example shown, operator O5 has logged off his AR terminal from server 11. The active, unfinished job (e.g. job T41) and the involved operator O5 are deleted from the active job list L2, which is again received in the unfinished job list L1 and may for example be allocated to another operator (see fig. 7, chart C). If the AR terminal of the free operator (e.g., O4) is unregistered, it is deleted from the free operator list L0.
Some aspects of the embodiments described herein are summarized below (see fig. 1-9). It should be noted, however, that it is not a complete enumeration but rather is only an exemplary enumeration. A system for assisting team work with Augmented Reality (AR) is described. According to examples described herein, a system has a production server (see FIG. 1) configured to store (see FIG. 2) three-dimensional geometry of at least a portion of a machine. Furthermore, a plurality of tasks (jobs) are stored, wherein each task can be assigned a position in space for completing the respective task and position-related information relating to the completion of the task (see fig. 4, diagrams a and B). The system also has a plurality of AR terminals (see fig. 1) which are connected to the production server via a computer network. In this case, each AR terminal can be assigned to a team person. The production server is designed to dynamically generate a "plan" for executing the process (and to adapt it if necessary), wherein each person of the team is assigned a specific task or a task group (cluster) is provided (for example, taking into account the skill level of the person) by displaying the associated location-related information (see fig. 4, diagram 4) on the AR terminal of the person concerned at the location in space assigned to the corresponding task (see fig. 7). Depending on the root implementation, the position of the individual operator can be detected and stored (e.g. together with the time and active jobs to which it belongs) in steps (i.e. during the process) by means of the AR terminal. These data can be used for later data analysis or also for real-time data analysis (real-time analysis) (see fig. 1). For example, the workflow may be (partially) automatically optimized by means of these data (e.g. assigning jobs to a cluster or the order of jobs in a cluster or avoiding a conflict of two or more operators).
The production server may also be configured to mark each person of the team who has been assigned or provided a task as busy and mark the assigned task as active. The tagging is done, for example, by receiving people and tasks into the list of active tasks L2 (see FIG. 6). Furthermore, if the (busy) person who assigned the task confirms completion of the task through his AR terminal, the production server may mark the task as complete. This marking is done, for example, by receiving the task into a list of completed tasks L3 (see FIG. 6).
If the AR terminal of the person assigned the task logs off from the production server without prior confirmation that the person completed the task, the task is again marked as active (the task is not marked as completed). For example, the flag may be cleared by re-receiving the involved task into the list of uncompleted tasks L1 (see fig. 6). The person whose AR terminal has been logged off by the production server is no longer available for assignment to other tasks.
The production server can assign new (unfinished) tasks to persons not currently assigned tasks (list of free operators L0, see fig. 7) as long as there are tasks that are neither marked as active nor as finished and that do not result in conflicts due to the assignment (the work areas of the two operators overlap due to jobs that are spatially close to each other). Additional people with additional AR terminals may be added to the team by registering the additional AR terminals by the production server (see figure 7, diagram a). In the example described here, the team personnel are identified or distinguished by the production server according to the identifier of the AR terminal assigned to them.
Each task of a process (e.g., to retrofit a production line or a portion thereof) may be assigned to a cluster, and each task may be marked as incomplete, active, or complete (see fig. 5 and 6). The production server may perform priority assignment when assigning tasks to personnel. For example, tasks in a cluster that includes the most outstanding tasks and the least active tasks may be considered first, as long as no conflict with other operators is expected here. To this end, a list of jobs for each operator may be generated or updated with the process maintenance steps taking into account defined edge conditions, such as the level of skill required for a particular job or the mutual correlation (logical or spatial correlation) of tasks.
- 上一篇：一种医用注射器针头装配设备
- 下一篇：使用区块链网络的多轮令牌分发系统和方法