Musical instrument auxiliary learning system, method and device based on AR augmented reality
阅读说明：本技术 一种基于ar增强现实的乐器辅助学习系统及方法和装置 (Musical instrument auxiliary learning system, method and device based on AR augmented reality ) 是由 刘飞 刘任凭 于 2019-11-13 设计创作，主要内容包括：本发明公开了一种基于AR增强现实的乐器辅助学习系统,包括定位标识和AR识别模块。定位标识设置在乐器的特定位置,AR识别模块能够从定位标识中获取乐器的类型和型号以及定位标识的设置位置,并能够以定位标识的设置位置为基点生成轮廓。此外AR识别模块能够获取用户弹奏乐器时的姿态和手势,并通过生成的轮廓呈现不同的颜色来显示用户弹奏乐器时的姿态和手势是否正确。定位标识设置简单,不用对乐器进行改造,成本较低。还能够和乐器本身产生互动,让学习方式更为直观、高效。由于通过视觉增强现实方式实现,提示效果和方式灵活全面、影响更加深刻、学习效果更好。(The invention discloses an instrument auxiliary learning system based on AR augmented reality, which comprises a positioning identifier and an AR identification module. The positioning mark is arranged at a specific position of the musical instrument, and the AR recognition module can acquire the type and the model of the musical instrument and the setting position of the positioning mark from the positioning mark and can generate the outline by taking the setting position of the positioning mark as a base point. In addition, the AR recognition module can acquire the gesture and the gesture of the user when playing the musical instrument, and displays whether the gesture and the gesture of the user when playing the musical instrument are correct or not by displaying different colors through the generated contour. The positioning mark is simple to set, the musical instrument is not required to be modified, and the cost is low. And the device can interact with the musical instrument, so that the learning mode is more visual and efficient. Because the visual augmented reality mode is adopted, the prompting effect and the mode are flexible and comprehensive, the influence is more profound, and the learning effect is better.)
1. An instrument assisted learning system based on AR augmented reality, comprising:
the positioning mark is arranged on the musical instrument and used for storing the type and model information of the musical instrument and the position information of the positioning mark on the musical instrument;
and the AR identification module is used for identifying the positioning identification of the musical instrument and generating a plurality of outlines corresponding to different playing positions of the musical instrument on the musical instrument.
2. The system of claim 1, wherein: the positioning identification is a two-dimensional code label.
3. The system of claim 1, wherein: the AR identification module includes:
the camera is used for acquiring musical instrument image information, positioning identification and gestures of a user;
the display screen is used for displaying a video of a user playing a musical instrument and displaying a contour on the musical instrument;
the recognition software unit is used for recognizing the positioning identifier and generating a plurality of outlines corresponding to different playing positions of the musical instrument on the musical instrument according to the positioning identifier; and for rendering the outline in different colors depending on whether the gesture and the gesture are correct at the time of playing.
4. The system of claim 1, further comprising: the pre-playing information receiving module is used for receiving the pre-playing information of the user.
5. The system of claim 1, further comprising: and the recording module is used for recording the playing video for the user to watch.
6. An AR augmented reality-based musical instrument assisted learning method is characterized by comprising the following steps:
acquiring a positioning identifier through a camera;
generating a plurality of outlines on a display screen according to the positioning marks, wherein each outline corresponds to one playing position on the musical instrument;
acquiring pre-playing information of a user and postures and gestures of the user when playing a musical instrument;
judging whether the gesture and the gesture are correct or not according to the pre-playing information;
and displaying the outlines of different colors according to the judgment result.
7. The method of claim 6, wherein: the generating a plurality of outlines on a display screen according to the positioning identification, wherein each outline corresponds to a playing position on the musical instrument and comprises the following steps:
determining the type and model of the musical instrument and the position of the positioning identifier on the musical instrument according to the positioning identifier;
and generating a contour matched with the musical instrument according to the type and the model of the musical instrument by taking the position of the positioning mark on the musical instrument as a base point, and displaying the contour on a display screen.
8. The method of claim 6, wherein: the judging whether the gesture and the gesture are correct according to the pre-playing information comprises:
acquiring a correct playing posture and gesture according to the pre-playing information;
and comparing the correct playing gesture and gesture with the gesture and gesture to obtain a judgment result.
9. The method of claim 6, wherein: the displaying the outlines of different colors according to the judgment result comprises the following steps:
when the playing gesture and the gesture are correct, the contour displays green;
when the playing gesture and the gesture are incorrect, the outline displays red.
10. An instrument aided learning device based on AR augmented reality, comprising:
the positioning identifier acquisition module is used for acquiring a positioning identifier through a camera;
the outline generating module is used for generating a plurality of outlines on a display screen according to the positioning marks, and each outline corresponds to one playing position on the musical instrument;
the pre-playing information and gesture acquisition module is used for acquiring pre-playing information of a user and gestures of the user when playing the musical instrument;
the judging module is used for judging whether the gesture and the gesture are correct or not according to the pre-playing information;
and the result display module is used for displaying the outlines of different colors according to the judgment result.
Technical Field
The invention relates to the field of AR (augmented reality), in particular to an instrument auxiliary learning system, method and device based on AR augmented reality.
Background
The existing musical instrument learning and teaching, no matter the user learns by oneself or the online one-to-one live broadcasting teaching, the content material lacking in learning or teaching and the user musical instrument generate interaction, such as a music score of explanation of a teacher, a key position, a chord handle position fingering and the like, can only be used for single demonstration, and whether the played gesture and gesture are correct or not can not be determined.
The existing market has a circuit design Bluetooth lighting mode which can partially solve the problems, musical instruments are modified by a circuit board design mode, the circuit board design mode comprises a simple MCU control unit and a Bluetooth transmission device, control information transmission is carried out by connecting Bluetooth and an equipment system, but the cost is too high, and charging and maintenance are needed; and the Bluetooth lighting mode can only show whether playing is correct through simple lighting, and the Bluetooth lighting mode cannot be connected with the musical instrument, so that information transmission is single.
Disclosure of Invention
In order to overcome the defects of the prior art, the invention provides the auxiliary learning system, the auxiliary learning method and the auxiliary learning device for the musical instrument, which are based on AR augmented reality, have low cost and simple use and can judge whether the posture and the gesture of the musical instrument are correct during playing.
The technical scheme adopted by the invention for solving the technical problems is as follows:
on the one hand, the method comprises the following steps of,
an AR augmented reality based musical instrument assisted learning system comprising:
the positioning mark is arranged on the musical instrument and used for storing the type and model information of the musical instrument and the position information of the positioning mark on the musical instrument;
and the AR identification module is used for identifying the positioning identification of the musical instrument and generating a plurality of outlines corresponding to different playing positions of the musical instrument on the musical instrument.
Further, the positioning identifier is a two-dimensional code label.
Further, the AR identification module includes:
the camera is used for acquiring musical instrument image information, positioning identification and gestures of a user;
the display screen is used for displaying a video of a user playing a musical instrument and displaying a contour on the musical instrument;
the recognition software unit is used for recognizing the positioning identifier and generating a plurality of outlines corresponding to different playing positions of the musical instrument on the musical instrument according to the positioning identifier; and for rendering the outline in different colors depending on whether the gesture and the gesture are correct at the time of playing.
Further, the pre-playing information receiving module is used for receiving the pre-playing information of the user.
Further, the recording module is used for recording the playing video for the user to watch and correct the error.
On the other hand, in the case of a liquid,
an AR augmented reality-based musical instrument assisted learning method comprises the following steps:
acquiring a positioning identifier through a camera;
generating a plurality of outlines on a display screen according to the positioning marks, wherein each outline corresponds to one playing position on the musical instrument;
acquiring pre-playing information of a user and postures and gestures of the user when playing a musical instrument;
judging whether the gesture and the gesture are correct or not according to the pre-playing information;
and displaying the outlines of different colors according to the judgment result.
Further, the generating a plurality of outlines on a display screen according to the positioning identification, wherein each outline corresponds to a playing position on the musical instrument comprises:
determining the type and model of the musical instrument and the position of the positioning identifier on the musical instrument according to the positioning identifier;
and generating a contour matched with the musical instrument according to the type and the model of the musical instrument by taking the position of the positioning mark on the musical instrument as a base point, and displaying the contour on a display screen.
Further, the judging whether the gesture and the gesture are correct according to the pre-playing information comprises:
acquiring a correct playing posture and gesture according to the pre-playing information;
and comparing the correct playing gesture and gesture with the gesture and gesture to obtain a judgment result.
Further, the displaying the outlines of different colors according to the judgment result includes:
when the playing gesture and the gesture are correct, the contour displays green;
when the playing gesture and the gesture are incorrect, the outline displays red.
On the other hand, in the case of a liquid,
an instrument learning aid based on AR augmented reality, comprising:
the positioning identifier acquisition module is used for acquiring a positioning identifier through a camera;
the outline generating module is used for generating a plurality of outlines on a display screen according to the positioning marks, and each outline corresponds to one playing position on the musical instrument;
the pre-playing information and gesture acquisition module is used for acquiring pre-playing information of a user and gestures of the user when playing the musical instrument;
the judging module is used for judging whether the gesture and the gesture are correct or not according to the pre-playing information;
and the result display module is used for displaying the outlines of different colors according to the judgment result.
This application adopts above technical scheme, possesses following beneficial effect at least:
the technical scheme of the invention discloses an instrument auxiliary learning system based on AR augmented reality, which comprises a positioning identifier and an AR identification module. The positioning identification is arranged at a specific position of the musical instrument, the AR identification module can acquire the type and the model of the musical instrument and the setting position of the positioning identification from the positioning identification, can calculate different playing positions of the musical instrument according to the type and the model of the musical instrument in the positioning identification, and generates outlines corresponding to the different playing positions one by one. The positioning mark is simple to set, the musical instrument is not required to be modified, and the cost is low. And the device can interact with the musical instrument, so that the learning mode is more visual and efficient. Because the visual augmented reality mode is adopted, the prompting effect and the mode are flexible and comprehensive, the influence is more profound, and the learning effect is better.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a schematic structural diagram of an instrument assisted learning system based on AR augmented reality according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of another instrument auxiliary learning system based on AR augmented reality according to an embodiment of the present invention;
fig. 3 is a flowchart of an instrument aided learning method based on AR augmented reality according to an embodiment of the present invention;
FIG. 4 is a flowchart of another instrument auxiliary learning method based on AR augmented reality according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of an instrument assisted learning device based on AR augmented reality according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the following detailed description of the technical solutions of the present invention is provided with reference to the accompanying drawings and examples. It is to be understood that the embodiments described are only a few embodiments of the present application and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the examples given herein without making any creative effort, shall fall within the protection scope of the present application.
Referring to fig. 1, an instrument aided learning system based on AR augmented reality includes:
a positioning mark 110 provided on the musical instrument for storing type and model information of the musical instrument and position information of the positioning mark on the musical instrument;
an AR recognition module 120, the AR recognition module is configured to recognize a positioning identifier of the musical instrument and generate a plurality of contours corresponding to different playing positions of the musical instrument on the musical instrument.
The embodiment of the invention provides an AR augmented reality-based musical instrument auxiliary learning system which comprises a positioning identifier and an AR identification module. The positioning identification is arranged at a specific position of the musical instrument, the AR identification module can acquire the type and the model of the musical instrument and the setting position of the positioning identification from the positioning identification, can calculate different playing positions of the musical instrument according to the type and the model of the musical instrument in the positioning identification, and generates outlines corresponding to the different playing positions one by one. The positioning mark is simple to set, the musical instrument is not required to be modified, and the cost is low. And the device can interact with the musical instrument, so that the learning mode is more visual and efficient. Because the visual augmented reality mode is adopted, the prompting effect and the mode are flexible and comprehensive, the influence is more profound, and the learning effect is better.
As a further improvement to the above embodiment, an embodiment of the present invention provides another instrument auxiliary learning system based on AR augmented reality, as shown in fig. 2, including:
a
as an optional implementation manner in the embodiment of the present invention, the location identifier is a two-dimensional code tag. In practical application, the two-dimensional code label is pasted to a determined position, the two-dimensional code is obtained through the camera, and the position information stored in the two-dimensional code, the type of the musical instrument and the type of the musical instrument are decoded. It should be noted that, other accessory information of some musical instruments, such as sound quality condition, maintenance record, etc., may also be stored in the two-dimensional code tag.
It can be understood that the setting position of the two-dimensional code can be set at will, and the position information stored in the two-dimensional code corresponds to the position set by the two-dimensional code label.
It should be noted that the positioning identifier is not limited to a two-dimensional code tag, but may be some pattern tags, and only a database needs to be established to correspond the pattern of the pattern tag to the information to be stored.
The AR recognition module 220 is configured to obtain the gesture and the gesture of the user, recognize the positioning identifier of the musical instrument, generate a contour, and display whether the gesture and the gesture of the user when playing the musical instrument are correct by presenting different colors through the contour.
As an optional implementation manner in this embodiment of the present invention, the AR identification module includes:
the
a
the
In some optional embodiments, the auxiliary learning system further comprises a pre-playing information receiving module 230 for receiving the pre-playing information of the user.
In some optional embodiments, the learning aid system further comprises a
In practical application, the positioning mark adopts a two-dimensional code label, the AR identification module is installed on a mobile phone or other mobile display equipment, a picture played by a user is shot through a front camera on the mobile phone, and when the two-dimensional code label is collected by the camera, stored information is read, and a contour is generated by setting a position base point of the two-dimensional code label. The method comprises the steps that pre-playing information is input by a user before playing, the pre-playing information comprises music scores, pictures or characters and the like, the posture and the gesture of the user when the user is to play can be obtained, a correct playing mode is generated by a system, and when the user actually plays, a playing scene is displayed on a mobile phone or other mobile display equipment. And displaying whether the gesture and the gesture of the user playing the musical instrument are correct or not by representing different colors through the outline.
It should be noted that, after the category and the model of the musical instrument in the two-dimensional code tag are obtained, the AR identification module calculates the outline and the operation area of the musical instrument, such as strings, frets, piano keys, and the like, with the position information stored in the two-dimensional code tag as a base point, divides different position operation areas according to common usage habits, and generates an outline for the operation area at each position.
The embodiment of the invention provides another instrument auxiliary learning system based on AR augmented reality, which stores information and provides positioning through a two-dimensional code label. The camera is used for acquiring images. The display screen displays a result, and the identification software unit is used for identifying the positioning identifier and generating a plurality of outlines corresponding to different playing positions of the musical instrument on the musical instrument according to the positioning identifier; and for rendering the outline in different colors depending on whether the gesture and the gesture are correct at the time of playing. The pre-playing information receiving module is used for receiving playing pre-playing information. And the recording module is used for recording the playing video for the user to watch and correct the errors. The display result is convenient, simple and clear, and the cost is low.
In an embodiment, the present invention further provides an instrument auxiliary learning method based on AR augmented reality, as shown in fig. 3, including the following steps:
acquiring a positioning identifier through a camera;
generating a plurality of outlines on the display screen according to the positioning marks, wherein each outline corresponds to one playing position on the musical instrument;
acquiring pre-playing information of a user and postures and gestures of the user when playing a musical instrument;
judging whether the gesture and the gesture are correct or not according to the pre-playing information;
and displaying the outlines of different colors according to the judgment result.
According to the AR augmented reality-based musical instrument auxiliary learning method provided by the embodiment of the invention, the positioning identifier is obtained through the camera, the outline is generated on the display screen according to the positioning identifier, and the color of the outline during playing indicates whether the posture and the gesture during playing are correct or not, so that the method is simple and clear, and is convenient for a user to learn.
As a supplementary explanation to the above embodiment, another instrument auxiliary learning method based on AR augmented reality is provided in the embodiment of the present invention, as shown in fig. 4, including the following steps:
acquiring a positioning identifier through a camera;
generating a plurality of outlines on the display screen according to the positioning marks, wherein each outline corresponds to one playing position on the musical instrument;
in some optional embodiments, generating a plurality of outlines on the display screen according to the positioning identifier, each outline corresponding to a playing position on the musical instrument comprises:
determining the type and model of the musical instrument and the position of the positioning mark on the musical instrument according to the positioning mark;
and generating a contour which conforms to the musical instrument according to the type and the model of the musical instrument by taking the position of the positioning mark on the musical instrument as a base point, and displaying the contour on the display screen.
Acquiring pre-playing information of a user and postures and gestures of the user when playing a musical instrument;
judging whether the gesture and the gesture are correct according to the pre-playing information, and acquiring the correct playing gesture and the gesture according to the pre-playing information in practical application;
and comparing the correct playing gesture and gesture with the gesture and gesture to obtain a judgment result.
Displaying outlines with different colors according to the judgment result, and displaying green on the outlines when the playing posture and the gesture are correct;
when the playing gesture and the gesture are incorrect, the outline displays red.
It should be understood that red and green are only one alternative implementation of the present invention, and that other colors may be used. In practical applications, the results may also be displayed using means other than color. Illustratively, when the playing gesture and the gesture are correct, the outline display becomes bright; when the playing gesture and the gesture are incorrect, the outline is darkened.
According to the AR augmented reality-based musical instrument auxiliary learning method provided by the embodiment of the invention, the positioning mark is used as the base point to generate the contour which accords with the musical instrument, the contour is not required to be marked on the musical instrument, the contour is directly generated and completely accords with the shape of the musical instrument, and whether the result is correct or not is judged through the color, so that the AR augmented reality-based musical instrument auxiliary learning method is convenient and accurate. The user can be effectively assisted in learning the musical instrument.
In an embodiment, the present invention further provides an instrument learning aid based on AR augmented reality, as shown in fig. 5, including:
a positioning identifier obtaining module 51, configured to obtain a positioning identifier through a camera;
a contour generating module 52, configured to generate a plurality of contours on the display screen according to the positioning identifier, where each contour corresponds to a playing position on the musical instrument;
a pre-playing information and gesture obtaining module 53, configured to obtain pre-playing information of a user and a gesture and gesture of the user when playing a musical instrument;
the judging module 54 is used for judging whether the gesture and the gesture are correct according to the pre-playing information;
and a result display module 55, configured to display the outlines of different colors according to the determination result.
According to the music instrument auxiliary learning device based on AR augmented reality, the positioning identification acquisition module acquires the positioning identification through the camera; the contour generation module generates a plurality of contours on the display screen according to the positioning identifiers; the pre-playing information and gesture acquisition module is used for acquiring pre-playing information of a user and gestures of the user when the user plays a musical instrument; the judging module judges whether the gesture and the gesture are correct or not according to the pre-playing information; the result display module displays the outlines of different colors according to the judgment result, posture correction and gesture reminding of the user can be achieved, and the user is assisted in musical instrument learning.
It is understood that the same or similar parts in the above embodiments may be mutually referred to, and the same or similar parts in other embodiments may be referred to for the content which is not described in detail in some embodiments.
It should be noted that, in the description of the present application, the terms "first", "second", etc. are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. Further, in the description of the present application, the meaning of "a plurality" means at least two unless otherwise specified.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and the scope of the preferred embodiments of the present application includes other implementations in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present application.
It should be understood that portions of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
It will be understood by those skilled in the art that all or part of the steps carried by the method for implementing the above embodiments may be implemented by hardware related to instructions of a program, which may be stored in a computer readable storage medium, and when the program is executed, the program includes one or a combination of the steps of the method embodiments.
In addition, functional units in the embodiments of the present application may be integrated into one processing module, or each unit may exist alone physically, or two or more units are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module, if implemented in the form of a software functional module and sold or used as a stand-alone product, may also be stored in a computer readable storage medium.
The storage medium mentioned above may be a read-only memory, a magnetic or optical disk, etc.
In the description herein, reference to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the application. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
Although embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present application, and that variations, modifications, substitutions and alterations may be made to the above embodiments by those of ordinary skill in the art within the scope of the present application.
- 上一篇：一种医用注射器针头装配设备
- 下一篇：一种大学教学用教学杆