阅读说明：本技术 混合扬声器和转换器 (Hybrid speaker and transducer ) 是由 松川武夫 俞在权 于 2020-02-10 设计创作，主要内容包括：提供了一种音频转换器系统。所述系统包括：音频输入,所述音频输入被配置成接收源音频；音频输出,所述音频输出被配置成耦合至包括至少两个非定向扬声器和定向扬声器的混合扬声器；以及处理器,所述处理器被配置成通过以下操作基于所述源音频生成所述混合扬声器的输出音频：识别所述源音频中的特定声音；将所述特定声音与所述源音频隔离；基于所述特定声音生成所述混合扬声器中的所述定向扬声器的定向扬声器输出；并且为所述混合扬声器中的所述至少两个非定向扬声器生成非定向扬声器输出的至少两个声道。(An audio transducer system is provided. The system comprises: an audio input configured to receive source audio; an audio output configured to be coupled to a hybrid speaker comprising at least two non-directional speakers and a directional speaker; and a processor configured to generate output audio for the hybrid speaker based on the source audio by: identifying a particular sound in the source audio; isolating the particular sound from the source audio; generating a directional speaker output for the directional speaker of the hybrid speakers based on the particular sound; and generating at least two channels of non-directional speaker output for the at least two non-directional speakers of the hybrid speakers.)

1. An audio transducer system, comprising:

an audio input configured to receive source audio;

an audio output configured to be coupled to a hybrid speaker comprising at least two non-directional speakers and a directional speaker; and

a processor configured to generate output audio for the hybrid speaker based on the source audio by performing steps comprising:

identifying a particular sound in the source audio;

isolating the particular sound from the source audio;

generating a directional speaker output for the directional speaker of the hybrid speakers based on the particular sound; and

generating at least two channels of non-directional speaker output for the at least two non-directional speakers of the hybrid speakers.

2. The system of claim 1, wherein the particular sound is identified by comparing the source audio to sounds in a sound database.

3. The system of claim 2, wherein the particular sound is identified by machine learning based on a generative countermeasure network (GAN) using sounds in the sound database as a learning set.

4. The system of claim 1, wherein the particular sound is identified based on sound frequency and/or amplitude differences between two or more channels of the source audio.

5. The system of claim 1, wherein the at least two channels of non-directional speaker output are generated based on subtracting the particular sound from the source audio.

6. The system of any of claims 1-5, wherein the processor is further configured to: determining a projection direction of the directional speaker of the hybrid speakers based on a user position, a magnitude difference between two or more channels of the source audio, and/or a time delay difference between two or more channels of the source audio.

7. The system of claim 6, wherein the projected direction of the directional speaker is output to the directional speaker to control a mechanical rotator configured to rotate the directional speaker.

8. The system of any one of claims 1 to 5, wherein the directional speaker comprises a plurality of panels and the directional speaker output comprises an output of each of the plurality of panels.

9. The system of claim 8, wherein the direction of projection of the directional speaker is controlled by modulating the output of the plurality of panels.

10. The system of any one of claims 1 to 5, wherein the hybrid speaker comprises a bar sound with the directional speaker located between a left non-directional speaker and a right non-directional speaker.

11. A method of audio conversion, comprising:

identifying, with a processor, a particular sound in source audio;

isolating the particular sound from the source audio;

generating directional speaker outputs for directional speakers of a hybrid speaker based on the particular sound; and

generating at least two channels of non-directional speaker output for at least two non-directional speakers of the hybrid speakers.

12. The method of claim 11, wherein the particular sound is identified by comparing the source audio to sounds in a sound database.

13. The method of claim 12, wherein the particular sound is identified by machine learning based on a generative countermeasure network (GAN) using sounds in the sound database as a learning set.

14. The method of claim 11, wherein the particular sound is identified based on sound frequency and/or amplitude differences between two or more channels of the source audio.

15. The method of claim 11, wherein the at least two channels of non-directional speaker output are generated based on subtracting the particular sound from the source audio.

16. The method of any one of claims 11 to 15, further comprising:

determining a projection direction of the directional speaker of the hybrid speakers based on a user position, a magnitude difference between two or more channels of the source audio, and/or a time delay difference between two or more channels of the source audio.

17. The method of claim 16, wherein the projected direction of the directional speaker is output to the directional speaker to control a mechanical rotator configured to rotate the directional speaker.

18. The method of any of claims 11 to 15, wherein the directional speaker comprises a plurality of panels and the directional speaker output comprises an output of each of the plurality of panels.

19. The method of claim 18, wherein the direction of projection of the directional speaker is controlled by modulating the output of the plurality of panels.

20. The method of any one of claims 11 to 15, wherein the hybrid speaker comprises a bar sound with the directional speaker located between a left non-directional speaker and a right non-directional speaker.

21. An audio speaker apparatus, comprising:

a speaker housing;

a left non-directional speaker and a right non-directional speaker enclosed in the speaker enclosure; and

a directional speaker located between the left non-directional speaker and the right non-directional speaker and enclosed in the speaker enclosure.

22. The apparatus of claim 21, further comprising:

an audio transducer configured to receive source audio comprising a non-directional audio channel and to generate an output of the directional speaker.

23. The apparatus of claim 21, wherein the directional speaker comprises one or more parametric panels.

24. The apparatus of claim 21, further comprising:

a mechanical rotator for controlling a projection direction of the directional speaker.

25. The apparatus of any one of claims 21 to 24, wherein the directional loudspeaker comprises a plurality of panels, and wherein a direction of projection of the directional loudspeaker is controlled by modulating an output of the plurality of panels.

26. The apparatus of any one of claims 21 to 24, further comprising:

a sensor to detect a position of a user, wherein a direction of projection of the directional speaker is controlled based on the position of the user.

1. Field of the invention

The present invention relates generally to audio processing and, more particularly, to generating audio signals for directional speakers.

2. Description of the related Art

A loudspeaker (or loudspeaker) is an electroacoustic transducer that converts an electrical audio signal into corresponding sound. For example, in a conventional dynamic cone speaker, a coil is suspended in a circular gap between the poles of a permanent magnet. When an alternating current audio signal is applied, the coil is forced to move back and forth rapidly due to faraday's law of induction, which causes a diaphragm attached to the coil to move back and forth, thereby forcing air to disperse sound waves around the diaphragm.

Background

Disclosure of Invention

One embodiment provides a system comprising: an audio input configured to receive source audio; an audio output configured to be coupled to a hybrid speaker comprising at least two non-directional speakers and a directional speaker; and a processor configured to generate output audio for the hybrid speaker based on the source audio by: the method includes identifying a particular sound of the source audio, isolating the particular sound from the source audio, generating a directional speaker output for a directional speaker of the hybrid speakers based on the particular sound, and generating at least two channels of the non-directional speaker output for at least two non-directional speakers of the hybrid speakers.

Another embodiment provides a method comprising: identifying, using a processor, a particular sound in source audio; isolating the particular sound from the source audio; generating directional speaker outputs for directional speakers of the hybrid speakers based on the particular sound; and generating at least two channels of non-directional speaker output for at least two of the hybrid speakers.

Another embodiment provides an audio speaker apparatus, comprising: a speaker housing; a left non-directional speaker and a right non-directional speaker enclosed within the speaker enclosure; and a directional speaker located between the left and right non-directional speakers and enclosed within the speaker enclosure.

A better understanding of the features and advantages of various embodiments of the present invention will be obtained by reference to the following detailed description that sets forth illustrative embodiments, in which the principles of various embodiments of the invention are utilized, and the accompanying drawings.

Drawings

The above and other aspects, features and advantages of embodiments of the present invention will become more apparent from the following more particular description, presented in conjunction with the following drawings, in which:

fig. 1 is a block diagram illustrating a system for driving a hybrid speaker according to some embodiments of the present invention;

FIG. 2 is a block diagram illustrating a system for transducing audio of a hybrid speaker according to some embodiments of the present invention;

FIG. 3 is a flow diagram illustrating a method for converting source video for a hybrid speaker according to some embodiments of the present invention; and

fig. 4A and 4B are illustrations of directional loudspeakers with multiple panels according to some embodiments of the invention.

Detailed Description

Directional loudspeakers generally refer to devices that produce less of a sound field than that propagated by conventional loudspeakers. In some embodiments, the directional speaker may include a parametric array speaker that generates sound from ultrasonic waves. The directional speaker device achieves high directivity by modulating audible sound onto high-frequency ultrasonic waves. Higher frequency sound waves have shorter wavelengths and therefore do not propagate rapidly, resulting in higher directivity than conventional non-directional loudspeakers.

In some embodiments described herein, a directional speaker is combined with one or more conventional non-directional speakers to simulate immersive sound and/or to selectively project selection sounds to particular users in a room. In some embodiments, the directional and non-directional speakers may be enclosed in the same enclosure to provide hybrid sound bars. In some embodiments, a converter is provided to convert conventional audio files and/or streams into signals for driving the hybrid speakers. The transducer may be configured to identify and isolate specific sounds to be reproduced by the directional speaker and to generate outputs of the directional speaker and the non-directional speaker in the speaker assembly.

Referring first to fig. 1, a system for driving a hybrid speaker is shown. The system includes an audio source 110, an audio transducer 120, and a hybrid speaker assembly 100 that produces the sound of a user 130.

Audio source 110 generally refers to a device that outputs sound to play at one or more speakers. In some embodiments, the audio source 110 may include a playback device, such as a television, cable box, media player, hard drive, media storage device, home entertainment system, computer, game console, mobile device, and the like. The audio source 110 may output mono sound, stereo sound, and/or surround sound. In some embodiments, audio source 110 may output one or more non-directional audio channels. In some embodiments, the audio source 110 may output any number of audio channels, such as two channels (e.g., left/right stereo), three channels (e.g., center, left, and right), six channels (e.g., 5.1 surround), eight channels (e.g., 7.1 surround), and so forth.

The audio converter 120 comprises a device configured to convert audio signals from the audio source 110 into signals configured to drive directional and non-directional speakers of the hybrid speaker assembly 100. In some embodiments, audio transducer 120 is configured to take one or more conventional audio channels and generate at least one output channel of directional speaker 103. In some embodiments, the audio converter 120 includes a processor configured to: identifying a particular sound in the source audio; isolating the particular sound from the source audio; generating directional speaker outputs for directional speakers of the hybrid speakers based on the particular sound; and generating at least two channels of non-directional speaker output for at least two of the hybrid speakers. In some embodiments, the audio transducer 120 may also include a sound database for identifying particular sounds in the source audio. In some embodiments, the sound database may include a cloud database shared among multiple audio transducers 120 and accessible via a network adapter on the audio transducers 120 configured to communicate over a network, such as a home network and/or the internet. In some embodiments, the audio converter 120 may be configured to run a machine learning algorithm to identify a particular sound based on an ever-growing and evolving sound learning set stored in a sound database. Additional details of the audio transducer 120 are provided herein with reference to fig. 2 and 3.

Hybrid speaker assembly 100 generally refers to a speaker device that includes at least one directional speaker and one or more non-directional speakers. In the embodiment shown in fig. 1, the hybrid speakers include left and right non-directional speakers enclosed in a speaker enclosure and a directional speaker located between the left and right non-directional speakers. In some embodiments, the hybrid speaker assembly 100 may include a form factor of a bar sound. In some embodiments, the hybrid speaker assembly 100 may include two or more individually movable housings.

The left speaker 101 and the right speaker 102 include non-directional speakers. Non-directional loudspeakers are generally referred to as conventional loudspeakers with a large propagation range and low directivity. For example, sound produced by non-directional speakers may generally be similarly heard by a listener positioned in any direction relative to the speakers. In some embodiments, left speaker 101 and right speaker 102 may comprise dynamic speakers comprising a diaphragm connected to a frame via a flexible suspension that constrains a voice coil. In some embodiments, left speaker 101 and right speaker 102 may include speaker elements similar to those in conventional bar audio. In some embodiments, the hybrid speaker assembly 100 may include additional non-directional speakers, such as a center speaker, subwoofer, and the like.

Directional speaker 103 is generally a speaker device having a narrow propagation range and high directivity. Typically, sound projected by a directional speaker can only be heard in a particular direction relative to the speaker. In some embodiments, directional speaker 103 may be configured to carry audible sound waves on inaudible high frequency waveforms (e.g., ultrasonic waves). In some embodiments, directional speaker 103 may include a parametric array speaker that generates sound from ultrasonic waves.

In some embodiments, directional speaker 103 may include multiple fixed panels. For example, directional loudspeaker 103 may include two or more fixed parametric array panels angled with respect to each other. An example of a two-panel directional speaker configuration is shown in fig. 4A, and an example of a three-panel directional speaker configuration is shown in fig. 4B. The panels of directional speakers 103 may be individually controlled to affect the overall projection direction of directional speakers 103. For example, the output of each panel may be modulated to sweep the projection direction of directional loudspeaker 103 through multiple angles in the projection area to selectively direct sound in different desired directions. In some embodiments, the scanning may be performed at a speed recognizable by the human ear. In some embodiments, the speed sweep may be between 0.01Hz to 10Hz or faster.

The projection angle, controlled by modulating the output of different panels, may also be used to direct sound to two or more separately seated listeners, respectively, and/or to create a simulated surround sound effect by projecting different sounds to the left and right ears of the listener. In some embodiments, each panel of directional speakers 103 may be driven by a separate audio stream output from audio transducer 120. In some embodiments, the hybrid speaker assembly 100 and/or the audio converter 120 may be configured to convert the audio and projection direction data into audio output for each panel of the directional speaker 103. For example, sound modulation may dynamically process sound signals over time using phase shift and/or amplitude control to provide sound effects by scanning the signal from left to right/right to left.

In some embodiments, directional loudspeaker 103 may include a mechanical rotator configured to physically rotate directional loudspeaker 103 to control the direction of projection of the loudspeaker. For example, the armature may steer the parameter array to the user 130 to direct sound to the user. In some embodiments, the rotor may sweep the projection direction of directional speaker 103 across the projection area, and directional speaker 103 may only produce sound at a desired angle during rotation, such that a single directional speaker may be perceived to project sound in multiple discrete or continuous directions. For example, directional speaker 103 may direct different sounds to different listeners sitting separately in the room by timing the output audio to rotate. In another example, the directional speaker 103 may direct different sounds to the left and right ears of the user to simulate surround sound or immersive sound. In some embodiments, directional speaker 103 is driven by a signal that includes audio and projection direction data associated with one or more temporal portions of the audio, and the rotor may be controlled by the projection direction data associated with the audio signal.

Although only one directional speaker 103 is shown in hybrid speaker assembly 100, in some embodiments, hybrid speaker assembly 100 may include multiple directional speakers 103 driven by different inputs. For example, a hybrid speaker may include two or more parametric arrays mounted on separate rotors and/or sets of parametric array panels. Although the audio transducer 120 is shown as being separate from the hybrid speaker assembly 100, in some embodiments, the audio transducer 120 may be integrated into the housing of the hybrid speaker assembly 100 as one physical device. In some embodiments, the audio converter 120 may be local to the hybrid speaker assembly 100 and/or include a cloud-based and/or server-based converter. In some embodiments, the audio transducer 120 may be integrated with the audio source 110, e.g., a home entertainment console, game console, computer system, etc. The audio source 110 may selectively engage the audio transducer 120 based on an audio format (e.g., a conventional audio or mixed speaker ready audio format) and/or a speaker arrangement (e.g., a conventional speaker or mixed speaker) coupled to the audio source 110.

With the system shown in fig. 1, a conventional audio source 110 may be converted by an audio transducer 120 into a signal for driving a hybrid speaker assembly 100 including a directional speaker 103. For a single user 130, the directional speaker 103 may direct different sounds to the user's left and right, which are mixed with the output from the conventional speakers to simulate surround sound and/or immersive sound effects. In some embodiments, directional speakers 103 may be configured to project sounds in the projection area that are only audible to a particular listener, while conventional speakers generate sounds that are audible to all listeners. For example, in a multiplayer game, different specific sounds (e.g., voice instructions, status alerts, warnings) may be projected to different players while ambient sounds or background music is played through non-directional speakers.

In some embodiments, the hybrid speaker assembly 100 may be used without the audio transducer 120. In some embodiments, the audio source 110 may output audio configured as a hybrid speaker assembly 100 including directional and non-directional speaker signals, and the audio transducer 120 may be omitted. For example, a hybrid speaker-compatible gaming machine may output select sound files (e.g., voice instructions, status alerts, warnings) via the directional speaker channels and the remaining sound files (e.g., background music) in its audio via one or more non-directional speaker channels. In another example, pre-recorded audio (e.g., movies, music, etc.) may be recorded with at least one directional audio channel that may be used directly by the mixing speakers without conversion.

Referring next to fig. 2, a system for switching audio sources for use with mixing speakers is shown. The audio transducer 200 is generally configured to acquire signals from an audio source 210 and output the signals to a non-directional speaker 221 and a directional speaker 222 of a mixing speaker assembly 220. In some embodiments, the audio source 210, the audio transducer 200, and the mixed speaker assembly 220 may include the audio source 110, the audio transducer 120, and the mixed speaker assembly 100 described herein with reference to fig. 1.

The audio converter 200 includes an audio input 201, a recognition module 202, a sound database 203, a sound filter 204, a non-directional speaker output 205, a projection direction module 206, and a directional speaker output 208.

The audio input 201 is typically configured to receive an analog or digital audio signal from an audio source 210. The audio input 201 may include wired and/or wireless connectors, such as audio connectors, audio/video connectors, HDMI connectors, USB connectors, bluetooth transceivers, wireless HDMI transceivers, and the like. In some embodiments, the audio input 201 may also include other components for filtering, converting, and/or enhancing the input signal for the rest of the processing circuitry.

The identification module 202 is generally configured to identify a particular sound in the input audio to play through the directional speaker. The identification module 202 may be implemented on a processor, control circuitry, an Application Specific Integrated Circuit (ASIC), and/or one or more digital or analog audio filter components. In some embodiments, the particular sound is identified by comparing the source audio to sound segments and/or sound profiles in sound database 203. For example, sound database 203 may include sound clips and/or sound profiles (e.g., spectral characteristics, frequency distributions, temporal frequency or amplitude variations, etc.) associated with multiple types of sounds suitable for playing via directional speakers. Examples of such sounds may include car sounds, bird sounds, ballistic throwing sounds, human speech, alarms, warning sounds, and the like. In some embodiments, the identification module 202 may include an audio event detection algorithm that utilizes a Convolutional Neural Network (CNN), a Recurrent Neural Network (RNN), and/or a Generate Antagonistic Network (GAN) that identifies particular sounds through machine learning using the sounds in the sound database 203 as a learning set. In some embodiments, the audience response may be used as feedback to a machine learning algorithm. In some embodiments, other types of feedback for machine learning may include camera input, face detection/tracking, and/or object detection/tracking. In some embodiments, user and environmental feedback may be used for directional and volume control. In some embodiments, a particular sound may be identified based on frequency, amplitude, and/or duration. For example, the sound recognition module 202 may be configured to detect sounds having high frequencies and short durations as specific sounds through directional speakers for playback. In some embodiments, the particular sound may be identified based on at least one of a sound frequency and an amplitude difference between two or more channels of the source audio. For example, if a sound within a sound stream is large (e.g., has an Interaural Intensity Difference (IID) or time delay (e.g., has an Interaural Time Difference (ITD)) between left or right speaker channels, the recognition module may identify the sound associated with the prominent IID or ITD to play on the directional speaker.

In some embodiments, sound database 203 may include a local and/or network-based database that stores audio clips and/or sound profiles. In some embodiments, the sound database 203 may be maintained by a central service shared between multiple audio transducers 200 associated with different users and different spaces. In some embodiments, the sound database 203 may be updated and pruned using audio and feedback received at multiple audio transducers by, for example, a machine learning algorithm that generates a countermeasure network (GAN). In some embodiments, the audio converter 200 may include a network interface (e.g., a network adapter, wireless transceiver, etc.) for communicating with the network-based sound database 203. In some embodiments, at least a portion of the functions of the recognition module 202 and the sound filter 204 may be performed by a remote server via a network interface.

The sound filter 204 is configured to isolate particular sounds identified by the identification module 202 from other sounds in the audio source 210. In some embodiments, sound filter 204 may isolate sound using temporal filtering, amplitude filtering, and/or frequency filtering such as high, low, or band pass filtering. In some embodiments, the sound filter 204 may use the difference between different channels of the audio source 210 to isolate a particular sound. For example, if a height ITD or IID is detected between channels of an input audio stream, one channel may be used to isolate a particular sound in the other channel by subtraction or masking. In some embodiments, filtering may only approximately isolate a particular sound; incomplete isolation may not significantly affect the user experience because the sound of the directional speakers may mix with the sound of the non-directional speakers during playback. In some embodiments, sounds from the sound database 203 that match a particular sound may be used to filter the sound and/or recreate the particular sound. The sound filter 204 then provides the particular sound isolated from the input audio to the projection direction module 206 and the directional speaker output 208. Sound filter 204 may also subtract the particular sound from one or more channels of the source audio and output a modified non-directional audio signal to non-directional speaker 221 of the mixing speaker. In some embodiments, the source audio may be provided to the non-directional speaker 221 without modification.

The non-directional speaker output 205 is generally configured to couple to the hybrid speaker assembly 220 and provide audio signals to be played at the non-directional speaker 221. In some embodiments, non-directional speaker output 205 may include wired and/or wireless connectors, such as audio connectors, audio/video connectors, HDMI connectors, USB connectors, bluetooth transceivers, wireless HDMI transceivers, and the like. In some embodiments, the non-directional speaker output 205 may also include other components for providing signals to the speaker, such as amplifiers, digital-to-analog converters, and the like. In some embodiments, the non-directional speaker output 205 may output a plurality of channels via a single line, multi-line, or wireless connection, each channel corresponding to a non-directional speaker (e.g., right speaker, left speaker) of the hybrid speaker assembly 220.

The projection direction module 206 is configured to determine a projection direction associated with the output to the directional speaker 222. In some embodiments, the casting direction module 206 determines the casting direction based on one or more of a time delay or/and a magnitude difference between the user position and two or more channels of the source audio. In some embodiments, the projection direction may be determined based on the user position detected by the user sensor 224 on the hybrid speaker assembly 220 and/or on the audio transducer 200 such that the directional speaker projects a particular sound towards one or more users. In some embodiments, the user location and/or projected area may be determined by a speaker calibration process. In some embodiments, the projection direction module 206 may be determined based on a difference in amplitude (e.g., loudness) of a particular sound in two or more channels of source audio (IID) and/or a difference in time delay of a particular sound between two or more channels of source audio (ITD). For example, if the particular sound of the left channel is significantly louder, the direction of projection of directional speaker 222 may be tilted to the left. In another example, the projection direction of the directional speaker 222 may be directed toward the center if the amplitude and timing of a particular sound in both channels is substantially the same. In some embodiments, the cast direction module 206 may determine the 2D soundscape to be output by the directional speaker 222 and determine the sound to be output at each angle as the directional speaker 222 sweeps across the cast area of the 2D soundscape. In some embodiments, the directional speaker 222 may be configured to sweep through the soundscape space in a fixed pattern, and the projection direction module 206 is configured to time the audio signal output to the scanning location of the directional speaker 222 to output the 2D soundscape. In some embodiments, an optical sensor may be used to detect user position and user detection/tracking information may be used to determine the projection direction. In some embodiments, the user position may be determined by a calibration process and/or manually adjusted by the user.

Directional speaker output 208 is generally configured to couple to a hybrid speaker and provide signals for playing at directional speaker 222 of hybrid speaker assembly 220. In some embodiments, directional speaker output 208 may include wired and/or wireless connectors, such as audio connectors, audio/video connectors, HDMI connectors, USB connectors, bluetooth transceivers, wireless HDMI transceivers, and the like. In some embodiments, the directional speaker input 201 may also include other components for providing signals to a speaker, such as an amplifier, digital-to-analog converter, and the like. In some embodiments, the output signal may include an audio signal synchronized with the scanning of the directional speaker 222. In some embodiments, the output signal may include one or more projection angles, each projection angle associated with a temporal portion of the audio stream. For example, the audio output may specify that the first 2 nanoseconds of audio are to be projected at 35 degrees and the next 3 nanoseconds of audio are to be projected at 50 degrees. In some embodiments, the output signal may include an audio stream and directional speaker movement instructions timed to the audio stream. For example, the directional speaker movement instruction may specify a scan speed, a scan range, and the like. In some embodiments, for directional speaker 222 with a mechanical rotator, the audio transducer may output rotator control signals separately from the audio stream. In some embodiments, for directional speaker 222 having multiple fixed panels, the output signal may include a separate audio channel to be output at each panel of directional speaker 222 that is modulated to control the direction of projection of directional speaker 222. In some embodiments, the output signal may include a 2D soundscape for playing by the directional speaker 222, and the hybrid speaker assembly 220 may include a processor configured to control the direction of projection of the directional speaker 222 and the audio output to reproduce the 2D soundscape.

In some embodiments, the outputs from non-directional speaker output 205 and directional speaker output 208 are timed such that when the sound of each speaker device reaches the listener, the outputs from non-directional speaker 221 and directional speaker 222 are synchronized and mixed. In some embodiments, machine learning, such as GAN, may be applied to selectively mix and synchronize outputs to non-directional speaker 221 and directional speaker 222 using sound database 203.

In some embodiments, one or more of the functions of the projection direction module 206 described herein may be performed instead by the hybrid speaker assembly 220. For example, in embodiments where the directional speaker 222 is configured to project a particular sound based only on the user location, the directional speaker output 208 may output only the particular sound and the hybrid speaker component 220 may determine the projection direction based on the user location detected by the user sensor 224. In this case, the user sensor 224 may not provide user location information to the audio transducer 200 and the projection direction module 206 may be omitted from the audio transducer 200.

User sensors 224 are typically configured to detect the location of one or more listeners in the vicinity of the directional speaker. In some embodiments, the user sensors 224 may include optical sensors, microphones, acoustic wave sensors, ultrasonic sensors, thermal sensors, and the like. In some embodiments, the user location may be determined by detecting sound reflections from sound produced by the directional speaker. For example, directional speaker 222 may scan the projected area with directional sound and user sensor 224, including a microphone, may measure the timing of sound reflections at each angle to determine the presence and/or location of the user.

In some embodiments, the connection between the audio transducer 200 and the hybrid speaker assembly 220, while shown with multiple arrows, may include a single wire or wireless connection. In some embodiments, the audio transducer 200 and the hybrid speaker assembly 220 may be an integrated device enclosed in a single housing. In some embodiments, the audio transducer 200 may be integrated with an audio source 210 that selectively engages the audio transducer based on audio format and/or speaker type.

Referring next to fig. 3, an example of a method for transducing audio of a hybrid speaker is shown. In some embodiments, the steps shown in fig. 3 may be performed by one or more of an audio transducer, a hybrid speaker controller, a computer system, and a server. In some embodiments, one or more of the steps shown in fig. 3 may be performed by the audio transducer 120 and/or the hybrid speaker assembly 100 described with reference to fig. 1, and/or the audio transducer 200 and/or the hybrid speaker assembly 220 described herein with reference to fig. 2. In general, the steps of FIG. 3 may be performed by one or more processor-based devices that include control circuitry and a memory that stores a set of computer-executable instructions.

In step 301, the system identifies a particular sound in the audio provided from audio source 310. A particular sound generally refers to a sound selected for playing through a directional speaker. In some embodiments, the particular sound is identified by comparing the source audio to the sounds and/or sound profiles in sound database 312. In some embodiments, the particular sounds are identified by the machine learning algorithm 311 using the sound database 312 as a learning set. In some embodiments, the machine learning algorithm 311 may include generating a countermeasure network (GAN). In some embodiments, the particular sound is identified based on one or more of the frequency, pitch, and amplitude of the sound. In some embodiments, sound database 312 may include sound segments and/or sound profiles (e.g., spectral characteristics, frequency distributions, temporal frequency or amplitude variations, etc.) associated with multiple types of sounds suitable for playing via directional speakers. Examples of such sounds may include car sounds, bird sounds, ballistic throwing sounds, human speech, alarms, warning sounds, and the like. In some embodiments, the sound may be initially identified based on the frequency, duration, magnitude, and/or IID and/or ITD between two or more channels of the source audio and added to the sound database 312. The machine learning algorithm 311 may use the sounds to identify other similar sounds. In some embodiments, the sound database 312 may be prioritized and/or pruned through a machine learning algorithm. In some embodiments, step 301 may be performed by identification module 202 described herein with reference to fig. 2.

In step 302, the system isolates a particular sound. Generally, in step 302, the particular sound identified in step 301 is isolated from other sounds in the audio received from the audio source 210. In some embodiments, step 302 may be performed based on temporal filtering, amplitude filtering, and/or frequency filtering, such as high, low, or band pass filtering. In some embodiments, the system may use different channels of the audio source 310 to isolate a particular sound. For example, if there is a large IID or ITD between channels of an input audio stream, one channel may be used to isolate a particular sound in the other channel by subtraction or masking. In some embodiments, matching sounds in the sound database 312 may be used to isolate a particular sound by subtraction or masking. In some embodiments, only approximate isolation of specific sounds is performed in step 302. In some embodiments, the sounds in the sound database may be used to recreate a particular sound. For example, the play speed, pitch, and/or amplitude of the sounds in the sound database may be modified to approximate the particular sound identified in the audio source 310. In some embodiments, step 302 may be performed by sound filter 204 described herein with reference to fig. 2.

In step 303, the system generates an output of directional speaker 321. In some embodiments, the output includes the particular sound isolated in step 302. In some embodiments, the directional speaker may be configured to perform a periodic scan of the projected area, and the directional speaker audio output may be timed to the periodic scan. In some embodiments, the directional speaker output may also include projection direction data. In some embodiments, the projection direction data may be determined by the projection direction module 206 described with reference to fig. 2. In some embodiments, the projection direction may be associated with a user location. In some embodiments, the projection direction data may include two or more discrete projection zones associated with two or more users, such that different sounds may be projected to different users in the same room using one directional speaker. In some embodiments, the projection direction data may associate a projection angle with a temporal portion of the directional speaker audio. In some embodiments, the directional speaker output may include a 2D soundscape to be produced by the directional speaker. In some embodiments, the directional speaker output may include a separate audio stream for each panel of the fixed multi-panel directional speaker. In some embodiments, the directional speaker output may include a rotation instruction for a rotor controlled directional speaker.

In step 304, the system generates an output of the non-directional speaker 322. In some embodiments, the directional speaker output may include the audio source 310 minus a particular sound. In some implementations, the directional speaker output may correspond to unchanged source audio. In some embodiments, the non-directional speaker output may include two or more channels, each channel corresponding to a non-directional speaker device (e.g., left speaker, right speaker) in the hybrid speaker assembly.

In some embodiments, the steps shown in fig. 3 may be repeated continuously for an audio stream in near real-time, such that the converted audio and the video playback associated with the source audio are synchronized. In some embodiments, the system may pre-buffer the source audio to allow video/audio playback synchronization. In some embodiments, the steps shown in fig. 3 may be performed on a pre-recorded audio file. In some embodiments, instead of outputting to speakers, the generated directional speaker output and non-directional speaker output may be stored as audio files for later playback.

Fig. 4A and 4B are illustrations of examples of multi-panel directional speakers according to some embodiments. Fig. 4A and 4B include top views of a panel of directional speakers arranged to project sound at multiple angles across projection area 420 by modulation. In some embodiments, panels 410a-e may each include a parametric array configured to project a narrow range of sound propagation through the ultrasound waves.

In fig. 4A, a two-panel configuration is shown. Panels 410a and 410b are positioned at an angle relative to each other and between non-directional left speaker 430 and right speaker 431. The angle between the two panels may be between 170 degrees and 90 degrees. The two panels 410a and 410b are configured to change the projection direction of sound within the projection area 420 by modulating the audio output of the two panels.

In fig. 4B, a three panel configuration is shown. Panels 410c, 410d, and 410e are positioned at an angle toward each other and between non-directional left speaker 430 and right speaker 431. In some implementations, the center panel 410d is generally parallel to a line drawn between the non-directional left speaker 430 and the right speaker 431 of the speaker assembly. The angle between each of the side panels 410c and 410e and the center panel 410d may be between 170 degrees and 70 degrees. The three panels 410c to 410e are configured to change the projection direction of the sound within the projection area 420 by modulating the audio output of the three panels.

In some embodiments, the hybrid speaker may include directional speakers in other configurations. For example, the directional loudspeaker may include four or more parametric panels. In another example, the directional speaker may include one or more bending parameter panels. In yet another example, the directional speaker may include a 2D matrix of panels (e.g., 2x2, 3x3, etc.) to project sound in two axes (e.g., left-right and up-down). In some embodiments, the hybrid speaker may include a directional speaker driven by a mechanical rotator configured to rotate and/or tilt one or more parameter arrays. The distances, proportions, and relative positions of elements in fig. 4A and 4B are provided for reference only and are not necessarily drawn to scale.

In some embodiments, one or more of the above-described embodiments, methods, approaches, and/or techniques may be implemented in one or more computer programs or software applications executable by a processor-based device or system. Such processor-based apparatuses or systems may include, for example, computers, entertainment systems, game consoles, workstations, graphics workstations, servers, clients, portable devices, tablet-type devices, and the like. Such computer programs may be used to perform various steps and/or features of the above-described methods and/or techniques. That is, the computer program may be adapted to cause or configure a processor-based device or system to perform and implement the functions described above. For example, such computer programs may be used to implement any embodiment of the above-described methods, steps, techniques, or features. As another example, such computer programs may be used to implement any type of tool or similar utility that uses any one or more of the above-described embodiments, methods, approaches, and/or techniques. In some embodiments, program code macros, modules, loops, subroutines, calls, etc., within or without a computer program, may be used to perform various steps and/or features of the methods and/or techniques described above. In some embodiments, the computer program may be stored or embodied on one or more computer-readable storage or recording media, such as any of the one or more computer-readable storage or recording media described herein.

Accordingly, in some embodiments, the present invention provides a computer program product comprising: a medium for embodying a computer program for input to a computer; and a computer program embodied in a medium to cause a computer to perform or implement steps comprising: any one or more steps involved in any one or more of the embodiments, methods, approaches, and/or methods described herein. For example, in some embodiments, the invention provides one or more non-transitory computer readable storage media storing one or more computer programs adapted or configured to cause a processor-based device or system to perform steps comprising: rendering a computer simulated scene for display to a user; detecting an onset of a saccade causing saccade masking in eye movements of a user viewing the computer simulated scene; and reducing computational resources used to render frames of the computer simulated scene during at least a portion of the duration of the glance.

While the invention disclosed herein has been described by means of specific embodiments and applications thereof, numerous modifications and variations could be made thereto by those skilled in the art without departing from the scope of the invention set forth in the claims.