阅读说明：本技术 音频回放的无线协调 (Wireless coordination of audio playback ) 是由 利奥·莱 哈罗德·阿德里安·査德 王海松 S·赵 L·李 盖洛德·于 于 2017-11-19 设计创作，主要内容包括：描述了对回放操作进行协调的电子装置。具体地,电子装置中的接口电路可以基于第二电子装置发送分组时的发送时间与分组的接收时间之间的差来计算作为接口电路中的时钟与第二电子装置中的第二时钟之间的时间的函数的相对漂移。然后,接口电路可以基于相对漂移来调整提供时钟的时钟电路以消除相对漂移,并且可以确定时钟和第二时钟之间的剩余时间偏移。接下来,接口电路可以基于剩余时间偏移来修改电子装置要执行回放操作时的未来时间以确定校正的未来时间,并且电子装置可以在校正的未来时间处执行回放操作。(An electronic device that coordinates playback operations is described. In particular, the interface circuit in the electronic device may calculate the relative drift as a function of time between a clock in the interface circuit and a second clock in the second electronic device based on a difference between a transmission time when the second electronic device transmits a packet and a reception time of the packet. The interface circuit may then adjust the clock circuit providing the clock based on the relative drift to eliminate the relative drift, and may determine a remaining time offset between the clock and the second clock. Next, the interface circuit may modify a future time at which the electronic device is to perform the playback operation based on the remaining time offset to determine a corrected future time, and the electronic device may perform the playback operation at the corrected future time.)

1. An electronic device, comprising:

one or more antennas;

a clock circuit configured to provide a clock; and

an interface circuit communicatively coupled to the one or more antennas and the clock circuit, wherein the electronic device is configured to:

receiving, from the one or more antennas, a packet associated with a second electronic device, wherein a given packet includes a transmit time based on a second clock in the second electronic device at the time the given packet was transmitted by the second electronic device;

storing a receive time when the packet is received, wherein the receive time is based on the clock and the receive time is determined in a physical layer of the electronic device;

calculating a relative drift as a function of time between the clock and the second clock based on a difference between the transmit time and the receive time, wherein a given difference lies between a given transmit time and a given receive time of a given packet;

adjusting the clock circuit providing the clock based on the relative drift to eliminate the relative drift;

determining a remaining time offset between the clock and the second clock;

receiving, from the one or more antennas, information associated with the second electronic device specifying a future time when the electronic device is to perform a playback operation; and

modifying the future time based on the remaining time offset to determine a corrected future time; and is

Wherein the electronic device is configured to perform the playback operation at the corrected future time.

2. The electronic device of claim 1, wherein the transmission time is included in one of a payload and a Media Access Control (MAC) header in the given packet.

3. The electronic device of claim 1, wherein the clock circuit further comprises:

an oscillator configured to provide a reference clock; and

a Frequency Locked Loop (FLL) circuit coupled to the oscillator configured to generate the clock based on the reference clock; and is

Wherein the electronics are configured to adjust the clock to eliminate the relative drift by modifying the FLL.

4. The electronic device of claim 3, wherein modifying the FLL comprises changing a seed of a synthesizer in the FLL.

5. The electronic device of claim 1, wherein the transmission time and the reception time are stored at opposite ends of a payload of the given packet; and is

Wherein the electronic device is configured to determine a duration of the payload and the electronic device is configured to add the duration to the remaining time offset.

6. The electronic device of claim 1, wherein the transmit time comprises a second counter value corresponding to the second clock and the receive time comprises a counter value corresponding to the clock.

7. The electronic device of claim 1, wherein the packet comprises audio data in a payload;

wherein the electronic device is configured to store the audio data in a queue; and is

Wherein the playback operation includes outputting the audio data from the queue.

8. The electronic device of claim 1, wherein adjusting the clock and modifying the future time coordinates the playback operation in a clock domain of the clock to within a predetermined value of a clock domain of the second clock.

9. The electronic device of claim 1, wherein the packet comprises a control packet.

10. The electronic device of claim 1, wherein the second electronic device is a master device and the electronic device is a slave device.

11. The electronic device of claim 1, wherein the receive time is associated with a wireless ranging capability of the interface circuit, and

wherein the given difference ignores a spatial distance between the electronic device and the second electronic device.

12. A computer-readable storage medium for an electronic device, the computer-readable storing program instructions that, when executed by the electronic device, coordinate playback operations by causing the electronic device to perform one or more operations comprising:

receiving, from one or more antennas in the electronic device, a packet associated with a second electronic device, wherein a given packet includes a transmit time based on a second clock in the second electronic device at the time the given packet was transmitted by the second electronic device;

storing a reception time when the packet is received, wherein the reception time is based on a clock in the electronic apparatus and the reception time is determined in a physical layer of the electronic apparatus;

calculating a relative drift as a function of time between the clock and the second clock based on a difference between the transmit time and the receive time, wherein a given difference lies between a given transmit time and a given receive time of a given packet;

adjusting a clock circuit providing the clock based on the relative drift to eliminate the relative drift;

determining a remaining time offset between the clock and the second clock;

receiving, from the one or more antennas, information associated with the second electronic device that specifies a future time when the electronic device is to perform the playback operation;

modifying the future time based on the remaining time offset to determine a corrected future time; and

performing the playback operation at the corrected future time, wherein adjusting the clock and modifying the future time coordinates the playback operation in the clock domain of the clock to within a predetermined value of the clock domain of the second clock.

13. The computer-readable storage medium of claim 12, wherein the transmission time is included in one of a payload and a Media Access Control (MAC) header in the given packet.

14. The computer-readable storage medium of claim 12, wherein the transmission time and the reception time are stored at opposite ends of a payload of the given packet; and is

Wherein the one or more operations include adding a duration of the packet to the remaining time offset.

15. The computer-readable storage medium of claim 12, wherein the packet comprises audio data in a payload;

wherein the one or more operations include storing the audio data in a queue; and is

Wherein the playback operation includes outputting the audio data from the queue.

16. A method for coordinating playback operations, comprising:

by an electronic device:

receiving, from one or more antennas in the electronic device, a packet associated with a second electronic device, wherein a given packet includes a transmit time based on a second clock in the second electronic device at the time the given packet was transmitted by the second electronic device;

storing a reception time when the packet is received, wherein the reception time is based on a clock in the electronic apparatus and the reception time is determined in a physical layer of the electronic apparatus;

calculating a relative drift as a function of time between the clock and the second clock based on a difference between the transmit time and the receive time, wherein a given difference lies between a given transmit time and a given receive time for a given packet;

adjusting a clock circuit providing the clock based on the relative drift to eliminate the relative drift;

determining a remaining time offset between the clock and the second clock;

receiving, from the one or more antennas, information associated with the second electronic device that specifies a future time when the electronic device is to perform the playback operation; and

modifying the future time based on the remaining time offset to determine a corrected future time; and is

Performing the playback operation at the corrected future time, wherein adjusting the clock and modifying the future time coordinates the playback operation in the clock domain of the clock to within a predetermined value of the clock domain of the second clock.

17. The method of claim 16, wherein the transmission time is included in one of a payload and a Media Access Control (MAC) header in the given packet.

18. The method of claim 16, wherein the adjusting comprises modifying a seed of a synthesizer in a Frequency Locked Loop (FLL) in the clock circuit.

19. The method of claim 16, wherein the transmission time and the reception time are stored at opposite ends of a payload of the given packet; and is

Wherein the method further comprises adding a duration of the packet to a remaining time offset.

20. The method of claim 16, wherein the packet comprises audio data in a payload;

wherein the method further comprises storing the audio data in a queue; and is

Wherein the playback operation includes outputting the audio data from the queue.

Technical Field

The described embodiments relate to coordination techniques. More specifically, the described embodiments include coordination techniques to wirelessly coordinate playback times of electronic devices that output sound.

RELATED ART

Music generally has a significant impact on an individual's mood and perception. This is thought to be the result of the connection or relationship between the regions in the brain that recognize, learn and remember music and the regions that produce emotional responses (e.g., frontal lobe and limbic system). In fact, emotion is considered to be related to the interpretation process of music, and emotion is also very important to the influence of music on the brain. In view of the ability of music to "feel" a listener, audio quality is often an important factor in user satisfaction when listening to audio content, and more generally, when viewing and listening to audio/video (a/V) content.

However, achieving high audio quality in an environment is often challenging. For example, the acoustic source (e.g., speaker) may not be properly placed in the environment. Alternatively or additionally, the listener may not be located at an ideal location in the environment. In particular, in stereo playback systems, the so-called "sweet spot" (at which the amplitude and arrival time differences are small enough that both the apparent image and localization of the original sound source are preserved) is typically limited to a fairly small region between the loudspeakers. When the listener is outside of this region, the apparent image collapses and may only hear one or the other independent audio channel output by the speakers. Furthermore, achieving high audio quality in the environment often places significant constraints on the synchronization of the speakers.

Thus, when one or more of these factors is suboptimal, the acoustic quality in the environment may be reduced. This, in turn, may adversely affect the listener's satisfaction and overall user experience when listening to audio content and/or a/V content.

Background

Disclosure of Invention

A first set of described embodiments includes an electronic device. The electronic device includes: one or more antennas; an interface circuit; and a clock circuit. During operation, the interface communicates with the second electronic device using wireless communication, and the clock circuit provides a clock in the electronic device. Further, the interface circuit may receive a packet (packet) from the second electronic device via wireless communication, wherein the given packet includes a transmission time based on a second clock in the second electronic device when the second electronic device transmits the given packet. In response, the interface circuit may store a receive time based on the clock when the packet was received, and may calculate a relative drift as a function of time between the clock and the second clock based on a difference between the transmit time and the receive time. The interface circuit may then adjust the clock circuit providing the clock based on the relative drift to eliminate the relative drift, and may determine a remaining time offset between the clock and the second clock. Further, the interface circuit may receive, via wireless communication, information from the second electronic device specifying a future time when the electronic device is to perform the playback operation. Next, the interface circuit may modify the future time based on the remaining time offset to determine a corrected future time, and the electronic device may perform a playback operation at the corrected future time.

It should be noted that the transmission time may be included in the payload and/or the Media Access Control (MAC) header in a given packet. In some embodiments, the packet comprises a control packet. Alternatively or additionally, the packet may comprise a data packet.

Further, the clock circuit may include: an oscillator providing a reference clock; and a Frequency Locked Loop (FLL) circuit that generates a clock based on the reference clock. The interface circuit may modify the FLL to adjust the clock. For example, modifying the FLL may include changing a seed of a synthesizer in the FLL.

Further, the transmission time and the reception time may be stored at opposite ends of the payload of a given packet. In these embodiments, the electronic device may determine the duration of the payload (e.g., using software executed by a processor or interface circuitry), and the interface circuitry may add the duration to the remaining offset time.

In addition, the transmission time may include a second counter value corresponding to the second clock, and the reception time may include a counter value corresponding to the clock.

In some embodiments, the packet includes audio data in a payload, and the electronic device stores the audio data in a queue. In these embodiments, the playback operation includes outputting audio data from the queue.

It should be noted that adjusting the clock and modifying the future time coordinates the playback operation in the clock domain of the clock to within a predetermined value of the clock domain of the second clock.

Further, the second electronic device may be a master device and the electronic device may be a slave device.

Further, the reception time is associated with a wireless ranging capability of the interface circuit.

Another embodiment provides a computer readable storage medium for an interface circuit in an electronic device. The computer-readable storage medium includes instructions for at least some of the operations performed by the electronic device.

Another embodiment provides a method for coordinating playback operations. The method includes at least some of the operations performed by the electronic device.

Another embodiment provides a second electronic device.

A second set of described embodiments includes an electronic device. The electronic device includes: one or more antennas; an interface circuit; and a clock circuit. During operation, the interface communicates with the second electronic device using wireless communication, and the clock circuit provides a clock in the electronic device. Further, the interface circuit may receive a packet from the second electronic device via wireless communication, where the given packet includes a transmission time based on a second clock in the second electronic device when the second electronic device transmits the given packet. In response, the interface circuit may store a receive time based on the clock when the packet was received, and may calculate a relative drift as a function of time between the clock and the second clock based on a difference between the transmit time and the receive time. The interface circuit may then adjust the clock circuit providing the clock based on the relative drift to eliminate the relative drift, and may determine a remaining time offset between the clock and the second clock. Further, the interface circuit may modify a future time at which the playback operation is to be performed by the second electronic device based on the remaining time offset to determine a corrected future time. Next, the interface circuit may transmit information specifying the corrected future time to the second electronic device via wireless communication.

It should be noted that the transmission time may be included in the payload and/or Media Access Control (MAC) header in a given packet. In some embodiments, the packet comprises a control packet. Alternatively or additionally, the packet may comprise a data packet.

Further, the clock circuit may include: an oscillator providing a reference clock; and a Frequency Locked Loop (FLL) circuit that generates a clock based on the reference clock. The interface circuit may modify the FLL to adjust the clock. For example, modifying the FLL may include changing a seed of a synthesizer (such as an accumulator in the synthesizer) in the FLL.

Further, the transmission time and the reception time may be stored at opposite ends of the payload of a given packet. In these embodiments, the electronic device may determine the duration of the payload (e.g., using software executed by a processor or interface circuitry), and the interface circuitry may add the duration to the remaining offset time.

In addition, the transmission time may include a second counter value corresponding to the second clock, and the reception time may include a counter value corresponding to the clock.

In some embodiments, the interface circuit transmits additional packets including audio data in the payload before transmitting the information, and the playback operation includes outputting the audio data. However, in some embodiments, at least some of the audio data is included in the same packet as the information.

It should be noted that adjusting the clock and modifying the future time coordinates the playback operation in the clock domain of the clock to within a predetermined value of the clock domain of the second clock.

Further, the electronic device may be a slave device, and the second electronic device may be a master device.

Further, the reception time is associated with a wireless ranging capability of the interface circuit.

Another embodiment provides a computer readable storage medium for an interface circuit in an electronic device. The computer-readable storage medium includes instructions for at least some of the operations performed by the electronic device.

Another embodiment provides a method for coordinating playback operations. The method includes at least some of the operations performed by the electronic device.

Another embodiment provides a second electronic device.

A third set of described embodiments includes an electronic device. The electronic device includes: one or more antennas; an interface circuit; and a clock circuit. During operation, the interface communicates with the second electronic device using wireless communication, and the clock circuit provides a clock in the electronic device. Further, the interface circuit may receive a packet from the second electronic device via wireless communication, where the given packet includes a transmission time based on a second clock in the second electronic device when the second electronic device transmits the given packet. In response, the interface circuit may store a receive time based on the clock when the packet was received, and may calculate a relative drift as a function of time between the clock and the second clock based on a difference between the transmit time and the receive time. The interface circuit may then adjust the clock circuit providing the clock based on the relative drift to eliminate the relative drift, and may store the adjustment to the clock circuit. Further, when the interface circuit or a second interface circuit in the second electronic device is reset, the interface circuit may adapt the clock circuit based on the stored adjustment to reduce the relative drift while the interface circuit recovers frequency lock with the second clock based on additional packets with additional transmit times received from the second electronic device.

It should be noted that the transmission time may be included in the payload and/or Media Access Control (MAC) header in a given packet. In some embodiments, the packet comprises a control packet. Alternatively or additionally, the packet may comprise a data packet.

Further, the clock circuit may include: an oscillator providing a reference clock; and a Frequency Locked Loop (FLL) circuit that generates a clock based on the reference clock. The interface circuit may modify the FLL to adjust the clock. For example, the adjustment may be applied to the seed of the first synthesizer in the FLL. In some embodiments, the FLL includes a second synthesizer that tracks adjustments to the first synthesizer.

Further, restoring frequency lock may occur over a time interval. Thus, prior to storing the adjustment, the interface circuit may average the adjustment over a time scale corresponding to the time interval. Alternatively or additionally, the adaptation is performed on a longer time scale than the adjustment. For example, the adaptation may be performed periodically, such as with a period that is part of a time interval.

In addition, the transmission time may include a second counter value corresponding to the second clock, and the reception time may include a counter value corresponding to the clock. It should be noted that the counter in the electronic device providing the counter value and/or the second counter in the second electronic device providing the second counter value may be reset when the interface circuit or the second interface circuit is reset. In these embodiments, the interface circuit may include a sample and hold circuit that mirrors the current counter value of the counter when the interface circuit or the second interface circuit is reset.

In some embodiments, the time of reception is associated with a wireless ranging capability of the interface circuit.

Another embodiment provides a computer readable storage medium for an interface circuit in an electronic device. The computer-readable storage medium includes instructions for at least some of the operations performed by the electronic device.

Another embodiment provides a method for reducing relative drift. The method includes at least some of the operations performed by the electronic device.

Another embodiment provides a second electronic device.

A fourth set of described embodiments includes an electronic device. The electronic device includes: a system clock circuit, a processor, one or more antennas, an interface circuit, and an interface clock circuit. During operation, the system clock circuit provides a system clock in the electronic device, the processor executes software, the interface communicates with the second electronic device using wireless communication, and the interface clock circuit provides an interface clock in the electronic device. Further, the interface circuit may receive a packet from the second electronic device via wireless communication, where the given packet includes a transmit time based on a second interface clock in the second electronic device when the second electronic device transmits the given packet. In response, the interface circuit may store a receive time based on the interface clock when the packet was received. Further, the interface circuit may receive, via wireless communication, information from the second electronic device specifying a future time when the electronic device is to perform the playback operation.

Additionally, the processor may maintain coordination between the system clock and the interface clock (e.g., by adjusting the system clock), where the interface clock has a higher frequency than the system clock. The processor may then calculate a relative drift as a function of time between the interface clock and the second interface clock based on a difference between the transmit time and the receive time. Further, the processor may adjust the system clock circuit based on the relative drift to eliminate the relative drift, wherein the relative drift based on the interface clock provides a higher resolution than the system clock. Next, the processor may determine a remaining time offset between the interface clock and the second interface clock. Further, the processor may modify the future time based on the remaining time offset to determine a corrected future time, and may perform a playback operation at the corrected future time.

It should be noted that the transmission time may be included in the payload and/or Media Access Control (MAC) header in a given packet. In some embodiments, the packet comprises a control packet. Alternatively or additionally, the packet may comprise a data packet.

In some embodiments, prior to performing the playback operation, the processor: disabling interrupts in the electronic device; and occupying at least a portion of the software stack by executing the loop to reduce latency associated with performing the playback operation.

Further, the electronic device may include a memory that stores instructions for maintaining, calculating, adjusting, determining, modifying, and executing.

Further, the transmission time and the reception time may be stored at opposite ends of the payload of a given packet. In these embodiments, the processor may determine the duration of the payload and may add the duration to the remaining offset time.

In addition, the packet may include audio data in the payload, and the electronic device stores the audio data in the queue. In these embodiments, the playback operation includes outputting audio data from the queue.

It should be noted that adjusting the system clock and modifying the future time may coordinate playback operations in the clock domain of the interface clock to within a predetermined value of the clock domain of the second interface clock.

Further, the second electronic device may be a master device, and the electronic device may be a slave device.

Further, the reception time may be associated with a wireless ranging capability of the interface circuit.

In addition, when a reset of the interface circuit or a second interface circuit in the second electronic device occurs, the processor may mirror the interface clock by sampling and holding a counter value corresponding to the interface clock in a counter in the interface circuit before resetting the counter. In these embodiments, the mirrored counter value may allow the interface circuit to reduce the relative drift while the interface circuit resumes frequency lock with the second interface clock based on additional packets with additional transmit times received by the interface circuit from the second electronic device.

Another embodiment provides a computer-readable storage medium for an interface circuit and/or a processor in an electronic device. The computer-readable storage medium includes instructions for at least some of the operations performed by the electronic device.

Another embodiment provides a method for coordinating playback operations. The method includes at least some of the operations performed by the electronic device.

Another embodiment provides a second electronic device.

A fifth set of described embodiments includes an electronic device. The electronic device includes: a system clock circuit, a processor, one or more antennas, an interface circuit, and an interface clock circuit. During operation, the system clock circuit provides a system clock in the electronic device, the processor executes software, the interface communicates with the second electronic device using wireless communication, and the interface clock circuit provides an interface clock in the electronic device. Further, the interface circuit may receive a packet from the second electronic device via wireless communication, where the given packet includes time coordination information based on a second interface clock in the second electronic device. In response, the interface circuit may coordinate the interface clock and the second interface clock based on the time coordination information. Further, the interface circuit may receive, via wireless communication, information from the second electronic device specifying a future time when the electronic device is to perform the playback operation.

In addition, the processor may capture timing information associated with the interface clock or the reference clock, which the interface clock circuit uses to generate the interface clock to increase the resolution of the system clock, where the interface clock has a higher frequency than the system clock. The processor may then track a relative drift as a function of time between the system clock and the interface clock using the timing information, and may determine an estimated time offset between the interface clock and the system clock at a future time based on the relative drift. Next, the processor may modify the future time based on the estimated time offset to determine a corrected future time, and may perform a playback operation at the corrected future time.

It should be noted that the transmission time may be included in the payload and/or Media Access Control (MAC) header in a given packet. In some embodiments, the packet comprises a control packet. Alternatively or additionally, the packet may comprise a data packet.

In some embodiments, prior to performing the playback operation, the processor: disabling interrupts in the electronic device; and occupying at least a portion of the software stack by executing the loop to reduce latency associated with performing the playback operation.

Further, the electronic device may include a memory that stores instructions for capturing, tracking, determining, modifying, and executing.

Additionally, capturing timing information may include storing a time value of an interface clock in a register or counter.

Additionally, the electronic device may include an oscillator that provides a reference clock, and the interface clock circuit may provide the interface clock based on the reference clock. In these embodiments, timing information is captured from a reference clock.

Further, the interface circuit may receive additional packets including audio data in the payload. Alternatively or additionally, at least some of the audio data may be received in the same packets as the information. In these embodiments, the electronic device stores the audio data in a queue, and the playback operation includes outputting the audio data from the queue.

It should be noted that the capturing, tracking, determining and modifying may coordinate the playback operation within predetermined values of the clock domain of the second interface clock.

Further, the second electronic device may be a master device, and the electronic device may be a slave device.

Additionally, the time of reception may be associated with a wireless ranging capability of the interface circuit.

Another embodiment provides a computer-readable storage medium for an interface circuit and/or a processor in an electronic device. The computer-readable storage medium includes instructions for at least some of the operations performed by the electronic device.

Another embodiment provides a method for coordinating playback operations. The method includes at least some of the operations performed by the electronic device.

Another embodiment provides a second electronic device.

This summary is provided for the purpose of illustrating some exemplary embodiments in order to provide a basic understanding of some aspects of the subject matter described herein. Accordingly, it should be understood that the above-described features are examples and should not be construed to narrow the scope or spirit of the subject matter described herein in any way. Other features, aspects, and advantages of the subject matter described herein will become apparent from the following detailed description, the drawings, and the claims.

Drawings

Fig. 1 is a block diagram illustrating an example of a system having an electronic device according to an embodiment of the present disclosure.

Fig. 2 is a flow diagram illustrating an example of a method for coordinating playback operations in accordance with an embodiment of the present disclosure.

Fig. 3 is a diagram illustrating an example of communication between the electronic devices in fig. 1 according to an embodiment of the present disclosure.

Fig. 4 is a flow diagram illustrating an example of a method for coordinating playback operations in accordance with an embodiment of the present disclosure.

Fig. 5 is a diagram illustrating an example of communication between the electronic devices in fig. 1 according to an embodiment of the present disclosure.

Fig. 6 is a block diagram illustrating an example of the electronic device in fig. 1, in accordance with an embodiment of the present disclosure.

Fig. 7 is a block diagram illustrating an example control circuit in the electronic device of fig. 6, in accordance with an embodiment of the present disclosure.

Fig. 8 is a timing diagram illustrating an example of a clock in the electronic device of fig. 1 according to an embodiment of the present disclosure.

Fig. 9 is a flow chart illustrating an example of a method for reducing drift in accordance with an embodiment of the present disclosure.

Fig. 10 is a diagram illustrating an example of communication between the electronic devices in fig. 1 according to an embodiment of the present disclosure.

Fig. 11 is a block diagram illustrating an example of a clock circuit in the electronic device in fig. 6 according to an embodiment of the present disclosure.

Fig. 12 is a timing diagram illustrating an example of a clock in the electronic device of fig. 1 as a function of time after a wireless reset in accordance with an embodiment of the present invention.

Fig. 13 is a flow diagram illustrating an example of a method for coordinating playback operations in accordance with an embodiment of the present disclosure.

Fig. 14 is a diagram illustrating an example of communication between the electronic devices in fig. 1 according to an embodiment of the present disclosure.

Fig. 15 is a block diagram illustrating an example of the electronic device in fig. 1, according to an embodiment of the present disclosure.

Fig. 16 is a flow diagram illustrating an example of a method for coordinating playback operations in accordance with an embodiment of the present disclosure.

Fig. 17 is a diagram illustrating an example of communication between the electronic devices in fig. 1 according to an embodiment of the present disclosure.

Fig. 18 is a block diagram illustrating an example of one of the electronic devices of fig. 1 in accordance with an embodiment of the present disclosure.

It should be noted that throughout the drawings, like reference numerals designate corresponding parts. Further, multiple instances of the same component are designated by a common prefix separated from the instance number by a dashed line.

Detailed Description

In a first set of embodiments, an electronic device coordinates playback operations. In particular, the interface circuit in the electronic device may calculate the relative drift as a function of time between a clock in the interface circuit and a second clock in the second electronic device based on a difference between a transmission time when the second electronic device transmits a packet and a reception time of the packet. The interface circuit may then adjust the clock circuit providing the clock based on the relative drift to eliminate the relative drift, and may determine a remaining time offset between the clock and the second clock. Next, the interface circuit may modify a future time at which the electronic device is to perform the playback operation based on the remaining time offset to determine a corrected future time, and the electronic device may perform the playback operation at the corrected future time.

By coordinating playback operations (e.g., playback of audio content), the coordination techniques may provide an improved acoustic experience in an environment that includes the electronic device and/or the second electronic device. For example, the coordination technique may ensure that playback is within a predetermined value of the clock domain of the second clock. This capability may eliminate the user's perception of drift or changes in the timing of playback operations, such as flutter echoes. Furthermore, such capabilities may facilitate surround sound or multi-channel sound. In these ways, the coordination techniques may improve the user experience when using the electronic device and/or the second electronic device. Thus, the coordination technique may increase customer loyalty and revenues of the provider of the electronic device and/or the second electronic device.

In a second set of embodiments, an electronic device coordinates playback operations. In particular, the interface circuit in the electronic device may calculate the relative drift as a function of time between a clock in the interface circuit and a second clock in the second electronic device based on a difference between a transmission time when the second electronic device transmits a packet and a reception time of the packet. The interface circuit may then adjust the clock circuit providing the clock based on the relative drift to eliminate the relative drift, and may determine a remaining time offset between the clock and the second clock. Next, the interface circuit may modify a future time at which the second electronic device is to perform the playback operation based on the remaining time offset to determine a corrected future time, and may transmit information specifying the corrected future time to the second electronic device.

By coordinating playback operations, such as playback of audio content, the coordination techniques may provide an improved acoustic experience in an environment that includes the electronic device and/or the second electronic device. For example, the coordination technique may ensure that playback is within a predetermined value of a clock domain of a clock. This capability may eliminate the user's perception of drift or changes in the timing of playback operations, such as flutter echoes. Furthermore, such capabilities may facilitate surround sound or multi-channel sound. In these ways, the coordination techniques may improve the user experience when using the electronic device and/or the second electronic device. Thus, the coordination technique may increase customer loyalty and revenues of the provider of the electronic device and/or the second electronic device.

In a third set of embodiments, the electronics reduce relative drift. In particular, the interface circuit in the electronic device may calculate the relative drift as a function of time between a clock in the interface circuit and a second clock in the second electronic device based on a difference between a transmission time when the second electronic device transmits a packet and a reception time of the packet. The interface circuit may then adjust the clock circuit providing the clock based on the relative drift to eliminate the relative drift, and may store the adjustment to the clock circuit. Further, when a wireless reset occurs, the interface circuit may adjust the clock circuit based on the stored adjustment to reduce the relative drift while the interface circuit recovers frequency lock with the second clock based on additional packets with additional transmit times received from the second electronic device.

By reducing drift, the coordination technique can maintain coordination of playback operations (such as playback of audio content) performed by the electronic device. In this process, the coordination techniques may provide an improved acoustic experience in an environment that includes the electronic device and/or the second electronic device. For example, the coordination technique may ensure that playback is within a predetermined value of the clock domain of the second clock. This capability may eliminate the user's perception of drift or changes in the timing of playback operations, such as flutter echoes. Furthermore, such capabilities may facilitate surround sound or multi-channel sound. In these ways, the coordination techniques may improve the user experience when using the electronic device and/or the second electronic device. Thus, the coordination technique may increase customer loyalty and revenues of the provider of the electronic device and/or the second electronic device.

In a fourth set of embodiments, the electronic device coordinates playback operations. In particular, a processor in an electronic device may maintain coordination between a system clock provided by a system clock circuit and an interface clock provided by a clock circuit. The processor may then calculate a relative drift as a function of time between the interface clock and a second interface clock in the second electronic device based on a difference between a transmission time when the second electronic device transmits the packet and a reception time of the packet. Further, the processor may adjust the system clock circuit based on the relative drift to eliminate the relative drift. Next, the processor may determine a remaining time offset between the interface clock and the second interface clock. Further, the processor may modify a future time at which the electronic device is to perform the playback operation based on the remaining time offset to determine a corrected future time, and may perform the playback operation at the corrected future time.

By coordinating playback operations, such as playback of audio content, the coordination techniques may provide an improved acoustic experience in an environment that includes the electronic device and/or the second electronic device. For example, the coordination technique may ensure that playback is within a predetermined value of the clock domain of the second clock. This capability may eliminate the user's perception of drift or changes in the timing of playback operations, such as flutter echoes. Furthermore, such capabilities may facilitate surround sound or multi-channel sound. In these ways, the coordination techniques may improve the user experience when using the electronic device and/or the second electronic device. Thus, the coordination technique may increase customer loyalty and revenues of the provider of the electronic device and/or the second electronic device.

In a fifth set of embodiments, an electronic device coordinates playback operations. In particular, the processor in the electronic device may coordinate the interface clock in the electronic device with the second interface clock in the second electronic device based on the time coordination information received in the packet from the second electronic device. The processor may then capture timing information associated with the interface clock provided by the interface clock circuit to increase the resolution of the system clock. Further, the processor may track a relative drift as a function of time between the system clock and the interface clock using the timing information, and may determine an estimated time offset between the interface clock and the system clock at a future time based on the relative drift. Next, the processor may modify a future time at which the electronic device is to perform the playback operation based on the estimated time offset to determine a corrected future time, and may perform the playback operation at the corrected future time.

By coordinating playback operations, such as playback of audio content, the coordination techniques may provide an improved acoustic experience in an environment that includes the electronic device and/or the second electronic device. For example, the coordination technique may ensure that playback is within a predetermined value of the clock domain of the second clock. This capability may eliminate the user's perception of drift or changes in the timing of playback operations, such as flutter echoes. Furthermore, such capabilities may facilitate surround sound or multi-channel sound. In these ways, the coordination techniques may improve the user experience when using the electronic device and/or the second electronic device. Thus, the coordination technique may increase customer loyalty and revenues of the provider of the electronic device and/or the second electronic device.

In the following discussion, the electronic device and/or a second electronic device (such as an audio/video (A/V) hub, an A/V display device, a portable electronic device, a receiver device, a speaker, etc.)Phonons and/or consumer electronics) may include radios that wirelessly communicate packets or frames according to one or more communication protocols, such as: the Institute of Electrical and Electronics Engineers (IEEE)802.11 standard (sometimes referred to as'', from Austin, TexAlliance)(from the bluetooth special interest group, corchland, washington), cellular telephone communication protocols, near field communication standards or specifications (from the NFC forum, weckfield, ma), and/or other types of wireless interfaces. For example, a cellular telephone communication protocol may include or may be compatible with: a second generation mobile telecommunications technology, a third generation mobile telecommunications technology (such as a communication protocol that conforms to the international mobile telecommunications-2000 specification of the international telecommunications union of geneva, switzerland), a fourth generation mobile telecommunications technology (e.g., a communication protocol that conforms to the international mobile telecommunications advanced specification of the international telecommunications union of geneva, switzerland), and/or another cellular telephone communication technology. In some embodiments, the communication protocol comprises long term evolution or LTE. However, various communication protocols (such as ethernet) may be used. Additionally, wireless communication may occur via various frequency bands, such as at or in the following frequency bands: a 2GHz radio band, a 5GHz radio band, an ISM band, a 60GHz radio band, and an ultra-wideband. It should be noted that the electronic device may communicate using infrared communications that are compatible with infrared communications standards, including one-way infrared communications or two-way infrared communications.

Further, the a/V content in the following discussion may include video and associated audio (such as music, sounds, conversations, etc.), video only or audio only.

Communication between electronic devices is illustrated in FIG. 1, where FIG. 1 presents a block diagram illustrating an example of a system 100 having a portable electronic device 110 (such as a remote control or cellular telephone), one or more A/V hubs (such as A/V hub 112, more generally a physical access point or a software-based access point), one or more A/V display devices 114 (such as a television, monitor, computer, more generally a display associated with an electronic device), one or more receiver devices (such as receiver device 116, e.g., a local wireless receiver associated with a proximate A/V display device 114-1, which proximate A/V display device 114-1 may receive frame-by-frame transcoded A/V content from A/V hub 112 for display on A/V display device 114-1), One or more speakers 118 (and more generally, one or more electronic devices including one or more speakers) that may receive and output audio data or content and/or one or more content sources 120 associated with one or more content providers. For example, one or more content sources 120 may include: a radio receiver, a video player, a satellite receiver, an access point providing connectivity to a wired network such as the internet, a media or content source, a consumer electronic device, an entertainment device, a set-top box, top-level content delivered over the internet or network without the involvement of a cable, satellite, or multiple system operator, a security camera, a surveillance camera, and the like. It should be noted that A/V hub 112, A/V display device 114, receiver device 116, and speaker 118 are sometimes collectively referred to as "components" in system 100. However, A/V hub 112, A/V display device 114, receiver device 116, and/or speakers 118 are sometimes referred to as "electronic devices".

In particular, portable electronic device 110 and A/V hub 112 may communicate with each other using wireless communication, and one or more other components in system 100 (such as at least one of A/V display devices 114, receiver device 116, one of speakers 118, and/or one of content sources 120) may communicate using wireless communication and/or wired communication. During wireless communication, these electronic devices may wirelessly communicate: sending advertising frames on the wireless channel, detecting each other by scanning the wireless channel, establishing a connection (e.g., by sending an association request), and/or sending and receiving packets or frames (which may include an association request and/or additional information as payload, such as information specifying communication performance, data, audio and/or video content, timing information, etc.).

As described further below with reference to fig. 18, portable electronic device 110, a/V hub 112, a/V display device 114, receiver device 116, speaker 118, and content source 120 may include subsystems such as: a network subsystem, a memory subsystem, and a processor subsystem. Further, one or more of portable electronic device 110, a/V hub 112, receiver device 116 and/or speakers 118, and optionally a/V display device 114 and/or content source 120, may include a radio 122 in a network subsystem. It should be noted that in some embodiments, the radio or receiver device is located in the A/V display device, for example, radio 122-5 is included in A/V display device 114-2. Further, it should be noted that the radios 122 may be examples of the same radio or may be different from each other. More generally, portable electronic device 110, A/V hub 112, sink device 116, and/or speaker 118 (and optionally A/V display device 114 and/or content source 120) may comprise or be included within any electronic device having a network subsystem that enables portable electronic device 110, A/V hub 112, sink device 116, and/or speaker 118 (and optionally A/V display device 114 and/or content source 120) to wirelessly communicate with one another. Wireless communication may include sending advertisements over the wireless channel to enable the electronic devices to make initial contact or detect each other, then exchanging subsequent data/management frames (such as association requests and responses) to establish a connection, configuring security options (e.g., internet protocol security), sending and receiving packets or frames over the connection, and so forth.

As can be seen in FIG. 1, a wireless signal 124 (represented by a sawtooth line) is transmitted from the radio 122-1 in the portable electronic device 110. The wireless signals are received by at least one of: at least one of a/V hub 112, sink device 116, and/or speakers 118 (and, optionally, one or more of a/V display device 114 and/or content source 120). For example, the portable electronic device 110 may transmit the packet. These packets, in turn, may be received by radio 122-2 in a/V hub 112. This may allow the portable electronic device 110 to communicate information to the a/V hub 112. Although fig. 1 shows the portable electronic device 110 sending packets, it should be noted that the portable electronic device 110 may also receive packets from the a/V hub 112 and/or one or more other components in the system 100. More generally, wireless signals may be transmitted and/or received by one or more components in system 100.

In the depicted embodiment, the processing of packets or frames in portable electronic device 110, a/V hub 112, sink device 116, and/or speaker 118 (and optionally one or more of a/V display device 114 and/or content source 120) includes: receiving a wireless signal 124 having packets or frames; decoding/extracting a packet or frame from the received wireless signal 124 to obtain a packet or frame; and processing the packets or frames to determine information contained in the packets or frames (such as information associated with the data stream). For example, the information from the portable electronic device 110 may include user interface activity information associated with a user interface displayed on a Touch Sensitive Display (TSD)128 in the portable electronic device 110, which the user of the portable electronic device 110 uses to at least control: at least one of a/V hub 112, a/V display device 114, at least one of speakers 118, and/or at least one of content sources 120. (in some embodiments, portable electronic device 110 includes a user interface with physical knobs and/or buttons that a user may use to at least control: A/V hub 112, one of A/V display devices 114, at least one of speakers 118, and/or one of content sources 120, in lieu of touch-sensitive display 128 or in addition to touch-sensitive display 128). Alternatively, information from one or more of portable electronic device 110, a/V hub 112, a/V display device 114, receiver device 116, one or more of speakers 118, and/or one or more of content sources 120 may specify communication capabilities related to communications between portable electronic device 110 and one or more other components in system 100. Further, the information from a/V hub 112 may include device status information (such as on, off, play, rewind, fast-forward, selected channel, selected a/V content, content source, etc.) related to a current device status of at least one of a/V display devices 114, at least one of speakers 118, and/or one of content sources 120, or may include user interface information for a user interface (which may be dynamically updated based on the device status information and/or user interface activity information). Further, at least information from one of A/V hub 112 and/or content sources 120 may include audio and/or video (which is sometimes denoted as "audio/video" or "A/V" content) that is displayed or presented on one or more of A/V display devices 114, as well as display instructions or presentation commands that specify how the audio and/or video is to be displayed, presented, or output. However, as previously described, audio and/or video may be sent between components in the system 100 via wired communication. Thus, as shown in FIG. 1, such as between A/V hub 112 and A/V display device 114-3, there may be a wired cable or link, such as a High Definition Multimedia Interface (HDMI) cable 126.

It should be noted that A/V hub 112 may determine display instructions (with a display layout) for A/V content based on the format of the display in A/V display device 114-1. Alternatively, a/V hub 112 may use predetermined display instructions, or a/V hub 112 may modify or transform a/V content based on the display layout such that the modified or transformed a/V content has an appropriate display format for display on the display. Further, the display instructions may specify information to be displayed on a display in A/V display device 114-1, including a location to display A/V content (such as in a central window, in a tiled window, etc.). Thus, the information to be displayed (i.e., an instance of the display instruction) may be based on the format of the display, such as: display size, display resolution, display aspect ratio, display contrast, display type, etc. In some embodiments, the A/V content comprises HDMI content. However, in other embodiments, the A/V content is compatible with another format or standard, such as: h.264, MPEG-2, QuickTime video format, MPEG-4, MP4, and/or TCP/IP. Further, the video mode of the a/V content may be 720p, 1080i, 1080p, 1440p, 2000, 2160p, 2540p, 4000p, and/or 4320 p.

Alternatively or additionally, the display instructions for the a/V content determined by a/V hub 112 may be based on a desired acoustic effect (such as mono sound, stereo sound, or multi-channel sound), a desired acoustic equalization, predefined acoustic characteristics of the ambient environment (such as an acoustic transfer function, an acoustic loss, an acoustic delay, acoustic noise in the environment, ambient sound in the environment, and/or one or more reflections), and/or a current location of one or more users in the environment relative to one or more of a/V display device 114-1 and/or speakers 118. For example, the display instructions may include a temporal relationship or coordination between playback times of audio output by the speakers 118 to achieve a desired acoustic effect.

Further, it should be noted that when A/V hub 112 receives A/V content from one of content sources 120, A/V hub 112 may provide the A/V content and display instructions to one or more of A/V display device 114-1 and/or speakers 118 as frames or packets having the A/V content are received from one of content sources 120 (e.g., in real-time) for display of the A/V content on a display in A/V display device 114-1 and/or output of the A/V content by one or more of speakers 118. For example, A/V hub 112 may collect A/V content in a buffer until an audio or video frame is received, and A/V hub 112 may then provide the complete frame to one or more of A/V display device 114-1 and/or speakers 118. Alternatively, A/V hub 112 may provide packets having portions of audio or video frames to one or more of A/V display device 114-1 and/or speakers 118. In some embodiments, display instructions may be provided to a/V display device 114-1 and/or one or more speakers 118 differently (such as when display instructions change), regularly or periodically (such as one out of every N frames or packets), or in each packet.

Further, it should be noted that communications between portable electronic device 110, a/V hub 112, one or more of a/V display devices 114, receiver device 116, one or more of speakers 118, and/or one or more content sources 120 may be characterized by various performance metrics, such as: received Signal Strength Indicator (RSSI), data rate discounting radio protocol overhead (sometimes referred to as "throughput"), error rate (such as packet error rate, or retry rate or retransmission rate), mean square error of the equalized signal relative to the equalization target, intersymbol interference, multipath interference, signal-to-noise ratio, eye diagram width, the ratio of the number of bytes successfully transmitted over a time interval (such as 1 to 10 seconds) to the estimated maximum number of bytes that can be transmitted over the time interval (the latter sometimes referred to as the 'capacity' of the channel or link), and/or the ratio of the actual data rate to the estimated maximum data rate (sometimes referred to as "utilization"). In addition, performance associated with different channels during communication may be monitored (e.g., to identify dropped packets) individually or jointly.

Communication between the portable electronic device 110, the a/V hub 112, one of the a/V display devices 114, the sink device 116, one of the speakers 118, and/or one or more content sources 120 in fig. 1 may involve one or more independent concurrent data streams in different wireless channels (or even different communication protocols, such as different Wi-Fi communication protocols) in one or more connections or links, which may be transmitted using multiple radios. It should be noted that one or more connections or links may each have a separate or different identifier (such as a different service set identifier) on a wireless network (which may be a private network or a public network) in system 100. Furthermore, one or more concurrent data streams may be partially or fully redundant on a dynamic or packet-by-packet basis to improve or maintain performance metrics even in the presence of transient changes (such as interference, changes in the amount of information that needs to be communicated, movement of the portable electronic device 110, etc.) and in order to be serviced (while remaining compatible with a communication protocol (e.g., Wi-Fi communication protocol)), such as: channel calibration, determining one or more performance indicators, performing quality of service characterization without interrupting communication (such as performing channel estimation, determining link quality, performing channel calibration, and/or performing spectral analysis associated with at least one channel), seamless switching between different wireless channels, coordinated communication between components, and the like. These features may reduce the number of retransmitted packets and, thus, may reduce latency and avoid communication interruptions, and may enhance the experience of one or more users watching a/V content on one or more of a/V display devices 114 and/or listening to audio output by one or more of speakers 118.

As previously described, a user may control at least one of a/V hub 112, at least one of a/V display device 114, at least one of speakers 118, and/or at least one of content sources 120 via a user interface displayed on touch-sensitive display 128 on portable electronic device 110. In particular, at a given time, the user interface may include one or more virtual icons that allow the user to activate, deactivate, or change at least the functions or capabilities of: at least one of a/V hub 112, a/V display device 114, at least one of speakers 118, and/or at least one of content sources 120. For example, a given virtual icon in the user interface may have an associated tap region located on the surface of the touch-sensitive display 128. If the user makes contact with the surface and then breaks contact with the surface within the tap region (e.g., using one or more fingers or toes, or using a stylus), the portable electronic device 110 (such as a processor executing program modules) may receive a user interface activity notification indicating activation of a command or instruction from a touch screen input/output (I/O) controller coupled to the touch-sensitive display 128. In response, program module may instruct interface circuitry in portable electronic device 110 to wirelessly transmit user interface activity information indicative of a command or instruction to A/V hub 112 and A/V hub 112 may transmit the command or instruction to a target component in system 100 (such as A/V display device 114-1). the instruction or command may cause A/V display device 114-1 to open or close, cause A/V display device 114-1 to display A/V content from a particular content source, control a display device 128 to display A/V content, and/or control a display device to display A/V content from a particular content source, Cause a/V display device 114-1 to perform a trick mode operation such as fast forward, rewind, fast rewind, or skip, etc. For example, A/V hub 112 may request A/V content from content source 120-1 and may then provide the A/V content to A/V display device 114-1 along with display instructions for A/V display device 114-1 to display the A/V content. Alternatively or additionally, a/V hub 112 may provide audio content associated with the video content from content source 120-1 to one or more of speakers 118.

As previously mentioned, achieving high audio quality in an environment (e.g., a room, building, vehicle, etc.) is often challenging. In particular, achieving high audio quality in the environment often places strong restrictions on the coordination of the loudspeakers (e.g., speaker 118). For example, the coordination may need to maintain an accuracy of 1-5 μ s. (it should be noted that these and other numerical values in the discussion are non-limiting exemplary values, therefore, the accuracy may be different, such as 10 or 50 μ s.) without proper coordination, the acoustic quality in the environment may be degraded, with the same amount of impact on listener satisfaction and overall user experience when listening to the audio content and/or A/V content.

This challenge may be addressed in the coordination technique by directly or indirectly coordinating speakers 118 with a/V hub 112 (and thus with each other). As described below with reference to fig. 2-17, in some embodiments, coordinated playback of audio content by speakers 118 may be facilitated using wireless communication. In particular, because the optical speed is approximately six orders of magnitude faster than the acoustic speed, the propagation delay of the wireless signal in the environment (e.g., a room) is negligible with respect to the desired accuracy of coordination of the speaker 118. For example, the desired coordination accuracy of the speaker 118 may be on the order of microseconds, while the propagation delay in a typical room (e.g., a distance of up to 10 meters to 30 meters) may be one or two orders of magnitude less. Thus, by including information specifying the time of transmission in packets transmitted by the a/V hub 112 to a given one of the speakers 118, and by recording or storing the time of receipt of these packets at the given speaker, the timing of playback operations (such as playing audio) can be coordinated within a predetermined value (such as, for example, within 1-5 μ s). Specifically, as described below with reference to fig. 2, 3, and 6-8, a/V hub 112 may transmit frames or packets that include a transmission time when a/V hub 112 transmits the frames or packets based on an interface clock provided by clock circuit 130-1 (such as an interface clock circuit in or associated with an interface circuit in a/V hub 112), and interface circuitry in one or more of speakers 118, such as speaker 118-1, may record or store the time of receipt of the packet based on an interface clock provided by clock circuitry 130-2, such as interface clock circuitry in interface circuitry in speaker 118-1 or interface clock circuitry associated with interface circuitry in speaker 118-1. Based on the difference between the transmit time and the receive time, the interface circuitry in speaker 118-1 may calculate the relative drift as a function of time between the interface clock provided by clock circuitry 130-1 and the interface clock provided by clock circuitry 130-2.

Interface circuitry in speaker 118-1 may then adjust clock circuit 130-2 based on the relative drift to eliminate the relative drift. For example, interface circuitry in speaker 118-1 may adjust Frequency Locked Loop (FLL) circuitry in clock circuitry 130-2 to frequency lock the interface clock provided by clock circuitry 130-1 and the interface clock provided by clock circuitry 130-2. In addition, interface circuitry in speaker 118-1 may determine a remaining time offset between the interface clock provided by clock circuitry 130-1 and the interface clock provided by clock circuitry 130-2.

This remaining time offset may be used to correct the phase between locking the interface clock provided by clock circuit 130-1 and the interface clock provided by clock circuit 130-2 when performing playback operations, such as outputting audio data received from a/V hub 112. In particular, interface circuitry in speaker 118-1 may receive, via wireless communication, a frame or packet from a/V hub 112 with information specifying a future time at which speaker 118-1 is to perform a playback operation. Next, interface circuitry in speaker 118-1 may modify the future time based on the remaining time offset to determine a corrected future time, and speaker 118-1 may perform a playback operation at the corrected future time.

Alternatively or additionally, as described further below with reference to fig. 4 and 5, the roles of a/V hub 112 and speakers 118-1 in the coordination technique may be reversed such that a/V hub 112 performs at least some of the above-described operations performed by speakers 118-1. Thus, instead of A/V hub 112 transmitting a packet with a transmission time to speaker 118-1, speaker 118-1 may transmit the packet to A/V hub 112. a/V hub 112 may then perform operations similar to those of speaker 118-1 as described above, and may transmit a frame or packet to speaker 118-1 with information specifying a future time of correction.

Furthermore, as further described below with reference to fig. 9-12, to reduce or eliminate drift between the interface clock provided by clock circuit 130-1 and the interface clock provided by clock circuit 130-2 after a wireless reset of the interface circuits in a/V hub 112 and/or the interface circuits in speakers 118-1, in some embodiments, the interface circuits in speakers 118-1 may adapt clock circuit 130-2 based on stored previous adjustments to clock circuit 130-2. For example, interface circuitry in speaker 118-1 may adjust an FLL circuit in clock circuit 130-2 (such as a seed of a synthesizer in the FLL circuit) based on an average of previous adjustments to clock circuit 130-2. In this manner, coordination of playback operations may be maintained within predetermined values while interface circuitry in speaker 118-1 recovers frequency locking with the interface clock provided by clock circuitry 130-1 based on additional packets having additional transmit times received from A/V hub 112. Alternatively or additionally, at least some of the above-described operations performed by speaker 118-1 during a wireless reset may be performed by a/V hub 112.

The foregoing embodiments use interface circuitry in a/V hub 112 and/or speaker 118-1 to enable and/or maintain coordination of playback operations between the clock domain of a/V hub 112 and the clock domain of speaker 118-1 within predetermined values, in other embodiments, the coordination of playback operations is performed, at least in part, using software executed by a processor. This will be further described below with reference to fig. 13 to 17. It should be noted that while these embodiments show the processor in speaker 118-1 executing software, in other embodiments at least some of the operations performed by the processor in speaker 118-1 are performed by the processor in a/V hub 112 executing software.

In some embodiments, techniques such as wireless ranging or radio-based distance measurement may be used to facilitate coordination of playback operations. For example, wireless ranging may be used to determine and correct the propagation delay of light between A/V hub 112 and/or speaker 118-1 when the propagation delay is not less than a predetermined value by at least one or two orders of magnitude, such as when A/V hub 112 and speaker 118-1 are in different rooms. (when the distance is in one room and the electronics are stationary, the propagation delay introduces negligible static contribution to the remaining time offset.) typically, the distance between the a/V hub 112 and the speaker 118-1 is determined based on the product of the time of flight (the difference between the transmit time and the receive time in the common clock domain) and the propagation speed.

Further, one or more additional techniques may be used to identify and/or exclude multipath wireless signals during coordination of playback operations. For example, a/V hub 112 and/or speakers 118 may determine an angle of arrival (including non-line-of-sight reception) using: directional antennas, differential times of arrival at an antenna array with known locations, and/or angles of arrival of two radios with known locations (e.g., trilateration or multilateration).

Although the preceding examples illustrate wireless ranging using a common clock domain in A/V hub 112 and/or speakers 118-1, in other embodiments, wireless ranging is performed when the interface clock provided by clock circuit 130-1 and the interface clock provided by clock circuit 130-2 are not coordinated. For example, even when the time of transmission is unknown or unavailable, the location of the a/V hub 112 and/or speakers 118 may be estimated based on the propagation speed and time of arrival data (which is sometimes referred to as "differential time of arrival") of the wireless signal 124 at several receivers at different known locations. More generally, various radiolocation techniques may be used, for example: determining a distance based on a difference in Received Signal Strength Indicator (RSSI) power relative to an original transmitted signal strength, which may include corrections for absorption, refraction, shadowing, and/or reflection; determining an angle of arrival (including non-line-of-sight reception) at a radio using a directional antenna or differential time of arrival based on an antenna array having a known location; determining a distance based on the backscattered wireless signal; and/or determining angles of arrival (i.e., trilateration or multilateration) at least two radios having known locations. It should be noted that wireless signal 124 may include transmissions over a GHz or multi-GHz bandwidth to produce pulses of short duration, such as, for example, approximately 1ns, which may allow range to be determined within 0.3m (e.g., 1 ft). In some embodiments, wireless ranging is facilitated using location information (such as the location of one or more of the electronic devices in fig. 1 determined or specified by a local positioning system, a global positioning system, a cellular telephone network, and/or a wireless network).

Although we describe the network environment shown in fig. 1 as an example, in alternative embodiments, there may be a different number or type of electronic devices. For example, some embodiments include more or fewer electronic devices. As another example, in another embodiment, a different electronic device is transmitting and/or receiving packets or frames. Although the electronic device in fig. 1 is shown with a single instance of radio 122, in other embodiments, one or more of these components may include multiple radios.

Coordinating playback operations using interface circuitry

We now describe embodiments of the coordination technique. In some embodiments, the coordination techniques are implemented, at least in part, using hardware such as interface circuitry. This is illustrated in fig. 2, where fig. 2 presents a flow chart illustrating an example of a method 200 for coordinating playback operations. Method 200 may be performed by interface circuitry in an electronic device, which may be a slave device, such as one of a/V display devices 114 (fig. 1) or one of speakers 118 (fig. 1).

During operation, the interface circuit may receive a packet from a second electronic device (which may be a master device) via wireless communication (operation 210), where the given packet includes a transmission time based on a second clock in the second electronic device when the second electronic device transmits the given packet. It should be noted that the transmission time may be included in the payload and/or Media Access Control (MAC) header in a given packet. In some embodiments, the packet comprises a control packet. Alternatively or additionally, the packet may comprise a data packet.

In response to receiving the packet, the interface circuit may store a receive time when the packet was received (operation 212), where the receive time is based on a clock in the electronic device. It should be noted that the transmission time may correspond to the leading edge or the trailing edge of the packet. Similarly, the reception time may correspond to the leading edge or the trailing edge of the packet.

The interface circuit may then calculate a relative drift as a function of time between the clock and the second clock based on the difference between the transmit time and the receive time (operation 214), and may adjust a clock circuit providing the clock (such as an interface clock circuit in or associated with the interface circuit) based on the relative drift to eliminate the relative drift (operation 216). For example, the adjustment may be based on a difference of consecutive packets, and the adjustment may frequency lock the clock and the second clock.

In addition, the interface circuit may determine a remaining time offset between the clock and the second clock (operation 218).

Further, the interface circuit may receive, via wireless communication, information from the second electronic device specifying a future time at which the electronic device is to perform the playback operation (operation 220).

Additionally, the interface circuit may modify the future time based on the remaining time offset (operation 222) to determine a corrected future time.

Next, the electronic device may perform a playback operation at the corrected future time (operation 224), wherein adjusting the clock and modifying the future time coordinates the playback operation in the clock domain of the clock to within a predetermined value of the clock domain of the second clock.

In some embodiments, the packet includes audio data in a payload, and the electronic device stores the audio data in a queue. In these embodiments, the playback operation includes outputting audio data from the queue. (however, in other embodiments, the playback operation includes displaying video, which may be coordinated with audio to prevent unintended timing offsets between sounds and images that may be noticed by a viewer.) it should be noted that adjusting the clock (operation 216) and modifying the future time (operation 222) may coordinate the playback operation.

Further, the interface circuitry (and/or the electronic device) may optionally perform one or more additional operations (operation 226). For example, the transmission time and the reception time may be stored at opposite ends of the payload of a given packet. Thus, the transmission time may be at the beginning of the payload and the reception time may be appended to the end of the payload. In these embodiments, interface circuitry in the electronic device or a processor executing software may determine the duration of the payload, and the interface circuitry may add the duration to the remaining offset time.

Fig. 3 presents a diagram illustrating an example of communication between a/V hub 112 and speakers 118-1. In particular, interface circuitry 310 in a/V hub 112 may transmit one or more packets (e.g., packet 312) to speaker 118-1. Each packet may include a corresponding transmit time 314 based on an interface clock 316 provided by an interface clock circuit 318 in interface circuit 310 in a/V hub 112 or an interface clock circuit 318 associated with interface circuit 310 in a/V hub 112 when a/V hub 112 transmits packet 312. When interface circuit 320 in speaker 118-1 receives a packet, interface circuit 320 may include a receive time in the packet (or interface circuit 320 may store the receive time in memory 330), where for each packet, the corresponding receive time 322 may be based on an interface clock 324 provided by interface clock circuit 326 in interface circuit 320 or interface clock circuit 326 associated with interface circuit 320.

The interface circuit 320 may then calculate a relative drift 332 as a function of time between the interface clock 316 and the interface clock 324 based on the difference between the transmit time and the receive time, and may adjust 334 the interface clock circuit 326 to eliminate the relative drift 332 based on the relative drift 332. In addition, interface circuit 320 may determine a remaining time offset 336 between interface clock 316 and interface clock 324.

In some embodiments, the transmission time and the reception time may be stored at opposite ends of the payload of the packet. In these embodiments, the interface circuit 320 or the processor 338 executing software in the speaker 118-1 may determine a duration 342 or time associated with the length 340 of the payload, and the interface circuit 320 may add the duration 342 to the remaining offset time 336.

In addition, interface circuit 310 may transmit packet 346, where packet 346 includes information specifying future time 344 at which speaker 118-1 is to perform playback operation 350. Upon receiving the packet 346, the interface circuit 320 may modify the future time 344 based on the remaining time offset 336 to determine a corrected future time 348.

Next, speaker 118-1 may perform playback operation 350 at corrected future time 348. For example, the interface circuit 318 or the processor 338 executing software may perform the playback operation 350. In particular, the packet and/or additional packets may include audio data 328 in the payload, and speaker 118-1 may store the audio data 328 in a queue in memory 330. In these embodiments, playback operation 350 includes outputting audio data 328 from the queue, which includes driving an electroacoustic transducer in speaker 118-1 based on audio data 328 so that speaker 118-1 outputs sound. It should be noted that adjusting 334 the interface clock 324 and modifying the future time 344 may coordinate the playback operation 350 in the clock domain of the interface clock 324 to within a predetermined value of the clock domain of the interface clock 316.

As previously described, the roles of the clock master and slave in the coordination technique may be reversed. This is illustrated in fig. 4, which presents a flow diagram illustrating an example of a method 400 for coordinating playback operations. Method 400 may be performed by a second interface circuit in a second electronic device (which may be a slave device), such as a/V hub 112 (fig. 1). During operation, the second interface circuit may receive a packet from an electronic device (which may be a slave device) via wireless communication (operation 410), where a given packet includes a transmit time based on a clock when an electronic device of the electronic devices transmits the given packet. It should be noted that the transmission time may be included in the payload and/or MAC header in a given packet. In some embodiments, the packet comprises a control packet. Alternatively or additionally, the packet may comprise a data packet.

In response to receiving the packet, the second interface circuit may store a receive time when the packet was received (operation 412), where the receive time is based on a second clock in the second electronic device. It should be noted that the transmission time may correspond to the leading edge or the trailing edge of the packet. Similarly, the reception time may correspond to the leading edge or the trailing edge of the packet.

The second interface circuit may then calculate a relative drift as a function of the second clock and the time between the clocks based on the difference between the transmit time and the receive time (operation 414), and may adjust a second clock circuit providing the second clock (such as a second interface clock circuit in or associated with the second interface circuit) based on the relative drift to eliminate the relative drift (operation 416). For example, the adjustment may be based on a difference of consecutive packets, and the adjustment may frequency lock the clock and the second clock.

In addition, the second interface circuit may determine a second clock and a remaining time offset between the clocks (operation 418).

Further, the second interface circuit may modify a future time at which the electronic device is to perform the playback operation based on the remaining time offset (operation 420) to determine a corrected future time.

Next, the second interface circuit may transmit information specifying the corrected future time to the electronic device via wireless communication (operation 422).

In some embodiments, the second interface circuit (and/or the second electronic device) optionally performs one or more additional operations (operation 424). For example, the transmission time and the reception time may be stored at opposite ends of the payload of a given packet. In these embodiments, a second interface circuit or processor executing software in the second electronic device may determine the duration of the payload, and the second interface circuit may add the duration to the remaining offset time.

Further, prior to transmitting the information, concurrently with transmitting the information and/or after transmitting the information (operation 422), the second interface circuit may transmit additional packets including the audio data in the payload, and the playback operation may include outputting the audio data. It should be noted (however, that in some embodiments, at least some of the audio data is included in the same packet as the information.) that adjusting the second clock (operation 416) and modifying the future time (operation 420) may coordinate playback operations in the clock domain of the second clock to within a predetermined value of the clock domain of the clock.

Fig. 5 presents a diagram illustrating an example of communication between a/V hub 112 and speakers 118-1. In particular, interface circuitry 320 in speaker 118-1 may transmit one or more packets (e.g., packet 510) to a/V hub 112. Each packet may include a corresponding transmit time 512 based on an interface clock 324 provided by interface clock circuit 326 in interface circuit 320 in speaker 118-1 or interface clock circuit 326 associated with interface circuit 320 when speaker 118-1 transmits packet 510. When interface circuit 310 in a/V hub 112 receives a packet, interface circuit 310 may include a receive time in the packet (or interface circuit 310 may store the receive time in memory), where for each packet, the corresponding receive time 514 may be based on an interface clock 316 (which is sometimes referred to as an "interface clock") provided by interface clock circuit 318 in interface circuit 310 or interface clock circuit 318 associated with interface circuit 310.

Interface circuit 310 may then calculate a relative drift 516 as a function of time between interface clock 316 and interface clock 324 based on the difference between the transmit time and the receive time, and may adjust 518 interface clock circuit 318 based on the relative drift to eliminate relative drift 516. In addition, interface circuit 310 may determine a remaining time offset 520 between interface clock 316 and interface clock 324.

In some embodiments, the transmission time and the reception time may be stored at opposite ends of the payload of the packet. In these embodiments, the interface circuit 310 or the processor 522 executing software in the a/V hub 112 may determine a duration 526 or time associated with the length 524 of the payload, and the interface circuit 310 may add the duration 526 to the remaining offset time 520.

Further, interface circuit 310 may modify a future time 528 at which speaker 118-1 is to perform playback operation 350 based on remaining time offset 520 to determine a corrected future time 530. Next, interface circuit 310 may transmit one or more packets 532 that include information specifying a corrected future time 530. Further, the packet 532 may include audio data 328 in the payload.

After interface circuit 320 receives packet 532, speaker 118-1 may store audio data 328 in a queue in memory 330, and speaker 118-1 may perform playback operation 350 at corrected future time 530. For example, the interface circuit 320 or a processor executing software may perform the playback operation 350. In these embodiments, playback operation 350 includes outputting audio data 328 from the queue, which includes driving an electroacoustic transducer in speaker 118-1 based on audio data 328 so that speaker 118-1 outputs sound. It should be noted that adjusting 518 the interface clock 316 and modifying the future time 528 may coordinate the playback operation 350 in the clock domain of the interface clock 316 to within a predetermined value of the clock domain of the interface clock 324.

In an exemplary embodiment, a coordination technique is used to provide channel coordination and phasing for surround sound or multi-channel sound. In particular, some people can perceive a 5 μ s change in playback coordination, which can produce an audible blurring effect. Furthermore, if the relative clock drift is large enough, audible chatter may occur between clock adjustments. Furthermore, global playback coordination between the speakers and the headphones (or headphones) may be required to avoid jumps or echoes that may degrade the user experience. Thus, the coordination technique may require maintaining the playback coordination of two or more speakers within, for example, 1-5 μ s.

To enable such coordination capabilities, in some embodiments, the coordination techniques may include transmit time information in packets transmitted by an interface circuit (i.e., in the physical layer), such as an interface circuit in an a/V hub (which may serve as an access point in a wireless local area network), or an audio receiver (and, more generally, a receiver) that provides data packets to one or more speakers in the system. In particular, the a/V hub may include a transmit timestamp in each User Datagram Protocol (UDP) data packet, such as in the payload. Thus, in some embodiments, coordination may not use access point beacons or dedicated packets. Further, the communication of coordination information may be unidirectional, such as from the A/V hub to the speakers or from the speakers to the A/V hub (as opposed to back-and-forth or bidirectional communication).

It should be noted that the timestamp may include a counter value corresponding to an interface clock in the interface circuitry in the a/V hub or an interface clock associated with the interface circuitry in the a/V hub. In some embodiments, the counter value is a high resolution, such as, for example, 32B. E.g. counter value or time stamp and integrated inter-IC sound bus (I)²S) are associated.

When the interface circuit in the recipient receives a packet from the a/V hub, the interface circuit may append the receive time to the payload in the data packet. For example, the receive time may include a counter value corresponding to an interface clock in or associated with an interface circuit in the receiver. In some embodiments, there may be 24B in the data packet for storing timing information, such as 4B at the beginning of the payload for the transmit time stored at the a/V hub and 4B at the end of the payload for the receive time stored at the receiver.

Then, using the transmit time (which may provide information about the master time base) and the receive time from the plurality of packets, the interface circuitry may track and correct for drift between the clocks in the interface circuitry in the a/V hub and the receiver, and may determine the remaining time offset. Next, the interface circuit may use the remaining time offset to modify the future time based on the remaining time offset to determine a corrected future time when the recipient performs a playback operation (e.g., playback of audio data included in the data packet).

It should be noted that in some embodiments, the transmit time and the receive time are included when data packets are transmitted and received, respectively, during the test mode of the interface circuits in the a/V hub and the receiver. The test mode may be set or selected by software executed by a processor in the a/V hub and/or the recipient.

Fig. 6 presents a block diagram illustrating an example of an electronic device 600. In the description that follows, the electronic device 600 may include the speaker 118-1 of FIG. 1. However, in other embodiments, the flow of coordination information is reversed, and the electronic device 600 includes the A/V hub 112 of FIG. 1.

The electronic device 600 may include: a Wi-Fi interface circuit 610 (which is an example of an interface circuit), a system on a chip (SOC)612, and a Control Circuit (CC)614 (e.g., a programmable logic device or a field programmable logic array). In addition, the electronic device 600 may include I²S-circuit 608, which samples the audio data in a/V hub 112 (fig. 1) and plays back the audio data in speaker 118-1 (fig. 1). To provide a high quality listening experience, I in the slave A/V hub 112 (FIG. 1) may be paired²Example of S-circuit 608 to I in speaker 118-1 (FIG. 1)²The timing of the instances of the S-circuit 608 are coordinated or the relative timing differences may be tracked and used to correct future playback times. In some embodiments, I²Coordination between the S-circuit 608 and the interface circuit 610 is maintained within the electronic device (such as by adjusting I)²S clock). The instances of interface circuit 610 are then frequency locked using a coordination technique.

In particular, packets 626 of audio data may be assembled and stored in a memory 624 in the SOC 612. The packet may include a space 628 to be used to store the transmission of Wi-Fi packets (which include packet 626)Time and reception time. It should be noted that packet 626 may include an I²S clock associated software time stamps, such as when SOC612 is based on I²The S clock samples the audio data.

The packet 626 may be provided to the interface circuit 610 via the host interface 630. The MAC layer 632 may then assemble MAC Protocol Data Units (MPDUs), which are stored in a first-in-first-out 634 buffer. The MPDUs may be provided to a Physical (PHY) layer 636, which assembles a Physical Layer Convergence Protocol (PLCP) protocol data unit (PPDU) into frames. Next, the frame may be sent by one of the radios 628.

During transmission, interface circuit 610 may wait for the shared communication channel to be available. When a communication channel is available, a current interface clock timestamp may be obtained from interface circuitry 610, stored in hardware in interface circuitry 610, and added to packet 626. For example, the transmission time may be added by the MAC layer 632 or the PHY layer 636.

After transmitting the frame including packet 626, interface circuit 610 may wait for an Acknowledgement (ACK). After a predefined time interval without an ACK, interface circuit 610 may repeat the process from waiting for the shared communication channel to be available. Thus, a modified transmit time may be included in packet 626 before each attempt to transmit a frame including packet 626.

If a timeout occurs or the number of retries is too large, the interface circuit 610 may signal a transmit failure to the SOC 612. Alternatively, if an ACK is received, the interface circuit 610 may provide a signal completion to the SOC612 and the transmit time stored in the interface circuit 610 may be provided to the SOC 612.

Similarly, when interface circuit 610 receives a frame, a receive timestamp may be added to an instance of packet 626 by PHY layer 636 or MAC layer 632. As previously described, the receive time may be associated with the leading or trailing edge of a frame. In particular, the receive time may be based on the receive time 640 (which is associated with the leading edge) or the receive clear 642 (which is associated with the trailing edge).

As further described below with reference to fig. 9-12Depicted, when reset occurs, interface circuit 610 may provide reset 616 to CC 614. In addition, CC 614 may reduce media independent interface (RMI I)620 and unidirectional I through bi-directional²S622 is coupled to SOC 612. In some embodiments, interface clock 618 has a fundamental frequency of approximately 24.576MHz (which is sometimes referred to as a "25 MHz clock"), and the I in or associated with CC 614²The clock in S622 has a sampling frequency between 44-192 kHz.

Fig. 7 presents a block diagram illustrating an example of CC 614 in fig. 1. In CC 614, clock circuit 710 may generate interface clock 618 using FLL 712 based on a reference clock 716 provided by an oscillator 714 (e.g., a crystal oscillator). Further, as shown, FLL 712 may include: a phase-locked loop (PLL)718 that effectively multiplies the reference clock 716 by N (such as 12, and more generally, an integer); and an accumulator 720-1, which effectively divides the output from PLL 718 by M (such as 16, and more generally, an integer that is the same or different than N). It should be noted that the accumulator 720-1 may be based on the seed 722-1 (such as hexadecimal 1AAA) divided by M. In some embodiments, accumulator 720-1 is included in the synthesizer, and FLL 712 generates interface clock 618 using direct digital synthesis.

In addition, control logic 724 in electronic device 600 (such as in CC 614 or interface circuit 610 in fig. 6) may adjust the fundamental frequency of interface clock 618 by adjusting seed 722-1. To achieve under-relaxation of the system, the adjustment to the seed 722-1 may be limited to incrementing or decrementing by one bit for each data packet.

Based on the relative drift between the a/V hub and the clock in the recipient (e.g., speaker 118-1 in fig. 1), the control logic 724 may adjust the seed 722-1 to eliminate the relative drift. For example, the relative drift as a function of time may be calculated by the control logic 724, and the control logic 724 may adjust the seed 722-1 to change the clock speed based on the relative drift. This may adjust the slope of the clock frequency versus time to lock the interface clock 618 frequency to the corresponding interface clock in the a/V hub (and, thus, to zero the relative drift). In some embodiments, the adjustment is based on an average or low-pass filtered value of the relative drift (which is determined using coordination information included in a plurality of data packets, such as data packets received within 0.1-1 ms), which may also provide under-relaxation. Alternatively or additionally, adjustments may be applied to clock circuit 710 at a reduced rate of adjustment (such as, for example, every millisecond).

In addition, control logic 724 may determine a remaining (DC) Time Offset (TO) between interface clock 618 and a corresponding interface clock in the a/V hub, which may be stored in a register 726 in electronic device 600 (fig. 6), such as in CC 614 or interface circuit 610 (fig. 6). It should be noted that the remaining time offset may be quasi-static. Further, electronic device 600 in fig. 6 (such as software executed by a processor in electronic device 600) may determine a duration of the data packet, which is then added to the remaining time offset by control logic 724. This may be necessary when the transmission time and the reception time are at opposite ends of the payload in the data packet.

In some embodiments, 8-16ms, for example, is required to frequency lock interface clock 618 and the corresponding interface clock in the A/V hub. To accommodate this, as well as the variable latency associated with the software executing on the electronic device 600 in fig. 6, the audio data in the data packet (which arrives based on the interface clock 618) may be stored in the queue 730. The queue may store, for example, up to 8 milliseconds of audio data. In addition, CC 614 may include a reorder buffer 728 that may reorder audio data arriving out of order from the a/V hub.

When interface circuit 610 (fig. 6) receives a future time at which electronic device 600 (fig. 6) is to perform a playback operation, control logic 724 may determine a corrected future time based on the received future time and the remaining time offset. Next, at the corrected future time, control logic 724 may perform a playback operation, such as outputting the audio data in queue 730 to Audio Integrated Circuit (AIC)732 and then to a speaker (not shown). In this way, all playback times of different recipients can be coordinated.

Fig. 8 presents a timing diagram illustrating an example of a fundamental frequency 810 of a clock 814 in the electronic device in fig. 1 (such as interface clocks in the a/V hub and the receiver). After the coordination interval 816, the receiving party may cancel or cancel out the relative drift 818 of the fundamental frequency 810 as a function of time 812. The future time 820 at which the recipient is to perform the playback operation may then be corrected by the remaining time offset 822 to determine a corrected future time 824 in order to coordinate the playback operation between the a/V hub and the recipient.

Although the foregoing description illustrates the use of transmit and receive times during a test mode of the interface circuit, in other embodiments, the transmit and receive times are associated with wireless ranging capabilities of the interface circuit. For example, when the a/V hub and the recipient are in the same room, the time of flight or time delay associated with the distance between the a/V hub and the recipient may be ignored or included in the remaining time offset. In some embodiments, wireless ranging is used to determine and correct the time delay associated with the distance between the a/V hub and the recipient when the a/V hub and recipient are in different rooms (such as more than 10-30m apart). In these embodiments, wireless ranging may be used in both directions of communication between the a/V hub and the recipient. Alternatively or additionally, the coordination of the a/V hub and the recipient may include or use a coordination technique such as network time protocol (NAP) when the a/V hub and the recipient are in different rooms. In some embodiments, the transmit time and/or the receive time are determined based on one or more external clocks provided to electronic device 600 (fig. 6), rather than using one or more clocks generated within electronic device 600 (fig. 6).

Coordinating playback operations after wireless reset

In some embodiments, the coordination technique is robust even when a wireless reset occurs, such as when communication performance degrades (e.g., due to interference) and either or both interface circuits on opposite sides of the link are reset to a default state or default link parameters, including resetting a counter associated with an interface clock circuit in or associated with the interface circuit. This is illustrated in fig. 9, which presents a flow chart illustrating an example of a method 900 for reducing drift. Method 900 may be performed by interface circuitry in an electronic device (which may be a slave device), such as one of a/V display devices 114 (fig. 1) or one of speakers 118 (fig. 1).

During operation, the interface circuit may receive a packet via wireless communication from a second electronic device (which may be a master device) (operation 210), where the given packet includes a transmit time based on a second clock in the second electronic device at the time the second electronic device transmits the given packet.

In response to receiving the packet, the interface circuit may store a receive time when the packet was received (operation 212), where the receive time is based on a clock in the electronic device.

The interface circuit may then calculate a relative drift as a function of time between the clock and the second clock based on the difference between the transmit time and the receive time (operation 214), and may adjust a clock circuit providing the clock (such as an interface clock circuit in or associated with the interface circuit) based on the relative drift to eliminate the relative drift (operation 216).

Further, the interface circuit may store an adjustment to the clock circuit (operation 910).

Further, when the interface circuit or a second interface circuit in the second electronic device is reset (operation 912), the interface circuit may adapt the clock circuit based on the stored adjustment (operation 914) to reduce the relative drift while the interface circuit recovers frequency lock with the second clock based on additional packets with additional transmit times received from the second electronic device.

Fig. 10 presents a diagram illustrating an example of communication between a/V hub 112 and speakers 118-1. In particular, interface circuitry 310 in a/V hub 112 may transmit packets (such as packet 312) to speaker 118-1. Each packet may include a corresponding transmit time 314 based on an interface clock 316 provided by interface clock circuit 318 in interface circuit 310 in a/V hub 112 or an interface clock circuit 318 associated with interface circuit 310 in a/V hub 112 when a/V hub 112 transmits packet 312. When interface circuit 320 in speaker 118-1 receives packet 312, interface circuit 320 may include a receive time 322 in packet 312 (or interface circuit 320 may store receive time 322 in memory 330), where for each packet, the corresponding receive time may be based on an interface clock 324 provided by interface clock circuit 326 in interface circuit 320 or interface clock circuit 326 associated with interface circuit 320.

The interface circuit 318 may then calculate a relative drift 332 as a function of time between the interface clock 316 and the interface clock 324 based on the difference between the transmit time and the receive time, and may adjust 334 the interface clock circuit 320 based on the relative drift 332 to eliminate the relative drift 332. In addition, interface circuit 318 may store adjustments 1010 to interface clock circuit 326 in memory 330.

Further, when interface circuit 310 and/or interface circuit 318 are reset 1012 (which may be transmitted by interface circuit 310 via packets 1014), interface circuit 318 may adapt 1016 interface clock circuit 320 based on stored adjustment 1010 to reduce relative drift 334 while interface circuit 320 resumes frequency lock with interface clock 316 based on additional packets 1018 having transmission times 1020 and corresponding reception times 1022 received by interface circuit 320 from interface circuit 310.

In an exemplary embodiment, due to variations in wireless communication between the a/V hub and the recipient, the radios or interface circuits in the a/V hub and/or the recipient are routinely and unpredictably reset. Referring back to fig. 6, during a wireless reset, CC 614 may receive a reset 616 from interface circuit 610. In addition to resetting the counter in interface circuit 610 during a wireless reset, reset 616 may reset the counter in clock circuit 710 (fig. 7). In this manner, electronic device 600 may internally coordinate and, upon a driver reset in interface circuit 610, may begin to determine and/or apply adjustments to clock circuit 710 (fig. 7) upon receipt of the next data packet (i.e., when the next transmit time/receive time pair is available). For example, when a driver in the interface circuit 610 resets, the counter may increment or decrement immediately upon receipt of the next data packet.

However, it may take, for example, 8-16ms to restore frequency lock between the A/V hub and the receiver. During this time interval, the clocks in the A/V hub and the receiver may drift, for example, by up to 50-60 μ s. Some users may perceive this relative drift when the audio output by the speakers is jittered.

To address this challenge, adjustments applied to the clock circuit during the coordination technique may be stored. Then, in the case of a wireless reset, the stored adjusted mean or variance during a previous time interval (such as the previous 8-16ms) may be applied to the clock circuit to reduce or eliminate relative drift in frequency lock recovery. This is illustrated in fig. 11, where fig. 11 presents a block diagram illustrating an example of clock circuit 1110 in electronic device 600 (fig. 6). In particular, clock circuit 1100 may include an optional accumulator 720-2 that tracks adjustments to accumulator 720-1 during the coordination technique. Further, the tracked adjustments may be stored in a register or memory 1112.

Then, when a wireless reset occurs, the control logic 724 may use the stored adjustment or the adjusted average or mean to adapt the clock circuit 1110 to reduce, constrain, or ideally eliminate the relative drift until frequency lock is restored. For example, the stored adjustments may be averaged over a time scale corresponding to the time interval required to restore frequency lock.

Using this method, the relative drift may be less than 2-8 μ s until frequency lock is restored. In some embodiments, the stored adjustments are averaged or low pass filtered over a time interval (e.g., 16 ms). Alternatively or additionally, an adjustment may be applied to the seed 722-1 of the accumulator 720-1 at a reduced update frequency or rate to provide damping or under-relaxation of the system. Thus, the adjustment may be performed over a longer time scale than the adjustment (such as every millisecond), and more generally, at a period that is a fraction of the time interval required to restore frequency lock.

Once frequency lock is restored, the remaining time offset may be determined by control logic 724 so that the phases of the playback operation may be coordinated.

Fig. 12 presents an example illustrating a clock 1210 in the electronic device of fig. 1 as a function of time after a wireless reset. Specifically, when a reset 1210 occurs, the stored adjustment may be applied to limit 1212 relative drift 1214 as a function of time 812 until frequency lock 1216 is restored.

Alternatively or in addition to the above-described method, in some embodiments, clock circuit 1100 includes an optional sample and hold circuit 1112 that mirrors the current counter value of the counter for interface clock 618 when interface circuit 610 (fig. 6) and/or an interface circuit in the a/V hub is reset. This counter value can be used to limit the relative drift when frequency lock is restored.

Coordinating playback operations using a processor

Instead of performing the coordination techniques in hardware (such as interface circuitry), in some embodiments the coordination techniques are performed at least in part by a processor executing software. In particular, the coordination techniques may include coordinating playback operations in different clock domains in the electronic device. This is illustrated in fig. 13, which presents a flow diagram illustrating an example of a method 1300 for coordinating playback operations. Method 1300 may be performed by a processor executing software, such as program modules, in an electronic apparatus, such as one of a/V display devices 114 (fig. 1) or one of speakers 118 (fig. 1), which may be a slave device. The processor may include control circuitry or control logic. It should be noted that the instructions for operation in the software may be stored in a memory of the electronic device.

During operation, the processor may maintain coordination (e.g., by adjusting a system clock) between a system clock provided by a system clock circuit in the electronic device and an interface clock provided by an interface clock circuit in the electronic device, where the interface clock has a higher frequency than the system clock (operation 1310).

The processor may then calculate a relative drift as a function of time between the interface clock and a second interface clock in the second electronic device based on a difference between a receive time when the interface circuit receives a packet from the second electronic device (which may be the master device) via wireless communication and a transmit time when a packet is transmitted by a second electronic device included in the packet, wherein the given packet includes a transmit time based on the second interface clock when the second electronic device transmits the given packet (operation 1312). It should be noted that the transmission time may correspond to the leading edge or the trailing edge of the packet. Similarly, the reception time may correspond to the leading edge or the trailing edge of the packet. The adjustment may be based on the difference of consecutive packets.

Further, the processor may adjust the system clock circuit based on the relative drift to eliminate the relative drift (operation 1314), wherein providing a higher resolution than the system clock based on the relative drift of the interface clock. For example, while coordination between the interface clock and the system clock may be highly accurate, the output registers that read the current value of the system clock may not have the same precision. Thus, the interface clock may be used to determine the relative drift, and as described below, the remaining offset. It should be noted that the adjustment may frequency lock the interface circuit and the second interface circuit.

Next, the processor may determine a remaining time offset between the interface clock and the second interface clock (operation 1316).

Further, the processor may modify a future time at which the electronic device is to perform the playback operation received from the second electronic device based on the remaining time offset (operation 1318) to determine a corrected future time.

Additionally, the processor may perform a playback operation at the corrected future time (operation 1320). In particular, the packet may include audio data in a payload, and the electronic device may store the audio data in a queue. In these embodiments, the playback operation may include outputting audio data from the queue. It should be noted that adjusting the system clock (operation 1314) and modifying the future time (operation 1318) may coordinate playback operations in the clock domain of the interface clock to within a predetermined value of the clock domain of the second interface clock.

In some embodiments, the processor optionally performs one or more additional operations (operation 1322). For example, prior to performing the playback operation (operation 1320), the processor may: disabling interrupts in the electronic device; and occupying at least a portion of the software stack by executing the loop to reduce latency associated with performing the playback operation.

Further, the transmission time and the reception time may be stored at opposite ends of the payload of a given packet. In these embodiments, the processor may determine the duration of the payload and may add the duration to the remaining offset time.

Further, when a reset of the interface circuit or a second interface circuit in the second electronic device occurs, the processor may mirror the interface clock by sampling and holding a counter value corresponding to the interface clock in a counter in the interface circuit before resetting the counter. In these embodiments, the mirrored counter value may allow the interface circuit to reduce the relative drift while the interface circuit resumes frequency lock with the second interface clock based on additional packets with additional transmit times received by the interface circuit from the second electronic device.

Fig. 14 presents a diagram illustrating an example of communication between a/V hub 112 and speakers 118-1. In particular, interface circuitry 310 in a/V hub 112 may transmit packets (e.g., packet 312) to speaker 118-1. Each packet may include a corresponding transmit time 314 based on an interface clock 316 provided by an interface clock circuit 318 in the interface circuit 310 in the a/V hub 112 or an interface clock circuit 318 associated with the interface circuit 310 in the a/V hub 112 when the a/V hub 112 transmits the packet 312. When interface circuit 320 in speaker 118-1 receives a packet, interface circuit 320 may store a receive time 322 in memory 330 (or it may include receive time 322 in packet 312), where for each packet, the corresponding receive time may be based on an interface clock 324 provided by interface clock circuit 326 in interface circuit 320 or interface clock circuit 326 associated with interface circuit 320.

The processor 338 may maintain coordination 1410 between the system clock 1412 provided by the system clock circuit 1414 in the speaker 118-1 and the interface clock 324 (e.g., by adjusting the system clock 1412), where the interface clock 324 has a higher frequency than the system clock 1412.

Processor 338 may then calculate a relative drift 1416 as a function of time between interface clock 324 and interface clock 316 based on the difference between the receive time and the transmit time.

Further, the processor 338 may adjust 1418 the system clock circuit 1414 to eliminate the relative drift 1416 based on the relative drift 1416, wherein the relative drift 1416 based on the interface clock 324 provides a higher resolution than the system clock 1412. Next, processor 338 may determine a remaining time offset 1420 between interface clock 324 and interface clock 316. For example, while coordination between interface clock 324 and system clock 1412 may be highly accurate, output registers that read the current value of system clock 1412 may not have the same accuracy. Thus, the interface clock 324 may be used to determine the relative drift, and as described below, the remaining offset.

In some embodiments, the transmission time and the reception time may be stored at opposite ends of the payload of the packet. In these embodiments, the processor 338 may determine a duration 1424 or time associated with the length 1422 of the payload, and the processor 338 may add the duration 1424 to the remaining offset time 1420.

In addition, interface circuit 310 may transmit packet 346 that includes information specifying future time 344 at which speaker 118-1 is to perform playback operation 350. After the interface circuit 320 receives the packet 346, the processor 338 may modify the future time 344 based on the remaining time offset 1420 to determine a corrected future time 1426.

Additionally, the processor 338 may perform the playback operation 350 at the corrected future time 1426. In particular, the packet and/or additional packets may include the audio data 328 in the payload, and the processor 338 may store the audio data 328 in a queue in the memory 330. In these embodiments, playback operation 350 may include outputting audio data 328 from the queue, which includes driving an electroacoustic transducer in speaker 118-1 based on audio data 328 so that speaker 118-1 outputs sound. It should be noted that adjustment 1418 of system clock 1412 and modification of future time 344 may coordinate playback operation 350 in the clock domain of interface clock 324 to within predetermined values of the clock domain of interface clock 316.

In some embodiments, prior to performing the playback operation 350, the processor 338: disabling interrupts in speaker 118-1; and occupies at least a portion of the software stack by executing a loop to reduce latency associated with performing playback operations 350.

Further, when interface circuit 310 and/or interface circuit 320 sends a reset, processor 338 may mirror interface clock 324, for example, by sampling and holding a counter value in a counter in interface circuit 320 that corresponds to interface clock 324, prior to resetting the counter. In these embodiments, the mirrored counter values may allow the interface circuit 320 to reduce the relative drift 1416 while the interface circuit 320 recovers frequency lock with the interface clock 316 (as previously described with reference to fig. 9-12) based on additional packets with transmit times received by the interface circuit 320 from the interface circuit 310.

In an exemplary embodiment, the coordination is performed at least in part by software/firmware executed by a processor. In particular, instead of hardware (such as interface circuitry) performing clock adjustment based on transmit and receive times (i.e., in-band time signaling rather than a separate side channel), software or firmware may reduce or accelerate I based on the difference between transmit and receive times (and more generally, coordination information)²An S-clock to lock the interface clocks in the A/V hub and the receiver at frequency. The processor may then determine a remaining time offset based on the transmit time and the receive time received in the data packet.

To avoid variable interrupt latency, when receiving future time and instructions to perform playback operations from the a/V hub, the processor in the recipient may: inhibit interrupts on a future basisThe time and remaining time offset determine a corrected future time and perform a busy cycle until the corrected future time. Then, based on I²S clock, the receiving side can perform the playback operation at the corrected future time.

Fig. 15 presents a block diagram illustrating an example of the electronic device 1500 in fig. 1. In the description that follows, the electronic device 1500 may include the speaker 118-1 of FIG. 1. However, in other embodiments, the coordination information flow is reversed, and the electronic device 1500 includes the A/V hub 112 of FIG. 1.

The electronic device 1500 may include: interface circuit 610, SOC612, CC 614, I²S-circuit 608, and processor 1512. I is²S-circuit 608 may be implemented by an inter-integrated circuit (I) that provides control information²C) The bus 1514 is coupled to the SOC 612. In addition, CC 614 (which may be optional in fig. 15) may generate and provide interface clock 1516, SOC clock 1518, and I²S clock 1520 (which is sometimes referred to as the "system clock"). Alternatively, interface clock 1516, SOC clock 1518, and/or I²S-clock 1520 may be provided by or based on an external oscillator (such as a crystal oscillator). In some embodiments, interface clock 1516 has a fundamental frequency of about 24.576MHz, and I²S1520 has a sampling frequency between 44-192 kHz.

In electronic device 1500, processor 1512 may use the relative drift (based on the transmit time and receive time from interface circuit 610) to adjust clock circuits in CC 614 so interface clock 1516 is frequency locked to a corresponding interface clock in the a/V hub, and (e.g., by adjusting I)²S clock 1520) maintain interface clocks 1516 and I²Coordination between S clocks 1520. It should be noted that interface clock 1516 may have a ratio of I²The higher resolution of the S-clock 1520, therefore, may improve the accuracy of the coordination in the electronic device 1500. For example, although interface clocks 1516 and I²Coordination between S clocks 1520 may be highly accurate, but reading I²The output registers for the current value of S clock 1520 may not have the same precision. Thus, interface clock 1516 may be used to perform the adjustment, and as suchAs described below, for determining the remaining offset.

Further, the processor 1512 may determine a remaining offset and may modify the future time based on the remaining time offset to determine a corrected future time. Then, after disabling the interrupt and performing the busy cycle, the processor 1512 may instruct the electronic device 1500 to perform the playback operation at the corrected future time.

In some embodiments, instead of using one or more clocks generated within the electronic device 1500 to determine the transmit time and/or the receive time, the transmit time and/or the receive time are based on one or more external clocks provided to the electronic device 1500.

Further, in the case of a wireless reset, I is due to the associated counters for interface clock 1516 and/or relative drift being reset²S clock 1520 may be lost. To address this challenge, CC 614 may mirror these counters. For example, the sample and hold operation may be performed at reset time. In particular, CC 614 may include one or more high resolution counters to mirror the counters used for interface circuitry 1516 and/or relative drift. For example, CC 614 may perform sampling and holding of counter values while asserting or toggling reset 616 (fig. 6).

Fig. 16 presents a flowchart illustrating an example of a method 1600 for coordinating playback operations, which may be performed by a processor executing software, such as program modules, in an electronic apparatus, such as one of a/V display devices 114 (fig. 1) or one of speakers 118 (fig. 1), which may be a slave apparatus. The processor may include control circuitry or control logic. It should be noted that the instructions for operation in the software may be stored in a memory of the electronic device.

During operation, the processor may capture timing information associated with an interface clock provided by an interface clock circuit in the electronic device or a reference clock used by the interface clock circuit to generate the interface clock to increase the resolution of the system clock (operation 1610), wherein the interface clock has a higher frequency than the system clock. For example, the capturing of timing information may include storing a time value of an interface clock in a register or counter. Alternatively or additionally, the electronic device may include an oscillator that provides a reference clock, and the interface clock circuit may provide the interface clock based on the reference clock. In these embodiments, timing information is captured from a reference clock.

The processor may then track a relative drift as a function of time between the system clock and the interface clock using the timing information (operation 1612), where the interface clock is coordinated with a second interface clock in the second electronic device based on time coordination information in a packet received from the second electronic device via the wireless communication. For example, the timing information may be included in a beacon transmitted by the second electronic device. Alternatively or additionally, the timing information may be included in a control packet and/or a data packet transmitted by the second electronic device.

Further, the processor may determine an estimated time offset between the interface clock and the system clock at a future time when the second electronic device is to perform the playback operation, received from the second electronic device, based on the relative drift (operation 1614).

Next, the processor may modify the future time based on the estimated time offset (operation 1616) to determine a corrected future time.

Further, the processor may perform a playback operation at the corrected future time (operation 1618). In particular, the interface circuit may receive additional packets including audio data in the payload. Alternatively or additionally, at least some of the audio data may be received in the same packets as the information. In these embodiments, the electronic device stores the audio data in a queue, and the playback operation includes outputting the audio data from the queue. It should be noted that the capturing (operation 1610), tracking (operation 1612), determining (operation 1614), and modifying (operation 1616) may coordinate the playback operation within a predetermined value of the clock domain of the second interface clock.

In some embodiments, the processor optionally performs one or more additional operations (operation 1620). For example, prior to performing the playback operation (operation 1618), the processor may: disabling interrupts in the electronic device; and occupying at least a portion of the software stack by executing the loop to reduce latency associated with performing the playback operation.

In some embodiments of method 200 (fig. 2), method 400 (fig. 4), method 900 (fig. 9), method 1300 (fig. 13), and/or method 1600 (fig. 16), there are more or fewer operations. For example, in method 400 (fig. 4), the second electronic device may resample audio data to facilitate coordinating playback operations. In addition, the order of the operations may be changed, and/or two or more operations may be combined into a single operation. Further, one or more operations may be modified. For example, operations performed by a second electronic device (such as a/V hub 112 in fig. 1) may be performed by an electronic device (such as speaker 118-1 in fig. 1) and/or vice versa. Additionally, instead of modifying the future time based on the remaining time offset, the electronic device may transmit the remaining time offset to the second electronic device, and the second electronic device may correct the future time for the remaining time offset (such as by subtracting the remaining time offset from the future time) before transmitting the modified future time to the second electronic device. Thus, in some embodiments, the second electronic device may pre-compensate the future time for the remaining time offset.

In some embodiments of method 200 (fig. 2), method 400 (fig. 4), method 900 (fig. 9), method 1300 (fig. 13), and/or method 1600 (fig. 16), the coordination includes synchronization in the time domain within time or phase accuracy and/or synchronization in the frequency domain within frequency accuracy.

Fig. 17 presents a diagram illustrating an example of communication between a/V hub 112 and speakers 118-1. In particular, interface circuit 310 in a/V hub 112 may send packet 1710 to speaker 118-1. Each packet may include time coordination information 1712 based on interface clock 316 provided by interface clock circuit 318 in interface circuit 310 in a/V hub 112 or an interface clock circuit 318 associated with interface circuit 310 in a/V hub 112. For example, the packet 1710 may include a beacon and the time coordination information (TSI)1712 may include a timing synchronization function.

After the interface 320 receives the packet 1710, the interface circuit 320 may coordinate the interface clock 324 provided by the interface clock circuit 326 in the interface circuit 320 or the interface clock circuit 326 associated with the interface circuit 320 based on the time coordination information 1712. This coordination may include frequency locking interface clock 324 to interface clock 316, or tracking the relative drift between interface clock 324 and interface clock 316 without frequency locking.

Processor 338 may capture timing information 1716 associated with interface clock 324 provided by interface clock circuit 326 or a reference clock used by interface clock circuit 326 to generate interface clock 324 to increase the resolution of system clock 1412 provided by system clock circuit 1414, where interface clock 324 has a higher frequency than system clock 1412.

Processor 1410 may then use timing information 1716 to track relative drift 1718 as a function of time between system clock 1412 and interface clock 324.

In addition, interface circuit 310 may transmit packet 346 that includes information specifying future time 344 at which speaker 118-1 is to perform playback operation 350. After the interface circuit 320 receives the packet 346, the processor 338 may determine an estimated time offset 1720 between the interface clock 324 and the system clock 1412 at the future time 344 based on the relative drift 1718. Next, the processor 338 may modify the future time 344 based on the estimated time offset 1720 to determine a corrected future time 1722.

Further, the processor 338 may perform the playback operation 350 at the corrected future time 1722. In particular, the interface circuit 310 may send a packet 1724 that may include the audio data 328 in the payload, and the processor 338 may store the audio data 328 in a queue in memory. In these embodiments, playback operation 350 may include outputting audio data 328 from the queue, which includes driving an electroacoustic transducer in speaker 118-1 based on audio data 328 so that speaker 118-1 outputs sound. It should be noted that the capturing, tracking, determining, and modifying may coordinate the playback operation 350 within predetermined values of the clock domain of the interface clock 316.

In some embodiments, prior to performing the playback operation 350, the processor 338: disabling interrupts in speaker 118-1; and occupies at least a portion of the software stack by executing a loop to reduce latency associated with performing playback operations 350.

Referring back to FIG. 15, based on I²The sample counter granularity of the 192kHz sample rate of the S-clock 1520 is 5.2 μ S. Therefore, the software cannot detect I²Whether the S-clock 1520 drifts until it drifts by at least one sample, which means that the coordination error of the playback operation is already 5.2 μ S. Furthermore, the coordination error is only worse for lower sampling rates. Thus, even though the hardware in the electronic device may have high precision, the software controlled PLL typically cannot determine beyond the threshold of I²Drift of capabilities defined by S-clock 1520.

Furthermore, ideally, I in different electronic devices²The S-clocks 1520 will start at the same time. However, even if the global clock domains are completed on the electronic device, they still need to coordinate to start at the same time. Alternatively, because there is usually no way to tell I²The S hardware starts at counter value + X (such as the corrected future time), so playback is typically specified by a bit in the start register that starts playback.

In principle, to address these challenges, in a coordination technique, the electronic device may perform a loop in which an interrupt is disabled when, for example, a system time counter is read until it reaches a start time. The electronics can then write to the start register to begin playback. However, due to I²The S hardware typically does not respond immediately, so the time to write to the start register may exceed 1 μ S.

Thus, in a coordination technique, various components (e.g., I) in an electronic device may be coordinated²The S-circuit may be in a slower clock domain, it may be buffered by an internal first-in-first-out buffer), and/or the processor may have an internal pipeline that needs to be flushed before the start instruction is executed in a coordinated manner. Furthermore, can be used for I²S hardware behavior analysis and adding an average or median delay to the corrected future time to correct or cancel I²S hardware delay.

In an exemplary embodiment, interface clock 1516 has a fundamental frequency of approximately 24.576 MHz. Further, processor 1512 may use time coordination information received from the a/V hub (such as a timing synchronization function in a beacon) to coordinate interface clock 1516 with a corresponding interface clock in the a/V hub. For example, interface clock 1516 may have a fixed fundamental frequency (such as 24.576MHz), and time coordination information may be used to track relative drift.

Since the update/read of time coordination information is expensive (in terms of overhead), SOC clock 1518 may be coordinated to interface clock 1516. In some embodiments, SOC clock 1518 has a fixed fundamental frequency.

SOC clock 1518 may be read to determine additional coordination information. In particular, I²S clock 1520 (which is sometimes referred to as a "system clock") may be coordinated with SOC clock 1518. However, because of I²The sampling frequency of S-clock 1520 may be between 44-192kHz, so a higher frequency (and higher resolution) SOC clock 1518 may be used in coordination techniques to improve I²Resolution of S-clock 1520.

Such software-based coordination techniques may allow the entire pipeline to be coordinated (or allow the relative drift associated with the entire pipeline to be determined), which includes (as previously described) a variable processing delay, so that playback operations may be coordinated within a predetermined value from the a/V hub to the recipient.

It should be noted that in the case of a wireless reset, the interface circuit 610, the SOC612, and I may be reset²All associated registers in S-circuit 608.

Alternatively or in addition to the foregoing methods, in some embodiments a high resolution counter is included in the clock circuit in CC 614 prior to dividing by M to generate or produce I²S clock 1520. This may allow for I to be used to coordinate playback operations within a predetermined value (and thus avoid or reduce jitter)²Direct sampling of S clock 1520. Depending on the clock frequency, this approach may reach an associated number of channels.

We now describe embodiments of an electronic device. Fig. 18 presents a block diagram illustrating an example of an electronic device 1800, such as one of the portable electronic devices 110, the a/V hub 112, the a/V display device 114, the receiver device 116, or the speaker 118 in fig. 1. The electronics include a processing subsystem 1810, a memory subsystem 1812, a network subsystem 1814, an optional feedback subsystem 1834, and a timing subsystem 1836. The processing subsystem 1810 includes one or more devices configured to perform computing operations. For example, processing subsystem 1810 may include one or more microprocessors, Application Specific Integrated Circuits (ASICs), microcontrollers, programmable logic devices, and/or one or more Digital Signal Processors (DSPs). One or more of these components in a processing subsystem are sometimes referred to as "control logic" or "control circuitry".

The memory subsystem 1812 includes one or more means for storing data and/or instructions for the processing subsystem 1810 and the network subsystem 1814. For example, the memory subsystem 1812 may include Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), and/or other types of memory. In some embodiments, instructions in memory subsystem 1812 for processing subsystem 1810 include: one or more program modules or sets of instructions (such as program modules 1822 or operating system 1824) that are executable by processing subsystem 1810. It should be noted that one or more computer programs or program modules may constitute a computer program mechanism. Further, instructions in various modules in the memory subsystem 1812 may be implemented as: a high-level programming language, an object-oriented programming language, and/or an assembly or machine language. Further, the programming language may be compiled or interpreted, e.g., configurable or configured (used interchangeably in this discussion), to be executed by the processing subsystem 1810.

Further, the memory subsystem 1812 may include circuitry or functionality to control access to memory. In some embodiments, the memory subsystem 1812 includes a memory hierarchy including one or more caches coupled to memory in the electronic device 1800. In some of these embodiments, one or more of the caches are located in the processing subsystem 1810.

In some embodiments, the memory subsystem 1812 is coupled to one or more high capacity mass storage devices (not shown). For example, the memory subsystem 1812 may be coupled to a magnetic or optical drive, solid state drive, or other type of mass storage device. In these embodiments, the memory subsystem 1812 may be used by the electronic device 1800 as fast-access memory for frequently used data, while mass storage is used to store less frequently used data.

The network subsystem 1814 includes one or more devices configured to couple to and communicate (i.e., perform network operations) over a wired and/or wireless network, including: control logic 1816, interface circuitry 1818, and associated antenna 1820. (although fig. 18 includes an antenna 1820, in some embodiments, the electronic device 1800 includes one or more nodes, such as node 1808, e.g., a pad, which may be coupled to the antenna 1820. thus, the electronic device 1800 may or may not include the antenna 1820.) for example, the network subsystem 1814 may include a bluetooth network system, a cellular network system (e.g., a 3G/4G network such as UMTS, LTE, etc.), a Universal Serial Bus (USB) network system, IEEE 802 based. 11 (e.g., Wi-Fi network system), ethernet network system, and/or other network systems. It should be noted that a combination of a given one of the interface circuits 1818 and at least one of the antennas 1820 may constitute a radio device. In some embodiments, the network subsystem 1814 includes a wired interface, such as an HDMI interface 1830.

The network subsystem 1814 includes a processor, controller, radio/antenna, jack/plug, and/or other means for coupling to, communicating over, and processing data and events for each supported network system. It should be noted that the components used to couple to, communicate on, and process data and events on each network system are sometimes collectively referred to as the "network interfaces" of the network system. Furthermore, in some embodiments, a "network" between electronic devices does not yet exist. Accordingly, the electronic device 1800 may use components in the network subsystem 1814 to perform simple wireless communication between electronic devices, e.g., transmitting advertisement or beacon frames and/or scanning for advertisement frames transmitted by other electronic devices as previously described.

Within the electronic device 1800, the processing subsystem 1810, the memory subsystem 1812, the network subsystem 1814, the optional feedback subsystem 1834, and the timing subsystem 1836 are coupled together using a bus 1828. Bus 1828 may include electrical, optical, and/or electro-optical connections that subsystems may use to transfer commands and data between each other. Although only one bus 1828 is shown for clarity, different embodiments can include different numbers or configurations of electrical, optical, and/or electro-optical connections between subsystems.

In some embodiments, the electronic device 1800 includes a display subsystem 1826 for displaying information (such as a request to clarify the identified environment) on a display, which may include a display driver, an I/O controller, and a display. It should be noted that a wide variety of display types may be used in display subsystem 1826, including: two-dimensional displays, three-dimensional displays (such as holographic displays or volumetric displays), head-mounted displays, retina-image projectors, head-up displays, cathode ray tubes, liquid crystal displays, projection displays, electroluminescent displays, electronic paper-based displays, thin film transistor displays, high performance addressing displays, organic light emitting diode displays, surface conduction electron emitter displays, laser displays, carbon nanotube displays, quantum dot displays, interferometric modulator displays, multi-touch screens (also sometimes referred to as touch-sensitive displays), and/or displays based on other types of display technologies or physical phenomena.

Further, optional feedback subsystem 1834 may include one or more sensor feedback components or devices, such as: a vibration device or vibration actuator (e.g., an eccentric rotating mass actuator or a linear resonant actuator), a light, one or more speakers, etc., which can be used to provide feedback (such as sensory feedback) to a user of the electronic device 1800. Alternatively or additionally, an optional feedback subsystem 1834 may be used to provide sensory input to the user. For example, one or more speakers may output sound, such as audio. It should be noted that one or more speakers may include an array of sensors that may be modified to adjust characteristics of the sound output by the one or more speakers. This capability may allow one or more speakers to modify sound in the environment to achieve a desired acoustic experience for the user, such as by changing the equalization or spectral content, phase, and/or direction of the propagating sound waves.

In addition, the timing subsystem 1836 may include one or more clock circuits 1838 for generating clocks in the electronic device 1800, such as based on one or more reference clocks.

The electronic device 1800 may be (or may be included in) any electronic device having at least one network interface. For example, the electronic device 1800 may be (or may be included in): desktop computers, laptop computers, sub-notebooks/netbooks, servers, tablet computers, smartphones, cellular phones, smart watches, consumer electronic devices (such as televisions, set-top boxes, audio devices, speakers, headsets, video devices, etc.), remote controls, portable computing devices, access points, routers, switches, communication devices, testing devices, and/or other electronic devices.

Although the electronic device 1800 is described using specific components, in alternative embodiments, different components and/or subsystems may be present in the electronic device 1800. For example, the electronic device 1800 may include one or more additional processing subsystems, memory subsystems, network subsystems, and/or display subsystems. Further, while one of the antennas 1820 is shown coupled to a given one of the interface circuits 1818, there may be multiple antennas coupled to a given one of the interface circuits 1818. For example, an example of a 3 x 3 radio may include three antennas. Additionally, one or more subsystems may not be present in the electronic device 1800. Further, in some embodiments, electronic device 1800 may include one or more additional subsystems not shown in fig. 18. Further, although separate subsystems are shown in fig. 18, in some embodiments, some or all of a given subsystem or component may be integrated into one or more of the other subsystems or components in the electronic device 1800. For example, in some embodiments, program modules 1822 are included in operating system 1824.

Further, the circuits and components in the electronic device 1800 can be implemented using any combination of analog and/or digital circuits, including: bipolar, PMOS and/or NMOS gates or transistors. Further, the signals in these embodiments may include digital signals having approximately discrete values and/or analog signals having continuous values. In addition, the components and circuits may be single ended or differential, and the power supply may be unipolar or bipolar.

The integrated circuit may implement some or all of the functionality of the network subsystem 1814 (such as one or more radios). Further, the integrated circuit may include hardware and/or software components for transmitting wireless signals from the electronic device 1800 and receiving signals at the electronic device 1800 from other electronic devices. Radios are generally known in the art, and therefore will not be described in detail, except for the components, circuits, or functions described herein. In general, the network subsystem 1814 and/or integrated circuit may include any number of radios.

In some embodiments, the network subsystem 1814 and/or integrated circuit includes configuration components (such as one or more hardware and/or software components) that configure the radio to transmit and/or receive on a given channel (e.g., a given carrier frequency). For example, in some embodiments, the configuration component may be used to switch the radio from monitoring and/or transmitting on a given channel to monitoring and/or transmitting on a different channel. (note that "monitoring" as used herein includes receiving signals from other electronic devices and possibly performing one or more processing operations on the received signals, e.g., determining whether the received signals include advertising frames, calculating performance metrics, performing spectral analysis, etc.). furthermore, network subsystem 1814 may include at least one port, such as HDMI port 1832, for receiving and/or providing information in a data stream to at least one a/V display device 114 (fig. 1), at least one speaker 118 (fig. 1), and/or at least one content source 120 (fig. 1).

Although a Wi-Fi compatible communication protocol is used as an illustrative example, the described embodiments may be used in a variety of network interfaces. For example, in some embodiments, the coordination technique is used with an ethernet communication protocol rather than a wireless communication protocol. In particular, the ethernet communication protocol may be used for room-to-room communication (i.e., communication over distances greater than 10 meters to 30 meters). In these embodiments, the Wi-Fi communication protocol can be used for in-room communication and playback coordination of multiple devices in a room, and the clocks used by the Wi-Fi interface circuitry and the Ethernet interface circuitry can be coordinated so that there is end-to-end coordination (i.e., from I in the content source)²I of S-circuits into receivers such as loudspeakers²S circuit). It should be noted that with room-to-room communication using the ethernet communication protocol, the coordination technique may be compatible with IEEE 802.11v so that the transmission time may be provided to the receiver after receiving the ACK.

Further, while some of the operations in the foregoing embodiments are implemented in hardware or software, in general, the operations in the foregoing embodiments may be implemented in a wide variety of configurations and architectures. Accordingly, some or all of the operations in the foregoing embodiments may be performed in hardware, software, or both. For example, at least some of the operations in the coordination techniques may be implemented using program modules 1822, an operating system 1824 (such as a driver for interface circuit 1818), and/or in firmware in interface circuit 1818. Alternatively or additionally, at least some operations in the coordination technique may be implemented in a physical layer (such as hardware in the interface circuit 1818).

Further, while the foregoing embodiments include a touch sensitive display in the portable electronic device that is touched by a user (e.g., with a finger or toe or stylus), in other embodiments, the user interface is displayed on the display of the portable electronic device and the user interacts with the user interface without contacting or contacting a surface of the display. For example, time-of-flight measurements, motion sensing (such as doppler measurements), or other non-contact measurements that allow for determination of the position, direction of motion, and/or velocity of the user's finger or toe (or stylus) relative to the position of the one or more virtual command icons may be used to determine the user's interaction with the user interface. In these embodiments, it should be noted that a user may activate a given virtual command icon by performing a gesture (such as "tapping" their finger in air without making contact with the surface of the display). In some embodiments, a user navigates through the user interface and/or activates/deactivates a function of one of the components in the system 100 (fig. 1) using spoken commands or instructions (i.e., via voice recognition) and/or based on where they look at one of the display or a/V display device 114 in the portable electronic device 110 in fig. 1 (e.g., by tracking the user's gaze or where the user is looking).

Further, although A/V hub 112 (FIG. 1) is shown as a separate component from A/V display device 114 (FIG. 1), in some embodiments, the A/V hub and A/V display device are combined into a single component or a single electronic device.

While the foregoing embodiments illustrate a coordination technique with audio and/or video content (such as HDMI content), in other embodiments, the coordination technique is used in the context of any type of data or information. For example, coordination techniques may be used with home automation data. In these embodiments, a/V hub 112 (fig. 1) may facilitate communication between and control of various electronic devices. Thus, a/V hub 112 (fig. 1) and coordination techniques may be used to facilitate or implement services in the so-called internet of things.

In the foregoing description, we have referred to "some embodiments". It should be noted that "some embodiments" describe a subset of all possible embodiments, but do not always specify the same subset of embodiments.

The previous description is presented to enable one of ordinary skill in the art to make and use the disclosure, and is provided in the context of a partial application and its requirements. Furthermore, the foregoing descriptions of embodiments of the present disclosure have been presented only for purposes of illustration and description. They are not intended to be exhaustive or to limit the disclosure to the forms disclosed. Accordingly, many modifications and variations will be apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of the disclosure. Additionally, the discussion of the preceding embodiments is not intended to limit the present disclosure. Thus, the present disclosure is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed herein.