阅读说明：本技术 使用自适应学习的波束管理 (Beam management using adaptive learning ) 是由 S·兰迪斯 A·图布尔 A·古贝斯基 A·Y·格洛科夫 A·D·坎得尔卡 于 2020-04-10 设计创作，主要内容包括：本公开的某些方面提供了用于使用自适应学习的波束管理的技术。某些方面提供了一种能由节点(诸如用户装备(UE)或基站(BS))执行的方法。该节点使用自适应学习来确定将用于波束管理规程的一个或多个波束。该节点使用所确定的一个或多个波束来执行波束管理规程。在一些方面,该节点使用自适应强化学习算法来选择用于波束发现规程中的测量的波束。该节点可基于与波束选择相关联的反馈,诸如基于使用在波束管理规程期间确定的波束配对来达成的吞吐量,来使波束管理算法自适应。(Certain aspects of the present disclosure provide techniques for beam management using adaptive learning. Certain aspects provide a method that can be performed by a node, such as a User Equipment (UE) or a Base Station (BS). The node uses adaptive learning to determine one or more beams to be used for a beam management procedure. The node performs a beam management procedure using the determined one or more beams. In some aspects, the node uses an adaptive reinforcement learning algorithm to select beams for measurement in a beam discovery procedure. The node may adapt the beam management algorithm based on feedback associated with beam selection, such as based on throughput achieved using beam pairings determined during a beam management procedure.)

1. A method for wireless communications by a node, comprising:

determining one or more beams to be used for a beam management procedure using adaptive learning; and

performing the beam management procedure using the determined one or more beams.

2. The method of claim 1, further comprising:

updating an adaptive learning algorithm for the adaptive learning based on feedback or training information or a combination thereof; and

another beam management procedure is performed using the updated adaptive learning algorithm.

3. The method of claim 2, wherein the feedback comprises feedback associated with the beam management procedure.

4. The method of claim 2, wherein the training information comprises:

training information obtained by deploying one or more User Equipments (UEs) in one or more simulated communication environments prior to network deployment of the one or more UEs;

training information obtained through feedback previously received while the one or more UEs are deployed in one or more communication environments;

training information from at least one of the network, one or more UEs, or a cloud; or

Training information received while the node is at least one of online or idle; or

Combinations thereof.

5. The method of claim 4, wherein the training information comprises training information received from one or more UEs different from the node after deployment of the node, wherein the training information comprises information associated with beam measurements made by the one or more UEs, or feedback associated with one or more beam management procedures performed by the one or more UEs, or a combination thereof.

6. The method of claim 4, wherein the node comprises a UE.

7. The method of claim 2, wherein:

using the adaptive learning algorithm comprises outputting an action based on one or more inputs;

the feedback is associated with the action; and is

Updating the adaptive learning algorithm based on the feedback includes adjusting one or more weights applied to the one or more inputs.

8. The method of claim 2, wherein the adaptive learning algorithm comprises an adaptive machine learning algorithm; a self-adaptive reinforcement learning algorithm; a self-adaptive deep learning algorithm; self-adaptive continuous infinite learning algorithm; or an adaptive policy-optimized reinforcement learning algorithm, or a combination thereof.

9. The method of claim 2, wherein the adaptive learning algorithm is modeled as a Partially Observable Markov Decision Process (POMDP).

10. The method of claim 2, wherein the adaptive learning algorithm is implemented by an artificial neural network.

11. The method of claim 10, wherein:

the artificial neural network comprises a Deep Q Network (DQN) comprising one or more Deep Neural Networks (DNNs); and is

Determining the one or more beams using the adaptive learning comprises:

passing one or more state parameters and one or more action parameters through the one or more DNNs;

for each state parameter, outputting the value of each action parameter; and

an action associated with the maximum output value is selected.

12. The method of claim 10, wherein updating the adaptive learning algorithm comprises adjusting one or more weights associated with one or more neuron connections in the artificial neural network.

13. The method of claim 1, wherein determining the one or more beams to be used for the beam management procedure using the adaptive learning comprises:

determining one or more beams to include in a codebook based on the adaptive learning; and

selecting one or more beams from the codebook to be used for the beam management procedure.

14. The method of claim 1, wherein determining the one or more beams to be used for the beam management procedure comprises using the adaptive learning to select one or more beams from a codebook to be used for the beam management procedure.

15. The method of claim 1, wherein the adaptive learning uses a state parameter associated with a channel measurement, a reward parameter associated with received signal throughput or spectral efficiency, and an action parameter associated with selection of a beam pair corresponding to the channel measurement.

16. The method of claim 15, wherein the channel measurements comprise Reference Signal Received Power (RSRP); spectral efficiency, channel flatness, or signal-to-noise ratio (SNR); or a combination thereof.

17. The method of claim 15, wherein the received signal comprises a Physical Downlink Shared Channel (PDSCH) transmission.

18. The method of claim 15, wherein the reward parameter is discounted by a penalty amount.

19. The method of claim 18, wherein the penalty amount depends on a number of one or more beams measured for the beam management procedure.

20. The method of claim 18, wherein the penalty amount depends on an amount of power consumption associated with the beam management procedure.

21. The method of claim 15, wherein the one or more beams are used for transmission, reception, or both of one or more Synchronization Signal Blocks (SSBs).

22. The method of claim 15, wherein performing the beam management procedure using the determined one or more beams comprises:

measuring a channel based on a Synchronization Signal Block (SSB) transmission from a Base Station (BS) using the determined one or more beams, the SSB transmission associated with one or more transmit beams of the BS; and

selecting one or more Beam Pair Links (BPLs), the one or more BPLs associated with one or more channel measurements above a channel measurement threshold; or one or more strongest channel measurements of all channel measurements associated with the SSB transmission; or a combination thereof.

23. The method of claim 22, wherein the determined one or more beams comprise a subset of available receive beams.

24. The method of claim 22, further comprising:

receiving a Physical Downlink Shared Channel (PDSCH) using one of the one or more BPLs;

determining a throughput associated with the PDSCH;

updating the adaptive learning algorithm based on the determined throughput; and

using the updated adaptive learning algorithm to determine another one or more beams to be used to perform another beam management procedure to select another one or more BPLs.

25. A node configured for wireless communication, comprising:

means for determining one or more beams to be used for a beam management procedure using adaptive learning; and

means for performing the beam management procedure using the determined one or more beams.

26. The node of claim 25, further comprising:

means for updating an adaptive learning algorithm for the adaptive learning based on feedback or training information, or a combination thereof; and

means for performing another beam management procedure using the updated adaptive learning algorithm.

27. The node of claim 25, wherein means for performing the beam management procedure using the determined one or more beams comprises:

means for measuring a channel based on a Synchronization Signal Block (SSB) transmission from a Base Station (BS) using the determined one or more beams, the SSB transmission associated with one or more transmit beams of the BS; and

means for selecting one or more Beam Pair Links (BPLs) that are measured with one or more channels above a channel measurement threshold; or one or more strongest channel measurements of all channel metrics associated with the SSB transmission; or a combination thereof.

28. The node of claim 27, further comprising:

means for receiving a Physical Downlink Shared Channel (PDSCH) using one of the one or more BPLs;

means for determining a throughput associated with the PDSCH;

means for updating the adaptive learning algorithm based on the determined throughput; and

means for determining, using the updated adaptive learning algorithm, another one or more beams to be used to perform another beam management procedure to select another one or more BPLs.

29. A node configured for wireless communication, comprising:

a memory; and

a processor coupled to the memory and configured to:

determining one or more beams to be used for a beam management procedure using adaptive learning; and

performing the beam management procedure using the determined one or more beams.

30. A non-transitory computer-readable medium storing computer-executable code, the computer-executable code comprising:

code for determining one or more beams to be used for a beam management procedure using adaptive learning; and

code for performing the beam management procedure using the determined one or more beams.

Disclosure of Invention

The systems, methods, and devices of the present disclosure each have several aspects, no single one of which is solely responsible for its desirable attributes. Without limiting the scope of the present disclosure as expressed by the claims which follow, some features will now be discussed briefly. After considering this discussion, and particularly after reading the section entitled "detailed description" one will understand how the features of this disclosure provide advantages that include an improved beam management procedure using adaptive learning.

Certain aspects provide a method for wireless communications by a node. The method generally includes determining one or more beams to be used for a beam management procedure using adaptive learning. The method generally includes performing a beam management procedure using the determined one or more beams.

In some examples, the node is a Base Station (BS).

In some examples, the node is a User Equipment (UE).

In some examples, the method includes updating an adaptive learning algorithm for adaptive learning. In some examples, the adaptive learning algorithm is updated based on feedback and/or training information. In some examples, the method includes performing another beam management procedure using the updated adaptive learning algorithm.

In some examples, the feedback includes feedback associated with a beam management procedure.

In some examples, the training information includes one or more of: training information obtained by deploying one or more UEs in one or more simulated communication environments prior to network deployment of the one or more UEs; training information obtained through feedback previously received while the one or more UEs are deployed in one or more communication environments; training information from a network, one or more UEs, and/or a cloud; and/or training information received while the node is online and/or idle.

In some examples, the training information includes training information received from one or more UEs different from the node after deployment of the node. In some examples, the training information includes information associated with a beam. In some examples, the training information includes measurements made by one or more UEs or feedback associated with one or more beam management procedures performed by the one or more UEs.

In some examples, using the adaptive learning algorithm includes outputting an action based on one or more inputs. In some examples, the feedback is associated with the action. In some examples, updating the adaptive learning algorithm based on the feedback includes adjusting one or more weights applied to the one or more inputs.

In some examples, the adaptive learning algorithm comprises an adaptive machine learning algorithm; a self-adaptive reinforcement learning algorithm; a self-adaptive deep learning algorithm; self-adaptive continuous infinite learning algorithm; and/or an adaptive strategy optimization reinforcement learning algorithm.

In some examples, the adaptive learning algorithm is modeled as a Partially Observable Markov Decision Process (POMDP).

In some examples, the adaptive learning algorithm is implemented by an artificial neural network.

In some examples, the artificial neural network comprises a Deep Q Network (DQN) comprising one or more Deep Neural Networks (DNNs). In some examples, determining one or more beams using adaptive learning includes communicating state parameters and action parameters through one or more DNNs; for each state parameter, outputting the value of each action parameter; and an act of selecting the associated maximum output value.

In some examples, updating the adaptive learning algorithm includes adjusting one or more weights associated with one or more neuron connections in the artificial neural network.

In some examples, determining one or more beams to be used for the beam management procedure using adaptive learning includes determining one or more beams to be included in a codebook based on the adaptive learning and selecting one or more beams to be used for the beam management procedure from the codebook.

In some examples, determining the one or more beams to be used for the beam management procedure includes using adaptive learning to select the one or more beams to be used for the beam management procedure from a codebook.

In some examples, the adaptive learning uses a state parameter associated with the channel measurements, a reward parameter associated with received signal throughput or spectral efficiency, and an action parameter associated with selection of a beam pair corresponding to the channel measurements.

In some examples, the channel measurements include Reference Signal Received Power (RSRP); spectral efficiency, channel flatness, and/or signal-to-noise ratio (SNR).

In some examples, the received signal comprises a Physical Downlink Shared Channel (PDSCH) transmission.

In some examples, the reward parameter is discounted by a penalty amount.

In some examples, the penalty amount depends on a number of one or more beams measured for the beam management procedure.

In some examples, the penalty amount depends on an amount of power consumption associated with the beam management procedure.

In some examples, the beams include one or more beams for transmission and/or reception of one or more Synchronization Signal Blocks (SSBs).

In some examples, performing the beam management procedure using the determined one or more beams includes measuring a channel based on an SSB transmission from the BS using the determined one or more beams, the SSB transmission associated with one or more transmit beams of the BS; and selecting one or more Beam Pair Links (BPLs) associated with one or more channel measurements above a channel measurement threshold and/or one or more strongest channel measurements of all channel measurements associated with the SSB transmission.

In some examples, the determined one or more beams comprise a subset of available receive beams.

In some examples, the method includes receiving the PDSCH using one of the one or more selected BPLs; determining a throughput associated with the PDSCH; updating an adaptive learning algorithm based on the determined throughput; and using the updated adaptive learning algorithm to determine another one or more beams to be used to perform another beam management procedure to select another one or more BPLs.

Certain aspects provide a node configured for wireless communication. The node generally includes means for determining one or more beams to be used for a beam management procedure using adaptive learning. The node generally includes means for performing a beam management procedure using the determined one or more beams.

Certain aspects provide a node configured for wireless communication. The node typically includes a memory. The node generally includes a processor coupled to a memory and configured to determine one or more beams to be used for a beam management procedure using adaptive learning. The processor and memory are generally configured to perform a beam management procedure using the determined one or more beams.

Certain aspects provide a computer-readable medium. The computer-readable medium typically stores computer-executable code. The computer-executable code generally includes code for determining one or more beams to be used for a beam management procedure using adaptive learning. The computer-executable code generally includes code for performing a beam management procedure using the determined one or more beams.

To the accomplishment of the foregoing and related ends, the one or more aspects comprise the features hereinafter fully described and particularly pointed out in the claims. The following description and the annexed drawings set forth in detail certain illustrative features of the one or more aspects. These features are indicative, however, of but a few of the various ways in which the principles of various aspects may be employed.

Brief Description of Drawings

So that the manner in which the above recited features of the present disclosure can be understood in detail, a more particular description of the disclosure, briefly summarized above, may be had by reference to aspects, some of which are illustrated in the appended drawings. It is to be noted, however, that the appended drawings illustrate only certain typical aspects of this disclosure and are therefore not to be considered limiting of its scope, for the description may admit to other equally effective aspects.

Fig. 1 is a block diagram conceptually illustrating an example telecommunications system, in accordance with certain aspects of the present disclosure.

Fig. 2 illustrates an example beam management procedure in accordance with certain aspects of the present disclosure.

Fig. 3 illustrates example Synchronization Signal Block (SSB) locations within an example semi-frame, in accordance with certain aspects of the present disclosure.

Fig. 4 illustrates example transmit and receive beams for SSB measurements, in accordance with certain aspects of the present disclosure.

Fig. 5 illustrates an example networking environment in which predictive models are used for beam management, in accordance with certain aspects of the present disclosure.

Figure 6 conceptually illustrates an example reinforcement learning model, in accordance with certain aspects of the present disclosure.

Fig. 7 conceptually illustrates an example deep Q-network (DQN) learning model, in accordance with certain aspects of the present disclosure.

Fig. 8 is a flow diagram illustrating example operations for wireless communications by a node in accordance with certain aspects of the present disclosure.

Fig. 9 is an example call flow diagram illustrating example signaling for beam management using adaptive learning, in accordance with certain aspects of the present disclosure.

Fig. 10 is an example call flow diagram illustrating example signaling for conducting a BPL discovery procedure using adaptive learning, in accordance with certain aspects of the present disclosure.

Fig. 11 illustrates a communication device that may include various components configured to perform operations for the techniques disclosed herein, in accordance with aspects of the present disclosure.

Fig. 12 is a block diagram conceptually illustrating a design of an example Base Station (BS) and User Equipment (UE), in accordance with certain aspects of the present disclosure.

To facilitate understanding, identical reference numerals have been used, where possible, to designate identical elements that are common to the figures. It is contemplated that elements disclosed in one aspect may be beneficially utilized on other aspects without specific recitation.

Detailed Description

Aspects of the present disclosure provide apparatuses (equipment), methods, processing systems, and computer-readable media for beam management using adaptive learning.

Some systems, such as new radio systems (e.g., 5G NR), support millimeter wave (mmW) communications. In mmW communication, a signal for device-to-device communication (referred to as a mmW signal) may have a high carrier frequency (e.g., within a 25GHz or higher frequency band, such as 30 to 300 GHz) and may have a wavelength in a range of 1mm to 10 mm. Based on such characteristics of mmW signals, mmW communication can provide high-speed (e.g., giga-speed) communication between devices. However, mmW signals may experience atmospheric effects and may not propagate well through materials compared to lower frequency signals. Thus, a mmW signal may experience relatively higher path loss as it propagates (e.g., attenuation or reduction in power density of a wave corresponding to the mmW signal) as compared to lower frequency signals.

To overcome path loss, mmW communication systems utilize directional beamforming. Beamforming may involve the use of Transmit (TX) beams and/or Receive (RX) beams. The TX beam corresponds to a transmitted mmW signal that is directed to have more power in a particular direction relative to other directions, such as toward a receiver. By directing the transmitted mmW signal to the receiver, more energy of the mmW signal is directed to the receiver, thereby overcoming the higher path loss. An RX beam corresponds to a technique performed at a receiver to apply a gain to a signal received in a particular direction while attenuating signals received in other directions. The use of RX beams also helps to overcome higher path loss, for example by improving the signal-to-noise ratio (SNR) of the desired mmW signal received at the receiver. In some aspects, hybrid beamforming (e.g., signal processing in the analog and digital domains) may be used.

Thus, in certain aspects, for a particular transmitter to communicate with a particular receiver, the transmitter needs to select a TX beam to use, and the receiver needs to select an RX beam to use. The TX and RX beams used for communication are referred to as beam pairs. In certain aspects, the RX and TX beams in a beam pair are selected such that sufficient communication coverage and/or capacity is provided.

In certain aspects, a beam pair may be selected (e.g., initial selection, updated selection, refinement to a narrower beam within a previously selected beam, etc.) using a beam management procedure. As will be discussed in more detail below with reference to fig. 2-4, a beam management procedure may involve measuring signals using different RX and/or TX beams for reception/transmission and selecting a beam for beam pairing based on the measurements. For example, the beam with the highest measured channel or link quality (e.g., throughput, SNR, etc.) among those measured beams may be selected.

In some cases, as discussed in more detail below with reference to fig. 2-4, there are a large number of RX and/or TX beams supported at the transmitter and/or receiver, which may mean that there are a large number of measurements that may be performed for beam management procedures. In addition, the communication environment between the transmitter and the receiver may be different at different times, such as due to obstructions (e.g., when a user's hand blocks a TX/RX beam at the transmitter/receiver (e.g., User Equipment (UE)), and/or objects block a line-of-sight (LOS) path between the transmitter and the receiver), movement and/or rotation of the transmitter/receiver, and so forth.

To account for such factors, in some cases, the beam management procedure is based on heuristics. Heuristic-based beam management procedures attempt to predict the true deployment scenario of the transmitter and receiver and typically update the beam management procedures used by the transmitter and receiver (such as using downloaded software patches) based on problems encountered (or expected) over time while the transmitter and receiver are communicating. For example, a heuristically based beam management procedure may only measure some RX and/or TX beams, but not all beams, of a transmitter and receiver based on parameters of the transmitter and/or receiver.

To further improve the beam management procedure, aspects of the present disclosure provide for using adaptive learning as part of the beam management procedure. For example, a UE (and/or BS) acting as a transmitter and/or receiver may use an adaptive learning-based beam management algorithm that is adaptive over time based on learning. In particular, the learning may be based on feedback associated with previous beam selections for the UE and/or the BS. The feedback may include an indication of a previous beam selection and parameters associated with the previous beam selection. The algorithm may be initially trained based on feedback in a laboratory environment and then updated (e.g., continuously) using feedback while the UE and/or BS are in deployment. In some examples, the algorithm is a deep reinforcement learning based beam management algorithm that uses machine learning and artificial neural networks to update and apply predictive models for beam selection during a beam management procedure. In this way, the adaptive learning based beam management algorithm learns from user behavior (e.g., frequently traversed paths, how the user holds the UE, etc.) and is therefore also personalized for the user.

The following description provides examples of using adaptive learning as part of a beam management procedure and is not intended to limit the scope, applicability, or examples set forth in the claims. Changes may be made in the function and arrangement of elements discussed without departing from the scope of the disclosure. Various examples may omit, substitute, or add various procedures or components as appropriate. For example, the described methods may be performed in an order different than described, and various steps may be added, omitted, or combined. Also, features described with reference to some examples may be combined in some other examples. For example, an apparatus may be implemented or a method may be practiced using any number of the aspects set forth herein. Moreover, the scope of the present disclosure is intended to cover such apparatus or methods as practiced using other structure, functionality, or structure and functionality in addition to or in addition to the various aspects of the present disclosure set forth herein. It should be understood that any aspect of the disclosure disclosed herein may be implemented by one or more elements of a claim. The word "exemplary" is used herein to mean "serving as an example, instance, or illustration. Any aspect described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other aspects.

Fig. 1 illustrates an example wireless communication network 100 in which aspects of the disclosure may be performed. For example, the wireless communication network 100 may be a new radio system (e.g., a 5G NR network). The wireless communication network 100 may support mmW communication through beamforming. Nodes (e.g., wireless nodes) in the wireless communication network 100, such as the UE 120a and/or the Base Station (BS)110a, may be configured to perform a beam management procedure to select a beam pair for communicating with another node. For example, UE 120a and BS 110a may perform a beam management procedure to determine to pair the receive beam of UE 120a and the transmit beam of BS 110a as a beam to be used for communication (e.g., downlink communication), also referred to as a Beam Pair Link (BPL). As will be described in greater detail herein, UE 120a and/or BS 110a may use adaptive learning based beam management procedures. The UE 120a and/or the BS 110a may use adaptive learning to determine one or more beams to be used for the beam management procedure. As shown in fig. 1, UE 120a has a beam selection manager 122. According to one or more aspects described herein, the beam selection manager 122 may be configured to determine/select beams to be used for a beam management procedure using an adaptive learning based algorithm. As shown in fig. 1, additionally or alternatively, BS 110a may have a beam selection manager 112. According to aspects described herein, the beam selection manager 112 may be configured to use an adaptive learning algorithm to determine/select beams to be used for a beam management procedure. UE 120a and/or BS 110a may then perform a beam management procedure using the determined one or more beams.

It should be noted that although certain aspects are described with respect to beam management procedures being performed by a wireless node, certain aspects of such a beam management procedure may be performed by other types of nodes, such as nodes connected to a BS by a wired connection.

As illustrated in fig. 1, wireless communication network 100 may include a number of BSs 110a-z (each also individually referred to herein as BS 110 or collectively as BS 110) and other network entities. BS 110 may communicate with UEs 120a-y (each also individually referred to herein as UE 120 or collectively as UE 120) in wireless communication network 100. Each BS 110 may provide communication coverage for a particular geographic area. In some examples, BSs 110 may be interconnected to each other and/or to one or more other BSs or network nodes (not shown) in the wireless communication network 100 by various types of backhaul interfaces, such as direct physical connections, wireless connections, virtual networks, or the like using any suitable transport network. In the example shown in fig. 1, BSs 110a, 110b, and 110c may be macro BSs for macro cells 102a, 102b, and 102c, respectively. BS 110x may be a pico BS for picocell 102 x. BSs 110y and 110z may be femto BSs for femto cells 102y and 102z, respectively. A BS may support one or more (e.g., three) cells.

The wireless communication network 100 may also include relay stations. A relay station is a station that receives a transmission of data and/or other information from an upstream station (e.g., a BS or a UE) and sends a transmission of the data and/or other information to a downstream station (e.g., a UE or a BS). A relay station may also be a UE that relays transmissions for other UEs. In the example shown in fig. 1, relay 110r may communicate with BS 110a and UE 120r to facilitate communication between BS 110a and UE 120 r. The relay station may also be referred to as a relay BS, relay, etc.

UEs 120 (e.g., 120x, 120y, etc.) may be dispersed throughout wireless communication network 100, and each UE may be stationary or mobile.

Network controller 130 may be coupled to a set of BSs and provide coordination and control for these BSs. Network controller 130 may communicate with BS 110 via a backhaul. BSs 110 may also communicate with one another via a wireless or wired backhaul (e.g., directly or indirectly).

In some examples, the wireless communication network 100 (e.g., a 5G NR network) may support mmW communication. As discussed above, such systems using mmW communication may use beamforming to overcome high path loss and may perform beam management procedures to select beams for beamforming.

The BS beam (e.g., TX or RX) and the UE beam (e.g., the other of TX or RX) form the BPL. Both the BS (e.g., BS 110a) and the UE (e.g., UE 120a) may determine (e.g., find/select) at least one eligible beam to form a communication link. For example, on the downlink, BS 110a transmits a downlink transmission using a transmit beam and UE 120a receives the downlink transmission using a receive beam. The combination of the transmit beam and the receive beam forms a BPL. UE 120a and BS 110a establish at least one BPL for UE 120a to wireless communication network 100. In some examples, multiple BPLs (e.g., a group of BPLs) may be configured for communication between UE 120a and one or more BSs 110. Different BPLs may be used for different purposes, such as for communicating different channels, for communicating with different BSs, and/or as fallback BPLs in case of failure of an existing BPL.

In some examples, for initial cell acquisition, a UE (e.g., UE 120a) may search for the strongest signal corresponding to a cell associated with a BS (e.g., BS 110a) and associated UE receive and BS transmit beams corresponding to BPLs used to receive/transmit reference signals. After initial acquisition, UE 120a may perform new cell detection and measurements. For example, UE 120a may measure a Primary Synchronization Signal (PSS) and a Secondary Synchronization Signal (SSS) to detect a new cell. As discussed in more detail below with reference to fig. 3, the PSS/SSS may be transmitted by a BS (e.g., BS 110a) in different Synchronization Signal Blocks (SSBs) across one or more sets of Synchronization Signal (SS) bursts. UE 120a may measure different SSBs within the set of SS bursts to perform beam management procedures, as discussed further herein.

In 5G NR, the beam management procedure for determining BPL may be referred to as a P1 procedure. Fig. 2 illustrates an example P1 procedure 202. BS 210 (e.g., such as BS 110a) may send a measurement request to UE 220 (e.g., such as UE 120a) and may then transmit one or more signals (sometimes referred to as "P1 signals") to UE 220 for measurement. In the P1 procedure 202, BS 210 transmits signals by beamforming in each symbol in a different spatial direction (corresponding to transmit beams 211, 212 … … 217) such that several (e.g., most or all) relevant spatial locations of the cell of BS 210 are reached. In this manner, BS 210 transmits signals using different transmit beams in different directions over time. In some examples, SSB is used as the P1 signal. In some examples, a channel state information reference signal (CSI-RS), a demodulation reference signal (DMRS), or another downlink signal may be used as the P1 signal.

In the P1 procedure 202, to successfully receive at least one symbol of the P1 signal, the UE 220 finds (e.g., determines/selects) an appropriate receive beam (221, 222 … … 226). Signals (e.g., SSBs) from multiple BSs may be measured simultaneously for a given signal index (e.g., SSB index) corresponding to a given time period. The UE 220 may apply a different receive beam during each occurrence (e.g., each symbol) of the P1 signal. Once the UE 220 successfully receives the symbol of the P1 signal, the UE 220 and the BS 210 find the BPL (i.e., the UE RX beam for receiving the P1 signal in the symbol and the BS TX beam for transmitting the P1 signal in the symbol). In some cases, the UE 220 does not search all its possible UE RX beams until it finds the best UE RX beam, as this causes additional delay. The UE 220 may instead select an RX beam once it is "good enough" (e.g., has a quality (e.g., SNR) that satisfies a threshold (e.g., a predefined threshold)). UE 220 may not know which beam is used by BS 210 to transmit the P1 signal in a symbol; however, UE 220 may report to BS 210 the time it observed the signal. For example, UE 220 may report to BS 210 the symbol index at which the P1 signal has been successfully received. BS 210 may receive the report and may determine which BS TX beam was used by the BS 210 at the indicated time. In some examples, the UE 220 measures a signal quality of the P1 signal, such as a Reference Signal Received Power (RSRP) or another signal quality parameter (e.g., SNR, channel flatness, etc.). UE 220 may report the measured signal quality (e.g., RSRP) to BS 210 along with the symbol index. In some cases, UE 220 may report to BS 210 a plurality of symbol indices corresponding to a plurality of BS TX beams.

The BPL used between the UE 220 and the BS 110 may be refined/changed as part of the beam management procedure. For example, the BPL may be periodically refined to accommodate changing channel conditions, such as fading due to movement of the UE 220 or other objects, doppler spread, and so forth. The UE 220 may monitor the quality of the BPL (e.g., the BPL found/selected during the P1 procedure and/or the previously refined BPL) in order to refine the BPL when the quality degrades (e.g., when the BPL quality falls below a threshold or when another BPL has a higher quality). In 5G NR, the beam management procedure for beam refinement of BPL may be referred to as P2 and P3 procedures to refine the BS beam and UE beam of individual BPL, respectively.

Fig. 2 illustrates example P2 and P3 procedures 204 and 206. As shown in fig. 2, for the P2 procedure 204, BS 210 transmits symbols of a signal with different BS beams (e.g., TX beams 215, 214, 213) that are spatially close to the BS beam of the current BPL. For example, BS 210 transmits signals in different symbols using adjacent TX beams (e.g., beam sweeps) around the TX beam of the current BPL. As shown in fig. 2, the TX beam used by BS 210 for the P2 procedure 204 may be different from the TX beam used by BS 210 for the P1 procedure 202. For example, the TX beams used by BS 210 for the P2 procedure 204 may be spaced closer together and/or may be more focused (e.g., narrower) than the TX beams used by BS 210 for the P1 procedure 202. During the P2 procedure 204, the UE 220 leaves its RX beam (e.g., RX beam 224) unchanged. UE 220 may measure the signal quality (e.g., RSRP) of the signal in different symbols and indicate the symbol in which the highest signal quality was measured. Based on the indication, BS 210 may determine the strongest (e.g., best or associated with the highest signal quality) TX beam (i.e., the TX beam used in the indicated symbol). The BPL may be refined accordingly to use the indicated TX beam.

As shown in fig. 2, for the P3 procedure 206, the BS 220 maintains a constant TX beam (e.g., the TX beam of the current BPL) and transmits symbols of a signal using the constant TX beam (e.g., TX beam 214). During the P3 procedure 206, the UE 220 scans for signals using different RX beams (e.g., RX beams 223, 224, 225) in different symbols. For example, the UE 220 may perform the sweep using RX beams that are adjacent to the RX beam in the current BPL (i.e., the BPL being refined). UE 220 may measure the signal quality (e.g., RSRP) of the signal for each RX beam and identify the strongest UE RX beam. The UE 220 may use the identified RX beam for BPL. UE 220 may report the signal quality to BS 210.

As discussed above, in some examples, SSB measurements may be used for beam management. Fig. 3 illustrates example SSB locations within an example NR radio frame format 302. The transmission timeline for each of the downlink and uplink may be partitioned into units of radio frames. As shown in fig. 3, an example 10ms NR radio frame format 302 may include ten 1ms subframes (subframes with indices of 0, 1 … … 9). In NR, a basic Transmission Time Interval (TTI) may be referred to as a slot. In NR, a subframe may contain a variable number of slots (e.g., 1, 2, 4, 8, 16 … … slots), depending on the subcarrier spacing (SCS). The NR may support a base SCS of 15KHz, and other SCSs may be defined relative to the base SCS (e.g., 30kHz, 60kHz, 120kHz, 240kHz, etc.). In the example shown in FIG. 3, the SCS is 120 kHz. As shown in fig. 3, subframe 304 (subframe 0) contains 8 slots (slots 0, 1 … … 7) with a duration of 0.125 ms. The symbol and slot lengths scale with subcarrier spacing. Each slot may include a variable number of symbol (e.g., OFDM symbol) periods (e.g., 7 or 14 symbols) depending on the SCS. For the 120kHz SCS shown in fig. 3, each of slot 306 (slot 0) and slot 308 (slot 1) includes 14 symbol periods (slots with indices of 0, 1 … … 13) with a duration of 0.25 ms.

In some examples, the SSB may be transmitted up to sixty four times in up to sixty four different beam directions. Up to sixty-four transmissions of an SSB are referred to as a set of SS bursts. SSBs in a set of SS bursts may be transmitted in the same frequency region, while SSBs in different sets of SS bursts may be transmitted in different frequency regions. In the example shown in fig. 3, in subframe 304, an SSB is transmitted in each slot (slot 0, 1 … … 6). In the example shown in fig. 3, in slot 306 (slot 0), SSB 310 is transmitted in symbols 4, 5, 6, 7, and SSB 312 is transmitted in symbols 8, 9, 10, 11 and in slot 308 (slot 1), SSB 314 is transmitted in symbols 2, 3, 4, 5, and SSB 316 is transmitted in symbols 6, 7, 8, 9, and so on. The SSB may include a PSS, an SSS, and a two-symbol Physical Broadcast Channel (PBCH). The PSS and SSS may be used by the UE for cell search and acquisition. For example, PSS may provide half frame timing, SSS may provide Control Protocol (CP) length and frame timing, and PSS and SSS may provide cell identity. The PBCH carries some basic system information such as downlink system bandwidth, timing information within the radio frame, SS burst set periodicity, system frame number, etc.

As shown in fig. 4, SSB may be used to make measurements using different transmit and receive beams, for example, according to a beam management procedure, such as the P1 procedure 202 shown in fig. 2. Fig. 4 illustrates an example of a BS410 (e.g., such as BS 110a) using 4 TX beams and a UE 420 (e.g., such as UE 120a) using 2 RX beams. For each SSB, BS410 transmits the SSB using a different TX beam BS. As shown in fig. 4, UE 420 may scan its RX beam 422 while BS410 transmits SSBs 310, 312, 314, 316 that sweep its four TX beams 412, 414, 416, 418, respectively. BPLs may be identified and used for data communication over a period of time, as discussed. As shown in fig. 4, BS410 uses TX beam 414 and UE 420 uses RX beam 422 for data communication over a period of time. UE 410 may then scan its RX beam 424 while BS410 transmits SSBs 426, 428 that sweep its TX beams 412, 414, and so on.

As can be seen, as the number of TX/RX beams increases, the number of scans by the UE of each of its RX beams on each TX beam may become larger. The power consumption may be linearly proportional to the number of SSBs measured. Thus, the time and power overhead associated with beam management can become large if all beams are actually scanned.

Accordingly, aspects of the present disclosure provide techniques for assisting a node in performing measurements on other nodes when beamforming is used, for example, by using adaptive learning, which may reduce the number of measurements used for beam management procedures and thereby reduce power consumption.

Example Beam management procedure Using adaptive learning

Non-adaptive algorithms are deterministic in their input as a function. If the algorithm were to face exactly the same input at different times, its output would be exactly the same. An adaptive algorithm is an algorithm that changes its behavior based on its past experience. This means that different devices using the adaptive algorithm may eventually have different algorithms over time.

According to certain aspects, the beam management procedure may be performed using an adaptive learning based beam management algorithm. Thus, the beam algorithm changes (e.g., adapts, updates) over time based on the new learning. The beam management procedure may be used for initial acquisition, cell discovery after initial acquisition, and/or determining the BPL for the strongest cell detected by the UE. For example, adaptive learning may be used to construct a UE codebook that indicates beams to be used (e.g., measured) for a beam management procedure. In some examples, adaptive learning may be used to select UE receive beams to be used for discovery of BPL. Adaptive learning may be used to intelligently select which UE receive beams will be used to measure signals based on training and experience so that fewer beams may be measured while still finding a suitable BPL (e.g., meeting a threshold signal quality).

In some examples, adaptive learning based beam management involves training a model, such as a predictive model. The model may be used during a beam management procedure to select which UEs will use which receive beams to measure signals. The model may be trained based on training data (e.g., training information), which may include feedback, such as feedback associated with a beam management procedure. Fig. 5 illustrates an example networking environment 500 in which predictive models 524 are used for beam management, according to certain aspects of the present disclosure.

As shown in fig. 5, the networked environment 500 includes a node 520, a training system 530, and a training repository 515 communicatively connected via a network 505. Node 520 may be a UE (e.g., such as UE 120a in wireless communication network 100) or a BS (e.g., such as BS 110a in wireless communication network 100). Network 505 may be a wireless network, such as wireless communication network 100, which may be a 5G NR network. Although training system 530, node 520, and training repository 515 are illustrated as separate components in fig. 5, one skilled in the art will recognize that training system 530, node 520, and training repository 515 may be implemented on any number of computing systems, as one or more independent systems, or in a distributed environment.

The training system 530 generally includes a predictive model training manager 532 that uses training data to generate the predictive models 524 for beam management. The predictive model 524 may be determined based on information in the training repository 515.

Training repository 515 may include training data acquired before and/or after deployment of node 520. The node 520 may be trained in a simulated communication environment (e.g., in field testing, driving testing) prior to deployment of the node 520. For example, various beam management procedures (e.g., various selections of UE RX beams for measuring signals) may be tested in various scenarios, such as at different UE speeds, with the UE stationary, with various rotations of the UE, with various BS deployments/geometries, and so on, to acquire training information related to the beam management procedures. This information may be stored in the training repository 515. After deployment, training repository 515 may be updated to include feedback associated with the beam management procedures performed by node 520. The training repository may also be updated with information from other BSs and/or UEs, e.g., based on learned experiences of the BSs and/or UEs that may be associated with beam management procedures performed by the BSs and/or UEs.

The predictive model training manager 532 may use information in the training repository 515 to determine a predictive model 524 (e.g., an algorithm) for beam management, such as to select a UE RX beam for measuring signals. As discussed in more detail herein, the predictive model training manager 532 may use various different types of adaptive learning, such as machine learning, depth information, reinforcement learning, and the like, to form the predictive model 524. The training system 530 may adapt (e.g., update/improve) the predictive model 524 over time. For example, when the training repository is updated with new training information (e.g., feedback), the model 524 is updated based on the new learning/experience.

Training system 530 may be located at node 520, a BS in network 505, or a different entity that determines predictive model 524. If located on a different entity, predictive model 524 is provided to node 520.

The training repository 515 may be a storage device, such as a memory. Training repository 515 may be located on node 520, training system 530, or another entity in network 505. The training repository 515 may be in cloud storage. The training repository 515 may receive training information from the node 520, an entity in the network 505 (e.g., a BS or UE in the network 505), the cloud, or other sources.

As described above, the node 520 is provided with (or generates, e.g., where the training system 530 is implemented in the node 520) a predictive model. As illustrated, node 520 may include a beam selection manager 522 configured to use a predictive model 524 for beam management (e.g., such as one of the beam management procedures discussed above with reference to fig. 2). In some examples, node 520 utilizes predictive model 524 to construct a UE codebook and/or to determine/select a beam from the UE codebook to be used for a beam management procedure. Predictive model 524 is updated as training system 530 adapts predictive model 524 with new learning.

Thus, the beam management algorithm (using the predictive model 524) of the node 520 is based on adaptive learning, as the algorithm used by the node 520 changes over time based on experience/feedback obtained by the node 520 in the deployment scenario (and/or through training information also provided by other entities), even after deployment.

According to certain aspects, the adaptive learning may use any suitable learning algorithm. As described above, the learning algorithm may be used by a training system (e.g., such as training system 530) to train a predictive model (e.g., such as predictive model 524) to obtain an adaptive learning-based beam management algorithm for use by a device (e.g., such as node 520) for beam management procedures. In some examples, the adaptive learning algorithm is an adaptive machine learning algorithm, an adaptive reinforcement learning algorithm, an adaptive deep learning algorithm, an adaptive continuous infinite learning algorithm, or an adaptive policy optimization reinforcement learning algorithm (e.g., a near-end policy optimization (PPO) algorithm, a policy gradient, a trust domain policy optimization (TRPO) algorithm, etc.). In some examples, the adaptive learning algorithm is modeled as a Partially Observable Markov Decision Process (POMDP). In some examples, the adaptive learning algorithm is implemented by an artificial neural network (e.g., a Deep Q Network (DQN) including one or more Deep Neural Networks (DNNs)).

In some examples, adaptive learning (e.g., used by the training system 530) is performed using a neural network. Neural networks can be designed with various modes of connectivity. In a feed-forward network, information is passed from a lower layer to a higher layer, with each neuron in a given layer communicating to a neuron in a higher layer. The hierarchical representation may be built in successive layers of the feed-forward network. The neural network may also have a backflow or feedback (also known as top-down) connection. In a reflow connection, output from a neuron in a given layer may be communicated to another neuron in the same layer. The reflow architecture may help identify patterns that span more than one input data chunk delivered to the neural network in sequence. The connection from a neuron in a given layer to a neuron in a lower layer is called a feedback (or top-down) connection. Networks with many feedback connections may be helpful when the identification of high-level concepts may assist in discerning particular low-level features of an input.

In some examples, adaptive learning (e.g., used by the training system 530) is performed using a Deep Belief Network (DBN). A DBN is a probabilistic model that includes multiple layers of hidden nodes. The DBN may be used to extract a hierarchical representation of the training data set. The DBN may be obtained by stacking multiple layers of a constrained boltzmann machine (RBM). RBMs are a class of artificial neural networks that can learn a probability distribution over a set of inputs. Since RBMs can learn probability distributions without information about which class each input can be classified into, RBMs are often used for unsupervised learning. Using a hybrid unsupervised and supervised paradigm, the bottom RBM of the DBN can be trained in an unsupervised fashion and can be used as a feature extractor, while the top RBM can be trained in a supervised fashion (over a joint distribution of inputs from previous layers and target classes) and can be used as a classifier.

In some examples, adaptive learning (e.g., used by the training system 530) is performed using a Deep Convolutional Network (DCN). A DCN is a network in a convolutional network configured with additional pooling and normalization layers. DCNs have achieved the most advanced performance available over many tasks. DCNs can be trained using supervised learning, where both input and output targets are known to many paradigms and used to modify the weights of the network by using a gradient descent method. The DCN may be a feed forward network. In addition, as described above, connections from neurons in the first layer of the DCN to neuron populations in the next higher layer are shared across neurons in the first layer. The feed-forward and shared connections of the DCN can be utilized for fast processing. The computational burden of a DCN may be much less than that of, for example, a similarly sized neural network including a reflow or feedback connection.

An artificial neural network, which may include a population of interconnected artificial neurons (e.g., a neuron model), is a computing device or represents a method performed by a computing device. These neural networks may be used for various applications and/or devices, such as Internet Protocol (IP) cameras, internet of things (IoT) devices, autonomous vehicles, and/or service robots. Individual nodes in an artificial neural network can mimic biological neurons by taking input data and performing simple operations on the data. The result of a simple operation performed on the input data is selectively communicated to other neurons. Weight values are associated with each vector and node in the network, and these values constrain how the input data relates to the output data. For example, the input data for each node may be multiplied by a respective weight value, and the products may be summed. The sum of these products can be adjusted by an optional offset, and an activation function can be applied to the result, producing an output signal or "output activation" of the node. The weight values may be initially determined by the iterative flow of training data in the network (e.g., the weight values are established during a training phase in which the network learns how to identify particular classes by their typical input data characteristics).

Adaptive learning (e.g., used by the training system 530) may be implemented using different types of artificial neural networks, such as Recurrent Neural Networks (RNNs), multi-layer perceptron (MLP) neural networks, Convolutional Neural Networks (CNNs), and so forth. The RNN works by saving the output of a layer and feeding the output back to the input to help predict the results of that layer. In an MLP neural network, data may be fed into an input layer, and one or more hidden layers provide several levels of abstraction for the data. A prediction may then be made for the output layer based on the abstracted data. MLP may be particularly suited to categorizing predictive problems, where the input is assigned a class or label. Convolutional Neural Networks (CNNs) are a kind of feedforward artificial neural networks. A convolutional neural network may include a collection of artificial neurons that each have a receptive field (e.g., a spatially local region of an input space) and collectively spell out an input space. Convolutional neural networks have numerous applications. In particular, CNNs have been widely used in the field of pattern recognition and classification. In a hierarchical neural network architecture, the output of a first layer of artificial neurons becomes the input of a second layer of artificial neurons, the output of a second layer of artificial neurons becomes the input of a third layer of artificial neurons, and so on. The convolutional neural network may be trained to identify feature classes. The computations in a convolutional neural network architecture may be distributed over a population of processing nodes, which may be configured in one or more computation chains. These multi-layer architectures can be trained one layer at a time and can be fine-tuned using back-propagation.

In some examples, using an adaptive machine learning algorithm, the training system 530 generates vectors from the information in the training repository 515. In some examples, the training repository 515 stores vectors. In some examples, the vector maps one or more features to a tag. For example, the features may correspond to various deployment scenario modes discussed herein, such as UE mobility, speed, rotation, channel conditions, BS deployment/geometry in the network, and so forth. The tag may correspond to a predicted optimal beam selection (e.g., for an RX beam) associated with a feature for performing a beam management procedure. Predictive model training manager 532 may use the vectors to train predictive models 524 for nodes 520. As discussed above, the vectors may be associated with weights in an adaptive learning algorithm. When the learning algorithm adapts (e.g., updates), the weights applied to the vectors may also be changed. Thus, when the beam management procedure is performed again under the same characteristics (e.g., under the same set of conditions), the model may give node 520 different results (e.g., different beam selections).

FIG. 6 conceptually illustrates an example reinforcement learning model. Reinforcement learning may be a semi-supervised learning model in machine learning. Reinforcement learning allows agent 604 (e.g., node 520 and/or training system 530) to take action (e.g., beam selection) and interact with environment 606 (e.g., current deployment scenario) based on the state observed by interpreter 602 (e.g., such as node 520) (e.g., RSPR of SSB using different beams) to maximize the total reward (e.g., Physical Downlink Shared Channel (PDSCH) throughput using selected beams) that can be observed by interpreter 602 and fed back to agent 604 as reinforcement. In some examples, agent 604 and interpreter 602 may be implemented as the same or separate component devices that may perform various functions of node 520, training system 530, and/or training repository 515.

In some examples, reinforcement learning is modeled as a Markov Decision Process (MDP). MDP is a discrete, time-random control process. MDP provides a mathematical framework for modeling decision making in situations where the results may be partially random and partially under the control of a decision maker. In MDP, at each time step, the process is in a state in the finite set of states S, and the decision maker can select any action in the finite set of actions a available in that state. The process responds at the next time step by randomly moving into a new state and giving the decision maker a corresponding reward, where R_α(s, s ') is the instant prize (or the expected instant prize) after the transition from state s to state s'. The probability of the process moving into its new state is subject to the selected actionE.g. according to a state transition function. The state transition can be made from P_α(s，s′)＝Pr(s_t+1＝s′|s_t＝s，α_tα) is given.

MDP seeks to find a policy for decision: a function of pi that specifies the action pi(s) that the decision maker will select when in state s. The goal is to select a strategy pi that maximizes the reward. For example, a policy that maximizes a reward cumulative function (such as a discount sum). An example function is shown below:

wherein

α_t＝π(s_t) I.e., the action given by the policy, and γ is a discount factor and satisfies 0 ≦ γ ≦ 1.

The solution to MDP is a policy that describes the best action (e.g., maximizing the expected discount reward) for each state in MDP.

In some examples, partially observable mdp (pomdp) is used. POMDP may be used when the state may not be known at the time the action is taken and thus the probability and/or reward may not be known. For POMDP, reinforcement learning may be used. The following functions may be defined:

Q(s，a)＝∑_s′P_α(s，s′)(R_α(s，s′)+γV(s′))。

experience during learning can be based on the (s, a) pair and the result s'. For example, in the case where the node was previously in state s and made beam selection a, and achieved throughput s'. In this example, the node may update the array Q based directly on learned experience. This may be referred to as Q learning. In some examples, the learning algorithm may be continuous.

In some examples, for an adaptive learning based beam management algorithm, the state may correspond to the M strongest beam quality measurements (e.g., Reference Signal Received Power (RSRP) of SSBs on different beams) in the environment (e.g., the current deployment scenario of the UE), including conditions discussed herein including UE mobility, BS deployment pattern (e.g., geometry), obstructions, and so on. The action may correspond to beam selection. The reward may be throughput achieved using beam selection, such as PDSCH throughput. The reward may be another parameter such as, for example, spectral efficiency. Thus, by using this MDP at a given state at a given time, a node may employ a strategy of finding a beam selection that specifies maximizing throughput. As discussed above, the reward may be discounted. For beam management, the reward may be offset by some penalty as a function of the measured SSB, e.g., to optimize for minimum power.

Referring back to the example network environment 500 in fig. 5 and the reinforcement learning model 600 in fig. 6, in some examples, the predictive model training manager 532 or the agent 604 may use reinforcement learning on the predictive model (e.g., the predictive model 524) to determine a policy (e.g., an MDP solution). Node 520 or agent 604 may take an action, such as beam selection for a beam management procedure, based on a policy given by a predictive model (e.g., predictive model 524) for a current state (e.g., observed by node 520 or interpreter 602) at a given time in the environment (e.g., environment 606). The reinforcement learning algorithm and the predictive model may be updated/adapted based on learned experience (e.g., may be stored in the training repository 515).

The framework of reinforcement learning provides tools to optimally solve for POMDP. This learning changes the weights of the multi-layered perceptrons (e.g., neural networks) that determine the next action to take. The algorithm in depth ML is coded with neural net weights. Thus, changing the weights changes the algorithm.

In some examples, adaptive learning based beam management uses an adaptive deep learning algorithm. The adaptive deep learning algorithm may be a Deep Q Network (DQN) implemented by a neural network. Fig. 7 conceptually illustrates an example DQN learning model 700, in accordance with certain aspects of the present disclosure. As shown in fig. 7, an agent 706 (e.g., such as agent 604 or node 520) includes an artificial neural network, e.g., a Deep Neural Network (DNN)708 such as shown in the example in fig. 7. For a current environment 702 (e.g., such as environment 606), which may be a real deployment scenario involving a UE (e.g., UE 120a) and a BS (e.g., BS 110a) and various conditions described herein, the agent 706 observes a state 704(s). For example, the observed states may be the M strongest RSRPs corresponding to SSBs measured using different beams for the beam management procedure.

In some examples, the adaptive learning algorithm is modeled as POMDP through reinforcement learning. POMDP may be used when the state may not be known at the time the action is taken and thus the probability and/or reward may not be known. For POMDP, reinforcement learning may be used. The Q array may be defined as:

Q_i+1(s，a)＝E{r+γmaxQ_i(s′，a′)|s，a}。

as shown in FIG. 7, a given state 704s (e.g., RSRP) and possible actions a are input to a DNN 708, which may execute an algorithm to output a value (e.g., parameter θ) per possible action a to determine a policy (e.g., π) based on the maximum value_θ(s, a)) the policy and corresponding action are taken and applied to the environment. For example, the agent 706 makes a beam selection and then uses the selected beam in the environment 702. As shown in fig. 7, the reward for the action is fed back to the agent 706 to update the algorithm. For example, the throughput achieved by the selected beam may be fed back. Based on this feedback, the agent 706 updates the DNN 708 (e.g., by changing the weights associated with the vectors).

According to certain aspects, adaptive learning based beam management allows continuous infinite learning. In some examples, learning may be augmented by joint learning. For example, when some machine learning methods use centralized training data on a single machine or in a data center, learning can be collaborative through joint learning, which involves multiple devices to form a predictive model. With joint learning, model training may be accomplished on multiple devices through cooperative learning from the devices. For example, referring back to fig. 5-7, node 520, agent 604, and agent 706 may receive training information and/or updated trained models from a variety of different devices.

In an illustrative example, beam management algorithms for a plurality of different UEs may be trained in a plurality of different operating scenarios, e.g., using deep reinforcement learning. The output of the training from different UEs may be combined to train a beam management algorithm for the UE. Once the beam management algorithm is trained, the algorithm may continue learning based on the actual deployment scenario. As discussed above, the state may be the best M RSRP metrics for the current time; the reward may be the measured PDSCH throughput for the current best beam pair; and the action may be a selection of which beam pair/pairs to measure.

According to certain aspects, adaptive learning based beam management allows for user personalization and design robustness. In some examples, adaptive learning based beam management may be optimized. For example, when a user (e.g., such as node 520) visits/traverses a path (e.g., an environment), the adaptive algorithm learns and optimizes for the environment. Furthermore, different BS vendors may have different beam management implementations, such as how SSBs are transmitted. For example, some BS vendors transmit many narrow TX beams that will also serve as data beams; while others transmit a few wide beams and use beam refinement (e.g., the P2 and/or P3 procedures) to narrow and track the data beams. In some examples, adaptive learning based beam management may be optimized for a particular beam management implementation for a vendor. In some examples, adaptive learning based beam management may be optimized for a user, e.g., the manner in which the user holds/uses the UE affects the possible blocking of the UE's beams.

Fig. 8 is a flowchart illustrating example operations 800 for wireless communication, in accordance with certain aspects of the present disclosure. Operations 800 may be performed, for example, by a node (e.g., such as node 520, which may be a wireless node, which may be BS 110a or UE 120a in wireless communication network 100). Operations 800 may be implemented as software components executed and run on one or more processors (e.g., controllers/processors 1240, 1280 of fig. 12). Moreover, signal transmission and reception by the node in operation 800 may be implemented, for example, by one or more antennas (e.g., antennas 1234, 1252 of fig. 12). In certain aspects, transmission and/or reception of signals by nodes may be implemented via a bus interface of one or more processors (e.g., controllers/processors 1240, 1280) that obtain and/or output the signals.

Operation 800 may begin at 805 with the use of adaptive learning to determine one or more beams to be used for a beam management procedure.

At 810, the node performs a beam management procedure using the determined one or more beams.

According to certain aspects, the adaptive learning uses an adaptive learning algorithm. The adaptive learning algorithm may be updated (e.g., adaptive) based on the feedback and/or training information. The node may perform another beam management procedure using the updated adaptive learning algorithm. The feedback may be feedback associated with a beam management procedure. For example, after performing a beam management procedure using the determined one or more beams, the node may receive feedback regarding the achieved throughput, and the beam management algorithm may be updated based on the feedback. In some examples, the feedback may be associated with beam management performed by different devices, such as different nodes.

Fig. 9 is an example call flow diagram illustrating example signaling 900 for beam management using adaptive learning, in accordance with certain aspects of the present disclosure. As shown in fig. 9, at 908, a UE 902 (e.g., such as UE 120a) may have an initial learning algorithm (e.g., including a predictive model). In some examples, the UE 902 may train the initial learning algorithm or the learning algorithm may be trained and then provided to the UE 902. At 910, the UE 902 performs a beam management procedure (e.g., such as the P1 procedure 202) with one or more BSs 904. For example, the UE 902 may use an adaptive learning algorithm to determine beams to use and/or measure. At 912, the UE 902 receives additional training information and/or feedback. For example, the UE 902 may receive feedback from the BS 904 (e.g., such as BS 110a) regarding the beam management procedure performed at 910, such as PDSCH throughput achieved using the selected beam. Additionally or alternatively, the UE 902 may receive additional training information from the BS 904 and/or another UE 906. At 914, the UE 902 determines an updated adaptive learning algorithm based on the additional training information and/or feedback. At 916, the UE 902 may perform another beam management with the BS 904 (or another BS) through the updated adaptive learning algorithm.

In some examples, the training information (and/or feedback) includes training information obtained by deploying one or more UEs in one or more simulated communication environments prior to network deployment of the one or more UEs; training information obtained through feedback previously received (e.g., based on measurements and/or beam management procedures performed by the UE) when the one or more UEs are deployed in one or more communication environments; training information from a network, one or more UEs, and/or a cloud; and/or training information received while the node is online and/or idle.

In some examples, using the adaptive learning algorithm at 805 includes the node outputting an action based on one or more inputs; wherein the feedback is associated with the action; and updating the adaptive learning algorithm based on the feedback includes adjusting one or more weights applied to the one or more inputs.

In some examples, the adaptive learning algorithm used by the node at 805 comprises an adaptive machine learning algorithm; a self-adaptive reinforcement learning algorithm; a self-adaptive deep learning algorithm; self-adaptive continuous infinite learning algorithm; and/or an adaptive strategy optimization reinforcement learning algorithm. As discussed above with reference to fig. 6-7, the adaptive learning algorithm may be modeled as POMDP. The adaptive learning algorithm may be implemented by an artificial neural network. In some examples, the artificial neural network may be a DQN comprising one or more DNNs. Determining one or more beams using adaptive learning may include passing state parameters and action parameters through one or more DNNs; for each state parameter, outputting the value of each action parameter; and an act of selecting the associated maximum output value. Updating the adaptive learning algorithm may include adjusting one or more weights associated with one or more neuron connections in the artificial neural network.

In some examples, determining, at 805, one or more beams to be used for the beam management procedure includes determining, based on adaptive learning, one or more beams to be included in a codebook and selecting, from the codebook, the one or more beams to be used for the beam management procedure.

In some examples, determining, using adaptive learning, at 805, one or more beams to be used for the beam management procedure includes selecting, using adaptive learning, one or more beams from a codebook to be used for the beam management procedure.

In some examples, adaptive learning is used to select BPL.

In some examples, the adaptive learning uses a state parameter associated with the channel measurements, a reward parameter associated with received signal throughput or spectral efficiency, and an action parameter associated with selection of a beam pair corresponding to the channel measurements. In some examples, the channel measurements include RSRP; spectral efficiency, channel flatness, and/or signal-to-noise ratio (SNR). In some examples, the received signal is a PDSCH transmission.

In some examples, the reward parameter is discounted by a penalty amount. In some examples, the penalty amount depends on the number of beams (e.g., beams used for transmission and/or reception of SSBs) measured for the beam management procedure. In some examples, the penalty amount depends on an amount of power consumption associated with the beam management procedure.

In some examples, performing the beam management procedure using the determined one or more beams at 810 includes measuring a channel based on SSB transmissions from the BS using the determined one or more beams, the SSB transmissions associated with a plurality of different transmit beams of the BS; and selecting one or more BPLs associated with channel measurements that are channel measurements above a channel measurement threshold and/or one or more strongest of all channel measurements associated with the measured SSB transmissions. In some examples, the determined one or more beams are a subset of available receive beams. In some examples, the node receives the PDSCH using one of the one or more selected BPLs; determining a throughput associated with the PDSCH; updating an adaptive learning algorithm based on the determined throughput; and using the updated adaptive learning algorithm to determine another one or more beams to be used to perform another beam management procedure to select another one or more BPLs.

Fig. 10 is an example call flow diagram illustrating example signaling 1000 for conducting a BPL discovery procedure (e.g., such as the P1 procedure 202) using adaptive learning, in accordance with certain aspects of the present disclosure. As shown in fig. 10, at 1008, a UE 1002 (e.g., such as UE 120a) may perform initial training in a simulation environment prior to deployment at 1010. Initial training at 1008 may train an initial learning algorithm (e.g., including a predictive model) at the UE 1002. At 1010, the UE 1002 may be deployed in a network having at least one BS 1004 (e.g., such as BS 110 a). The UE 1002 may perform beam management procedures (e.g., such as the P1 procedure 202) with one or more BSs 1004 in the network. For example, as shown in fig. 10, at 1012, the UE 1002 may select a beam or RX/TX beam pair using an adaptive learning algorithm. At 1016, UE 1002 measures the SSB transmission(s) received from BS 1004 at 1014 using the beam(s) selected at 1012. At 1018, the UE 1002 reports the measurements and/or BPL selection to the BS 1004. Then, at 1020, the BS 1004 transmits the PDSCH to the UE 1002 using the BPL indicated by the UE 1002 (or selected based on measurements reported by the UE 1002). At 1022, the UE 1002 may determine PDSCH throughput. PDSCH throughput may serve as feedback or reinforcement to adaptive learning. At 1026, the UE 1002 updates the adaptive learning algorithm based on the feedback. Optionally, the UE 1002 may receive additional training information and/or feedback from the BS 1004 and/or another UE 1006 (e.g., UE 2) that the UE 1002 may use to update the adaptive learning algorithm at 1026. The UE 1002 may then perform another beam management with the BS 1004 (or another BS) through the updated adaptive learning algorithm.

Fig. 11 illustrates a communication apparatus 1100 that may include various components (e.g., corresponding to means plus function components) configured to perform operations of the techniques disclosed herein, such as the operations illustrated in fig. 8. The communication device 1100 includes a processing system 1102 coupled to a transceiver 1108. The transceiver 1108 is configured to transmit and receive signals (such as various signals as described herein) for the communication device 1100 via the antenna 1110. The processing system 1102 can be configured to perform processing functions for the communication device 1100, including processing signals received by and/or to be transmitted by the communication device 1100.

The processing system 1102 includes a processor 1104 coupled to a computer-readable medium/memory 1112 via a bus 1106. In certain aspects, the computer-readable medium/memory 1112 is configured to store instructions (e.g., computer-executable code) that, when executed by the processor 1104, cause the processor 1104 to perform the operations illustrated in fig. 8 or other operations for performing the various techniques for adaptive learning based beam management discussed herein. In certain aspects, the computer-readable medium/memory 1112 stores code 1114 for determining one or more beams to be used for a beam management procedure using adaptive learning; and code 1116 for performing a beam management procedure using the determined one or more beams. In certain aspects, the processor 1104 has circuitry configured to implement code stored in the computer-readable medium/memory 1112. The processor 1104 includes circuitry 1118 for using adaptive learning to determine one or more beams to be used for beam management procedures; and circuitry 1120 for performing a beam management procedure using the determined one or more beams.

In some examples, the communication device 1100 may include a system on a chip (SOC) (not shown) that may include a Central Processing Unit (CPU) or a multi-core CPU configured to perform adaptive learning-based beam management according to certain aspects of the present disclosure. Variables (e.g., neural signals and synaptic weights), system parameters associated with a computing device (e.g., neural networks with weights), delays, frequency bin information, and task information may be stored in a memory block associated with a Neural Processing Unit (NPU), a memory block associated with a CPU, a memory block associated with a Digital Signal Processor (DSP), in a different memory block, or may be distributed across multiple memory blocks. The instructions executed at the CPU may be loaded from a program memory associated with the CPU or may be loaded from a different memory block.

In some examples, adaptive learning-based beam management described herein may allow for improvements in the P1 procedure by adaptively updating beam management algorithms such that beam selection may be refined to more intelligently select beams to measure based on learning. Thus, the UE may find the BPL while measuring fewer beams.

Methods disclosed herein comprise one or more steps or actions for achieving the method. The method steps and/or actions may be interchanged with one another without departing from the scope of the claims. In other words, unless a specific order of steps or actions is specified, the order and/or use of specific steps and/or actions may be modified without departing from the scope of the claims.

As used herein, a phrase referring to "at least one of" a list of items refers to any combination of these items, including a single member. By way of example, "at least one of a, b, or c" is intended to encompass: a. b, c, a-b, a-c, b-c, and a-b-c, and any combination of multiple identical elements (e.g., a-a-a, a-a-b, a-a-c, a-b-b, a-c-c, b-b-b, b-b-c, c-c, and c-c-c, or any other ordering of a, b, and c).

As used herein, the term "determining" encompasses a wide variety of actions. For example, "determining" can include calculating, computing, processing, deriving, studying, looking up (e.g., looking up in a table, a database, or another data structure), ascertaining, and the like. Also, "determining" may include receiving (e.g., receiving information), accessing (e.g., accessing data in a memory), and the like. Also, "determining" may include resolving, selecting, choosing, establishing, and the like.

Fig. 12 illustrates example components of a BS 110a and a UE 120a (e.g., in the wireless communication network 100 of fig. 1) that may be used to implement aspects of the present disclosure. For example, antennas 1252, processors 1266, 1258, 1264, and/or controller/processor 1280 of UE 120a and/or antennas 1234, processors 1220, 1230, 1238, and/or controller/processor 1240 of BS 110a may be used to perform various techniques and methods described herein. As shown in fig. 12, controller/processor 1280 of UE 120a has a beam selection manager 1281, which, according to aspects described herein, may be configured to use adaptive learning to determine, for example, a beam to be used for a beam management procedure. As shown in fig. 12, additionally or alternatively, controller/processor 1240 of BS 110a may have a beam selection manager 1241, which beam selection manager 1241 may be configured to determine beams using adaptive learning according to aspects described herein.

At BS 110a, a transmit processor 1220 may receive data from a data source 1212 and control information from a controller/processor 1240. The control information may be used for a Physical Broadcast Channel (PBCH), a Physical Control Format Indicator Channel (PCFICH), a physical hybrid ARQ indicator channel (PHICH), a Physical Downlink Control Channel (PDCCH), a group common PDCCH (gc PDCCH), etc. The data may be for a Physical Downlink Shared Channel (PDSCH), etc. Processor 1220 may process (e.g., encode and symbol map) the data and control information to obtain data symbols and control symbols, respectively. Transmit processor 1220 may also generate reference symbols, such as for a Primary Synchronization Signal (PSS), a Secondary Synchronization Signal (SSS), and a cell-specific reference signal (CRS). A Transmit (TX) multiple-input multiple-output (MIMO) processor 1230 may perform spatial processing (e.g., precoding) on the data symbols, the control symbols, and/or the reference symbols, if applicable, and may provide output symbol streams to Modulators (MODs) 1232a-1232 t. Each modulator 1232 may process a respective output symbol stream (e.g., for OFDM, etc.) to obtain an output sample stream. Each modulator may further process (e.g., convert to analog, amplify, filter, and upconvert) the output sample stream to obtain a downlink signal. Downlink signals from modulators 1232a-1232t may be transmitted via antennas 1234a-1234t, respectively.

At UE 120a, antennas 1252a-1252r may receive the downlink signals from BS 110a and may provide received signals to demodulators (DEMODs) in transceivers 1254a-1254r, respectively. Each demodulator 1254 may condition (e.g., filter, amplify, downconvert, and digitize) a respective received signal to obtain input samples. Each demodulator may further process the input samples (e.g., for OFDM, etc.) to obtain received symbols. A MIMO detector 1256 may obtain received symbols from all demodulators 1254a through 1254r, perform MIMO detection on the received symbols if applicable, and provide detected symbols. A receive processor 1258 may process (e.g., demodulate, deinterleave, and decode) the detected symbols, provide decoded data for UE 120a to a data sink 1260, and provide decoded control information to a controller/processor 1280.

On the uplink, at UE 120a, a transmit processor 1264 may receive and process data from a data source 1262 (e.g., for the Physical Uplink Shared Channel (PUSCH)) and control information from a controller/processor 1280 (e.g., for the Physical Uplink Control Channel (PUCCH)). Transmit processor 1264 may also generate reference symbols for a reference signal (e.g., a Sounding Reference Signal (SRS)). The symbols from transmit processor 1264 may be precoded by a TX MIMO processor 1266 if applicable, further processed by demodulators 1254a through 1254r in the transceiver (e.g., for SC-FDM, etc.), and transmitted to the base station 110. At BS 110a, the uplink signals from UE 120a may be received by antennas 1234, processed by modulators 1232, detected by a MIMO detector 1236 if applicable, and further processed by a receive processor 1238 to obtain decoded data and control information sent by UE 120 a. The receive processor 1238 may provide the decoded data to a data sink 1239 and the decoded control information to the controller/processor 1240.

Controllers/processors 1240 and 1280 may direct the operation at BS 110a and UE 120a, respectively. Controller/processor 1240 and/or other processors and modules at BS 110a may perform or direct the performance of various processes for the techniques described herein. Memories 1242 and 1282 may store data and program codes for BS 110a and UE 120a, respectively. A scheduler 1244 may schedule UEs for data transmission on the downlink and/or uplink.

The techniques described herein may be used for various wireless communication technologies such as 3GPP Long Term Evolution (LTE), LTE-advanced (LTE-a), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Frequency Division Multiple Access (FDMA), Orthogonal Frequency Division Multiple Access (OFDMA), single carrier frequency division multiple access (SC-FDMA), time division synchronous code division multiple access (TD-SCDMA), and other networks. The terms "network" and "system" are often used interchangeably. A CDMA network may implement radio technologies such as Universal Terrestrial Radio Access (UTRA), CDMA2000, and so on. UTRA includes wideband CDMA (wcdma) and other variants of CDMA. cdma2000 covers IS-2000, IS-95 and IS-856 standards. TDMA networks may implement radio technologies such as global system for mobile communications (GSM). An OFDMA network may implement radio technologies such as NR (e.g., 5G RA), evolved UTRA (E-UTRA), Ultra Mobile Broadband (UMB), IEEE 802.11(Wi-Fi), IEEE 802.16(WiMAX), IEEE 802.20, Flash-OFDMA, and the like. UTRA and E-UTRA are parts of the Universal Mobile Telecommunications System (UMTS). LTE and LTE-A are UMTS releases using E-UTRA. UTRA, E-UTRA, UMTS, LTE-A and GSM are described in literature from an organization named "third Generation partnership project" (3 GPP). cdma2000 and UMB are described in documents from an organization named "third generation partnership project 2" (3GPP 2). NR is an emerging wireless communication technology that is being developed in conjunction with the 5G technology forum (5 GTF). NR access (e.g., 5G NR) may support various wireless communication services, such as mmW. The NR may utilize OFDM with CP on the uplink and downlink and include support for half-duplex operation using TDD. Beamforming may be supported and beam directions may be dynamically configured. MIMO transmission with precoding may also be supported. In some examples, MIMO configuration in DL may support up to 8 transmit antennas (multi-layer DL transmission with up to 8 streams) and up to 2 streams per UE. In some examples, multi-layer transmission of up to 2 streams per UE may be supported.

In 3GPP, the term "cell" can refer to a coverage area of a Node B (NB) and/or an NB subsystem serving the coverage area, depending on the context in which the term is used. In the NR system, the terms "cell", BS, next generation node B (gNB or g B node), Access Point (AP), Distributed Unit (DU), carrier, or Transmission Reception Point (TRP) may be used interchangeably. In some examples, the cell may not necessarily be stationary, and the geographic area of the cell may move according to the location of the mobile BS.

A UE may also be referred to as a mobile station, a terminal, an access terminal, a subscriber unit, a station, a client equipment (CPE), a cellular phone, a smartphone, a Personal Digital Assistant (PDA), a wireless modem, a wireless communication device, a handheld device, a laptop, a cordless phone, a Wireless Local Loop (WLL) station, a tablet computer, a camera, a gaming device, a netbook, a smartbook, an ultrabook, an appliance, a medical device or medical equipment, a biometric sensor/device, a wearable device (such as a smartwatch, a smart garment, smart glasses, a smart wristband, smart jewelry (e.g., smart ring, smart bracelet, etc.)), an entertainment device (e.g., a music device, a video device, a satellite radio, etc.), a vehicle component or sensor, a smart meter/sensor, industrial manufacturing equipment, a global positioning system device, a satellite radio, etc, Or any other suitable device configured to communicate via a wireless or wired medium. Some UEs may be considered Machine Type Communication (MTC) devices or evolved MTC (emtc) devices. MTC and eMTC UEs include, for example, a robot, a drone, a remote device, a sensor, a meter, a monitor, a location tag, etc., which may communicate with a BS, another device (e.g., a remote device), or some other entity. A node, such as a wireless node, may provide connectivity for or to a network, e.g., a wide area network (such as the internet) or a cellular network, e.g., via a wired or wireless communication link. Some UEs may be considered internet of things (IoT) devices, which may be narrowband IoT (NB-IoT) devices.

The techniques described herein may be used for the above-mentioned wireless networks and radio technologies as well as other wireless networks and radio technologies. For clarity, although aspects may be described herein using terms commonly associated with 3G, 4G, and/or 5G wireless technologies, aspects of the present disclosure may be applied in other generation-based communication systems.

The previous description is provided to enable any person skilled in the art to practice the various aspects described herein. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects. Thus, the claims are not intended to be limited to the aspects shown herein, but is to be accorded the full scope consistent with the language claims, wherein reference to an element in the singular is not intended to mean "one and only one" (unless specifically so stated) but rather "one or more". The term "some" or "an" refers to one or more, unless specifically stated otherwise. All structural and functional equivalents to the elements of the various aspects described throughout this disclosure that are known or later come to be known to those of ordinary skill in the art are expressly incorporated herein by reference and are intended to be encompassed by the claims. Moreover, nothing disclosed herein is intended to be dedicated to the public regardless of whether such disclosure is explicitly recited in the claims. No element of the claims should be construed under the provisions of 35 u.s.c. § 112(f), unless the element is explicitly recited using the phrase "means for … …" or in the case of method claims the element is recited using the phrase "step for … …".

The various operations of the methods described above may be performed by any suitable means capable of performing the corresponding functions. These means may include various hardware and/or software components and/or modules, including but not limited to, circuits, Application Specific Integrated Circuits (ASICs), or processors. Generally, where there are operations illustrated in the figures, the operations may have corresponding counterpart means plus functional components with similar numbering.

The various illustrative logical blocks, modules, and circuits described in connection with the disclosure may be implemented or performed with a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable Logic Device (PLD), discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but in the alternative, the processor may be any commercially available processor, controller, microcontroller or state machine. A processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.

If implemented in hardware, an example hardware configuration may include a processing system in a node. The processing system may be implemented with a bus architecture. The bus may include any number of interconnecting buses and bridges depending on the specific application of the processing system and the overall design constraints. The bus may link together various circuits including the processor, the machine-readable medium, and the bus interface. A bus interface may be used to connect a network adapter or the like to the processing system via the bus. A network adapter may be used to implement the signal processing functions of the PHY layer. In the case of UE 120a (see fig. 1), a user interface (e.g., keypad, display, mouse, joystick, etc.) may also be connected to the bus. The bus may also link various other circuits such as timing sources, peripherals, voltage regulators, power management circuits, and the like, which are well known in the art, and therefore, will not be described any further. A processor may be implemented with one or more general and/or special purpose processors. Examples include microprocessors, microcontrollers, DSP processors, and other circuitry capable of executing software. Those skilled in the art will recognize how best to implement the functionality described with respect to the processing system, depending on the particular application and the overall design constraints imposed on the overall system.

If implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Software should be construed broadly to mean instructions, data, or any combination thereof, whether referred to as software, firmware, middleware, microcode, hardware description language, or otherwise. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. The processor may be responsible for managing the bus and general processing, including the execution of software modules stored on the machine-readable storage medium. A computer readable storage medium may be coupled to the processor such that the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium may be integral to the processor. By way of example, a machine-readable medium may include a transmission line, a carrier wave modulated by data, and/or a computer-readable storage medium separate from the node having instructions stored thereon, all of which may be accessed by a processor through a bus interface. Alternatively or additionally, the machine-readable medium or any portion thereof may be integrated into a processor, such as a cache and/or a general register file, as may be the case. Examples of a machine-readable storage medium may include RAM (random access memory), flash memory, ROM (read only memory), PROM (programmable read only memory), EPROM (erasable programmable read only memory), EEPROM (electrically erasable programmable read only memory), registers, magnetic disk, optical disk, hard drive, or any other suitable storage medium, or any combination thereof, as examples. The machine-readable medium may be embodied in a computer program product.

A software module may comprise a single instruction, or many instructions, and may be distributed over several different code segments, among different programs, and across multiple storage media. The computer readable medium may include several software modules. These software modules include instructions that, when executed by an apparatus, such as a processor, cause a processing system to perform various functions. These software modules may include a transmitting module and a receiving module. Each software module may reside in a single storage device or be distributed across multiple storage devices. As an example, a software module may be loaded into RAM from a hard drive when a triggering event occurs. During execution of the software module, the processor may load some instructions into the cache to increase access speed. One or more cache lines may then be loaded into a general register file for execution by the processor. When referring to the functionality of a software module below, it will be understood that such functionality is implemented by the processor when executing instructions from the software module.

Any connection is properly termed a computer-readable medium. For example, if the software is transmitted from a web site, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL), or wireless technologies such as Infrared (IR), radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. Disk (disk) and disc (disc), as used herein, includes Compact Disc (CD), laser disc, optical discDigital Versatile Disk (DVD), floppy disk, and blue-rayDisks, where a disk (disk) usually reproduces data magnetically, and a disk (disc) reproduces data optically with a laser. Thus, in some aspects, computer-readable media may comprise non-transitory computer-readable media (e.g., tangible media). Additionally, for other aspects, the computer-readable medium may comprise a transitory computer-readable medium (e.g., a signal). Combinations of the above should also be included within the scope of computer-readable media.

Thus, certain aspects may comprise a computer program product for performing the operations presented herein. For example, such a computer program product may include a computer-readable medium having stored (and/or encoded) thereon instructions executable by one or more processors to perform operations described herein, such as instructions for performing operations described herein and illustrated in fig. 8.

Further, it is to be appreciated that modules and/or other appropriate means for performing the methods and techniques described herein can be downloaded and/or otherwise obtained by a user terminal and/or base station where applicable. For example, such a device can be coupled to a server to facilitate the transfer of an apparatus for performing the methods described herein. Alternatively, the various methods described herein can be provided via a storage device (e.g., RAM, ROM, a physical storage medium such as a Compact Disc (CD) or floppy disk, etc.), such that the apparatus can obtain the various methods upon coupling or providing the storage device to a user terminal and/or base station. Further, any other suitable technique suitable for providing the methods and techniques described herein to a device may be utilized.

It is to be understood that the claims are not limited to the precise configuration and components illustrated above. Various changes, substitutions and alterations in the arrangement, operation and details of the method and apparatus described above may be made without departing from the scope of the claims.