阅读说明：本技术 用于优化移动设备中视频通信的功率消耗的系统和方法 (System and method for optimizing power consumption for video communications in a mobile device ) 是由 纳拉辛汉·维杰·阿南德 于 2021-04-14 设计创作，主要内容包括：本发明提供了用于优化移动设备中视频通信的功率消耗的系统和方法。系统(100)包括视频编解码器编码器模块(101)、视频编解码器解码器模块(106)以及后处理滤波模块(解块滤波器)(103、107)模块。后处理模块是在DSP/VEIW处理器上被实施的,而视频编码器和解码器模块是在具有SIMD扩展的CPU上被实施的。与上述模块在单个/多个DSP/VLIW核中的实施相比,模块在多核中的这种流水线式实施使SoC中的电流消耗减少最高至50％。模块(101、102、106)在电流消耗上的显著减少使得能够减少视频呼叫时间中的功率消耗。因此,本发明提供了通过模块的多核实施来优化在移动设备中视频呼叫的功率消耗的简单方法。(The present invention provides systems and methods for optimizing power consumption for video communications in a mobile device. The system (100) includes a video codec encoder module (101), a video codec decoder module (106), and a post-processing filtering module (deblocking filter) (103, 107) module. The post-processing module is implemented on a DSP/VEIW processor, while the video encoder and decoder modules are implemented on a CPU with SIMD extensions. Such a pipelined implementation of modules in multiple cores reduces the current consumption in the SoC by up to 50% compared to an implementation of the above modules in single/multiple DSP/VLIW cores. The significant reduction in current consumption by the modules (101, 102, 106) enables a reduction in power consumption during video call times. The present invention thus provides a simple method of optimizing power consumption of a video call in a mobile device through multi-core implementation of the module.)

1. A system (100) for optimizing power consumption for video communication in a mobile device, the system (100) comprising:

a. a camera (101) integrated in the mobile device, the camera receiving and recording input video frame samples at 8 bit resolution, 10 bit resolution, or any resolution;

b. a video codec encoder module (102) configured to compress/encode a video signal for efficient transmission over a wireless channel, wherein the above implementation is performed on a CPU with SIMD extensions, wherein the encoded signal is transmitted via a network (105);

c. a filter module including, but not limited to, a deblocking filter (103), wherein the filter module (103) receives previously processed video frame samples from an encoder (101), wherein the filter module (103) is implemented in a DSP/VLIW processor;

d. a video codec decoder module (106) configured to decompress/decode a compressed video signal received from a network (105), wherein the video codec decoder is implemented on a CPU with SIMD extensions, wherein the video codec decoder module (106) resides at a receiving end;

e. a filtering module, including but not limited to a deblocking filter (107), wherein decoded video frame samples are filtered on a DSP/VLIW core;

f. turning off saturation in the video codec encoder/decoder module; wherein the encoder/decoder module is implemented on a CPU with SIMD extensions;

soc is designed with a CPU with SIMD extensions and a DSP/VLIW processing core to enable pipelined implementation of the video communication module between the two cores; wherein there is no saturation designed in the Arithmetic Logic Unit (ALU) of the CPU in critical instructions including, but not limited to, multiply-accumulate (MAC) and shift instructions, and

h. a video receiver (108) configured to receive video frame samples from a filtering module (107).

2. The system of claim 1, wherein the video codec modules (102, 106) include, but are not limited to, MPEG-1, MPEG-2, MPEG-4, h.264, h.265, AV1, VP8, VP10 standard video codecs.

3. The system of claim 1, wherein the mobile device comprises a portable cellular telephone, a mobile handheld terminal, a mobile telephone, a wireless telephone, a cellular telephone, a portable telephone, a Personal Digital Assistant (PDA), and a smartphone.

4. The system of claim 1, wherein optimizing power consumption is applicable to video playback and video recording without transmission.

5. The system of claim 1, wherein the current consumption for video communication is reduced by up to 50% compared to implementing all modules (video codec encoder, video codec decoder, filtering (deblocking filter), and post-processing) on a single or multiple DSP/VLIW processors in the SoC.

6. A method for optimizing power consumption in a mobile device, the method comprising the steps of:

a. a camera (101) integrated in the mobile device, the camera receiving and recording input video frame samples at 8 bit resolution, 10 bit resolution, or any resolution;

b. a video codec encoder module (102) configured to compress/encode a video signal for efficient transmission over a wireless channel, wherein the above implementation is performed on a CPU with SIMD extensions, wherein the encoded signal is transmitted via a network (105);

c. a filter module including, but not limited to, a deblocking filter (103), wherein the filter module (103) receives previously processed video frame samples from the encoder (101), wherein the filter module (103) is implemented in a DSP/VLIW processor;

d. a video codec decoder module (106) configured to decompress/decode a compressed video signal received from a network (105), wherein the video codec decoder is implemented on a CPU with SIMD extensions, wherein the video codec decoder module (106) resides at a receiving end;

e. a filtering module, including but not limited to a deblocking filter (107), wherein decoded video frame samples are filtered on a DSP/VLIW core;

f. turning off saturation in the video codec encoder/decoder module; wherein the encoder/decoder module is implemented on a CPU with SIMD extensions;

soc is designed with a CPU with SIMD extensions and a DSP/VLIW processing core to enable pipelined implementation of the video communication module between the two cores; wherein there is no saturation designed in the Arithmetic Logic Unit (ALU) of the CPU in critical instructions including, but not limited to, multiply-accumulate (MAC) and shift instructions, and

h. a video receiver (108) configured to receive video frame samples from a filtering module (107).

7. The method of claim 6, wherein the video codec module (102, 106) comprises, but is not limited to, an MPEG-1, MPEG-2, MPEG-4, h.264, h.265, AV1, VP8, VP10 standard video codec.

8. The method of claim 7, wherein the mobile device comprises a portable cellular telephone, a mobile handheld terminal, a mobile telephone, a wireless telephone, a cellular telephone, a portable telephone, a Personal Digital Assistant (PDA), and a smartphone.

Technical Field

The present invention relates to a system and method for optimizing power consumption in a mobile device. More particularly, the present invention relates to reducing current consumption in video communications by implementing a video codec on a Central Processing Unit (CPU) with Single Instruction Multiple Data (SIMD) extensions and a post-processing module including a filter (deblocking filtering, which is part of the codec) on a Digital Signal Processor (DSP) core, thereby optimizing power consumption in mobile devices.

Background

With the adoption of 5G technology that provides the high data rates of cores, video communication that is typically characterized by such high data rates is rapidly becoming possible. There are different small devices, such as mobile devices, smart phones and other portable wireless devices, that will be designed to employ 5G technology.

Video communication involves the encoding (compression) and decoding (decompression) of digital video data. There are different video compression standards such as MPEG-1, MPEG-2, MPEG-4, h.264, h.265, AV1, VP8, VP10 and many more upcoming standards. These video codecs encode video signals by exploiting spatial redundancy (intra prediction), temporal redundancy (motion compensation), statistical redundancy of the coded symbols (variable length coding (VLC), context adaptive VLC (cavlc), Context Adaptive Binary Arithmetic Coding (CABAC)). The computational complexity of processing these modules requires a large number of processing cycles, and thus, during video communications conducted in battery-driven accessories/devices, such as smartphone mobile devices, video encoder/decoder operations consume a large amount of current.

The high load on the processor in video communication causes a large amount of power consumption, and thus the battery life used by the device in video communication is greatly reduced.

For all wireless communication systems, it is important to minimize power consumption and/or improve data rates and user experience in User Equipment (UE) devices. As UE devices become more complex, they increasingly consume greater amounts of power. The UE device has an onboard battery of limited capacity. Therefore, there is a problem of obtaining the best possible user experience under the constraint of a limited battery.

US10390309 entitled "System and method for optimizing power consumption in a mobile device" discloses a method and apparatus for optimizing power consumption in a mobile device by appropriate instruction set architectural feature changes and optimal implementation of a speech codec. However, this scheme is mainly directed to the voice call use case.

Accordingly, there is a need for a system and method for optimizing power consumption in a mobile device for optimizing power consumption in video communications.

Disclosure of Invention

The present invention overcomes the disadvantages of the prior art and provides a system and method for optimizing power consumption for communications in a mobile device.

The system includes a camera integrated with the mobile device. The camera sensor captures the input video and converts to digital video with a typical 8-bit pixel size. Now, a higher resolution camera sensor capable of capturing 10 bit pixel sizes can be used.

In embodiments of the invention, the digital video signal is encoded according to the compression standard h.264 or any other suitable standard for applications. Different encoding tools such as intra prediction, motion compensation, variable length encoding are implemented in a CPU with SIMD extended instruction sets but without critical single cycle instruction product accumulation (MAC). A post-processing module including a codec module deblocking filter is implemented on the DSP. By implementing the deblocking filter and post-processing modules on a DSP/VLIW core in a system-on-a-chip (SoC), the current consumption of the SoC can be reduced, while the video encoder is implemented in a CPU with SIMD extensions. The encoded signal is then transmitted over a network. The system also includes a video codec decoder module disposed at the receiving end. The video codec decoder module is configured to decompress/decode a compressed video signal received from a network. The decoded video signal is then post-processed using a deblocking filter module. The post-processing module is implemented in a DSP/VLIW core, while the video codec decoder is implemented in a CPU with SIMD extensions.

Accordingly, the present invention also provides a method of improving power consumption for a video call in a mobile device. The method results in up to 50% savings in current consumption for video calls compared to video codecs and post-processing modules implemented on Digital Signal Processors (DSPs)/Very Long Instruction Word (VLIW) processors.

Video talk time is increased with less power consumption, reducing thermal aspects and extending battery life.

Drawings

The foregoing and other features of the embodiments will become more apparent from the following detailed description of the embodiments, when read in conjunction with the accompanying drawings. In the drawings, like numbering represents like elements.

FIG. 1 illustrates a block diagram of a system for optimizing power consumption for video communications in a mobile device, according to one embodiment of the invention.

Fig. 2 illustrates a method for optimizing power consumption in a mobile device according to one embodiment of the invention.

Fig. 3 illustrates a method for optimizing power consumption in a mobile device according to one embodiment of the invention.

Detailed Description

Reference will now be made in detail to the description of the present subject matter, one or more examples of which are illustrated in the figures. Each example is provided to explain the subject matter and not as a limitation. Various changes and modifications apparent to those skilled in the art to which the invention pertains are deemed to be within the spirit, scope and concept of the invention.

To more clearly and concisely describe and point out the subject matter of the claimed invention, the following definitions are provided for specific terms, which are used in the following written description.

The term "talk time" refers to the time a mobile phone/mobile device is used to process a video call, in particular as a measure of the duration of the battery of the phone/mobile device.

The present invention provides systems and methods for optimizing power consumption for video communications in a mobile device. The system includes a camera, a video codec encoder, a video codec decoder, a video post-processing module including, but not limited to, a deblocking filter. Pipelined implementations of video codec and post-processing modules (deblocking filters) in a CPU with SIMD extensions and DSP/VLIW, respectively, result in a reduction of power consumption in talk time by up to 50% compared to implementing all these modules in a single or multiple DSP/VLIW processors.

FIG. 1 illustrates a block diagram of a system for optimizing power consumption for video communications in a mobile device, according to one embodiment of the invention. In a preferred embodiment, the system includes a camera (101) integrated with the mobile device. The camera (101) is configured to receive input video and to feed video frame samples to a video encoder module (102). The processed video samples are then passed to a deblocking filter (post-processing module) (103). The video samples are encoded using the encoding tools described in the video codec standard (104).

At the transmitting end, the video frame samples are fed to a video codec encoder module (101). The video codec encoder module is configured to compress/encode a video signal running on a processor, CPU with SIMD extensions. Different coding tools of the digital video compression standard specification, i.e. intra prediction, motion estimation/inter prediction, transform, quantization, bitstream coding including VLC, CAVLC, CABAC, are implemented on a CPU with SIMD extensions. The processing of video frames is pipelined between the CPU and the DSP/VLIW, with deblocking filters implemented on the DSP/VLIW core. Current consumption in the SoC is reduced by up to 50% in the transmission path (102, 103, 104) compared to module implementations on single or multiple DSP/VLIW cores. The encoded signal is then transmitted over a transmission network (105).

In addition, a video codec decoder module (106) is present at the receiving end. The video codec decoder module (106) is configured to decompress/decode a received compressed video signal (running on a CPU with SIMD extensions). The decoded video samples are then post-processed (deblocking filter). A post-processing module (deblocking filter) (107) is implemented in the DSP/VLIW processor. This pipelined implementation results in reduced current consumption (up to 50%) by the mobile device.

Fig. 2 in an embodiment of the invention, pipelined implementation of video codec and post-processing (deblocking filter) modules on different processing cores CPU and DSP/VLIW cores, respectively, results in a reduction of current consumption up to 50% (102, 103, 104 and 106, 107). The architecture of the SoC includes a DSP/VLIW and a CPU with SIMD extensions. Thus, low power video communication is achieved using the present invention.

Fig. 3 illustrates a method for optimizing power consumption for video communication in a mobile device according to one embodiment of the invention. In a preferred embodiment, the method begins with the step of receiving and recording raw video at step 301.

At step 302, the video frame samples are compressed/encoded by the video codec encoder module. The video encoder is implemented on a CPU with SIMD extensions. At step 303, deblocking filters and other post-processing are implemented on the DSP/VLIW core. With such a multi-core implementation (102, 103, 104), current consumption in video calls is reduced by up to 50%. The encoded signal is transmitted via a network (105).

At step 304, a compressed video signal is received from a network. Where they are decompressed/decoded by a video codec decoder (106).

At step 305, the decoded video is deblock filtered and post-processed (107) to obtain an output video frame. The post-processed video frame samples are then displayed (108).

The inventive in a video encoder/decoder module implemented on a CPU with SIMD extensions is now described. The video frame samples are typically 8 bits. Now supporting 10-bit video samples. In an intra prediction module, the intra block size may vary between 4x4 and 32x32 depending on the video standard. The reference samples are also 8 bits or 10 bits and are filtered. Its output is still 8 bits or 10 bits and can fit within the 16-bit range (span) of a 32-bit register. The filtering operation is a 3-tap filter and then downscaling is performed with intermediate results still in the 16-bit range. Therefore, it is safe to turn off saturation (saturation), and the instruction set may have MAC instructions embedded therein without saturation and SIMD optimization is possible. Thus, the input to the transform module in the intra prediction module is 9 or 11 bits. The transformation module is implemented to ensure that the intermediate results and the output results do not cross the 16-bit range. Therefore, saturation can be turned off again and embedded MAC instructions without saturation are useful, and SIMD optimization is possible. In the inter-block coding tool, the input pixel value is 8 or 10 bits. Based on the Sum of Absolute Differences (SAD), motion estimation can be performed efficiently and correctly so that the motion estimation does not cross the 16-bit range. The input to transform between coded blocks is 9 or 11 bits and can also be processed within 16 bits of the intermediate/final result. Thus, saturation can be turned off, enabling Instruction Set Architecture (ISA) to have MAC embedded in instructions without saturation, and SIMD optimization is possible, thus saving material list (BOM) overhead and giving power savings in optimization.

Accordingly, the present invention provides a method of improving power consumption for video communications in a mobile device. Talk time is increased with less power consumption, reducing thermal aspects and extending battery life.