阅读说明：本技术 用于检测视频内容的基于硬件或基于软件的解码的配置 (Configuration for detecting hardware-based or software-based decoding of video content ) 是由 D·L·普雷斯特加德 J·莱斯 M·阿拉纳 于 2019-08-28 设计创作，主要内容包括：过程从媒体内容服务器接收媒体内容。此外,该过程利用媒体回放设备处的处理器对与媒体内容对应的一个或多个样本文件执行第一回放。一个或多个样本文件均对应于不同的编解码器格式。另外,该过程利用媒体回放设备处的处理器确定一个或多个样本文件中的每一个的置信度得分。置信度得分与指示不同的编解码器格式是基于软件的或基于硬件的的一个或多个因素相关联。最后,该过程利用媒体回放设备处的处理器根据基于硬件的不同的编解码器格式对媒体内容的至少大部分执行第二回放。(The process receives media content from a media content server. Further, the process performs, with a processor at the media playback device, a first playback of one or more sample files corresponding to the media content. The one or more sample files each correspond to a different codec format. In addition, the process determines, with a processor at the media playback device, a confidence score for each of the one or more sample files. The confidence scores are associated with one or more factors indicating that the different codec formats are software-based or hardware-based. Finally, the process performs a second playback of at least a majority of the media content according to the hardware-based disparate codec format with a processor at the media playback device.)

1. A computer program product comprising a non-transitory computer readable storage device having a computer readable program stored thereon, wherein the computer readable program when executed on a computer causes the computer to:

receiving media content from a media content server;

performing, with a processor at the media playback device, a first playback on one or more sample files corresponding to the media content, the one or more sample files each corresponding to a different codec format;

determining, with a processor at the media playback device, a confidence score for each of the one or more sample files, the confidence score associated with one or more factors indicating that different codec formats are software-based or hardware-based; and

performing, with a processor at the media playback device, a second playback of at least a majority of the media content according to a hardware-based disparate codec format.

2. The computer program product of claim 1, wherein the first playback occurs before the second playback.

3. The computer program product of claim 1, wherein the one or more factors comprise utilization of resources by the processor.

4. The computer program product of claim 3, wherein the confidence score is weighted according to utilization of resources by the processor.

5. The computer program product of claim 1, wherein the one or more factors include memory constraints.

6. The computer program product of claim 1, wherein the one or more sample files each have a duration that is less than a duration of the media content.

7. The computer program product of claim 1, wherein the one or more sample files each have a high resolution.

8. The computer program product of claim 1, wherein the different codec format is selected from the group consisting of: HEVC, AVC, and AV 1.

9. The computer program product of claim 1, wherein during the first playback, the first playback is invisible to a user viewing a display device in operable communication with the media playback device.

10. The computer program product of claim 9, wherein the computer is further caused to render an overlay on the first playback such that the first playback is invisible.

11. The computer program product of claim 10, wherein the overlay is selected from the group consisting of: a login screen, a diagnostic screen, and imagery associated with the media content.

12. A computer program product comprising a non-transitory computer readable storage device having a computer readable program stored thereon, wherein the computer readable program when executed on a computer causes the computer to:

receiving media content from a media content server;

performing, with a processor at the media playback device, a first playback of a first portion of the media content, the first portion of the media content corresponding to a first codec format;

determining, with a processor at the media playback device, a confidence score for a second portion of the media content after the first playback start, the confidence score associated with one or more factors indicating that the second portion corresponds to a second software-based or hardware-based codec format that is different from the first codec format;

interrupting, with the processor, the first playback; and

performing, with a processor at the media playback device, a second playback of a second portion of the media content according to the second codec format.

13. The computer program product of claim 12, wherein the first playback occurs before the second playback.

14. The computer program product of claim 12, wherein the one or more factors include utilization of resources by the processor.

15. The computer program product of claim 14, wherein the confidence score is weighted according to utilization of resources by the processor.

16. The computer program product of claim 12, wherein the one or more factors include memory constraints.

17. An apparatus, comprising:

a processor that receives media content from a media content server, performs a first playback of one or more sample files corresponding to the media content, determines a confidence score for each of the one or more sample files, performs a second playback of at least a majority of the media content according to a hardware-based disparate codec format, the one or more sample files each corresponding to a disparate codec format, the confidence score associated with one or more factors indicating that the disparate codec format is software-based or hardware-based.

18. The apparatus of claim 17, wherein the processor is located within a media playback device.

19. The apparatus of claim 17, wherein the different codec format is selected from the group consisting of: HEVC, AVC, and AV 1.

20. The apparatus of claim 17, wherein the one or more factors comprise utilization of resources by the processor.

Technical Field

The present disclosure relates generally to the field of video content.

Background

When receiving video content for playback (e.g., via streaming, downloading, etc.), the playback device typically receives the video content in an encoded format. In addition, the playback device executes a codec to decode the video content for playback. With the advent of higher resolution display devices (e.g., 4K televisions), video content developers often want to know what codec format to encode video content that is sent to users via playback devices. Thus, current playback devices may perform decoding according to a codec selected by a developer, which may not be consistent with the optimal codec for the particular playback device. Therefore, current playback devices are not configured to perform optimal playback of video content.

Disclosure of Invention

In one aspect, a computer program product includes a non-transitory computer readable storage device having a computer readable program stored thereon. The computer readable program when executed on a computer causes the computer to receive media content from a media content server. Further, the computer is caused to perform, with a processor at the media playback device, a first playback on one or more sample files corresponding to the media content. The one or more sample files each correspond to a different codec format. Additionally, the computer is caused to determine, with a processor at the media playback device, a confidence score for each of the one or more sample files. The confidence scores are associated with one or more factors indicating that the different codec formats are software-based or hardware-based. Finally, the computer is caused to perform, with a processor at the media playback device, a second playback of at least a majority of the media content according to the hardware-based disparate codec format. In an alternative aspect, the first playback may be interrupted with the processor before the second playback is performed according to a different codec format.

In another aspect, the functions performed by the computer readable program may be implemented by an apparatus. In yet another aspect, the functions performed by the computer-readable program may be implemented by a process.

Drawings

The above features of the present disclosure will become more apparent with reference to the following description taken in conjunction with the accompanying drawings, in which like reference numerals denote like elements, and in which:

fig. 1 shows a block diagram of internal components of a media playback device that determines an optimal state for playing back content.

Fig. 2 illustrates a system for decoding media content using the media playback device shown in fig. 1.

Fig. 3A illustrates an example of one of the sample media files shown in fig. 1 being played back by a media playback device to determine a status.

Fig. 3B illustrates an overlay that may be positioned over the sample media file shown in fig. 3A during playback of the sample media file.

FIG. 4 illustrates a process that may be used to determine a confidence score for one or more of the sample media files shown in FIG. 1.

FIG. 5 illustrates another process that may be used to determine a confidence score for one or more of the sample media files shown in FIG. 1.

Detailed Description

The configuration detects whether the media player is performing hardware-based native (native) decoding of video content (e.g., via hardware installed on a device chipset) or software-based decoding. By determining the hardware-based capabilities of the respective media playback device (e.g., mobile device, connected television, blu-ray player, personal computer, virtual reality apparatus, augmented reality apparatus, etc.), the media player can then determine whether to use hardware-based decoding or software-based decoding to optimize the functionality and viewer experience of the playback device.

Fig. 1 shows a block diagram of the internal components of a media playback device 100 that determines the best state (i.e., hardware-based or software-based decoding) for playing back content. The media playback device 100 may have a central processing unit ("CPU") 101, a graphics processor unit ("GPU") 108, a memory device 102, a transceiver 104, an input/output ("I/O") device 103 (e.g., a remote control, a mouse, a joystick, a keyboard, etc.), and a data storage device 105.

Further, the data storage device 105 may store media content 106, a media player 107, status detection code 108, and/or a sample media file 109. Examples of media content 106 include, but are not limited to, movies, television programs, documents, video games, music, advertisements, and the like. Additionally, one or more sample media files 109 (each corresponding to a particular codec format (e.g., high efficiency video coding ("HEVC"), advanced video coding ("MPEG-4 AVC"), AOMedia video 1 ("AV 1"), etc.) may be part of the media content 106 or different from the media content 106. the one or more sample files 109 may be associated with particular media content for playback.

Further, the state detection code 108 may be executed by the CPU101 to determine the best state (i.e., a particular codec format) for playback of the particular media content 106. For example, the CPU101 may execute the state detection code 108 to analyze one or more sample files 109 in various codec formats. As a result, the CPU101 may determine the optimal codec format (i.e., native hardware based or software based) for a particular media player 107 and media playback device 100. For example, the CPU101 may determine that AVC is a native hardware-based codec format for the media player 107 and the media playback device 100, while HEVC is a software-based codec format. Furthermore, the CPU101 may determine that performing video content decoding using HEVC will result in significantly more battery consumption by the media playback device 100 than AVC, since software-based decoding is typically more battery-consuming than hardware-based decoding. Accordingly, the CPU101 may execute the media player 107 via a hardware-based codec (e.g., AVC) at a lower resolution than a software-based codec (e.g., HEVC) to improve the functionality of the media player 107 and the media playback device 100.

In contrast, previous media players were not configured to determine the type of codec native to hardware but what codec is software-based. The reason that previous media players lack this knowledge is that the chipsets for two identical playback devices may be configured with different codec formats. For example, a hardware manufacturer may have implemented HEVC as a native hardware-based codec format on a first set of playback devices, but then implemented AVC as a native hardware-based codec format on a second set of playback devices. Thus, previous media players typically do not know the particular codec format native to the hardware of the previous playback device. As a result, previous media players are often operated in an energy-inefficient and memory-inefficient manner.

On the other hand, the media player 107 shown in fig. 1 improves the functions of the CPU101 and the memory device 102 by determining the optimal codec format. For example, the CPU101 may execute the state detection code 108 to run various queries/polls on system resources (e.g., CPU utilization, memory constraints, etc.) used by the media playback device 100 during playback of one or more sample media files 109. In one aspect, the CPU101 executes the state detection code 108 to calculate a score based on one or more factors (e.g., system resources) that are affected during playback of one or more sample media files 109.

The one or more sample media files 109 may be configured to provide a large level of computational intensity over a short duration relative to the length of the media content 106. For example, the one or more sample media files 109 may be high resolution files (e.g., 4K, 1080i, 720p, etc.), such that the difference in resource execution of a software-based codec as compared to a hardware-based codec would be apparent.

Examples of one or more factors include, but are not limited to, CPU utilization, memory constraints, data received from an operating system of the media playback device 100, a version of the operating system, a model of the media playback device 100, a golden process (i.e., a minimum functional test to determine distortion), data received from a thermal sensor (e.g., CPU temperature), a type of the media content 106 (e.g., primary content or additions), and/or a length of the media content 106. For example, software-based codec execution is typically associated with CPU spikes, as compared to native hardware-based. Thus, when the state detection code 108 is executed on various sample media files 109, the CPU101 can determine which of the sample media files 109 are causing CPU spikes and which are not. This factor may be used to generate a confidence score (i.e., a confidence level at which a particular sample media file 109 is encoded according to a native hardware-based or software-based codec), among one or more other factors.

In one aspect, the confidence score may be determined by tabulating (tabulates) sub-scores associated with each factor. In another aspect, the sub-scores may be weighted. For example, a CPU utilization factor may be assigned a higher weight than the model of the media playback device 100. Further, a predetermined tolerance threshold may be established for the confidence score (i.e., the sum of the sub-scores, weighted or unweighted). The codec format for the sample media file 109 may be considered software-based if the sample media file 109 has a confidence score that exceeds a predetermined tolerance threshold; in contrast to being hardware based if below a predetermined tolerance threshold.

In one aspect, the CPU101 executes the state detection code 108 to determine a state prior to playback of the media content 106 or at least a substantial portion of the media content 106. In another aspect, the CPU101 may continue to execute the state detection code 108 during playback of the media content 106. For example, the media content 106 may be streamed to the media playback device 100 such that a first portion is encoded according to HEVC and a second portion is encoded according to AVC. The CPU101 may monitor the playback of the media content 106 to determine whether the confidence score exceeds a predetermined tolerance threshold. For example, CPU utilization spikes may occur after a change from native hardware-based codec format halfway from HEVC to AVC. Since the media player 107 may be using the HEVC codec to avoid battery drain, as determined by sample media file 109 analysis, for example, the media player 107 may still use the HEVC codec when the factors change part way through. Thus, the CPU101 can determine such changes and modify the state by continuous monitoring.

Fig. 2 illustrates a system 200 for decoding media content 106 using the media playback device 100 shown in fig. 1. In one aspect, the media playback device 100 receives encoded media content 106 from a remote media content server 201 via a computerized network 202. Further, the media playback device 100 decodes the encoded media content 106 for display on a display device 204 (e.g., a television) in operable communication with the media playback device 100. For example, the display device 204 may be a device separate from the media playback device 100 that provides output of the decoded media content (e.g., via a wired or wireless connection). Alternatively, the display device 204 may be integrated in the media playback device 100. In another aspect, the media playback device 100 receives encoded media content locally (e.g., via DVD insertion, USB connection, etc.).

Fig. 3A illustrates an example of one of the sample media files 109 shown in fig. 1 being played back by the media playback device 100 to determine a state. For example, the sample media file 109 may be a high-resolution or other resource-intensive portion of the media content 106.

In addition, fig. 3B shows an overlay 301 that may be positioned over the sample media file 109 shown in fig. 3A during playback of the sample media file 109. In other words, the media playback device 100 may hide playback of the sample media file 109 via the overlay 301 such that the user 203 (fig. 2) does not have to view the sample media file 109 during the media playback device 100 determination state. For example, the overlay 301 may be a login screen, a diagnostic image, a graphic (e.g., a character image) corresponding to the media content 106, and the like.

Fig. 4 illustrates a process 400 that may be used to determine a confidence score for one or more of the sample media files 109 shown in fig. 1. At process block 401, process 400 receives media content 106 (fig. 1) from media content server 201 (fig. 2). Further, at process block 402, the process 400 performs, with the processor 101 at the media playback device 100, a first playback of one or more sample files 109 corresponding to the media content 106. The one or more sample files each correspond to a different codec format. Additionally, at process block 403, the process 400 determines, with the processor 101 at the media playback device 100, a confidence score for each of the one or more sample files 109. The confidence scores are associated with one or more factors indicating that the different codec formats are software-based or hardware-based. Finally, at process block 404, the process 400 performs a second playback of at least a majority of the media content 106 according to the hardware-based disparate codec format with the processor 101 at the media playback device 100.

Fig. 5 illustrates another process 500 that may be used to determine a confidence score for one or more of the sample media files 109 shown in fig. 1. At process block 501, process 500 receives media content from media content server 201 (FIG. 2). Further, at process block 502, the process 500 performs a first playback of a first portion of media content with the processor 101 at the media playback device 100. The first portion of the media content 106 corresponds to a first codec format. Additionally, at process block 503, the process 500 determines, with the processor 101 at the media playback device 100, a confidence score for the second portion of the media content 106 after the first playback has begun. The confidence score is associated with one or more factors indicating that the second portion corresponds to a second software-based or hardware-based codec format. The second codec format is different from the first codec format.

Further, at process block 504, the process 500 interrupts the first playback with the processor 101. Finally, at process block 505, the process 500 performs a second playback of the second portion of the media content according to a second codec format with the processor 101 at the media playback device 100.

The processes described herein may be implemented in a dedicated, general purpose, multi-purpose, or single purpose processor. Such a processor will execute instructions at the assembly, compilation or machine level to perform the processing. These instructions may be written by one of ordinary skill in the art after a description of the figures corresponding to the processes and stored or transmitted on a computer-readable medium. The instructions may also be created using source code or any other known computer-aided design tool. The computer readable medium can be any medium capable of carrying those instructions, such as a computer readable storage device, and includes CD-ROM, DVD, magnetic or other optical disks, tapes, silicon memory (e.g., removable, non-removable, volatile or non-volatile), packetized or non-packetized data for wired or wireless transmission over a network, either locally or remotely. A computer herein is intended to include any device having a general, multi-purpose, or single-purpose processor, as described above. For example, the computer may be a desktop computer, a laptop computer, a smart phone, a tablet device, a set-top box, and so forth.

It should be understood that the devices, systems, computer program products, and processes described herein may also be applied to other types of devices, systems, computer program products, and processes. Those skilled in the art will appreciate that various adaptations and modifications of the aspects of the apparatus, systems, computer program products, and processes described herein may be configured without departing from the scope and spirit of the apparatus, systems, computer program products, and processes. It is therefore to be understood that within the scope of the appended claims, the inventive apparatus, system, computer program product, and process may be practiced otherwise than as specifically described herein.