阅读说明：本技术 一种基于脑电特征的视频播放质量评价方法及装置 (Video playing quality evaluation method and device based on electroencephalogram characteristics ) 是由 陶晓明 李哲 耿冰蕊 段一平 高定成 胡舒展 于 2021-08-09 设计创作，主要内容包括：本申请提供了一种基于脑电特征的视频播放质量评价方法及装置,涉及视频技术领域和生物神经科学领域,旨在基于用户的真实感知对视频播放质量进行评价。所述方法包括：采集用户观看待评价视频时的脑电信号；提取预设通道的所述脑电信号的功率谱密度特征；根据所述功率谱密度特征,获取所述脑电信号在预设频带内的能量值；根据获取到的所述预设频带内的能量值,得到待评价视频的播放质量分数。(The application provides a video playing quality evaluation method and device based on electroencephalogram characteristics, relates to the technical field of videos and the field of biological neuroscience, and aims to evaluate the video playing quality based on real perception of a user. The method comprises the following steps: collecting electroencephalogram signals when a user watches a video to be evaluated; extracting power spectral density characteristics of the electroencephalogram signals of a preset channel; acquiring an energy value of the electroencephalogram signal in a preset frequency band according to the power spectral density characteristic; and obtaining the playing quality score of the video to be evaluated according to the obtained energy value in the preset frequency band.)

1. A video playing quality evaluation method based on electroencephalogram features is characterized by comprising the following steps:

collecting electroencephalogram signals when a user watches a video to be evaluated;

extracting power spectral density characteristics of the electroencephalogram signals of a preset channel;

acquiring an energy value of the electroencephalogram signal in a preset frequency band according to the power spectral density characteristic;

and obtaining the playing quality score of the video to be evaluated according to the obtained energy value in the preset frequency band.

2. The method of claim 1, extracting power spectral density features of the brain electrical signal of a preset channel, comprising:

processing the electroencephalogram signals of the preset channel by utilizing a Hamming window function;

performing fast Fourier transform on the electroencephalogram signals of the preset channels on each window;

and averaging the EEG signals of the preset channel after the fast Fourier transform to obtain the power spectral density characteristic of the EEG signals of the preset channel.

3. The method according to claim 1, wherein obtaining the playing quality score of the video to be evaluated according to the obtained energy value in the preset frequency band comprises:

normalizing the acquired energy value in the preset frequency band;

and obtaining the playing quality score of the video to be evaluated according to the corresponding relation pre-established between the normalized energy value and the playing quality score.

4. The method of claim 1, the preset frequency band being determined by:

acquiring a subjective score of a sample video watched by a user;

acquiring energy values in a plurality of frequency bands of the electroencephalogram signals when a user watches the sample video;

obtaining quality scores corresponding to the multiple frequency bands of the sample video according to the obtained energy values in the multiple frequency bands;

and determining the frequency band corresponding to the most relevant quality score as a preset frequency band according to the quality scores corresponding to the plurality of frequency bands and the correlation degree of the subjective score.

5. The method of claim 1, the preset channel being determined by:

collecting a plurality of sections of sample electroencephalograms, wherein the sample electroencephalograms are electroencephalograms when a plurality of users watch videos with different buffer levels;

preprocessing the sample electroencephalogram signals;

carrying out superposition averaging on the preprocessed multiple sample electroencephalogram signals with the same buffer level;

carrying out two-tailed t test on the sample electroencephalograms when each user watches the buffered and unbuffered videos after the superposition averaging, and recording a channel with p < 0.005;

for any channel, the number of users with p <0.005 is greater than a preset number, and the channel is the preset channel.

6. The method of claim 5, collecting a plurality of segments of a sample brain electrical signal, comprising:

obtaining a plurality of segments of video clips;

adding buffering time lengths with different buffering grades in each video clip to obtain a plurality of sections of sample video clips;

and collecting the multiple sections of sample electroencephalogram signals when multiple users watch the sample video clips.

7. The method of claim 5, preprocessing the sample brain electrical signal, comprising:

re-referencing the sample brain electrical signal with a TP9 electrode and a TP10 electrode;

performing band-pass filtering of 0.2-100HZ on the sample electroencephalogram signal after being subjected to the re-reference by adopting an FIR filter;

carrying out independent component analysis on the filtered sample electroencephalogram signal, and removing impurity signals;

dividing the sample electroencephalogram signals with the impurity signals removed into multiple sections to obtain the preprocessed sample electroencephalogram signals.

8. A video playing quality evaluation device based on electroencephalogram characteristics is characterized by comprising:

the acquisition module is used for acquiring electroencephalogram signals when a user watches a video to be evaluated;

the extraction module is used for extracting the power spectral density characteristic of the electroencephalogram signal of a preset channel;

the energy value module is used for acquiring an energy value of the electroencephalogram signal in a preset frequency band according to the power spectral density characteristic;

and the score module is used for obtaining the playing quality score of the video to be evaluated according to the obtained energy value in the preset frequency band.

9. The apparatus of claim 8, the extraction module comprising:

the processing submodule is used for processing the electroencephalogram signals of the preset channel by utilizing a Hamming window function;

the Fourier transform sub-module is used for performing fast Fourier transform on the electroencephalogram signals of the preset channel on each window;

the characteristic submodule is used for averaging the electroencephalogram signals of the preset channel after the fast Fourier transform is carried out, and obtaining the power spectral density characteristic of the electroencephalogram signals of the preset channel.

10. The apparatus of claim 8, the score module comprising:

the normalization submodule is used for performing normalization processing on the acquired energy value in the preset frequency band;

and the corresponding submodule is used for obtaining the playing quality score of the video to be evaluated according to the corresponding relation which is pre-established between the energy value after the normalization processing and the playing quality score.

Technical Field

The application relates to the technical field of videos and the field of biological neuroscience, in particular to a method and a device for evaluating video playing quality based on electroencephalogram characteristics.

Background

With the rapid development of wireless communication technology and multimedia services, the main value of video services is not only to accurately transmit information, but also to meet the experience requirements of people. However, the objective technical index evaluation method based on the quality of service (QoS) level is difficult to match the development requirement of the current video service.

Common subjective quality evaluation methods in the related art are user-based and business-based evaluation methods, respectively. However, the user-based evaluation method requires the user to perform behavior feedback and is easily influenced by human thinking cognitive deviation; the service-based evaluation method is difficult to accurately evaluate the comprehensive experience quality of the service in an actual service scene. Therefore, the subjective quality evaluation methods in the related art all have certain limitations.

Disclosure of Invention

In view of the foregoing problems, embodiments of the present invention provide a method and an apparatus for evaluating video playing quality based on electroencephalogram characteristics, so as to overcome the foregoing problems or at least partially solve the foregoing problems.

The embodiment of the invention provides a video playing quality evaluation method based on electroencephalogram characteristics, which comprises the following steps:

collecting electroencephalogram signals when a user watches a video to be evaluated;

extracting power spectral density characteristics of the electroencephalogram signals of a preset channel;

acquiring an energy value of the electroencephalogram signal in a preset frequency band according to the power spectral density characteristic;

and obtaining the playing quality score of the video to be evaluated according to the obtained energy value in the preset frequency band.

Optionally, extracting the power spectral density feature of the electroencephalogram signal of the preset channel includes:

processing the electroencephalogram signals of the preset channel by utilizing a Hamming window function;

performing fast Fourier transform on the electroencephalogram signals of the preset channels on each window;

and averaging the EEG signals of the preset channel after the fast Fourier transform to obtain the power spectral density characteristic of the EEG signals of the preset channel.

Optionally, obtaining the playing quality score of the video to be evaluated according to the obtained energy value in the preset frequency band, including:

normalizing the acquired energy value in the preset frequency band;

and obtaining the playing quality score of the video to be evaluated according to the corresponding relation pre-established between the normalized energy value and the playing quality score.

Optionally, the preset frequency band is determined by:

acquiring a subjective score of a sample video watched by a user;

acquiring energy values in a plurality of frequency bands of the electroencephalogram signals when a user watches the sample video;

obtaining quality scores corresponding to the multiple frequency bands of the sample video according to the obtained energy values in the multiple frequency bands;

and determining the frequency band corresponding to the most relevant quality score as a preset frequency band according to the quality scores corresponding to the plurality of frequency bands and the correlation degree of the subjective score.

Optionally, the preset channel is determined by:

collecting a plurality of sections of sample electroencephalograms, wherein the sample electroencephalograms are electroencephalograms when a plurality of users watch videos with different buffer levels;

preprocessing the sample electroencephalogram signals;

carrying out superposition averaging on the preprocessed multiple sample electroencephalogram signals with the same buffer level;

carrying out two-tailed t test on the sample electroencephalograms when each user watches the buffered and unbuffered videos after the superposition averaging, and recording a channel with p < 0.005;

for any channel, the number of users with p <0.005 is greater than a preset number, and the channel is the preset channel.

Optionally, acquiring a plurality of segments of sample brain electrical signals comprises:

obtaining a plurality of segments of video clips;

adding buffering time lengths with different buffering grades in each video clip to obtain a plurality of sections of sample video clips;

and collecting the multiple sections of sample electroencephalogram signals when multiple users watch the sample video clips.

Optionally, the preprocessing the sample electroencephalogram signal includes:

re-referencing the sample brain electrical signal with a TP9 electrode and a TP10 electrode;

performing band-pass filtering of 0.2-100HZ on the sample electroencephalogram signal after being subjected to the re-reference by adopting an FIR filter;

carrying out independent component analysis on the filtered sample electroencephalogram signal, and removing impurity signals;

dividing the sample electroencephalogram signals with the impurity signals removed into multiple sections to obtain the preprocessed sample electroencephalogram signals.

In a second aspect of the embodiments of the present invention, there is provided a video playing quality evaluation device based on electroencephalogram characteristics, the device including:

the acquisition module is used for acquiring electroencephalogram signals when a user watches a video to be evaluated;

the extraction module is used for extracting the power spectral density characteristic of the electroencephalogram signal of a preset channel;

the energy value module is used for acquiring an energy value of the electroencephalogram signal in a preset frequency band according to the power spectral density characteristic;

and the score module is used for obtaining the playing quality score of the video to be evaluated according to the obtained energy value in the preset frequency band.

Optionally, the extraction module comprises:

the processing submodule is used for processing the electroencephalogram signals of the preset channel by utilizing a Hamming window function;

the Fourier transform sub-module is used for performing fast Fourier transform on the electroencephalogram signals of the preset channel on each window;

the characteristic submodule is used for averaging the electroencephalogram signals of the preset channel after the fast Fourier transform is carried out, and obtaining the power spectral density characteristic of the electroencephalogram signals of the preset channel.

Optionally, the score module comprises:

the normalization submodule is used for performing normalization processing on the acquired energy value in the preset frequency band;

and the corresponding submodule is used for obtaining the playing quality score of the video to be evaluated according to the corresponding relation which is pre-established between the energy value after the normalization processing and the playing quality score.

The embodiment of the invention has the following advantages:

in the embodiment, the electroencephalogram signals of the user watching the video to be evaluated can be collected; extracting power spectral density characteristics of the electroencephalogram signals of a preset channel; acquiring energy values of the electroencephalogram signals in a plurality of preset frequency bands according to the power spectral density characteristics; and obtaining the playing quality scores of the video to be evaluated according to the obtained energy values in the preset frequency bands. Therefore, the playing quality score of the video to be evaluated is obtained according to the electroencephalogram signal, the electroencephalogram signal reflects human thinking activity, and the quantification of the electroencephalogram signal can realize perception evaluation without subjective deviation, so that the user's perception of the video playing quality can be accurately reflected according to the playing quality score of the video to be evaluated, which is obtained according to the electroencephalogram signal. Therefore, the method has the advantage of accurately quantifying and evaluating QoE (Quality of Experience) of the user on the video playing Quality.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the description of the embodiments of the present application will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without inventive exercise.

FIG. 1 is a flowchart illustrating steps of a method for evaluating video playing quality based on electroencephalogram characteristics according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of electrode locations for EEG standard leads in an embodiment of the present invention;

FIG. 3 is a flowchart illustrating steps for determining a default channel according to an embodiment of the present invention;

FIG. 4 is a schematic diagram of a sample video clip in an embodiment of the present application;

FIG. 5 is a schematic diagram illustrating the steps of preprocessing a sample electroencephalogram signal according to an embodiment of the present invention;

fig. 6 is a schematic structural diagram of a video playing quality evaluation device based on electroencephalogram characteristics in an embodiment of the present invention.

Detailed Description

In order to make the aforementioned objects, features and advantages of the present application more comprehensible, the present application is described in further detail with reference to the accompanying drawings and the detailed description.

Electroencephalogram (EEG) can capture early nerve signals of the bottom layer of the brain, and has the characteristics of short time consumption, tropism, quantification, high precision and the like. The method can be used for quantitatively analyzing the QoE change when a user watches videos with different playing qualities by mining and learning the electroencephalogram signals.

In order to solve the problems that objective parameters are used in the video playing quality evaluation method in the related art, the video playing quality is not evaluated from the perspective of a user, and the like, the applicant proposes: the playing quality score of the video to be evaluated is obtained through the electroencephalogram signals when the user watches the video to be evaluated, and therefore the QoE for accurately measuring the playing quality of the video by the user is obtained.

Referring to fig. 1, a flowchart illustrating steps of a video playing quality evaluation method based on electroencephalogram characteristics in an embodiment of the present invention is shown, and as shown in fig. 1, the video playing quality evaluation method based on electroencephalogram characteristics may specifically include the following steps:

step S110: and acquiring an electroencephalogram signal when a user watches a video to be evaluated.

Electroencephalograms of a user viewing a video to be evaluated are acquired and recorded by a 64-channel electroencephalogram amplification system (Brain Products, Brain amps, germany) and a matched signal recording software (Brain Vision Recorder). The sampling frequency of the brain electrical signal is 500 Hz. The electrode cap of the acquisition system was equipped with 64 actiCAP active electrodes (a mini-electrode) arranged according to the international standard 10-20 system. Referring to fig. 2, a schematic diagram of electrode locations for EEG standard leads is shown. The ground electrode is located between forehead electrodes Fp1 and Fp2, the reference potential being the average potential of electrodes TP9 and TP 10. After applying the scalp gel uniformly between the electrodes and the scalp, the impedance of all electrodes was kept below 10K Ω.

Optionally, after acquiring an electroencephalogram signal when a user watches a video to be evaluated, preprocessing the electroencephalogram signal, including: re-referencing the electroencephalogram signals, filtering and analyzing independent components of the re-referenced electroencephalogram signals, removing impurity signals, segmenting and the like.

Step S120: and extracting the power spectral density characteristic of the electroencephalogram signal of a preset channel.

The power spectral density characteristic of the electroencephalogram signal in a preset channel is extracted, wherein the preset channel is a channel determined before the electroencephalogram characteristic-based video playing quality evaluation method is used, and the method for determining the preset channel is described in detail in the embodiment below. The channel refers to an electrode for collecting electroencephalogram signals through the electrode. The preset channel is a channel closely related to emotion related to video quality evaluation, and reflects the response of the human brain to video playing quality. The preset channel can be POz, 3-4, 7-8; pz, 1-6, 8.

Optionally, as an embodiment, the power spectral density characteristic of the brain electrical signal is calculated by using a WELCH algorithm, which allows the brain electrical signals to overlap, thereby increasing the number of segments, and the principle is shown in the following formula:

wherein p (w) represents the power spectral density characteristic, U represents a normalization factor, d (N) is a data window, the electroencephalogram signal x with the length of N is divided into L segments, each segment is M, i is 1,2, … …, L; n-1, 2, … …, M-1; j denotes a complex number and ω denotes an amplitude.

The power spectral density characteristic of the electroencephalogram signal is calculated by using a WELCH algorithm, and the calculation specifically includes: processing the electroencephalogram signals of the preset channel by utilizing a Hamming window function; performing fast Fourier transform on the electroencephalogram signals of the preset channels on each window; and averaging the EEG signals of the preset channel after the fast Fourier transform to obtain the power spectral density characteristic of the EEG signals of the preset channel.

The EEG signal of the preset channel is processed by using Hamming windows which are overlapped by 50%, the EEG signal of the preset channel is subjected to fast Fourier transform on each window, and the EEG signal of the preset channel after being subjected to fast Fourier transform is averaged at different time to obtain the power spectral density characteristic of the EEG signal of the preset channel.

Alternatively, other EM algorithms may be used to obtain the power spectral density characteristics of the brain electrical signal.

Step S130: and acquiring the energy values of the electroencephalogram signals in a plurality of preset frequency bands according to the power spectral density characteristics.

According to the power spectral density characteristics of the acquired electroencephalogram signals of the preset channels, energy values of the electroencephalogram signals in a plurality of preset frequency bands are acquired, wherein the preset frequency bands are the frequency bands most relevant to subjective scoring and are determined before the electroencephalogram characteristic-based video playing quality evaluation method is used, and the method for determining the preset frequency bands is described in detail in the following embodiments. The energy values of the brain electrical signals in a plurality of preset frequency bands can be obtained through commands in matlab software (a commercial mathematical software).

Step S140: and obtaining the playing quality score of the video to be evaluated according to the obtained energy value in the preset frequency band.

Normalizing the acquired energy value in the preset frequency band; and obtaining the playing quality score of the video to be evaluated according to the corresponding relation pre-established between the normalized energy value and the playing quality score. And normalizing the energy value of the preset frequency band, and obtaining the playing quality score of the video to be evaluated according to the corresponding relation between the normalized energy value and the playing quality score. Wherein, the energy value in the corresponding relation is the energy value of the electroencephalogram signals in a plurality of preset channels.

By adopting the technical scheme of the embodiment of the application, the electroencephalogram signals when a user watches the video to be evaluated can be collected, then the power spectral density characteristics are extracted from the electroencephalogram signals, the energy values of the electroencephalogram signals in a plurality of preset frequency bands are obtained, and the playing quality score of the video to be evaluated is obtained according to the obtained energy values. The electroencephalogram signals are quantized, and perception evaluation without subjective deviation can be achieved. The playing quality score of the video to be evaluated, which is obtained according to the electroencephalogram signal, can accurately reflect the perception of the user on the video playing quality, so that the QoE of the user on the video playing quality can be accurately quantified and evaluated.

Optionally, as an embodiment, the preset frequency band is determined by: acquiring a subjective score of a sample video watched by a user; acquiring energy values in a plurality of frequency bands of the electroencephalogram signals when a user watches the sample video; obtaining quality scores corresponding to the multiple frequency bands of the sample video according to the obtained energy values in the multiple frequency bands; and determining the frequency band corresponding to the most relevant quality score as a preset frequency band according to the quality scores corresponding to the plurality of frequency bands and the correlation degree of the subjective score.

The sample videos are a plurality of videos added with different buffer durations, subjective MOS scores of the sample videos watched by a user are obtained firstly, and the subjective scores are determined according to answers of the user to subjective questions. Acquiring an electroencephalogram signal when a user watches a sample video, and acquiring corresponding quality scores of three frequency bands which are relatively related to subjective expression in the electroencephalogram signal according to energy values of the three frequency bands, wherein the three frequency bands which are relatively related to the subjective expression are theta (4-8 Hz), alpha (8-13 Hz) and beta (13-30 Hz) frequency bands respectively.

The method comprises the steps of obtaining the average scores of subjective scores of a plurality of users for a plurality of sample videos and the average scores of quality scores corresponding to a plurality of frequency bands, and calculating a correlation coefficient between the average score of the quality score corresponding to each frequency band and the average score of the corresponding subjective score, so as to determine the frequency band corresponding to the quality score most relevant to the subjective score. And determining the frequency band corresponding to the quality score most relevant to the subjective score as a preset frequency band.

Optionally, as an embodiment, referring to fig. 3, a flowchart illustrating steps of determining a preset channel is shown, where the preset channel may be determined through the following steps:

step S210: collecting a plurality of sections of sample electroencephalograms, wherein the sample electroencephalograms are electroencephalograms when a plurality of users watch videos with different buffer levels.

Step S210 includes substep S211, substep S212, and substep S213.

Substep S211: a multi-segment video clip is obtained.

To ensure a high signal-to-noise ratio, video segments are selected that produce as little signal noise as possible. The inventor randomly cuts 100 non-overlapping 4s video clips from three 50-minute animal record films by using the animal record film as a stimulating material, the video resolution is 720P, and the video frame rate is 50 fps.

Substep S212: and adding buffering time lengths with different buffering grades in each video clip to obtain a plurality of sections of sample video clips.

As shown in fig. 4, each video segment is added with a buffering time of different buffering levels after playing for 2s, so as to obtain a plurality of sample video segments. The buffering duration added by the inventor is totally 12 levels, respectively 0ms, 20ms, 40ms, 60ms, 80ms, 100ms, 200ms, 300ms, 400ms, 500ms, 700ms, 1000ms, thereby generating 1200 sample video segments. It will be appreciated that other buffer lengths may alternatively be added.

Substep S213: and collecting the multiple sections of sample electroencephalogram signals when multiple users watch the sample video clips.

And selecting 7 users who have not participated in the video playing quality evaluation experiment before and have good eyesight, and informing the users of necessary contents in the acquisition process before acquiring the electroencephalogram signals. A 24 inch liquid crystal display is used to present a multi-segment sample video clip with a display resolution of 1920 x 1080 and a refresh rate of 144 Hz. The sample video clip has a size of 16 x 9 cm on the display screen. The distance each user views from the display is set to 6 times the height of the video content, i.e. 54 cm.

In the acquisition process, 1200 sample video clips with different buffer levels are randomly played. After each sample video clip is played, the user must answer two subjective questions, 1) do there are buffers observed in the video? Answer options yes or no; 2) how does this video buffering time experience your user? Answer options are 5 points: imperceptible; and 4, dividing: perceptible, but not objectionable; and 3, dividing: is slightly unpleasant; and 2, dividing: unpleasant; 1 minute: it is very annoying (the option is strictly in accordance with the MOS standard). The user stares at the display at all times while the sample video segment is played, and unnecessary blinking and each segment of physical activity are reduced. After the sample video clip is played, the user answers the two subjective questions by pressing the corresponding buttons on the keyboard. In the experimental process, the user can choose to have a rest after any sample video is played. In order to minimize electromagnetic and noise interference, a user completes the acquisition of electroencephalogram signals in an electromagnetic shielding room, wherein the illumination simulates the color temperature of sunlight.

The method comprises the steps of collecting and recording multi-segment sample electroencephalogram signals when a user watches multi-segment sample video segments through a 64-channel electroencephalogram amplification system and a matched signal recording software. The sampling frequency of the sample brain electrical signals is 500 Hz. The electrode cap of the acquisition system is provided with 64 active electrodes of actiCAP, and the electrodes are arranged according to an international standard 10-20 system. The ground electrode is located between forehead electrodes Fp1 and Fp2, the reference potential being the average potential of electrodes TP9 and TP 10. After applying the scalp gel uniformly between the electrodes and the scalp, the impedance of all electrodes was kept below 10K Ω.

It can be understood that, the above setting is to acquire the electroencephalogram signal with better quality and capable of reflecting the QoE of the user on the video playing quality, as long as the same purpose can be achieved, other settings may be adopted, for example, 8 users may be selected to acquire the electroencephalogram signal, and other displays may also be adopted.

Step S220: and preprocessing the sample electroencephalogram signals.

The acquired sample brain electrical signals contain a lot of noises, and the noises are not from the perception of the brain on the playing quality of the sample video clips, but are caused by the interference of heart activity, eye movement, muscle activity and power supply frequency. These disturbances can severely degrade the brain electrical signal quality, and therefore require pre-processing of each sample brain electrical signal to remove these disturbances. The sample electroencephalogram signal can be preprocessed by using an EEGLAB tool box of Matlab 2016b, and can also be preprocessed by using other signal processing software.

Optionally, as an embodiment, step S220 may include sub-step S221, sub-step S222, sub-step S223, and sub-step S224.

Referring to fig. 5, a schematic diagram illustrating a step of preprocessing a sample brain electrical signal is shown, as shown in fig. 5, including:

substep S221: the sample brain electrical signals were re-referenced using TP9 and TP10 electrodes.

The electroencephalogram cap can collect signals at a plurality of electrode positions of a human body, firstly, electroencephalogram signals are loaded, scalp position information is led in, and the electrodes are positioned.

The TP9 and TP10 electrodes are electrodes of left and right earlobes of a human body, the TP9 and TP10 electrodes are used as reference electrodes, and the average value of the potentials of the two electrodes is used for re-referencing the sample electroencephalogram signal.

Substep S222: and performing band-pass filtering of 0.2-100HZ on the sample electroencephalogram signals after being subjected to the re-reference by adopting an FIR filter.

And performing band-pass filtering of 0.2-100Hz on the re-referenced sample electroencephalogram signal by adopting an FIR filter (finite impulse filter). In order to eliminate the influence of power frequency interference, a 50Hz notch filter is adopted for notch filtering.

Substep S223: and carrying out independent component analysis on the filtered sample electroencephalogram signal, and removing impurity signals.

The collected sample brain electrical signals can be regarded as mixed signals from various independent sources, such as heart activity signals, eyeball motion signals, muscle activity signals, power supply frequency noise and the like, and the interfering signals are regarded as linear signals. Therefore, the Independent Component Analysis (ICA) is adopted to remove other impurity signals except the sample brain electricity signal in the filtered sample brain electricity signal.

Substep S224: dividing the sample electroencephalogram signals with the impurity signals removed into multiple sections to obtain the preprocessed sample electroencephalogram signals.

The sample brain electrical signals with the impurity signals removed are divided into a plurality of sections of 1200 segments of 6s, each sample brain electrical signal segment is 1s before the beginning of watching the sample video segment to 5s after the beginning of watching the sample video segment, and each user can obtain 100 sample brain electrical signals aiming at the video segment of each buffer level.

The above pre-treatment steps may have different sequences according to actual situations. The technical scheme for preprocessing the sample electroencephalogram signals is also suitable for preprocessing the electroencephalogram signals when the user watches the video to be evaluated. By adopting the technical scheme of the embodiment, the collected electroencephalogram signals of each section of the sample are preprocessed, so that the accuracy of determining the preset channel through the electroencephalogram signals of the sample can be improved.

Step S230: and carrying out superposition averaging on the preprocessed multiple sample electroencephalogram signals with the same buffer level.

In order to avoid individual difference of single sample electroencephalogram signals, the preprocessed 100 sample electroencephalogram signals with the same buffer level are subjected to superposition averaging to obtain the potential average value of the sample electroencephalogram signals with the same buffer level. Specifically, the method comprises the following steps: selecting electroencephalogram signals of a plurality of preset buffer levels to perform superposition average calculation of a potential average value, wherein the plurality of preset buffer levels can be preferably 12 buffer levels added with buffer duration of 0ms (or 20ms), 20ms, 40ms, 60ms, 80ms, 100ms, 200ms, 300ms, 400ms, 500ms, 700ms and 1000 ms. The superposition averaging can be achieved by the following formula:

wherein X represents electroencephalogram, c represents a channel, t represents time, l represents buffer level, h represents the number of sample electroencephalograms of each buffer level, k represents a user, and mean represents averaging.

Step S240: and carrying out two-tailed t test on the sample electroencephalograms when each user after the superposition averaging watches the buffered and unbuffered videos, and recording a channel with p < 0.005.

And respectively carrying out superposition averaging on the sample electroencephalogram signals when each user watches the buffered video and the unbuffered video, wherein the buffered video can be a 1000ms video with the buffering time length of 1000ms, and the unbuffered video can be a 1000ms video with the buffering time length of 0 ms. And carrying out double-tail t test on the superposed and averaged sample electroencephalogram signals to obtain a channel with p <0.005, wherein the channel with p <0.005 is a channel with obvious difference.

Step S250: for any channel, the number of users with p <0.005 is greater than a preset number, and the channel is the preset channel.

For any channel, the sample electroencephalograms of users with the number larger than the preset number in the multiple users participating in sample electroencephalogram signal acquisition are subjected to two-tailed t test to obtain p <0.005, and the channel is determined to be the preset channel. For example: the result of performing double-tail t test on the sample electroencephalograms of the same channel of 8 users among the 10 users is that p is less than 0.005, and the channel is considered to be a channel with significant differences in common users and is a preset channel.

By adopting the technical scheme of the embodiment, the electroencephalogram signals of the sample are subjected to superposition averaging, then the channel with larger difference in the electroencephalogram signals when the user watches buffered video segments and unbuffered video segments is determined, and the channel is determined as the preset channel. The preset channel is a channel closely related to emotion related to video quality evaluation, and reflects the response of the human brain to video playing quality. Therefore, when the relevant steps of the video playing quality evaluation method based on the electroencephalogram characteristics provided by the embodiment of the invention are executed, the determined electroencephalogram signals of the preset channel can be directly used for processing.

It should be noted that, for simplicity of description, the method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present invention is not limited by the illustrated order of acts, as some steps may occur in other orders or concurrently in accordance with the embodiments of the present invention. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred and that no particular act is required to implement the invention.

Fig. 6 is a schematic structural diagram of a video playing quality evaluation device based on electroencephalogram characteristics according to an embodiment of the present invention, and as shown in fig. 6, the video playing quality evaluation device based on electroencephalogram characteristics includes an acquisition module, an extraction module, an energy value module, and a score module, where:

the acquisition module is used for acquiring electroencephalogram signals when a user watches a video to be evaluated;

the extraction module is used for extracting the power spectral density characteristic of the electroencephalogram signal of a preset channel;

the energy value module is used for acquiring the energy of the electroencephalogram signal in a plurality of preset frequency bands according to the power spectral density characteristics;

and the score module is used for obtaining the playing quality scores of the video to be evaluated according to the obtained energy values in the preset frequency bands.

Optionally, as an embodiment, the extracting module includes:

the processing submodule is used for processing the electroencephalogram signals of the preset channel by utilizing a Hamming window function;

the Fourier transform sub-module is used for performing fast Fourier transform on the electroencephalogram signals of the preset channel on each window;

the characteristic submodule is used for averaging the electroencephalogram signals of the preset channel after the fast Fourier transform is carried out, and obtaining the power spectral density characteristic of the electroencephalogram signals of the preset channel.

Optionally, as an embodiment, the score module includes:

the normalization submodule is used for performing normalization processing on the acquired energy values in the plurality of preset frequency bands;

and the mapping submodule is used for obtaining the playing quality score of the video to be evaluated according to the mapping relation which is pre-established between the energy value after the normalization processing and the playing quality score.

Optionally, as an embodiment, the preset channel is determined by:

collecting a plurality of sections of sample electroencephalograms, wherein the sample electroencephalograms are electroencephalograms when a plurality of users watch videos with different buffer levels;

preprocessing the sample electroencephalogram signals;

carrying out superposition averaging on the preprocessed multiple sample electroencephalogram signals with the same buffer level;

carrying out two-tailed t test on the sample electroencephalograms when each user watches the buffered and unbuffered videos after the superposition averaging, and recording a channel with p < 0.005;

for any channel, the number of users with p <0.005 is greater than a preset number, and the channel is the preset channel.

Optionally, as an embodiment, acquiring a plurality of segments of sample brain electrical signals includes:

obtaining a plurality of segments of video clips;

adding buffering time lengths with different buffering grades in each video clip to obtain a plurality of sections of sample video clips;

and collecting the multiple sections of sample electroencephalogram signals when multiple users watch the sample video clips.

Optionally, as an embodiment, the preprocessing the sample brain electrical signal includes:

re-referencing the sample brain electrical signal with a TP9 electrode and a TP10 electrode;

performing band-pass filtering of 0.2-100HZ on the sample electroencephalogram signal after being subjected to the re-reference by adopting an FIR filter;

carrying out independent component analysis on the filtered sample electroencephalogram signal, and removing impurity signals;

dividing the sample electroencephalogram signals with the impurity signals removed into multiple sections to obtain the preprocessed sample electroencephalogram signals.

Optionally, as an embodiment, the preset frequency band is determined by:

acquiring energy values of the sample electroencephalogram signals in different frequency bands under different buffer levels;

and carrying out variance analysis on the energy value of the acquired sample electroencephalogram signal in each frequency band, and determining the frequency band with the energy value change range exceeding a preset range under different buffer levels as the frequency band being a preset frequency band.

By adopting the technical scheme of the embodiment of the application, the electroencephalogram signals when a user watches the video to be evaluated can be collected, then the power spectral density characteristics are extracted from the electroencephalogram signals, the energy values of the electroencephalogram signals in a plurality of preset frequency bands are obtained, and the playing quality score of the video to be evaluated is obtained according to the obtained energy values. The electroencephalogram signals are quantized, and perception evaluation without subjective deviation can be achieved. The playing quality score of the video to be evaluated, which is obtained according to the electroencephalogram signal, can accurately reflect the perception of the user on the video playing quality, so that the QoE of the user on the video playing quality can be accurately quantified and evaluated.

It should be noted that the device embodiments are similar to the method embodiments, so that the description is simple, and reference may be made to the method embodiments for relevant points.

The embodiments in the present specification are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other.

As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, apparatus or computer program product. Accordingly, embodiments of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

Embodiments of the present invention are described with reference to flowchart illustrations and/or block diagrams of methods, apparatus, electronic devices and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing terminal to cause a series of operational steps to be performed on the computer or other programmable terminal to produce a computer implemented process such that the instructions which execute on the computer or other programmable terminal provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

While preferred embodiments of the present invention have been described, additional variations and modifications of these embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the embodiments of the invention.

Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or terminal that comprises the element.

The method and the device for rating the video playing quality based on the electroencephalogram characteristics are introduced in detail, specific examples are applied in the method for explaining the principle and the implementation mode of the method, and the description of the embodiments is only used for helping to understand the method and the core idea of the method; meanwhile, for a person skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.