阅读说明：本技术 基于fpga接收mipi信号的方法、fpga、终端和介质 (FPGA-based MIPI signal receiving method, FPGA, terminal and medium ) 是由 叶红磊 温建新 叶红波 蒋亮亮 姚清志 于 2021-09-06 设计创作，主要内容包括：本发明提供了一种基于FPGA接收MIPI信号的方法、FPGA、终端和介质,该方法包括：在FPGA接收MIPI信号的过程中,当LVDS接口确定MIPI信号从高速传输模式切换为低功耗传输模式时,在低功耗传输模式下,LVDS接口确定所接收的MIPI信号为消隐数据；当GPIO接口确定MIPI信号从低功耗传输模式切换为高速传输模式时,在高速传输模式下LVDS接口接收并解码MIPI信号的一行串行有效图像数据；重复执行上述步骤,直至接收并解码得到MIPI信号的最后一行串行有效图像数据后,FPGA将各行串行有效图像数据转换为并行图像数据。该方法可以实现在不借助其它芯片的情况下,由FPGA接收MIPI信号。(The invention provides a method for receiving MIPI signals based on an FPGA, the FPGA, a terminal and a medium, wherein the method comprises the following steps: in the process that the FPGA receives the MIPI signal, when the LVDS interface determines that the MIPI signal is switched from a high-speed transmission mode to a low-power-consumption transmission mode, the LVDS interface determines that the received MIPI signal is blanking data in the low-power-consumption transmission mode; when the GPIO interface determines that the MIPI signal is switched from a low-power-consumption transmission mode to a high-speed transmission mode, the LVDS interface receives and decodes a row of serial effective image data of the MIPI signal in the high-speed transmission mode; and repeatedly executing the steps until the last line of serial effective image data of the MIPI signal is obtained through receiving and decoding, and converting the serial effective image data of each line into parallel image data by the FPGA. The method can realize that the FPGA receives the MIPI signal without other chips.)

1. A method for receiving MIPI signals based on an FPGA is characterized in that the method is applied to a field programmable gate array FPGA, and the FPGA comprises a low voltage differential signal LVDS interface and a general purpose input and output GPIO interface; the LVDS interface is used for receiving MIPI signals from the image sensor; the GPIO interface is used for receiving the boosted MIPI signal while the LVDS interface receives the MIPI signal; the method comprises the following steps:

in the process that the FPGA receives the MIPI signal, when the LVDS interface determines that the MIPI signal is switched from a high-speed transmission mode to a low-power transmission mode, the LVDS interface determines that the received MIPI signal is blanking data in the low-power transmission mode; when the GPIO interface determines that the MIPI signal is switched from a low-power-consumption transmission mode to a high-speed transmission mode, the LVDS interface receives and decodes a line of serial effective image data of the MIPI signal in the high-speed transmission mode;

and repeatedly executing the steps until the LVDS interface receives and decodes the last row of serial effective image data of the MIPI signal, and converting the serial effective image data of each row into parallel image data by the FPGA.

2. The method of claim 1, wherein determining that the MIPI signal is switched from a high-speed transmission mode to a low-power transmission mode by the LVDS interface comprises:

the LVDS interface acquires tail information of the received MIPI signal;

and determining that the MIPI signal is switched from a high-speed transmission mode to a low-power consumption transmission mode according to the tail information.

3. The method of claim 1 or 2, wherein the GPIO interface determining that the MIPI signal is switched from a low power consumption transmission mode to a high speed transmission mode comprises:

according to the change that the GPIO interface can receive the MIPI signal to cannot receive the MIPI signal, the MIPI signal is determined to be switched from a low-power-consumption transmission mode to a high-speed transmission mode; the boosted MIPI signal is obtained by boosting the voltage to 2.5V through a boosting voltage chip.

4. The method according to claim 1 or 2, characterized in that the method further comprises:

and the FPGA sends the frame signal, the line signal and the parallel image data to an image signal processor.

5. The method of claim 1 or 2, wherein the FPGA receives and decodes a row of serial valid image data from a MIPI signal via the LVDS interface, comprises:

the FPGA acquires the header information in the MIPI through the LVDS interface and starts to receive and decode a row of serial effective image data from the MIPI.

6. An FPGA, comprising a Low Voltage Differential Signaling (LVDS) interface and a General Purpose Input Output (GPIO) interface:

the LVDS interface is used for receiving MIPI signals from the image sensor;

the GPIO interface is used for receiving the MIPI signal after voltage boosting while the LVDS interface receives the MIPI signal;

the FPGA is used for determining the MIPI signal received by the LVDS interface as blanking data in a low-power-consumption transmission mode when the LVDS interface of the FPGA determines that the MIPI signal is switched from a high-speed transmission mode to a low-power-consumption transmission mode in the process that the FPGA receives the MIPI signal; when the GPIO interface of the FPGA determines that the MIPI signal is switched from a low-power-consumption transmission mode to a high-speed transmission mode, determining the MIPI signal received by the LVDS interface as effective image data in the high-speed transmission mode, and receiving and decoding to obtain a line of serial effective image data of the MIPI signal;

and repeatedly executing the steps until the LVDS interface receives and decodes the last row of serial effective image data of the MIPI signal, and converting the serial effective image data of each row into parallel image data by the FPGA.

7. The FPGA of claim 6, wherein the LVDS interface is specifically configured to, when determining that the MIPI signal is switched from the high-speed transmission mode to the low-power transmission mode: acquiring tail information of a received MIPI signal; and determining that the MIPI signal is switched from a high-speed transmission mode to a low-power consumption transmission mode according to the tail information.

8. The FPGA of claim 6 or 7, wherein the GPIO interface is specifically configured to, when determining that the MIPI signal is switched from a low-power-consumption transmission mode to a high-speed transmission mode: and determining that the MIPI signal is switched from a low-power-consumption transmission mode to a high-speed transmission mode according to the change from the condition that the GPIO interface can receive the MIPI signal to the condition that the MIPI signal cannot be received, wherein the boosted MIPI signal is obtained by boosting the voltage to 2.5V through a boosting voltage chip.

9. A terminal device, characterized in that it comprises an FPGA according to any one of claims 6 to 8.

10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the method of any one of claims 1 to 5.

Technical Field

The invention relates to the technical field of communication, in particular to a method for receiving MIPI signals based on an FPGA, the FPGA, a terminal and a medium.

Background

For an intelligent terminal, if various interfaces are included in the equipment, great difficulty is brought to the design and component selection of a mobile phone. Therefore, the MIPI (mobile industry processor interface) alliance standardizes the interfaces inside the intelligent terminal, such as a camera, a display screen interface, a radio frequency/baseband interface and the like, thereby reducing the complexity of the intelligent terminal design and increasing the design flexibility. For example, MIPI defines a series of internal interface standards for mobile phones, such as camera interface, display interface, radio frequency interface, microphone/speaker interface, and the like. The advantage of unifying the interface standard is that cell-phone manufacturer can be from the nimble different chips and the module of selecting on the market as required, and is more convenient when changing design and function.

D-PHY (physical layer) and signal levels of MIPI As shown in FIG. 1, the D-PHY includes HS-TX (high speed transmitter), LP-TX (low power transmitter), HS-RX (high speed receiver), and LP-RX (low power receiver). The MIPI transmission mode is divided into a High Speed (HS) mode and a low power consumption (LP) mode, a low-voltage differential signal is adopted in the HS mode, the amplitude of a high level is typically 200mV, the power consumption is high, but a very high data rate (such as a data rate of 80M-1 Gbps) can be transmitted; in the LP mode, a single-ended signal is used, the high level is typically 1.2V in amplitude, the data rate is low (e.g., the data rate is less than 10Mbps), and the corresponding power consumption is low. The combination of the two transmission modes ensures that the MIPI bus can transmit at high speed when a large amount of data (such as images) needs to be transmitted, and can reduce power consumption when the large amount of data is not needed to be transmitted.

However, when the FPGA is used to receive the MIPI signal, the high level amplitude of the FPGA is typically 3.3V, and the FPGA cannot correctly receive the MIPI signal because the high level amplitudes are not uniform.

Disclosure of Invention

The invention provides a method, a device, equipment and a medium for receiving an MIPI signal based on an FPGA (field programmable gate array), which are used for realizing the correct receiving of the MIPI signal by utilizing the FPGA.

In a first aspect, the invention provides a method for receiving an MIPI signal based on an FPGA, which can be applied to the FPGA, wherein the FPGA includes an LVDS interface and a GPIO interface, and the LVDS interface is used for receiving the MIPI signal from an image sensor; the GPIO interface is used for receiving the boosted MIPI signal while receiving the MIPI signal through the LVDS interface, and when the LVDS interface determines that the MIPI signal is switched from a high-speed transmission mode to a low-power transmission mode in the process that the FPGA receives the MIPI signal, the LVDS interface determines that the received MIPI signal is blanking data in the low-power transmission mode; when the GPIO interface determines that the MIPI signal is switched from a low-power-consumption transmission mode to a high-speed transmission mode, the LVDS interface receives and decodes a line of serial effective image data of the MIPI signal in the high-speed transmission mode;

and repeating the steps until the LVDS interface receives and decodes the last row of serial effective image data of the MIPI signal, and converting the serial effective image data of each row into parallel image data by the FPGA.

The method for receiving the MIPI signal based on the FPGA has the advantages that: according to the characteristics that the MIPI signal has a high-speed transmission mode and a low-power-consumption transmission mode, the method utilizes the LVDS interface and the GPIO interface of the FPGA to simultaneously receive the MIPI signal, and the MIPI signal is boosted before being connected to the GPIO interface, so that the problem that the MIPI signal cannot be correctly received due to the fact that the high-level amplitude of the FPGA is inconsistent is solved. In addition, the GPIO interface can identify the transmission mode of the MIPI signal and switch the transmission mode into a high-speed transmission mode, and under the high-speed transmission mode, the LVDS interface can receive serial effective image data of each row of the MIPI signal; the LVDS interface can identify that the transmission mode of the MIPI signal is switched to a low-power-consumption transmission mode, and the FPGA determines that the received MIPI signal is blanking data in the low-power-consumption transmission mode, so that the MIPI signal is received by the FPGA without the help of other chips.

In one possible implementation, the LVDS interface determines a manner in which the MIPI signal is switched from the high-speed transmission mode to the low-power transmission mode, including: the LVDS interface acquires tail information of the received MIPI signal; and determining that the MIPI signal is switched from a high-speed transmission mode to a low-power consumption transmission mode according to the tail information. In the method, the LVDS can determine that the MIPI signal is switched from the high-speed transmission mode to the low-power-consumption transmission mode according to the tail information of the received MIPI signal, and the tail information can be preset characters.

In one possible implementation, the method for determining the mode of switching the MIPI signal from the low power consumption transmission mode to the high speed transmission mode by the GPIO interface includes: and determining that the MIPI signal is switched from a low-power-consumption transmission mode to a high-speed transmission mode according to the change from the condition that the MIPI signal can be received to the condition that the MIPI signal cannot be received by the GPIO interface. That is to say, because the MIPI signal can be received by the GPIO after being boosted from 1.2V to 2.5 and still cannot be received after being boosted from 200mV, the GPIO interface may determine, according to the timing at which the MIPI signal is changed from being received to being unable to be received, that the MIPI signal is switched from the low-power-consumption transmission mode to the high-speed transmission mode, and at this time, the FPGA may receive valid data of the MIPI signal through LVDS.

In one possible implementation, the boosted MIPI signal is obtained by boosting the voltage of the voltage chip to 2.5V. Because the voltage is increased by the voltage increasing chip, the problem that the FPGA cannot correctly receive MIPI signals due to inconsistent high-level amplitude can be solved.

In one possible design, the method further includes: the FPGA sends the frame signal, the line signal and the parallel image data to an image signal processor. In the method, the image signal processor can distinguish which frame and which line the parallel image data respectively belong to based on the frame signal and the line signal, and then perform further image processing such as denoising on the image data, thereby outputting a signal with higher image quality.

In one possible implementation, the FPGA receives a line of image data from the MIPI signal through the LVDS interface, including: when the FPGA acquires the header information in the MIPI signal through the LVDS interface and comprises the set characters, the FPGA starts to receive and decode a row of serial effective image data from the MIPI signal.

In a second aspect, an embodiment of the present invention provides an FPGA, which includes an LVDS interface and a GPIO interface, where the LVDS interface is configured to receive an MIPI signal from an image sensor; the GPIO interface is used for receiving the MIPI signal after voltage boosting while the LVDS interface receives the MIPI signal. Wherein the memory is used to store one or more computer programs; the one or more computer programs stored in the memory, when executed by the processor, enable the FPGA to implement any one of the possible design methodologies of the first aspect described above.

In a third aspect, an embodiment of the present invention further provides a terminal device, where the terminal device includes a computer program, and when the computer program runs on the terminal device, the terminal device is caused to execute any one of the possible design methods in any one of the above aspects.

In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, where the computer-readable storage medium includes a computer program, and when the computer program runs on a terminal device, the computer program causes the terminal device to perform any one of the possible design methods of any one of the aspects.

In a fifth aspect, an embodiment of the present invention further provides a method including a computer program product, when the computer program product runs on a terminal device, causing the terminal device to execute any one of the possible designs of any one of the aspects.

For the beneficial effects of the second to fifth aspects, reference may be made to the description of the first aspect, and the description is not repeated.

Drawings

Fig. 1 is a schematic diagram of a physical layer and signal levels of an MIPI according to an embodiment of the present invention;

fig. 2 is a communication system according to an embodiment of the present invention;

fig. 3 is a schematic flowchart of a method for receiving an MIPI signal based on an FPGA according to an embodiment of the present invention;

FIG. 4 is a schematic circuit diagram according to an embodiment of the present invention;

fig. 5 is a schematic diagram of another communication system according to an embodiment of the present invention;

fig. 6 is a schematic diagram of a terminal according to an embodiment of the present invention.

Detailed Description

The technical solution of the present invention will be described with reference to the accompanying drawings.

Hereinafter, some terms referred to hereinafter will be explained to facilitate understanding by those skilled in the art.

(1) Application Specific Integrated Circuit (ASIC)

An ASIC is a special purpose chip that is specialized chip for certain specific requirements. Such as a dedicated audio processor and a video processor, and currently, many dedicated Artificial Intelligence (AI) chips can also be regarded as an ASIC.

(2) FPGA (field programmable gate array)

An FPGA is a logic device composed of many logic units, wherein the logic units include gates, lookup tables and flip-flops, and the FPGA has rich hardware resources, strong parallel processing capability and flexible reconfigurable capability, and is increasingly and widely applied in many fields such as data processing, communication, network and the like. The FPGA is a product developed further on the basis of programmable devices such as PAL, GAL, CPLD and the like, and appears as a semi-custom circuit in the field of Application Specific Integrated Circuit (ASIC), thereby not only solving the defects of the custom circuit, but also overcoming the defect of limited gate circuit number of the original programmable device.

The FPGA comprises a general purpose input/output (GPIO) interface and a Low Voltage Differential Signaling (LVDS) interface, and each pin of the FPGA is defined as an LVDS interface or a GPIO interface after being electrified, so that the electrical characteristics of the pins of the FPGA are stable and unchangeable after being electrified.

The FPGA-based MIPI signal receiving method provided by the invention can simultaneously receive MIPI signals by adopting the LVDS interface and the GPIO interface of the FPGA based on the signal level characteristics of the MIPI signals in different transmission modes, wherein the MIPI signals are boosted and then accessed into the GPIO interface, so that the problem that the MIPI signals cannot be correctly received due to inconsistent high level amplitudes of the FPGA can be solved. Because the LVDS interface can identify the transmission mode of the MIPI signal and switch the transmission mode into the low-power transmission mode, the FPGA determines the received MIPI signal as blanking data in the low-power transmission mode; the GPIO interface can identify the transmission mode of the MIPI signal and switch the transmission mode into a high-speed transmission mode, and under the high-speed transmission mode, the LVDS interface can receive serial effective image data of each row of the MIPI signal, so that the MIPI signal can be received by the FPGA without the help of other chips.

The technical solution in the embodiment of the present invention is described below with reference to the drawings in the embodiment of the present invention. In the description of the embodiments of the present invention, the terminology used in the following embodiments is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in the description of the invention and the appended claims, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, such as "one or more", unless the context clearly indicates otherwise. It should also be understood that in the following embodiments of the present invention, "at least one", "one or more" means one or more than two (including two). The term "and/or" is used to describe an association relationship that associates objects, meaning that three relationships may exist; for example, a and/or B, may represent: a alone, both A and B, and B alone, where A, B may be singular or plural. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship.

Reference throughout this specification to "one embodiment" or "some embodiments," or the like, means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the present invention. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," or the like, in various places throughout this specification are not necessarily all referring to the same embodiment, but rather "one or more but not all embodiments" unless specifically stated otherwise. The terms "comprising," "including," "having," and variations thereof mean "including, but not limited to," unless expressly specified otherwise. The term "coupled" includes both direct and indirect connections, unless otherwise noted. "first" and "second" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated.

In the embodiments of the present invention, words such as "exemplary" or "for example" are used to mean serving as examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "e.g.," an embodiment of the present invention is not necessarily to be construed as preferred or advantageous over other embodiments or designs. Rather, use of the word "exemplary" or "such as" is intended to present concepts related in a concrete fashion.

As shown in fig. 2, a communication system suitable for use in embodiments of the present invention includes an image sensor 10, an FPGA 20. The image sensor 10 sends an MIPI signal to the FPGA through the MIPI interface. The MIPI signal includes MIPI CLK (clock) information in addition to MIPI data (e.g., MIPI data0, MIPI data1, MIPI data2, and MIPI data 3). The MIPI signal may be a digital-to-speech interpolation (DSI) signal from a display interface, or a complementary metal oxide semiconductor image sensor (CMOS sensor interface, CSI) signal from a camera interface, or may be a signal from other MIPI interfaces of the mobile terminal, and the MIPI interface 410 may be an interface such as GTX, GTH, or the like.

The FPGA20 comprises an LVDS interface and a GPIO interface, wherein the LVDS interface is used for receiving the MIPI signal from the image sensor, and the GPIO interface is used for receiving the MIPI signal after boosting while the LVDS interface receives the MIPI signal. Because the MIPI signal is boosted before being accessed into the GPIO interface, the problem that the MIPI signal cannot be correctly received due to the fact that the amplitude of the high level of the FPGA is inconsistent can be solved. The FPGA20 may alternatively be a terminal device containing an ASIC chip or a terminal device containing an FPGA or an embedded system containing an ASIC/FPGA. Since the MIPI signal output by the image sensor 10 is a differential signal, the FPGA20 can convert the received image data from the image sensor 10 into parallel image data, and output a frame synchronization signal (vsync) and a row synchronization signal (hsync) and image data corresponding to the row, so that an image signal processor or an upper computer connected to the FPGA can distinguish which frame and which row the parallel image data belongs to.

The terminal device comprising the ASIC/FPGA of the present invention may also be referred to as a User Equipment (UE), an access terminal, a subscriber unit, a subscriber station, a mobile station, a remote terminal, a mobile device, a user terminal, a wireless communication device, a user agent, or a user device. The terminal device in the embodiment of the present invention may be a mobile phone (mobile phone), a tablet computer (Pad), a smart printer, a train detector, a gas station detector, a computer with a wireless transceiving function, a Virtual Reality (VR) terminal device, an Augmented Reality (AR) terminal device, a wireless terminal in industrial control (industrial control), a wireless terminal in self driving (self driving), a wireless terminal in remote medical (remote medical), a wireless terminal in smart grid (smart grid), a wireless terminal in transportation safety, a wireless terminal in city (smart city), a wireless terminal in smart home (smart home), and the like. The embodiment of the invention does not limit the application scenarios.

It should be understood that fig. 2 is a simplified schematic diagram of an example for ease of understanding only, and that other image signal processors may be included in the communication system or other terminal devices may be included, which are not shown in fig. 2.

An embodiment of the present invention provides a method for receiving an MIPI signal based on an FPGA, where the method may be performed by the FPGA20 in the communication system shown in fig. 1. It is to be understood that the steps performed by FPGA20 may also be specifically performed by one or more modules or components of FPGA20 in the present invention. As shown in fig. 3, a method for receiving an MIPI signal based on an FPGA according to an embodiment of the present invention is shown, where the method includes:

s301, in the process that the FPGA receives the MIPI signal, when the LVDS interface determines that the MIPI signal is switched from a high-speed transmission mode to a low-power transmission mode, the received MIPI signal is determined to be blanking data in the low-power transmission mode; when the GPIO interface determines that the MIPI signal is switched from the low-power-consumption transmission mode to the high-speed transmission mode, the LVDS interface receives and decodes the MIPI signal to obtain a row of serial effective image data in the high-speed transmission mode.

In this step, in a possible implementation manner, the determining, by the LVDS interface, a manner of switching the MIPI signal from the high-speed transmission mode to the low-power transmission mode includes: the LVDS interface acquires tail information of the received MIPI signal; and determining that the MIPI signal is switched from a high-speed transmission mode to a low-power consumption transmission mode according to the tail information. In the method, the LVDS can determine that the MIPI signal is switched from the high-speed transmission mode to the low-power-consumption transmission mode according to the tail information of the received MIPI signal, and the tail information can be preset characters.

In one possible implementation, the mode of the GPIO interface determining that the MIPI signal is switched from the low power consumption transmission mode to the high speed transmission mode includes: and determining that the MIPI signal is switched from a low-power-consumption transmission mode to a high-speed transmission mode according to the change from the condition that the MIPI signal can be received to the condition that the MIPI signal cannot be received by the GPIO interface. That is to say, because the MIPI signal can be received by the GPIO interface after being boosted from 1.2V to 2.5 and still cannot be received by the GPIO interface after being boosted from 200mV, the GPIO interface can determine that the MIPI signal is switched from the low-power-consumption transmission mode to the high-speed transmission mode according to the timing at which the MIPI signal can be received and cannot be received, and at this time, the FPGA can receive valid data of the MIPI signal through the LVDS interface.

And S302, repeatedly executing the steps until the LVDS interface receives and decodes the serial effective image data of the last line of the MIPI signal, and converting the serial effective image data of each line into parallel image data by the FPGA.

In the method, the MIPI signal is a low-voltage differential signal in a high-speed transmission mode, so that the method has the advantage of high anti-interference performance, and the MIPI signal is a single-ended signal in a low-power transmission mode, so that the method has the advantage of low power consumption. The LVDS interface of the FPGA can correctly decode the MIPI signal in the high-speed transmission mode, but cannot correctly decode the MIPI signal in the low-power-consumption transmission mode. Therefore, in the method, the GPIO interface can identify the transmission mode of the MIPI signal and switch to the high-speed transmission mode, and in the high-speed transmission mode, the LVDS interface can receive serial effective image data of each line of the MIPI signal; the LVDS interface can identify that the transmission mode of the MIPI signal is switched to a low-power-consumption transmission mode, and the FPGA determines that the received MIPI signal is blanking data in the low-power-consumption transmission mode, so that the MIPI signal is received by the FPGA without the help of other chips.

Generally, one frame of image data is composed of 1080 lines, so that after the LVDS interface receives 1080 lines of serial effective image data, one frame of image data can be obtained, and then the serial effective image data is finally converted into a parallel data signal in a form similar to a digital video port (digital video port) so as to facilitate a digital video display device to correctly display an image. Thus, the FPGA can receive MIPI signals.

Referring to fig. 4, the MIPI signal includes MIPI _ R _ DP1 and MIPI _ R _ DN1, and in the low power transmission mode, MIPI _ R _ DP1 and MIPI _ R _ DN1 are not differential signals but are independent of each other; in high-speed transmission mode, MIPI _ R _ DP1 and MIPI _ R _ DN1 are differential signals. As can be seen from fig. 4, MIPI _ R _ DP1 and MIPI _ R _ DN1 access an LVDS interface of the FPGA, and in addition, MIPI _ R _ DP1 and MIPI _ R _ DN1 also access a boost circuit chip U5, and MIPI _ R _ DP1 and MIPI _ R _ DN1 access a GPIO interface of the FPGA after being boosted by the boost circuit chip U5, so that, in a high-speed transmission mode, the FPGA can receive MIPI signals through the LVDS interface, and when the MIPI signals are switched from the high-speed transmission mode to a low-power mode, the FPGA can determine that the transmission mode is switched through the LVDS interface, so that the MIPI signals are in a line blanking (lv-blanking) time, that is, image data received by the FPGA is invalid data in the low-power mode. When the MIPI signal is switched from the low-power-consumption transmission mode to the high-speed transmission mode, the FPGA can determine that the transmission mode is switched through the GPIO interface, and therefore the FPGA receives one line of image data from the MIPI signal through the LVDS interface. Specifically, when the header of the valid data of the MIPI signal acquired by the FPGA through the LVDS interface includes a set character (e.g., 0XB8), the FPGA starts receiving a line of image data from the MIPI signal. Repeating the steps until the FPGA receives the last line of image data of the MIPI signal through the LVDS interface, and finally converting the serial image data of each line of the received MIPI signal into parallel image data by the FPGA.

Embodiments of the present invention also provide a communication system as shown in fig. 5, including an image sensor 10, an FPGA20, and an image signal processor 30. Referring to fig. 5, the FPGA20 includes a MIPI decoding module 501 and a DVP data generating module 502. Wherein, the MIPI decoding module 501 is connected to the MIPI interface and adapted to decode an encoded MIPI signal (the MIPI signal includes image data and recovered clock information), the MIPI decoding module 501 may be an 8b/10b decoder, a 64b/66b decoder, or a manchester decoder. In one embodiment, the MIPI decoding module 501 recovers clock information through a dedicated clock recovery chip of a built-in PLL circuit, and the recovered clock information provides a clock source and a data sampling reference for a subsequent circuit. A Digital Video Port (DVP) data generating module 502 for generating parallel image data. The image signal processor 30 is configured to send the frame signal, the line signal, and the parallel image data corresponding to the line signal to the image signal processor 30, and the image signal processor 30 performs subsequent processing such as noise reduction.

Fig. 6 shows a schematic structural diagram of a terminal device 600. The terminal device 600 may be configured to implement the method described in the above method embodiment, and refer to the description in the above method embodiment. The terminal device 600 may be a chip, a network device (e.g., a base station), a terminal device or other network devices, etc.

The terminal device 600 comprises one or more processors 601. The processor 601 may be a general purpose processor or a special purpose processor, etc. For example, a baseband processor, or a central processor. The baseband processor may be used to process communication protocols and communication data, and the central processor may be used to control a communication device (e.g., a base station, a terminal, or a chip), execute a software program, and process data of the software program. The communication device may include a transceiving unit to enable input (reception) and output (transmission) of signals. For example, the communication device may be a chip, and the transceiving unit may be an input and/or output circuit of the chip, or a communication interface. The chip can be used for a terminal or a base station or other network equipment. As another example, the communication device may be a terminal or a base station or other network equipment, and the transceiver unit may be a transceiver, a radio frequency chip, or the like.

The terminal device 600 comprises one or more processors 601, and the one or more processors 601 can implement the method shown in the FPGA in the embodiment shown in fig. 3.

Optionally, the processor 601 may also implement other functions besides the method of the embodiment shown in fig. 3.

Optionally, in one design, the processor 601 may also include instructions 603, which may be executed on the processor, so that the communication device 700 performs the method described in the above method embodiment.

In yet another possible design, the terminal device 600 may include one or more memories 602 having instructions 604 stored thereon, which are executable on the processor, so that the terminal device 600 performs the methods described in the above method embodiments. Optionally, the memory may further store data therein. Instructions and/or data may also be stored in the optional processor. For example, the one or more memories 602 may store the corresponding relations described in the above embodiments, or the related parameters or tables referred to in the above embodiments, and the like. The processor and the memory may be provided separately or may be integrated together.

In yet another possible design, the terminal device 600 may also include a communication interface 605 and an antenna 606. The processor 601 may be referred to as a processing unit and controls a communication device (terminal or base station). The communication interface 605 may be referred to as a transceiver, a transceiving circuit, a transceiver, or the like, and is used for implementing transceiving functions of a communication device through the antenna 606.

It should be understood that the processor in the embodiments of the present invention may be a Central Processing Unit (CPU), and the processor may also be other general purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.

It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

In the embodiments provided in the present invention, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.

The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.

The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention, and all the changes or substitutions should be covered within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the appended claims.

It should be noted that the processor in the embodiments of the present invention may be an integrated circuit chip having signal processing capability. In implementation, the steps of the above method embodiments may be performed by integrated logic circuits of hardware in a processor or instructions in the form of software. The Processor may be a general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, or discrete hardware components. The various methods, steps and logic blocks disclosed in the embodiments of the present invention may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present invention may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The storage medium is located in a memory, and a processor reads information in the memory and completes the steps of the method in combination with hardware of the processor.

It will be appreciated that the memory in embodiments of the invention may be either volatile memory or nonvolatile memory, or may include both volatile and nonvolatile memory. The non-volatile Memory may be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable PROM (EEPROM), or a flash Memory. Volatile Memory can be Random Access Memory (RAM), which acts as external cache Memory. By way of example, but not limitation, many forms of RAM are available, such as Static random access memory (Static RAM, SRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic random access memory (Synchronous DRAM, SDRAM), Double Data Rate Synchronous Dynamic random access memory (DDR SDRAM), Enhanced Synchronous SDRAM (ESDRAM), Synchronous link SDRAM (SLDRAM), and Direct Rambus RAM (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to comprise, without being limited to, these and any other suitable types of memory.

An embodiment of the present invention further provides a computer-readable medium, on which a computer program is stored, where the computer program, when executed by a computer, implements the method of any of the above method embodiments.

The embodiment of the invention also provides a computer program product, and the computer program product realizes the method of any one of the above method embodiments when being executed by a computer.

In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with embodiments of the invention, to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored on a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website, computer, server, or data center to another website, computer, server, or data center via wire (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., Digital Video Disk (DVD)), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.

It should be understood that the processing device may be a chip, the processor may be implemented by hardware or software, and when implemented by hardware, the processor may be a logic circuit, an integrated circuit, or the like; when implemented in software, the processor may be a general-purpose processor implemented by reading software code stored in a memory, which may be integrated in the processor, located external to the processor, or stand-alone.

Those of ordinary skill in the art will appreciate that the elements and algorithm steps of the examples described in connection with the embodiments disclosed herein may be embodied in electronic hardware, computer software, or combinations of both, and that the components and steps of the examples have been described in a functional general in the foregoing description for the purpose of illustrating clearly the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.

In short, the above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.