阅读说明：本技术 视觉假体装置、系统及其控制方法、存储介质 (Visual prosthesis device, system, control method thereof and storage medium ) 是由 吴天准 徐臻 于 2019-11-27 设计创作，主要内容包括：本发明实施例公开一种视觉假体装置、系统及其控制方法、存储介质,其中,视觉假体装置的主控模块根据第一通信模块接收到的电极控制指令控制刺激脉冲生成模块产生刺激脉冲信号,电极阵列不仅可以接收刺激脉冲信号以向视觉神经元组织释放电刺激,而且可以接收视觉神经组织产生的模拟神经元信号,再结合信号放大和模数转换模块对模拟神经元信号放大和模数转换,得到数字神经元信号,主控模块可以控制第一通信模块向外传输数字神经元信号。而包括体外控制模块、第二通信模块和视觉假体装置的视觉假体系统及其控制方法,可以接收并处理分析数字神经元信号,找到视觉假体装置的每个电极的最优刺激参数,提高了电极的最优刺激参数的获取效率和准确度。(The embodiment of the invention discloses a visual prosthesis device, a system, a control method thereof and a storage medium, wherein a main control module of the visual prosthesis device controls a stimulation pulse generation module to generate stimulation pulse signals according to an electrode control instruction received by a first communication module, an electrode array not only can receive the stimulation pulse signals to release electrical stimulation to visual neuron tissues, but also can receive analog neuron signals generated by the visual neuron tissues, and then the signal amplification and analog-to-digital conversion module is combined to amplify and convert the analog neuron signals to obtain digital neuron signals, and the main control module can control the first communication module to transmit the digital neuron signals outwards. The visual prosthesis system comprising the in-vitro control module, the second communication module and the visual prosthesis device and the control method thereof can receive, process and analyze the digital neuron signals, find the optimal stimulation parameters of each electrode of the visual prosthesis device, and improve the acquisition efficiency and accuracy of the optimal stimulation parameters of the electrodes.)

1. A visual prosthesis device is characterized by comprising a first communication module, a main control module, a stimulation pulse generation module, an electrode array and a signal amplification and analog-to-digital conversion module, wherein the first communication module is connected with the main control module, the output end of the main control module is connected with the input end of the stimulation pulse generation module, the output end of the stimulation pulse generation module is connected with the input end of an electrode in the electrode array, the output end of the electrode is connected with the input end of the signal amplification and analog-to-digital conversion module, and the output end of the signal amplification and analog-to-digital conversion module is connected with the input end of the main control module; wherein:

the first communication module is used for receiving an electrode control instruction and transmitting a digital neuron signal;

the main control module is used for controlling the stimulation pulse generation module to generate a stimulation pulse signal according to the electrode control instruction;

the stimulation pulse generation module is used for generating the stimulation pulse signal according to the electrode control instruction;

the electrodes in the electrode array are used for receiving the stimulation pulse signals, releasing electrical stimulation to the visual nerve tissues to generate phosphene, and receiving simulated neuron signals generated by the visual nerve tissues;

the signal amplification and analog-to-digital conversion module is used for amplifying the analog neuron signals and carrying out digital processing to obtain the digital neuron signals.

2. The apparatus of claim 1, wherein the main control module comprises a first processor, a second processor and a data register module including a plurality of data registers, an output of the signal amplifying and analog-to-digital converting module is connected to an input of the first processor, an output of the first processor is connected to an input of the data register module, the data register module is connected to the second processor, the second processor is connected to the first communication module, the second processor is configured to fetch the digital neuron signals stored in the data registers and send the digital neuron signals to the first communication module, and the first processor is configured to store new digital neuron signals in the emptied data registers.

3. The apparatus of claim 1, wherein the stimulation pulse generation module comprises a plurality of sub-stimulation pulse generation modules, the number of the sub-stimulation pulse generation modules is the same as the number of the electrodes of the electrode array, and an output terminal of one of the sub-stimulation pulse generation modules is connected to an input terminal of one of the electrodes of the electrode array.

4. The device according to any one of claims 1 to 3, further comprising a temperature sensor for detecting the temperature of the optic nerve tissue and/or an impedance sensor for detecting the electrode impedance of the electrode array, wherein the output end of the temperature sensor and the output end of the impedance sensor are respectively connected with the input end of the main control module.

5. A visual prosthesis system comprising an extracorporeal control module, a second communication module and the visual prosthesis device of any one of claims 1 to 4, the extracorporeal control module being connected to the second communication module, the second communication module being connected to the first communication module, the extracorporeal control module being configured to send the electrode control instructions to the first communication module and to receive and process the digital neuron signals.

6. A visual prosthesis system according to claim 5, further comprising an alarm module for outputting an alarm signal, the output of the extracorporeal control module being connected to the input of the alarm module.

7. A control method of a visual prosthesis system applied to the visual prosthesis system of claim 5 or 6, characterized by comprising:

sending the electrode control instruction to the visual prosthesis device, wherein the electrode control instruction comprises an electrode number of a stimulation electrode, an initial stimulation parameter of the stimulation electrode and an electrode number of a recording electrode group, and the electrode control instruction is used for indicating the visual prosthesis device to control the electrode with the corresponding number of the electrode array to send out electrical stimulation to visual nerve tissue according to the electrode number and the initial stimulation parameter of the stimulation electrode;

receiving a first digital neuron signal of an electrode corresponding to the electrode array returned by the visual prosthesis device according to the electrode number of the recording electrode group;

judging whether a preset condition is met or not according to a first digital neuron signal of the recording electrode group and a second digital neuron signal of each electrode in the recording electrode group in a resting state; if the preset conditions are met, taking the initial stimulation parameters as the optimal stimulation parameters of the stimulation electrode; and if the preset condition is not met, modifying the initial stimulation parameter until the preset condition is met.

8. The method of claim 7, wherein the electrode numbers of the sets of recording electrodes include an electrode number of a first set of recording electrodes that does not include the stimulating electrode, an electrode number of a second set of recording electrodes that includes the stimulating electrode;

judging whether a preset condition is met or not according to a first digital neuron signal of the recording electrode group and a second digital neuron signal of each electrode in the recording electrode group in a resting state; if the preset conditions are met, taking the initial stimulation parameters as the optimal stimulation parameters of the stimulation electrode; if the preset condition is not met, modifying the initial stimulation parameter until the preset condition is met, wherein the step of modifying the initial stimulation parameter comprises the following steps:

acquiring the number of first electrodes in the first recording electrode group, of which first digital neuron signals reach a first preset threshold value according to first digital neuron signals of the first recording electrode group and second digital neuron signals of all electrodes in the first recording electrode group in a resting state, wherein the first preset threshold value is the product of the second digital neuron signals of the electrodes corresponding to the first digital neuron signals and a first preset multiple;

acquiring the number of second electrodes in the second recording electrode group, of which the first digital neuron signals reach a second preset threshold value according to the first digital neuron signals of the second recording electrode group and second digital neuron signals of all electrodes in the second recording electrode group in a resting state, wherein the second preset threshold value is the product of the second digital neuron signals of the electrodes corresponding to the first digital neuron signals and a second preset multiple;

adding the number of the first electrodes and the number of the second electrodes to obtain an effective stimulation range, and judging according to the effective stimulation range and a preset value, wherein when the effective stimulation range is equal to the preset value, the initial stimulation parameters are used as the optimal stimulation parameters of the stimulation electrodes; if the effective stimulation range is greater than or less than the preset value, modifying the initial stimulation parameter until the effective stimulation range is equal to the preset value.

9. The method according to claim 7 or 8, characterized in that the method further comprises:

sending a test instruction to the visual prosthetic device, the test instruction including optimal stimulation parameters of stimulation electrodes and a standard test image, the standard test image specifying stimulation electrodes, the test instruction being for instructing the visual prosthetic device to project the standard test image on the electrode array and to drive the stimulation electrodes according to the optimal stimulation parameters of the stimulation electrodes;

receiving a third digital neuron signal of each electrode of the electrode array corresponding to the test instruction returned by the visual prosthesis device;

carrying out image reconstruction according to the third digital neuron signal to obtain a perception image;

calculating the similarity of the standard test image and the perception image;

and judging whether the optimal stimulation parameters of the stimulation electrodes meet requirements or not according to the image similarity and a preset similarity, wherein the image similarity is greater than or equal to the preset similarity, the optimal stimulation parameters of the stimulation electrodes meet the requirements, the image similarity is smaller than the preset similarity, and the optimal stimulation parameters of the stimulation electrodes do not meet the requirements.

10. A computer storage medium, characterized in that it stores a computer program comprising program instructions which, when executed by a processor, perform the method of controlling a visual prosthesis system according to any one of claims 7-9.

Technical Field

The present invention relates to the field of visual prosthesis technology, and in particular, to a visual prosthesis device, a visual prosthesis system, a control method for a visual prosthesis system, and a computer storage medium.

Background

Visual prosthesis technology belongs to one of functional electrical stimulation. The blind person stimulation device applies specific artificial electrical stimulation to the intact part of the visual pathway to excite nerve cells and simulate the effect of natural light stimulation to enable the blind person to generate visual perception by utilizing the characteristic that most blind persons often only have lesion on one part of the visual pathway and the structure and the function of the nerve tissues of the rest part are intact. The visual perception produced by the fixed-point electrical stimulation of a single electrode is called optical illusion. The visual prosthesis system comprises a video acquisition device (usually a small-sized camera) positioned outside the human body, a video processing module, an electric stimulation coding module and a multi-electrode array implanted to a specific part of a visual passage in the human body, wherein the video acquisition device, the video processing module, the electric stimulation coding module and the multi-electrode array form a visual prosthesis on-body chip. The working principle of the visual prosthesis is as follows: real-time video images acquired by the video acquisition equipment are processed and converted into signals for driving the multi-electrode array. The multi-electrode array applies current stimulation with certain amplitude, waveform and frequency to the visual nerve tissue to excite the visual neurons, so that the patient can generate visual feeling.

Due to the different distances between different sub-electrodes and the visual neurons in the multi-electrode array, the impedance of different sub-electrodes is different, so that the minimum stimulation intensity required by different sub-electrodes to activate the adjacent visual neurons is different, and therefore, the stimulation parameters of each sub-electrode need to be optimized separately.

In the prior art, the optimization process of the electrical stimulation parameters of the electrodes of the visual prosthesis is roughly as follows: the medical staff transmits a set of continuously enhanced stimulation parameters to the visual prosthesis on-body chip, and tests which electrodes in the electrode array can enable the testee to perceive the phosphenes with 50% probability under which parameters. The healthcare selects from these parameters, setting optimal stimulation parameters for each sub-electrode in the electrode array. However, relying on the verbal description of the patient for electrode stimulation parameter optimization would consume a significant amount of time and effort on the part of the medical practitioner and the patient, since the visual prosthesis chip contains a large number of electrodes. Furthermore, the verbal description does not accurately reflect the effective range of electrode stimulation, which significantly reduces the spatial resolution of electrode stimulation, which also means that electrical stimulation is often too high, which causes heat accumulation and increases the risk of thermally damaging nerve tissue. Also, over time, contact between the body chip and the neural tissue changes, causing changes in the electrode impedance, resulting in failure of the previously determined electrical stimulation parameters, which the patient needs to regularly seek the assistance of medical personnel, further increasing the burden on medical workers and patients. Finally, when the subject is an animal, the animal cannot verbally describe the phosphenes it perceives, limiting the development of electrode stimulation parameter optimization.

Disclosure of Invention

The embodiment of the invention provides a visual prosthesis device, a visual prosthesis system, a control method and a storage medium thereof, which can improve the accuracy and the acquisition efficiency of the optimal stimulation parameters of a stimulation electrode.

In one aspect, an embodiment of the present invention provides a visual prosthesis device, including a first communication module, a main control module, a stimulation pulse generation module, an electrode array, and a signal amplification and analog-to-digital conversion module, where the first communication module is connected to the main control module, an output end of the main control module is connected to an input end of the stimulation pulse generation module, an output end of the stimulation pulse generation module is connected to an input end of an electrode in the electrode array, an output end of the electrode is connected to an input end of the signal amplification and analog-to-digital conversion module, and an output end of the signal amplification and analog-to-digital conversion module is connected to an input end of the main control module; wherein:

the first communication module is used for receiving an electrode control instruction and transmitting a digital neuron signal;

the main control module is used for controlling the stimulation pulse generation module to generate a stimulation pulse signal according to the electrode control instruction;

the stimulation pulse generation module is used for generating the stimulation pulse signal according to the electrode control instruction;

the electrodes in the electrode array are used for receiving the stimulation pulse signals, releasing electrical stimulation to the visual nerve tissues to generate phosphene, and receiving simulated neuron signals generated by the visual nerve tissues;

the signal amplification and analog-to-digital conversion module is used for amplifying the analog neuron signals and carrying out digital processing to obtain the digital neuron signals.

Optionally, the main control module includes a first processor, a second processor and a data register module including a plurality of data registers, an output end of the signal amplification and analog-to-digital conversion module is connected with an input end of the first processor, an output end of the first processor is connected with an input end of the data register module, the data register module is connected with the second processor, the second processor is connected with the first communication module, the second processor is used for taking out the digital neuron signals stored in the data register and then sending the digital neuron signals to the first communication module, and the first processor is used for storing the digital neuron signals in the vacated data register.

Optionally, the stimulation pulse generation module includes a plurality of sub-stimulation pulse generation modules, the number of the sub-stimulation pulse generation modules is the same as the number of the electrodes of the electrode array, and an output end of one sub-stimulation pulse generation module is connected to an input end of one electrode of the electrode array.

Optionally, the device further includes a temperature sensor for detecting a temperature of the optic nerve tissue and/or an impedance sensor for detecting an electrode impedance of the electrode array, and an output end of the temperature sensor and an output end of the impedance sensor are respectively connected to an input end of the main control module.

In another aspect, an embodiment of the present invention provides a visual prosthesis system, including an extracorporeal control module, a second communication module, and the visual prosthesis device, where the extracorporeal control module is connected to the second communication module, the second communication module is connected to the first communication module, and the extracorporeal control module is configured to send the electrode control command to the first communication module, and receive and process the digital neuron signal.

Optionally, the visual prosthesis system further comprises an alarm module for outputting an alarm signal, and an output end of the extracorporeal control module is connected with an input end of the alarm module.

In another aspect, an embodiment of the present invention provides a method for controlling a visual prosthesis system, which is applied to the visual prosthesis system, and includes:

sending the electrode control instruction to the visual prosthesis device, wherein the electrode control instruction comprises an electrode number of a stimulation electrode, an initial stimulation parameter of the stimulation electrode and an electrode number of a recording electrode group, and the electrode control instruction is used for indicating the visual prosthesis device to control the electrode with the corresponding number of the electrode array to send out electrical stimulation to visual nerve tissue according to the electrode number and the initial stimulation parameter of the stimulation electrode;

receiving a first digital neuron signal of an electrode corresponding to the electrode array returned by the visual prosthesis device according to the electrode number of the recording electrode group;

judging whether a preset condition is met or not according to a first digital neuron signal of the recording electrode group and a second digital neuron signal of each electrode in the recording electrode group in a resting state; if the preset conditions are met, taking the initial stimulation parameters as the optimal stimulation parameters of the stimulation electrode; and if the preset condition is not met, modifying the initial stimulation parameter until the preset condition is met.

Optionally, the electrode numbers of the recording electrode groups include an electrode number of a first recording electrode group, an electrode number of a second recording electrode group, the first recording electrode group does not include the stimulation electrode, and the second recording electrode group includes the stimulation electrode;

judging whether a preset condition is met or not according to a first digital neuron signal of the recording electrode group and a second digital neuron signal of each electrode in the recording electrode group in a resting state; if the preset conditions are met, taking the initial stimulation parameters as the optimal stimulation parameters of the stimulation electrode; if the preset condition is not met, modifying the initial stimulation parameter until the preset condition is met, wherein the step of modifying the initial stimulation parameter comprises the following steps:

acquiring the number of first electrodes in the first recording electrode group, of which first digital neuron signals reach a first preset threshold value according to first digital neuron signals of the first recording electrode group and second digital neuron signals of all electrodes in the first recording electrode group in a resting state, wherein the first preset threshold value is the product of the second digital neuron signals of the electrodes corresponding to the first digital neuron signals and a first preset multiple;

acquiring the number of second electrodes in the second recording electrode group, of which the first digital neuron signals reach a second preset threshold value according to the first digital neuron signals of the second recording electrode group and second digital neuron signals of all electrodes in the second recording electrode group in a resting state, wherein the second preset threshold value is the product of the second digital neuron signals of the electrodes corresponding to the first digital neuron signals and a second preset multiple;

adding the number of the first electrodes and the number of the second electrodes to obtain an effective stimulation range, and judging according to the effective stimulation range and a preset value, wherein when the effective stimulation range is equal to the preset value, the initial stimulation parameters are used as the optimal stimulation parameters of the stimulation electrodes; if the effective stimulation range is greater than or less than the preset value, modifying the initial stimulation parameter until the effective stimulation range is equal to the preset value.

Optionally, the method further comprises:

sending a test instruction to the visual prosthetic device, the test instruction including optimal stimulation parameters of stimulation electrodes and a standard test image, the standard test image specifying stimulation electrodes, the test instruction being for instructing the visual prosthetic device to project the standard test image on the electrode array and to drive the stimulation electrodes according to the optimal stimulation parameters of the stimulation electrodes;

receiving a third digital neuron signal of each electrode of the electrode array corresponding to the test instruction returned by the visual prosthesis device;

carrying out image reconstruction according to the third digital neuron signal to obtain a perception image;

calculating the similarity of the standard test image and the perception image;

and judging whether the optimal stimulation parameters of the stimulation electrodes meet requirements or not according to the image similarity and a preset similarity, wherein the image similarity is greater than or equal to the preset similarity, the optimal stimulation parameters of the stimulation electrodes meet the requirements, the image similarity is smaller than the preset similarity, and the optimal stimulation parameters of the stimulation electrodes do not meet the requirements.

In another aspect, an embodiment of the present invention provides a computer storage medium storing a computer program comprising program instructions that, when executed by a processor, perform the control method of a visual prosthesis system.

In the visual prosthesis device in the embodiment of the invention, the main control module controls the stimulation pulse generation module to generate the stimulation pulse signal according to the electrode control instruction received by the first communication module, the electrode array not only can receive the stimulation pulse signal to release electrical stimulation to the visual neuron tissue, but also can receive the analog neuron signal generated by the visual neuron tissue, and then the signal amplification and analog-to-digital conversion module is combined to amplify and convert the analog neuron signal to obtain the digital neuron signal, and the main control module can control the first communication module to transmit the digital neuron signal outwards. The visual prosthesis system comprising the in-vitro control module, the second communication module and the visual prosthesis device can receive, process and analyze the digital neuron signals, and find the optimal stimulation parameters of each electrode of the visual prosthesis device in a closed-loop feedback mode, so that the acquisition efficiency and accuracy of the optimal stimulation parameters of the electrodes are improved. In addition, in the control method of the visual prosthesis system, the visual prosthesis device is instructed to control the stimulation electrodes to send out electric stimulation according to the initial stimulation parameters according to the electrode control instruction, and then whether preset conditions are met or not is judged according to a first digital neuron signal of an electrode corresponding to the electrode array returned by the visual prosthesis device according to the electrode number of the recording electrode group and according to the first digital neuron signal of the recording electrode group and a second digital neuron signal of each electrode in the recording electrode group in a resting state; if the preset conditions are met, the initial stimulation parameters are used as the optimal stimulation parameters of the stimulation electrode; if the preset conditions are not met, modifying the initial stimulation parameters until the preset conditions are met; the optimal stimulation parameters of the stimulation electrode are obtained, the accuracy and the obtaining efficiency of the optimal stimulation parameters of the stimulation electrode are effectively improved, and the development cost of the visual prosthesis device is reduced.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

FIG. 1 is a schematic diagram of a visual prosthesis system according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of a visual prosthetic device according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of a visual prosthesis system according to an embodiment of the present invention;

FIG. 4 is a flow chart of a control method of a visual prosthesis system according to an embodiment of the present invention;

FIG. 5 is a flow chart of a control method of a visual prosthesis system according to an embodiment of the present invention;

FIG. 6 is a flow chart of a control method of a visual prosthesis system according to an embodiment of the present invention;

FIG. 7 is a flowchart illustrating a method for controlling a visual prosthesis system according to an embodiment of the present invention;

FIG. 8 is a flow chart of a control method of a visual prosthesis system according to an embodiment of the present invention;

FIG. 9 is a schematic structural diagram of a control device of a visual prosthesis system according to an embodiment of the present invention;

FIG. 10 is a schematic structural diagram of a control device of a visual prosthesis system according to an embodiment of the present invention;

FIG. 11 is a schematic structural diagram of a control device of a visual prosthesis system according to an embodiment of the present invention;

fig. 12 is a schematic structural diagram of a terminal device according to an embodiment of the present invention.

Detailed Description

The technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

It should be understood that the terms "first," "second," and the like in the description and claims of this application and in the drawings are used for distinguishing between different objects and not for describing a particular order. Furthermore, the terms "include" and "have," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or apparatus.

Referring to fig. 1 and 2, fig. 1 is a schematic structural diagram of a visual prosthesis system according to an embodiment of the present invention, and fig. 2 is a schematic structural diagram of a visual prosthesis device according to an embodiment of the present invention; the visual prosthesis system comprises an in vitro control module 13, a second communication module (not shown) and a visual prosthesis device 12, the visual prosthesis device 12 comprises a first communication module 201, a master control module 203, a stimulation pulse generation module 206, an electrode array 207 and a signal amplification and analog-to-digital conversion module 205, wherein:

the extracorporeal control module 13 is connected to the second communication module, the second communication module is connected to the first communication module 201, and the extracorporeal control module 13 is configured to send an electrode control instruction to the first communication module 201, and receive and process a digital neuron signal; the first communication module 201 is connected to the main control module 203, the output end of the main control module 203 is connected to the input end of the stimulation pulse generation module 206, the output end of the stimulation pulse generation module 206 is connected to the input end of the electrode in the electrode array 207, the output end of the electrode is connected to the input end of the signal amplification and analog-to-digital conversion module 205, and the output end of the signal amplification and analog-to-digital conversion module 205 is connected to the input end of the main control module 203.

Specifically, the first communication module is configured to receive an electrode control instruction sent by the external control module 13 through the second communication module and transmit the digital neuron signal to the external control module 13 (through the second communication module), where the electrode control instruction includes a stimulation parameter of an electrode;

the main control module is used for controlling the stimulation pulse generation module to generate stimulation pulse signals according to the electrode control instruction, wherein the stimulation pulse signals can be stimulation pulse voltage signals or stimulation pulse current signals;

the stimulation pulse generation module is used for generating the stimulation pulse signal according to the electrode control instruction;

the electrodes in the electrode array are used for receiving the stimulation pulse signals, releasing electrical stimulation to the visual nerve tissues to generate phosphene, and receiving simulated neuron signals generated by the visual nerve tissues;

the signal amplification and analog-to-digital conversion module is used for amplifying the analog neuron signals and carrying out digital processing to obtain the digital neuron signals.

The visual prosthesis system in the embodiment of the invention comprises an in-vivo implanted visual prosthesis device and an in-vitro control module, wherein the in-vitro control module can send an electrode control command to the visual prosthesis device, the main control module in the visual prosthesis device controls the stimulation pulse generation module to generate stimulation pulse signals according to the electrode control instruction received by the first communication module, the electrode array not only can receive the stimulation pulse signals to release electrical stimulation to the visual neuron tissue, but also can receive the analog neuron signal generated by the visual nerve tissue, and then combines the signal amplification and analog-to-digital conversion module to amplify and convert the analog neuron signal to obtain the digital neuron signal, the main control module can control the first communication module to transmit the digital neuron signal to the outside, in the embodiment, the simulated neuron signals detected by the electrode array are transmitted to the in-vitro control module (through the second communication module); after the in-vitro control module processes and analyzes the digital neuron signals, the electrode control instruction can be adjusted, and finally, the stimulation parameter optimization of the closed-loop in-vivo visual prosthesis device is completed by the in-vitro control module under the weak manual supervision condition, so that the optimization efficiency and the accuracy of the optimal stimulation parameters of the electrodes are improved; according to digital neuron signals recorded by the visual prosthesis device, any electrode in the body electrode array is subjected to accurate stimulation parameter adjustment, the rationality and effectiveness of single electrode electrical stimulation and the accuracy of electrical stimulation are improved, and the risk of thermal injury of the electrical stimulation to nerve tissues is reduced. Because the device does not depend on the oral report of an experimental object and depends on the objective index of a digital neuron signal, the device can carry out large-scale animal in-vivo experiment on the visual prosthesis device (chip), thereby reducing the development cost of the visual prosthesis device and improving the safety and the effectiveness of the visual prosthesis device before clinical test.

Further, referring to fig. 3, fig. 3 is a schematic structural diagram of a visual prosthesis system according to an embodiment of the present invention, where the first communication module may be a wired communication module or a wireless communication module, and the wireless communication module is, for example, a WiFi module, a bluetooth module, a ZigBee module, or the like, as long as wireless or wired communication between the in-vitro control module 301 and the visual prosthesis device can be achieved, and the configuration is not particularly limited herein. Preferably, the first communication module and the second communication module are wireless communication modules, and wireless transmission is closer to clinical requirements.

Further, referring to fig. 3, the main control module includes a first processor 307, a second processor 303 and a data register module 305 including a plurality of data registers, an output terminal of the signal amplifying and analog-to-digital converting module 310 is connected to an input terminal of the first processor 307, an output terminal of the first processor 307 is connected to an input terminal of the data register module 305, the data register module 305 is connected to the second processor 303, the second processor 303 is connected to the first communication module, the second processor 303 is configured to take out the digital neuron signal stored in the data register 305 and send the digital neuron signal to the first communication module, and the first processor 307 is configured to store a new digital neuron signal in an empty data register. The first processor 307 and the second processor 303 realize transmission of digital neuron signals to the external control module by using a ping-pong mechanism, that is, the second processor 303 transmits digital neuron signal data in a part of data registers (k2 data registers) in the data register module 305 to the external control module, and the first processor 307 stores newly received digital neuron signal data in a data register (k1 data registers) which is just vacated, and realizes data access by using the ping-pong mechanism, so that data can be stored in the visual prosthesis device and transmitted to the external control module at the same time, thereby not only realizing continuous recording and transmission of data, avoiding incomplete recorded data, but also reducing the storage capacity of the data register module and reducing the volume of the visual prosthesis device.

Further, referring to fig. 3, the main control module further includes a third processor 304, a parameter register module 302, and a fourth processor 309, the first communication module is connected to the parameter register module 302, the parameter register module 302 is connected to the third processor 304, the third processor 304 is connected to the stimulation pulse generation module 306, the electrodes of the electrode array 308 are connected to the fourth processor 309, the parameter register module 302 is connected to the fourth processor 309, and the fourth processor 309 is connected to the signal amplification and analog-to-digital conversion module 310; wherein, the parameter register module 302 is configured to store an electrode control instruction, the electrode control instruction includes an electrode number of a stimulation electrode (i.e. an electrode of an electrode array that is designated to send electrical stimulation to the optic nerve tissue), stimulation parameters (amplitude, frequency, pulse width, duration) of the stimulation electrode, recording mode parameters (e.g. delay time and sampling frequency), a parameter register clear flag, an electrode number of a designated recording electrode group (i.e. an analog neuron signal of an electrode corresponding to the electrode number is processed by the designated signal amplification and analog-to-digital conversion module), the delay time is a time interval between the time when the stimulation electrode gives electrical stimulation and the time when the electrode with the designated electrode number starts recording the analog neuron signal, and the sampling frequency refers to the time interval of processing the simulated neuron signals of the electrodes with the appointed electrode numbers by the signal amplification and analog-to-digital conversion module. The third processor 304 controls the electrodes with the numbers corresponding to the electrode array 308 to emit electrical stimulation according to the electrode numbers of the stimulation electrodes and the corresponding stimulation parameters, the fourth processor 309 controls the signal amplification and analog-to-digital conversion module 310 to process the simulated neuron signals of the electrodes with the numbers corresponding to the sampling frequencies and the electrode numbers of the specified recording electrode groups according to the specified frequencies, and the parameter register clearing flag is used for clearing the data stored in the register in the parameter register module 302.

It is easy to think that the master control module can only adopt one processor to realize the functions of the first processor, the second processor, the third processor and the fourth processor as long as the processing capability of the adopted processor is strong enough.

Further, referring to fig. 3, the stimulation pulse generation module 306 includes a plurality of sub-stimulation pulse generation modules, the number of the sub-stimulation pulse generation modules is the same as the number of the electrodes of the electrode array 308, and the output end of one sub-stimulation pulse generation module is connected to the input end of one electrode of the electrode array. The sub-stimulation pulse generation module may be implemented using a micro-current stimulator chip, but is not limited to such an implementation.

Further, referring to fig. 2 and fig. 3, the apparatus further includes a sensor module 311, the sensor module 311 includes a temperature sensor 204 for detecting a temperature of the optic nerve tissue 312 and/or an impedance sensor 202 for detecting an impedance of the electrode array 308, an output terminal of the temperature sensor 204 and an output terminal of the impedance sensor 202 are respectively connected to an input terminal of the main control module 203; specifically, the output end of the temperature sensor 204 and the output end of the impedance sensor 202 are respectively connected to the input end of the first processor 307, and similarly, the first processor 307 and the second processor 303 implement storage and transmission of sensor parameters through a ping-pong mechanism, and send the sensor parameters to the extracorporeal control module 301.

Further, the visual prosthesis system may further include a display module for displaying information such as digital neuron signals, sensor parameters, optimal stimulation parameters of the electrodes, and the like, and an information input module for inputting information, an output end of the in vitro control module is connected with an input end of the display module, an output end of the information input module is connected with an input end of the in vitro control module, and the information input module may be a keyboard, a touch screen, and the like, and is used for inputting information such as electrical stimulation parameters.

In practical application, when the electrical stimulation parameters of the electrodes are optimized, the first optimization mode is a weak manual supervision optimization mode, the external control module autonomously completes the optimization of the electrical stimulation parameters according to a pre-stored program, namely the external control module is used for generating and sending electrode control instructions to the first communication module and receiving and processing digital neuron signals, the external control module processes the digital neuron signals according to the pre-stored program to judge whether the electrical stimulation parameters are the optimized electrical stimulation parameters, and the external control module continuously adjusts the electrical stimulation parameters and then judges to complete the optimization of the electrical stimulation parameters.

And secondly, the scheme of completely manually executing the optimization of the electrical stimulation parameters is that medical workers manually input stimulation parameters or images which are considered to be suitable by the medical workers to an in-vitro control module according to digital neuron signals and other information recorded by a specified electrode (group) under any stimulation parameters and displayed by a display module, and then the stimulation parameters or images are transmitted to the visual prosthesis device through the in-vitro control module, and the medical workers judge whether the requirements are met according to the digital neuron signals returned by the visual prosthesis device, so that the optimization of the stimulation parameters of the visual prosthesis device under the strong supervision condition is realized.

And thirdly, the optimization process of the in-vitro control module is supervised by medical care or laboratory staff between the first type and the second type, the medical care or laboratory staff can intervene or take over the execution process of the in-vitro control module at any time according to the digital neuron signals transmitted back by the visual prosthesis device, and after the states of the visual prosthesis device and the nerve tissues are confirmed, manual parameter optimization is selected or automatic optimization is continued or brand-new automatic optimization is started.

Therefore, the invention can be implemented by combining the in-vitro control module with manual intervention under the condition of manual supervision with any intensity, and has the capability of greatly reducing the burden of medical staff and patients.

Further, referring to fig. 1, the visual prosthesis system further includes an alarm module 14 for outputting an alarm signal, an output end of the external control module 13 is connected to an input end of the alarm module 14, the alarm module 14 may be an audible and visual alarm module, an optical alarm module or an acoustic alarm module, and respectively emits an audible and visual alarm signal, an optical alarm signal or an acoustic alarm signal, and when the external control module 13 determines that the received temperature exceeds a preset temperature (e.g., 37 ℃), it indicates that the temperature of the visual nerve tissue is abnormal; or when the received electrode impedance exceeds the preset impedance, the sub-stimulation pulse generation module is indicated to be abnormal or the electrode is abnormal, at the moment, the in-vitro control module 13 controls the alarm module 14 to work so as to prompt related personnel to be abnormal, avoid the permanent damage of nerve tissues caused by electric stimulation heating, and conveniently and timely eliminate faults.

It should be noted that the in vitro control module may be a single chip, a computer, or other devices with processing and control capabilities, and is not particularly limited. For the visual prosthesis device, external image acquisition equipment can be configured, and the main control module processes images or videos acquired by the image acquisition equipment to obtain stimulation parameters of the control electrode array so as to stimulate visual nerve tissues and enable the blind person to generate visual feeling.

On the basis of the above embodiments, please refer to fig. 1, fig. 2 and fig. 4, fig. 4 is a flow chart of a control method of a visual prosthesis system according to an embodiment of the present invention; the control method of the visual prosthesis system is applied to the visual prosthesis system and used for determining the optimal stimulation parameters of each electrode in the electrode array, and specifically comprises the following steps:

step S401, sending the electrode control instruction to the visual prosthesis device, wherein the electrode control instruction comprises an electrode number of a stimulation electrode, an initial stimulation parameter of the stimulation electrode and an electrode number of a recording electrode group, and the electrode control instruction is used for indicating the visual prosthesis device to control the electrode with the corresponding number of the electrode array to send out electric stimulation to visual nerve tissue according to the electrode number of the stimulation electrode and the initial stimulation parameter;

specifically, the stimulation electrode is a certain electrode which points to the optic nerve tissue to send out electrical stimulation, and the stimulation parameters are parameters for controlling the stimulation pulse generation module to generate stimulation pulse signals, including amplitude, frequency, pulse width, duration and the like of the stimulation pulse signals; the recording electrode group comprises a plurality of electrodes and is used for detecting simulated neuron signals generated by visual nerve tissues, namely the signal amplification and analog-to-digital conversion module processes the simulated neuron signals of the electrodes with corresponding numbers according to the electrode numbers of the recording electrode group; in addition, the electrode control instructions further comprise recording mode parameters (such as delay time and sampling frequency) for controlling the time interval for processing the simulated neuron signals, and the specific size of the recording mode parameters can be determined in advance according to animal ex-vivo experiments. In this step, the external control module may automatically generate the initial stimulation parameters according to a preset program, or an experimenter or other personnel may input the initial stimulation parameters to the external control module by using the input module.

Step S402, receiving a first digital neuron signal of an electrode corresponding to the electrode array returned by the visual prosthesis device according to the electrode number of the recording electrode group;

step S403, judging whether a preset condition is met according to a first digital neuron signal of the recording electrode group and a second digital neuron signal of each electrode in the recording electrode group in a resting state; if the preset conditions are met, taking the initial stimulation parameters as the optimal stimulation parameters of the stimulation electrode; and if the preset condition is not met, modifying the initial stimulation parameter until the preset condition is met.

Specifically, whether a preset condition is met or not is judged according to a first digital neuron signal of the recording electrode group and a second digital neuron signal of each electrode in the recording electrode group in a resting state, so that the optimal stimulation parameters of the stimulation electrodes are determined. In step S403, the initial stimulation parameters may be automatically modified by the in-vitro control module until the preset conditions are met, or the initial stimulation parameters may be modified by an experimenter or the like according to information such as digital neuron signals displayed on the display module and experience to reach the preset conditions, or the in-vitro control module may be manually supervised and judged by the experimenter or the like, and when the initial stimulation parameters automatically modified by the in-vitro control module are not suitable, the experimenter modifies the initial stimulation parameters.

By utilizing the method provided by the embodiment of the invention, the optimal stimulation parameters of the stimulation electrode can be obtained, the accuracy and the obtaining efficiency of the optimal stimulation parameters of the stimulation electrode are effectively improved, any electrode in the in-vivo electrode array is accurately adjusted in electrical stimulation parameters according to the nerve electrical signals recorded by the visual prosthesis device, the rationality and the effectiveness of single electrode electrical stimulation and the accuracy of electrical stimulation are improved, and the risk of thermal injury of electrical stimulation to nerve tissues is reduced. Because the device does not depend on the oral report of an experimental object, but depends on the objective index of a digital neuron signal, the device can carry out large-scale animal in-vivo experiment on the visual prosthesis device, reduces the development cost of the visual prosthesis device, and improves the safety and the effectiveness of the visual prosthesis device before clinical test. The scheme of the invention can be implemented by combining the in-vitro control module with manual intervention under the condition of manual supervision with any intensity, and has the capability of greatly reducing the burden of medical staff and patients.

Further, the electrode numbers of the recording electrode groups include an electrode number of a first recording electrode group and an electrode number of a second recording electrode group, the first recording electrode group does not include the stimulation electrode, and the number of the electrodes of the first recording electrode group is a first preset number; the second recording electrode group comprises the stimulation electrodes, and the number of the electrodes of the second recording electrode group is a second preset number. Referring to fig. 5, fig. 5 is a flowchart illustrating a control method of a visual prosthesis system according to an embodiment of the present invention; the step S403 includes:

step S501, acquiring the number of first electrodes in the first recording electrode group, of which the first digital neuron signals reach a first preset threshold value according to first digital neuron signals of the first recording electrode group and second digital neuron signals of all electrodes in the first recording electrode group in a resting state, wherein the first preset threshold value is the product of the second digital neuron signals of the electrodes corresponding to the first digital neuron signals and a first preset multiple;

specifically, the first preset number and the second preset number may be the same or different, and may be set as needed, for example, if the first preset number is 5, and the second preset number is 6, then 5 electrodes are selected from the rest of the electrodes in the electrode array except for the stimulation electrode as the first recording electrode group, assuming that the numbers are a1, a2, A3, a4, and a5, respectively, and 6 electrodes are selected from the electrode array including the stimulation electrode as the second recording electrode group, assuming that the numbers are B1, B2, B3, B4, B5, and B6, respectively (one of B1 to B6 is the stimulation electrode). When the number of the first electrodes is obtained, judging according to a first digital neuron signal S1 of A1 and a second digital neuron signal S2 of A1 in a resting state, judging whether the S1 reaches a first preset threshold value, wherein the first preset threshold value is the product of S2 and a first preset multiple, judging from A2 to A5 in a way that the judgment is the same as that of A1, and counting the number of all electrodes of which S1 is larger than or equal to the first preset threshold value to obtain the number of the first electrodes. The first digital neuron signal may be the number of action potentials or the frequency of the signal.

Step S502, acquiring the number of second electrodes in the second recording electrode group, of which the first digital neuron signals reach a second preset threshold value according to the first digital neuron signals of the second recording electrode group and the second digital neuron signals of all electrodes in the second recording electrode group in a resting state, wherein the second preset threshold value is the product of the second digital neuron signals of the electrodes corresponding to the first digital neuron signals and a second preset multiple;

specifically, similar to the first preset number, when the number of the second electrodes is obtained, judging according to a second digital neuron signal S4 of a first digital neuron signal S3 and a second digital neuron signal S1 of B1 in a resting state, judging whether S3 reaches a second preset threshold value, wherein the second preset threshold value is a product of S3 and a second preset multiple, for B2 to B6, the same judgment as that of B1 is performed, and the number of all electrodes of which S3 is greater than or equal to the first preset threshold value is counted, so that the number of the second electrodes is obtained. The first preset multiple and the second preset multiple may be the same or different, and may be modified as required.

Step S503, adding the number of the first electrodes and the number of the second electrodes to obtain an effective stimulation range, and judging according to the effective stimulation range and a preset value, wherein when the effective stimulation range is equal to the preset value, the initial stimulation parameter is used as the optimal stimulation parameter of the stimulation electrodes; if the effective stimulation range is greater than or less than the preset value, modifying the initial stimulation parameter until the effective stimulation range is equal to the preset value.

Specifically, if the effective stimulation range exceeds a preset value, the stimulation intensity needs to be reduced (modification of the reduction amplitude, frequency, pulse width, duration of the initial stimulation parameters) until the effective stimulation range equals the preset value. If the effective stimulation range does not reach the preset value, the stimulation intensity needs to be increased until the effective stimulation range is equal to the preset value.

Further, referring to fig. 6, fig. 6 is a schematic flowchart of a control method of a visual prosthesis system according to an embodiment of the present invention, where the method further includes:

step S601, sending a test instruction to the visual prosthesis device, wherein the test instruction comprises optimal stimulation parameters of stimulation electrodes and a standard test image, the standard test image designates the stimulation electrodes, and the test instruction is used for instructing the visual prosthesis device to project the standard test image on the electrode array and driving the stimulation electrodes according to the optimal stimulation parameters of the stimulation electrodes;

specifically, one or more stimulation electrodes are specified in the standard test image, and the stimulation electrodes are respectively controlled to send out electric stimulation to the optic nerve tissue according to the optimal stimulation parameters of the specified stimulation electrodes. The standard test images include static and dynamic images. Referring to fig. 7, fig. 7 is a flowchart illustrating a control method of a visual prosthesis system according to an embodiment of the present invention; in this embodiment, the standard test image is only black and white, and only the light and dark contrast lines are white, the standard test image is projected into the electrode array space, it is determined whether the pixel corresponding to each electrode is white or black, and the electrode corresponding to the white pixel in the image is calibrated as the stimulation electrode.

Step S602, receiving a third digital neuron signal of each electrode of the electrode array corresponding to the test instruction returned by the visual prosthesis device;

step S603, carrying out image reconstruction according to the third digital neuron signal to obtain a perception image;

specifically, the perceptual image may be obtained by performing image reconstruction by an interpolation reconstruction method.

Step S604, calculating the similarity of the standard test image and the perception image;

specifically, the image similarity may be calculated by using an image hashing algorithm, or other methods for calculating the image similarity may also be used, which is not particularly limited herein.

Step S605, judging whether the optimal stimulation parameter of the stimulation electrode meets the requirement or not according to the image similarity and the preset similarity, wherein the image similarity is greater than or equal to the preset similarity, the optimal stimulation parameter of the stimulation electrode meets the requirement, the image similarity is smaller than the preset similarity, and the optimal stimulation parameter of the stimulation electrode does not meet the requirement.

The method of fig. 6 can verify the optimal stimulation parameters of the stimulation electrode obtained in the previous step, and determine whether the optimal stimulation parameters meet the requirements. In order to improve the verification accuracy, the verification is carried out according to a plurality of standard test images, if the image similarity of a preset proportion (for example, 50%) of the standard test images and the corresponding perception images reaches the preset similarity, the optimal stimulation parameters of the stimulation electrodes are considered to meet the requirements, and the electrical stimulation parameters of the in-vivo visual prosthesis device are successfully optimized; otherwise, the optimal stimulation parameters of the stimulation electrode do not meet the requirements, the parameter optimization fails, and new optimal stimulation parameters of the stimulation electrode need to be acquired.

It should be noted that, in the embodiment of the present invention, after obtaining the optimal stimulation parameters of the stimulation electrodes, further fine tuning may be performed on the basis to find out an optimal stimulation parameter set of the stimulation electrodes, the finding out principle is to ensure that the effective stimulation range is always maintained at a set value, multiple optimal stimulation parameters corresponding to a certain stimulation electrode may be obtained to form an optimal stimulation parameter set thereof, and then a table may be established according to a correspondence between the numbers of the stimulation electrodes and the corresponding optimal stimulation parameter set for storage. The control method of the present invention is actually the working process of the extracorporeal control module, and the table established above will be retained in the extracorporeal control module.

Referring to fig. 8, fig. 8 is a flowchart illustrating a control method of a visual prosthesis system according to an embodiment of the present invention; the following describes the process of optimal stimulation parameter tuning:

after the effective stimulation range of a certain stimulation electrode is obtained by using the method of fig. 4, whether the requirement is met is determined according to the effective stimulation range and a preset value, if the requirement is met, the initial stimulation parameter corresponding to the effective stimulation range is stored as the optimal stimulation parameter of the stimulation electrode, otherwise, a new stimulation parameter is set as the initial stimulation parameter, and the effective stimulation range is continuously obtained for judgment. And when the effective stimulation range of the stimulation parameters after fine adjustment is not equal to the preset value, stopping fine adjustment of the parameters, and entering a flow for determining the optimal stimulation parameters of the next stimulation electrode. After the optimal stimulation parameters are finely adjusted, a plurality of optimal stimulation parameters corresponding to a certain stimulation electrode can be obtained, and an optimal stimulation parameter set is formed.

Further, after obtaining the optimal stimulation parameter set corresponding to each electrode of the electrode array, when performing the parameter verification of the embodiment corresponding to fig. 6, one optimal stimulation parameter is taken out from the optimal stimulation parameter set of each electrode for verification, and when the verification fails, a new optimal stimulation parameter is continuously taken out from the set for verification. The whole verification process can be constructed as a machine learning method, and the machine learning method runs under the condition of weak manual supervision, so that the acquisition efficiency of the optimal stimulation parameters of the stimulation electrode is effectively improved.

Based on the description of the above embodiment of the control method of the visual prosthesis system, the embodiment of the present invention further discloses a control device of the visual prosthesis system, referring to fig. 9, fig. 9 is a schematic structural diagram of the control device of the visual prosthesis system provided in the embodiment of the present invention, fig. 9 is a functional partition inside an external control module, where the control device of the visual prosthesis system includes a control instruction sending module 901, a signal receiving module 902, and a determining module 903; wherein:

a control instruction sending module 901, configured to send the electrode control instruction to the visual prosthesis device, where the electrode control instruction includes an electrode number of a stimulation electrode, an initial stimulation parameter of the stimulation electrode, and an electrode number of a recording electrode group, and the electrode control instruction is used to instruct the visual prosthesis device to control, according to the electrode number of the stimulation electrode and the initial stimulation parameter, the electrode with the corresponding number of the electrode array to send electrical stimulation to a visual nerve tissue;

a signal receiving module 902, configured to receive a first digital neuron signal of an electrode corresponding to the electrode array returned by the visual prosthesis apparatus according to the electrode number of the recording electrode group;

a determining module 903, configured to determine whether a preset condition is met according to a first digital neuron signal of the recording electrode set and a second digital neuron signal of each electrode in the recording electrode set in a resting state; if the preset conditions are met, taking the initial stimulation parameters as the optimal stimulation parameters of the stimulation electrode; and if the preset condition is not met, modifying the initial stimulation parameter until the preset condition is met.

For specific functional implementation manners of the control instruction sending module 901, the signal receiving module 902, and the determining module 903, reference may be made to steps S401 to S403 in the embodiment corresponding to fig. 4, which is not described herein again.

Further, the electrode numbers of the recording electrode groups include an electrode number of a first recording electrode group, an electrode number of a second recording electrode group, the first recording electrode group does not include the stimulating electrode, and the second recording electrode group includes the stimulating electrode; referring to fig. 10, fig. 10 is a schematic structural diagram of a control device of a visual prosthesis system according to an embodiment of the present invention; the determining module 903 includes a first electrode number obtaining submodule 101, a second electrode number obtaining submodule 102, and a processing submodule 103, where:

a first electrode number obtaining sub-module 101, configured to obtain, according to a first digital neuron signal of the first recording electrode group and a second digital neuron signal of each electrode in the first recording electrode group in a resting state, a first electrode number of the first recording electrode group for which a first digital neuron signal reaches a first preset threshold, where the first preset threshold is a product of a second digital neuron signal of an electrode corresponding to the first digital neuron signal and a first preset multiple;

a second electrode number obtaining sub-module 102, configured to obtain, according to a first digital neuron signal of the second recording electrode group and a second digital neuron signal of each electrode in the second recording electrode group in a resting state, a second electrode number of the second recording electrode group where the first digital neuron signal reaches a second preset threshold, where the second preset threshold is a product of a second digital neuron signal of an electrode corresponding to the first digital neuron signal and a second preset multiple;

the processing submodule 103 is configured to add the number of the first electrodes and the number of the second electrodes to obtain an effective stimulation range, and perform judgment according to the effective stimulation range and a preset value, where when the effective stimulation range is equal to the preset value, the initial stimulation parameter is used as an optimal stimulation parameter of the stimulation electrodes; if the effective stimulation range is greater than or less than the preset value, modifying the initial stimulation parameter until the effective stimulation range is equal to the preset value.

For specific functional implementation manners of the first electrode number obtaining submodule 101, the second electrode number obtaining submodule 102 and the processing submodule 103, reference may be made to steps S501 to S503 in the embodiment corresponding to fig. 5, which is not described herein again.

Further, referring to fig. 9 and fig. 11, fig. 11 is a schematic structural diagram of a control apparatus of a visual prosthesis system according to an embodiment of the present invention, where the apparatus further includes a test instruction sending module 111, a reconstruction module 112, a similarity calculation module 113, and a similarity determination module 114, where:

a test instruction sending module 111, configured to send a test instruction to the visual prosthesis apparatus, where the test instruction includes an optimal stimulation parameter of a stimulation electrode and a standard test image, the standard test image specifies the stimulation electrode, and the test instruction is configured to instruct the visual prosthesis apparatus to project the standard test image on the electrode array and drive the stimulation electrode according to the optimal stimulation parameter of the stimulation electrode;

a signal receiving module 902, further configured to receive a third digital neuron signal, which is returned by the visual prosthesis device and corresponds to each electrode of the electrode array of the test instruction;

a reconstruction module 112, configured to perform image reconstruction according to the third digital neuron signal to obtain a perceptual image;

a similarity calculation module 113, configured to calculate image similarity between the standard test image and the perceptual image;

a similarity determination module 114, configured to determine whether the optimal stimulation parameter of the stimulation electrode meets the requirement according to the image similarity and a preset similarity, where the image similarity is greater than or equal to the preset similarity, the optimal stimulation parameter of the stimulation electrode meets the requirement, the image similarity is less than the preset similarity, and the optimal stimulation parameter of the stimulation electrode does not meet the requirement.

For specific functional implementation manners of the test instruction sending module 111, the signal receiving module 902, the reconstructing module 112, the similarity calculating module 113, and the similarity determining module 114, reference may be made to steps S601 to S605 in the embodiment corresponding to fig. 6, which is not described herein again.

It is to be noted that the respective units or modules in the control apparatus of the visual prosthesis system shown in fig. 9, 10 and 11 may be respectively or entirely combined into one or several additional units or modules to constitute, or some unit(s) or module(s) thereof may be further split into a plurality of functionally smaller units or modules to constitute, which may achieve the same operation without affecting the achievement of the technical effects of the embodiments of the present invention. The above units or modules are divided based on logic functions, and in practical applications, the functions of one unit (or module) may also be implemented by a plurality of units (or modules), or the functions of a plurality of units (or modules) may be implemented by one unit (or module).

Based on the description of the method embodiment and the device embodiment, the embodiment of the invention also provides the terminal equipment.

Fig. 12 is a schematic structural diagram of a terminal device according to an embodiment of the present invention. As shown in fig. 12, the control apparatus of the visual prosthesis system of fig. 9 to 11 described above may be applied to the terminal device 120, and the terminal device 120 may include: processor 121, network interface 124 and memory 125, and the terminal device 120 may further include: a user interface 123, and at least one communication bus 122. Wherein a communication bus 122 is used to enable the connection communication between these components. The user interface 123 may include a Display screen (Display) and a Keyboard (Keyboard), and the optional user interface 123 may further include a standard wired interface and a standard wireless interface. The network interface 124 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface). The memory 125 may be a high-speed RAM memory or a non-volatile memory (e.g., at least one disk memory). The memory 125 may optionally be at least one storage device located remotely from the processor 121. As shown in fig. 12, the memory 125, which is a kind of computer storage medium, may include therein an operating system, a network communication module, a user interface module, and a device control application program.

In the terminal device 120 shown in fig. 12, the network interface 124 may provide a network communication function; and user interface 123 is primarily an interface for providing input to a user; and processor 121 may be used to invoke a device control application stored in memory 125 to implement:

sending the electrode control instruction to the visual prosthesis device, wherein the electrode control instruction comprises an electrode number of a stimulation electrode, an initial stimulation parameter of the stimulation electrode and an electrode number of a recording electrode group, and the electrode control instruction is used for indicating the visual prosthesis device to control the electrode with the corresponding number of the electrode array to send out electrical stimulation to visual nerve tissue according to the electrode number and the initial stimulation parameter of the stimulation electrode;

receiving a first digital neuron signal of an electrode corresponding to the electrode array returned by the visual prosthesis device according to the electrode number of the recording electrode group;

judging whether a preset condition is met or not according to a first digital neuron signal of the recording electrode group and a second digital neuron signal of each electrode in the recording electrode group in a resting state; if the preset conditions are met, taking the initial stimulation parameters as the optimal stimulation parameters of the stimulation electrode; and if the preset condition is not met, modifying the initial stimulation parameter until the preset condition is met.

Further, the electrode numbers of the recording electrode groups include an electrode number of a first recording electrode group, an electrode number of a second recording electrode group, the first recording electrode group does not include the stimulating electrode, and the second recording electrode group includes the stimulating electrode; in one embodiment, the processor 121 performs the determining whether the preset condition is satisfied according to the first digital neuron signal of the recording electrode group and the second digital neuron signal of each electrode in the recording electrode group in a resting state; if the preset conditions are met, taking the initial stimulation parameters as the optimal stimulation parameters of the stimulation electrode; if the preset condition is not met, modifying the initial stimulation parameter until the preset condition is met, and specifically executing the following steps:

acquiring the number of first electrodes in the first recording electrode group, of which first digital neuron signals reach a first preset threshold value according to first digital neuron signals of the first recording electrode group and second digital neuron signals of all electrodes in the first recording electrode group in a resting state, wherein the first preset threshold value is the product of the second digital neuron signals of the electrodes corresponding to the first digital neuron signals and a first preset multiple;

acquiring the number of second electrodes in the second recording electrode group, of which the first digital neuron signals reach a second preset threshold value according to the first digital neuron signals of the second recording electrode group and second digital neuron signals of all electrodes in the second recording electrode group in a resting state, wherein the second preset threshold value is the product of the second digital neuron signals of the electrodes corresponding to the first digital neuron signals and a second preset multiple;

adding the number of the first electrodes and the number of the second electrodes to obtain an effective stimulation range, and judging according to the effective stimulation range and a preset value, wherein when the effective stimulation range is equal to the preset value, the initial stimulation parameters are used as the optimal stimulation parameters of the stimulation electrodes; if the effective stimulation range is greater than or less than the preset value, modifying the initial stimulation parameter until the effective stimulation range is equal to the preset value.

In one embodiment, processor 121 is further configured to perform the steps of:

sending a test instruction to the visual prosthetic device, the test instruction including optimal stimulation parameters of stimulation electrodes and a standard test image, the standard test image specifying stimulation electrodes, the test instruction being for instructing the visual prosthetic device to project the standard test image on the electrode array and to drive the stimulation electrodes according to the optimal stimulation parameters of the stimulation electrodes;

receiving a third digital neuron signal of each electrode of the electrode array corresponding to the test instruction returned by the visual prosthesis device;

carrying out image reconstruction according to the third digital neuron signal to obtain a perception image;

calculating the similarity of the standard test image and the perception image;

and judging whether the optimal stimulation parameters of the stimulation electrodes meet requirements or not according to the image similarity and a preset similarity, wherein the image similarity is greater than or equal to the preset similarity, the optimal stimulation parameters of the stimulation electrodes meet the requirements, the image similarity is smaller than the preset similarity, and the optimal stimulation parameters of the stimulation electrodes do not meet the requirements.

It should be understood that the terminal device 120 described in the embodiment of the present invention may perform the description of the control method of the visual prosthesis system in the embodiment corresponding to fig. 4 to fig. 8, and may also perform the description of the control device of the visual prosthesis system in the embodiment corresponding to fig. 9 to fig. 11, which is not repeated herein. In addition, the beneficial effects of the same method are not described in detail.

Further, here, it is to be noted that: an embodiment of the present invention further provides a computer storage medium, where a computer program executed by the control device of the visual prosthesis system mentioned above is stored in the computer storage medium, and the computer program includes program instructions, and when the processor executes the program instructions, the description of the control method of the visual prosthesis system in the embodiments corresponding to fig. 4 to 8 can be executed, so that details are not repeated here. In addition, the beneficial effects of the same method are not described in detail. For technical details not disclosed in the embodiments of the computer storage medium to which the present invention relates, reference is made to the description of the method embodiments of the present invention.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.

The above disclosure is only for the purpose of illustrating the preferred embodiments of the present invention, and it is therefore to be understood that the invention is not limited by the scope of the appended claims.