阅读说明：本技术 Rpu阵列的更新管理 (The more new management of RPU array ) 是由 T.戈克曼 O.M.奥宁 于 2018-03-13 设计创作，主要内容包括：提供了一种计算机实现的方法和计算机处理系统,用于神经网络的更新管理。该方法包括使用电阻处理单元在神经网络上执行各向同性更新过程。各向同性更新过程使用乘法运算中的被乘数和乘数。执行步骤包括缩放被乘数和乘数以具有相同的数量级。(A method of computer implementation and computer processing system are provided, the more new management for neural network.This method includes that isotropism renewal process is executed in neural network using resistance processing unit.Isotropism renewal process uses the multiplicand and multiplier in multiplying.Executing step includes scaling multiplicand and multiplier with the order of magnitude having the same.)

1. a kind of computer implemented method of the more new management for neural network, this method comprises:

Isotropism renewal process is executed in neural network using resistance processing unit RPU, isotropism renewal process uses The multiplicand and multiplier of multiplying,

Wherein the execution step includes scaling multiplicand and multiplier with the order of magnitude having the same.

2. computer implemented method as described in claim 1, wherein scaling multiplicand and multiplier with before scaling and it After keep identical product.

3. computer implemented method as described in claim 1, wherein scaling step is being applied to isotropism renewal process Input input adjustment process in execute.

4. computer implemented method according to claim 3, wherein input adjustment process is removed by multiplicand and multiplier The order of magnitude between difference caused by wrong spatial correlation.

5. computer implemented method as described in claim 1, wherein executing isotropism using only the single update cycle Renewal process.

6. computer implemented method as described in claim 1, wherein RPU array is configured as executing simulation vector matrix Multiplication.

7. computer implemented method as described in claim 1, further includes that will be corresponded to by one or more random transition devices Digital group of the neuron of neural network translates into random bit stream.

8. computer implemented method as claimed in claim 7, wherein to the ratio with the one or more preassigneds of satisfaction Each random bit stream of bit length executes isotropism renewal process.

9. computer implemented method as claimed in claim 8, wherein one or more preassigneds include having minimum ratio Spy's stream length.

10. computer implemented method as claimed in claim 7, wherein zoom factor is applied to one or more random The amplification factor of converter.

11. computer implemented method as claimed in claim 7, wherein one or more random transition devices include first with Machine converter and the second random transition device, and wherein, scaling step include by the amplification factor of the first random transition device multiplied by The zoom factor, and by the amplification factor of the second random transition device divided by the zoom factor.

12. computer implemented method as described in claim 1, further includes that will be corresponded to by one or more certainty converters Digital group in the neuron of neural network is converted into certainty bit stream.

13. computer implemented method as claimed in claim 12, wherein meet one or more preassigneds to having Corresponding bits stream in the certainty bit stream of bit length executes isotropism renewal process.

14. computer implemented method as claimed in claim 13, wherein one or more preassigneds include having minimum Bitstream length.

15. computer implemented method as claimed in claim 12, wherein zoom factor is applied to one or more determinations The amplification factor of property converter.

16. computer implemented method as claimed in claim 12, wherein one or more certainty converters include first Certainty converter and the second certainty converter, and wherein, scaling step includes by the amplification of the first certainty converter The factor is multiplied by the zoom factor and by the amplification factor of the second certainty converter divided by the zoom factor.

17. a kind of computer program product of the more new management for neural network, which includes computer Readable storage medium storing program for executing, has the program instruction therewith realized, which can be executed by computer so that computer is held Row includes the following method:

Isotropism renewal process is executed in neural network using resistance processing unit RPU, isotropism renewal process uses The multiplicand and multiplier of multiplying,

Wherein the execution step includes scaling multiplicand and multiplier with the order of magnitude having the same.

18. computer program product as claimed in claim 17, wherein scaling multiplicand and multiplier with before scaling and it After keep identical product.

19. computer program product as claimed in claim 17, wherein scaling step is being applied to isotropism renewal process Input input adjustment process in execute.

20. computer program product according to claim 19, wherein input adjustment process is removed by multiplicand and multiplier The order of magnitude between difference caused by wrong spatial correlation.

21. computer program product as claimed in claim 17, wherein executing isotropism using only the single update cycle Renewal process.

22. computer program product as claimed in claim 17, wherein RPU array is configured as executing simulation vector matrix Multiplication.

23. a kind of computer processing system of the more new management for neural network, the computer processing system include:

Resistance processing unit (RPU) is configured as executing isotropism update processing in neural network, and isotropism is updated Journey uses the multiplicand and multiplier of multiplying,

Wherein, RPU is configured as updating by scaling multiplicand and multiplier with the order of magnitude having the same to execute isotropism Process.

24. circuit as claimed in claim 23, wherein the computer processing system further includes nonlinear filter.

25. circuit as claimed in claim 23, wherein the RPU includes in application-specific integrated circuit.

Technical field

The present invention relates generally to resistance processing unit more particularly to the more new managements of resistance processing unit (RPU) array.

Background technique

Summary of the invention

According to an aspect of the invention, there is provided a kind of computer implemented side of the more new management for neural network Method.This method includes that isotropism renewal process is executed in neural network using resistance processing unit.Isotropism is updated Journey uses the multiplicand and multiplier in multiplying.Executing step includes scaling multiplicand and multiplier with quantity having the same Grade.

According to another aspect of the present invention, the computer program for providing a kind of more new management for neural network produces Product.The computer program product includes non-transitory computer-readable storage media, has the program instruction therewith embodied. Program instruction can be executed by computer so that computer implemented method.This method includes using resistance processing unit in neural network Upper execution isotropism renewal process.Isotropism renewal process uses the multiplicand and multiplier in multiplying.Execute step Including scaling multiplicand and multiplier with the order of magnitude having the same.

According to another aspect of the invention, a kind of computer disposal system of more new management for neural network is provided System.The computer processing system includes resistance processing unit (RPU), is configured as executing isotropism update in neural network Processing.Isotropism renewal process uses the multiplicand and multiplier in multiplying.RPU is configured as by multiplicand and will multiply Number scaling executes isotropism renewal process with the order of magnitude having the same.

From the detailed description below to the illustrative embodiments being read in conjunction with the accompanying drawings, these and other feature and advantage It will become obvious.

Detailed description of the invention

The details of preferred embodiment will be provided with reference to the following drawings by being described below, in which:

Fig. 1 shows the example processing system according to an embodiment of the present invention that can apply the principle of the invention；

Fig. 2 shows the exemplary simulated vector-matrix multiplications on RPU array according to an embodiment of the present invention；

Fig. 3 shows the simulation vector-matrix multiplication of the another exemplary on RPU array according to an embodiment of the present invention；

Fig. 4 shows and according to an embodiment of the present invention can operate using exemplary update of the invention；

Fig. 5, which is shown, according to an embodiment of the present invention randomly updates Regular Circuit using RPU array of the invention；

Fig. 6 shows the update cycle for randomly updating Regular Circuit for corresponding to Fig. 5 according to one embodiment of present invention； And

Fig. 7 shows the illustrative methods of the more new management according to an embodiment of the present invention for RPU array.

Specific embodiment

The present invention relates to the more new managements of resistance processing unit (RPU) array.

In one embodiment, isotropism update scheme is proposed for RPU, so as to while improving overall performance Using the least period for making full use of the update cycle of RPU needed for training.

Present invention could apply to any one of random bit streams and certainty bit stream.

It in one embodiment, include filling random bit stream and using the random of coincidence for the RPU update scheme proposed Bit stream executes update.

In one embodiment, the present invention adjusts input before operation randomly updates the period, to eliminate due to necessary Multiplied by wrong spatial correlation caused by two numbers of multiple orders of magnitude different from each other.That is, in one embodiment In, randomly update the multiplicand order of magnitude having the same in scheme.Using the method, RPU test accuracy can dramatically raising The training of CNN and DNN is completed RPU array using single iteration and is updated, to accelerate the update cycle.

Keep the update cycle as fast as possible using shortest bit stream.Such case corresponds to BL=1, and wherein the update cycle is logical Single pulse operation is crossed (so that minimum and maximum possible update is identical, while relative to x_iδ_jValue still maintains probability).

In order to use this pulse window as efficiently as possible, we have proposed one kind in a manner of more isotropic The method being updated.

In one embodiment, the present invention can solve by hardware not perfect the problem of causing.In one embodiment, originally Invention reduces the influence (in RPU) with the resistive element of asymmetric behavior by reducing the correlation between node.This Invention can be related to scale value to increase randomness and minimize imperfect influence of the equipment on test result.

Fig. 1 shows the example processing system 100 according to an embodiment of the present invention that can apply the principle of the invention.

Processing system 100 includes bus 102, (uniformly and single for interconnecting one or more nonlinear filters (NLF) Solely indicated by drawing reference numeral 110), one or more resistance processing units (RPU) are (uniformly and individually by appended drawing reference 120 indicate), one or more memories (uniformly and individually being indicated by appended drawing reference 130), and it is one or more defeated Enter/export (I/O) circuit (uniformly and individually being indicated by appended drawing reference 140).

In the case where the integrated circuit (IC) of processing system 100 is realized, chip-on communication can be provided by bus 102, and Communicating off-chip can be provided by I/O circuit 140.

Certainly, as those skilled in the art are readily apparent that, processing system 100 can also include that other elements (are not shown Out), and certain elements are omitted.For example, various other input equipments and/or output equipment may include in processing system 100 In, this depends on its specific implementation, as those of ordinary skill in the art are readily comprehensible.It is, for example, possible to use various types of Wireless and/or wired input and/or output equipment.In addition, can also be used as those of ordinary skill in the art are readily comprehensible The additional RPU of various configurations, processor, controller, memory etc..In view of the introduction of present invention provided herein, this field is general Logical technical staff is readily apparent that these and other modifications of processing system 100.

Fig. 2 shows the exemplary simulated vector-matrix multiplications 200 on RPU array according to an embodiment of the present invention.

Simulation vector-matrix multiplication 200 is related to set of number input value (δ) 210, wherein each digital input value (δ) 210 It is indicated by corresponding analog signal pulse width 220.Analog signal pulse width 220 is provided to operational amplifier 231 Operational amplifier (op-amp) integrating circuit 230, capacitor (C_int) 232 inverting input terminals for being connected to operational amplifier 231 The output end of (and leap) operational amplifier 231.The non-inverting input of operational amplifier 231 is grounded.Operational amplifier 231 Output is also connected to the input of analog-digital converter (ADC) 240.240 output signal y1 of ADC indicates to simulate vector on RPU array (digitlization) result of matrix multiplication 200.

Fig. 3 shows the simulation vector-matrix multiplication 300 of the another exemplary on RPU array according to an embodiment of the present invention. Multiplication 300 is substantially multiplication shown in Fig. 2, uses different formats (expression).

Simulation vector-matrix multiplication 300 be related to being applied to one group of the anti-phase input of operational amplifier integrating circuit 330 it is defeated Enter value 320.Operational amplifier integrating circuit 330 includes having capacitor (C_int) 332 operational amplifier 331.Input value 320 Corresponding to input voltage vin and corresponding mutual conductanceIt arrivesWithIt arrivesThe non-inverting input of operational amplifier 331 terminates Ground.Capacitor 332 is connected to the output end of inverting input terminal (and leap) operational amplifier 331 of operational amplifier 331.Operation The output end of amplifier is also connected to the input terminal of analog-digital converter (ADC) 340.331 output signal V of operational amplifier_out, table Show the result of the simulation vector-matrix multiplication 300 on RPU array.ADC340 is by the simulation output V from operational amplifier 331_out Be converted to digital signal.

Fig. 4 show it is according to an embodiment of the present invention can be using exemplary update operation 400 of the invention.

Update operation 400 is related to inputting original vector into δ 411 and original vector input x 412 is supplied to resistance processing list Member (RPU) 420.It is following that the output of RPU 420 is provided:

w_ij=w_ij+η(x_i×δ_j)

Wherein w_ijIndicate the weight of the connection between ith row and jth column, η indicates learning rate (scalar value), x_iIndicate defeated Enter the activity at neuron, δ_jIndicate the error calculated by output neuron.

In one embodiment, using back-propagation method training RPU, which includes three periods, i.e., Forward direction period, backward period and weight update cycle (abbreviated here as " weight update ").The forward and backward period relates generally to Vector-matrix multiplication is calculated in forward and backward.The invention relates essentially to the weight update cycles.

Fig. 5, which is shown, according to an embodiment of the present invention randomly updates Regular Circuit using RPU array of the invention 500.Fig. 6 shows update cycle 600 of the embodiment according to the present invention corresponding to the update Regular Circuit 500 of Fig. 5.

Randomly updating Regular Circuit 500 includes converter (TR) 510, TR 520, AND (&) and door 530 and adder 540. TR can be random transition device (STR) or certainty converter (DTR).For explanation, below by 510 He of converter (TR) 520 are known as random transition device (STR), and description is related to stochastic flow.However, as those of ordinary skill in the art are providing this Be easy determining in the case where the teachings of the present invention that text provides, while keeping spirit of the invention, STR can easily by DTR is substituted and is related to certainty stream.

STR 510 receives input x_iAnd it exportsInput x_iIndicate the activity at input neuron.It indicates to exert with uncle The stochastic variable that sharp process is characterized, subscript n indicate the bit position in Test Sequences.

STR 520 receives input δ_jAnd it exportsInput δ_jIndicate the error that output neuron calculates.It indicates with primary The stochastic variable that sharp process is characterized is exerted, subscript n indicates the bit position in Test Sequences.

Each of STR510 and 520 is related to parameter C, is STR amplification factor.Particularly, STR 510 be related to x to The amplification factor C of amount_x, STR 520 is related to the amplification factor C of delta vector_δ.Therefore,WithRespectively by C_xx_iAnd C_δδ_jIt provides general Rate.In one embodiment, amplification factor C_xAnd C_δIt is controlled by nonlinear filter (for example, NLF 110, etc. of Fig. 1).

Adder 540 receives input Δ w_minAnd w_ij, and export (update) w_ij.Parameter, Δ w_minCorresponding in RPU equipment The variation of increment conductance because single coincidence events are converted to the variation of increment weight.The electric conductivity value being stored in RPU equipment is similar Ground is converted to weighted value.

By randomly update that Regular Circuit 500 realizes to randomly update rule as follows:

Wherein w_ijIndicate the weight connected between ith row and jth column, Δ w_minIndicate due to single coincidence events (and by Be considered to control possible device parameter as voltage) caused by the variation of increment conductance (that is, the variation of weighted value, BL indicates random The length (at the output of the STR used during the update cycle) of bit stream,WithIt indicates characterized by Bernoulli process Stochastic variable, and subscript n indicate Test Sequences in bit position.

In randomly updating Regular Circuit 500, by neuron (x_iAnd δ_j) coding number be converted to by STR510 and 520 Random bit stream.

Error will be reduced by increasing random bit stream length BL, but will increase renewal time in turn.In one embodiment, It is as follows to allow to reach the acceptable BL value range similar to the error in classification of baseline model: being instructed using different BL values Practice, is sequentially arranged simultaneouslyThe learning rate for baseline model is matched with C=1.It has been determined that as low as 10 BL value is enough that stochastic model is made to become to cannot be distinguished with baseline model.

In addition, the variation of the weighted value of single update cycle is by BL Δ w for randomly updating rule_minLimitation, and If from the (Cx of STR 510 and 520_i) and (C δ_j) generate the probability of pulse then may this thing happens close to one or bigger. For example, it is also contemplated that influence of this editing in weight update, and will not make the reduced performance of BL to 10.

In addition, (overall situation) learning rate η is the important hyper parameter to be controlled.

Learning rate control is accomplished by the following way:

η←BLΔw_minC²

In form most typically, the mean change of the weighted value of stochastic model can be written as follow:

Therefore, the learning rate of stochastic model is by three state modulators, i.e. BL and Δ w_minAnd C.These three parameters are adjustable To match learning rate used in baseline model.

When training continues, since network becomes more preferably, δ value becomes smaller (therefore network needs less update).Separately On the one hand, due to having used tanh activation primitive, becoming 1 or -1, (if it is sigmoid, then it will be 0 to x value (Δ w=η x δ) Or 1).

When multiplicand is in the different orders of magnitude (for example, 1 and 10^-6), randomly updating rule (being overlapped detection) becomes more Grain, it means that in population of individuals with "high" pulse probability be it is very different (that is, one more or less determine, and The other is be very unlikely to).

When δ column triggering, all nodes in the column can all update because x row be certain to triggering (value for 1 or- 1).This will lead to false spatial coherence and damages the process.

According to one embodiment of present invention, can be with scale value, while keeping their product identical, so that they have Comparativity (identical).For example,

η←BLΔw_minC²

Wherein, in η equation in front, " (C/ γ) indicates C_x, " (C γ) " indicates C_δ。

In one embodiment, bit length BL is reduced to one, to allow the update cycle as short as possible.

Isotropism update scheme according to the present invention is drawn by eliminating by the difference in height of the order of magnitude between x and δ value The wrong spatial correlation risen, improves the measuring accuracy in convolutional neural networks (CNN) and deep neural network (DNN).

The entropy of update mechanism is increased using the stream of the likelihood probability comprising "high" pulse, to provide better knot Fruit.

In one embodiment, it is related to all operations of the invention all to complete in the digital domain, it is complicated without increasing circuit Property.

With reference to Fig. 6, following equation is applicable in:

P_i=C_xx_i

P_j=C_δδ_j.

In Fig. 6, updates pulse (for example, randomly updating pulse) 611 and be applied to RPU array 612.In particular row j The probability of pulse is generated by P_jIt provides, P_jIt is controlled by the STR of the row.Similarly, the probability of pulse is generated in particular column i by P_i It provides, P_iIt is controlled by the STR of the column.Some of them may be overlapped (weight from the pulse that the i-th column and jth row generate in RPU equipment Close), cause increment conductance to change.This increment conductance variation is equivalent to increment weight variation (Δ w_min)。

In the case where certainty updates pulse, if those of ordinary skill in the art are readily comprehensible, random transition device (STR) it can be replaced by DTR certainty converter (STR), while the introduction for keeping spirit of the invention to provide.

Fig. 7 shows the illustrative methods 700 of the more new management according to an embodiment of the present invention for RPU array.

In step 710, number is received (for example, x from neuron_iAnd δ_j)。

In step 720, it is determined whether use more new management.If it is, entering step 730.Otherwise, 790 are entered step.

In step 730, zoom factor γ is determined, so that:

In step 740, number (x is scaled using zoom factor γ_iAnd δ_j), so that scaling number C_xx_iAnd C_δδ_jBetween number Magnitude is equal.

In step 750, scaled number is converted into (for example, random or certainty bit by bit stream by one group of TR Stream), each bit stream has corresponding bit length (BL).

In step 760, updated using neural network.

In step 770, by the Application of Neural Network updated by the neural network update cycle in corresponding to the defeated of special object Enter signal.

In step 780, output neural network based executes movement related with special object or changes special object State (arrives another state).

In step 790, C is used_xAnd C_δScale number (x_iAnd δ_j), so that:

It should be appreciated that present invention could apply to be related to the myriad applications of neural network, including but not limited to speech recognition, Speaker Identification, gesture identification, audio identification, natural language processing, computer vision, bioinformatics etc..Therefore, step 770 and 780 can be related to any aforementioned applications.Accordingly, with respect to speech recognition, for example, acoustic utterances can be converted into voice Expression.In addition, further to speech recognition, it can be in the bio-identification identifier (example for identifying the password, submission said Such as, fingerprint), speaker, any one of object etc. when unlock hardware or other kinds of lock.

In any possible technical detail combination level, the present invention can be system, method and/or computer program and produce Product.Computer program product may include computer readable storage medium, containing of the invention for realizing processor The computer-readable program instructions of various aspects.

Computer readable storage medium, which can be, can keep and store the tangible of the instruction used by instruction execution equipment Equipment.Computer readable storage medium for example can be -- but being not limited to -- storage device electric, magnetic storage apparatus, optical storage are set Standby, electric magnetic storage apparatus, semiconductor memory apparatus or above-mentioned any appropriate combination.Computer readable storage medium is more Specific example (non exhaustive list) includes: portable computer diskette, hard disk, random access memory (RAM), read-only storage Device (ROM), erasable programmable read only memory (EPROM or flash memory), static random access memory (SRAM), portable pressure Contracting disk read-only memory (CD-ROM), memory stick, floppy disk, mechanical coding equipment, is for example deposited digital versatile disc (DVD) thereon Contain punch card or groove internal projection structure and the above-mentioned any appropriate combination of instruction.Computer used herein above Readable storage medium storing program for executing is not interpreted that instantaneous signal itself, the electromagnetic wave of such as radio wave or other Free propagations pass through The electromagnetic wave (for example, the light pulse for passing through fiber optic cables) or pass through electric wire transmission that waveguide or other transmission mediums are propagated Electric signal.

Computer-readable program instructions as described herein can be downloaded to from computer readable storage medium it is each calculate/ Processing equipment, or outer computer or outer is downloaded to by network, such as internet, local area network, wide area network and/or wireless network Portion stores equipment.Network may include copper transmission cable, optical fiber transmission, wireless transmission, router, firewall, interchanger, gateway Computer and/or Edge Server.Adapter or network interface in each calculating/processing equipment are received from network to be counted Calculation machine readable program instructions, and the computer-readable program instructions are forwarded, for the meter being stored in each calculating/processing equipment In calculation machine readable storage medium storing program for executing.

Computer program instructions for executing operation of the present invention can be assembly instruction, instruction set architecture (ISA) instructs, Machine instruction, machine-dependent instructions, microcode, firmware instructions, condition setup data or with one or more programming languages The source code or object code that any combination is write, the programming language include object-oriented programming language-such as Java, Smalltalk, C++ etc., and conventional procedural programming languages-such as " C " language or similar programming language.Computer can Reader instruction can be executed fully on the user computer, partly be executed on the user computer, as an independence Software package execute, part on the user computer part execute on the remote computer or completely in remote computer or It is executed on server.In situations involving remote computers, remote computer can include office by the network-of any kind Domain net (LAN) or wide area network (WAN)-are connected to subscriber computer, or, it may be connected to outer computer (such as using because Spy nets service provider to connect by internet).In some embodiments, pass through the shape using computer-readable program instructions State information comes personalized customization electronic circuit, such as programmable logic circuit, field programmable gate array (FPGA) or programmable Logic array (PLA), which can execute computer-readable program instructions, to realize various aspects of the invention.

Referring herein to according to the method for the embodiment of the present invention, the flow chart of device (system) and computer program product and/ Or block diagram describes various aspects of the invention.It should be appreciated that flowchart and or block diagram each box and flow chart and/ Or in block diagram each box combination, can be realized by computer-readable program instructions.

These computer-readable program instructions can be supplied to general purpose computer, special purpose computer or other programmable datas The processor of processing unit, so that a kind of machine is produced, so that these instructions are passing through computer or other programmable datas When the processor of processing unit executes, function specified in one or more boxes in implementation flow chart and/or block diagram is produced The device of energy/movement.These computer-readable program instructions can also be stored in a computer-readable storage medium, these refer to It enables so that computer, programmable data processing unit and/or other equipment work in a specific way, thus, it is stored with instruction Computer-readable medium then includes a manufacture comprising in one or more boxes in implementation flow chart and/or block diagram The instruction of the various aspects of defined function action.

Computer-readable program instructions can also be loaded into computer, other programmable data processing units or other In equipment, so that series of operation steps are executed in computer, other programmable data processing units or other equipment, to produce Raw computer implemented process, so that executed in computer, other programmable data processing units or other equipment Instruct function action specified in one or more boxes in implementation flow chart and/or block diagram.

The flow chart and block diagram in the drawings show the system of multiple embodiments according to the present invention, method and computer journeys The architecture, function and operation in the cards of sequence product.In this regard, each box in flowchart or block diagram can generation One module of table, program segment or a part of instruction, the module, program segment or a part of instruction include one or more use The executable instruction of the logic function as defined in realizing.In some implementations as replacements, function marked in the box It can occur in a different order than that indicated in the drawings.For example, two continuous boxes can actually be held substantially in parallel Row, they can also be executed in the opposite order sometimes, and this depends on the function involved.It is also noted that block diagram and/or The combination of each box in flow chart and the box in block diagram and or flow chart, can the function as defined in executing or dynamic The dedicated hardware based system made is realized, or can be realized using a combination of dedicated hardware and computer instructions.

The reference of " one embodiment " or " embodiment " of the invention and other modifications are meaned to combine in specification Special characteristic, structure, the characteristic etc. of embodiment description are included at least one embodiment of the present invention.Therefore, entire The phrase " in one embodiment " or " in embodiment " and any other modification in each place are appeared in specification Appearance is not necessarily all referring to identical embodiment.

It should be appreciated that using "/", "and/or" and "at least one" any one, for example, " A/B ", " A and/or B ", " at least one of A and B " are intended to include: only being selected first option (A) listed, or only selected second and list Option (B), or two options (A and B) of selection.As another example, in " A, B and/or C " and " at least one in A, B and C It is a " in the case where, this wording is intended to include: only being selected first option (A) listed, or is only selected second choosing listed Item (B), or the option (C) that only selection third is listed, or first and second option (A and B) listed only is selected, or choosing The option (A and C) that only limit first and third are listed is selected, or only selects second and a option (B and C) listed of third, or Select all three options (A and B and C).For listed numerous items, such as the ordinary skill people of this field and related fields Member is it will be apparent that this can be extended.

The preferred embodiment (it is intended to be illustrative and be not restrictive) that system and method have been described, should infuse Meaning, those skilled in the art can modify and change according to the above instruction.It should therefore be understood that can be disclosed It is changed in specific embodiment, these change in the scope of the present invention that appended claims are summarized.So retouch Each aspect of the present invention has been stated, there is details and particularity required by Patent Law, be described in the accompanying claims patent Certificate is required and it is expected the content of protection.