阅读说明：本技术 用以收集在自主交通工具中产生的传感器数据的系统、方法及设备 (Systems, methods, and apparatus to collect sensor data generated in an autonomous vehicle ) 是由 G·戈洛夫 于 2020-01-08 设计创作，主要内容包括：本发明涉及用以收集在自主交通工具中产生的传感器数据的系统、方法及设备。所述交通工具的传感器产生传感器数据流,所述传感器数据流分别并行且以循环方式缓冲于第一循环缓冲器及较大的第二循环缓冲器中。所述交通工具的高级驾驶员辅助系统在检测或预测到事故时产生事故信号且在检测到物体检测、辨识、识别或分类中的错误时提供训练信号。所述事故信号致使将传感器数据流区段从所述第一循环缓冲器复制到非易失性存储器的以轮循方式从多个槽中选择的槽中。所述训练信号致使将传感器数据流区段从所述第二循环缓冲器复制到所述非易失性存储器的在为所述第一循环缓冲器保留的所述槽之外的区域中。(The present invention relates to systems, methods, and apparatus for collecting sensor data generated in an autonomous vehicle. Sensors of the vehicle generate sensor data streams that are respectively buffered in parallel and in a circular manner in a first circular buffer and a larger second circular buffer. Advanced driver assistance systems of the vehicle generate an accident signal when an accident is detected or predicted and provide a training signal when an error in object detection, recognition or classification is detected. The fault signal causes copying of a sensor data stream segment from the first circular buffer into a slot of a non-volatile memory selected from a plurality of slots in a round robin fashion. The training signal causes copying of a sensor data stream segment from the second circular buffer into a region of the non-volatile memory outside of the slot reserved for the first circular buffer.)

1. An autonomous vehicle, comprising:

a sensor configured to generate a sensor data stream during operation of the autonomous vehicle on a road;

an advanced driver assistance system configured to operate the autonomous vehicle on the road based on the sensor data stream, wherein the advanced driver assistance system is further configured to generate an accident signal in response to detecting or predicting an accident and to generate a training signal in response to an error in object detection, recognition, identification, or classification;

a non-volatile memory;

a first circular buffer having a capacity to buffer first sensor data generated by the sensor up to a first time period;

a second circular buffer having a capacity to buffer second sensor data generated by the sensor for a second time period up to longer than the first time period, wherein the first and second circular buffers are configured to buffer the stream of sensor data within the capacity of the first and second circular buffers in parallel and in a circular manner, respectively, in the absence of the incident signal and the training signal; and

a controller configured to copy the first sensor data from the first circular buffer into the non-volatile memory in response to the incident signal and copy the second sensor data from the second circular buffer into the non-volatile memory in response to the training signal.

2. The autonomous vehicle of claim 1, wherein the sensor comprises a digital camera, radar, lidar, or an ultrasonic sonar, or any combination thereof.

3. The autonomous vehicle of claim 2, wherein the advanced driver assistance system includes a map of the road and an artificial neural network; and the advanced driver assistance system is configured to generate the training signal in response to a mismatch between an identification of an object in the map of the road and an identification of a corresponding object via application of the sensor data stream in the artificial neural network.

4. The autonomous vehicle of claim 3, wherein after the training signal, the autonomous vehicle is configured to transmit the second sensor data from the non-volatile memory to a remote server configured to update the map of the road at least according to the second sensor data, update the artificial neural network through machine learning using the second sensor data, or update both the map and the artificial neural network.

5. The autonomous vehicle of claim 4, wherein the first sensor data is generated entirely by the sensor prior to the accident signal; and the second sensor data includes a first portion generated by the sensor before the training signal and a second portion generated by the sensor during a predetermined time period after the training signal.

6. The autonomous vehicle of claim 5, wherein the controller is further configured to:

stopping buffering data from the sensor data stream into the first circular buffer in response to the incident signal, thereafter copying at least a portion of the first sensor data from the first circular buffer into the non-volatile memory; and is

Continuing to buffer data from the sensor data stream into the second circular buffer for a predetermined period of time in response to the training signal, thereafter ceasing to buffer further data from the sensor data stream into the second circular buffer.

7. An autonomous vehicle data logger, comprising:

a non-volatile memory;

a communication interface configured to receive a sensor data stream from a sensor of the autonomous vehicle;

a first circular buffer having a capacity to buffer first sensor data from the stream of sensor data, the first sensor data generated during a first time period;

a second circular buffer having a capacity to buffer second sensor data from the stream of sensor data, the second sensor data generated during a second time period that is longer than the first time period, wherein the first and second circular buffers are connected to the communication interface to perform buffering concurrently and in parallel; and

a controller configured to:

copying the first sensor data from the first circular buffer into the non-volatile memory in response to a first type of signal; and is

Copying the second sensor data from the second circular buffer to the non-volatile memory in response to a second type of signal.

8. A data logger as claimed in claim 7, wherein the first type of signal is indicative of an accident; and the second type of signal indicates a training opportunity for an advanced driver assistance system of the vehicle.

9. The data recorder according to claim 8, wherein in response to the signal of the first type, the controller is configured to stop buffering data from the stream of sensor data into the first circular buffer, after which at least a portion of the first sensor data is copied from the first circular buffer into the non-volatile memory.

10. The data recorder according to claim 9, wherein in response to the signal of the second type, the controller is configured to continue buffering data from the sensor data stream into the second circular buffer for a predetermined period of time, after which buffering of more data from the sensor data stream into the second circular buffer stops.

11. The data logger of claim 10 wherein the first sensor data is generated entirely by the sensor prior to the first type of the signal; and the second sensor data includes a first portion generated by the sensor before the signal of the second type and a second portion generated by the sensor during the predetermined time period after the signal of the second type.

12. The data recorder according to claim 11, wherein said first and second circular buffers are volatile memories; and the non-volatile memory comprises NAND flash memory.

13. The data recorder according to claim 7, wherein both the first circular buffer and the second circular buffer are configured to buffer a common portion of the sensor data stream prior to receiving the first or second type of signal.

14. A data logger as claimed in claim 13, wherein the common portion is generated by the sensor during the same time period.

15. A method, comprising:

receiving, in a data logger of an autonomous vehicle, a sensor data stream generated by a sensor of the autonomous vehicle during operation of the autonomous vehicle on a road;

buffering the sensor data stream in parallel and in a circular manner into a capacity of a first circular buffer of the data recorder and a capacity of a second circular buffer of the data recorder, respectively, wherein:

the capacity of the first circular buffer is configured to buffer up to a first section of the sensor data stream, the first section being generated by the sensor in a first time period; and is

The capacity of the second circular buffer is configured to buffer up to a second section of the sensor data stream, the second section being generated by the sensor in a second time period that is longer than the first time period;

receiving in the data logger an accident signal generated by an advanced driver assistance system detecting or predicting an accident;

copying the first segment of the sensor data stream from the first circular buffer into a non-volatile memory in response to the incident signal;

receiving in the data logger a training accident signal resulting from the advanced driver assistance system detecting an error in object detection, recognition, identification or classification; and

copying the second section of the sensor data stream from the second circular buffer into the non-volatile memory in response to the training signal.

16. The method of claim 15, further comprising:

the sensor data stream is generated by the sensor, which includes a digital camera, radar, lidar or ultrasonic sonar, or any combination thereof.

17. The method of claim 16, wherein the error includes a mismatch between object identification provided in a map of the road and object identification by applying the sensor data stream in an artificial neural network of the advanced driver assistance system of the autonomous vehicle; and the method further comprises:

transmitting the second segment of the sensor data stream from the non-volatile memory to a remote server after the training signal;

update the map of the road according to the second sensor data using the second section of the stream of sensor data, update the artificial neural network through machine learning using the second sensor data, or update both the map and the artificial neural network.

18. The method of claim 17, further comprising:

dividing the non-volatile memory into a first region and a second region, wherein the second section of the sensor data stream is copied into the second region in response to the training signal;

organizing the first region into a plurality of slots, each slot sufficient to store the entire contents of the first circular buffer; and

maintaining, in the slot, a plurality of data stream segments copied from the first circular buffer in response to a most recent accident signal generated by the advanced driver assistance system of the autonomous vehicle.

19. The method of claim 18, further comprising:

copying data in the first circular buffer into a slot selected from the plurality of slots according to a round robin scheme.

20. The method of claim 18, further comprising:

organizing the second region into a plurality of slots, each slot sufficient to store the entire contents of the second circular buffer;

storing a priority indicator for data stored in the slot; and

select a selected slot from the plurality of slots in the second region based on the priority indicator and a priority indicator associated with the training signal in response to the training signal, wherein the second section is copied from the second circular buffer into the selected slot in the second region.

Technical Field

At least some embodiments disclosed herein relate to recording sensor data of autonomous vehicles for subsequent analysis, such as accident review and/or updating data, maps, and artificial neural network models used by Advanced Driver Assistance Systems (ADAS).

Background

Autonomous vehicles often include a number of sensors to help control the operation of the autonomous vehicle. In the case of an accident, collision or near-collision involving an autonomous vehicle, it may be beneficial to review sensor data recorded just prior to and/or during the accident to help possibly determine the cause of the accident and/or whether there may be a design defect and/or vehicle failure.

In the event power is lost during an accident, vehicle sensor data stored in volatile memory may be lost. Volatile memory requires power to maintain the data stored therein. Examples of volatile memory include Dynamic Random Access Memory (DRAM) and Static Random Access Memory (SRAM).

Some memory integrated circuits are non-volatile and can retain stored data even when not powered for long periods of time (e.g., days, months, years). Examples of non-volatile memory include flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), and electronically erasable programmable read-only memory (EEPROM) memory, among others.

Some non-volatile memories have a useful life span limited by the cycles of programming and erasing to store new data, a program-erase (P/E) budget represents a predetermined number of cycles of programming and erasing that can be reliably performed for replacing data in an erasable media, after the predetermined number of cycles of erasing, the program-erase (P/E) budget for such an erasable media is exhausted, and thus, the media can become statistically unreliable and thus deemed to be the end of its useful life, for example, a single-level-cell (S L C) flash memory can have a P/E budget near 100,000 cycles, a multi-level-cell (M L C) flash memory can have a P/E budget in the range from 10,000 to 30,000 cycles, and a three-level-cell (T L C) or four-level-cell (Q L C) flash memory can have a P/E budget between 3,000 to 5,000 cycles.

Disclosure of Invention

According to an aspect of the present application, an autonomous vehicle is provided. The autonomous vehicle includes: a sensor configured to generate a sensor data stream during operation of the autonomous vehicle on a road; an advanced driver assistance system configured to operate the autonomous vehicle on the road based on the sensor data stream, wherein the advanced driver assistance system is further configured to generate an accident signal in response to detecting or predicting an accident and to generate a training signal in response to an error in object detection, recognition, identification, or classification; a non-volatile memory; a first circular buffer having a capacity to buffer first sensor data generated by the sensor up to a first time period; a second circular buffer having a capacity to buffer second sensor data generated by the sensor for a second time period up to longer than the first time period, wherein the first and second circular buffers are configured to buffer the stream of sensor data within the capacity of the first and second circular buffers in parallel and in a circular manner, respectively, in the absence of the incident signal and the training signal; and a controller configured to copy the first sensor data from the first circular buffer into the non-volatile memory in response to the incident signal, and copy the second sensor data from the second circular buffer into the non-volatile memory in response to the training signal.

According to another aspect of the present application, a data logger of an autonomous vehicle is provided. The data recorder comprises: a non-volatile memory; a communication interface configured to receive a sensor data stream from a sensor of the autonomous vehicle; a first circular buffer having a capacity to buffer first sensor data from the stream of sensor data, the first sensor data generated during a first time period; a second circular buffer having a capacity to buffer second sensor data from the stream of sensor data, the second sensor data generated during a second time period that is longer than the first time period, wherein the first and second circular buffers are connected to the communication interface to perform buffering concurrently and in parallel; and a controller configured to: copying the first sensor data from the first circular buffer into the non-volatile memory in response to a first type of signal; and copying the second sensor data from the second circular buffer to the non-volatile memory in response to a second type of signal.

According to yet another aspect of the present application, a method is provided. The method comprises the following steps: receiving, in a data logger of an autonomous vehicle, a sensor data stream generated by a sensor of the autonomous vehicle during operation of the autonomous vehicle on a road; buffering the sensor data stream in parallel and in a circular manner into a capacity of a first circular buffer of the data recorder and a capacity of a second circular buffer of the data recorder, respectively, wherein: the capacity of the first circular buffer is configured to buffer up to a first section of the sensor data stream, the first section being generated by the sensor in a first time period; and the capacity of the second circular buffer is configured to buffer up to a second section of the sensor data stream, the second section being generated by the sensor in a second time period that is longer than the first time period; receiving in the data logger an accident signal generated by an advanced driver assistance system detecting or predicting an accident; copying the first segment of the sensor data stream from the first circular buffer into a non-volatile memory in response to the incident signal; receiving in the data logger a training accident signal resulting from the advanced driver assistance system detecting an error in object detection, recognition, identification or classification; and copying the second section of the sensor data stream from the second circular buffer into the non-volatile memory in response to the training signal.

Drawings

Embodiments are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings in which like references indicate similar elements.

Fig. 1 shows a system in which a vehicle is configured with a data logger to collect sensor data for improving its Advanced Driver Assistance System (ADAS) and/or for accident review.

FIG. 2 shows an autonomous vehicle having a data logger, according to one embodiment.

FIG. 3 shows a data recorder according to one embodiment.

FIG. 4 shows a method of operating a data recorder, according to one embodiment.

FIG. 5 shows another method of operating a data recorder according to one embodiment.

Detailed Description

At least some embodiments disclosed herein provide systems, methods, and apparatus to collect and store sensor data generated in an autonomous vehicle or another vehicle with an Advanced Driver Assistance System (ADAS) under different conditions and for different purposes.

In one example, after an accident (e.g., a collision or near-collision) or near-accident event, the sensor data may be reviewed to determine the cause of the event. This sensor data may be referred to herein as accident sensor data. The accident sensor data may be analyzed to identify the cause of the accident or proximity accident event and/or unsafe aspects or designs of the autonomous vehicle, for example. The analysis may result in improved control designs and/or configurations for safety operation of the autonomous vehicle and/or similar vehicles.

In another example, Advanced Driver Assistance Systems (ADAS) of a vehicle (e.g., autonomous driving systems) may be used to improve Advanced Driver Assistance Systems (ADAS) after the ADAS detect errors/mistakes in processing sensor data or mismatches in identifying objects from sensor data or failure to identify/classify objects from sensor data or to recognize, detect, identify or classify objects with a minimum level of confidence. Such sensor data that is not associated with an accident or a proximity accident event may be referred to herein as training sensor data. Improvements may be made, for example, by performing machine learning using sensor data and/or updating a data model in the ADAS used to classify or identify objects. Such data models may include High Definition (HD) maps of road systems in which autonomous vehicles travel and/or artificial neural networks for object detection, classification, recognition, and/or identification.

High Definition (HD) maps are commonly used in autonomous vehicles for motion planning. This map can have a high accuracy in the centimeter level. HD maps may provide geometric and semantic information about the environment in which the vehicle is operating (e.g., a road system). The geometric information provided in the HD map may include raw sensor data collected by light detection and ranging (lidar), radar, cameras, sonar, GPS, and the like. Semantic information may identify objects such as lane boundaries, intersections, parking spaces, parking signs, traffic lights, and other information such as traffic speed, lane change limits, and the like.

Some techniques have been developed that further use HD maps to generate representations/inputs for object detection, classification, recognition and/or recognition performed using an Artificial Neural Network (ANN).

Objects identified in the HD map may be compared to objects identified via sensor data generated by digital cameras, lidar (light detection and ranging), radar (radio detection and ranging), ultrasonic sonar (acoustic navigation and ranging), and so forth. When there is a mismatch in object detection, classification, recognition, or identification, the sensor data may be used in the analysis to update the HD map and/or train an Artificial Neural Network (ANN) for object detection, classification, recognition, or identification.

In some embodiments disclosed herein, the autonomous vehicle's data recorder is configured with two separate circular buffers (e.g., HD maps and/or ANN models) for buffering accident sensor data for subsequent accident review and for buffering training sensor data for improving the ADAS. Separate circular buffers may be separately configured to meet different needs for buffering sensor data for different uses, such as accident review and ADAS improvement. In other embodiments disclosed herein, a single circular buffer is controlled by a controller to meet the different needs of two separate circular buffers.

For example, the data logger may be configured with an incident data circular buffer to buffer incident sensor data for subsequent incident review and a training data circular buffer to buffer training sensor data for improving ADAS.

For example, a data recorder may receive a data stream containing real-time video data from various cameras mounted on a vehicle and various output signals from other sensors (e.g., radar, lidar, ultrasonic sonar, etc.). The sensor data may be raw data from the sensor or compressed image/video data.

Each of the circular buffers is configured to buffer data in a circular manner, with incoming data filling into the buffer until the buffer is full, and then other new data is buffered to replace the oldest data in the circular buffer. The circular buffer may be implemented using a type of memory that has very high endurance (e.g., allowing terabytes of data to be written and rewritten in the circular buffer). For example, the circular buffer may be implemented via SRAM or DRAM, which is volatile and requires power to retain its contents. Alternatively, the circular buffer may be implemented using non-volatile memory with very high endurance (e.g., cross-point memory).

Cross-point memories use transistor-less memory elements, each of which has memory cells and selectors stacked together in a column. The columns of memory elements are connected via two vertical layers of conductive lines, one above and one below the columns of memory elements. Each memory element may be individually selected at the intersection of one conductive line on each of the two layers. Cross-point memory devices are fast and non-volatile, and can be used as a unified memory pool for processing and storage.

Some non-volatile memories with very high endurance suffer from short data retention (e.g., minutes or hours).

The autonomous vehicle may be configured to generate a signal indicative of the occurrence of the accident or a prediction of a potential accident. This signal may be referred to herein as an accident signal.

The accident signal may be generated by an ADAS (e.g., autonomous driving system) based on the sensor data. For example, activating automatic emergency braking may cause an accident signal to be generated for the data recorder. For example, activating the anti-lock brake system may cause an accident signal to be generated to the data logger. For example, acceleration faster than a threshold may cause an accident signal to be generated to the data recorder. For example, an accident signal may be generated for the data logger when a collision avoidance system of the vehicle detects a threat of collision or determines that the probability of collision is above a threshold. For example, an accident signal may be generated for the data logger when the airbag sensor determines that a crash is occurring during the initial impact of the crash. For example, when the ADAS detects a dangerous situation from the sensor data (e.g., another car or object is closer than a threshold to the autonomous vehicle), an accident signal is generated for the data logger.

The non-volatile memory of the data logger may be implemented using high density and high data retention flash memory that can retain data for months and/or years without being powered, for example, a three level cell (T L C) or a four level cell (Q L C) NAND flash memory may be used as the non-volatile memory of the data logger.

In response to the crash signal, the data logger copies the contents of the crash data circular buffer into non-volatile memory so that crash sensor data can be retained after a crash that can prevent power to the data logger for a long period of time.

Preferably, the non-volatile memory of the data logger is configured to contain several slots for accident sensor data. Each of the slots is a dedicated storage space for holding sensor data copied from the incident data circular buffer. In response to each crash signal, the contents of the crash data circular buffer are copied into an available one of the slots in the non-volatile memory. Initially, all slots may be used to store sensor data copied from the circular buffer of incident data. After all of the slots have been used in response to past incident signals, the slot storing the sensor data associated with the oldest incident signal may be made available to store new current sensor data copied from the incident data circular buffer in response to the current incident signal.

The data logger is configured to ensure that copying of sensor data from the incident data circular buffer to the non-volatile memory is complete even when an incident causes the power supply of the data logger to be disconnected. For example, the data logger may have a backup power supply sufficient to power operations to complete copying of sensor data from the incident data circular buffer to the non-volatile memory even when the data logger's external power supply is removed immediately following the incident signal. The backup power source may be implemented via a capacitor that is charged to store power for operation, while the data logger is typically powered via an external power supply (e.g., during operation to log data in an accident data circular buffer prior to an accident). Alternatively or in combination, the backup power source may include a battery mounted within the housing of the data recorder.

The incident data circular buffer is configured to buffer the sensor data for a period of time (e.g., 30 seconds) immediately preceding the incident signal. Copying the sensor data buffered during the time period into a slot in a non-volatile memory of the data logger in response to the incident signal; and existing data in the fault data circular buffer at the time of the fault signal is not overwritten with new data from the sensor before copying the corresponding data into the slot.

In one implementation, in response to the crash signal, the crash data circular buffer temporarily stops buffering incoming new sensor data to retain its contents and resumes buffering data after at least the oldest portion of the contents in the crash data circular buffer has been copied to a slot in the data logger's non-volatile memory. Optionally, the crash data circular buffer may resume buffering incoming new sensor data after completing the storage of the existing data of the crash signal into the non-volatile memory. Optionally, the crash data circular buffer may resume buffering incoming new sensor data after a predetermined portion of the contents of the crash data circular buffer has been copied into a slot in the non-volatile memory and before the entire contents of the crash data circular buffer are copied into the slot, wherein buffering incoming new sensor data does not overwrite the crash sensor data to be copied to the non-volatile memory of the data recorder.

In another implementation, in response to the crash signal, the crash data circular buffer is configured to read the oldest portion of the contents in the crash data circular buffer for recording in non-volatile memory and then record the incoming new sensor data in the appropriate location to overwrite the oldest portion in the crash data circular buffer. Thus, the data logger empties the contents of the crash data circular buffer into a slot in the non-volatile memory while continuing to log the incoming sensor data in the portion of the emptied portion of the crash data circular buffer; and the data logger can continue to buffer incoming sensor data without being able to buffer any incoming sensor signals while copying content associated with the incident signal (e.g., generated during a 30 second time period immediately preceding the incident signal) into a slot in non-volatile memory.

Optionally, the data logger and/or autonomous vehicle may be configured to upload data content in slots in the non-volatile memory to a remote server via a communication network/connection. For example, the data transfer may be configured to be performed in one of several options. One option is to transmit the data wirelessly to a server via a cellular communication network. Another option is to transmit data wirelessly to the server via a wireless local area network (e.g., Wi-Fi). Yet another option is to transmit over a wired connection on a diagnostic port of the autonomous vehicle. Yet another option is to remove the data logger from the autonomous vehicle to enable a wired connection to a communication port of the data logger.

The incident sensor data recorded in the slots of the non-volatile memory of the data logger can be used to investigate the incident or proximity incident event.

The autonomous vehicle may generate a signal indicative of a request to save sensor data for training and/or improving the ADAS. This signal may be referred to herein as a training signal.

For example, the training signal may be generated by an ADAS (e.g., autonomous driving system) in response to a mismatch between road objects (e.g., lane boundaries, obstacles, traffic signs, etc.) identified in the HD map and corresponding objects detected or recognized from the sensor data. Sensor data associated with the mismatch may be used to train the ADAS in recognizing objects and/or update the HD map for corresponding sections of the road.

The training data loop buffer is configured to buffer the training sensor data in parallel with the incident data loop buffer buffering the incident sensor data. The training data circular buffer is configured to have a capacity to store data collected over a longer period of time (e.g., 3 minutes) than the incident data circular buffer (e.g., 30 seconds).

In response to the training signal, the training sensor data is copied from the training data circular buffer into the non-volatile memory of the data logger. The training sensor data that is replicated in response to the training signal may include a portion that is generated before the training signal and a remaining portion that is generated after the training signal. In some examples, the portion generated before the training signal is for a period of time equal to or longer than the length of the stream segment buffered in the accident data circular buffer (e.g., 30 seconds or more). In other examples, all of the training sensor data that is replicated in response to the training signal is generated prior to the training signal. Optionally, a time period selection indication is provided with the training signal such that the data logger can selectively replicate the portion generated before the training signal and the remainder generated after the training signal. Alternatively, the ADAS may select the time period by adjusting the timing of the training signal such that the predefined time window relative to the training signal is the time period of the sensor data stream segment to be saved as training sensor data. In some examples, upon receiving a training signal in the data logger, the training data circular buffer continues to buffer incoming sensor data signals until the remainder of the data generated after the training signal is buffered in the training data circular buffer for copying into the non-volatile memory of the data logger. Optionally, the training data circular buffer may temporarily stop buffering more incoming sensor data after the rest of the training signal generated after the training signal is buffered after the training data circular buffer and before copying of the training sensor data associated with the training signal to the non-volatile memory of the data recorder is complete. The copying of the training sensor data may be configured to begin in response to the training signal and before completion of buffering all training sensor data associated with the training signal. Alternatively, the replication of the training sensor data may be configured to begin at or after the completion of buffering all of the training sensor data associated with the training signal.

Preferably, the non-volatile memory of the data logger is configured with a dedicated area reserved to store training sensor data. The training data area in the non-volatile memory is separate from the slots configured for sensor data for incident review.

Optionally, the training data region may have a plurality of slots for storing training sensor data associated with a plurality of training signals. The data logger may be configured to maintain training sensor data associated with the last training signal.

Optionally, the data logger is further configured to receive a priority indication of the training signal. For example, the ADAS may evaluate a degree of mismatch between an object identified in the HD map and a corresponding object detected/recognized from the sensor data. A high mismatch is assigned a high priority. Similarly, a low confidence level in detecting, classifying, recognizing, or identifying an object may be assigned a high priority. Thus, the data logger may be configured to maintain training sensor data based on the priority indication when the data logger has insufficient slots to store training data that has not been transmitted to the server.

The content copied into the training data area of the non-volatile memory of the data recorder may be automatically transmitted to the remote server via some of the communication options discussed above for transmitting incident data, such as a wireless cellular communication network (e.g., 5G cellular communication) and/or a wireless local area network (e.g., Wi-Fi). Depending on network bandwidth and coverage, data transmission may take a period of time from minutes to hours/days. The non-volatile memory of the data logger may retain the data without power until the data transmission is completed.

Fig. 1 shows a system in which a vehicle is configured with a data logger to collect sensor data for improving its Advanced Driver Assistance System (ADAS) and for accident review.

The system of fig. 1 includes a vehicle 111 having a data logger 101. The data logger 101 may be configured according to any of the embodiments disclosed herein. The vehicle 111 has an Advanced Driver Assistance System (ADAS)105 and one or more sensors 103 that provide sensor data inputs to the ADAS 105.

For example, the sensors 103 may include digital cameras, lidar, radar, ultrasonic sonar, brake sensors, speed sensors, acceleration sensors, airbag sensors, GPS (global positioning system) receivers, and the like.

The sensor 103 time-dependent output is provided as a sensor data stream to the ADAS 105 to support its operation and to the data recorder 101 for buffering and/or recording. The ADAS 105 may generate an incident signal and/or a training signal to initiate transfer of sensor data from the circular buffer to the non-volatile memory of the data logger 101, as discussed above.

For example, in response to detecting an accident or proximity accident event, the ADAS 105 may generate an accident signal to cause the data logger 101 to store accident sensor data (e.g., 127) for a duration B (e.g., 30 seconds). The incident sensor data 127 is typically generated by the sensor for a duration of time B (e.g., 30 seconds) and buffered in a circular buffer of the data logger 101 before the ADAS 105 generates the incident signal.

For example, in response to detecting an opportunity to train or improve the ADAS 105 in addition to an accident or near-accident event, the ADAS 105 may generate a training signal to cause the data logger 101 to store training sensor data (e.g., 121) for a duration a (e.g., 3 minutes). Training sensor data 121 is generated by the sensor for a duration of a (e.g., 3 minutes) and buffered in a circular buffer of data logger 101. The duration a (e.g., 3 minutes) may include a portion before and optionally a portion after the training signal generated by the ADAS 105.

The vehicle 111 is configured with a wireless communication device to transmit accident sensor data 127 and/or training sensor data 121 to a remote server 119 via wireless signals 113 and a communication network 117. The remote server 119 is typically deployed at a location remote from the roadway 102 on which the vehicle 111 is used. One example of a communication network 117 is a mobile phone network having one or more base stations (e.g., 115) to receive wireless signals (e.g., 113). Another example of a communication network 117 is the internet, where wireless local area network signals (e.g., 113) transmitted by the vehicle 111 are received in an access point (e.g., 115) for further communication to the server 119. In some embodiments, the vehicle 111 transmits the sensor data (e.g., 121 or 127) to the server 119 using a communication link 107 or communication balloon to the satellite 109.

The server 119 may be configured to include an incident module 129. The accident sensor data 127 may be reviewed and/or analyzed in the accident module 129 to identify the cause of the accident and/or possible design or configuration changes in order to improve autonomous driving and/or ADAS.

Server 119 may be configured to include training and updating model 123. The training sensor data 121 may be used in the training and updating module 123 to improve an Artificial Neural Network (ANN)125 of the ADAS 105 and/or to update a High Definition (HD) map 126 of the roads 102 used by the vehicle 111.

In some implementations, the ADAS 105 may optionally include a machine learning module configured to use training sensor data 121 stored in non-volatile memory of the data logger 101 to improve its ANN, with or without assistance from the server 119.

FIG. 2 shows an autonomous vehicle 111 with a data logger 101 according to one embodiment. For example, the vehicle 111 in the system of fig. 1 may be implemented using the autonomous vehicle 111 of fig. 2.

The vehicle 111 of fig. 2 is configured with an Advanced Driver Assistance System (ADAS) 105. The ADAS 105 of the vehicle 111 may have a High Definition (HD) map 126 for motion planning and an Artificial Neural Network (ANN)125 for object detection, recognition, identification, and/or classification. Optionally, the HD map 126 may also be used for object detection, recognition, identification, and/or classification.

The ADAS 105 may generate a training signal for the data recorder 101 when there is a mismatch between the object identified in the HD map 126 and the identification of the object via the ANN 125.

When the ADAS 105 detects an accident or proximity accident event, the ADAS 105 may generate an accident signal for the data logger 101.

The vehicle 111 typically includes an infotainment system 149, a communication device 139, one or more sensors 103, and a computer system 131 connected to some controls of the vehicle 111, such as a steering control 141 for guiding the vehicle 111, a braking control 143 for stopping the vehicle 111, an acceleration control 145 for the speed of the vehicle 111, and so forth. In some embodiments, the vehicles 111 in the system of fig. 1 have similar configurations and/or similar components.

The computer system 131 of the vehicle 111 includes one or more processors 133, a data recorder 101, and a memory 135 that stores firmware (or software) 147 including computer instructions and a data model for the ADAS 105. The data model of the ADAS 105 may include the HD map 126 and the ANN 125.

The one or more sensors 103 may include a visible light camera, an infrared camera, a lidar, a radar or sonar system, a peripheral sensor, a Global Positioning System (GPS) receiver, a satellite positioning system receiver, a brake sensor, and/or an airbag sensor. The sensors may provide a real-time sensor data stream to computer system 131. The sensor data generated by the vehicle's sensors 103 may include capturing images of objects using a camera that images with light visible to the human eye or a camera that images with infrared light or a sonar, radar, or lidar system. The image data obtained from the at least one sensor of the vehicle is part of the collected sensor data for recording in the data logger.

The output of the ADAS 105 may be used to control (e.g., 141, 143, 145) the acceleration of the vehicle 111, the speed of the vehicle 111, and/or the direction of the vehicle 111 during autonomous driving.

In some embodiments, after server 119 updates ANN 125 and/or HD map 126, server 119 may update the ADAS of vehicle 111 over-the-air by downloading ANN 125 and/or HD map 126 to computer system 131 of vehicle 111 via communication device 139 and/or wireless signal 113.

FIG. 3 shows a data recorder according to one embodiment. For example, the data recorder 101 in the system of fig. 1 and/or in the vehicle 111 of fig. 2 may be implemented using the data recorder 101 of fig. 3.

The data logger 101 of fig. 3 may include a housing 137 that encloses its components. Optionally, the housing 137 of the data logger 101 seals the components of the data logger 101 and protects the data logger 101 and/or data stored in the non-volatile memory 165 from damage during an accident; and thus the data stored in the non-volatile memory 165 of the data logger 101 may be retrieved following a crash/collision, fire and/or flood that typically occurs in an accident involving a vehicle.

The data logger 101 of fig. 3 includes one or more communication interfaces or devices 153, a fault data circular buffer 161, a training data circular buffer 163, a controller 159, and a non-volatile memory 165.

The communication interface or device 153 may be used to receive the sensor data stream 151 from the sensors 103 of the vehicle 111 in which the data logger 101 is installed. The sensor data stream 151 is provided in parallel to an incident data circular buffer 161 and a training data circular buffer 163.

Further, the communication interface or device 153 may be used to receive training signals (e.g., 157) and accident signals 155 from the vehicle 111 and/or its ADAS 105.

In response to the incident signal 155, the controller 159 stops recording the incoming sensor data stream 151 into the incident data circular buffer 161 and begins copying the contents of the incident data circular buffer 161 to the non-volatile memory 165.

The non-volatile memory 165 has a plurality of slots 171, 173, …, 179. Each of the slots is sufficient to store the entire contents of the crash data circular buffer 161. The controller 159 may use the slots 171, 173, …, 179 in a round robin fashion and/or in a round robin fashion such that after a certain number of incident signals, the slots 171, 173, …, 179 store the latest set of incident sensor data associated with and/or identified by the most recent incident signal.

In response to the training signal 157, the controller 159 may continue to buffer the incoming sensor data stream 151 into the training data circular buffer 163 such that the training data circular buffer 163 contains a training sensor data set recorded partially before the training signal 157 and partially after the training signal 157. The controller 159 then copies the training sensor data set into a region 167 of the non-volatile memory 165 reserved for training sensor data.

Optionally, the region 167 may also include a plurality of slots, each sufficient to store the contents of the training data circular buffer 163.

Optionally, data logger 101 includes a backup power source, such as a capacitor or rechargeable battery.

FIG. 4 shows a method of operating a data recorder, according to one embodiment. For example, the method of fig. 4 may be implemented in the data logger 101 of fig. 3 in the vehicle 111 of fig. 2 in the system of fig. 1.

At blocks 201 and 211, the data recorder 101 provides a first circular buffer 161 and a second circular buffer 163.

At block 203, the data logger 101 buffers a first sensor data stream of a first duration (e.g., 30 seconds) in a first circular buffer 161.

At block 213, the data logger 101 buffers a second stream of sensor data for a second duration (e.g., 3 minutes) in a second circular buffer 163.

The first circular buffer 161 and the second circular buffer 163 operate in parallel. Data is buffered in a circular manner into each of circular buffers 161 and 163 such that when the buffers are full, the newest data overwrites the oldest data. For example, data is recorded from the beginning of the buffer towards the end of the buffer. Once the end of the buffer is reached, data is again written from the beginning of the buffer towards the end of the buffer as if the end of the buffer followed the beginning of the buffer.

At block 205, the data logger 101 receives a request to log sensor data. The request may be an accident signal or a training signal.

At block 207, the data logger 101 determines whether the request is for recording an incident (e.g., a collision or near collision).

If the request is for recording a crash, then at block 209, the data recorder 101 copies the first contents of the first circular buffer 161 to the non-volatile memory 165.

Optionally, in response to the request being to record an incident, data recorder 101 stops buffering incoming sensor data into first circular buffer 161 for a period of time (e.g., to prevent overwriting of existing data in first circular buffer 161 and/or to reserve backup power when an external power supply of data recorder 101 is interrupted).

At block 221, the data logger 101 transmits or provides the first content from the non-volatile memory 165 to the server 119 after the incident.

At block 223, the server 119 reviews the incident based on the first content.

If the request is not a recording incident, then at block 219 the data recorder 101 copies the second content from the second circular buffer 163 to the non-volatile memory 165.

Optionally, in response to the request, the data logger 101 continues to buffer incoming sensor data into the first circular buffer 161 for a period of time, and thus buffers second content containing sensor data for a duration that is partially before the request and partially after the request. After a period of continued buffering, the data recorder 101 may stop buffering for another period of time, after which copying of the second content into the non-volatile memory 165 is completed.

At block 231, the data logger 101 transmits the second content in the non-volatile memory 165 to the server 119.

At block 233, the server 119 refines an Advanced Driver Assistance System (ADAS) based on the second content.

FIG. 5 shows another method of operating a data recorder according to one embodiment. For example, the method of fig. 5 may be implemented in the data logger 101 of fig. 3 in the vehicle 111 of fig. 2 in the system of fig. 1. The method of fig. 5 may be performed in combination with the method of fig. 4.

At block 251, the data logger 101 provides a first circular buffer 161 for buffering sensor data related to incidents and a second circular buffer 163 for buffering sensor data not related to incidents.

At block 253, the data logger 101 provides the non-volatile memory 165.

At block 255, the data recorder 101 divides the non-volatile memory 165 into a first portion 171, 173, …, 179 and a second portion 167, wherein the first portion has a storage capacity that is a multiple of the capacity of the first circular buffer 161 and the second portion is sufficient to store data buffered in the second circular buffer 167.

At block 257, the data recorder 101 organizes the first portion into a predetermined number of slots 171, 173, …, 179, each slot having a capacity to store data buffered in the first circular buffer 161.

At block 259, the data logger 101 stores sensor data for up to a predetermined number of recent incidents in a predetermined number of slots 171, 173, …, 179.

For example, the data logger 101 copies the incident sensor data from the incident data circular buffer 161 into the first slot 171 in response to the initial incident signal. For each subsequent incident signal, the data logger 101 stores the incident sensor data in the next slot (e.g., 173, …, 179). After reaching the last slot 179, the data recorder 101 uses the first slot 171 as the next slot, as if the first slot 171 were the next slot following the last slot 179.

Optionally, the second portion 167 reserved for training sensor data buffered in the training data circular buffer 163 is also configured with a plurality of slots. Each of the slots stores an indicator that identifies a priority of training sensor data stored in the respective slot. When the slot in the second portion 167 is full, the data logger 101 reuses the slot with the lowest priority lower than the current training sensor data in the training data circular buffer 163.

In one example, the autonomous vehicle 111 has a sensor 103 configured to generate a sensor data stream 151 during operation of the autonomous vehicle 111 on the road 102. An Advanced Driver Assistance System (ADAS)105 of the vehicle 111 may be configured to operate the autonomous vehicle 111 on the roadway 102 based on the sensor data stream 151. The Advanced Driver Assistance System (ADAS)105 is further configured to generate an accident signal 155 in response to detecting or predicting an accident and generate a training signal 157 in response to errors in object detection, recognition, identification, or classification.

The data recorder 101 of the vehicle 111 in the example includes a non-volatile memory 165, a first circular buffer 161, a second circular buffer 163, and a controller 159. First circular buffer 161 has a capacity to buffer a first section (e.g., 30 seconds) of sensor data stream 151; and a second circular buffer has a capacity to buffer a second section (e.g., 3 minutes) of the sensor data stream 151. In the absence of the incident signal and the training signal, the first circular buffer 161 and the second circular buffer 163 are configured to buffer the sensor data stream 151 within their respective capacities in parallel and in a circular manner, respectively. The sensor data stream segment buffered in the second circular buffer 163 may include the sensor data stream segment buffered in the first circular buffer 161 prior to generating the accident signal 155 and the training signal 157.

In response to the fault signal 155, the controller 159 is configured to copy the sensor data stream segment in the first circular buffer 161 into the non-volatile memory 165. In response to the training signal 157, the controller 159 is configured to copy the sensor data stream segment in the second circular buffer 163 into the non-volatile memory 165.

The sensor data stream 151 from the sensor 103 may include the output of the sensor 103 (e.g., a digital camera, radar, lidar, or ultrasonic sonar, or any combination thereof). An Advanced Driver Assistance System (ADAS)105 may include a map 126 of the road 102 and an artificial neural network 125 for detecting, recognizing, identifying, and/or classifying objects on the road 102 based on a sensor data stream 151. In response to a mismatch between the identification of an object in the map 126 of the road 102 and the identification of a corresponding object in the artificial neural network 125 performed by applying the sensor data stream 151, the Advanced Driver Assistance System (ADAS)105 may generate a training signal 155.

Preferably, after the training signal 157, the autonomous vehicle 111 is configured to automatically transmit the sensor data stream segment, copy it into the non-volatile memory 165 in response to the training signal 157, transmit it to the remote server 119. The server 119 may be configured with a training and updating module 123 to update the map 126 of the road 102 according to the sensor data stream segments received from the vehicles 111, and/or to update the artificial neural network 125 using machine learning techniques, such as supervised machine learning techniques.

In response to the incident signal 155, a section of the sensor data stream 151 that was generated prior to the incident signal 155 and/or that was buffered in a first circular buffer 161 at the time of the incident signal 155 is copied into the non-volatile memory 165. Optionally, the controller 159 may cause the first circular buffer 161 to stop buffering incoming segments of the sensor data stream 151 that immediately follow the fault signal 155 in order to prevent a portion of the segments of the sensor data stream 151 that were in the first circular buffer 161 at the time of the fault signal 155 from being lost before at least a portion of the content was copied from the first circular buffer 161 to the non-volatile memory 165.

Optionally, in response to the training signal 157, a section of the sensor data stream 151 generated partially before the training signal 157 and partially after the training signal 157 is buffered in the second circular buffer 163 and copied into the non-volatile memory 165. For example, the controller 159 may continue to use the second circular buffer 163 to buffer the section of the sensor data stream 151 that immediately follows the training signal 157 to complete the buffering of the section for copying into the non-volatile memory 165. Once the section of the sensor data stream 151 to be copied into the non-volatile memory 165 is in the second circular buffer 163, the controller 159 can then cause the second circular buffer 163 to stop buffering another incoming section of the sensor data stream 151.

In some embodiments, the crash data circular buffer 161 is not used. In response to the fault signal 155, the controller 159 copies a portion of the contents of the training data circular buffer 163 corresponding to the section to be buffered in the fault data circular buffer 161 into a slot in the non-volatile memory 165. In other implementations, the training circular buffer 163 is not used. In response to the training signal 157, the controller 159 copies at least a portion of the contents of the crash data circular buffer 161 corresponding to the section to be buffered in the training data circular buffer 163 into a slot 173 in the non-volatile memory 165, while storing the section of the sensor data stream 151 following the training signal 155 directly into the region 167 of the non-volatile memory 165. Training sensor data corresponding to training signal 157 is provided from the combination of the sector copied into the corresponding slot (e.g., 173) by the accident data circular buffer 161 and the sector saved directly into region 167.

In some implementations, in response to the incident signal 155, the controller 159 may stop operation of the training data circular buffer 163 to reserve power for copying data in the incident data circular buffer 161 to a non-volatile memory (e.g., 165) (e.g., in response to detecting loss of an external power supply of the data recorder 101).

The first and second circular buffers 161, 163 can be volatile memory (e.g., DRAM or SRAM), and the non-volatile memory 165 can include T L C/Q L C NAND flash memory the first and second circular buffers 161, 163 can buffer a common section of the sensor data stream 151 prior to receiving the fault signal 155 or the training signal 157.

Optionally, the controller 159 divides the non-volatile memory 165 into a first area 171, 173, …, 179) and a second area 167. Further, the controller 159 organizes the first region 171, 173, …, 179 into a plurality of slots 171, 173, …, 179, wherein each slot (e.g., 171) has sufficient storage capacity to store the entire contents of the first circular buffer 161. The sensor data stream segment associated with the training signal 157 and/or identified by the training signal 157 is copied into the second region 167; and the sensor data stream segment associated with the incident signal 155 and/or identified by the incident signal 155 is copied into one of the slots 171, 173, …, 179 in the first region. The controller 159 may use the slots to maintain up to a predetermined number of accident sensor data stream segments in the slots 171, 173, …, 179 associated with and/or identified by the most recent accident signals generated by the Advanced Driver Assistance System (ADAS)105 of the autonomous vehicle 111 according to a round robin scheme.

Optionally, the controller 159 may also organize the second region 167 of the non-volatile memory 165 into slots, wherein each slot has sufficient storage capacity to store the entire contents of the second circular buffer 163. The controller 159 may store (e.g., in non-volatile memory or another memory) a priority indicator for data stored in the slot. In response to the training signal 157, the controller 159 selects a slot from the slots in the second region 167 based on the priority indicator of the existing data in the slots in the second region 167 and the priority indicator associated with the training signal 157. The training sensor data is copied from the second circular buffer 163 into a selected slot in the second region 167.

The present invention includes methods and apparatus to perform the methods described above, including data processing systems to perform these methods and computer readable media containing instructions that when executed on a data processing system cause the system to perform these methods.

Server 119, computer system 131, and/or data logger 101 may each be implemented as one or more data processing systems.

A typical data processing system may include an interconnect (e.g., a bus and system core logic) that interconnects a microprocessor and a memory. The microprocessor is typically coupled to a cache memory.

The interconnect interconnects the microprocessor and the memory together and also interconnects them to input/output (I/O) devices via an I/O controller. The I/O devices may include display devices and/or peripheral devices such as mice, keyboards, modems, network interfaces, printers, scanners, cameras, and other devices known in the art. In one embodiment, when the data processing system is a server system, some of the I/O devices (e.g., printer, scanner, mouse, and/or keyboard) are optional.

The interconnect may include one or more buses connected to each other through various bridges, controllers, and/or adapters. In one embodiment, the I/O controller includes a USB adapter for controlling USB (Universal Serial bus) peripheral devices and/or an IEEE-1394 bus adapter for controlling IEEE-1394 peripheral devices.

The memory may include one or more of: ROM (read only memory), volatile RAM (random access memory) and non-volatile memory such as hard drives, flash memory, etc.

Volatile RAM is typically implemented as dynamic RAM (dram) which requires power continuously in order to refresh or maintain the data in the memory. The non-volatile memory is typically a magnetic hard drive, a magnetic optical drive, an optical drive (e.g., DVDRAM), or other type of memory system that maintains data even after power is removed from the system. The non-volatile memory may also be a random access memory.

The non-volatile memory may be a local device coupled directly to the rest of the components in the data processing system. Non-volatile memory that is remote from the system, such as a network storage device coupled to the data processing system through a network interface (e.g., a modem or ethernet interface), may also be used.

In the present invention, some functions and operations are described as being performed by or caused by software code to simplify explanation. However, such expressions are also used to specify the functions caused by the execution of the code/instructions by a processor, such as a microprocessor.

Alternatively or in combination, the functions and operations described herein may be implemented using special purpose circuits, such as using an Application Specific Integrated Circuit (ASIC) or a Field Programmable Gate Array (FPGA), with or without software instructions. Embodiments may be implemented using hardwired circuitry, without or in combination with software instructions. Thus, the techniques are not limited to any specific combination of hardware circuitry and software nor to any particular source for the instructions executed by the data processing system.

While one embodiment may be implemented in a fully functional computer and computer system, the various embodiments are capable of being distributed as a computing product in a variety of forms and of being applied regardless of the particular type of machine or computer-readable medium used to actually effect the distribution.

At least some aspects of the disclosure may be embodied, at least in part, in software. That is, the techniques may be performed in a computer system or other data processing system in response to its processor (e.g., microprocessor) executing sequences of instructions contained in a memory (e.g., ROM, volatile RAM, non-volatile memory, cache memory, or remote storage).

The routines executed to implement the embodiments, may be implemented as part of an operating system or a specific application, component, program, object, module or sequence of instructions referred to as a "computer program". The computer programs typically include one or more instructions that are set at various times in various memories and storage devices in the computer, and that, when read and executed by one or more processors in the computer, cause the computer to perform operations necessary to execute elements relating to various aspects.

A machine-readable medium may be used to store software and data which, when executed by a data processing system, cause the system to perform various methods. Executable software and data may be stored in various places including, for example, ROM, volatile RAM, non-volatile memory, and/or cache memory. Portions of such software and/or data may be stored in any of these storage devices. Further, the data and instructions may be obtained from a centralized server or a peer-to-peer network. Different portions of the data and instructions may be obtained from different centralized servers and/or peer-to-peer networks at different times and in different communication sessions or in the same communication session. The data and instructions may be obtained entirely prior to execution of the application. Alternatively, portions of data and instructions may be obtained dynamically (just in real time) as needed for execution. Thus, data and instructions are not required to be entirely on the machine-readable medium at a particular instance in time.

Examples of computer readable media include, but are not limited to, non-transitory recordable and non-recordable type media such as volatile and non-volatile memory devices, Read Only Memory (ROM), Random Access Memory (RAM), flash memory devices, floppy and other removable disks, magnetic disk storage media, optical storage media (e.g., Compact Disc Read Only Memory (CDROM), Digital Versatile Disks (DVD), etc.), among others. A computer-readable medium may store instructions.

The instructions may also be embodied in digital and analog communications links of electrical, optical, acoustical or other forms of propagated signals, such as carrier waves, infrared signals, digital signals, etc. However, a propagated signal (e.g., a carrier wave, an infrared signal, a digital signal, etc.) is not a tangible machine-readable medium and is not configured to store instructions.

In general, a machine-readable medium includes any mechanism that provides (i.e., stores and/or transmits) information in a form accessible by a machine (e.g., a computer, network device, personal digital assistant, manufacturing tool, any device with a set of one or more processors, etc.).

In various embodiments, hardwired circuitry may be used in combination with software instructions to implement techniques. Thus, the techniques are not limited to any specific combination of hardware circuitry and software nor to any particular source for the instructions executed by the data processing system.

The foregoing description and drawings are illustrative and are not to be considered limiting. Numerous specific details are described to provide a thorough understanding. However, in certain instances, well-known or conventional details are not described in order to avoid obscuring the description. References to one or an embodiment in the present disclosure are not necessarily to the same embodiment; and such references mean at least one.

In the foregoing specification, the invention has been described with reference to specific exemplary embodiments thereof. It will be evident that various modifications may be made thereto without departing from the broader spirit and scope as set forth in the following claims. The specification and drawings are, accordingly, to be regarded in an illustrative sense rather than a restrictive sense.