阅读说明：本技术 进行车辆自诊断的方法和控制装置 (Method and control device for self-diagnosis of vehicle ) 是由 L-G·桑德尔 于 2021-04-08 设计创作，主要内容包括：本发明提供了在车辆(100)中进行车辆自诊断的方法(400)和控制装置(310)。所述方法(400)包括执行所述车辆(100)的预期会触发警报的第一动作(401)；检测由所执行的第一动作(401)触发的所述警报(402)；执行所述车辆(100)的预期会禁用所述警报的第二动作(403)；检测由所执行的第一动作(401)触发的所述警报被禁用(404)；以及基于所执行的第一动作(401)、所触发的警报的检测(402)、所执行的第二动作(403)和/或所禁用的警报的检测(404)的结果,确定允许/禁止所述车辆(100)的行驶(410)。(A method (400) and control device (310) for making a vehicle self-diagnosis in a vehicle (100) are provided. The method (400) comprises performing a first action (401) of the vehicle (100) that is expected to trigger an alarm; detecting the alarm (402) triggered by the performed first action (401); performing a second action (403) of the vehicle (100) that is expected to disable the alert; detecting that the alarm triggered by the performed first action (401) is disabled (404); and determining to allow/prohibit travel (410) of the vehicle (100) based on results of the performed first action (401), the detection of the triggered alert (402), the performed second action (403), and/or the detection of the disabled alert (404).)

1. A method (400) of making a vehicle self-diagnosis in a vehicle (100), the method (400) comprising the steps of:

performing a first action (401) of the vehicle (100) that is expected to trigger an alarm;

detecting the alarm (402) triggered by the performed first action (401);

performing a second action (403) of the vehicle (100) that is expected to disable the alert;

detecting that the alarm triggered by the performed first action (401) is disabled (404); and

determining to enable/disable travel (410) of the vehicle (100) based on results of the performed first action (401), the detection (402) of the triggered alert, the performed second action (403) and/or the detection (404) of the disabled alert.

2. The method (400) of claim 1, further comprising the steps of:

obtaining a confirmation (405) confirming that a result of the performed action corresponds to an expected result; and wherein said determination (410) of enabling/disabling of driving is performed further based on the obtained confirmation (405).

3. The method (400) of claim 2, when no acknowledgement is obtained (405), further comprising the steps of:

performing a third action (406) in order to determine a reason for failing to obtain the acknowledgement (405), eliminating or at least reducing the effect of failing to obtain the acknowledgement (405); and wherein the step of determining permission/prohibition of travel (410) is further performed based on the result of the performed third action (406).

4. The method (400) according to any one of claims 1 to 3, further comprising the steps of:

checking a wireless communication function (407) between a control device (310) of the vehicle (100) and a vehicle external entity (200); and wherein the step of determining permission/prohibition of travel (410) is further performed based on the result of the performed functional check (407).

5. The method (400) according to any one of claims 1 to 4, further comprising the steps of:

checking a communication function (408) between a control device (310) of the vehicle (100) and a vehicle sensor (110,120,130, 140); and wherein the step of determining permission/prohibition of travel (410) is further performed based on the result of the performed functional check (408).

6. The method (400) according to any one of claims 1 to 5, further comprising the steps of:

estimating braking performance and friction (409) between the tyres and the bottom (115) of the vehicle (100); and wherein the step of determining permission/prohibition of travel (410) is further performed based on the estimated braking performance and friction (409) force.

7. The method (400) according to any of claims 1-6, when it has been determined that the driving (410) of the vehicle (100) is prohibited, further comprising the steps of:

-performing an activity (411) of correcting any anomalies to achieve a driving permission of said vehicle (100).

8. The method (400) of claim 7, wherein the performed activity (411) comprises outputting an alarm to inform a responsible person of the vehicle (100) about disabled driving of the vehicle (100).

9. A control device (310) for making a vehicle self-diagnosis in a vehicle (100), wherein the control device (310) is configured to:

generating a command for performing a first action of the vehicle (100) that is expected to trigger an alarm;

detecting the alarm triggered by the performed first action;

generating a command for performing a second action of the vehicle (100) that is expected to disable the alert;

detecting that the alarm triggered by the performed first action is disabled; and

determining to allow/prohibit travel of the vehicle (100) based on a result of the performed first action, the detection of the triggered alert, the performed second action, and/or the detection of the disabled alert.

10. The control device (310) of claim 9, further configured to:

obtaining a confirmation that a result of the performed action corresponds to an expected result; and

determining permission/prohibition of travel of the vehicle (100) further based on the obtained confirmation.

11. The control device (310) of claim 10, further configured to:

when no confirmation is obtained confirming that the result of the performed action corresponds to the expected result, a third action is performed to

Determining a reason for failing to obtain the confirmation;

eliminating or at least reducing the effect of failing to obtain the acknowledgement; and

determining to permit/prohibit travel of the vehicle (100) based on a result of the executed third action.

12. The control device (310) of any of claims 9-11, further configured to:

checking a wireless communication function between the control device (310) and a vehicle external entity (200); and

determining permission/prohibition of travel of the vehicle (100) based on a result of the executed functional check.

13. The control device (310) of any of claims 9-12, further configured to:

checking a communication function between the control device (310) and a vehicle sensor (110,120,130, 140); and

determining permission/prohibition of travel of the vehicle (100) based on a result of the executed functional check.

14. The control device (310) of any of claims 9-13, further configured to:

estimating braking performance and friction between tyres and a bottom (115) of the vehicle (100); and

determining to permit/prohibit travel of the vehicle (100) based on the estimated braking performance and friction.

15. The control device (310) according to any one of claims 9 to 14, further configured to, when it has been determined that the travel of the vehicle (100) is prohibited, perform an activity of correcting any abnormality so as to achieve the travel permission of the vehicle (100).

16. The control device (310) of claim 15, wherein the performed activity includes outputting an alert to inform a responsible person of the vehicle (100) about disabled driving of the vehicle (100).

17. A computer program comprising instructions which, when executed by a computer, cause the computer to perform the method (400) according to any one of claims 1 to 8.

18. A computer-readable medium comprising instructions that, when executed by a computer, cause the computer to perform the method (400) of any of claims 1-8.

19. A vehicle (100) comprising a control device (310) according to any one of claims 9 to 16.

Technical Field

The present document relates to a method and a control arrangement in a vehicle. More specifically, a method and control apparatus for self-diagnosis in a vehicle are described.

Background

From the viewpoint of traffic safety, it is very important to periodically check the health condition of the vehicle before starting driving, and it is also important to keep the vehicle in a good operable state and maintain its functional and economic values. One example might be to walk around the vehicle when the engine is running and the lights are on and then leave to check if the lights are working, if the lens is broken or cracked (which may affect the light distribution), if the rear view mirror is intact and clean, if there is no oil stain under the vehicle (which can be seen as an indirect indication that shop repairs need to be done), etc.

Fortunately, there are electrical tests in many vehicles to detect and present a visual indication to the driver, for example, when the lights are broken. However, any other of the above-described defects cannot be detected by such known electrical tests. In addition, in the event of a failure of the display of the vehicle (where the visual indication is presented), the driver may not be able to notice the visual indication being issued, resulting in a potentially dangerous traffic safety situation. It is desirable to improve error code management and to be able to detect, for example, sensor or communication disconnection, error display failure, etc.

Vehicles often rely on wireless communication with ambient vehicles and other off-vehicle entities, as well as functional location services, such as for navigation. Traffic safety may be compromised if wireless communication does not function properly.

A vehicle as discussed herein may include a broad range of transportation means such as a truck, car, motorcycle, trailer, bus, bicycle, train, tram, airplane, boat or other similar manned or unmanned conveyance.

Due to lack of time and/or interest, many vehicle drivers may not regularly check their vehicles, at least not intervene as desired.

Additionally, some vehicles may be unmanned, so-called autonomous vehicles. Therefore, no driver performs any check at all about the vehicle condition.

In the case of an inattentive driver and/or an unmanned vehicle, knowing, for example, how long the error has been, how it has occurred, whether there are any consecutive errors, etc., is very helpful to the vehicle mechanic and/or vehicle manufacturer when eventually taken to the workshop. An inattentive driver in such a situation may not be much helpful to the mechanic if regular vehicle inspections are not performed.

Another problem relates to digital maps used in the navigator of the vehicle. Such maps are typically based on collected data, which may become obsolete due to road reconstruction. This is dangerous for blindly trusting the pilot's driver. For autonomous vehicles, a reliable digital map is critical for successful driving; however, updating digital maps requires a great deal of work.

Another problem may be particularly relevant to autonomous vehicles, i.e. vehicles may be injured or damaged due to accidents, theft, vandalism, etc. during parking, which existing alarms may not be able to detect. The damage that occurs may cause traffic safety problems, but it may also be important to detect the event so that damage can be reported to the insurance company and/or the police department.

Document US20170269593 discloses a control system for an autonomous vehicle that checks the function of a plurality of components before starting to drive the autonomous vehicle. Detection of a defective component can result in failure of the autonomous driving function. The vehicle functions examined are described as steering, braking, acceleration and/or gear shifting.

However, the disclosed control system does not provide a solution to the problem of vehicle internal communication and/or failure of vehicle external communication with other entities.

Document US20170205824 proposes a method for checking various data of a parked autonomous vehicle before starting to travel.

This document also does not propose a solution to the problem of the internal communication of the vehicle and/or the external communication of the vehicle with other entities failing.

The document WO2019/088893 describes a vehicle adapted to be driven manually or in an autonomous mode. Before allowing a change from the manual driving mode to the autonomous driving mode, certain functions of the vehicle, such as lights, stabilizing systems and brakes, are checked.

This document relates to mode change/allowing mode change without triggering a safe parking or driving permission.

Therefore, it is desirable to improve the false detection of a vehicle before releasing the vehicle.

Disclosure of Invention

It is therefore an object of the present invention to address at least some of the above problems and to improve traffic safety by improving the detection of anomalies in vehicles.

According to a first aspect of the invention, this object is achieved by a method of self-diagnosing a vehicle in a vehicle. The method includes performing a first action of the vehicle that is expected to trigger an alarm. The method also includes detecting an alarm triggered by the performed first action. Additionally, the method includes performing a second action of the vehicle that is expected to disable the alert. Also, the method includes detecting that an alarm triggered by the performed first action is disabled. The method comprises determining to enable/disable travel of the vehicle based on a result of the performed first action, the detection of the triggered alert, the performed second action, and/or the detection of the disabled alert.

According to a second aspect of the invention, the object is achieved by a control device in a vehicle. The control device aims at vehicle self-diagnosis of the vehicle. The control device is configured to generate a command for performing a first action of the vehicle that is expected to trigger an alarm. In addition, the control device is further configured to detect that an alarm triggered by the performed first action is disabled. The control device is further configured to determine to enable/disable travel of the vehicle based on a result of the performed first action, the detection of the triggered alarm, the performed second action, and/or the detection of the disabled alarm.

Due to the described aspects, the vehicle self-diagnosis is performed by first performing a first action of intentionally causing an alarm, then performing a second action of disabling the alarm and simultaneously monitoring the behavior of the alarm, thereby determining that the internal communication and the alarm system associated with the performed action are operating normally before issuing the vehicle travel permission.

In any instance of the first action to be performed, the pressure in the brake system of the vehicle may be reduced by braking when the pressure in the brake system is below a threshold limit, an alarm/error code may be triggered. When a low pressure alarm is detected, a second action including increasing the pressure of the brake system may be performed by, for example, operating an on-board compressor until the low pressure alarm disappears.

In this way, the success of the action performed is confirmed, and the function of the warning process is also verified before the vehicle is allowed to start running, and various problems that may occur on the vehicle, which a careful driver would normally detect, can be detected by the control routine for self-diagnosis provided, thereby improving traffic safety.

Other advantages and additional novel features will become apparent from the detailed description that follows.

Drawings

Embodiments of the invention will now be described in further detail with reference to the accompanying drawings, in which:

fig. 1A shows a side view of a vehicle according to an embodiment of the invention.

FIG. 1B shows a vehicle according to an embodiment of the invention, as described above.

Fig. 2 shows a vehicle and a vehicle exterior entity according to an embodiment of the invention.

Fig. 3A illustrates an example of anomaly detection in an image according to an embodiment of the present invention.

Fig. 3B illustrates an example of anomaly detection in an image according to an embodiment of the present invention.

Fig. 3C shows an example of anomaly detection in an image according to an embodiment of the present invention.

Fig. 3D illustrates an example of anomaly detection in an image according to an embodiment of the present invention.

Fig. 4A is a flow chart illustrating an embodiment of a first portion of a method.

Fig. 4B is a flow chart illustrating an embodiment of a second portion of the method.

Fig. 5 is a diagram depicting a system according to an embodiment.

Detailed Description

The embodiments of the invention described herein are defined as methods and control devices that may be put into practice in the embodiments described below. These embodiments may, however, be illustrated and implemented in many different forms and are not limited to the examples set forth herein; but rather to provide these illustrative examples of implementations so that this disclosure will be thorough and complete.

Other objects and features may become apparent from the following detailed description considered in conjunction with the accompanying drawings. It is to be understood, however, that the drawings are designed solely for purposes of illustration and not as a definition of the limits of the embodiments disclosed herein, for which reference should be made to the appended claims. In addition, the drawings are not necessarily to scale and, unless otherwise indicated, they are merely intended to conceptually illustrate the structures and procedures described herein.

Fig. 1A shows a scenario of a vehicle 100. Vehicle 100 travels along travel direction 105 on road 115.

Vehicle 100 may include a broad range of transportation vehicles such as trucks, automobiles, motorcycles, trailers, buses, bicycles, trains, trams, airplanes, boats, drones, spacecraft, or other similar manned or unmanned vehicles.

In various embodiments, the vehicle 100 may be driver controlled or driver free (i.e., autonomously controlled). However, the disclosed solution may have particular advantages for autonomous vehicles.

The vehicle 100 may include various sensors, such as one or more forward pointing sensors 110, one or more right pointing sensors 120, one or more backward pointing sensors 130, and/or one or more left pointing sensors 140, as shown along the direction of travel 105. This is also disclosed in overview in fig. 1B.

The sensors 110,120,130,140 may be of the same or different types, and may include, for example, cameras, stereo cameras, infrared cameras, video cameras, radar, lidar, ultrasound devices, time-of-flight cameras, or the like in different embodiments.

In some embodiments, the sensor may comprise, for example, a motion detector and/or be based on a Passive Infrared (PIR) sensor that is sensitive to human skin temperature by emitting black body radiation at mid-infrared wavelengths, as opposed to background objects at room temperature; or by emitting continuous microwave radiation waves and detecting motion by doppler radar principles; or by emitting ultrasonic waves to detect and analyze the reflections; or by a tomographic motion detection system based on radio wave interference detection, etc.

The sensors 110,120,130,140 may be located in the vehicle 100 to perform various functions, such as providing environmental information for Advanced Driver Assistance Systems (ADAS) or systems for autonomously driving the vehicle 100.

Such sensors may be located in the vehicle 100 and output directly from the vehicle 100, for example, for detecting obstacles in front of the vehicle 100. However, a portion of the images, video sequences, or other information captured by the sensors will cover the own vehicle 100.

By using onboard sensors to collect information about the own vehicle 100, images, video sequences, or other sensor information, and analyzing this information, for example, by comparing it to stored information of ideal images, video sequences, or other sensor information of the vehicle 100 or a portion thereof, anomalies that may be caused by damaged components on the vehicle 100, or signs of an accident approaching the vehicle 100, ice on the road 115, or the like, may be detected. This solution works like a second pair of eyes (or, in the case of an autonomous vehicle, a unique pair of eyes) constantly looking for abnormal situations that may indicate a malfunction of the vehicle 100. In this way, the self-diagnosis of the vehicle 100 (or a portion thereof) may be performed before starting traveling and/or before allowing the vehicle 100 to leave the starting zone. The launch area may be a parking section, a loading/unloading area, a fenced area, or the like where the vehicle 100 travels at a low or limited speed.

The solution provided involves making an action for triggering the activation of an alarm of the vehicle 100, i.e. making an action known to result in an alarm or warning, such as reducing the pressure in the brake system of the vehicle 100 by braking. Braking is performed until a low pressure level alarm is triggered, thereby determining that the alarm is operating as intended. The pressure of the brake system may then be increased by operating, for example, an on-board compressor or an external compressed air supply until the pressure level of the brake system reaches a sufficient/acceptable level and the alarm is disabled. Additionally, in some embodiments, monitoring the action for triggering an alarm does not trigger any other alarms.

In other embodiments, the vehicle 100 may be checked for the correct functional product Feature (FPC) code. The FPC code describes the functional features of the vehicle 100.

Before granting the driving permission, it can also be checked whether the vehicle 100 has a valid Diagnostic Trouble Code (DTC) and/or has a valid DTC in error.

Additionally, each ECU of the vehicle 100 may be checked for the correct ECU source address or identity, for example by comparing the extracted ECU source address/identity with database information. Further, a reference number for uniquely identifying the vehicle 100 may be extracted and checked, for example, for verification against a database. In the event that the vehicle 100 does not include an ECU with a capacity exceeding a minimum standard limit for autonomous driving, or the vehicle 100 is on a hot list of land and/or stolen vehicles, the vehicle 100 may be shut down.

From this determination, the intercom and alarm system associated with the particular action operates normally prior to issuing a driving license to vehicle 100. Thereby improving traffic safety.

Another example of the vehicle self-diagnosis may be to open a door of the vehicle 100, check whether an alarm about the opened door is generated. Thereafter, the door is closed and it is checked whether the alarm is disabled.

Yet another example may be to detect (passenger) pressure on a seat of the vehicle 100, determine if the seat belt of the seat is closed and trigger the vehicle to stop running until the belt is closed or the passenger leaves the vehicle 100.

In some embodiments, receiving confirmation of correspondence between expected results of an action and the resulting results of the action performed may be a prerequisite to issuing a driving license for the vehicle 100. Therefore, failure to confirm the result of the action with the expected result may result in the vehicle 100 not being provided with the running permission.

Failure to confirm the outcome of an action with an expected outcome may be, for example, due to the expected action not being performed, or the expected action not being performed as expected, an alarm not being generated/disabled as expected, and/or other unexpected behavior related to the vehicle 100, such as a change in light distribution implying a broken or mismatched lamp, a dirty or broken lamp lens, etc., e.g., by any of the sensors 110,120,130, 140; the absence of a turn signal light when using a turn signal means that the light becomes dirty or broken; changes in cab roll, yaw or pitch angle mean cab suspension breakage; a false detection of the high beam area may trigger an adjustment of the light; false detection of the adaptive main beam (i.e. dimming down the high beam when encountering other vehicles or traffic users); detection of unsafe cargo; false detection of a rain sensor, etc.

In some embodiments, a command for an action is generated, the result of the command is examined and compared to an expected result. If they correspond (within the threshold limits), the correctness of the action is confirmed. Otherwise, deviations from the expected results of the commands/actions will be detected.

In one example, a command may be generated for performing a defined action to be performed by the on-board actuator, e.g. turning the driving wheel to the left/right to some extent. After a certain defined period of time, the result of the issued command (i.e. the drive wheel angle) can be measured. A comparison can then be made between the expected value according to the generated command and the measurement of the command. In case the difference between these entities exceeds a threshold limit (which may be e.g. a difference of 5%, a difference of 10% or other predefined or configurable value), then the detected deviation may trigger e.g. a limited driving permission, e.g. driving slower than a certain speed limit, driving with increased distance to the vehicle in front, only driving during off-peak hours, only driving on certain roads, only driving to the vehicle in front, completely prohibiting a driving permission, etc.

Failure to obtain confirmation may also involve poor communication with the sensors 110,120,130,140 and/or sensor 110,120,130,140 or actuator failure; or positioning errors. In addition, wireless communications with entities external to the vehicle may be checked, such as illustrated in fig. 2.

Fig. 2 shows the vehicle 100 in wireless communication with a vehicle-external entity 200 via a wireless transceiver 210.

Wireless communication may be conducted through a wireless communication interface, such as vehicle-to-vehicle (V2V) communication or vehicle-to-infrastructure (V2I) communication. The general term car-to-all (V2X) is sometimes used.

In some embodiments, communication between entities 100, 200 may be performed via V2V communication, e.g., based on Dedicated Short Range Communication (DSRC) devices. In some embodiments, the DSRC operates in the 5.9GHz band, with a bandwidth of 75MHz, and in the approximate range of 1000 m.

The wireless communication may be in accordance with any IEEE standard for wireless vehicle communication, such as a special operating mode of IEEE802.11 for vehicle networks, known as Wireless Access (WAVE) in an on-board environment. IEEE802.11 p is an extension of the 802.11 wireless LAN medium access layer (MAC) and physical layer (PHY) specifications.

Other wireless communication options include Wi-Fi, Bluetooth, RFID, 3GPP LTE, and the like.

In some embodiments, tests may be conducted regarding the functionality of wireless communication between the vehicle 100 and the vehicle-external entity 200 and/or the functionality of the vehicle-external entity 200.

The test may be performed by signaling the vehicle-external entity 200 to stimulate it to return a response message. In case of receiving a return signal from the vehicle-outside entity 200, a conclusion is drawn that the wireless communication between the respective entities 100, 200 is functioning properly.

However, more complex tests may be a challenge-response mechanism. A message (challenge) may be sent from the vehicle 100 to the vehicle-external entity 200. The vehicle-external entity 200 may perform contracted operations on the challenge, such as adding a shared secret value to the challenge and calculating a hash value according to a known algorithm. The result of the calculation (response) is then returned to the vehicle 100 where the comparison can be made by doing the same at the vehicle 100, i.e. adding the shared secret value to the challenge and calculating the hash value. In the case of the same value, the vehicle exterior entity 200 may be regarded as operating normally. It is also verified that the vehicle exterior entity 200 has not been spoofed.

Other functional tests may include cryptography, mutual authentication using a two-way challenge-response mechanism, and the like.

In the illustrated embodiment, the vehicle-external entity 200 is attached to a database 220 and vehicle-external sensors 230 such as cameras, lidar, radar, and the like.

In some embodiments, an anomaly of vehicle 100 may be detected based on sensor data captured by vehicle exterior sensors 230, which may capture sensor data of vehicle 100, such as damage due to rogue behavior or flat tires.

Accordingly, the vehicle 100 may request sensor data from the vehicle exterior sensors 230 from the vehicle exterior entity 200 to detect an abnormality related to the vehicle 100.

Fig. 3A shows an example of a vehicle interior of the vehicle 100, and depicts how a potential driver or passenger of the vehicle 100 perceives the previous scenario in fig. 1A, 1B, and/or 2.

The vehicle 100 includes a control device 310 for vehicle self-diagnosis of the vehicle 100, thereby determining whether the vehicle 100 is allowed or prohibited from running. The control device 310 may comprise or be attached to a database 320 in which expected reference values of the vehicle 100 may be stored.

In the illustrated embodiment, an alarm of the display 330 may be activated by an excitation. In this case, the pressure in the brake system of the vehicle 100 may be reduced by sending a brake signal to the brake actuator. When an alarm is caused due to low brake pressure, braking may be disabled, or pressure may be increased by generating an activation signal to the compressor. The pressure of the brake system may then be increased by operating, for example, an on-board compressor until the pressure level of the brake system reaches a sufficient/acceptable level and the alarm is disabled. Additionally, in some embodiments, monitoring the action for triggering an alarm does not trigger any other alarms.

Fig. 3B shows an example of a vehicle interior of the vehicle 100, and depicts how a potential driver or passenger of the vehicle 100 perceives the previous scenario in fig. 1A, 1B, and/or 2.

The control device 310 may collect information from the sensors 110,120,130,140 of the vehicle 100 and/or the vehicle exterior sensors 230. This information may be stored in database 320. Subsequently, a comparison may be made between the currently collected sensor data and previously stored sensor data retrieved from database 320.

The sensors 110,120,130,140 may be steered and/or reoriented in different directions, and a device intended to display objects outside the direct field of view of the driver may present an adjusted view of the relevant sensor 110,120,130, 140.

The control device 310 may communicate with other vehicle interior units, such as sensors 110,120,130,140 and/or actuators, via, for example, a communication bus. The communication bus may comprise, for example, a Controller Area Network (CAN) bus, a Media Oriented System Transport (MOST) bus, or the like. However, the data link may alternatively be established over a wireless connection that includes or is at least motivated by any wireless communication technology (such as Wi-Fi, bluetooth, etc.).

In the illustrated embodiment, a message is displayed on the display 330 to bring the potential driver/passenger to the attention of the observed and detected abnormality and to facilitate the reader's understanding. However, there may not be any display 330 in the vehicle 100 at all.

The control device 310 may be configured for image recognition/computer vision and object recognition.

Computer vision is a technical field including methods for acquiring, processing, analyzing and understanding images and, generally, high-dimensional data from the real world to produce numerical or symbolic information. The subject of development in this field is the ability to replicate human vision by electronically perceiving and understanding images. In this case, understanding means converting the visual image (input to the retina) into a description of the world that can interact with other mental processes and causing appropriate actions. This image understanding can be seen as a stripping of symbolic information from image data using models constructed by geometric, physical, statistical, and learning theories. Computer vision can also be described as a field of automation and integration of various processes and representations for visual perception.

The image data of the sensors 110,120,130,140 may take many forms, such as images, video sequences, views from multiple cameras, or multi-dimensional data from a scanner.

The expected state may be confirmed by comparing the expected/normal state extracted from the database 320 with current state sensor data captured from one or more sensors 110,120,130, 140. Alternatively, an anomaly may be detected. In the illustrated scene, a crack on the headlamp glass is detected.

In some embodiments, recommendations may also be made for the driver/passenger/vehicle owner regarding the operations that should be performed. In the event that a crack is detected on the windshield, the driver/passenger/vehicle owner may be advised to cover the crack from the outside with a piece of scotch tape and then travel to a glass repair shop. In some embodiments, if there is no person in the vehicle 100, the vehicle 100 may decide to travel to the plant anyway. Via the wireless internet connection, such workshops may be searched near the geographic location of the vehicle 100 and/or along the direction of travel 105 of the vehicle 100, and recommendations may be made based on, for example, price comparisons, instant service availability checks, and/or customer satisfaction of previous customers (if provided with such information).

Fig. 3C shows yet another example of a vehicle interior of the vehicle 100, and depicts a scene of how a broken headlamp glass is perceived from the interior of the autonomous vehicle 100 and by the owner (or other responsible person of the vehicle 100) via the vehicle exterior display device 200 located at a distance from the vehicle 100.

In addition to the already presented control means 310, data storage 320 and sensors 110,120,130,140, the vehicle 100 may also comprise a wireless transmitter or transceiver 150. The transmitter 150 may be in wireless communication with the display device 200 of the vehicle owner/supervisor.

The communication may be through a wireless communication interface such as any of the previously enumerated or similar devices.

In the illustrated embodiment, the sensor data comparison results in detection of a headlamp glass breakage on the left side of the vehicle 100. Thus, the detected abnormality is caused by a physical vehicle part failure. Since driving is illegal in case of a headlamp failure (at least in some jurisdictions), the vehicle 100 is parked at the roadside and the geographic location of the vehicle 100 is sent to the display device 200.

Thus, the owner/corresponding responsible/autonomous maintenance robot may be notified of the relevant situation and may take appropriate action, such as carrying the correct spare parts and tools, notifying the transport recipient of the delay (if any), and requiring someone to travel to the vehicle 100 for maintenance.

The geographic location of the vehicle 100 may be determined by a positioning unit 340 in the vehicle 100, which may be based on a satellite navigation system, such as navigation signal timing and ranging (Navstar) Global Positioning System (GPS), differential GPS (dgps), Galileo, GLONASS, and the like.

According to various embodiments, the geographic location of the positioning unit 340 (and thus the vehicle 100) may be generated continuously at some predetermined or configurable time interval.

Positioning by satellite navigation is based on distance measurements using triangulation from a plurality of satellites 350a, 350b, 350c, 350 d. In this example, four satellites 350a, 350b, 350c, 350d are depicted, but this is merely an example. More than four satellites 350a, 350b, 350c, 350d may be used to improve accuracy or create redundancy. The satellites 350a, 350b, 350c, 350d continuously transmit information about the time and date (e.g., in encoded form), the identity (the broadcasting satellite 350a, 350b, 350c, 350d), the status, and the location of the satellite 350a, 350b, 350c, 350d at any given time. The GPS satellites 350a, 350b, 350c, 350d transmit information that is encoded, for example, with different codes, but not necessarily based on Code Division Multiple Access (CDMA). This allows information from a single satellite 350a, 350b, 350c, 350d to be distinguished from information of other satellites based on the unique code of each respective satellite 350a, 350b, 350c, 350 d. This information may then be transmitted for receipt by a suitably adapted locating device included in the vehicle 100.

According to some embodiments, the range measurements may include measuring the difference in time taken for each respective satellite signal transmitted by a respective satellite 350a, 350b, 350c, 350d to reach the positioning unit 340. When the radio signal travels at the speed of light, the distance to each satellite 350a, 350b, 350c, 350d can be calculated by measuring the signal propagation time.

The positions of the satellites 350a, 350b, 350c, 350d are known because they are continuously monitored by approximately 15-30 ground satellite receiving stations located primarily along and near the earth's equator. Accordingly, the geographic location, i.e., latitude and longitude, of the vehicle 100 may be calculated by triangulating to determine distances to at least three satellites 350a, 350b, 350c, 350 d. To determine altitude, signals from four satellites 350a, 350b, 350c, 350d may be used, according to some embodiments.

The geographic location of the vehicle 100 may alternatively be determined, for example, by transponders positioned at known locations around the route of the vehicle 100 and dedicated sensors in the vehicle 100 to identify the transponders and thereby determine the location; by detecting and identifying WiFi networks (WiFi networks along a route may be mapped with certain corresponding geographic locations in the database); by receiving bluetooth beacon signals or other signal signatures of wireless signals associated with a geographic location, such as by triangulating signals transmitted by a plurality of fixed base stations having known geographic locations. The location may alternatively be input by a passenger in the vehicle 100.

In some embodiments, upon determining the geographic location of the positioning unit 340 (or in another manner), it may be presented on the display device 200, for example on a map that may mark the location of the vehicle 100.

Fig. 3D shows another example of a vehicle interior of the vehicle 100, and depicts a scenario in which, based on at least a portion of the sensor data, the detected anomaly includes a reasonable deviation of information in the digital map, which depicts the vehicle surroundings.

According to some embodiments, the captured sensor data may be compared to a map state. Such a comparison may result in a difference in the number of lanes actually detected compared to from the map data due to recent road construction or the like, or simply due to an error in the stored map data or a malfunction of the navigator 340. This is merely any example of such possible deviations between reality captured by the vehicle sensors 110,120,130, 140. Other examples may be a new road/inlet/outlet; new speed limits for existing roads; the road has become unidirectional, etc. The sensors 110,120,130,140 may continuously capture and collect information from roads, traffic signs, etc. while driving, and may be compared to map data as well as information associated with the geographic location of the vehicle 100, such as speed limits and other limits.

In the case where the correspondence between reality and stored map data cannot be confirmed, an alarm may be output to inform a person in charge of the digital map relating to the detected abnormality. Such alerts may also be provided to passengers via display 330 (if any) and/or to the owner/supervisor on display device 200.

For autonomous vehicles, it is important that the map data can be trusted for navigation, since it is possible that no driver will notice and react to deviations from the map data due to e.g. road construction, accidents or the like.

In some embodiments, the onboard sensors 110,120,130,140 may detect indications of accidents or dangerous situations on or near roads, such as stationary vehicles on roads; reversing vehicles on a highway; a vehicle traveling on a road against a traveling direction; a vehicle traveling on a pedestrian-dedicated area or a bicycle lane; human or animal lying on the road, etc. In this case, information about the detected accident indication may be transmitted to a police department, a traffic monitoring center, an emergency center, or the like, in addition to decelerating and/or stopping the own vehicle 100.

Fig. 4A-4B illustrate an example of a method 400 according to one embodiment. The flow chart in fig. 4A-4B illustrates a method 400 for use in the vehicle 100. The method 400 is directed to providing vehicle self-diagnostics.

The vehicle 100 may be, for example, a truck, bus, automobile, or similar conveyance previously mentioned configured for autonomous travel.

The vehicle 100 may include a plurality of sensors 110,120,130,140 that may be directed in different directions around the vehicle 100 and have respective monitoring areas that at least partially cover a portion of the own vehicle 100.

In order to be able to properly perform the vehicle self-diagnosis, the method 400 may include a plurality of steps 401-411. However, some of these steps 401-411 may be performed in various alternative ways. Some method steps may be performed only in some optional embodiments; such as steps 405-409 and/or 411. In addition, the described steps 401-411 may be performed in a temporal order slightly different from the numbering proposal order. The method 400 may include the subsequent steps of:

step 401 includes performing a first action of the vehicle 100 that is expected to trigger an alarm or error code. The first action may include reducing the pressure of the vehicle brake system, attempting to initiate travel with the parking brake activated, opening a door or hood, attempting to initiate travel with the travel brake activated, etc.

The purpose of performing the first action is to trigger the triggering of an alarm/error code, thereby verifying that the action has been performed and that the alarm system is functioning properly as expected.

Step 402 comprises detecting an alarm or error code triggered by the performed first action 401.

The alarm/error code triggered depends on the first action and may include, for example, a warning of low brake pressure, a warning that the parking brake has been activated, a warning that the door/bonnet has been opened, a warning that the service brake is active when a drive is attempted, etc.

By checking the triggered alarm/error code it is verified that the executed first action 401 has been executed correctly and that the alarm system is functioning properly as expected at least in respect of the triggered alarm.

Step 403 includes performing a second action of the vehicle 100 that is expected to disable the alert/error code.

Thus, the second action may be related to the first action 401 being performed and the alarm/error code being triggered and include, for example, activating a compressor to establish brake pressure, deactivating a parking brake, closing a door/bonnet, releasing a service brake, etc.

Step 404 comprises detecting that the alarm/error code triggered by the performed first action 401 is disabled, i.e. the previously detected alarm 402 is disabled, such as a warning of low brake pressure, a warning that the parking brake has been activated, a warning that the door/bonnet has been opened, a warning that the service brake is active when trying to drive, etc.

From which it is determined that the second action has been performed correctly and the verification/validation alarm is functioning properly as expected.

In some embodiments, it may also be checked that the performed first action 401 and/or second action 402 has not activated any other alarm than expected.

Step 405, which may only be performed in some specific embodiments, includes obtaining a confirmation that the result of the performed action corresponds to the expected result.

In some embodiments, the failure to obtain confirmation may be due to a physical vehicle part failure or an in-vehicle communication failure.

Additionally, in some embodiments, the action performed may not be confirmed under current driving conditions due to a deviation from the expected conditions of the vehicle 100. For example, the anomaly/deviation may include sending a defined input command to an actuator of the vehicle 100, such as a command to turn the drive wheels 10 degrees. The angle of rotation of the drive wheel may then be measured after a certain period of time and a comparison may be made between the command to the actuator and the resulting result of the operation of the actuator. In case the difference between the comparison values exceeds a threshold limit, such as 5%, a conclusion may be drawn that an anomaly is detected.

Another example may include activating a speaker, listening to the sound of the speaker with an acoustic sensor, and confirming that the result of the action performed corresponds to the expected result; or alternatively draw a conclusion that an abnormality is detected when the horn sound cannot be detected within an expected time frame.

Other examples may include increasing/decreasing vehicle speed, determining the speed of the vehicle 100 after a certain period of time, comparing the resulting speed with an expected speed and confirming that the result of the performed action corresponds to the expected result; or alternatively draw a conclusion that an anomaly is detected when the difference between the two values exceeds a threshold limit, such as 5%, 10%, 20%, etc.

Other examples may be activating lights of the vehicle 100 such as headlights, parking lights, flashing lights, etc., searching for the activated lights with on-board sensors and confirming/failing to confirm the correctness of the action performed, and/or drawing a conclusion that an anomaly is detected when the ignition light is not detected within an expected time frame.

In some embodiments, the sensor data obtained from the vehicle exterior sensors 230 may be validated based on image recognition.

Step 406, which may only be performed in some specific implementations in which step 405 has been performed, comprises performing a third action in order to determine the reason for failure to obtain acknowledgement 405, eliminating or at least reducing the effect of failure to obtain acknowledgement 405.

The third action may include, for example, extracting sensor data from the database 320, obtaining error code information, requesting service/maintenance/cleaning from a service provider, contacting the owner or other responsible person and notifying the action of a failed validation, and the like.

Step 407, which may only be performed in some specific embodiments, includes checking wireless communication functionality between the control device 310 of the vehicle 100 and the vehicle-external entity 200.

Accordingly, a communication failure in the wireless communication of the vehicle 100 with the environmental entity 200 can be detected. Since the traffic safety of autonomous vehicles, especially in 5G traffic environments, is highly dependent on proper wireless communication, a communication failure may have devastating consequences, which is why the failure to make wireless communication may prohibit the vehicle 100 from traveling. In some embodiments, the vehicle 100 may be allowed to travel to the nearest workshop for maintenance only at speeds below the maximum speed, for example 50 km/h.

Step 408, which may only be performed in some specific embodiments, includes checking the communication functionality between the control device 310 of the vehicle 100 and the vehicle sensors 110,120,130, 140.

For autonomous vehicles, lack of environmental sensor data or insufficient environmental data may be devastating to road safety. The inability to obtain sensor data from the vehicle sensors 110,120,130,140 may be equivalent to blinding the eyes to drive the manned vehicle, which is why the inability to receive communications may completely prohibit the vehicle 100 from traveling.

However, this of course depends on the sensor 110,120,130,140 for which no information can be received. In the event that a redundant sensor (i.e., a sensor having a coverage area that may be monitored by one or more other sensors) is unable to communicate, vehicle 100 may be allowed to travel to the nearest plant for maintenance at a limited speed, such as 50 km/h.

Step 409, which may only be performed in some specific embodiments, includes estimating the braking performance and friction between the tires of vehicle 100 and bottom 115.

Accordingly, for example, an icy road can be detected early during travel, and a more discreet travel mode can be selected; or the tire change service vehicle may be prompted to change tires to non-skid tires. If the estimated braking performance and/or friction is below a first threshold level, driving may be prohibited altogether. In some embodiments, the vehicle 100 may be allowed to travel at a limited speed, such as 50km/h, when the estimated braking performance and/or friction is above a first threshold level but below a second threshold level.

Step 410 comprises determining to enable/disable the driving of the vehicle 100 based on the results of the performed first action 401, the detection 402 of the triggered alarm, the performed second action 403 and/or the detection 404 of the disabled alarm.

Thus, in case the performed first action 401 does not result in that the triggering alarm 402 can be detected and/or disabled, it is determined that driving is prohibited.

In some embodiments where step 405 has been performed, the determination 410 to enable/disable travel may be further based on an obtained confirmation 405/failure to obtain a confirmation 405.

In some embodiments, where step 406 has been performed, it may be further determined 410 to enable/disable travel based on the result of the performed third action 406.

In some embodiments, where step 407 has been performed, it may be further determined 410 to enable/disable travel based on the results of the performed functional check 407.

In some embodiments, where step 408 has been performed, it may be further determined 410 to enable/disable travel based on the results of the performed functional check 408.

In some embodiments, where step 409 has been performed, the determination 410 to enable/disable travel may be further based on the estimated braking performance and friction 409.

Thus, an "error detection" is deliberately triggered by performing a first action, and then checking whether the alarm/error code has been confirmed; the process is then reversed by performing a second action for disabling the alarm/error code, ensuring that the vehicle 100 can continue in the autonomous mode with a so-called "safe start". Due to the actions performed it is ensured that the entire work chain for handling error codes is really functional. In the case where the driving permission cannot be given, the deviation may be handled differently according to the type of the deviation and the severity thereof.

Step 411, which may only be performed in some particular embodiments, may include, where it has been determined 410 that travel of vehicle 100 is prohibited, performing an activity or measure to correct any anomalies or causes that fail to confirm the action performed and to enable travel permission of vehicle 100.

The activity/action may be, for example, sending a signal to the communication device 200 of the owner or other responsible person in the traffic monitoring tower.

The action 411 performed may include outputting an alarm to inform the person in charge of the vehicle 100 that confirmation of the action performed was not obtained 405.

In various embodiments, information regarding error/anomaly detection/validation failures may be stored in a vehicle inspection file, which may be located in the on/off-board data storage device 220, 320 or any other convenient location.

Thus, vehicle inspection is facilitated since critical information may be saved and provided to the mechanic, while the mechanic may immediately understand and draw conclusions about what kind of problem occurred on the vehicle 100 due to the vehicle inspection documentation. After the vehicle check is successfully performed and any detected abnormality is corrected, the vehicle 100 may be given a driving permission.

Fig. 5 illustrates an embodiment of a system 500 for performing vehicle self-diagnostics in the vehicle 100. The system 500 may perform at least some of the method steps 401 and 411 previously described in accordance with the method 400 described above and illustrated in fig. 4A-B.

The system 500 includes at least one control device 310 that makes a vehicle self-diagnosis in the vehicle 100. The control device 310 is configured to generate a command for performing a first action of the vehicle 100 that is expected to trigger an alarm/error code. In addition, the control device 310 is configured to detect an alarm/error code triggered by the performed first action. Furthermore, the control device 310 is configured to generate a command for performing a second action of the vehicle 100 that is expected to disable the alert. The control device 310 is further configured to detect that an alarm triggered by the performed first action is disabled. The control device 310 is further configured to determine to allow/prohibit the travel of the vehicle 100 based on the results of the performed first action, the detection of the triggered alert, the performed second action, and/or the detection of the disabled alert.

In some embodiments, the control device 310 is configured to obtain a confirmation that the result of the performed action corresponds to the expected result. Also, the control device 310 is configured to determine permission/prohibition of travel of the vehicle 100 further based on the obtained confirmation/non-confirmation.

In some embodiments, the non-confirmation may be due to a physical vehicle part failure. Additionally, according to some embodiments, the non-confirmation may be due to a deviation from the expected conditions of the vehicle 100 under the current driving conditions. In some particular embodiments, the anomaly may include information deviations in the digital map and/or errors in the positioning device 340.

In some embodiments, control device 310 may be configured to perform actions in order to determine the cause of the non-confirmation, eliminate or at least reduce the effect of the non-confirmation, and/or detect additional anomalies of vehicle 100 resulting from the same cause that caused the non-confirmation. In addition, the control device 310 may also be configured to determine permission/prohibition of travel of the vehicle 100 based on a result of the executed third action.

Further, in some embodiments, the control device 310 may check the wireless communication function between the control device 310 of the vehicle 100 and the vehicle-external entity 200. The control device 310 may also be configured to determine to permit/prohibit the travel of the vehicle 100 based on the result of the executed function check.

The control device 310 may be configured to check the communication function between the control device 310 and the vehicle sensors 110,120,130, 140. Communication is typically via a wired or wireless bus. The control device 310 may be additionally configured to determine permission/prohibition of travel of the vehicle 100 based on the result of the executed function check.

In still other embodiments, the control device 310 may also be configured to estimate braking performance and/or friction between the tires of the vehicle 100 and the bottom 115. Further, the control device 310 may also be configured to determine to permit/prohibit travel of the vehicle 100 based on the estimated braking performance/friction.

The control device 310 may also be configured to execute measures for achieving the travel permission of the vehicle 100 when it has been determined that the travel of the vehicle 100 is prohibited.

Also, the control device 310 may be configured to, when the success of the performed action cannot be confirmed, output an alarm to inform the person in charge of the vehicle 100 of the lack of confirmation of the action.

The system 500 may include an actuator for performing the first action and/or the second action.

Additionally, the system 500 may also include sensors 110,120,130,140 of the vehicle 100 for capturing sensor data related to the vehicle 100.

In some embodiments, the system 500 may include a vehicle exterior sensor 230.

The system 500 may also include a data storage device 220, 320 for storing captured sensor data for the normal state of the vehicle 100.

Moreover, the system 500 may also include a vehicle-external entity 200 for wirelessly communicating with the control device 310.

The control apparatus 310 may include a receiving circuit 510 configured to receive signals from the sensors 110,120,130,140, 230 and/or the data storage device 320.

Additionally, according to some embodiments, the control device 310 may comprise processing circuitry 520 configured to perform at least some of the method steps 401-411 of the method 400 described above.

Such processing circuitry 520 may include one or more instances of processing circuitry, i.e., a Central Processing Unit (CPU), processing unit, processing circuit, Application Specific Integrated Circuit (ASIC), microprocessor, or other processing logic that may interpret and execute instructions. Thus, the expression "processing circuitry" as utilized herein may represent/include a plurality of processing circuitry, such as any, some, or all of those enumerated above.

Further, in some embodiments, control device 310 may include memory 525. Optional memory 525 may include a physical device for temporarily or permanently storing data or programs (i.e., sequences of instructions). According to some embodiments, memory 525 may comprise an integrated circuit comprising silicon-based transistors. In various embodiments, the memory 525 may include, for example, a memory card, a flash memory, a USB memory, a hard disk, or other similar volatile or non-volatile storage unit for storing data, such as a ROM (read only memory), a PROM (programmable read only memory), an EPROM (erasable PROM), an EEPROM (electrically erasable PROM), or the like.

Additionally, in some embodiments, the control device 310 may include a signal transmitter 530. Signal transmitter 530 may be configured to transmit a signal to, for example, display device 200, data storage device 320, and/or an actuator.

The above-described steps 401-411 to be performed in the vehicle 100 may be implemented by one or more processing circuits 520 within the control device 310 together with a computer program product for performing at least some of the functions of the steps 401-411. Thus, a computer program product comprising instructions for performing steps 401-.

Additionally, some embodiments of the invention may include a vehicle 100 including a control device 310 for vehicle self-diagnostics, in accordance with at least some of method steps 401-411.

The computer program product mentioned above may be provided, for example, in the form of a data carrier carrying computer program code for performing at least some of the steps 401 and 411 according to some embodiments when being loaded into the one or more processing circuits 520 of the control device 310. The data carrier may be, for example, a hard disk, a CD ROM disk, a memory stick, an optical storage device, a magnetic storage device, or any other suitable medium such as a disk or tape that can store machine-readable data in a non-transitory manner. Furthermore, the computer program product may be provided as computer program code on a server and downloaded to the controlling device 310 remotely, e.g. over an internet or an intranet connection.

As shown in the figures, the terminology used in the description of the embodiments is not intended to limit the described method 400, control device 310, computer program, system 500, and/or vehicle 100. Various changes, substitutions and/or alterations may be made herein without departing from the embodiments of the invention as defined by the appended claims.

As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items. As used herein, unless explicitly stated otherwise, the term "OR" should be interpreted as a mathematical OR, i.e., as an inclusive disjunction, and not as a mathematical exclusive OR (XOR). In addition, the singular forms "a", "an" and "the" should be construed as "at least one" and thus may include a plurality of the same kind of entities unless expressly stated otherwise. It will be further understood that the terms "comprises," "comprising," "includes" and/or "including," when used in this specification, specify the presence of stated features, actions, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, actions, integers, steps, operations, elements, components, and/or groups thereof. A single unit, such as a processing circuit, may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. A computer program may be stored/distributed on a suitable medium, such as an optical storage medium or a solid-state medium supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the internet or other wired or wireless telecommunication systems.