阅读说明：本技术 一种基于车联网大数据的车辆油耗统计方法 (Vehicle oil consumption statistical method based on Internet of vehicles big data ) 是由 李皓 徐国强 孔伟伟 邱梅芳 樊佳明 于 2021-01-22 设计创作，主要内容包括：本发明属于汽车制造技术领域,涉及一种基于车联网大数据的车辆油耗统计方法,包括如下步骤：步骤一：通过车载终端获取车辆的油耗累计使用量、数据发生时间,记为第一数据；步骤二：根据预设阈值对第一数据进行过滤,过滤异常数据后的数据,记为第二数据；步骤三：将统计结束时间的油耗累计使用量减去统计开始时间的油耗累计使用量得到车辆油耗结果数据,记为第三数据；步骤四：将第三数据写入数据库,供业务系统调用查询使用。本发明解决了现有车辆油耗统计方法中车载终端会上传海量的工业数据,其中包含了大量的无效、错误数据而导致的计算车辆油耗误差大,计算速度慢的问题。(The invention belongs to the technical field of automobile manufacturing, and relates to a vehicle oil consumption statistical method based on Internet of vehicles big data, which comprises the following steps: the method comprises the following steps: acquiring the accumulated usage amount of oil consumption and the data occurrence time of a vehicle through a vehicle-mounted terminal, and recording as first data; step two: filtering the first data according to a preset threshold, and recording the data after abnormal data filtering as second data; step three: subtracting the oil consumption accumulated usage amount at the starting time from the oil consumption accumulated usage amount at the ending time of the statistics to obtain vehicle oil consumption result data, and recording the vehicle oil consumption result data as third data; step four: and writing the third data into a database for the service system to call and query. The invention solves the problems of large error and low calculation speed of vehicle oil consumption calculation caused by the fact that a vehicle-mounted terminal uploads massive industrial data which contains a large amount of invalid and wrong data in the conventional vehicle oil consumption statistical method.)

1. A vehicle oil consumption statistical method based on Internet of vehicles big data is characterized by comprising the following steps:

the method comprises the following steps: acquiring the accumulated usage amount of oil consumption and the data occurrence time of a vehicle through a vehicle-mounted terminal, and recording as first data;

step two: filtering the first data according to a preset threshold, and recording the data after abnormal data filtering as second data;

step three: on the basis of the second data, subtracting the oil consumption accumulated usage amount at the starting time from the oil consumption accumulated usage amount at the ending time of the statistics to obtain vehicle oil consumption result data, and recording the vehicle oil consumption result data as third data;

step four: and writing the third data into a database for the service system to call and query.

2. The vehicle oil consumption statistical method based on the internet of vehicles big data as claimed in claim 1, wherein the first data in the first step is stored according to a built-up table by using an Hbase database, wherein the Rowkey design mode is vehicle-mounted terminal ID + data generation time + data type.

3. The vehicle oil consumption statistical method based on the internet of vehicles big data according to claim 1, wherein the filtering manner in the second step comprises: and (4) using sequencing filtration, filtering the median and the accumulated oil consumption, and filtering the change of the data occurrence time in a credible interval.

4. The vehicle oil consumption statistical method based on the internet of vehicles big data as claimed in claim 3, wherein the filtering process in the second step comprises the following steps:

A. acquiring the vehicle oil consumption accumulated usage in the first data, and recording as an OilList set;

B. taking out the median in the OilList set, marking the median as credible oil consumption accumulated usage data, and marking the data as media oil;

C. sorting according to the data occurrence time, recording as a timeList set, acquiring the corresponding data occurrence time according to the mediaoil, and recording as a mediatime;

D. if the position of the mediatime in the timeList set exceeds a preset range, removing mediaOil and mediatime from the set;

E. and repeating the step A to the step D until the credible accumulated oil consumption amount crediOil and the credible data occurrence time crediTime are obtained.

5. The vehicle oil consumption statistical method based on the internet of vehicles big data according to claim 1, characterized in that the third data in the third step is used for calculating the oil consumption data of the vehicle in the statistical time range in an off-line batch processing mode; and processing the non-whole-day oil consumption data of the vehicle in an online real-time calculation mode, and accumulating the whole-day oil consumption data of the vehicle and the non-whole-day oil consumption data of the vehicle to calculate the oil consumption data of the vehicle.

6. The vehicle fuel consumption statistical method based on the internet of vehicles big data as claimed in claim 5, wherein the offline batch processing mode comprises using Map-Reduce computing framework, and the online real-time computing mode comprises using Flink computing framework.

7. The vehicle fuel consumption statistical method based on the internet of vehicles big data as claimed in claim 1, wherein the database written by the third data in step four comprises an Oracle database.

Technical Field

The invention belongs to the technical field of automobile manufacturing, and relates to a vehicle oil consumption statistical method based on Internet of vehicles big data.

Background

In enterprise fleet management and enterprise logistics transportation, abnormal oil loss caused by various abnormal reasons such as oil stealing and selling by personnel in enterprises causes endless oil cost, and the serious abnormal oil loss needs strict time and interval statistics of the enterprises to eliminate the abnormal oil loss. The conventional vehicle oil consumption statistical method is used for counting vehicle oil consumption data uploaded by a vehicle terminal, however, in the driving process of a vehicle, the vehicle terminal uploads massive data, which contains a large amount of invalid and wrong data for vehicle oil consumption statistics. Therefore, a significant error is caused by collecting the oil consumption data uploaded by the vehicle terminal to calculate the vehicle oil consumption in a statistical manner; meanwhile, in the process of processing massive data, the processing speed of the program is a key technical index influencing user experience, and a large amount of processing data causes additional burden on a processor. How to rapidly acquire vehicle oil consumption data with high accuracy on the premise of eliminating redundant, invalid and wrong data is always a difficult problem of enterprise vehicle oil consumption statistics.

Disclosure of Invention

The technical scheme adopted by the invention for solving the technical problem is as follows: a vehicle oil consumption statistical method based on Internet of vehicles big data comprises the following steps: the method comprises the following steps: acquiring the accumulated usage amount of oil consumption and the data occurrence time of a vehicle through a vehicle-mounted terminal, and recording as first data; step two: filtering the first data according to a preset threshold, and recording the data after abnormal data filtering as second data; step three: on the basis of the second data, subtracting the oil consumption accumulated usage amount at the starting time from the oil consumption accumulated usage amount at the ending time of the statistics to obtain vehicle oil consumption result data, and recording the vehicle oil consumption result data as third data; step four: and writing the third data into a database for the service system to call and query.

Preferably, the first data in the first step is stored according to a daily table by using an Hbase database, wherein the Rowkey design mode is vehicle-mounted terminal ID + data generation time + data type.

Preferably, the filtration mode in the second step includes: and (4) using sequencing filtration, filtering the median and the accumulated oil consumption, and filtering the change of the data occurrence time in a credible interval.

Preferably, the filtering process in the second step includes the following steps: A. acquiring the vehicle oil consumption accumulated usage in the first data, and recording as an OilList set; B. taking out the median in the OilList set, marking the median as credible oil consumption accumulated usage data, and marking the data as media oil; C. sorting according to the data occurrence time, recording as a timeList set, acquiring the corresponding data occurrence time according to the mediaoil, and recording as a mediatime; D. if the position of the mediatime in the timeList set exceeds a preset range, removing mediaOil and mediatime from the set; E. and repeating the step A to the step D until the credible accumulated oil consumption amount crediOil and the credible data occurrence time crediTime are obtained.

Preferably, the third data in the third step is data of fuel consumption of the vehicle in the statistical time range in a whole day calculated in an off-line batch processing mode; and processing the non-whole-day oil consumption data of the vehicle in an online real-time calculation mode, and accumulating the whole-day oil consumption data of the vehicle and the non-whole-day oil consumption data of the vehicle to calculate the oil consumption data of the vehicle.

Preferably, the offline batch processing mode comprises using a Map-Reduce computing framework, and the online real-time computing mode comprises using a Flink computing framework.

Preferably, the database written by the third data in the fourth step includes an Oracle database. The invention has the beneficial effects that:

1. the method and the device have the advantages that the vehicle oil consumption data acquired by the vehicle-mounted terminal are filtered for many times by using various filtering modes such as sequencing filtering, median and oil consumption accumulated usage filtering, data generation time variation filtering in a credible interval and the like, redundant, invalid and wrong non-oil consumption data are removed, and then data processing is carried out, so that the method and the device have the advantages of small data processing amount and small processor load compared with the conventional vehicle oil consumption statistical method, and the oil consumption statistical speed is obviously improved.

2. The method comprises the steps of carrying out off-line batch processing on vehicle whole-day oil consumption data by using a Map-Reduce calculation framework, carrying out on-line real-time calculation on vehicle non-whole-day oil consumption data by using a Flink calculation framework, and then accumulating the vehicle whole-day oil consumption data and the non-whole-day oil consumption data to calculate the vehicle oil consumption data; the Map-Reduce calculation framework is suitable for large-scale data set parallel operation and is suitable for multi-day vehicle whole-day oil consumption data; the Flink calculation framework is suitable for the open source process processing operation and is suitable for non-whole-day oil consumption data of a single-day open source vehicle, so that the processing speed is higher, and the oil consumption statistical processing performance is higher.

Drawings

FIG. 1 is a step diagram of a vehicle fuel consumption statistical method based on Internet of vehicles big data;

FIG. 2 is a filtration flow diagram;

fig. 3 is a flow chart of vehicle fuel consumption data classification and accumulation.

Detailed Description

The related art in the present invention will be described clearly and completely with reference to the accompanying drawings in the following embodiments, and it is to be understood that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

Referring to fig. 1 to 3, a vehicle oil consumption statistical method based on internet of vehicles big data includes the following steps: the method comprises the following steps: acquiring the accumulated usage amount of oil consumption and the data occurrence time of a vehicle through a vehicle-mounted terminal, and recording as first data; step two: filtering the first data according to a preset threshold, and recording the data after abnormal data filtering as second data; step three: on the basis of the second data, subtracting the oil consumption accumulated usage amount at the starting time from the oil consumption accumulated usage amount at the ending time of the statistics to obtain vehicle oil consumption result data, and recording the vehicle oil consumption result data as third data; step four: and writing the third data into a database for the service system to call and query.

Further, the first data in the first step are stored according to a built-up table by using an Hbase database, wherein the Rowkey design mode is vehicle-mounted terminal ID + data generation time + data type.

Further, the filtering method in the second step includes: the method comprises the steps of using sorting filtering, median and oil consumption accumulated usage filtering, and filtering the change of data occurrence time in a credible interval, wherein abnormal data with repeated time points can be filtered out through the sorting filtering, and abnormal data with overlarge or undersize oil consumption can be filtered out through the median and the oil consumption accumulated usage filtering.

Further, the filtering process in the second step includes the following steps: A. acquiring the vehicle oil consumption accumulated usage in the first data, and recording as an OilList set; B. taking out the median in the OilList set, marking the median as credible oil consumption accumulated usage data, and marking the data as media oil; C. sorting according to the data occurrence time, recording as a timeList set, acquiring the corresponding data occurrence time according to the mediaoil, and recording as a mediatime; D. if the position of the mediatime in the timeList set exceeds a preset range, removing mediaOil and mediatime from the set; E. and repeating the step A to the step D until the credible accumulated oil consumption amount crediOil and the credible data occurrence time crediTime are obtained.

Further, the third data in the third step adopts an off-line batch processing mode to calculate the whole day oil consumption data of the vehicle within the statistical time range; and processing the non-whole-day oil consumption data of the vehicle in an online real-time calculation mode, and accumulating the whole-day oil consumption data of the vehicle and the non-whole-day oil consumption data of the vehicle to calculate the oil consumption data of the vehicle.

Furthermore, the offline batch processing mode comprises using a Map-Reduce computing framework, and the online real-time computing mode comprises using a Flink computing framework.

Further, the database written by the third data in the fourth step includes an Oracle database.

Examples

In the embodiment, the vehicle oil consumption accumulated usage amount and the data generation time are obtained through the vehicle-mounted terminal and recorded as first data; the first data are stored in an Hbase database according to a daily table, and the Rowkey design mode is vehicle-mounted terminal ID + data generation time + data type; the characteristic of high concurrency and real-time data processing of HBase is utilized, so that the off-line and on-line calculation of the vehicle oil consumption is facilitated.

Filtering the first data according to a preset threshold, and recording the data after abnormal data filtering as second data; and (4) filtering abnormal data by using logic such as sequencing, median and oil consumption accumulated usage, data occurrence time change in a credible interval and the like.

The specific filtering process comprises the following steps of obtaining the vehicle oil consumption accumulated usage in the first data, and recording the vehicle oil consumption accumulated usage as an OilList set; taking out the median in the OilList set, marking the median as credible oil consumption accumulated usage data, and marking the data as media oil; sorting according to the data occurrence time, recording as a timeList set, acquiring the corresponding data occurrence time according to the mediaoil, and recording as a mediatime; if the position of the mediatime in the timeList set exceeds a preset range (namely, the subscript of the mediatime is greater than (timeList. size ()/2+ threshold value) or the subscript of the mediatime is less than (timeList. size ()/2-threshold value)), removing mediaoil and mediatime from the set; and repeating the steps until the credible accumulated oil consumption crediOil and the credible data occurrence time crediTime are obtained.

According to the credible data occurrence time crediTime, sequentially traversing the crediTime to the data of the accumulated usage of the oil consumption within the starting time of statistics, and if (mediaoil-current accumulated usage of the oil consumption)/(crediTime-current time) is greater than the threshold value of the change of the oil consumption within unit time, considering the data as an abnormal value and removing the abnormal value; similarly, the accumulated usage data of the oil consumption from the crediTime to the counting end time is sequentially traversed, and relevant abnormal data are removed.

In this embodiment, the first data is a set of unfiltered accumulated usage of fuel consumption and data occurrence time { (1,12:01), (3,12:01), (1,12:02), (2,12:03), (2,12:04), (50,12:06), (4,12:07), (4,12:08), (5,12:09), (4,12:10) }; using the data as an oilList set, searching a median mediaoil, firstly finding a mediaoil which is 3 and a corresponding time mediatime which is 12:01, wherein the mediatime exceeds a preset range, and removing (3,12: 01); continuing to search the median mediaoil until the credible accumulated usage of oil consumption and credible data occurrence time (2,12:04) are obtained; when the group of data is traversed according to credible crediOil and crediTime in sequence, two groups of abnormal data are removed (50,12:06) and (4,12: 10).

And on the basis of the second data, subtracting the oil consumption accumulated usage amount at the starting time from the oil consumption accumulated usage amount at the ending time of the statistics to obtain the vehicle oil consumption statistical data, and recording the vehicle oil consumption statistical data as third data. In this embodiment, the second data is data obtained by removing the abnormal data from the first data in the above embodiment, and the specific data is as follows: { (1,12:01), (1,12:02), (2,12:03), (2,12:04), (4,12:07), (4,12:08), (5,12:09) }, the vehicle fuel consumption is 4 when counted in a statistical time range of 12:01 to 12: 10.

The statistical and accumulation process of the third data is as follows: performing offline batch processing calculation by using a Map-Reduce calculation framework, and performing online real-time calculation by using a Flink calculation framework; the vehicle oil consumption data of the whole day in the statistical time range is obtained from the vehicle oil consumption data calculated off line, and the vehicle oil consumption data of the whole day is obtained through on-line real-time calculation; and accumulating the acquired offline calculated vehicle oil consumption data and the real-time calculated vehicle oil consumption data to serve as third data.

And finally, writing the third data into a database for the service system to call and query. Therefore, statistics of vehicle oil consumption based on the big data of the Internet of vehicles is achieved.

In summary, the invention provides a vehicle oil consumption statistical method based on internet of vehicles big data, which comprises the steps of filtering oil consumption data of a vehicle obtained by a vehicle-mounted terminal for multiple times by using various filtering modes such as sequencing filtering, median and oil consumption accumulated usage filtering, data occurrence time variation filtering in a credible interval and the like, eliminating redundant, invalid and wrong non-oil consumption data, then processing the data, processing the non-whole-day oil consumption data of the vehicle in an online real-time calculation mode by using a Flink calculation frame, accumulating the whole-day oil consumption data of the vehicle and the non-whole-day oil consumption data of the vehicle to calculate the oil consumption data of the vehicle, solving the problem that the vehicle-mounted terminal in the existing vehicle oil consumption statistical method can upload massive industrial data, wherein the large amount of invalid and wrong data cause large errors in calculating the oil consumption of the vehicle, the invention has the problem of low calculation speed, so the invention has wide application prospect.

It is to be emphasized that: the above embodiments are only preferred embodiments of the present invention, and are not intended to limit the present invention in any way, and all simple modifications, equivalent changes and modifications made to the above embodiments according to the technical spirit of the present invention are within the scope of the technical solution of the present invention.