阅读说明：本技术 设备间的数据变换和同步 (Data transformation and synchronization between devices ) 是由 S·K·阿加瓦尔 J·丹吉 于 2020-03-30 设计创作，主要内容包括：一种计算机实现的方法包括：在边缘服务器处从多个设备接收数据；对从所述多个设备接收的所述数据进行变换；将所变换的数据存储在边缘服务器数据库中；对所述数据库中的所变换的数据执行分析；以及选择性地将数据经由网络上传到远程服务器。(A computer-implemented method comprising: receiving data from a plurality of devices at an edge server; transforming the data received from the plurality of devices; storing the transformed data in an edge server database; performing an analysis on the transformed data in the database; and selectively uploading the data to a remote server via a network.)

1. A computer-implemented method, comprising:

receiving data from a plurality of devices at an edge server;

transforming the data received from the plurality of devices;

storing the transformed data in an edge server database;

performing an analysis on the transformed data in the database; and

data is selectively uploaded to a remote server via a network.

2. The method of claim 1, wherein the data received from each of the plurality of devices is received periodically from a plurality of wireless devices.

3. The method of claim 1, wherein transforming the data received from each of the plurality of devices comprises: data received during a specified time window is buffered and transformed.

4. The method of claim 3, wherein the specified time window is device dependent.

5. The method of any of claims 1-4, further comprising performing a function on the data stored in the edge server database.

6. The method of claim 5, wherein the function comprises executing a machine learning model to perform scoring, wherein the machine learning model is provided from the remote server.

7. The method of claim 6, further comprising:

determining an action based on the output of the function performed;

generating a control signal based on the determined action; and

transmitting the control signal to one or more of the devices for causing the action to be performed.

8. The method of claim 5, wherein the function comprises one or more of: relational database functions, binary classification, linear regression, query execution, and aggregation.

9. The method of any of claims 1-4, wherein selectively uploading data to the remote server via the network comprises: data is uploaded in response to an on-demand request from an application.

10. A machine-readable storage device having instructions for execution by a processor of a machine to cause the processor to perform operations to perform a method, the operations comprising:

receiving data from a plurality of devices at an edge server;

transforming the data received from the plurality of devices;

storing the transformed data in an edge server database;

performing an analysis on the transformed data in the database; and

data is selectively uploaded to a remote server via a network.

11. The device of claim 10, wherein the data received from each of the plurality of devices is periodically received from a plurality of wireless devices, and wherein transforming the data received from each of the plurality of wireless devices comprises: data received during a specified time window is buffered and transformed.

12. The device of any of claims 10-11, wherein the operations further comprise:

performing a function on the data stored in the edge server database;

determining an action based on the output of the function performed;

generating a control signal based on the determined action; and

transmitting the control signal to one or more of the devices for causing the action to be performed.

13. The apparatus of claim 12, wherein the function comprises one or more of: relational database functions, binary classification, linear regression, query execution, and aggregation.

14. An apparatus, comprising:

a processor; and

a memory device coupled to the processor and having a program stored thereon for execution by the processor to perform operations comprising:

receiving data from a plurality of devices at an edge server;

transforming the data received from the plurality of devices;

storing the transformed data in an edge server database;

performing an analysis on the transformed data in the database; and

data is selectively uploaded to a remote server via a network.

15. The device of claim 14, wherein the data received from each of the wireless devices is received periodically from the plurality of wireless devices, and wherein transforming the data received from each of the plurality of wireless devices comprises: buffering and transforming data received during a specified time window, and wherein the operations further comprise:

performing a function on the data stored in the edge server database;

determining an action based on the output of the function performed;

generating a control signal based on the determined action; and

transmitting the control signal to one or more of the wireless devices for causing the action to be performed.

Background

Internet of things (IoT) devices are expected to generate large amounts of data. The number of IoT devices in use is expected to exceed hundreds of millions. Such devices may be battery powered. The battery-powered limitation means that many, if not most, IoT devices will only have short-range wireless communication capabilities.

The cloud computing resources will likely be used to process data generated by a large number of IoT devices. Networks used to transfer data to cloud computing resources may become burdened with massive amounts of data and may not receive and process the data in a timely manner.

Disclosure of Invention

A computer-implemented method includes receiving data from a plurality of devices at an edge server, transforming the data received from the plurality of devices, storing the transformed data in an edge server database, performing analysis on the transformed data in the database, and selectively uploading the data to a remote server via a network.

Drawings

FIG. 1 is a block diagram of a system for processing device data at a network edge according to an example embodiment.

FIG. 2 is a flow diagram illustrating a computer-implemented method of processing data from a device coupled to an edge server, according to an example embodiment.

Fig. 3 is a block diagram of a system that collects streaming data for an application corresponding to a selected scene, according to an example embodiment.

FIG. 4 is a schematic block diagram of a computer system implementing one or more example embodiments.

Detailed Description

In the following description, reference is made to the accompanying drawings which form a part hereof, and in which is shown by way of illustration specific embodiments which may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention, and it is to be understood that other embodiments may be utilized and that structural, logical and electrical changes may be made without departing from the scope of the present invention. The following description of example embodiments is, therefore, not to be taken in a limiting sense, and the scope of the present invention is defined by the appended claims.

In one embodiment, the functions or algorithms described herein may be implemented in software. Software may be comprised of computer executable instructions stored on a computer readable medium or computer readable storage device, such as one or more non-transitory memories or other types of hardware-based storage devices (local or networked). Further, these functions correspond to modules, which may be software, hardware, firmware, or any combination thereof. Various functions may be performed in one or more modules as desired, and the described embodiments are merely examples. The software may be executed on a digital signal processor, ASIC, microprocessor, or other type of processor running on a computer system, such as a personal computer, server, or other computer system, turning such computer system into a specially programmed machine.

The functions may be configured to perform operations using, for example, software, hardware, firmware, and so forth. For example, the phrase "configured to" may refer to a logical circuit arrangement of hardware elements that is to implement the associated functionality. The phrase "configured to" may also refer to a logical circuit configuration of hardware elements, a coded design for implementing the associated functions of firmware or software. The term "module" refers to a structural element that may be implemented using any suitable hardware (e.g., processor, etc.), software (e.g., application, etc.), firmware, or any combination of hardware, software, and firmware. The term "logic" encompasses any functionality for performing a task. For example, each operation illustrated in the flowcharts corresponds to logic to perform the operation. Operations may be performed using software, hardware, firmware, or the like. The terms "component," "system," and the like can refer to a computer-related entity, hardware, and software in execution, firmware, or a combination thereof. A component may be a process running on a processor, an object, an executable, a program, a function, a subroutine, a computer, or a combination of software and hardware. The term "processor" may refer to a hardware component, such as a processing unit of a computer system.

Furthermore, the claimed subject matter may be implemented as a method, apparatus, or article of manufacture using standard programming and engineering techniques to produce software, firmware, hardware, or any combination thereof to control a computing device to implement the disclosed subject matter. The term "article of manufacture" as used herein is intended to encompass a computer program accessible from any computer-readable storage device or media. The computer-readable storage medium may include, but is not limited to, magnetic storage devices such as hard disks, floppy disks, magnetic strips, optical disks, Compact Disks (CDs), Digital Versatile Disks (DVDs), smart cards, flash memory devices, and the like. In contrast, computer-readable media, i.e., non-storage media, may additionally include communication media such as transmission media for wireless signals and the like.

IoT and edge computing are characterized by the vast amount of data generated by the millions of IoT sensors deployed. Depending on connectivity and bandwidth limitations, these large amounts of data cannot all be migrated into the cloud for storage and analysis, and therefore, there is a need to be able to store, process, aggregate, filter, and predict/analyze these data sets before migrating the aggregated/filtered cloud.

Given the volume of data and logistical constraints, it has become extremely important to organize deployment data solutions that provide native capabilities to store, process, analyze, filter, and then move this data between the edge and public/private clouds. None of the existing participants in the edge data space provide this built-in capability.

Fig. 1 is a block diagram of a system 100 for such a data solution. The system 100 includes a number of data generating devices, such as IoT sensors 110 that generate large amounts of data. The sensors 110 may also include wireless devices that function as sensors or even as actuators that respond to control signals and perform actions. In various embodiments, the number of sensors 110 may vary from one to hundreds, thousands, or more, thereby generating large amounts of data. The sensor may be battery powered and include a transceiver for wirelessly transmitting data. Some sensors may sense and transmit data at a high rate, such as multiple times per second, while other sensors may transmit once every few seconds or minutes or more to conserve battery power.

As shown at 115, the data is transmitted to an edge device, such as edge server 120, which includes programming for implementing database functionality, such as SQL, which stands for structured query language. The edge server 120 is a computing device running software having the capability to receive and process data with very low latency delay. Examples of edge servers 120 include servers within an organization, such as servers within a data center, that can quickly receive data with a very short time or delay between the sending of the data and the receipt of the data by the edge servers. The edge server 120 is configured with software to ingest large amounts of data from sensors (such as hundreds or thousands of sensors that transmit data per second or other time period) as well as from other running devices.

In some embodiments, the edge server 120 may communicate with a remote server 125 coupled via a network 130. The remote server may include cloud resources, such as SQL servers, data lake storage, and other services. The network 130 may be a public network, such as the internet, or a private network. Network 130 may include a number of nodes including routers, switches, hubs, and other devices designed to receive and route data, such as data packets, and possibly perform functions on the data during transmission. Thus, there may be significant delays in the network before the data even reaches the destination. Edge server 120 is a server located between sensor 110 and network 130. The edge server 120 may be coupled to receive data directly from the sensors 110 or from one or more other devices that receive data from the sensors 110. The edge server 120 may be coupled near the edge of the network 130 and may also be referred to as a gateway so that data from the sensors 110 quickly reaches the edge server 120 without the delay associated with sending the data to other devices via the network.

In one embodiment, the edge server 120 includes a windowing and transformation component 135 that performs aggregation of data or other transformation of data. The windowing and transformation component 135 may operate on the data as it is received, or may receive a series of data during one or more time-based windows associated with each sensor 110. The windowing and transformation component 135 can perform functions on data received from sensors during a time window associated with the sensors. Such functions may include buffering received data and transforming the data during a time window. Transforming the data may include performing windowed aggregation and statistical analysis, such as mean, outlier identification, derivative, standard deviation, and other types of data transformations, which may vary depending on the environment/application sensed by the sensor. Further transformations may include aggregation of data, transforming data between rows and columns, grouping by aggregation, connecting data from different sensors, and other functions performed on an ad-hoc basis. In this context, ad-hoc refers to the performance of data received and buffered during a corresponding time window of a sensor, rather than storing and retrieving data outside such time window.

The transformed data is provided to a database engine 140, such as an SQL (structured query language) database engine, where the transformed data may be stored, indexed, and available in a relational database 142. The data in the database 142 may access the capabilities of other functions 145 for execution on the engine 140. Functions 145 may include various analyses on data stored in database 142, such as many different relational database functions, binary classification, linear regression, SQL execution, aggregation, relational queries, graph-based queries, or other types of analyses/functions that may be performed on data stored in a relational database format.

One such function includes one or more machine learning models. The machine learning model in one embodiment may be trained via remote server 125 based on data provided from edge server 120. The data provided from the edge server 120 may include the sensor data 115 and analysis performed on the raw sensor data 115 that may also be stored in the database 142. Several different applications, referred to as scenarios, for which models can be generated to make predictions associated with the applications, are described below. Since model training can be a resource intensive task, moving model training to a remote server can ensure that the local resources of the edge server are not limited and can be applied to collection and local analysis as data is collected. In one embodiment, the Azure ML (machine learning) model is provided via remote server 125. Various functions written in different programming languages may also be performed, including programs written in Python, Spark, Java, and other languages.

The remote server 125, in addition to receiving data from the edge server 120, may also provide device synchronization by providing data and deploying software packages to the edge server 120. The business rules or security policies may be received from a remote server for implementation on the edge server 120.

The edge server 120 offloads the transformed and aggregated data (data from the sensors) to the cloud/enterprise data center, remote server 125. Note that the edge server 120 is described in terms of software that can be executed on a processing resource. In addition, the edge server 120 provides the customer with the ability to configure custom retention policies for data on the edge before offloading the data to the cloud or private data center.

In view of the large amount of data generated at the edge of the network, the edge server 120 performs transformations on the generated data, such as aggregating the data streams 115 flowing into the edge server 120. Most internet of things sensors generate data at fixed time intervals, with each data point having a time attribute attached to it. Thus, the edge server 120, via the windowing and transformation component 135, can perform windowing and time series specific processing on the edges.

In addition, the edge server 120 has the ability to execute ML within the database using native T-SQL functionality or using the model 145 or algorithms written in R, Python, Spark, or Java. Customers will be able to train their models in the cloud, remote server 125 using cloud-based scalable resources and analytics, and push the trained models to the edge server 120 for scoring via the models 145.

The edge server 120 provides the above capabilities through native integration of analytics functions, such as Azure flow analysis, without the need for external solutions to synchronize data between the edge and the public/private cloud. The edge server 120 is also scalable in that more processing resources can be added to perform functions that are performed with increasing amounts of data received from more and more sensors.

Fig. 2 is a flow diagram illustrating a computer-implemented method 200 of processing data from a device 110 coupled to an edge server 120. The method 200 begins with a plurality of devices, such as IoT devices, generating data and transmitting it such that it is received by an edge server in operation 210. Data may be received from each of the plurality of devices periodically. The time period between the received values may be the same or different for different devices, e.g. sensors. The same type of sensor, such as a temperature sensor, may have the same sensing and transmission period or frequency. Each value of the data representing the sensed parameter may be time stamped. The edge server may wirelessly receive data from a plurality of devices without sending the data to the edge server via a public network.

In operation 220, the edge server transforms data received from each of the plurality of devices. The transformation may include aggregating or otherwise combining the data via one or more statistical processes or other analysis. In some embodiments, data received during a specified time window, such as a device-specific window, may be transformed. The time windows may be the same length for multiple or all devices, or the length of the time windows may be different for each device or device type. The statistical or mathematical computation may be one of the transformations of the data received during each device-specific window.

The transformed data is stored by the edge server into an edge server database in operation 230. The data so stored may include the transformed data.

At operation 240, the method 200 may also include performing various functions described above with respect to the analysis function 145. The results may also be stored in database 142.

At operation 255, an action may be determined based on the result of the performed function. The action may be a function of implementing a particular scenario of the system 100. Example scenarios are described below and may include generating an alarm and/or controlling various functions based on parameters sensed by the sensors 110. At operation 260, a control signal may be generated based on the function, and the control signal may be transmitted to one or more of the wireless devices at 265 for causing actions associated with the plurality of wireless devices to be performed. At operation 270, data and/or package updates are received at the edge server from the remote server.

Fig. 3 is a block diagram of a system 300 that collects streaming data for an application 310 corresponding to a selected scene. Like components in fig. 1 and 3 are numbered the same. Real-time sensor data 115 is streamed from the plurality of sensors 110 to the edge server 120. The edge server 120 includes a windowing and transformation component 135 to transform streaming data, and also includes an engine 140 and functionality 145. Network 130 provides data from database 142 to cloud resource 325, which may be the same as remote server 125 or include remote server 125.

The application 310 may include a dashboard or other user interface for one or more scenes. The application 310 may upload data from the edge server 120 to the cloud resources 325 via a user interface trigger based on user-defined logic, or such upload may be performed periodically. The uploaded data may be selected from database 142 or from output provided by function 145, or a combination thereof in different embodiments. By performing function 145 on the data prior to uploading, local control actions may be taken in a timely manner, and the amount of data uploaded may be significantly less than the amount of data received from sensors and other devices, resulting in less network bandwidth being consumed.

The following is an example use case scenario for local scoring on the edge server 120 and the application 310. There are many more scenarios that can be implemented directly with system 100 or system 300. Each scenario will use a model or other analysis as part of the function 145 trained by historical data associated with the scenario.

Agricultural farm-an agricultural farm is characterized by the presence of a large number of sensors (temperature, humidity, moisture, etc.), each of which produces a large amount of data every few seconds. All of this data generated from these sensors is used to predict several factors, such as fertilizer demand, crop spray plan, irrigation demand, via a model or other analysis provided by the functions 145 specific to this scenario. Uploading these data sets to Azure or other public/private clouds may be counterproductive in view of the large amount of data generated by these sensors and the geographic location of these farms. The data set is typically aggregated based on a time window (e.g., average 1 minute, average 5 minutes, average 15 minutes, etc.) before uploading to the cloud. In such a scenario, the edge server 120 may be located in a farm, such as in a WiFi router or local area network device.

Networked vehicles-networked vehicles are another example of a scenario where large amounts of data are generated at the edge. Large amounts of data need to be aggregated, processed, and then uploaded to cloud storage for further analysis through scalable computing resources. Furthermore, in the case of networked vehicles, scoring data on the edges via an ML model running on the edge server 120 is important to provide real-time feedback to facilitate scenarios of dynamic routing, vehicle wear management, perishable item temperature control, and the like. In such a scenario, the edge server 120 may be located in each car, or possibly in a cell tower in direct communication with the car. However, switching between towers may add complexity, making it more likely that the edge server 120 will be located with the vehicle.

Oil and gas drilling platforms-those located in the middle of the ocean or in remote geographic locations are characterized by the need to provide low-latency real-time analysis of data on the edges. These drilling platforms are generally time sensitive in nature and in most cases cannot withstand the delay of pushing data to and performing analysis on the cloud. The data (primarily the last few minutes or hours of data) needs to be analyzed quickly to drive actions/results on the rig. For example, real-time analysis and correlation of temperature, pressure and volume data is required to control the pressure on the valves or the overall cooling of the drilling platform. By moving the analytics and ML models to the edge server 120, data can be analyzed and correlated in real-time for such control. The edge server 120 in this scenario may be processing resources local to the rig.

Security and monitoring-the security and monitoring industry is another example where real-time analysis of data on edges is critical to ensure that the system can identify fraudulent attempts to circumvent security.

Fig. 4 is a schematic block diagram of a computer system 400 for implementing edge devices, servers, IoT devices, and other devices to perform methods and algorithms in accordance with an example embodiment. Not all components need be used in various embodiments.

An example computing device in the form of a computer 400 may include a processing unit 402, memory 403, removable storage 410, and non-removable storage 412. Although the example computing device is illustrated and described as computer 400, in different embodiments, the computing device may be in different forms. For example, the computing device may alternatively be a smartphone, tablet, smart watch, Smart Storage Device (SSD), or other computing device that includes the same or similar elements as shown and described with respect to fig. 4. Devices such as smartphones, tablets, and smartwatches are commonly referred to collectively as mobile devices or user equipment.

Although various data storage elements are shown as part of computer 400, storage may also or alternatively include cloud-based storage accessible via a network, such as internet or server-based storage. Note also that the SSD may include a processor on which the parser may run, allowing the parsed, filtered data to be transferred over the I/O channel between the SSD and the main memory.

Memory 403 may include volatile memory 414 and non-volatile memory 408. Computer 400 may include a variety of computer-readable media-or have access to a computing environment that includes a variety of computer-readable media-such as volatile memory 414 and non-volatile memory 408, removable memory 410, and non-removable memory 412. Computer memory includes Random Access Memory (RAM), Read Only Memory (ROM), Erasable Programmable Read Only Memory (EPROM) or Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, Compact Disc Read Only Memory (CDROM), Digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium capable of storing computer-readable instructions.

Computer 400 may include or have access to a computing environment that includes input interface 406, output interface 404, and communication interface 416. The output interface 404 may include a display device, such as a touch screen, which may also serve as an input device. The input interface 406 may include one or more of: a touch screen, touch pad, mouse, keyboard, camera, one or more device-specific buttons, one or more sensors integrated within computer 400 or coupled to computer 400 via a wired or wireless data connection, and other input devices. The computer may operate in a networked environment using communication connections to connect to one or more remote computers, such as a database server. The remote computers may include Personal Computers (PCs), servers, routers, network PCs, peer devices or other common data stream network switches, and the like. The communication connection may include a Local Area Network (LAN), a Wide Area Network (WAN), cellular, Wi-Fi, Bluetooth, or other network. According to one embodiment, various components of computer 400 are connected to system bus 420.

Computer readable instructions stored on a computer readable medium may be executed by processing unit 402 of computer 400, such as program 418. In some embodiments, the program 418 includes software for implementing one or more methods described herein, such as sensing data, transmitting data, performing analysis, executing machine learning models, storing data, and other functions described herein. Hard drives, CD-ROMs, and RAM are some examples of articles including a non-transitory computer-readable medium, such as a storage device. The terms computer-readable medium and storage device do not include a carrier wave because a carrier wave is considered too transitory. Storage may also include networked storage, such as a Storage Area Network (SAN). The computer programs 418, along with the workspace manager 422, may be used to cause the processing unit 402 to perform one or more of the methods or algorithms described herein.

Example (c):

1. a computer-implemented method includes receiving data from a plurality of devices at an edge server, transforming the data received from the plurality of devices, storing the transformed data in an edge server database, performing an analysis on the transformed data in the database, and selectively uploading the data to a remote server via a network.

2. The method of example 1, wherein the data received from each of the plurality of devices is received periodically from a plurality of wireless devices.

3. The method of any of examples 1-2, wherein transforming data received from each of the plurality of devices comprises buffering and transforming data received during a specified time window.

4. The method of example 3, wherein the specified time window is device-dependent.

5. The method of any of examples 1-4, and further comprising performing a function on data stored in an edge server database.

6. The method of example 5, wherein the function comprises executing a machine learning model for scoring, wherein the machine learning model is provided from a remote server.

7. The method of example 6, and further comprising determining an action based on an output of the performed function, generating a control signal based on the determined action, and transmitting the control signal to one or more of the devices to cause the action to be performed.

8. The method of any of examples 5-7, wherein the function includes one or more of: relational database functions, binary classification, linear regression, query execution, and aggregation.

9. The method of example 8, further comprising storing the output in an edge server database.

10. The method of any of examples 1-9, and further comprising receiving, at the edge server, the update from the remote server.

11. The method of any of examples 1-10, wherein the network comprises a public network or a private network, and wherein the remote server comprises a cloud resource.

12. The method of any of examples 1-11, wherein the plurality of devices comprise IoT (IoT) wireless sensors.

13. The method of any of examples 1-12, wherein selectively uploading data to a remote server via a network comprises uploading data in response to an on-demand request from an application.

14. The method of any of examples 1-13, wherein selectively uploading data to a remote server via a network comprises periodically uploading data.

15. A machine-readable storage device has instructions for execution by a processor of a machine to cause the processor to perform operations to perform a method of managing communication accounts. The operations include receiving data from a plurality of devices at an edge server; transforming the data received from the plurality of devices; storing the transformed data in an edge server database; performing an analysis on the transformed data in the database; and selectively uploading the data to a remote server via a network.

16. The device of example 15, wherein the data received from each of the plurality of devices is periodically received from a plurality of wireless devices, and wherein transforming the data received from each of the plurality of wireless devices comprises buffering and transforming data received during a specified time window.

17. The apparatus of any of examples 15-16, wherein the operations further comprise performing a function on data stored in the edge server database; determining an action based on an output of the performed function; generating a control signal based on the determined action; and transmitting the control signal to one or more of the devices to cause the action to be performed.

18. The apparatus of example 17, wherein the functions comprise one or more of: relational database functions, binary classification, linear regression, query execution, and aggregation.

19. An apparatus includes a processor and a storage device coupled to the processor and having stored thereon a program for execution by the processor to perform operations. These operations include: receiving data from a plurality of devices at an edge server; transforming the data received from the plurality of devices; storing the transformed data in an edge server database; performing an analysis on the transformed data in the database; and selectively uploading the data to a remote server via a network.

20. The device of example 19, wherein the data received from each of the wireless devices is periodically received from the plurality of wireless devices, and wherein transforming the data received from each of the plurality of wireless devices comprises buffering and transforming the data received during a specified time window. The operations also include performing a function on data stored in the edge server database, determining an action based on an output of the performed function, generating a control signal based on the determined action, and transmitting the control signal to one or more wireless devices to cause the action to be performed.

Although several embodiments have been described in detail above, other modifications are possible. For example, the logic flows depicted in the figures do not require the particular order shown, or sequential order, to achieve desirable results. Other steps may be provided from the described flows, or steps may be eliminated, and other components may be added to, or removed from, the described systems. Other embodiments are possible within the scope of the following claims.