阅读说明：本技术 用于在云端和iot网络之间共享机器学习功能的系统和方法 (System and method for sharing machine learning functionality between cloud and IOT networks ) 是由 O·加尔西亚-马尔雄 A·莫蒂 于 2018-12-13 设计创作，主要内容包括：提供了用于基于被应用于包括多个感测节点和聚集节点的物联网(IoT)网络的卷积神经网络(CNN)使用深度学习的系统和方法。感兴趣事件基于所收集的数据而以更高可靠性被检测,并且IoT网络通过在IoT网络和云计算网络之间划分处理功能而改善带宽使用情况。(Systems and methods are provided for using deep learning based on a Convolutional Neural Network (CNN) applied to an internet of things (IoT) network including a plurality of sensing nodes and aggregation nodes. Events of interest are detected with higher reliability based on the collected data, and the IoT network improves bandwidth usage by dividing processing functions between the IoT network and the cloud computing network.)

1. A computer-implemented method for a plurality of nodes (10) using ML learning in an IoT network (100), comprising the steps of:

obtaining a trained ML model (22), physical location data of the plurality of nodes (10), and communication connection data of the nodes (10);

using a clustering algorithm to determine which nodes of the plurality of nodes (10) should be sensing nodes (10) and which nodes should be aggregation nodes (10), wherein a sensing node (10) senses and sends sensed data to one of the aggregation nodes (10), and the functionality of the aggregation node (10) comprises one or more of the following actions: (i) sensing; (ii) receiving the sensed data from the sensing node (10); (iii) performing a convolution of the sensed data received from the sensing nodes (10) with a weighted window; (iv) applying Sigmoid function to the convolution output; (v) sub-sampling the convolution output; (vi) sending a message containing a result of the action to an ML unit (21) portion of a cloud computing network (20); and is

Sending configuration information to the IoT network (100) regarding which of the plurality of nodes should be the sensing nodes or the aggregation nodes (10).

2. The method of claim 1, further comprising the step of the ML unit (21) determining that each of the aggregation nodes 10 requires an operation implemented with the sensed data and logic for sending the message to the ML unit (21).

3. The method of claim 1, wherein the sensed data is an occupancy measure of the region of interest 11 or a soil movement measure of the region of interest 11.

4. The method of claim 1, wherein the sensing node (10) sends the sensed data to the aggregation node (10) using a Local Area Network (LAN) interface (16), and the aggregation node (10) sends the message to the ML unit (21) using a Wide Area Network (WAN) interface (15).

5. The method of claim 1, wherein the aggregation node (10) sends the message to the ML unit (21) via a control unit (200) part of the IoT network (100).

6. A method of improving bandwidth utilization by using a CNN model (22) that can be partitioned and run partly in an IoT network (100) comprising a plurality of nodes (10) and partly in a cloud computing network (20) comprising ML units (21), the method comprising the steps of:

first processing a first layer of the CNN model (22) using the IoT network (100), wherein the IoT network (100) comprises one or more aggregator nodes (10) and a plurality of sensor nodes (10), wherein the plurality of sensor nodes (10) sense and send sensed data to the aggregator nodes (10) via a LAN interface (16), and the functionality of the aggregator nodes (10) comprises one or more of the following actions: (i) sensing; (ii) receiving the sensed data from the sensing node (10); (iii) performing a convolution of the sensed data received from the sensing nodes (10) with a weighted window; (iv) applying Sigmoid function to the convolution output; (v) sub-sampling the convolution output; (vi) -sending a message containing the result of said action to said ML unit (21);

second processing of the acted message by the ML unit (21) in one or more higher layers of the CNN model (22); and is

A feature of interest (FOI) prediction is determined based on the first and second processes.

7. The method according to claim 6, wherein the sensed data is an occupancy measure of the region of interest (11) or a soil movement measure of the region of interest (11).

8. The method of claim 6, wherein the IoT network is an intelligent lighting system (100).

9. The method of claim 6, wherein the sensing node (10) sends the sensed data to the aggregation node (10) using a Local Area Network (LAN) interface (16), and the aggregation node (10) sends the message to the ML unit (21) using a Wide Area Network (WAN) interface (15).

10. The method of claim 6, wherein the aggregation node (10) sends the message to the ML unit (21) via a control unit (200) part of the IoT network (100).

11. The method of claim 6, wherein the first processing step is performed using the aggregation node 10, the aggregation node 10 performing a first layer of the CNN model (22) by performing a convolution with a time window.

12. Method according to claim 6, wherein said first processing step is performed using said aggregation node 10, said aggregation node 10 performing a first layer of said CNN model (22) with initial weights of a weighting window, said initial weights being pre-initialized according to a given model tailored to the result of the action to be determined.

13. The method of claim 6, wherein the first processing step is performed using the aggregation node 10, the aggregation node 10 performing a first layer of the CNN model (22) by performing a temporal layer convolution of the sensed data in a given temporal-spatial window.

14. An intelligent lighting network (100), comprising:

a plurality of sensing nodes (10), each sensing node (10) comprising at least a first sensor (12) and a first LAN interface (16); and

a plurality of aggregation nodes (10), each aggregation node (10) comprising at least a second sensor (12), a second LAN interface (16), a WAN interface (15) and a processor (13), wherein the aggregation nodes (10) are configured to perform one or more of the following actions: (i) sensing; (ii) receiving the sensed data from one or more of the sensing nodes (10); (iii) performing a convolution of the sensed data received from the one or more sensing nodes (10) with a weighted window; (iv) applying Sigmoid function to the convolution output; (v) sub-sampling the convolution output; (vi) sending a message containing a result of the action to an ML unit (21) that is part of a cloud computing network (20),

wherein the determination of which sensing nodes (10) of the plurality of sensing nodes (10) should send the sensed data to which aggregation nodes (10) of the plurality of aggregation nodes (10) is determined in dependence on an ML model (22) and bandwidth communication limitations of the intelligent lighting network (100), the ML model (22) taking into account the number of aggregation nodes (10) determined by a window size of the ML model 22.

15. The intelligent lighting network according to claim 14, wherein the sensing node (10) sends the sensed data to the aggregation node (10) using the first LAN interface (16), and the aggregation node (10) sends the message to the ML unit (21) using the WAN interface (15).

Technical Field

The present invention relates to a system and method that uses deep learning based on convolutional neural networks applied to IoT networks, more particularly to detect events with higher reliability based on collected data and to save bandwidth by partitioning processing functions between IoT networks and the cloud.

Background

Smart lighting systems with multiple light emitters and sensors are experiencing a steady growth in the market. Intelligent lighting systems are a lighting technology designed for energy efficiency. This may include efficient means and automated control of adjustments based on conditions such as occupancy or daylight availability. Lighting is a deliberate light application that is done to achieve some aesthetic or practical effect. Including task lighting, accent lighting, and general lighting.

Such intelligent lighting systems may use multimode sensor inputs, e.g. in the form of occupancy and light measurements, to control the light output of the light emitters and adapt the artificial lighting conditions to prevailing environmental conditions. With the spatial granularity achieved by such sensor deployments in the context of lighting systems, it is possible to use sensor data to learn about the operating environment. For example, one such aspect relates to occupancy. There is an increased interest in learning about occupancy environments beyond basic presence. In this regard, occupancy modeling is closely related to building energy efficiency, lighting control, safety monitoring, emergency evacuation, and rescue operations. In some applications, occupancy modeling may be used when making automatic decisions, e.g., regarding HVAC control, etc.

Connecting the light sources to the lighting management system also enables a number of advanced features, such as: property management by tracking the position and status of light sources, reduced energy consumption by adapting lighting schedules, etc. Such intelligent lighting systems may also enable other applications such as positioning or visible light communication.

The intelligent lighting system may also enable other applications beyond illumination. Such applications can run on existing lighting infrastructure and bring added value. Examples of such other applications include people counting and soil motion monitoring.

People counting applications may be implemented using Passive Infrared (PIR) sensors. Such PIR sensors are traditionally used to reduce power consumption by turning on the lights in those areas that are occupied. PIR sensors are already widely available on the market. There is also the possibility to use such PIR sensors for other functions such as people counting in offices, activity monitoring, etc.

The soil motion monitoring application may be implemented using GPS data. For example, each intelligent outdoor light emitter may have a GPS sensor so that the light emitter may be automatically located once installed. It is known to track the amount of soil movement with relatively high accuracy using two GPS sensors, one located in a static area and the other located in an area subject to movement. However, it is not known whether there are better algorithms that can be used to generate insight (insight) about the amount of soil movement.

In this regard, as discussed below, aspects of the present invention that utilize machine and deep learning algorithms may be used to provide improved algorithms.

Machine Learning (ML) is a field of computer science that gives computers the ability to learn without being explicitly programmed. In this regard, machine learning refers to an algorithm that allows a computer to "learn" from data to adapt program actions accordingly. Machine learning algorithms are classified as supervised and unsupervised. Unsupervised learning involves drawing conclusions from the data table, for example, by classifying data items into difference classes. The learning algorithm is not given labels but lets itself find the structure in its input. Unsupervised learning can be the target itself (hidden patterns in the discovery data) or a means to go to the endpoint (feature learning). Supervised algorithms use learning in past data to apply it to new data. Example inputs and their desired outputs are given by a "teacher" and the goal is to learn an overall rule that maps inputs to outputs. As a special case, the input signal may only be partially available or limited to special feedback.

Deep learning is a specific type of machine learning algorithm that is motivated by the way the brain works. This approach attempts to model the way the human brain processes light and sound into vision and hearing. Some successful applications of deep learning are computer vision and speech recognition. The neurons are interconnected so that an answer is triggered based on the input. Deep learning aims to define a network of neurons organized in multiple layers, whereby the input data is processed layer by layer, so that the last layer can provide a high level of abstraction of the input data, with the weights between the links correctly selected.

There are many different alternative designs for deep learning algorithms. For example, Convolutional Neural Networks (CNNs), in which the tissue pattern of neurons is excited by the visual cortex. Such CNNs are a special multi-layer neural network that is trained using a version of the back-propagation algorithm. CNN uses a variant of the multilayer perceptron designed to require minimal preprocessing. Based on their shared weight architecture and translation invariance properties, they are also referred to as shift-invariant or space-invariant artificial neural networks (SIANN).

LeNet-5 is often considered the first CNN to work in practice for character recognition tasks in pictures. The design of CNNs opens up space, and this is desirable for applications beyond lighting, such as those described above, where multiple sensors are deployed in a given zone of interest to monitor a feature of interest, such as the number of people in a landslide or room. For example, if a landslide occurs, the "data pattern" captured by the sensor will be independent of the location of the landslide itself.

The main operational features of CNNs when classified are as follows:

n times (N layers)

Convolution layers compute the convolution of the input data using convolution filters (called "weighted windows"). The convolution will be performed on the entire input data, typically an array or matrix, so that the convolution emphasizes a specific pattern. This has three main meanings: (i) only local connectivity (with filter size) is required between the input and output nodes of the CNN; (ii) it shows that the spatial arrangement of data in the sensed data that is relevant for the filter is derived from closely located partitions in the input (vector/matrix); (iii) it shows that the parameters of the filter can be shared-this means that the input is time/space invariant.

Sub-sampling or pooling layers extract the most important features after each convolution. The main idea is that after convolution some features may appear in the tightly located areas. Redundant information can then be removed by sub-sampling. Typically, the output of the convolution is divided into a grid (e.g., cells of side 2 × 2) and a single value, such as an average or maximum value, is output from each cell.

The modified linear unit (ReLU) layer takes the output of the sub-sampled region and modifies it to a value in a given range, typically between 0 and the maximum value. One way to interpret this layer is to treat it as a binary decision that determines whether a given feature has been determined in a given region (after convolution and sub-sampling). The ReLu layer may utilize Sigmoid function f (x) = (1 + e)^-x)^-1That is, if x is very small, f (x) is 0, if x is about, if x is large, f (x) tends to 1.

In general, if the sub-sampling layer has a size of 2 × 2, then for a size of N²And N layers, the feature will have N ∗ 2^-NThe size of (2).

The fully connected layer is the last layer that connects all outputs of previous layers to obtain the final answer as a feature combination of N-1 layers. This layer may be as simple as a matrix operation that multiplies the inputs generated by the last layer to quantify the likelihood of each potential event/classification.

The process of learning the parameters of CNN is summarized as follows:

all parameters (weights and biases) are initialized in a random manner.

-calculating an output for the training data.

-calculating a cost function (error) in the last layer (C = shen ∑ (target-output)²）。

-back-propagating the error and deriving the error in each neuron within the network.

-obtaining a gradient of the cost function with respect to weight and bias, given an error in each neuron in the network.

-updating the updated values of the weights and the deviations to w_i+1= w_iƞ dC/dW, where w_iIs the weight of the current iteration, ƞ is the learning rate, and dC/dW is the calculated gradient.

Disclosure of Invention

One aspect of the present invention, which relates to an improved method of deep learning using convolutional neural network-based, may be applied to an IoT network. The method uses data obtained by a sensor network, thereby enabling detection of events with higher reliability.

Another aspect of the invention relates to a method of using a CNN model that can be partitioned and run partially in an IoT network and partially in a cloud. This allows bandwidth to be saved. The cloud may automate the computation of nodes in the IoT network that have different roles (sensing and aggregation) and how the model may be partitioned and deployed.

Yet another aspect of the present invention relates to optimizing bandwidth utilization for IoT networks and cloud ends.

Yet another aspect of the invention enables real-time applications based on deep learning networks. This may be used to ensure that a gateway or other intermediate infrastructure that is part of an IoT network or cloud computing network is not overwhelmed by processing incoming data and performing deep learning operations.

One embodiment of the present invention is directed to a computer-implemented method for a plurality of nodes using ML learning in an IoT network. The method comprises the following steps: a trained ML model, physical location data of a plurality of nodes, and communication connection data of the nodes are obtained. A clustering algorithm is used to determine which nodes should be sensing nodes and which should be aggregation nodes. The sensing node senses and sends the sensed data to the aggregation node. The functions of the aggregation node include one or more of the following actions: (i) sensing; (ii) receiving sensed data from a sensing node; (iii) performing a convolution on the sensed data received from the sensing nodes with a weighted window; (iv) applying Sigmoid function to the convolution output; (v) sub-sampling the convolution output; (vi) and sending a message containing the action result to the ML unit part of the cloud computing network. Configuration information regarding which of the plurality of nodes should be sensing or aggregation nodes is sent to the IoT network.

One advantage of this approach is that the latency of the IoT network is reduced.

Another embodiment of the present invention is directed to a method for improving bandwidth utilization by using a CNN model that may be partitioned and run in part in an IoT network comprising a plurality of nodes and in part in a cloud computing network comprising ML units. The method includes the step of first processing a first layer of the CNN model using the IoT network. The IoT network includes one or more aggregation nodes and a plurality of sensing nodes. The sensing node senses and sends the sensed data to the aggregation node via the LAN interface. The functions of the aggregation node include one or more of the following actions: (i) sensing; (ii) receiving sensed data from a sensing node; (iii) performing a convolution on the sensed data received from the sensing nodes with a weighted window; (iv) applying Sigmoid function to the convolution output; (v) sub-sampling the convolution output; (vi) a message containing the action result is sent to the ML unit. The method further comprises the following steps: the message of the action is second processed by the ML unit in one or more higher layers of the CNN model, and a feature of interest (FOI) prediction is determined based on the first and second processing.

Yet another embodiment of the present invention is directed to an intelligent lighting network, the plurality of sensing units each including at least a first sensor and a first LAN interface, and the plurality of aggregation nodes including at least a second sensor, a second LAN interface, a WAN interface, and a processor. The aggregation node is configured to perform one or more of the following actions: (i) sensing; (ii) receiving sensed data from one or more of the sensing nodes; (iii) performing a convolution on sensed data received from one or more sensing nodes with a weighted window; (iv) applying Sigmoid function to the convolution output; (v) sub-sampling the convolution output; (vi) sending a message containing the action result to an ML unit that is part of the cloud computing network. The determination of which sensing nodes should send sensed data to which aggregation nodes is determined according to an ML model that takes into account the number of aggregation nodes determined by the window size of the ML model, and the bandwidth communication limitations of the intelligent lighting network.

Drawings

Further details, aspects and embodiments of the invention will be described, by way of example only, with reference to the accompanying drawings. Elements in the figures are illustrated for simplicity and clarity and have not necessarily been drawn to scale. In the drawings, elements corresponding to elements already described may have the same reference numerals. In the drawings:

fig. 1 schematically shows an example of an embodiment of a system element.

Fig. 1a schematically shows an embodiment of an outdoor lighting system.

Fig. 2 schematically shows details of an example of an embodiment of components in a node of the system element of fig. 1.

FIG. 3 schematically illustrates an example of an embodiment of centralized operation of the system elements of FIG. 1.

Fig. 4 schematically shows an example of an embodiment of distributing the first layer of CNN to an IoT network.

Fig. 5 schematically illustrates an example of multiple local communications in 2 x 2 and 3 x 3 convolution windows.

Fig. 6 schematically shows an example of an embodiment of a plurality of windows and aggregation nodes.

Fig. 7 schematically illustrates an example of a method to optimize the manner in which ML models are deployed in an IoT network.

FIG. 8 shows an example of a spatial window function of a node that may be used to profile the system elements of FIG. 1.

Detailed Description

While this invention is susceptible of embodiment in many different forms, there is shown in the drawings and will herein be described in detail one or more specific embodiments, with the understanding that the present disclosure is to be considered as an exemplification of the principles of the invention and is not intended to limit the invention to the specific embodiments illustrated and described.

Hereinafter, elements of the embodiment are described in operation for ease of understanding. It will be apparent, however, that the corresponding elements are arranged to perform the functions described as being performed by them.

Furthermore, the invention is not limited to the embodiments, and the invention lies in each and every novel feature or combination of features described herein or recited in mutually different dependent claims.

FIG. 1 shows a representation of system elements according to one embodiment of the invention. As shown in fig. 1, n nodes 10 are deployed in a region of interest (ROI) 11. The node 10 monitors a feature of interest (FOI) in the ROI 11. As mentioned above, the FOI may be, for example, occupancy, soil movement, or any other characteristic or variable in the ROI. In one embodiment, the FOI is an occupancy measure of the ROI, such as number of people or density of people. The FOI may be obtained by some means other than the conventional organization of the illumination system. For example, a camera may be used to count people, or a person may count people on the ground. The person may be tagged to detect their presence, for example by their mobile phone.

The node 10 collects data that is then sent (possibly after some degree of preprocessing) to the cloud 20 (or cloud computing network), and a Machine Learning (ML) unit at the cloud 20 contains algorithms that process the data from the node 10 to obtain a given insight (insight) about the FOI. This process is done according to the trained ML model 22. The process of training an ML model involves providing the ML algorithm (i.e., the learning algorithm) with training data to learn from. The training data must contain the correct answer, referred to as the target or target attribute. The learning algorithm finds patterns in the training data that map the input data attributes to the target (the answer to be predicted), and its output captures the trained ML model 22 of these patterns. Trained ML model 22 may be used to obtain predictions about new data whose targets are unknown.

Fig. 1a shows another configuration of an outdoor lighting system according to an embodiment of the invention.

As shown in FIG. 1a, the outdoor lighting system 100 includes one or more lighting units (LUs 1-LU 8) configured to act as nodes 10. LU (LU 1-LU 8) may include light emitting organ 101, one or more sensors 102, database 103, communication interface 104, and light level controller 105.

The sensors 102 may be used to detect one or more objects/Features (FOIs) within a predetermined sensing Range (ROI). The sensor 102 may be any suitable sensor to achieve this result. For example, a passive infrared, radar sensor, GPS or camera may be used to give the detection result. Such a sensor 102 may send a "detection" in the form of a sensed data result if an object or feature is detected within the sensing range of the sensor 102. The sensor 102 may also periodically attempt to detect an object within the sensing range and be a "detect" result if an object is detected and a "no detect" result otherwise.

The communication interface 104 may be, for example, a hard-wired link and/or a wireless interface compatible with DSRC, 3G, LTE, WiFi, RFID, wireless mesh, or another type of wireless communication system and/or visible light communication. The communication interface 104 may be any suitable communication arrangement to transmit data between one or more of the LUs (1-8), the control unit 200, and/or the cloud 20.

The database 103 need not be included in the LUs (1-8). Since LUs (1-8) may communicate with one or more other LUs (1-8) and/or intermediate nodes (not shown in FIG. 1 a), any data that would need to be stored or accessed by a particular LU (LU 1-LU 8) may be stored in and accessed from database 103 in another LU (LU 1-LU 8), in an intermediate node, or in other network storage as needed.

As shown in fig. 1a, the lighting system 100 may further comprise a control unit 200 (e.g. a service center, a logistics office, a maintenance center, etc.). The control unit 200 may be located near LU (LU 1-LU 8) or at a remote location. The central control unit 200 comprises a communication unit 201 and may also comprise a database 202. The communication unit 201 is used to communicate with LUs (LUs 1-LU 8) and/or other external networks such as the cloud 20 (not shown in FIG. 1 a). The control unit 200 is directly or indirectly communicatively coupled to LUs (LUs 1-LU 8) and/or the cloud 20. For example, control unit 200 may communicate directly via wired and/or wireless/wireless mesh connections or indirectly via a network, such as the internet, an intranet, a Wide Area Network (WAN), a Metropolitan Area Network (MAN), a Local Area Network (LAN), a terrestrial broadcast system, a cable network, a satellite network, a wireless network, a wire or telephone network (POTS), as well as portions or combinations of these and other types of networks.

The control unit 200 includes algorithms for operating, invoking on/off times and sequencing, dimming times and percentages, and other control functions. The control unit 200 may also perform data logging of parameters such as hours of operation or energy usage, alarm and scheduling functions.

The communication interface 104 as mentioned above in relation to the communication unit 201 may be any suitable communication arrangement to transmit data to and/or from the control unit 200. In this regard, via the communication interface 104, each LU (LU 1-LU 8) may communicate with the control unit 200 directly and/or via another LU (LU 1-LU 8) as desired. The communication interface 104 implements remote command, control, and monitoring of LUs (LUs 1-LU 8).

Sensors 102 disposed throughout the lighting system 100 capture data. The data may relate to various features, objects, characteristics (FOI) within range of the sensor 102. Raw data and/or pre-processed data (referred to as "data") may be transmitted to the control unit 200, the cloud 20, or other network devices for processing as discussed below.

It should be understood that the embodiments of fig. 1 and/or 1a may be deployed (or modified to be deployed) in a building, such as an office building, hospital, etc. A connected lighting system is not essential to the embodiment, e.g. sensors or the like may be installed without or differently from the connected lighting system. However, the inventors have found that the infrastructure of the connected lighting system lends itself well to the installation of embodiments of the present invention.

Fig. 2 shows system components of another embodiment of the node 10. In this embodiment, the node 10 includes at least one sensor 12 (e.g., a PIR sensor, a GPS sensor, or an accelerometer, etc.). In other embodiments, the FOI may include (1) immediate characteristics showing the immediate output of the sensor 12 when the data is queried, including, for example, light level, binary motion, CO2 concentration, temperature, humidity, binary PIR, and door status (on/off); (2) counting features (net counts of movement, net counts of PIR, and net counts of doors) that record the number of times the output of the sensor 12 has changed within the last minute; (3) the average characteristics of the average of the outputs of the sensor 12 over a certain period of time are displayed (occupancy sensor, sound average (e.g. every 5 seconds)).

Data from the sensors 12 may be processed by the CPU 13 and/or stored in the local memory 14. The node 10 may then send information to other nodes 10 in the LAN, to the control unit 200, and/or over a Wide Area Network (WAN) 20, for example using a Local Area Network (LAN) interface 16, and/or over the WAN.

In other embodiments, some of the above-mentioned communication interfaces between the node 10, the cloud 20 and the control unit 200 may comprise a wired interface such as an ethernet cable, or a wireless interface such as a Wi-Fi or ZigBee interface, etc.

The nodes 10 in fig. 1, 1a and 2 may be IoT (internet of things) devices. IoT refers to an increasing network of physical objects characterized by IP addresses for internet connectivity, and the communications that occur between these objects and other internet-enabled devices and systems. An IoT is a network of physical devices, vehicles, home appliances, and other items that are embedded with electronics, software, sensors, actuators, and network connections that enable these objects to connect and exchange data. Each object can be uniquely identified by its embedded computing system, but can interoperate within the existing internet infrastructure.

IoT allows objects to be sensed or controlled remotely across existing network infrastructure, which creates an opportunity to integrate the physical world more directly into computer-based systems and brings increased efficiency, accuracy, and economic benefits in addition to reduced human intervention. When IoT is enhanced by sensors and actuators, this technology is an example of a more general type of cyber-physical system, which also encompasses technologies such as smart grid, virtual power plants, smart homes, smart transportation, and smart cities.

The node 10 (i.e., IoT device) may process raw data from the sensor 12 or may transfer or share the processing of the raw data to a remote device such as the cloud 20.

Before describing how processing functions may be divided between the cloud 20 and the node 10 and/or the control unit 200 and within the node 10 and/or the control unit 200, it is first described how a CNN network may be centrally applied at the cloud 20. In this regard, fig. 3 depicts a multi-layer architecture, where a first layer corresponds to nodes 10 (shown as lighting units in the ceiling) deployed in the ROI (see fig. 1). Fig. 3 shows an example in an office setting where occupancy is to be determined. In this case, layer 1 corresponds to the output of one node 10 at a given time T. The particular filter depicted in fig. 3 has a given dimension representing a given spatial region to be scanned. It should be understood that other dimensions may be used. After applying the ReLU function, the convolution of this filter with the values of the input data is used to obtain the values of layer 2 (which are then used for subsequent layers, etc.). The convolution is performed using a weighted window. Note also that the above description does not include the sub-sampling phase, however this phase may also be included.

Having now described the above description of a centralized system, a description is made of how processing functions may be divided between the cloud 20 and the nodes 10 and/or control units 200 (i.e., IoT networks). This is depicted in fig. 4. Here, the first layer of the CNN may be deployed to the IoT network such that operations corresponding to this layer are performed locally (i.e., in the node 10 and/or the control unit 200).

Two types of nodes 10 are identified: a sensing node and an aggregation node. The functionality of the sensing node is limited to sensing and sending (as it is in the centralized embodiment explained before) the sensed values to the aggregation node. The functions of the aggregation node include one or several of the following actions: (i) sensing; (ii) receiving sensed values from closely located nodes 10; (iii) performing a convolution on data received from the closely located nodes using a weighted window; (iv) applying a sigmoid function to the convolved outputs; (v) sub-sampling the output; (vi) (iv) sending a message to the cloud containing each value after steps (iii), (iv) and (v).

The cloud receives input from aggregator 10 that is input to higher layers of CNN. In this example, it would be the input to layer 2.

Communication between the sensing node 10 and the aggregation node 10 may be performed by using a LAN interface 16. The communication between the aggregation node 10 and the cloud 12 may use the LAN interface 16 to reach a gateway (not shown) that includes its own WAN network interface, or alternatively directly over the WAN interface 15.

The process of deploying ML parameters to the node 10 and/or control unit 200 (i.e., IoT network) as part of the software defined deep learning is now described. As mentioned above, once the trained ML model 22 has been learned, the cloud 20 will have a pattern of the IoT network (typically a deep learning algorithm or a machine learning algorithm). At this stage, cloud 20 must optimize the manner in which trained ML model 22 is deployed to the IoT network for optimal performance.

The overall process is depicted in fig. 5. As inputs, the process receives the trained ML model 22, the physical location of the nodes 10 (e.g., GPS coordinates of the nodes placed outdoors, or the layout of the nodes 10 in a building), and the LAN connection matrix of the nodes 10 (i.e., how the nodes may communicate with each other — this may include the signal strength of the physical layer, the packet throughput of the MAC layer, the routing structure, etc.).

Given these inputs, the ML unit 21 will determine the appropriate sense nodes 10 and aggregation nodes 10. This step may be done using a clustering algorithm. Clustering is the task of dividing a population or data point into groups such that the similarity of the data point in the same group to other data points in the same group is higher than to data points in other groups. The purpose is to isolate groups with similar characteristics and assign them as clusters. There are many types of clustering algorithms, such as connection models, center of gravity models, distribution models, and density models, as well as cluster types such as k-means and hierarchy.

In one embodiment of the present invention, a k-means clustering algorithm is used that takes into account that the number of aggregated nodes 10 is determined entirely by the window size of the ML model 22 and the overall communication constraints (the amount of bytes and messages that can be sent from the network of nodes 10 to the cloud 20). Given the initial number of aggregation nodes 10, the nodes 10 may then be placed according to a given grid and usage data about physical locations and LAN connections to determine a set of nodes 10 that minimizes local communications and maximizes performance.

Once the aggregation nodes 10 are determined, the ML unit 21 will determine which sensing nodes 10 should send their data to which aggregation nodes 10. Alternatively, the ML unit 21 may determine which sensing nodes 10 the aggregation node 10 should subscribe to receive their data. This is determined in such a way that the aggregation node 10 receives all the data generated by the peripheral nodes 10 and required to generate the input data of the next layer (including sub-sampling). Furthermore, the ML unit 21 will determine the operations (typically convolutions, Sigmoid functions, sub-sampling) each aggregator node 10 needs to implement with the collected data, and then determine the logic for sending messages to the cloud 20.

The final step is to send a message to each node 10 in the network in a specific configuration: a sensing or aggregation node; how the sensed information should be distributed; the child ML models in the node 10 are aggregated. There may also be some handshaking in the communication to ensure that the information in the cloud 20 has been correctly received by the node 10 (reliability).

The advantages and disadvantages of the centralized and distributed deep learning processing methods described above are now compared. In this regard, three different configurations of the distributed architecture are described:

(1) wherein the first CNN layer operates in an IoT network and has a weighting window of size 2 x 2,

(2) wherein the first CNN layer operates in an IoT network and has a weighting window of size w x w,

(3) with the first two CNN layers running in the IoT network and having a weighting window of size 2 x 2.

Note that this calculation does not include the effect of sub-sampling, but only a similar reduction due to window size similar to that shown in the above diagram.

A performance analysis is analyzed that takes into account (1) the communication overhead from the nodes to the cloud, (2) local communication requirements, and (3) deep learning iterations in the IoT network.

The following is an analysis framework to estimate the bandwidth savings that will be achieved by aspects and embodiments of the present invention. Given a mesh network of light emitters (as shown in fig. 1), local communication over one hop can be generalized to any number of hops in the analysis framework.

FIG. 6 shows the number of local communication messages that need to be exchanged in order to compute the convolution over such a window, that is, three in the case of 2 × 2 nodes 10 and eight in the case of 3 × nodes 10. for an n × n grid of nodes 10 (e.g., luminaires in outdoor lighting system 100), the number of 2 × 2 convolution windows is (n-1)²It is given.

Next, consider the number of aggregation nodes 10 of the n × n grid of nodes 10 depending on whether n is odd or even, if n is odd, the number of aggregation nodes 10 is represented by ((n-1)/2)²Given that, if n is an even number, the number of aggregation nodes is given by (n/2)²Given below.

See fig. 7(6) for an example of calculating the convolution window and the number of aggregation nodes 10. The total number of local communication messages is less than or equal to (w 2-1) x the number of windows. The total number of messages from the aggregation node 10 to the gateway is equal to the number of aggregation nodes 10 x the number of windows (one value per window). For 2 x 2 windows, one aggregation node 10 handles a maximum of four windows.

As shown in the embodiments described above, moving functionality to the IoT network (e.g., node 10 and/or outdoor lighting system 100) results in significantly reduced communication from the IoT network to cloud 20 because the first layer(s) are executed in the IoT network, which has abstracted higher level features. Although this raises the price of local communications, it does not involve high costs as long as the node 10 includes the LAN network interface 16, since it is local. With these aspects of the invention, some of the processing of raw data from sensors 12 may be readily distributed and processed in aggregation node 10 and/or control unit 200. This means that fewer computations need to be done centrally or at the cloud 20.

In another embodiment, the weighting of the weighting window, the convolution filter, may be determined by the function Wx_o,y_o(x, y) (spatial window) where (x)_o, y_o) Determine where the function is sampled, and (x, y) is applied to the function with respect to (x)_o, y_o) Weight of the output of the sensor located at position (x, y). Such a function Wx is shown in fig. 8_o,y_oAn example of (x, y). For example, this embodiment is advantageous in intelligent lighting networks, where the ML algorithm requires sensor data at locations where the light emitters are not located. In such a case, the value from the nearest light emitter may be used to interpolate the value at the desired point.

By taking into account the above spatial window running on the input data generated by the sensors 12 and only at some locations (x)_o, y_o) Several output values are obtained that will correspond to the inputs of the second layer in CNN, which may be combined with the CNN sub-sampling step. Such a function is very useful because the sensors 12 in a node 10 (e.g., a portion of a LU) may not actually be distributed according to a completely regular grid.

In another embodiment, the input for layer 1 may be obtained by outputting a value from sensor 21 calculated over a period of time, which is equal to the sum, maximum, average. This means that the first layer of the CNN network does not need to work on only the raw sensor data. The node 10 may also operate with aggregate values such as average, maximum, minimum, etc.

In another embodiment involving gunshot detection, the node 10 may run the first tier of CNN by performing a convolution with a time window. In this regard, the node 10 includes a sensor as a microphone, the first layer corresponding to convolution with a signal from the gunshot trigger.

In a further embodiment, the initial weights of the weighting windows are pre-initialized according to a given ML model, which is customized for the signal to be calculated. In more detail, the above embodiments have covered the deployment of the model across nodes 10, and then the ML model is executed using the aggregation nodes 10 and the sensing nodes 10. This embodiment is a process preceding these two steps: optimal parameters of the ML model are learned. In this embodiment, data used to learn the ML model itself across the nodes 10 is determined. More specifically, an initial seed value for the iterative learning process is generated.

In various embodiments, the input/communication interface may be selected from a variety of alternatives. For example, the input interface may be a network interface for a local area network or a wide area network such as the internet, a storage interface for internal or external data storage, a keyboard, and the like.

The storage or memory may be implemented as electronic memory, flash or magnetic memory, a hard disk, etc. The storage may include a plurality of discrete memories that collectively make up the storage. The storage may also be a temporary memory, i.e. a RAM. In the case of temporary storage.

Typically, the cloud 12, the node 10, the control unit 200 each comprise a microprocessor, CPU or processor circuit executing appropriate software stored therein; for example, the software may have been downloaded and/or stored in a corresponding memory, e.g. a volatile memory such as a RAM or a non-volatile memory such as a flash memory (not separately shown). Alternatively, such devices may be implemented in whole or in part in programmable logic, for example as Field Programmable Gate Arrays (FPGAs), or as so-called Application Specific Integrated Circuits (ASICs), i.e., Integrated Circuits (ICs) tailored to their specific use. For example, the circuit may be implemented in CMOS, e.g., using a hardware description language such as Verilog, VHDL, etc. The processor circuit may be implemented in a distributed manner, for example as a plurality of sub-processor circuits. The storage may be distributed over a plurality of distributed sub-stores. Some or all of the memory may be electronic memory, magnetic memory, or the like. For example, the storage may have volatile and non-volatile portions. The stored portion may be read-only.

In one embodiment, the outdoor lighting system 100 may include sensors 102 having different modalities. The outdoor lighting system 100 may have hierarchical levels, such as a hierarchy in which devices communicate with corresponding higher or lower level devices. Note that in a lighting system, a plurality of light emitters and sensors are grouped together in a control zone. Multiple control zones may be defined within the same room, e.g. one control zone for light emitters close to the window and one for the remaining light emitters. Next, a plurality of rooms are located on the same floor, and so on. There is one local control area at each hierarchical level. The local controller may play the role of a controller at multiple hierarchical levels.

As will be clear to a person skilled in the art, many different ways of performing the above described method are possible. For example, the order of the steps may be varied or some steps may be performed in parallel. In addition, other method steps may be inserted between the steps. The inserted steps may represent modifications of the method such as described herein or may be unrelated to the method.

The method according to the invention may be performed using software comprising instructions for causing a processor system to perform the method. The software may include only those steps taken by specific sub-entities of the system. The software may be stored in a suitable storage medium such as a hard disk, floppy disk, memory, optical disk, etc. The software may be transmitted as signals, either wirelessly or along wires or using a data network such as the internet. The software may be made available for download and/or for remote use on a server. The method according to the invention may be performed using a bitstream arranged to configure programmable logic, such as a Field Programmable Gate Array (FPGA), to perform the method.

It will be appreciated that the invention also extends to computer programs, particularly computer programs on or in a carrier, adapted for putting the invention into practice. The program may be in the form of source code, object code, a code intermediate source and object code such as partially compiled form, or in any other form suitable for use in the implementation of the method according to the invention. Embodiments related to computer program products include computer-executable instructions corresponding to each processing step of at least one of the set forth methods. These instructions may be subdivided into subroutines and/or stored in one or more files that may be linked in a static or dynamic manner. Another embodiment relating to a computer program product comprises computer-executable instructions corresponding to each means of at least one of the systems and/or products set forth.

For example, the computer readable medium has a writeable part comprising a computer program comprising instructions for causing a processor system to perform the method of the invention according to an embodiment. The computer program may be embodied on a computer readable medium as physical marking or by magnetization of the computer readable medium. However, any other suitable embodiment is also contemplated. Further, it will be understood that while the computer-readable medium may be an optical disk, the computer-readable medium may be any suitable computer-readable medium, such as a hard disk, solid state memory, flash memory, etc., and may be non-recordable or recordable. The computer program comprises instructions for causing a processor system to perform the method.

For example, in an embodiment, the node 10, the control unit 200 and/or the cloud 20 may comprise a processor circuit and a memory circuit, the processor being arranged to execute software stored in the memory circuit. For example, the processor circuit may be an Intel Core i7 processor, an ARM Cortex-R8, or the like. The memory circuit may be a ROM circuit or a non-volatile memory, such as a flash memory. The memory circuit may be a volatile memory, such as an SRAM memory. In the latter case, the verification device may comprise a non-volatile software interface, such as a hard disk, a network interface, etc., arranged to provide software.

The foregoing detailed description has set forth a few of the many forms that this invention can take. The above examples are merely illustrative of several possible embodiments of various aspects of the invention, wherein equivalent alterations and/or modifications will occur to others skilled in the art upon the reading and understanding of this specification and the annexed drawings. In particular regard to the various functions performed by the above described components (devices, systems, etc.), the terms (including a reference to a "means") used to describe such components are intended to correspond, unless otherwise indicated, to any component, such as hardware or combinations thereof, which performs the specified function of the described component (i.e., that is functionally equivalent), even though not structurally equivalent to the disclosed structure which performs the function in the herein illustrated implementations of the disclosure.

The principles of the present invention may be implemented as any combination of hardware, firmware and software. Furthermore, the software is preferably implemented as an application program tangibly embodied on a program storage unit or computer-readable storage medium constituted by certain devices and/or combinations of devices or parts thereof. The application program may be uploaded to, and executed by, a machine comprising any suitable architecture. The computer platform may also include an operating system and micro-architectural code. The various processes and functions described herein may be either part of the micro-architectural code or part of the application program, or any combination thereof, which may be executed by a CPU, whether or not such computer or processor is explicitly shown. In addition, various other peripheral units may be connected to the computer platform such as an additional data storage unit and a printing unit.

While a particular feature of the invention may have been illustrated and/or described with respect to only one of several implementations, such feature may be combined with one or more other features of the other implementations as may be desired and advantageous for any given or particular application. Furthermore, unless otherwise specified, reference to a singular component or item is intended to encompass two or more such components or items. Furthermore, to the extent that the terms "includes," including, "" has, "" having, "" with, "or variants thereof are used in either the detailed description and/or the claims, such terms are intended to be inclusive in a manner similar to the term" comprising.

The invention has been described with reference to the preferred embodiments. However, modifications and alterations will occur to others upon reading and understanding the preceding detailed description. It is intended that the invention be construed as including all such modifications and alterations. It is intended that the scope of the invention be defined only by the claims, which include all equivalents.

In the claims, numerals in parentheses refer to numerals in the drawings of the exemplary embodiments or formulas of the embodiments to thereby enhance intelligibility of the claims. These references should not be construed as limitations on the claims.