Distributed Intelligence

The scope of this project is shown in Fig. 1, where the focus of this project is the visual information summarization as the application and the distributed intelligence as the required base technology. The gray parts in Fig. 1 are not under the focus of this project. That is, we will not focus on the human context inferring, and we will majorly consider DNN based algorithms in the distributed intelligence framework.


Figure. 1. Scope of this project. 

  • Visual information summarization

The concept of visual information summarization is shown in Fig. 2. The visual sensors deployed in the physical world sense the visual information continuously and generate ultra-big data. To handle such ultra-big data, the redundant data should be filtered out at different nodes of the network, including sensors and the aggregators. The huge raw data from the visual sensors are trimmed and summarized during the process, and only the important information is transmitted to the cloud servers for high-level analysis. Finally, the summarized information from the cloud servers is demonstrated to the human users to extend the human perception.

From our previous project, we have demonstrated that we can achieve 91.3% bitrate reduction for multi-view surveillance videos with sparse objects. However, the high bandwidth reduction rate cannot be well sustained for videos with dense objects. We plan to develop a new object-based system, which can further reduce 50% of the bandwidth for the situations with dense objects.

Moreover, video summarization from multiple cameras with re-identification technique is our next research target, where higher level object identification is conducted to track individual object in a video camera network. Rather than most re-identification works focusing on images of an object captured from different cameras, our work will focus on videos of objects. Furthermore, on-line and unsupervised approaches are more suitable for augmented collective beings (ACB) applications.

Figure. 2 Information summarization to support super sensing of ACB. 

  • Distributed intelligence

The technology behind is “distributed intelligence,” where the computation of visual data analysis is distributed on every part of the network, including sensors, aggregators, and cloud servers. Learning from our previous work, the distributed approach has the advantage of much lower transmission bandwidth requirement, compared with the centralized approach, where all the data analysis workloads are assigned to the cloud servers. The concept of distributed intelligence can be shown in Fig. 3. The data processing from the sensors, aggregators, to the cloud servers can be viewed as a data filtering process, where the amount of data becomes smaller and smaller during transmission from layer to layer; it can also be viewed as a context inferring process, where the semantics of the information become higher and higher from layer to layer. Note that, the computing resources at different layers should be different: light-weight learning/recognition engine at sensor layer, heterogeneous system architecture (HSA) with mobile CPU and GPU at the aggregator layer, and CPU/GPU clusters at the cloud server layer. In this project, we will focus on the sensor layer.

Among many classifiers using machine learning, we aim at the deep neural network (DNN) based classifier because of its high potential of performance. Both the network optimization of DNN and distributed computing of DNN are included in this project.

In summary, this project will focus on the following three topics:

A.  Visual Information summarization and visualization platform

B.  Learning hardware module design for distributed sensors

C.  Distributed strategy for deep neural networks

As shown in Fig. 1, B and C will be the technology base of A.


Fig. 3. Concept of distributed intelligence.