Big Data and AI Driven Innovations

1. Summary of the impact

The HKPolyU team has conducted research in Big Data Analytics and AI Driven Innovations for smart transportation and advanced image processing. Major achievements include: 1) development of big data analytics for highly-efficient transportation, logistics and airport resource management; 2) deep learning-based approaches for enhanced image recognition and computer vision technologies. The research has been granted HK$20.58M amount of funding support, resulting in 7 patents, 11 sponsorships, and the deployment of 8 systems with industrial and government partners, including the leading logistics in HK (HKIA, GoGoVan) and one of the largest technology companies in China (Alibaba)

2. Underpinning research

Data-Driven Smart Transportation. Emerging on-demand transport services, such as Uber and GoGoVan, usually face the dilemma of demand supply imbalance, i.e., the spatial distributions of orders and drivers are imbalanced. To solve the challenges above, our staff Prof. Jiannong Cao and his team proposed big data analytics for urban complex systems [R1], including distributed big data sharing techniques to collect and exchange data from various sources, designed cross-domain big data fusion methods to combine multi-source data for comprehensive analytics, and devised structure-aware big data analytic algorithms to capture the dependency in urban complex systems, such as transportation, logistics and airport resource management. The related big data analytics are as follows.

• Distributed big data sharing: Current big data sharing schemas heavily rely on trustworthy third parties. We studied the challenging issues in designing a decentralized big data sharing platform for trustless data sharing and proposed mechanisms and algorithms for various system functions, including homogeneous representation of heterogeneous big data, distributed and high-performance data retrieval, flexible and secure data sharing approaches, and traceable and reliable storage of data sharing records. In this research we developed a fully distributed trustworthy data-sharing platform comprised of the basic functions of data publishing, data retrieval, and data transaction. All the mechanisms and algorithms have been proved to be effective and efficient by extensive experiments over our platform.
• Cross-domain big data fusion: Big data analytics using cross-domain datasets allows us to study an issue by fusing views from multiple angles. However, there are open challenges of analyzing massive cross-domain datasets including heterogeneity, uncertainty and high dimensionality. We have proposed an effective solution to address these challenges through cross-domain data representation learning and coupled matrix factorization [R2].
• Structure-aware big data analytics: Structure-aware big data analytics learns knowledge from all concerned components of a complex system by considering the dependency between them. We proposed a new analytic model, which can facilitate prediction and detection in complex systems to improve their efficiency and stability [R3].

AI-Driven Smart Imaging. Our staff Prof. Lei Zhang and his team developed a series of AI based approaches to enhance the image quality and image recognition, including performance advanced sparse representation, low-rank analysis and deep learning based.

• Sparse/collaborative representation: The ASDS method (TIP2011, 979 citations) was proposed to cluster image patches into a few sparse domains and adaptively selects the most suitable domain to represent each patch. The NCSR (TIP2013, 732 citations) method seamlessly integrates sparse coding with image nonlocal self-similarity for image restoration. A pioneering work in [R4] is among the early works that successfully apply CNN for image denoising. The proposed DnCNN model can tackle several general image denoising tasks, such as Gaussian denoising, super-resolution, and JPEG image deblocking, using a single model. DnCNN has now become a benchmark for image denoising.
• Low-rank minimization: A low-rank based robust online background subtraction method was proposed in [R5], which can be automatically adjusted to fit a wide range of video background transformations and camera movements. It demonstrated superior performance to state-of-the-art online and offline background subtraction methods in both accuracy and speed.
• Pattern classification: The collaborative representation nature of the sparse representation based classifier was revealed and demonstrated that collaborative representation plays the key role in representation based pattern recognition (ICCV 2011, 1583 citations). This work is the third most cited paper in ICCV 2011 (https://aminer.org/bestpaper). The team is also one of the first that studied discriminative dictionary learning for image recognition and applied the Fisher discrimination criterion to learn discriminative dictionaries. The developed FDDL model (ICCV 2011, 823 citations) has become a benchmark in discriminative dictionary learning. This work was later extended to [R6].

3. References to the research

[R1] Wengen Li, Jiannong Cao, Jihong Guan, Shuigeng Zhou, Guanqing Liang, Winnie K.Y. So, Michal Szczecinski, “A General Framework for Unmet Demand Prediction in On-demand Transport Services”, IEEE Transactions on Intelligent Transportation Systems, vol. 20, Issue 8, Aug. 2019, pp. 2820-2830.

[R2] Yuqi Wang, Jiannong Cao, Lifang He, Wengen Li, Lichao Sun, Philip S. Yu, “Coupled Sparse Matrix Factorization for Response Time Prediction in Logistics Services”, The 26th ACM International Conference on Information and Knowledge Management (CIKM 2017). November 6-10, 2017. Singapore.

[R3] Linchuan Xu, Xiaokai Wei, Jiannong Cao, Philip S. Yu, "On Exploring Semantic Meanings of Links for Embedding Social Networks", ACM World Wide Web conference (WWW 2018), April 23-27, 2018, Lyon, France, pp. 479-488.

[R4] Kai Zhang, Wangmeng Zuo, Yunjin Chen, Deyu Meng, Lei Zhang, “Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising”, IEEE Transactions on Image Processing, vol. 26, issue 7, pp. 3142-3155, July 2017. (Cited by 725 times; ESI highly cited paper)

[R5] H. Yong, D. Meng, W. Zuo, L. Zhang, "Robust Online Matrix Factorization for Dynamic Background Subtraction," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 40, issue 7, pp. 1726-1740, July 2018.

[R6] M. Yang, L. Zhang, X. Feng, and D. Zhang, “Sparse Representation based Fisher Discrimination Dictionary Learning for Image Classification,” International Journal of Computer Vision, vol. 109, issue 3, pp. 209-232, Sept. 2014. (Cited by 300 times; ESI highly cited paper)

4. Details of the impact

The impact of the research on big data and AI driven innovations is demonstrated by the increasing collaboration with, and impressive funding support from, various departments of government and industrial organizations, further development and deployment of the proposed systems, as well as by the productive industrial collaboration and sponsorships.

Data-Driven Smart Transportation: The impact of our work on big data analytics for urban complex systems mainly reflects from the following three aspects [S1]-[S4].

• With the support of Huawei, we develop a big data sharing platform supporting fully distributed data sharing without trustworthy third parties. In collaborating with Huawei, the research results have been incorporated in the design of Huawei’s future data sharing tools. The developed distributed data sharing platform is promising to be adopted for data sharing among institutes and organizations due to its feature of requiring no trust. The related funding project “AI3–A Layered-federation Information Sharing Architecture” [S2] has won the highest international/non-local consultancy award from PolyU Technology and Consultancy Company Limited (PTec) in 2018 [S1]. Furthermore, the underlying technology Blockchain has attracted companies like Alibaba and iNAS for cooperation with us.
• With the support of HK ITF and the collaboration with Hong Kong R&D Centre for Logistics and Supply Chain Management Enabling Technologies (LSCM) and Hong Kong International Airport Authority (HKIA), we applied our research outputs to airport resource management such as baggage carousel allocation and aircraft stand management. Managing resources in facilitating the flow of people and goods through the airport is always one big challenge for HKIA. Multi-source data such as flight/origin arrival patterns, passenger profile, baggage dwelling time, seasonal factors, and historical patterns, would have to be included for consideration. We developed a Big Data-Driven Airport Resource Management (BigARM) Engine [S3] for intelligent airport resource management, providing functions of making intelligent initial resource allocation plans, providing informed adjustment recommendations and alerting resource managers abrupt events. Based on the performance on testbed, “BigARM will reduce the initial generation plan from hours to minutes and greatly reduce workload of operators” as mentioned in the supporting letter from HKIA [S3].
• With the collaboration with GoGoVan, a leading on-demand logistics company in Hong Kong, we collected a large amount of data about customers and drivers in logistics services. With the collected data and the big data analytic techniques we proposed, we developed multiple applications to solve a series of issues in demand-supply management in on-demand logistics [S4], such as unmet demand prediction, order response time prediction, order accepting probability prediction, and dynamic order dispatching. These applications greatly improved the service quality of GoGoVan and improved its running efficiency.

AI-Driven Smart Imaging: The research achievements on machine learning based image enhancement and recognition attracted the attention from industries, including Alibaba Group, DJI, Huawei Technologies Ltd., and Sony Inc. for donation and research collaboration [S5]-[S9].

• Since April 2017, Prof. Zhang has been collaborating with the Institute of Data Science and Technology (iDST), Alibaba Group, starting from a collaborative project entitled “Quality Enhancement of Surveillance Images and Videos” [S5], which has as main objective of enhancing the video quality to improve the performance of the algorithm and the visual sense of the video image. At the same time, to directly combine some low-level visual processing ideas and methods in the advanced visual problem to improve the performance of the algorithm and model. It is the first research project Alibaba ever supported in Hong Kong. The progress is going very well, and some developed image enhancement algorithms have been adopted in Alibaba's “City Brain” project, improving much the surveillance image quality and consequently the license plate recognition and person re-identification rates. By employing the robust online background subtraction techniques developed in [R5], “Prof. Zhang and his team developed a foreground gating and background refining network, which significantly improved the generalization capability and accuracy (by 5%) of Alibaba's traffic video object detection system without increasing the computational cost” as commented by Alibaba Group in its supporting letter [S5].
• From 2014 to 2018, Prof. Zhang has been receiving research donations from DJI Innovations, the world-leading company in easy-to-fly drones and aerial photography systems, and collaborating with them on PhD student training and developing new camera image processing techniques [S6]. Through student summer internship, some new algorithms on in-camera white balance, fast face detection and tracking have been developed for DJI's use in their drone cameras. They are now collaborating on developing next generation end-to-end in-camera imaging pipeline, intelligent image enhancement, and human centered image quality/aesthetic evaluation for future cameras.
• In 2015, Prof. Zhang’s group also received research donation from Nopluz Technologies Co., Ltd, Shanghai, for research in computer vision technologies for intelligent transportation [S7]. The object detection, tracking and recognition techniques developed by Prof. Zhang’s group were integrated into Nopluz’s product to monitor the driving status of trucks, and were tested by Shanghai Qin-Shun Construction Engineering Company, a Construction Engineering Company in Shanghai, which owns more than 200 trucks. According to Nopluz’s supporting letter [S7], “the use of these technologies largely reduced more than 80% of the truck traffic accidents… This achievement was highly appreciated by Shanghai Traffic Police Bureau”.
• From 2015 to 2016, Prof. Zhang collaborated with Sony Corporation, Japan, on a project entitled “3D Data Convert and Repair Ecosystem” [S8]. Five patents have been filed simultaneously in the US and China. They also wrote a recommendation letter to us for bidding the PolyU Distinguished Technology Transfer Excellence Award.
• Prof. Zhang’s research in deep learning based image denoising and enhancement [R4] has attracted attention from the multimedia department of Huawei Technology Ltd., which is among the top 3 smartphone manufactures in the world. They have invited Prof. Zhang as a consultant to provide consultancy service [S9] on low-level vision and image processing techniques for Huawei smartphones.

5. Sources to corroborate the impact

[S1] Highest International/Non-local Consultancy (Winner) “AI3–A Layered-Federation Information Sharing Architecture” (https://www.comp.polyu.edu.hk/en-us/news/awards_detail/450)

[S2] AI3- A Layered-Federation Information Sharing Architecture, July 2016 – May 2018, Huawei Technologies Co. Ltd., HK$1,920,000. (https://www.comp.polyu.edu.hk/en-us/research/consultancy-project)

[S3] Big Data-Driven Airport Resources Management (BigARM) Engine and Application Tools, May 2018 – October 2019, HK ITC Innovation and Technology, HK$6,594,312, with supporting letter from HKAA as attached.

[S4] “Demand Dispatching for Emerging On-demand Services: A Data Driven Approach”, 01/2018-12/2020, HK PolyU HK RGC General Research Fund (GRF), HK$973,750, with supporting letter from GoGoVan as attached.

[S5] Collaborative research project with Alibaba, “Quality Enhancement of Surveillance Images and Videos”, HK$ 0.7M, 2017-2018, with supporting letter from Alibaba as attached.

[S6] Donation from DJI Innovations, “General Research in Image Processing and Computer Vision”, HK$8M, 2014 – 2018 (HK$2M/year). (https://www.comp.polyu.edu.hk/en-us/research/consultancy-project)

[S7] Donation from Nopluz Technologies Co., Ltd. “General Research in Computer Vision Techniques for Intelligent Transportation,” HK$1M, 2015, with supporting letter from Nopluz as attached.

[S8] Consultancy project with Sony Corporation, Japan, “3D Data Convert and Repair Ecosystem,” HK$1,030,750, December 2015 – November 2016. (https://www.comp.polyu.edu.hk/en-us/research/consultancy-project)

[S9] Consultancy project with Huawei, “Low-level vision and image processing techniques,” HK$0.32M, 2018. (https://www.comp.polyu.edu.hk/en-us/research/consultancy-project)