Super-resolution in millimetre-wave compressive computational imaging

Abstract

Imaging at millimetre wave (mmW) has many advantages over infrared (IR), X-ray and optical imaging. MmWs can penetrate through materials that are opaque at optical wavelengths. They do not possess any ionizing effects, and hence are harmless to human exposure. They can also be operated in all weather conditions, making them suitable for both indoor and outdoor use. Because of all these advantages, mmWs have found applications in many fields, ranging from security screening, and remote sensing to medical imaging. However, imaging at mmW frequencies exhibits a fundamental resolution limit, known as diffraction-limited resolution. Several techniques can be employed to enhance the resolution, such as increasing the size of the aperture, increasing the operating frequency, or reducing the imaging distance. Although these methods improve the resolution capability of the imaging system, they bring other challenges, such as increased hardware complexities and increased size of the aperture, hence limiting the system to a small range of applications. It also increases the data acquisition and processing time, hence posing significant challenges in real-time applications. An alternate solution to enhancing the resolution of the imaging system could be the use of super-resolution (SR) technique in the signal processing layer. SR is the process of recovering high-resolution (HR) version of a given low-resolution (LR) image. The presented thesis focuses on leveraging deep learning techniques to facilitate SR in mmW images in real-time. The main challenge in deploying any learning algorithm for image processing tasks, particular for mmW images, is the generation of the dataset. As SR is an ill-posed problem, the dataset required to achieve efficient learning is large. To address this challenge, instead of relying on experimentally generated datasets (which can be time consuming), or on already available datasets in the public domain, a numerical model of a compressive computational imaging (CI) system is developed. The role of this numerical model in this work is to generate the necessary dataset for the development of the deep learning models.

The first part of the thesis covers the development of a CI numerical model. Although CI techniques significantly reduces hardware complexity, however, they require processing of large matrices, hence increasing the computational cost. An Field Programmable Gate Array (FPGA)-enabled hardware layer is integrated with the CI numerical model to reduce the computational cost. In the second part of the thesis, two deep learning models are developed. The first model is a classifier, wherein, a Convolutional Neural Network (CNN) is designed to perform a classification task on mmW reconstructed images of different threat objects. A dataset consisting of simulated reconstructed images of Computer-Aided Design (CAD) models of threat objects is generated using the numerical model developed previously. To test the classifier, both simulated and experimentally generated images were used. The accuracy obtained in these tests establishes the fact that a learning algorithm trained with simulated data can perform accurately on experimental data as well. After this validation, a second deep learning model is developed, which deals with the SR problem. The same numerical model is used to generate the training dataset for this task. The SR is achieved using a complex-valued CNN layer that leverages a sub-network architecture. As often is the case in SR problems, the resolution difference between the input and output images is very large for any neural network to efficiently learn the mapping between the two sets of data. To address this challenge, sub-networks are introduced in the neural architecture that partitions the SR problems into multiple sub-problems. As the training dataset consists of both real and imaginary parts, the CNN architecture is designed accordingly to fit in the complex data. The final step in this research was to integrate the super-resolution
model with the developed classification model. The final system is an end-to-end mmW super-resolution classifier system that has the capability of improving the resolution of any input near-field mmW reconstruction data and classifying the reconstructed data into its appropriate classes.

Thesis is embargoed until 31 July 2024.

Date of Award	Jul 2023
Original language	English
Awarding Institution	Queen's University Belfast
Sponsors	Tezpur University
Supervisor	Okan Yurduseven (Supervisor), Vincent Fusco (Supervisor) & Bhabesh Deka (Supervisor)