Learning with limited data and its applications on video understanding

Abstract

Data plays a vital role in machine learning and computer vision research, as well as their applications. However, limited data poses a significant challenge due to the increasing complexity of tasks and the growing size of neural networks which require larger datasets for training. In this thesis, we examine the problem of limited sample learning, which aims to improve deep neural network performance when given a limited number of samples.

Chapter 3 focuses on learning with limited samples using generative models. We propose an improved version of the variational auto-encoder (VAE) using normalising flow (NF). We demonstrate that by replacing the Gaussian prior in the VAE with NF, overfitting in the low data regime can be reduced.

In Chapter 4, we present an extended version of FlowReg, which aims to regularise the training process by transferring knowledge from one or more related domains to the target domain. We propose a cross-domain flow to link the source domain and the target domain, which aligns the posterior distributions in multi-domain learning. Learning with limited samples can also be addressed by model adaptation.

This is explored in Chapter 5 where we study an application of limited sample learning in videos. We reframe video object detection as a limited sample learning problem and show that by using high-quality frames of a video as a reference set, a feature extractor can be adapted to produce more discriminative features in low-quality frames.

In Chapter 6, we study the topic of applying those tools in learning with limited labelled samples. We use Knowledge Distillation (KD)-based regularisation for the few-shot image classification problem. We demonstrate that an improved version of KD, which introduces high order relations between class representatives, can further improve the classification performance.

Date of Award	Dec 2023
Original language	English
Awarding Institution	Queen's University Belfast
Sponsors	Oosto
Supervisor	Hui Wang (Supervisor) & Yang Hua (Supervisor)